From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E9F74C433F5 for ; Wed, 23 Mar 2022 11:42:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233449AbiCWLoV (ORCPT ); Wed, 23 Mar 2022 07:44:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60988 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233318AbiCWLoV (ORCPT ); Wed, 23 Mar 2022 07:44:21 -0400 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3FD9879385; Wed, 23 Mar 2022 04:42:52 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id F00DCB81E85; Wed, 23 Mar 2022 11:42:50 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0E0E1C340E8; Wed, 23 Mar 2022 11:42:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1648035769; bh=AX3abbornC9FM40TMDR9pJEAVnDGmX3E+mfdfSSC5xI=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=Apt6NVYWGYqoV9pWyMXsDMiJOfEiyhunqbrscSTMrQR+PW+kTafTW02Cm/97eg84W 75nUv0lOOwEHQMrgiEd0+NcdbvjqP4rVYfiK+u/hU1AH9NgjLzZ0tO/mAuzzV/vuFS SeicNiFy08qDfL13M1ouV92Ks/on9bSEE/K9AkN0= Date: Wed, 23 Mar 2022 12:42:46 +0100 From: Greg KH To: Bernd Schubert Cc: Miklos Szeredi , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-api@vger.kernel.org, linux-man@vger.kernel.org, linux-security-module@vger.kernel.org, Karel Zak , Ian Kent , David Howells , Linus Torvalds , Al Viro , Christian Brauner , Amir Goldstein , James Bottomley , Dharmendra Singh Subject: Re: [RFC PATCH] getvalues(2) prototype Message-ID: References: <20220322192712.709170-1-mszeredi@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: linux-man@vger.kernel.org On Wed, Mar 23, 2022 at 11:26:11AM +0100, Bernd Schubert wrote: > On 3/23/22 08:16, Greg KH wrote: > > On Tue, Mar 22, 2022 at 08:27:12PM +0100, Miklos Szeredi wrote: > > > Add a new userspace API that allows getting multiple short values in a > > > single syscall. > > > > > > This would be useful for the following reasons: > > > > > > - Calling open/read/close for many small files is inefficient. E.g. on my > > > desktop invoking lsof(1) results in ~60k open + read + close calls under > > > /proc and 90% of those are 128 bytes or less. > > > > As I found out in testing readfile(): > > https://lore.kernel.org/r/20200704140250.423345-1-gregkh@linuxfoundation.org > > > > microbenchmarks do show a tiny improvement in doing something like this, > > but that's not a real-world application. > > > > Do you have anything real that can use this that shows a speedup? > > Add in network file systems. Demonstrating that this is useful locally and > with micro benchmarks - yeah, helps a bit to make it locally faster. But the > real case is when thousands of clients are handled by a few network servers. > Even reducing wire latency for a single client would make a difference here. I think I tried running readfile on NFS. Didn't see any improvements. But please, try it again. Also note that this proposal isn't for NFS, or any other "real" filesystem :) > There is a bit of chicken-egg problem - it is a bit of work to add to file > systems like NFS (or others that are not the kernel), but the work won't be > made there before there is no syscall for it. To demonstrate it on NFS one > also needs a an official protocol change first. And then applications also > need to support that new syscall first. > I had a hard time explaining weather physicist back in 2009 that it is not a > good idea to have millions of 512B files on Lustre. With recent AI workload > this gets even worse. Can you try using the readfile() patch to see if that helps you all out on Lustre? If so, that's a good reason to consider it. But again, has nothing to do with this getvalues(2) api. thanks, greg k-h