From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9EC4CE7E644 for ; Tue, 26 Sep 2023 14:40:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234935AbjIZOkG (ORCPT ); Tue, 26 Sep 2023 10:40:06 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46390 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234855AbjIZOkE (ORCPT ); Tue, 26 Sep 2023 10:40:04 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E5FB4120 for ; Tue, 26 Sep 2023 07:39:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1695739151; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=XllbBqxTvNzfuhH1RFsYE8Y8P6H7hYsJSRNMnTexg+M=; b=GC5kb3tVC7t/WmGQ3HfCI6ZTrylsPE2nFdWqFwbunBsjUIzF+vbL/E0qXXUlWwtRBmOK46 xNpJaqWy0VVn4Kl2lyb33ZyD+zOwU67STyv51Iq2dBcvzUqy2N1uN4cb6S28HmAQynVtlu rCQsPI3LJ2XRKFEmxfCfhlC7xAjaxbs= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-182-Fk0Y759GO-SfTiZBEIRSlA-1; Tue, 26 Sep 2023 10:39:08 -0400 X-MC-Unique: Fk0Y759GO-SfTiZBEIRSlA-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 0CB3F8039D2; Tue, 26 Sep 2023 14:39:07 +0000 (UTC) Received: from oldenburg.str.redhat.com (unknown [10.2.16.53]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 268471004058; Tue, 26 Sep 2023 14:39:05 +0000 (UTC) From: Florian Weimer To: Miklos Szeredi Cc: Miklos Szeredi , Christian Brauner , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-api@vger.kernel.org, linux-man@vger.kernel.org, linux-security-module@vger.kernel.org, Karel Zak , Ian Kent , David Howells , Al Viro , Christian Brauner , Amir Goldstein Subject: Re: [RFC PATCH 2/3] add statmnt(2) syscall References: <20230913152238.905247-1-mszeredi@redhat.com> <20230913152238.905247-3-mszeredi@redhat.com> <20230914-salzig-manifest-f6c3adb1b7b4@brauner> <20230914-lockmittel-verknallen-d1a18d76ba44@brauner> <20230918-grafik-zutreffen-995b321017ae@brauner> <871qeloxj0.fsf@oldenburg.str.redhat.com> <87wmwdnhj1.fsf@oldenburg.str.redhat.com> Date: Tue, 26 Sep 2023 16:39:03 +0200 In-Reply-To: (Miklos Szeredi's message of "Tue, 26 Sep 2023 16:33:50 +0200") Message-ID: <87bkdpngmw.fsf@oldenburg.str.redhat.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Scanned-By: MIMEDefang 3.1 on 10.11.54.3 Precedence: bulk List-ID: X-Mailing-List: linux-api@vger.kernel.org * Miklos Szeredi: > On Tue, 26 Sept 2023 at 16:19, Florian Weimer wrote: > >> getdents gets away with this buffer size because applications can copy >> out all the data from struct dirent if they need long-term storage. >> They have to do that because the usual readdir interface overwrites the >> buffer, potentially at the next readdir call. This means the buffer >> size does not introduce an amount of memory fragmention that is >> dependent on the directory size. >> >> With an opaque, pointer-carrying struct, copying out the data is not >> possible in a generic fashion. Only the parts that the application >> knows about can be copied out. So I think it's desirable to have a >> fairly exact allocation. > > Okay, so let's add a 'size' field to the struct, which is set to the > size used (as opposed to the size of the buffer). That should solve > copying without wasting a single byte of memory. That would be helpful. > Otherwise the format is fully copyable, since the strings are denoted > with an offset, which doesn't change after the buffer is copied. I missed the development in that direction. Yes, offsets would work nicely in this context. They help with compat syscalls, too. If the buffer is relocatable like that, we can even try first with a reasonably sized on-stack buffer and create an exactly-sized heap allocation from that. Thanks, Florian