From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 03D99E7E643 for ; Tue, 26 Sep 2023 14:39:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234804AbjIZOj4 (ORCPT ); Tue, 26 Sep 2023 10:39:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46338 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234855AbjIZOjz (ORCPT ); Tue, 26 Sep 2023 10:39:55 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3D9DB10E for ; Tue, 26 Sep 2023 07:39:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1695739149; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=XllbBqxTvNzfuhH1RFsYE8Y8P6H7hYsJSRNMnTexg+M=; b=eCM9VuS1fBxlZInEL+JDtxX7GykuomozP1qi7SoDoh/c7Eh7I8JcEOTumOyKjKfubZGLQX G4+0qTHfQko4A5JOBACeH/7Gpro8tzDDrr6TCx6uZbf3HpQ/KKL59f//4EY0OLmr3NTp6C 9ZXDcFYMhwZhv2zTgZznQwfjPAj00p8= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-182-Fk0Y759GO-SfTiZBEIRSlA-1; Tue, 26 Sep 2023 10:39:08 -0400 X-MC-Unique: Fk0Y759GO-SfTiZBEIRSlA-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 0CB3F8039D2; Tue, 26 Sep 2023 14:39:07 +0000 (UTC) Received: from oldenburg.str.redhat.com (unknown [10.2.16.53]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 268471004058; Tue, 26 Sep 2023 14:39:05 +0000 (UTC) From: Florian Weimer To: Miklos Szeredi Cc: Miklos Szeredi , Christian Brauner , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-api@vger.kernel.org, linux-man@vger.kernel.org, linux-security-module@vger.kernel.org, Karel Zak , Ian Kent , David Howells , Al Viro , Christian Brauner , Amir Goldstein Subject: Re: [RFC PATCH 2/3] add statmnt(2) syscall References: <20230913152238.905247-1-mszeredi@redhat.com> <20230913152238.905247-3-mszeredi@redhat.com> <20230914-salzig-manifest-f6c3adb1b7b4@brauner> <20230914-lockmittel-verknallen-d1a18d76ba44@brauner> <20230918-grafik-zutreffen-995b321017ae@brauner> <871qeloxj0.fsf@oldenburg.str.redhat.com> <87wmwdnhj1.fsf@oldenburg.str.redhat.com> Date: Tue, 26 Sep 2023 16:39:03 +0200 In-Reply-To: (Miklos Szeredi's message of "Tue, 26 Sep 2023 16:33:50 +0200") Message-ID: <87bkdpngmw.fsf@oldenburg.str.redhat.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Scanned-By: MIMEDefang 3.1 on 10.11.54.3 Precedence: bulk List-ID: * Miklos Szeredi: > On Tue, 26 Sept 2023 at 16:19, Florian Weimer wrote: > >> getdents gets away with this buffer size because applications can copy >> out all the data from struct dirent if they need long-term storage. >> They have to do that because the usual readdir interface overwrites the >> buffer, potentially at the next readdir call. This means the buffer >> size does not introduce an amount of memory fragmention that is >> dependent on the directory size. >> >> With an opaque, pointer-carrying struct, copying out the data is not >> possible in a generic fashion. Only the parts that the application >> knows about can be copied out. So I think it's desirable to have a >> fairly exact allocation. > > Okay, so let's add a 'size' field to the struct, which is set to the > size used (as opposed to the size of the buffer). That should solve > copying without wasting a single byte of memory. That would be helpful. > Otherwise the format is fully copyable, since the strings are denoted > with an offset, which doesn't change after the buffer is copied. I missed the development in that direction. Yes, offsets would work nicely in this context. They help with compat syscalls, too. If the buffer is relocatable like that, we can even try first with a reasonably sized on-stack buffer and create an exactly-sized heap allocation from that. Thanks, Florian