From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C250BC05027 for ; Sat, 18 Feb 2023 02:42:16 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CFC666B0071; Fri, 17 Feb 2023 21:42:15 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id C85626B0072; Fri, 17 Feb 2023 21:42:15 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AFEF86B0073; Fri, 17 Feb 2023 21:42:15 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 9C51C6B0071 for ; Fri, 17 Feb 2023 21:42:15 -0500 (EST) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 6738CAB767 for ; Sat, 18 Feb 2023 02:42:15 +0000 (UTC) X-FDA: 80478863430.05.DD03509 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf03.hostedemail.com (Postfix) with ESMTP id 84FAD20004 for ; Sat, 18 Feb 2023 02:42:12 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=UJ1lp4m9; spf=pass (imf03.hostedemail.com: domain of ming.lei@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=ming.lei@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1676688133; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=PWOvznM2L9QyDWoJX6erFzsz89hT+rHvd14btB9G+Ac=; b=ecYWKxyAcwhG94+44sFDhFzYlxmbb/UVr7ojSlYtAUvr8bSOOO6R+qebzH6hq++HXPw9XI WiNaHeAXjZVRdQdhehSA6FvWx8nfdODHYATIEWL2dfSwurJZ+kSDA2MT/vx8ZVaCue27WG LhiZnKLGZ+Bz/JRp/CHyMJbr8/AFqjo= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=UJ1lp4m9; spf=pass (imf03.hostedemail.com: domain of ming.lei@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=ming.lei@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1676688133; a=rsa-sha256; cv=none; b=uW/adRrm63IoRf39xAtqtpoo2MOK693xGexbobwoGuUaI1fxYxqkXWHzsXfBR7OHGEue4e x1kV3oYNzKEaEej+MkR9FucqNIuWIwdrrSMyee5b5M3inbjBdjbK2YhddPlSzeU/yE7ygU t3KtL3Ct2CjY7BbA/UF/wM+uapXZFw0= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1676688131; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=PWOvznM2L9QyDWoJX6erFzsz89hT+rHvd14btB9G+Ac=; b=UJ1lp4m9UOU31wAWh/6IGrM8h1y/OobbBTCAwGB51yxtu63875Nrgutv7jhUMf7sPoZ8OU xj1LKM+ez424HO8ZFVsCh/FDtzY5Qu3S/z2w9ZzkK9xtSgo5h11QuVLNmajUTfJhAxkMsr 0kQ2D8jcu+mTnRT7gW8qqoB2nfun+xA= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-124-9bYswzT2PVWZCNa2oCU2DA-1; Fri, 17 Feb 2023 21:42:06 -0500 X-MC-Unique: 9bYswzT2PVWZCNa2oCU2DA-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 61C4129AA38E; Sat, 18 Feb 2023 02:42:05 +0000 (UTC) Received: from T590 (ovpn-8-16.pek2.redhat.com [10.72.8.16]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 618CD2026D4B; Sat, 18 Feb 2023 02:41:54 +0000 (UTC) Date: Sat, 18 Feb 2023 10:41:49 +0800 From: Ming Lei To: David Howells Cc: Jens Axboe , Al Viro , Christoph Hellwig , Matthew Wilcox , Jan Kara , Jeff Layton , David Hildenbrand , Jason Gunthorpe , Logan Gunthorpe , Hillf Danton , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Christoph Hellwig , John Hubbard , ming.lei@redhat.com Subject: Re: [PATCH v14 02/17] splice: Add a func to do a splice from a buffered file without ITER_PIPE Message-ID: References: <20230214171330.2722188-1-dhowells@redhat.com> <20230214171330.2722188-3-dhowells@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20230214171330.2722188-3-dhowells@redhat.com> X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 84FAD20004 X-Stat-Signature: oxd3d7b5njxtt39ufxhnej69pu5nuc5x X-Rspam-User: X-HE-Tag: 1676688132-415645 X-HE-Meta: U2FsdGVkX18PxaL3embGQo1RSnsja5dUv64JaT9XF+TOOvibB+q79x+gWnEAm2+zkSATEdJAkl090UtkVy23dzsGSYVUYvMFEKEC05DhhhkUqGP7WIKd0Ut3rmIko44Syu1ySd5CJfkEcg2Xa8pQzv4qVz51gMRF4La1u8rNjmFz4uTzUtQ46lpfoCRZGgaw7/+wiouGx/GvqWQmGUfvNQ7vxKd+DAid4jNin6OJba2bTAKJlpmiDHimrB7ywt6V47K1WTc09jf+U8as/f+J2XFqgB+KgyKvk9IhtrIyaV+x0qpuaZ/hF1IM4+6s/d7gFjJ93oM2OXZpndZ/1mafpMkkK0IfiFd0HkTgnj2wR1dQ7FR+hnbQiZtsLI5FaWPFUrEy+aTkLvFZHfoIGXFEM+D26E7/7owXtT6TEruauAXC7OA3KBgIQBn1gcTiM52nHJWj0+KU1D3CgxZ5vjwWlL0Ozjh3yFQAfe3glQ5ujzsI6ZUqEbnHP6emWQfbn8hkmcK4SYDaqesoo5IPPe9QPa5zD+18JIH7xcRd2taM6K3XbmS7W3xmIJUwunanNMUqZ/K6ZVlnCr5ZIXRmXdbDAtbQUtJlvXf4qRMW/ExA8GslCD3QjQj+CObpv2Hfu/bxT1RnxPKaoXoBcq6BHua2z8948/nhfSVF+SpB37YXufttN5pgAlp/R1/mvkrHgXZkCnMqPckFFBYgzfcriAPSKK04dClM4W+xgXyz0SWHQM4sYnP5BdaEhC/csP1P4ByqxcIPfnSyxrKNKS93XwMAkGNRfCETUYoRyA2lsgqaVf+ts9jJhi+qgQiD+rDCjwtLyoP+w2973Hgakyy/DQUxvul4d8/t8eAg+A40JSBGMaKBQZVhRXer7HKGNEhMk19qnAaUcXhiewxQKySw/nfUxtr3F7wYf+cFhkyFKfn53w9SKJi+pUDfb1EG/x5HBFUWnnSibGwVQHqoFMDy8L1 XTq5FH4W sGKVOrLDKCuLxgXKoAzKReWFGIRhQQPoFJgPpU6TFE9bDew8l2sLTuQAtRVtZm4qGUzk7/xNmCoKKrMwxQBQT5cc2zTEltw2RTared/D/UHMk+33VgJdLHaQNf+7vv8cp6m/7nG3UxQg0f0h+PWjkLB15XLbPeb2A7whZ7ZKRMbxVpG3il4ueWt87EYQTI7elM+SHONEuhMvpQ6+ZNYexsq3ecdu+xc78tuaICyJ5wKHQ/3CsFTTFcEXiaN0WBq/egtD0yuqn5sL6Vj/uHu07JfzmhUSEGLis4CkCO7OWnxzC86V9UZ2szV1xsBIoKMQ2pPcwz8bCwIiJZtg+Wv1ZA2d8fUiDMaFUguvPhg+ET/DJAIQI/KeTDoK7elkv0jqTPonIGLldQEOM50jLNkjneA/vSwYwIBsCbD2f8vHoh+4Yk1Uarsd1lUgYC5/7H+Z5UPF5uSxb1QbMQedRAt2SzlNdq3CDYLYKQcUSleLJBm6mkDZCBAAjIS/JM+kD6UjUTfBipM4G/vMs7jY= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, Feb 14, 2023 at 05:13:15PM +0000, David Howells wrote: > Provide a function to do splice read from a buffered file, pulling the > folios out of the pagecache directly by calling filemap_get_pages() to do > any required reading and then pasting the returned folios into the pipe. > > A helper function is provided to do the actual folio pasting and will > handle multipage folios by splicing as many of the relevant subpages as > will fit into the pipe. > > The code is loosely based on filemap_read() and might belong in > mm/filemap.c with that as it needs to use filemap_get_pages(). > > Signed-off-by: David Howells > cc: Jens Axboe > cc: Christoph Hellwig > cc: Al Viro > cc: David Hildenbrand > cc: John Hubbard > cc: linux-mm@kvack.org > cc: linux-block@vger.kernel.org > cc: linux-fsdevel@vger.kernel.org > --- > > Notes: > ver #14) > - Rename to filemap_splice_read(). > - Create a helper, pipe_head_buf(), to get the head buffer. > - Use init_sync_kiocb(). > - Move to mm/filemap.c. > - Split the implementation of filemap_splice_read() from the patch to > make generic_file_splice_read() use it and direct_splice_read(). > > include/linux/fs.h | 3 ++ > mm/filemap.c | 128 +++++++++++++++++++++++++++++++++++++++++++++ > mm/internal.h | 6 +++ > 3 files changed, 137 insertions(+) > > diff --git a/include/linux/fs.h b/include/linux/fs.h > index c1769a2c5d70..28743e38df91 100644 > --- a/include/linux/fs.h > +++ b/include/linux/fs.h > @@ -3163,6 +3163,9 @@ ssize_t vfs_iocb_iter_write(struct file *file, struct kiocb *iocb, > struct iov_iter *iter); > > /* fs/splice.c */ > +ssize_t filemap_splice_read(struct file *in, loff_t *ppos, > + struct pipe_inode_info *pipe, > + size_t len, unsigned int flags); > extern ssize_t generic_file_splice_read(struct file *, loff_t *, > struct pipe_inode_info *, size_t, unsigned int); > extern ssize_t iter_file_splice_write(struct pipe_inode_info *, > diff --git a/mm/filemap.c b/mm/filemap.c > index 876e77278d2a..8c7b135c8e23 100644 > --- a/mm/filemap.c > +++ b/mm/filemap.c > @@ -42,6 +42,8 @@ > #include > #include > #include > +#include > +#include > #include > #include > #include "internal.h" > @@ -2842,6 +2844,132 @@ generic_file_read_iter(struct kiocb *iocb, struct iov_iter *iter) > } > EXPORT_SYMBOL(generic_file_read_iter); > > +/* > + * Splice subpages from a folio into a pipe. > + */ > +size_t splice_folio_into_pipe(struct pipe_inode_info *pipe, > + struct folio *folio, loff_t fpos, size_t size) > +{ > + struct page *page; > + size_t spliced = 0, offset = offset_in_folio(folio, fpos); > + > + page = folio_page(folio, offset / PAGE_SIZE); > + size = min(size, folio_size(folio) - offset); > + offset %= PAGE_SIZE; > + > + while (spliced < size && > + !pipe_full(pipe->head, pipe->tail, pipe->max_usage)) { > + struct pipe_buffer *buf = pipe_head_buf(pipe); > + size_t part = min_t(size_t, PAGE_SIZE - offset, size - spliced); > + > + *buf = (struct pipe_buffer) { > + .ops = &page_cache_pipe_buf_ops, > + .page = page, > + .offset = offset, > + .len = part, > + }; > + folio_get(folio); > + pipe->head++; > + page++; > + spliced += part; > + offset = 0; It should be better to replace above with add_to_pipe(). > + } > + > + return spliced; > +} > + > +/* > + * Splice folios from the pagecache of a buffered (ie. non-O_DIRECT) file into > + * a pipe. > + */ > +ssize_t filemap_splice_read(struct file *in, loff_t *ppos, > + struct pipe_inode_info *pipe, > + size_t len, unsigned int flags) > +{ > + struct folio_batch fbatch; > + struct kiocb iocb; > + size_t total_spliced = 0, used, npages; > + loff_t isize, end_offset; > + bool writably_mapped; > + int i, error = 0; > + > + init_sync_kiocb(&iocb, in); > + iocb.ki_pos = *ppos; > + > + /* Work out how much data we can actually add into the pipe */ > + used = pipe_occupancy(pipe->head, pipe->tail); > + npages = max_t(ssize_t, pipe->max_usage - used, 0); > + len = min_t(size_t, len, npages * PAGE_SIZE); Do we need to consider offset in 1st page here? thanks, Ming