From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B7E70C61CE8 for ; Tue, 3 Jun 2025 09:54:01 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 099A76B03F8; Tue, 3 Jun 2025 05:53:58 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0229D6B03F9; Tue, 3 Jun 2025 05:53:57 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E52FD6B03FC; Tue, 3 Jun 2025 05:53:57 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id BEDD36B03FA for ; Tue, 3 Jun 2025 05:53:57 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 75D991A026C for ; Tue, 3 Jun 2025 09:53:57 +0000 (UTC) X-FDA: 83513628114.30.CA63F50 Received: from mta20.hihonor.com (mta20.hihonor.com [81.70.206.69]) by imf19.hostedemail.com (Postfix) with ESMTP id 243CE1A0004 for ; Tue, 3 Jun 2025 09:53:54 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=none; spf=pass (imf19.hostedemail.com: domain of tao.wangtao@honor.com designates 81.70.206.69 as permitted sender) smtp.mailfrom=tao.wangtao@honor.com; dmarc=pass (policy=none) header.from=honor.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1748944435; a=rsa-sha256; cv=none; b=YUFuiU4gQo/JfxX66S4GsAPJ5uZ/PsSK4Q/bzQ4m0y/pfEOwEP7UlNrtEezaoETnIIiIeS rzyg/K2wSDLMXET2lbJj2T4Zy36RA1NtcMMWplEIN4A23QGvu2zFvwDEXYVRvWgA3Ij/OO JOQsHe56CmtfIUjW4w9Wb86IQjh5nhE= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=none; spf=pass (imf19.hostedemail.com: domain of tao.wangtao@honor.com designates 81.70.206.69 as permitted sender) smtp.mailfrom=tao.wangtao@honor.com; dmarc=pass (policy=none) header.from=honor.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1748944435; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1bGSEKJUabCA9LkgtoA3wAHKG50wkDKzdpOf2c5yANk=; b=VF343/Ezelm3hrp3SZv8Jyxv5ieqPnjZSupqpE0KSv8l0tAxL1G8if/4+OEBOshtasqN28 6bYG3F9CirTLi2OAb9dt5z96FpWxpUCcOoNqmEhXLlwp3ZY9vd8HVBYpQPBLpT3xbKJolo RBjNOq4HHSMwdOmItjCEBVyQ573xJMc= Received: from w012.hihonor.com (unknown [10.68.27.189]) by mta20.hihonor.com (SkyGuard) with ESMTPS id 4bBQw571wWzYlP63; Tue, 3 Jun 2025 17:51:33 +0800 (CST) Received: from a010.hihonor.com (10.68.16.52) by w012.hihonor.com (10.68.27.189) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.11; Tue, 3 Jun 2025 17:53:50 +0800 Received: from localhost.localdomain (10.144.18.117) by a010.hihonor.com (10.68.16.52) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.11; Tue, 3 Jun 2025 17:53:50 +0800 From: wangtao To: , , , , , , , , CC: , , , , , , , , , , , , , , , , wangtao Subject: [PATCH v4 3/4] udmabuf: Implement udmabuf direct I/O Date: Tue, 3 Jun 2025 17:52:44 +0800 Message-ID: <20250603095245.17478-4-tao.wangtao@honor.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250603095245.17478-1-tao.wangtao@honor.com> References: <20250603095245.17478-1-tao.wangtao@honor.com> MIME-Version: 1.0 Content-Type: text/plain X-Originating-IP: [10.144.18.117] X-ClientProxiedBy: w002.hihonor.com (10.68.28.120) To a010.hihonor.com (10.68.16.52) X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 243CE1A0004 X-Stat-Signature: 13we81jgbqkcheu5xauxpmych1yahoqs X-Rspam-User: X-HE-Tag: 1748944434-613955 X-HE-Meta: U2FsdGVkX1+RAM7hmz8Cej+/glP6iDajwc2xFjyaihLaxLVWOcDrfvCLlJrUNSGN01XStQQEz/7dGUIDraJOwjXGnJfDkedAyDfokTMOr4Idm21WAR9+xISlzmUT0QUkW+UHcobI1nQtdVFiLz7636ziKbKAQcGqkXUnRPS8zbD/sxGpfTvBbynuN/2LxcI68qxBLK4dzxwHuVjcvFNQX61OVr/SkA/J8dpLBUOena2+qfaD6bcXmZLWDC1HtURO5ltN8M9HNveqq5o0JKohIsqpHdBwS4qQLaWBCZLW/WagOKnH2uh76q9yX8msa2m7ikfje5m+m0xu40pPuAdSwHInBdSJZv9EUuzz3omhrtzicrkWeXF1bpN34rck4Oigxiux1dhQAeBeZebt1hvMuAR3krypea7a/3PDX6oNzSWzA7iBO5Qd62Cvf9QE3nIy9ve5qNdao26oLdSRwlwOdbsTrpIAsWHCqAD0HF3ryUT4xCedthjmHC3QkCddgZcqGvPlGBJogRMxiFddpaBQE3jCj7A45VAvD1X4JW4Ughsz+wHne8MuipkSo5gmyZx70WbHeQC0OZRR3kr/fS2P1q3MhzUG0zXdFLqRpcyvS3eYERtFO+2GOlyr9BkmwqR9vmEcZWjr29THnR3FMLE5cQFkH9LZMwnB8EPMxpE/MDX3O4GERBcDpuWiHdpL59e6yQBC0Ae2C0jSjab1/AYIMgRUQ72clz2M1PRDrnm5aLf1/G6caubUP+8JbMvf0bPDc+0A2srXiKjorXohxk4PrxiaWg4Ziixm6HQsNwtMQirbKJcMuUErVeoW4wJPyfNX6R/FE1kfG5EUrbqJ2uJ1QMcGNm3/elFG9cegsN4/v6zWz20+tIfB4BENLh0+IiNQ0lS3jnzhmF4qfCG8RDVgmDYxLpyf+ugvQpQQhtnQBoC+fi2zpLq09po8JD4t45gzo8bnWK8r+OGIuRlJZ6F 5Tnnbf69 MwckHbJM11wPTNsXhrBRlxPSpT+vuhq6En6I9tUST0OUpo4j2pauvuUb67YYiOP8o2qXMQ+tV2q3AyIvjwTKmoMo0qilohJskqpjxOiniVXHTQ2qHjVexOiU0QrXbr9oig8SB/Xs/Jq3frXrRIw1RvH7bkUTva4FkPkbjw57pSFbNcnunFvTdOvXd+8mAFZgfwrl+2jEScI+5pCLyzy3CKHvnXhie8sDI4H0ivrM0acBEiVPoHO1OWheimQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Construct bio_vec from folios, then call the other file's r/w callbacks for IO operations. Test data shows direct I/O copy_file_range improves performance by over 50% vs direct I/O mmap&read (2557 vs 1534). Test data: | 32x32MB Read 1024MB |Creat-ms|Close-ms| I/O-ms|I/O-MB/s| I/O% |-------------------------|--------|--------|--------|--------|----- | 1)Beg udmabuf buffer R/W| 580 | 323 | 1238 | 867 | 100% | 2) dmabuf buffer R/W| 48 | 5 | 1149 | 934 | 107% | 3) udma+memfd buffer R/W| 597 | 340 | 2157 | 497 | 57% | 4) udma+memfd direct R/W| 573 | 340 | 700 | 1534 | 176% | 5) u+mfd buffer sendfile| 577 | 340 | 1204 | 891 | 102% | 6) u+mfd direct sendfile| 567 | 339 | 2272 | 472 | 54% | 7) u+mfd buffer splice| 570 | 337 | 1114 | 964 | 111% | 8) u+mfd direct splice| 564 | 335 | 793 | 1355 | 156% | 9) udmabuf buffer c_f_r| 577 | 323 | 1059 | 1014 | 116% |10) udmabuf direct c_f_r| 582 | 325 | 420 | 2557 | 294% |11)End udmabuf buffer R/W| 586 | 323 | 1188 | 903 | 104% Signed-off-by: wangtao --- drivers/dma-buf/udmabuf.c | 54 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 54 insertions(+) diff --git a/drivers/dma-buf/udmabuf.c b/drivers/dma-buf/udmabuf.c index e74e36a8ecda..511567b15340 100644 --- a/drivers/dma-buf/udmabuf.c +++ b/drivers/dma-buf/udmabuf.c @@ -15,6 +15,8 @@ #include #include #include +#include +#include static int list_limit = 1024; module_param(list_limit, int, 0644); @@ -284,6 +286,55 @@ static int end_cpu_udmabuf(struct dma_buf *buf, return 0; } +static ssize_t udmabuf_rw_file(struct dma_buf *dmabuf, loff_t my_pos, + struct file *other, loff_t pos, + size_t count, bool is_write) +{ + struct udmabuf *ubuf = dmabuf->priv; + loff_t my_end = my_pos + count, bv_beg, bv_end = 0; + size_t i, bv_off, bv_len, bv_idx = 0; + struct bio_vec *bvec; + struct kiocb kiocb; + struct iov_iter iter; + unsigned int direction = is_write ? ITER_SOURCE : ITER_DEST; + ssize_t ret = 0; + struct folio *folio; + + bvec = kvcalloc(ubuf->nr_pinned, sizeof(*bvec), GFP_KERNEL); + if (!bvec) + return -ENOMEM; + + init_sync_kiocb(&kiocb, other); + kiocb.ki_pos = pos; + + for (i = 0; i < ubuf->nr_pinned; i++) { + folio = ubuf->pinned_folios[i]; + bv_beg = bv_end; + if (bv_beg >= my_end) + break; + bv_end += folio_size(folio); + if (bv_end <= my_pos) + continue; + + bv_len = min(bv_end, my_end) - max(my_pos, bv_beg); + bv_off = my_pos > bv_beg ? my_pos - bv_beg : 0; + bvec_set_page(&bvec[bv_idx], &folio->page, bv_len, bv_off); + ++bv_idx; + } + + if (bv_idx > 0) { + /* start R/W. */ + iov_iter_bvec(&iter, direction, bvec, bv_idx, count); + if (is_write) + ret = other->f_op->write_iter(&kiocb, &iter); + else + ret = other->f_op->read_iter(&kiocb, &iter); + } + kvfree(bvec); + + return ret; +} + static const struct dma_buf_ops udmabuf_ops = { .cache_sgt_mapping = true, .map_dma_buf = map_udmabuf, @@ -294,6 +345,7 @@ static const struct dma_buf_ops udmabuf_ops = { .vunmap = vunmap_udmabuf, .begin_cpu_access = begin_cpu_udmabuf, .end_cpu_access = end_cpu_udmabuf, + .rw_file = udmabuf_rw_file, }; #define SEALS_WANTED (F_SEAL_SHRINK) @@ -455,6 +507,8 @@ static long udmabuf_create(struct miscdevice *device, ret = PTR_ERR(dmabuf); goto err; } + /* Support direct I/O */ + dmabuf->file->f_mode |= FMODE_CAN_ODIRECT; /* * Ownership of ubuf is held by the dmabuf from here. * If the following dma_buf_fd() fails, dma_buf_put() cleans up both the -- 2.17.1