From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 35520C5B559 for ; Tue, 3 Jun 2025 09:54:06 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AD22B6B03FD; Tue, 3 Jun 2025 05:53:58 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A34756B03FE; Tue, 3 Jun 2025 05:53:58 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 923816B03FF; Tue, 3 Jun 2025 05:53:58 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 6DBD66B03FD for ; Tue, 3 Jun 2025 05:53:58 -0400 (EDT) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 3A6A6B7C20 for ; Tue, 3 Jun 2025 09:53:58 +0000 (UTC) X-FDA: 83513628156.14.B161C8B Received: from mta22.hihonor.com (mta22.hihonor.com [81.70.192.198]) by imf16.hostedemail.com (Postfix) with ESMTP id B4EE7180005 for ; Tue, 3 Jun 2025 09:53:55 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=honor.com; spf=pass (imf16.hostedemail.com: domain of tao.wangtao@honor.com designates 81.70.192.198 as permitted sender) smtp.mailfrom=tao.wangtao@honor.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1748944436; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=hvBWlrswWXlV+H8Ejb5hOpw71DeTzhhQcgOfiKOK+wE=; b=jI5Jsulyd7u7gxvpdtDStQznAfXM/xGlAelfmSxUkVgSOG7nM6drBJQU6wossv0dtIpMtr sFb876oRzvl5a0mTpwEjk3zBHEW6sQSlPlz1kGJy/ZzLV+DwCSvTqsOeCb5pqIgGWbbP02 Cj4W6LI78RrS89Fn2lFJaZrc1Fn9jr8= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1748944436; a=rsa-sha256; cv=none; b=fsmseI8rkcOyyUYGSFau6GMglgtX/ZHEhUoIQlrRPvV+TK2U0i87B0y+gske+1oLo6nFnk XbTjPUMTMSgt/THpXVIXR6NXj6X3oaLkBAsFwn7Mec0HKF6hY88j6JuHS2dwxtqL1+3sLv VZhPvil0YFVVmYN7wwdV97NjjyfBscw= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=honor.com; spf=pass (imf16.hostedemail.com: domain of tao.wangtao@honor.com designates 81.70.192.198 as permitted sender) smtp.mailfrom=tao.wangtao@honor.com Received: from w011.hihonor.com (unknown [10.68.20.122]) by mta22.hihonor.com (SkyGuard) with ESMTPS id 4bBQwW0q9hzYl8XY; Tue, 3 Jun 2025 17:51:55 +0800 (CST) Received: from a010.hihonor.com (10.68.16.52) by w011.hihonor.com (10.68.20.122) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.11; Tue, 3 Jun 2025 17:53:51 +0800 Received: from localhost.localdomain (10.144.18.117) by a010.hihonor.com (10.68.16.52) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.11; Tue, 3 Jun 2025 17:53:50 +0800 From: wangtao To: , , , , , , , , CC: , , , , , , , , , , , , , , , , wangtao Subject: [PATCH v4 4/4] dmabuf:system_heap Implement system_heap dmabuf direct I/O Date: Tue, 3 Jun 2025 17:52:45 +0800 Message-ID: <20250603095245.17478-5-tao.wangtao@honor.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250603095245.17478-1-tao.wangtao@honor.com> References: <20250603095245.17478-1-tao.wangtao@honor.com> MIME-Version: 1.0 Content-Type: text/plain X-Originating-IP: [10.144.18.117] X-ClientProxiedBy: w002.hihonor.com (10.68.28.120) To a010.hihonor.com (10.68.16.52) X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: B4EE7180005 X-Stat-Signature: i9g936t4t695kz6hah9ku8pxoeojg7go X-Rspam-User: X-HE-Tag: 1748944435-345808 X-HE-Meta: U2FsdGVkX1/jmxsDxbDLYwtWV6mXmCyOqn/TYImWHY+igZrZStVKhoHOp5lIKelfHbBZHMdaPrO9NE8vFKabQUXMrZeSMzKQ+bNq/laKDBAcBlVdRsMDPu1erg2/EDm00FP73qQtRj1wMPYHM1Dg/EThTMLFzYqOLVmGovxpKQYgiZyN94G47egZ7SWkHxGGa9xvyE95NQxVpuRfAtwMIjhEkjPUGqYWcezPDHVjjgwroYuPAtGjYDkiKlb2nDQJZNLmFMveWanTV+UraRAm4/rW2soF7hccBuuhsrQQbxNbVnEhY1SsPOy7cFewWK7R9p95yZ0HCVHKSJtmQJyOs7uckIi/lFOUwNnNK1hrKY5HW0KwRnnocsk1LKFqAb9pcBxuMyT8V6hA4Ewup6pLj6w5ncRtXzXpmUeie2tm6KkcV+BUm8Sic9YRJ4VWHlwLnb6ji3b9Ch1pL0zIVA4I+L5KGUM9t8DbWGyLGE28n71BzkoLAm0xEvldDmOMrb8bxDqdLDxmTKpD0kI824P2DFJNqVmv0RFGKBIhtxV647cv9E/xCianAI36VJrvUMUb4hMsY5QS4+ouIeGats1MoNnjwFk830QQCGlIeuCe8OjChCswcF6idQQ+fpFCRtshrGgoUnDIVBF3jrCN1OtxgpKiGoDY4t47r7n2J/rbYyeBHoHs2gF/KCLmw6+HBJ5k6AvLJvEQMCChXkJHCAXC/Uvvni87SrEmumqFVQ4oP57kdjzf7UxLmA1OoXhWcqJTRLGZf174UuWplJYw8PX1scOoYQJmhAKeibkZckR9hJJE0l5Cy7fLsKbuJl+BRZbphEi2QLLiBgwlONV6nNTUpcVBTR5mPPxKNihwXgyDK7sQa5PoOUvyuURlxRHh6yi1gBFz8dAvzAbQTQSam+WB7aS4Rge1ssiJeYuzwXheZZ7EpVPPJ2OHcornb66iQrLnCdYsuXCAGHb7p0h8S3M b6iDoYao 5s2we/a/5Zkp4M0KCAx0IUewtDolJxFCXz3NrqSRVIbLkhlfefTRXtlGLs/dTwJnaLGCNl6u5+WqoOc4KJ9mGdoKdzeKHYjKUziIuTvxuo7v1n87VwdgfR4gwVTwe/8OrAN+6AfoO7kD5S6cbpn9ucYBmlWKbaJwZxJDDyjeDIVbAe82jmhwAy//xAH6rGOgkUO6MNzUH4jAdDIl+49uhBdgFoqUkJKsdvi5ZY7csrlVgXad+yIbOMBahWmef9/tht1oYFvBwXZ3w9GKUtfdBbX8u8xY7r6H1IgENS3ZHV4Lke0E= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: First verify system_heap exporter has exclusive dmabuf access. Build bio_vec from sgtable, then invoke target file's r/w callbacks for IO. Outperforms buffer IO mmap/read by 250%, beats direct I/O udmabuf copy_file_range by over 30% with initialization time significantly lower than udmabuf. Test data: | 32x32MB Read 1024MB |Creat-ms|Close-ms| I/O-ms|I/O-MB/s| I/O% |-------------------------|--------|--------|--------|--------|----- | 1)Beg dmabuf buffer R/W| 47 | 5 | 1125 | 954 | 100% | 2) udmabuf buffer R/W| 576 | 323 | 1228 | 874 | 91% | 3) udma+memfd buffer R/W| 596 | 340 | 2166 | 495 | 51% | 4) udma+memfd direct R/W| 570 | 338 | 711 | 1510 | 158% | 5) udmabuf buffer c_f_r| 578 | 329 | 1128 | 952 | 99% | 6) udmabuf direct c_f_r| 570 | 324 | 405 | 2651 | 277% | 7) dmabuf buffer c_f_r| 47 | 5 | 1035 | 1037 | 108% | 8) dmabuf direct c_f_r| 51 | 5 | 309 | 3480 | 364% | 9)End dmabuf buffer R/W| 48 | 5 | 1153 | 931 | 97% | 32x32MB Write 1024MB |Creat-ms|Close-ms| I/O-ms|I/O-MB/s| I/O% |-------------------------|--------|--------|--------|--------|----- | 1)Beg dmabuf buffer R/W| 50 | 5 | 1405 | 764 | 100% | 2) udmabuf buffer R/W| 580 | 341 | 1337 | 803 | 105% | 3) udma+memfd buffer R/W| 588 | 331 | 1820 | 590 | 77% | 4) udma+memfd direct R/W| 585 | 333 | 662 | 1622 | 212% | 5) udmabuf buffer c_f_r| 577 | 329 | 1326 | 810 | 106% | 6) udmabuf direct c_f_r| 580 | 330 | 602 | 1784 | 233% | 7) dmabuf buffer c_f_r| 49 | 5 | 1330 | 807 | 105% | 8) dmabuf direct c_f_r| 49 | 5 | 344 | 3127 | 409% | 9)End dmabuf buffer R/W| 50 | 5 | 1442 | 745 | 97% Signed-off-by: wangtao --- drivers/dma-buf/heaps/system_heap.c | 69 +++++++++++++++++++++++++++++ 1 file changed, 69 insertions(+) diff --git a/drivers/dma-buf/heaps/system_heap.c b/drivers/dma-buf/heaps/system_heap.c index 26d5dc89ea16..85ffff7ef855 100644 --- a/drivers/dma-buf/heaps/system_heap.c +++ b/drivers/dma-buf/heaps/system_heap.c @@ -20,6 +20,8 @@ #include #include #include +#include +#include static struct dma_heap *sys_heap; @@ -281,6 +283,70 @@ static void system_heap_vunmap(struct dma_buf *dmabuf, struct iosys_map *map) iosys_map_clear(map); } +static ssize_t system_heap_buffer_rw_other(struct system_heap_buffer *buffer, + loff_t my_pos, struct file *other, loff_t pos, + size_t count, bool is_write) +{ + struct sg_table *sgt = &buffer->sg_table; + struct scatterlist *sg; + loff_t my_end = my_pos + count, bv_beg, bv_end = 0; + size_t i, bv_off, bv_len, bv_idx = 0; + struct bio_vec *bvec; + struct kiocb kiocb; + struct iov_iter iter; + unsigned int direction = is_write ? ITER_SOURCE : ITER_DEST; + ssize_t ret = 0; + + bvec = kvcalloc(sgt->orig_nents, sizeof(*bvec), GFP_KERNEL); + if (!bvec) + return -ENOMEM; + + init_sync_kiocb(&kiocb, other); + kiocb.ki_pos = pos; + + for_each_sgtable_sg(sgt, sg, i) { + bv_beg = bv_end; + if (bv_beg >= my_end) + break; + bv_end += sg->offset + sg->length; + if (bv_end <= my_pos) + continue; + + bv_len = min(bv_end, my_end) - max(my_pos, bv_beg); + bv_off = sg->offset + (my_pos > bv_beg ? my_pos - bv_beg : 0); + bvec_set_page(&bvec[bv_idx], sg_page(sg), bv_len, bv_off); + ++bv_idx; + } + + if (bv_idx > 0) { + /* start R/W. */ + iov_iter_bvec(&iter, direction, bvec, bv_idx, count); + if (is_write) + ret = other->f_op->write_iter(&kiocb, &iter); + else + ret = other->f_op->read_iter(&kiocb, &iter); + } + kvfree(bvec); + + return ret; +} + +static ssize_t system_heap_dma_buf_rw_file(struct dma_buf *dmabuf, + loff_t my_pos, struct file *file, loff_t pos, + size_t count, bool is_write) +{ + struct system_heap_buffer *buffer = dmabuf->priv; + ssize_t ret = -EBUSY; + + mutex_lock(&buffer->lock); + if (list_empty(&buffer->attachments) && !buffer->vmap_cnt) + ret = system_heap_buffer_rw_other(buffer, my_pos, + file, pos, count, is_write); + mutex_unlock(&buffer->lock); + + return ret; +} + static void system_heap_dma_buf_release(struct dma_buf *dmabuf) { struct system_heap_buffer *buffer = dmabuf->priv; @@ -308,6 +374,7 @@ static const struct dma_buf_ops system_heap_buf_ops = { .mmap = system_heap_mmap, .vmap = system_heap_vmap, .vunmap = system_heap_vunmap, + .rw_file = system_heap_dma_buf_rw_file, .release = system_heap_dma_buf_release, }; @@ -400,6 +467,8 @@ static struct dma_buf *system_heap_allocate(struct dma_heap *heap, ret = PTR_ERR(dmabuf); goto free_pages; } + /* Support direct I/O */ + dmabuf->file->f_mode |= FMODE_CAN_ODIRECT; return dmabuf; free_pages: -- 2.17.1