From: Chao Gao <chao.gao@intel.com>
To: linux-kernel@vger.kernel.org
Cc: dave.hansen@intel.com, len.brown@intel.com, tony.luck@intel.com,
rafael.j.wysocki@intel.com, reinette.chatre@intel.com,
dan.j.williams@intel.com, kirill.shutemov@linux.intel.com,
sathyanarayanan.kuppuswamy@linux.intel.com,
ilpo.jarvinen@linux.intel.com, Chao Gao <chao.gao@intel.com>,
Andrew Morton <akpm@linux-foundation.org>,
Borislav Petkov <bp@suse.de>,
Damien Le Moal <damien.lemoal@opensource.wdc.com>,
iommu@lists.linux-foundation.org,
Kees Cook <keescook@chromium.org>,
linux-doc@vger.kernel.org, linux-pm@vger.kernel.org,
Muchun Song <songmuchun@bytedance.com>,
"Paul E. McKenney" <paulmck@kernel.org>,
Randy Dunlap <rdunlap@infradead.org>
Subject: [PATCH v1 0/3] swiotlb performance optimizations
Date: Tue, 28 Jun 2022 15:01:31 +0800 [thread overview]
Message-ID: <20220628070136.419163-1-chao.gao@intel.com> (raw)
Intent of this post:
Seek reviews from Intel reviewers and anyone else in the list
interested in IO performance in confidential VMs. Need some acked-by
reviewed-by tags before I can add swiotlb maintainers to "to/cc" lists
and ask for a review from them.
swiotlb is now widely used by confidential VMs. This series optimizes
swiotlb to reduce cache misses and lock contention during bounce buffer
allocation/free and memory bouncing to improve IO workload performance in
confidential VMs.
Here are some FIO tests we did to demonstrate the improvement.
Test setup
----------
A normal VM with 8vCPU and 32G memory, swiotlb is enabled by swiotlb=force.
100 in Host/Guest CPU utilization means 1 logical processor. FIO block size
is 4K and iodepth is 256. Note that a normal VM is used so that others lack
of necessary hardware to host confidential VMs can reproduce results below.
Results
-------
1 FIO job read/write Throughput IOPS Host CPU Guest CPU
(MB/s) (k) utilization utilization
vanilla read 1037 253 228.48 101.92
write 1148 280 233.28 100.96
optimized read 1160 283 232.32 101.12
write 1195 292 233.28 100.64
1-job FIO sequential read/write perf increase by 12% and 4% respectively.
4 FIO jobs read/write Throughput IOPS Host CPU Guest CPU
(MB/s) (k) utilization utilization
vanilla read 885 214.9 527.04 401.12
write 868 212.1 531.84 400.64
optimized read 2320 567 344.64 202.8
write 1998 488 312 173.92
4-job FIO sequential read/write perf increase by 164% and 130% respectively.
This series is based on 5.19-rc2.
Andi Kleen (1):
swiotlb: Split up single swiotlb lock
Chao Gao (2):
swiotlb: Use bitmap to track free slots
swiotlb: Allocate memory in a cache-friendly way
.../admin-guide/kernel-parameters.txt | 4 +-
arch/x86/kernel/acpi/boot.c | 4 +
include/linux/swiotlb.h | 47 +++-
kernel/dma/swiotlb.c | 263 +++++++++++++-----
4 files changed, 229 insertions(+), 89 deletions(-)
--
2.25.1
WARNING: multiple messages have this Message-ID (diff)
From: Chao Gao <chao.gao@intel.com>
To: linux-kernel@vger.kernel.org
Cc: len.brown@intel.com, tony.luck@intel.com,
Kees Cook <keescook@chromium.org>,
"Paul E. McKenney" <paulmck@kernel.org>,
linux-doc@vger.kernel.org,
Damien Le Moal <damien.lemoal@opensource.wdc.com>,
rafael.j.wysocki@intel.com,
Muchun Song <songmuchun@bytedance.com>,
dave.hansen@intel.com, iommu@lists.linux-foundation.org,
Randy Dunlap <rdunlap@infradead.org>,
Borislav Petkov <bp@suse.de>,
linux-pm@vger.kernel.org, ilpo.jarvinen@linux.intel.com,
dan.j.williams@intel.com, reinette.chatre@intel.com,
Andrew Morton <akpm@linux-foundation.org>,
kirill.shutemov@linux.intel.com
Subject: [PATCH v1 0/3] swiotlb performance optimizations
Date: Tue, 28 Jun 2022 15:01:31 +0800 [thread overview]
Message-ID: <20220628070136.419163-1-chao.gao@intel.com> (raw)
Intent of this post:
Seek reviews from Intel reviewers and anyone else in the list
interested in IO performance in confidential VMs. Need some acked-by
reviewed-by tags before I can add swiotlb maintainers to "to/cc" lists
and ask for a review from them.
swiotlb is now widely used by confidential VMs. This series optimizes
swiotlb to reduce cache misses and lock contention during bounce buffer
allocation/free and memory bouncing to improve IO workload performance in
confidential VMs.
Here are some FIO tests we did to demonstrate the improvement.
Test setup
----------
A normal VM with 8vCPU and 32G memory, swiotlb is enabled by swiotlb=force.
100 in Host/Guest CPU utilization means 1 logical processor. FIO block size
is 4K and iodepth is 256. Note that a normal VM is used so that others lack
of necessary hardware to host confidential VMs can reproduce results below.
Results
-------
1 FIO job read/write Throughput IOPS Host CPU Guest CPU
(MB/s) (k) utilization utilization
vanilla read 1037 253 228.48 101.92
write 1148 280 233.28 100.96
optimized read 1160 283 232.32 101.12
write 1195 292 233.28 100.64
1-job FIO sequential read/write perf increase by 12% and 4% respectively.
4 FIO jobs read/write Throughput IOPS Host CPU Guest CPU
(MB/s) (k) utilization utilization
vanilla read 885 214.9 527.04 401.12
write 868 212.1 531.84 400.64
optimized read 2320 567 344.64 202.8
write 1998 488 312 173.92
4-job FIO sequential read/write perf increase by 164% and 130% respectively.
This series is based on 5.19-rc2.
Andi Kleen (1):
swiotlb: Split up single swiotlb lock
Chao Gao (2):
swiotlb: Use bitmap to track free slots
swiotlb: Allocate memory in a cache-friendly way
.../admin-guide/kernel-parameters.txt | 4 +-
arch/x86/kernel/acpi/boot.c | 4 +
include/linux/swiotlb.h | 47 +++-
kernel/dma/swiotlb.c | 263 +++++++++++++-----
4 files changed, 229 insertions(+), 89 deletions(-)
--
2.25.1
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
next reply other threads:[~2022-06-28 7:01 UTC|newest]
Thread overview: 12+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-06-28 7:01 Chao Gao [this message]
2022-06-28 7:01 ` [PATCH v1 0/3] swiotlb performance optimizations Chao Gao
2022-06-28 7:01 ` [PATCH v1 1/3] swiotlb: Use bitmap to track free slots Chao Gao
2022-06-28 7:01 ` Chao Gao
2022-06-28 13:11 ` Ilpo Järvinen
2022-06-28 13:11 ` Ilpo Järvinen
2022-06-28 7:01 ` [PATCH v1 2/3] swiotlb: Allocate memory in a cache-friendly way Chao Gao
2022-06-28 7:01 ` Chao Gao
2022-06-28 7:01 ` [PATCH v1 3/3] swiotlb: Split up single swiotlb lock Chao Gao
2022-06-28 7:01 ` Chao Gao
2022-06-30 2:42 ` Chao Gao
2022-06-30 2:42 ` Chao Gao
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20220628070136.419163-1-chao.gao@intel.com \
--to=chao.gao@intel.com \
--cc=akpm@linux-foundation.org \
--cc=bp@suse.de \
--cc=damien.lemoal@opensource.wdc.com \
--cc=dan.j.williams@intel.com \
--cc=dave.hansen@intel.com \
--cc=ilpo.jarvinen@linux.intel.com \
--cc=iommu@lists.linux-foundation.org \
--cc=keescook@chromium.org \
--cc=kirill.shutemov@linux.intel.com \
--cc=len.brown@intel.com \
--cc=linux-doc@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-pm@vger.kernel.org \
--cc=paulmck@kernel.org \
--cc=rafael.j.wysocki@intel.com \
--cc=rdunlap@infradead.org \
--cc=reinette.chatre@intel.com \
--cc=sathyanarayanan.kuppuswamy@linux.intel.com \
--cc=songmuchun@bytedance.com \
--cc=tony.luck@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.