From: Dongsheng Yang <dongsheng.yang@linux.dev>
To: axboe@kernel.dk, hch@lst.de, dan.j.williams@intel.com,
gregory.price@memverge.com, John@groves.net,
Jonathan.Cameron@Huawei.com, bbhushan2@marvell.com,
chaitanyak@nvidia.com, rdunlap@infradead.org
Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org,
linux-cxl@vger.kernel.org, linux-bcache@vger.kernel.org,
nvdimm@lists.linux.dev, Dongsheng Yang <dongsheng.yang@linux.dev>
Subject: [RFC PATCH 00/11] pcache: Persistent Memory Cache for Block Devices
Date: Mon, 14 Apr 2025 01:44:54 +0000 [thread overview]
Message-ID: <20250414014505.20477-1-dongsheng.yang@linux.dev> (raw)
Hi All,
This patchset introduces a new Linux block layer module called
**pcache**, which uses persistent memory (pmem) as a cache for block
devices.
Originally, this functionality was implemented as `cbd_cache` within the
CBD (CXL Block Device). However, after thorough consideration,
it became clear that the cache design was not limited to CBD's pmem
device or infrastructure. Instead, it is broadly applicable to **any**
persistent memory device that supports DAX. Therefore, I have split
pcache out of cbd and refactored it into a standalone module.
Although Intel's Optane product line has been discontinued, the Storage
Class Memory (SCM) field continues to evolve. For instance, Numemory
recently launched their Optane successor product, the NM101 SCM:
https://www.techpowerup.com/332914/numemory-releases-optane-successor-nm101-storage-class-memory
### About pcache
+-------------------------------+------------------------------+------------------------------+------------------------------+
| Feature | pcache | bcache | dm-writecache |
+-------------------------------+------------------------------+------------------------------+------------------------------+
| pmem access method | DAX | bio | DAX |
+-------------------------------+------------------------------+------------------------------+------------------------------+
| Write Latency (4K randwrite) | ~7us | ~20us | ~7us |
+-------------------------------+------------------------------+------------------------------+------------------------------+
| Concurrency | Multi-tree per backend, | Shared global index tree, | single indexing tree and |
| | fully utilizing pmem | | global wc_lock |
+-------------------------------+------------------------------+------------------------------+------------------------------+
| IOPS (4K randwrite 32 numjobs)| 2107K | 352K | 283K |
+-------------------------------+------------------------------+------------------------------+------------------------------+
| Read Cache Support | YES | YES | NO |
+-------------------------------+------------------------------+------------------------------+------------------------------+
| Deployment Flexibility | No reformat needed | Requires formatting backend | Depends on dm framework, |
| | | devices | less intuitive to deploy |
+-------------------------------+------------------------------+------------------------------+------------------------------+
| Writeback Model | log-structure; preserves | no guarantee between | no guarantee writeback |
| | backing crash-consistency; | flush order and app IO order;| ordering |
| | important for checkpoint | may lose ordering in backing | |
+-------------------------------+------------------------------+------------------------------+------------------------------+
| Data Integrity | CRC on both metadata and | CRC on metadata only | No CRC |
| | data (data crc is optional) | | |
+-------------------------------+------------------------------+------------------------------+------------------------------+
### Repository
- Kernel code: https://github.com/DataTravelGuide/linux/tree/pcache
- Userspace tool: https://github.com/DataTravelGuide/pcache-utils
### Example Usage
```bash
$ insmod /workspace/linux_compile/drivers/block/pcache/pcache.ko
$ pcache cache-start --path /dev/pmem1 --format --force
$ pcache backing-start --path /dev/vdh --cache-size 50G --queues 16
/dev/pcache0
$ pcache backing-list
[
{
"backing_id": 0,
"backing_path": "/dev/vdh",
"cache_segs": 3200,
"cache_gc_percent": 70,
"cache_used_segs": 2238,
"logic_dev": "/dev/pcache0"
}
]
```
Thanks for reviewing!
Dongsheng Yang (11):
pcache: introduce cache_dev for managing persistent memory-based cache
devices
pcache: introduce segment abstraction
pcache: introduce meta_segment abstraction
pcache: introduce cache_segment abstraction
pcache: introduce lifecycle management of pcache_cache
pcache: gc and writeback
pcache: introduce cache_key infrastructure for persistent metadata
management
pcache: implement request processing and cache I/O path in cache_req
pcache: introduce logic block device and request handling
pcache: add backing device management
block: introduce pcache (persistent memory to be cache for block
device)
MAINTAINERS | 8 +
drivers/block/Kconfig | 2 +
drivers/block/Makefile | 2 +
drivers/block/pcache/Kconfig | 16 +
drivers/block/pcache/Makefile | 4 +
drivers/block/pcache/backing_dev.c | 593 +++++++++++++++++
drivers/block/pcache/backing_dev.h | 105 +++
drivers/block/pcache/cache.c | 394 +++++++++++
drivers/block/pcache/cache.h | 612 +++++++++++++++++
drivers/block/pcache/cache_dev.c | 808 ++++++++++++++++++++++
drivers/block/pcache/cache_dev.h | 81 +++
drivers/block/pcache/cache_gc.c | 150 +++++
drivers/block/pcache/cache_key.c | 885 +++++++++++++++++++++++++
drivers/block/pcache/cache_req.c | 812 +++++++++++++++++++++++
drivers/block/pcache/cache_segment.c | 247 +++++++
drivers/block/pcache/cache_writeback.c | 183 +++++
drivers/block/pcache/logic_dev.c | 348 ++++++++++
drivers/block/pcache/logic_dev.h | 73 ++
drivers/block/pcache/main.c | 194 ++++++
drivers/block/pcache/meta_segment.c | 61 ++
drivers/block/pcache/meta_segment.h | 46 ++
drivers/block/pcache/pcache_internal.h | 185 ++++++
drivers/block/pcache/segment.c | 175 +++++
drivers/block/pcache/segment.h | 78 +++
24 files changed, 6062 insertions(+)
create mode 100644 drivers/block/pcache/Kconfig
create mode 100644 drivers/block/pcache/Makefile
create mode 100644 drivers/block/pcache/backing_dev.c
create mode 100644 drivers/block/pcache/backing_dev.h
create mode 100644 drivers/block/pcache/cache.c
create mode 100644 drivers/block/pcache/cache.h
create mode 100644 drivers/block/pcache/cache_dev.c
create mode 100644 drivers/block/pcache/cache_dev.h
create mode 100644 drivers/block/pcache/cache_gc.c
create mode 100644 drivers/block/pcache/cache_key.c
create mode 100644 drivers/block/pcache/cache_req.c
create mode 100644 drivers/block/pcache/cache_segment.c
create mode 100644 drivers/block/pcache/cache_writeback.c
create mode 100644 drivers/block/pcache/logic_dev.c
create mode 100644 drivers/block/pcache/logic_dev.h
create mode 100644 drivers/block/pcache/main.c
create mode 100644 drivers/block/pcache/meta_segment.c
create mode 100644 drivers/block/pcache/meta_segment.h
create mode 100644 drivers/block/pcache/pcache_internal.h
create mode 100644 drivers/block/pcache/segment.c
create mode 100644 drivers/block/pcache/segment.h
--
2.34.1
next reply other threads:[~2025-04-14 1:45 UTC|newest]
Thread overview: 19+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-04-14 1:44 Dongsheng Yang [this message]
2025-04-14 1:44 ` [RFC PATCH 01/11] pcache: introduce cache_dev for managing persistent memory-based cache devices Dongsheng Yang
2025-04-14 1:44 ` [RFC PATCH 02/11] pcache: introduce segment abstraction Dongsheng Yang
2025-04-14 1:44 ` [RFC PATCH 03/11] pcache: introduce meta_segment abstraction Dongsheng Yang
2025-04-14 1:44 ` [RFC PATCH 04/11] pcache: introduce cache_segment abstraction Dongsheng Yang
2025-04-14 1:44 ` [RFC PATCH 05/11] pcache: introduce lifecycle management of pcache_cache Dongsheng Yang
2025-04-14 1:45 ` [RFC PATCH 06/11] pcache: gc and writeback Dongsheng Yang
2025-04-14 1:45 ` [RFC PATCH 07/11] pcache: introduce cache_key infrastructure for persistent metadata management Dongsheng Yang
2025-04-14 1:45 ` [RFC PATCH 08/11] pcache: implement request processing and cache I/O path in cache_req Dongsheng Yang
2025-04-14 1:45 ` [RFC PATCH 09/11] pcache: introduce logic block device and request handling Dongsheng Yang
2025-04-14 1:45 ` [RFC PATCH 10/11] pcache: add backing device management Dongsheng Yang
2025-04-14 1:45 ` [RFC PATCH 11/11] block: introduce pcache (persistent memory to be cache for block device) Dongsheng Yang
2025-04-15 18:00 ` [RFC PATCH 00/11] pcache: Persistent Memory Cache for Block Devices Dan Williams
2025-04-16 1:04 ` Jens Axboe
2025-04-16 6:08 ` Dongsheng Yang
2025-04-16 15:10 ` Jens Axboe
2025-04-16 21:40 ` Dongsheng Yang
2025-04-22 10:29 ` Mikulas Patocka
2025-04-22 13:23 ` Dongsheng Yang
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20250414014505.20477-1-dongsheng.yang@linux.dev \
--to=dongsheng.yang@linux.dev \
--cc=John@groves.net \
--cc=Jonathan.Cameron@Huawei.com \
--cc=axboe@kernel.dk \
--cc=bbhushan2@marvell.com \
--cc=chaitanyak@nvidia.com \
--cc=dan.j.williams@intel.com \
--cc=gregory.price@memverge.com \
--cc=hch@lst.de \
--cc=linux-bcache@vger.kernel.org \
--cc=linux-block@vger.kernel.org \
--cc=linux-cxl@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=nvdimm@lists.linux.dev \
--cc=rdunlap@infradead.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).