public inbox for fio@vger.kernel.org
 help / color / mirror / Atom feed
* [PATCH v3 0/2] page_fault: add mmap-backed ioengine for anonymous faults
@ 2026-04-08  1:20 Nico Pache
  2026-04-08  1:20 ` [PATCH v3 1/2] " Nico Pache
                   ` (3 more replies)
  0 siblings, 4 replies; 6+ messages in thread
From: Nico Pache @ 2026-04-08  1:20 UTC (permalink / raw)
  To: vincentfu, axboe, fio
  Cc: jvozar, jjurca, jhladky, peterx, spetrovi, npache, david

This series introduces a new page_fault ioengine for Anonymous memory
testing. This enables using fio’s existing framework and job files for
memory management style workloads without relying on a filesystem. An
example job file is included to demonstrate usage and lays the groundwork
for how we plan on utilizing fio to test a number of MM related workloads.

The first patch adds the mmap‑backed ioengine that allocates anonymous
memory and copies data on read/write to intentionally trigger faults.

The second patch updates the documentation to guide usage of the new
feature.

V3:
- Correctly use clang-format to adhere to formatting conventions
- check (and fail/warn) if user sets the start_offset
- directly utilize io_ops_data as there is no files
- updated error/print to follow proper format
- Copy docs update to fio.1

V2: https://lore.kernel.org/fio/20260311155114.198006-1-npache@redhat.com/
V1: https://lore.kernel.org/fio/20260129184302.34887-1-npache@redhat.com/

Signed-off-by: Nico Pache <npache@redhat.com>

Nico Pache (2):
  page_fault: add mmap-backed ioengine for anonymous faults
  Documentation: update the documentation to include the page_fault
    engine

 HOWTO.rst               |   5 ++
 Makefile                |   2 +-
 engines/page_fault.c    | 125 ++++++++++++++++++++++++++++++++++++++++
 examples/page_fault.fio |  60 +++++++++++++++++++
 fio.1                   |   5 ++
 5 files changed, 196 insertions(+), 1 deletion(-)
 create mode 100644 engines/page_fault.c
 create mode 100644 examples/page_fault.fio

-- 
2.53.0


^ permalink raw reply	[flat|nested] 6+ messages in thread

* [PATCH v3 1/2] page_fault: add mmap-backed ioengine for anonymous faults
  2026-04-08  1:20 [PATCH v3 0/2] page_fault: add mmap-backed ioengine for anonymous faults Nico Pache
@ 2026-04-08  1:20 ` Nico Pache
  2026-04-08  1:20 ` [PATCH v3 2/2] Documentation: update the documentation to include the page_fault engine Nico Pache
                   ` (2 subsequent siblings)
  3 siblings, 0 replies; 6+ messages in thread
From: Nico Pache @ 2026-04-08  1:20 UTC (permalink / raw)
  To: vincentfu, axboe, fio
  Cc: jvozar, jjurca, jhladky, peterx, spetrovi, npache, david

Introduce a new ioengine that mmaps anonymous memory and copies data
on read/write to trigger page faults. This allows us to leverage FIOs
powerful framework for MM related testing, and will ideally allow us to
quickly expand testing, by leveraging previously FS related fio scripts.

Signed-off-by: Nico Pache <npache@redhat.com>
---
 Makefile                |   2 +-
 engines/page_fault.c    | 125 ++++++++++++++++++++++++++++++++++++++++
 examples/page_fault.fio |  60 +++++++++++++++++++
 3 files changed, 186 insertions(+), 1 deletion(-)
 create mode 100644 engines/page_fault.c
 create mode 100644 examples/page_fault.fio

diff --git a/Makefile b/Makefile
index 0337e8fe..099e2f94 100644
--- a/Makefile
+++ b/Makefile
@@ -57,7 +57,7 @@ SOURCE :=	$(sort $(patsubst $(SRCDIR)/%,%,$(wildcard $(SRCDIR)/crc/*.c)) \
 		smalloc.c filehash.c profile.c debug.c engines/cpu.c \
 		engines/mmap.c engines/sync.c engines/null.c engines/net.c \
 		engines/ftruncate.c engines/fileoperations.c \
-		engines/exec.c \
+		engines/exec.c engines/page_fault.c \
 		server.c client.c iolog.c backend.c libfio.c flow.c cconv.c \
 		gettime-thread.c helpers.c json.c idletime.c td_error.c \
 		profiles/tiobench.c profiles/act.c io_u_queue.c filelock.c \
diff --git a/engines/page_fault.c b/engines/page_fault.c
new file mode 100644
index 00000000..31e5177c
--- /dev/null
+++ b/engines/page_fault.c
@@ -0,0 +1,125 @@
+/*
+ * page_fault engine
+ *
+ * IO engine that reads/writes directly to/from anonymous memory
+ * by triggering page faults.
+ */
+#include "fio.h"
+#include "ioengines.h"
+#include <sys/mman.h>
+
+struct fio_page_fault_data {
+	void *mmap_ptr;
+	size_t mmap_sz;
+};
+
+static int fio_page_fault_init(struct thread_data *td)
+{
+	size_t total_io_size;
+	struct fio_page_fault_data *fpd;
+
+	if (td->o.nr_files > 1) {
+		log_err("fio: page_fault ioengine does not support multiple files\n");
+		return 1;
+	}
+
+	if (td->o.start_offset != 0) {
+		log_err("fio: page_fault engine does not support start_offset\n");
+		return 1;
+	}
+
+	fpd = calloc(1, sizeof(*fpd));
+	if (!fpd)
+		return 1;
+
+	total_io_size = td->o.size;
+	fpd->mmap_sz = total_io_size;
+	fpd->mmap_ptr = mmap(NULL, total_io_size, PROT_READ | PROT_WRITE,
+			     MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
+	if (fpd->mmap_ptr == MAP_FAILED) {
+		free(fpd);
+		return 1;
+	}
+
+	td->io_ops_data = fpd;
+	return 0;
+}
+
+static enum fio_q_status fio_page_fault_queue(struct thread_data *td,
+					      struct io_u *io_u)
+{
+	void *mmap_head;
+	struct fio_page_fault_data *fpd = td->io_ops_data;
+	if (!fpd) {
+		io_u->error = EINVAL;
+		return FIO_Q_COMPLETED;
+	}
+
+	if (io_u->offset + io_u->buflen > fpd->mmap_sz) {
+		io_u->error = EINVAL;
+		return FIO_Q_COMPLETED;
+	}
+
+	mmap_head = fpd->mmap_ptr + io_u->offset;
+	switch (io_u->ddir) {
+	case DDIR_READ:
+		memcpy(io_u->xfer_buf, mmap_head, io_u->buflen);
+		break;
+	case DDIR_WRITE:
+		memcpy(mmap_head, io_u->xfer_buf, io_u->buflen);
+		break;
+	case DDIR_SYNC:
+	case DDIR_DATASYNC:
+	case DDIR_SYNC_FILE_RANGE:
+	case DDIR_SYNCFS:
+		break;
+	default:
+		io_u->error = EINVAL;
+		break;
+	}
+
+	return FIO_Q_COMPLETED;
+}
+
+static int fio_page_fault_open_file(struct thread_data *td, struct fio_file *f)
+{
+	return 0;
+}
+
+static int fio_page_fault_close_file(struct thread_data *td, struct fio_file *f)
+{
+	return 0;
+}
+
+static void fio_page_fault_cleanup(struct thread_data *td)
+{
+	struct fio_page_fault_data *fpd = td->io_ops_data;
+
+	if (!fpd)
+		return;
+	if (fpd->mmap_ptr && fpd->mmap_sz)
+		munmap(fpd->mmap_ptr, fpd->mmap_sz);
+	free(fpd);
+}
+
+static struct ioengine_ops ioengine = {
+	.name = "page_fault",
+	.version = FIO_IOOPS_VERSION,
+	.init = fio_page_fault_init,
+	.cleanup = fio_page_fault_cleanup,
+	.queue = fio_page_fault_queue,
+	.open_file = fio_page_fault_open_file,
+	.close_file = fio_page_fault_close_file,
+	.get_file_size = generic_get_file_size,
+	.flags = FIO_SYNCIO | FIO_NOEXTEND | FIO_DISKLESSIO,
+};
+
+static void fio_init fio_page_fault_register(void)
+{
+	register_ioengine(&ioengine);
+}
+
+static void fio_exit fio_page_fault_unregister(void)
+{
+	unregister_ioengine(&ioengine);
+}
\ No newline at end of file
diff --git a/examples/page_fault.fio b/examples/page_fault.fio
new file mode 100644
index 00000000..4a49fb7d
--- /dev/null
+++ b/examples/page_fault.fio
@@ -0,0 +1,60 @@
+[global]
+stonewall
+ioengine=page_fault
+size=5*1024*1024*$mb_memory/8 # 5/8 of the memory
+bs=$pagesize
+
+[page_fault_write]
+rw=write
+
+[page_fault_mixed]
+rw=randrw
+rwmixread=50
+
+[page_fault_process_storm]
+size=8*1024*$pagesize
+rw=randrw
+rwmixread=50
+numjobs=$ncpus
+
+[page_fault_mthp]
+bsrange=16k-1M
+blockalign=2M
+rw=write
+
+[page_fault_seq_read]
+rw=read
+
+[page_fault_rand_read]
+rw=randread
+
+[page_fault_rand_write]
+rw=randwrite
+
+[page_fault_warm]
+rw=randrw
+rwmixread=50
+loops=3
+
+[page_fault_read_heavy]
+rw=randrw
+rwmixread=90
+
+[page_fault_write_heavy]
+rw=randrw
+rwmixread=10
+
+[page_fault_large_block]
+bs=2M
+blockalign=2M
+rw=write
+
+[page_fault_cache_hot]
+size=64*$pagesize
+rw=randrw
+rwmixread=50
+loops=100
+
+[page_fault_pressure]
+size=7*1024*1024*$mb_memory/8 # 7/8 of the memory
+rw=write
-- 
2.53.0


^ permalink raw reply related	[flat|nested] 6+ messages in thread

* [PATCH v3 2/2] Documentation: update the documentation to include the page_fault engine
  2026-04-08  1:20 [PATCH v3 0/2] page_fault: add mmap-backed ioengine for anonymous faults Nico Pache
  2026-04-08  1:20 ` [PATCH v3 1/2] " Nico Pache
@ 2026-04-08  1:20 ` Nico Pache
  2026-04-08  2:01 ` [PATCH v3 0/2] page_fault: add mmap-backed ioengine for anonymous faults Jens Axboe
  2026-04-08  4:12 ` fiotestbot
  3 siblings, 0 replies; 6+ messages in thread
From: Nico Pache @ 2026-04-08  1:20 UTC (permalink / raw)
  To: vincentfu, axboe, fio
  Cc: jvozar, jjurca, jhladky, peterx, spetrovi, npache, david

Document the new page fault engine.

Signed-off-by: Nico Pache <npache@redhat.com>
---
 HOWTO.rst | 5 +++++
 fio.1     | 5 +++++
 2 files changed, 10 insertions(+)

diff --git a/HOWTO.rst b/HOWTO.rst
index 5e3266df..bb97ee22 100644
--- a/HOWTO.rst
+++ b/HOWTO.rst
@@ -2379,6 +2379,11 @@ I/O engine
 			several instances to access the same device or file
 			simultaneously, but allow it for threads.
 
+		**page_fault**
+			I/O engine that uses an mmap region to simulate data transfer by
+			allocating anonymous memory and copying data on read/write to
+			intentionally trigger page faults.
+
 	File/directory operation engines define how the job operates file or directory. The
 	following types are defined:
 
diff --git a/fio.1 b/fio.1
index 664d7e3b..14ea6427 100644
--- a/fio.1
+++ b/fio.1
@@ -2168,6 +2168,11 @@ instance is used per process, so all jobs setting option \fBthread\fR will share
 a single instance (with one queue per thread) and must specify compatible
 options. Note that some drivers don't allow several instances to access the same
 device or file simultaneously, but allow it for threads.
+.TP
+.B page_fault
+I/O engine that uses an mmap region to simulate data transfer by allocating
+anonymous memory and copying data on read/write to intentionally trigger page
+faults.
 .RE
 .P
 File/directory operation engines define how the job operates file or directory.
-- 
2.53.0


^ permalink raw reply related	[flat|nested] 6+ messages in thread

* Re: [PATCH v3 0/2] page_fault: add mmap-backed ioengine for anonymous faults
  2026-04-08  1:20 [PATCH v3 0/2] page_fault: add mmap-backed ioengine for anonymous faults Nico Pache
  2026-04-08  1:20 ` [PATCH v3 1/2] " Nico Pache
  2026-04-08  1:20 ` [PATCH v3 2/2] Documentation: update the documentation to include the page_fault engine Nico Pache
@ 2026-04-08  2:01 ` Jens Axboe
  2026-04-08 13:21   ` Nico Pache
  2026-04-08  4:12 ` fiotestbot
  3 siblings, 1 reply; 6+ messages in thread
From: Jens Axboe @ 2026-04-08  2:01 UTC (permalink / raw)
  To: vincentfu, fio, Nico Pache
  Cc: jvozar, jjurca, jhladky, peterx, spetrovi, david


On Tue, 07 Apr 2026 19:20:01 -0600, Nico Pache wrote:
> This series introduces a new page_fault ioengine for Anonymous memory
> testing. This enables using fio’s existing framework and job files for
> memory management style workloads without relying on a filesystem. An
> example job file is included to demonstrate usage and lays the groundwork
> for how we plan on utilizing fio to test a number of MM related workloads.
> 
> The first patch adds the mmap‑backed ioengine that allocates anonymous
> memory and copies data on read/write to intentionally trigger faults.
> 
> [...]

Applied, thanks!

[1/2] page_fault: add mmap-backed ioengine for anonymous faults
      commit: f5ea7d9e1d7c07053f23e764b281a5e47bf01ba3
[2/2] Documentation: update the documentation to include the page_fault engine
      commit: dceeff3a1dd87b5194c84845d77096b3cdb3cee3

Best regards,
-- 
Jens Axboe




^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [PATCH v3 0/2] page_fault: add mmap-backed ioengine for anonymous faults
  2026-04-08  1:20 [PATCH v3 0/2] page_fault: add mmap-backed ioengine for anonymous faults Nico Pache
                   ` (2 preceding siblings ...)
  2026-04-08  2:01 ` [PATCH v3 0/2] page_fault: add mmap-backed ioengine for anonymous faults Jens Axboe
@ 2026-04-08  4:12 ` fiotestbot
  3 siblings, 0 replies; 6+ messages in thread
From: fiotestbot @ 2026-04-08  4:12 UTC (permalink / raw)
  To: fio

[-- Attachment #1: Type: text/plain, Size: 144 bytes --]


The result of fio's continuous integration tests was: success

For more details see https://github.com/fiotestbot/fio/actions/runs/24112588366

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [PATCH v3 0/2] page_fault: add mmap-backed ioengine for anonymous faults
  2026-04-08  2:01 ` [PATCH v3 0/2] page_fault: add mmap-backed ioengine for anonymous faults Jens Axboe
@ 2026-04-08 13:21   ` Nico Pache
  0 siblings, 0 replies; 6+ messages in thread
From: Nico Pache @ 2026-04-08 13:21 UTC (permalink / raw)
  To: Jens Axboe
  Cc: vincentfu, fio, jvozar, jjurca, jhladky, peterx, spetrovi, david

On Tue, Apr 7, 2026 at 8:01 PM Jens Axboe <axboe@kernel.dk> wrote:
>
>
> On Tue, 07 Apr 2026 19:20:01 -0600, Nico Pache wrote:
> > This series introduces a new page_fault ioengine for Anonymous memory
> > testing. This enables using fio’s existing framework and job files for
> > memory management style workloads without relying on a filesystem. An
> > example job file is included to demonstrate usage and lays the groundwork
> > for how we plan on utilizing fio to test a number of MM related workloads.
> >
> > The first patch adds the mmap‑backed ioengine that allocates anonymous
> > memory and copies data on read/write to intentionally trigger faults.
> >
> > [...]
>
> Applied, thanks!

Thank you! I have more ideas in mind but they will come in waves given
my current workload. Please let me know if you spot any errors, bugs,
or have suggestions for features to add.

-- Nico

>
> [1/2] page_fault: add mmap-backed ioengine for anonymous faults
>       commit: f5ea7d9e1d7c07053f23e764b281a5e47bf01ba3
> [2/2] Documentation: update the documentation to include the page_fault engine
>       commit: dceeff3a1dd87b5194c84845d77096b3cdb3cee3
>
> Best regards,
> --
> Jens Axboe
>
>
>


^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2026-04-08 13:23 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-04-08  1:20 [PATCH v3 0/2] page_fault: add mmap-backed ioengine for anonymous faults Nico Pache
2026-04-08  1:20 ` [PATCH v3 1/2] " Nico Pache
2026-04-08  1:20 ` [PATCH v3 2/2] Documentation: update the documentation to include the page_fault engine Nico Pache
2026-04-08  2:01 ` [PATCH v3 0/2] page_fault: add mmap-backed ioengine for anonymous faults Jens Axboe
2026-04-08 13:21   ` Nico Pache
2026-04-08  4:12 ` fiotestbot

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox