Linux block layer
 help / color / mirror / Atom feed
From: Mohamed Khalfella <mkhalfella@purestorage.com>
To: linux-block@vger.kernel.org
Cc: shinichiro.kawasaki@wdc.com, Keith Busch <kbusch@kernel.org>,
	Jens Axboe <axboe@kernel.dk>, Christoph Hellwig <hch@lst.de>,
	Sagi Grimberg <sagi@grimberg.me>, Hannes Reinecke <hare@suse.de>,
	John Meneghini <jmeneghi@redhat.com>,
	Jesse Taube <jtaubepe@redhat.com>,
	Randy Jennings <randyj@purestorage.com>,
	Dhaval Giani <dgiani@purestorage.com>,
	Mohamed Khalfella <mkhalfella@purestorage.com>
Subject: [PATCH blktests 4/5] src/nvme-ghost-write-detector: add an ABA ghost write detector
Date: Wed, 16 Sep 2026 20:06:24 -0600	[thread overview]
Message-ID: <20260917020752.1672578-5-mkhalfella@purestorage.com> (raw)
In-Reply-To: <20260917020752.1672578-1-mkhalfella@purestorage.com>

Consider a write X that is issued to an LBA and times out without ever
being acknowledged. The initiator retries it on another path, where it
lands. A write Y to the same LBA follows and lands as well. The original
X is still in flight somewhere in the fabric, and when it finally
reaches the device it overwrites Y. A read now returns X, a write the
initiator gave up on long ago.

Add a program to catch this. It opens the device O_DIRECT and, on each
iteration, issues a series of writes of distinct byte patterns to the
same LBA, the 4k block at offset 0, waits, then reads that block back
and checks every byte holds the pattern written last. Because the
patterns differ, the value that comes back identifies which write
reappeared.

Signed-off-by: Mohamed Khalfella <mkhalfella@purestorage.com>
---
 src/.gitignore                  |  1 +
 src/Makefile                    |  1 +
 src/nvme-ghost-write-detector.c | 86 +++++++++++++++++++++++++++++++++
 3 files changed, 88 insertions(+)
 create mode 100644 src/nvme-ghost-write-detector.c

diff --git a/src/.gitignore b/src/.gitignore
index e9869e1..9673be8 100644
--- a/src/.gitignore
+++ b/src/.gitignore
@@ -15,6 +15,7 @@
 /zbdioctl
 /miniublk
 /nvme-passthrough-meta
+/nvme-ghost-write-detector
 /ioctl-lbmd-query
 /nvme-passthru-admin-uring
 /nvme-delay-ioctl
diff --git a/src/Makefile b/src/Makefile
index dd64694..92b4d0c 100644
--- a/src/Makefile
+++ b/src/Makefile
@@ -24,6 +24,7 @@ C_TARGETS := \
 	mount_clear_sock \
 	nvme-delay-ioctl \
 	nvme-passthrough-meta \
+	nvme-ghost-write-detector \
 	ioctl-lbmd-query \
 	nbdsetsize \
 	openclose \
diff --git a/src/nvme-ghost-write-detector.c b/src/nvme-ghost-write-detector.c
new file mode 100644
index 0000000..bd42dde
--- /dev/null
+++ b/src/nvme-ghost-write-detector.c
@@ -0,0 +1,86 @@
+// SPDX-License-Identifier: GPL-3.0+
+// Copyright (C) 2026 Mohamed Khalfella
+
+#define _GNU_SOURCE
+#include <stdio.h>
+#include <stdlib.h>
+#include <fcntl.h>
+#include <unistd.h>
+#include <string.h>
+#include <malloc.h>
+#include <errno.h>
+#include <libgen.h>
+
+#define BUF_SIZE	4096
+#define ITERATIONS	10
+#define DELAY		3		/* seconds delay between iterations */
+#define WRITE_COUNT	10
+
+#define WRITE_OFFSET	0
+#define READ_OFFSET	WRITE_OFFSET
+
+int main(int argc, char **argv)
+{
+	int fd, i, w, off, ret;
+	char *buff;
+
+	fprintf(stdout, "starting %s test program\n", basename(argv[0]));
+
+	if (argc < 2) {
+		fprintf(stderr, "usage: %s /dev/nvmeXnY", argv[0]);
+		return 1;
+	}
+
+	fd = open(argv[1], O_RDWR | O_DIRECT);
+	if (fd < 0) {
+		fprintf(stderr, "failed to open device, errno = %d\n", errno);
+		return 1;
+	}
+
+	ret = posix_memalign((void **)&buff, BUF_SIZE, BUF_SIZE);
+	if (ret) {
+		fprintf(stderr, "failed to allocate buffer, ret = %d\n", ret);
+		goto out;
+	}
+
+	for (i = 0; i < ITERATIONS; i++) {
+		fprintf(stdout, "iteration number %d, writing data\n", i);
+
+		for (w = 0; w < WRITE_COUNT; w++) {
+			memset(buff, w, BUF_SIZE);
+			ret = pwrite(fd, buff, BUF_SIZE, WRITE_OFFSET);
+			if (ret != BUF_SIZE) {
+				fprintf(stderr, "failed to write buff, "
+						"ret = %d, errno = %d\n",
+						ret, errno);
+				goto out;
+			}
+		}
+
+		sleep(5);
+		fprintf(stdout, "validating written data\n");
+
+		ret = pread(fd, buff, BUF_SIZE, READ_OFFSET);
+		if (ret != BUF_SIZE) {
+			fprintf(stderr, "failed to read buff, "
+					"ret = %d, errno = %d\n",
+					ret, errno);
+			goto out;
+		}
+
+		for (off = 0; off < BUF_SIZE; off++) {
+			if (buff[off] != WRITE_COUNT - 1) {
+				fprintf(stdout, "validation failed\n");
+				goto out;
+			}
+		}
+
+		fprintf(stdout, "successfully validated\n");
+		sleep(DELAY);
+	}
+
+out:
+	fprintf(stdout, "finished %s test program\n", basename(argv[0]));
+	close(fd);
+	return ret;
+}
-- 
2.55.0


  parent reply	other threads:[~2026-09-17  2:09 UTC|newest]

Thread overview: 13+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-09-17  2:06 [PATCH blktests 0/5] nvme: detect ABA ghost writes on multipath fabrics Mohamed Khalfella
2026-09-17  2:06 ` [PATCH blktests 1/5] src/miniublk: add a control channel to the daemon Mohamed Khalfella
2026-09-17  2:06 ` [PATCH blktests 2/5] src/miniublk: add IO delay injection Mohamed Khalfella
2026-09-17  2:06 ` [PATCH blktests 3/5] src/miniublk: add the inject command Mohamed Khalfella
2026-09-17  2:06 ` Mohamed Khalfella [this message]
2026-09-21 18:38   ` [PATCH blktests 4/5] src/nvme-ghost-write-detector: add an ABA ghost write detector Jesse Taube
2026-09-23 17:00     ` Mohamed Khalfella
2026-09-17  2:06 ` [PATCH blktests 5/5] nvme/070: test for ABA ghost writes on a multipath fabrics namespace Mohamed Khalfella
2026-09-22 17:07   ` Jesse Taube
2026-09-23 16:56     ` Mohamed Khalfella
2026-09-23  8:18 ` [PATCH blktests 0/5] nvme: detect ABA ghost writes on multipath fabrics Shin'ichiro Kawasaki
2026-09-23 17:02   ` Mohamed Khalfella
2026-10-02  3:03     ` Shin'ichiro Kawasaki

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20260917020752.1672578-5-mkhalfella@purestorage.com \
    --to=mkhalfella@purestorage.com \
    --cc=axboe@kernel.dk \
    --cc=dgiani@purestorage.com \
    --cc=hare@suse.de \
    --cc=hch@lst.de \
    --cc=jmeneghi@redhat.com \
    --cc=jtaubepe@redhat.com \
    --cc=kbusch@kernel.org \
    --cc=linux-block@vger.kernel.org \
    --cc=randyj@purestorage.com \
    --cc=sagi@grimberg.me \
    --cc=shinichiro.kawasaki@wdc.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox