[PATCH 0/2] dm: add new loop and ram targets

[PATCH 0/2] dm: add new loop and ram targets

[Heinz Mauelshagen]

Enhancing IO performance compared to the kernels existing
loop driver thus better suiting respective requirements in
test setups, this patch series adds new "loop" and "ram" targets.

For measures see test results below.


The "loop" target maps segments to backing files.
Mapping table example:
0 4192256 loop /tmp/mapper_loop1
4192256 2097152 loop /dev/nvm/mapper_loop0


The "ram" target accesses RAM directly rather than through
tmpfs additionally enhancing performance compared to "loop"
thus avoding filesystem overhead.
Mapping table example:
0 8388608 ram

"ram" is a singleton target.


Performance test results for 4K and 32K IOPS comparing the loop driver
with dm-loop backed by tmpfs and dm-ram (all 2GiB backing size):

<TESTSCRIPT>
#!/bin/sh
for f in /tmp/loop0 /tmp/mapper_loop0
do
	dd if=/dev/zero of=$f bs=256M count=8 iflag=fullblock
done

losetup /dev/loop0 /tmp/loop0
sectors=`du -s /tmp/mapper_loop0|cut -f1`
dmsetup create loop0 --table "0 $sectors loop /tmp/mapper_loop0"
dmsetup create ram --table "0 $sectors ram"

for bs in 4K 32K
do
	for d in /dev/loop0 /dev/mapper/loop0 /dev/mapper/ram
	do
		echo 3 > /proc/sys/vm/drop_caches
		fio --bs=$bs --rw=randrw --numjobs=99 --group_reporting --iodepth=12 --runtime=3 --ioengine=libaio \
		    --loops=1 --direct=1 --exitall --name dc --filename=$d | egrep "read|write"
	done
done
</TESTSCRIPT>

<4K_RESULTS>
loop driver:
   read: IOPS=226k, BW=881MiB/s (924MB/s)(2645MiB/3003msec)
  write: IOPS=225k, BW=880MiB/s (923MB/s)(2643MiB/3003msec)
dm-loop target:
   read: IOPS=425k, BW=1661MiB/s (1742MB/s)(4990MiB/3004msec)
  write: IOPS=425k, BW=1662MiB/s (1743MB/s)(4992MiB/3004msec)
dm-ram target:
   read: IOPS=636k, BW=2484MiB/s (2605MB/s)(7464MiB/3005msec)
  write: IOPS=636k, BW=2484MiB/s (2605MB/s)(7464MiB/3005msec)
</4K_RESULTS>

<32K_RESULTS>
loop driver:
  read: IOPS=55.5k, BW=1733MiB/s (1817MB/s)(5215MiB/3009msec)
  write: IOPS=55.2k, BW=1726MiB/s (1810MB/s)(5195MiB/3009msec)
dm-loop target:
   read: IOPS=110k, BW=3452MiB/s (3620MB/s)(10.1GiB/3006msec)
  write: IOPS=110k, BW=3448MiB/s (3615MB/s)(10.1GiB/3006msec)
dm-ram target:
   read: IOPS=355k, BW=10.8GiB/s (11.6GB/s)(32.6GiB/3008msec)
  write: IOPS=355k, BW=10.8GiB/s (11.6GB/s)(32.6GiB/3008msec)
</32K_RESULTS>


Signed-off-by: Heinz Mauelshagen <heinzm@redhat.com>

Heinz Mauelshagen (2):
  dm loop: new target redirecting io to backing file(s)
  dm ram: new target redirecting io to RAM

 Documentation/device-mapper/loop.txt |  20 ++
 Documentation/device-mapper/ram.txt  |  15 ++
 drivers/md/Kconfig                   |  14 ++
 drivers/md/Makefile                  |   2 +
 drivers/md/dm-loop.c                 | 352 +++++++++++++++++++++++++++++++++++
 drivers/md/dm-ram.c                  | 101 ++++++++++
 6 files changed, 504 insertions(+)
 create mode 100644 Documentation/device-mapper/loop.txt
 create mode 100644 Documentation/device-mapper/ram.txt
 create mode 100644 drivers/md/dm-loop.c
 create mode 100644 drivers/md/dm-ram.c

-- 
2.14.3

[PATCH 1/2] dm loop: new target redirecting io to backing file(s)

[Heinz Mauelshagen]

Signed-off-by: Heinz Mauelshagen <heinzm@redhat.com>
---
 Documentation/device-mapper/loop.txt |  20 ++
 drivers/md/Kconfig                   |   7 +
 drivers/md/Makefile                  |   1 +
 drivers/md/dm-loop.c                 | 352 +++++++++++++++++++++++++++++++++++
 4 files changed, 380 insertions(+)
 create mode 100644 Documentation/device-mapper/loop.txt
 create mode 100644 drivers/md/dm-loop.c

diff --git a/Documentation/device-mapper/loop.txt b/Documentation/device-mapper/loop.txt
new file mode 100644
index 000000000000..a8c1e0cae62e
--- /dev/null
+++ b/Documentation/device-mapper/loop.txt
@@ -0,0 +1,20 @@
+dm-loop
+=======
+
+Device-Mapper's "loop" target provides a mapping to a
+backing file. This is similar to a loop device created
+by losetup with less overhead, hence higher iops and bandwidth.
+
+
+Parameters: <path_name>
+
+<path_name> path to existing file to map block io to
+
+
+Example:
+
+dmsetup create loop --table "0 $TWO_GiB loop /tmp/loopfile"
+
+This will create a 2GiB loop device /dev/mapper/loop mapped
+to existing /tmp/loopfile which has to be 2GiB in size or
+bigger for the creation to succeed.
diff --git a/drivers/md/Kconfig b/drivers/md/Kconfig
index 83b9362be09c..1d80783b9ee8 100644
--- a/drivers/md/Kconfig
+++ b/drivers/md/Kconfig
@@ -390,6 +390,13 @@ config DM_ZERO
 	  A target that discards writes, and returns all zeroes for
 	  reads.  Useful in some recovery situations.
 
+config DM_LOOP
+	tristate "Loop target (EXPERIMENTAL)"
+	depends on BLK_DEV_DM
+	---help---
+	  A target that redirects IOs to a backing file.
+	  E.g. useful in testing.
+
 config DM_MULTIPATH
 	tristate "Multipath target"
 	depends on BLK_DEV_DM
diff --git a/drivers/md/Makefile b/drivers/md/Makefile
index f701bb211783..68baf79c5536 100644
--- a/drivers/md/Makefile
+++ b/drivers/md/Makefile
@@ -57,6 +57,7 @@ obj-$(CONFIG_DM_PERSISTENT_DATA)	+= persistent-data/
 obj-$(CONFIG_DM_MIRROR)		+= dm-mirror.o dm-log.o dm-region-hash.o
 obj-$(CONFIG_DM_LOG_USERSPACE)	+= dm-log-userspace.o
 obj-$(CONFIG_DM_ZERO)		+= dm-zero.o
+obj-$(CONFIG_DM_LOOP)		+= dm-loop.o
 obj-$(CONFIG_DM_RAID)	+= dm-raid.o
 obj-$(CONFIG_DM_THIN_PROVISIONING)	+= dm-thin-pool.o
 obj-$(CONFIG_DM_VERITY)		+= dm-verity.o
diff --git a/drivers/md/dm-loop.c b/drivers/md/dm-loop.c
new file mode 100644
index 000000000000..35adde3f64e0
--- /dev/null
+++ b/drivers/md/dm-loop.c
@@ -0,0 +1,352 @@
+/*
+ * Copyright (C) 2018 Red Hat GmbH
+ *
+ * Simple loop target which redirects
+ * io in parallel to a backing file.
+ *
+ * This file is released under the GPL.
+ */
+
+#include <linux/device-mapper.h>
+#include <linux/falloc.h>
+#include <linux/uio.h>
+#include <linux/module.h>
+
+#define DM_MSG_PREFIX "loop"
+#define	WORKQUEUE_NAME	"dm-kloopd"
+
+/* Global workqueue shared by all loop mappings */
+static struct workqueue_struct *kloopd_wq = NULL;
+static atomic_t kloopd_wq_users = ATOMIC_INIT(0);
+
+/* Registry of all loop devices to prevent using the same files multiple times */
+static LIST_HEAD(loop_devs);
+
+/* loop context */
+struct loop_c {
+	struct file *file; /* Backing file */
+
+	/* Workqueue */
+	spinlock_t lock;
+	struct bio_list bios;
+	struct work_struct bios_ws;
+
+	struct dm_target *ti;
+	char *path; /* Status table output */
+	struct list_head list;
+};
+
+/* bio context for wrokqueue */
+struct bio_c {
+	struct work_struct bio_ws;
+	struct bio *bio;
+	struct loop_c *lc;
+};
+
+/* Is file of @lc already in use? */
+static int __file_in_use(struct loop_c *lc)
+{
+	struct loop_c *cur;
+
+	list_for_each_entry(cur, &loop_devs, list)
+		if (cur != lc && cur->file->f_inode == lc->file->f_inode)
+			return -EPERM;
+	return 0;
+}
+
+/* Use punch hole to discard bio_sectors(@bio) in backing file starting at @pos */
+static void loop_discard(struct loop_c *lc, struct bio *bio)
+{
+	if (lc->file->f_op->fallocate) {
+		int r = lc->file->f_op->fallocate(lc->file,
+						  FALLOC_FL_PUNCH_HOLE | FALLOC_FL_KEEP_SIZE,
+						  to_bytes(bio->bi_iter.bi_sector),
+						  to_bytes(bio_sectors(bio)));
+		if (unlikely(r && r != -EOPNOTSUPP && r != -EINVAL))
+			bio->bi_status = errno_to_blk_status(-EIO);
+	}
+}
+
+/* Sync a backing file range @pos - @end (FUA, PREFLUSH) */
+static void loop_fsync_range(struct loop_c *lc, loff_t pos, loff_t end)
+{
+	int r = vfs_fsync_range(lc->file, pos, end, 0);
+
+	if (unlikely(r && r != -EINVAL))
+		DMERR("Error fsync range");
+	else
+		cond_resched();
+}
+
+/* Check for any IO error after reading or writing a bio_vec */
+static int loop_check_io_error(ssize_t bytes, loff_t pos,
+			       struct bio_vec *bvec, const char *what)
+{
+	if (likely(bytes == bvec->bv_len))
+		return 0;
+
+	DMERR_LIMIT("%s error[%lld] at byte offset %llu, length %u",
+		    what, (long long) bytes, (unsigned long long) pos, bvec->bv_len);
+	return (bytes < 0) ? (int) bytes : -EIO;
+}
+
+/*
+ * Read/write @bio payload from/to backing file at @pos.
+ *
+ * Returns 0 on success and < 0 on error (e.g. -EIO).
+ */
+static void loop_rw_bio(struct loop_c *lc, struct bio *bio)
+{
+	int r = 0;
+	bool write = op_is_write(bio_op(bio));
+	ssize_t bytes;
+	loff_t bio_begin, bio_end = 0, pos = to_bytes(bio->bi_iter.bi_sector);
+	struct bio_vec bvec;
+	struct iov_iter io_iter;
+
+	if (unlikely(write && (bio->bi_opf & (REQ_FUA | REQ_PREFLUSH)))) {
+		bio_begin = pos;
+		bio_end = pos + bio_cur_bytes(bio);
+	}
+
+	bio_for_each_segment(bvec, bio, bio->bi_iter) {
+		iov_iter_bvec(&io_iter, ITER_BVEC, &bvec, 1, bvec.bv_len);
+
+		if (write) {
+			file_start_write(lc->file);
+			bytes = vfs_iter_write(lc->file, &io_iter, &pos, 0);
+			file_end_write(lc->file);
+			r = loop_check_io_error(bytes, pos, &bvec, "write");
+			if (r)
+				break;
+		} else {
+			bytes = vfs_iter_read(lc->file, &io_iter, &pos, 0);
+			r = loop_check_io_error(bytes, pos, &bvec, "read");
+			if (r) {
+				zero_fill_bio(bio);
+				break;
+			}
+
+			flush_dcache_page(bvec.bv_page);
+		}
+
+		cond_resched();
+	}
+
+	if (unlikely(r < 0))
+		bio->bi_status = errno_to_blk_status(r);
+
+	if (unlikely(bio_end))
+		/* FUA, ... requested -> flush the bio defined range */
+		loop_fsync_range(lc, bio_begin, bio_end);
+}
+
+/* Worker thread function to process file IO for single bio */
+static void loop_process_bio(struct work_struct *work)
+{
+	struct bio_c *bio_c = container_of(work, struct bio_c, bio_ws);
+	struct bio *bio = bio_c->bio;
+
+	current->flags |= PF_LESS_THROTTLE;
+
+	switch (bio_op(bio)) {
+	case REQ_OP_READ:
+	case REQ_OP_WRITE:
+		loop_rw_bio(bio_c->lc, bio);
+		break;
+	case REQ_OP_FLUSH:
+		loop_fsync_range(bio_c->lc, 0, LLONG_MAX);
+		break;
+	case REQ_OP_DISCARD:
+		loop_discard(bio_c->lc, bio);
+		break;
+	default:
+		bio->bi_status = errno_to_blk_status(-EIO);
+	}
+
+	bio_endio(bio);
+}
+
+/* Worker thread function to process all bios */
+static void loop_process_bios(struct work_struct *work)
+{
+	struct loop_c *lc = container_of(work, struct loop_c, bios_ws);
+	struct bio_list bl;
+	struct bio *bio;
+	struct bio_c *bio_c;
+
+	current->flags |= PF_LESS_THROTTLE;
+
+	/* Take out input bios to process... */
+	bio_list_init(&bl);
+	spin_lock_irq(&lc->lock);
+	bio_list_merge(&bl, &lc->bios);
+	bio_list_init(&lc->bios);
+	spin_unlock_irq(&lc->lock);
+
+	while ((bio = bio_list_pop(&bl))) {
+		bio_c = dm_per_bio_data(bio, lc->ti->per_io_data_size);
+		INIT_WORK(&bio_c->bio_ws, loop_process_bio);
+		bio_c->bio = bio;
+		bio_c->lc = lc;
+		queue_work(kloopd_wq, &bio_c->bio_ws);
+	}
+}
+
+/* Release loop context resources of @lc */
+static void destroy_loop(struct loop_c *lc)
+{
+	if (lc) {
+		list_del(&lc->list);
+		if (kloopd_wq && atomic_dec_and_test(&kloopd_wq_users)) {
+			destroy_workqueue(kloopd_wq);
+			kloopd_wq = NULL;
+		}
+		if (lc->file)
+			filp_close(lc->file, NULL);
+		if (lc->path)
+			kfree(lc->path);
+		kfree(lc);
+	}
+}
+
+/*
+ * Construct a loop mapping on a (sparse) file.
+ *
+ * Argument:
+ *    <file_path>: path to backing file
+ */
+static int loop_ctr(struct dm_target *ti, unsigned int argc, char **argv)
+{
+	int r = -ENOMEM;
+	struct loop_c *lc;
+
+	if (argc != 1) {
+		ti->error = "Invalid argument count";
+		return -EINVAL;
+	}
+
+	lc = ti->private = kzalloc(sizeof(*lc), GFP_KERNEL);
+	if (!lc) {
+		ti->error = "Cannot allocate context";
+		goto err;
+	}
+
+	spin_lock_init(&lc->lock);
+	bio_list_init(&lc->bios);
+	INIT_WORK(&lc->bios_ws, loop_process_bios);
+	list_add(&lc->list, &loop_devs);
+
+	ti->num_discard_bios = 1;
+	ti->discards_supported = true;
+	ti->flush_supported = true;
+	ti->per_io_data_size = sizeof(struct bio_c);
+	lc->ti = ti;
+
+	lc->path = kstrdup(argv[0], GFP_KERNEL);
+	if (!lc->path) {
+		ti->error = "Cannot allocate path";
+		goto err;
+	}
+
+	/* Open existing backing file */
+	lc->file = filp_open(lc->path, O_EXCL | O_LARGEFILE | O_RDWR, 0);
+	if (IS_ERR(lc->file)) {
+		ti->error = "Cannot open backing file";
+		r = PTR_ERR(lc->file);
+		lc->file = NULL;
+		goto err;
+	}
+
+	r = __file_in_use(lc);
+	if (r) {
+		ti->error = "Cannot use same file multiple times";
+		goto err;
+	}
+
+	if (ti->len > to_sector(i_size_read(lc->file->f_mapping->host))) {
+		ti->error = "Backing file too small";
+		r = -ENOSPC;
+		goto err;
+	}
+
+	r = dm_set_target_max_io_len(ti, min(ti->len, (sector_t) UINT_MAX));
+	if (r)
+		goto err;
+
+	/* Alloc global workqueue with first loop mapping construction */
+	if (atomic_inc_return(&kloopd_wq_users) == 1) {
+		kloopd_wq = alloc_workqueue(WORKQUEUE_NAME, WQ_MEM_RECLAIM | WQ_UNBOUND, 0);
+		if (!kloopd_wq) {
+			DMERR("Cannot start workqueue %s", WORKQUEUE_NAME);
+			atomic_set(&kloopd_wq_users, 0);
+			r = -ENOMEM;
+			goto err;
+		}
+	}
+
+	return 0;
+err:
+	destroy_loop(lc);
+	return r;
+}
+
+static void loop_dtr(struct dm_target *ti)
+{
+	destroy_loop(ti->private);
+}
+
+static int loop_map(struct dm_target *ti, struct bio *bio)
+{
+	struct loop_c *lc = ti->private;
+
+	/* Not a singleton target... */
+	bio->bi_iter.bi_sector = dm_target_offset(ti, bio->bi_iter.bi_sector);
+
+	spin_lock_irq(&lc->lock);
+	bio_list_add(&lc->bios, bio);
+	spin_unlock_irq(&lc->lock);
+
+	queue_work(kloopd_wq, &lc->bios_ws);
+
+	return DM_MAPIO_SUBMITTED;
+}
+
+static void loop_status(struct dm_target *ti, status_type_t type,
+			unsigned status_flags, char *result, unsigned maxlen)
+{
+	if (type == STATUSTYPE_TABLE) {
+		struct loop_c *lc = ti->private;
+		int sz = 0;
+
+		DMEMIT("%s", lc->path);
+	}
+}
+
+static struct target_type loop_target = {
+	.name	     = "loop",
+	.version     = {1, 0, 0},
+	.module      = THIS_MODULE,
+	.ctr	     = loop_ctr,
+	.dtr	     = loop_dtr,
+	.map	     = loop_map,
+	.status	     = loop_status,
+};
+
+static int __init dm_loop_init(void)
+{
+	return dm_register_target(&loop_target);
+}
+
+static void __exit dm_loop_exit(void)
+{
+	dm_unregister_target(&loop_target);
+}
+
+/* Module hooks */
+module_init(dm_loop_init);
+module_exit(dm_loop_exit);
+
+MODULE_DESCRIPTION(DM_NAME " loop target");
+MODULE_AUTHOR("Heinz Mauelshagen <dm-devel@redhat.com>");
+MODULE_LICENSE("GPL");
-- 
2.14.3

[PATCH 2/2] dm ram: new target redirecting io to RAM

[Heinz Mauelshagen]

Signed-off-by: Heinz Mauelshagen <heinzm@redhat.com>
---
 Documentation/device-mapper/ram.txt |  15 ++++++
 drivers/md/Kconfig                  |   7 +++
 drivers/md/Makefile                 |   1 +
 drivers/md/dm-ram.c                 | 101 ++++++++++++++++++++++++++++++++++++
 4 files changed, 124 insertions(+)
 create mode 100644 Documentation/device-mapper/ram.txt
 create mode 100644 drivers/md/dm-ram.c

diff --git a/Documentation/device-mapper/ram.txt b/Documentation/device-mapper/ram.txt
new file mode 100644
index 000000000000..4358fbf57cae
--- /dev/null
+++ b/Documentation/device-mapper/ram.txt
@@ -0,0 +1,15 @@
+dm-ram
+======
+
+Device-Mapper's "ram" target provides a mapping to RAM.
+This is comparible to a loop device mapped to a tmpfs
+file but has less overhead, hence higher IOPS and bandwidth.
+
+dm-ram has no target-specific parameters.
+
+Example:
+
+dmsetup create ramdisk --table "0 $((2048*1024*2)) ram"
+
+This will create a 2GiB ramdisk /dev/mapper/ramdisk and
+will suceed presuming more than 2GiB free RAM are available.
diff --git a/drivers/md/Kconfig b/drivers/md/Kconfig
index 1d80783b9ee8..abea7b8771f4 100644
--- a/drivers/md/Kconfig
+++ b/drivers/md/Kconfig
@@ -397,6 +397,13 @@ config DM_LOOP
 	  A target that redirects IOs to a backing file.
 	  E.g. useful in testing.
 
+config DM_RAM
+	tristate "RAM disk target (EXPERIMENTAL)"
+	depends on BLK_DEV_DM
+	---help---
+	  A target that redirects IO to RAM.
+	  E.g. useful in testing.
+
 config DM_MULTIPATH
 	tristate "Multipath target"
 	depends on BLK_DEV_DM
diff --git a/drivers/md/Makefile b/drivers/md/Makefile
index 68baf79c5536..f404223629ab 100644
--- a/drivers/md/Makefile
+++ b/drivers/md/Makefile
@@ -58,6 +58,7 @@ obj-$(CONFIG_DM_MIRROR)		+= dm-mirror.o dm-log.o dm-region-hash.o
 obj-$(CONFIG_DM_LOG_USERSPACE)	+= dm-log-userspace.o
 obj-$(CONFIG_DM_ZERO)		+= dm-zero.o
 obj-$(CONFIG_DM_LOOP)		+= dm-loop.o
+obj-$(CONFIG_DM_RAM)		+= dm-ram.o
 obj-$(CONFIG_DM_RAID)	+= dm-raid.o
 obj-$(CONFIG_DM_THIN_PROVISIONING)	+= dm-thin-pool.o
 obj-$(CONFIG_DM_VERITY)		+= dm-verity.o
diff --git a/drivers/md/dm-ram.c b/drivers/md/dm-ram.c
new file mode 100644
index 000000000000..9ef1a8516e1a
--- /dev/null
+++ b/drivers/md/dm-ram.c
@@ -0,0 +1,101 @@
+/*
+ * Copyright (C) 2018 Red Hat GmbH
+ *
+ * Ram disk target.
+ *
+ * This file is released under the GPL.
+ */
+
+#include <linux/device-mapper.h>
+#include <linux/module.h>
+
+static int ram_ctr(struct dm_target *ti, unsigned int argc, char **argv)
+{
+	int r;
+
+	if (argc) {
+		ti->error = "No arguments required";
+		return -EINVAL;
+	}
+
+	/* Allocation will fail anyway for the time being, but... */
+	if (ti->len != to_sector(to_bytes(ti->len))) {
+		ti->error = "Invalid target length";
+		return -EINVAL;
+	}
+
+	r = dm_set_target_max_io_len(ti, min(ti->len, (sector_t) UINT_MAX));
+	if (r)
+		return r;
+
+	ti->private = vmalloc(to_bytes(ti->len));
+	if (!ti->private) {
+		ti->error = "Cannot allocate ram disk";
+		return -ENOMEM;
+	}
+
+	ti->num_discard_bios = 1;
+	ti->discards_supported = true;
+
+	return 0;
+}
+
+static void ram_dtr(struct dm_target *ti)
+{
+	vfree(ti->private);
+}
+
+static int ram_map(struct dm_target *ti, struct bio *bio)
+{
+	struct bio_vec bvec;
+
+	switch (bio_op(bio)) {
+	case REQ_OP_READ:
+		bio_for_each_segment(bvec, bio, bio->bi_iter)
+			memcpy(page_address(bvec.bv_page) + bvec.bv_offset,
+			       ti->private + to_bytes(bio->bi_iter.bi_sector), bvec.bv_len);
+		break;
+	case REQ_OP_WRITE:
+		bio_for_each_segment(bvec, bio, bio->bi_iter)
+			memcpy(ti->private + to_bytes(bio->bi_iter.bi_sector),
+			       page_address(bvec.bv_page) + bvec.bv_offset, bvec.bv_len);
+		break;
+	case REQ_OP_DISCARD:
+		memset(ti->private + to_bytes(bio->bi_iter.bi_sector), 0, bio_cur_bytes(bio));
+		break;
+	default:
+		return DM_MAPIO_KILL;
+	}
+
+	bio_endio(bio);
+
+	return DM_MAPIO_SUBMITTED;
+}
+
+static struct target_type ram_target = {
+	.name	     = "ram",
+	.version     = {1, 0, 0},
+	.features    = DM_TARGET_SINGLETON | DM_TARGET_ALWAYS_WRITEABLE,
+	.module      = THIS_MODULE,
+	.ctr	     = ram_ctr,
+	.dtr	     = ram_dtr,
+	.map	     = ram_map
+};
+
+static int __init dm_ram_init(void)
+{
+	return dm_register_target(&ram_target);
+}
+
+static void __exit dm_ram_exit(void)
+{
+	dm_unregister_target(&ram_target);
+}
+
+/* Module hooks */
+module_init(dm_ram_init);
+module_exit(dm_ram_exit);
+
+MODULE_DESCRIPTION(DM_NAME " ram disk target");
+MODULE_AUTHOR("Heinz Mauelshagen <dm-devel@redhat.com>");
+MODULE_LICENSE("GPL");
-- 
2.14.3

Re: [PATCH 0/2] dm: add new loop and ram targets

[Mike Snitzer]

On Wed, Jan 17 2018 at  2:33pm -0500,
Heinz Mauelshagen <heinzm@redhat.com> wrote:

> Enhancing IO performance compared to the kernels existing
> loop driver thus better suiting respective requirements in
> test setups, this patch series adds new "loop" and "ram" targets.
> 
> For measures see test results below.
> 
> 
> The "loop" target maps segments to backing files.
> Mapping table example:
> 0 4192256 loop /tmp/mapper_loop1
> 4192256 2097152 loop /dev/nvm/mapper_loop0
> 
> 
> The "ram" target accesses RAM directly rather than through
> tmpfs additionally enhancing performance compared to "loop"
> thus avoding filesystem overhead.
> Mapping table example:
> 0 8388608 ram
> 
> "ram" is a singleton target.
> 
> 
> Performance test results for 4K and 32K IOPS comparing the loop driver
> with dm-loop backed by tmpfs and dm-ram (all 2GiB backing size):
> 
> <TESTSCRIPT>
> #!/bin/sh
> for f in /tmp/loop0 /tmp/mapper_loop0
> do
> 	dd if=/dev/zero of=$f bs=256M count=8 iflag=fullblock
> done
> 
> losetup /dev/loop0 /tmp/loop0
> sectors=`du -s /tmp/mapper_loop0|cut -f1`
> dmsetup create loop0 --table "0 $sectors loop /tmp/mapper_loop0"
> dmsetup create ram --table "0 $sectors ram"
> 
> for bs in 4K 32K
> do
> 	for d in /dev/loop0 /dev/mapper/loop0 /dev/mapper/ram
> 	do
> 		echo 3 > /proc/sys/vm/drop_caches
> 		fio --bs=$bs --rw=randrw --numjobs=99 --group_reporting --iodepth=12 --runtime=3 --ioengine=libaio \
> 		    --loops=1 --direct=1 --exitall --name dc --filename=$d | egrep "read|write"
> 	done
> done
> </TESTSCRIPT>
> 
> <4K_RESULTS>
> loop driver:
>    read: IOPS=226k, BW=881MiB/s (924MB/s)(2645MiB/3003msec)
>   write: IOPS=225k, BW=880MiB/s (923MB/s)(2643MiB/3003msec)
> dm-loop target:
>    read: IOPS=425k, BW=1661MiB/s (1742MB/s)(4990MiB/3004msec)
>   write: IOPS=425k, BW=1662MiB/s (1743MB/s)(4992MiB/3004msec)
> dm-ram target:
>    read: IOPS=636k, BW=2484MiB/s (2605MB/s)(7464MiB/3005msec)
>   write: IOPS=636k, BW=2484MiB/s (2605MB/s)(7464MiB/3005msec)
> </4K_RESULTS>
> 
> <32K_RESULTS>
> loop driver:
>   read: IOPS=55.5k, BW=1733MiB/s (1817MB/s)(5215MiB/3009msec)
>   write: IOPS=55.2k, BW=1726MiB/s (1810MB/s)(5195MiB/3009msec)
> dm-loop target:
>    read: IOPS=110k, BW=3452MiB/s (3620MB/s)(10.1GiB/3006msec)
>   write: IOPS=110k, BW=3448MiB/s (3615MB/s)(10.1GiB/3006msec)
> dm-ram target:
>    read: IOPS=355k, BW=10.8GiB/s (11.6GB/s)(32.6GiB/3008msec)
>   write: IOPS=355k, BW=10.8GiB/s (11.6GB/s)(32.6GiB/3008msec)
> </32K_RESULTS>
> 
> 
> Signed-off-by: Heinz Mauelshagen <heinzm@redhat.com>
> 
> Heinz Mauelshagen (2):
>   dm loop: new target redirecting io to backing file(s)
>   dm ram: new target redirecting io to RAM
> 
>  Documentation/device-mapper/loop.txt |  20 ++
>  Documentation/device-mapper/ram.txt  |  15 ++
>  drivers/md/Kconfig                   |  14 ++
>  drivers/md/Makefile                  |   2 +
>  drivers/md/dm-loop.c                 | 352 +++++++++++++++++++++++++++++++++++
>  drivers/md/dm-ram.c                  | 101 ++++++++++
>  6 files changed, 504 insertions(+)
>  create mode 100644 Documentation/device-mapper/loop.txt
>  create mode 100644 Documentation/device-mapper/ram.txt
>  create mode 100644 drivers/md/dm-loop.c
>  create mode 100644 drivers/md/dm-ram.c

My initial thought for dm-ram was: why?  (considering we have brd and
pmem and null_blk).  But for 100 lines of code if nothing else it could
serve as yet another example DM target for those interested in learning
more about how to implement a DM target.  Would be good to compare its
performance with brd, null_blk and pmem though.

As for dm-loop, doubling the performance of the loopback driver is quite
nice (especially with only 1/7 the number of lines of code as
drives/block/loop.c).

I'll review both of these closer in the coming days but it is getting
_really_ close to the 4.16 merge window (likley opens Sunday)  So they
may have to wait until 4.17.  We'll see.

Thanks,
Mike

Re: [PATCH 0/2] dm: add new loop and ram targets

[Heinz Mauelshagen]

On 01/17/2018 10:29 PM, Mike Snitzer wrote:
> On Wed, Jan 17 2018 at  2:33pm -0500,
> Heinz Mauelshagen <heinzm@redhat.com> wrote:
>
>> Enhancing IO performance compared to the kernels existing
>> loop driver thus better suiting respective requirements in
>> test setups, this patch series adds new "loop" and "ram" targets.
>>
>> For measures see test results below.
>>
>>
>> The "loop" target maps segments to backing files.
>> Mapping table example:
>> 0 4192256 loop /tmp/mapper_loop1
>> 4192256 2097152 loop /dev/nvm/mapper_loop0
>>
>>
>> The "ram" target accesses RAM directly rather than through
>> tmpfs additionally enhancing performance compared to "loop"
>> thus avoding filesystem overhead.
>> Mapping table example:
>> 0 8388608 ram
>>
>> "ram" is a singleton target.
>>
>>
>> Performance test results for 4K and 32K IOPS comparing the loop driver
>> with dm-loop backed by tmpfs and dm-ram (all 2GiB backing size):
>>
>> <TESTSCRIPT>
>> #!/bin/sh
>> for f in /tmp/loop0 /tmp/mapper_loop0
>> do
>> 	dd if=/dev/zero of=$f bs=256M count=8 iflag=fullblock
>> done
>>
>> losetup /dev/loop0 /tmp/loop0
>> sectors=`du -s /tmp/mapper_loop0|cut -f1`
>> dmsetup create loop0 --table "0 $sectors loop /tmp/mapper_loop0"
>> dmsetup create ram --table "0 $sectors ram"
>>
>> for bs in 4K 32K
>> do
>> 	for d in /dev/loop0 /dev/mapper/loop0 /dev/mapper/ram
>> 	do
>> 		echo 3 > /proc/sys/vm/drop_caches
>> 		fio --bs=$bs --rw=randrw --numjobs=99 --group_reporting --iodepth=12 --runtime=3 --ioengine=libaio \
>> 		    --loops=1 --direct=1 --exitall --name dc --filename=$d | egrep "read|write"
>> 	done
>> done
>> </TESTSCRIPT>
>>
>> <4K_RESULTS>
>> loop driver:
>>     read: IOPS=226k, BW=881MiB/s (924MB/s)(2645MiB/3003msec)
>>    write: IOPS=225k, BW=880MiB/s (923MB/s)(2643MiB/3003msec)
>> dm-loop target:
>>     read: IOPS=425k, BW=1661MiB/s (1742MB/s)(4990MiB/3004msec)
>>    write: IOPS=425k, BW=1662MiB/s (1743MB/s)(4992MiB/3004msec)
>> dm-ram target:
>>     read: IOPS=636k, BW=2484MiB/s (2605MB/s)(7464MiB/3005msec)
>>    write: IOPS=636k, BW=2484MiB/s (2605MB/s)(7464MiB/3005msec)
>> </4K_RESULTS>
>>
>> <32K_RESULTS>
>> loop driver:
>>    read: IOPS=55.5k, BW=1733MiB/s (1817MB/s)(5215MiB/3009msec)
>>    write: IOPS=55.2k, BW=1726MiB/s (1810MB/s)(5195MiB/3009msec)
>> dm-loop target:
>>     read: IOPS=110k, BW=3452MiB/s (3620MB/s)(10.1GiB/3006msec)
>>    write: IOPS=110k, BW=3448MiB/s (3615MB/s)(10.1GiB/3006msec)
>> dm-ram target:
>>     read: IOPS=355k, BW=10.8GiB/s (11.6GB/s)(32.6GiB/3008msec)
>>    write: IOPS=355k, BW=10.8GiB/s (11.6GB/s)(32.6GiB/3008msec)
>> </32K_RESULTS>
>>
>>
>> Signed-off-by: Heinz Mauelshagen <heinzm@redhat.com>
>>
>> Heinz Mauelshagen (2):
>>    dm loop: new target redirecting io to backing file(s)
>>    dm ram: new target redirecting io to RAM
>>
>>   Documentation/device-mapper/loop.txt |  20 ++
>>   Documentation/device-mapper/ram.txt  |  15 ++
>>   drivers/md/Kconfig                   |  14 ++
>>   drivers/md/Makefile                  |   2 +
>>   drivers/md/dm-loop.c                 | 352 +++++++++++++++++++++++++++++++++++
>>   drivers/md/dm-ram.c                  | 101 ++++++++++
>>   6 files changed, 504 insertions(+)
>>   create mode 100644 Documentation/device-mapper/loop.txt
>>   create mode 100644 Documentation/device-mapper/ram.txt
>>   create mode 100644 drivers/md/dm-loop.c
>>   create mode 100644 drivers/md/dm-ram.c
> My initial thought for dm-ram was: why?  (considering we have brd and
> pmem and null_blk).  But for 100 lines of code if nothing else it could
> serve as yet another example DM target for those interested in learning
> more about how to implement a DM target.  Would be good to compare its
> performance with brd, null_blk and pmem though.

With it we get the dm flexibility to set up ramdisks as opposed to
brd module parameters.  It's performance is pretty similar
to brd but it's faster for larger block sizes.
Yes, the value of its simplicity for beginners is an additonal goody.

null_blk doesn't quite fit the list lagging backing store support?

Some numbers in brd, dm-ram, null_blk order:
# fio --bs=32k --rw=randrw --numjobs=99 --group_reporting --iodepth=12 
--runtime=3  --ioengine=libaio --loops=1 --direct=1 --exitall --name 
pipi --filename=/dev/ram0|egrep "read|write"
    read: IOPS=334k, BW=10.2GiB/s (10.0GB/s)(30.7GiB/3009msec)
   write: IOPS=334k, BW=10.2GiB/s (10.0GB/s)(30.7GiB/3009msec)

# fio --bs=32k --rw=randrw --numjobs=99 --group_reporting --iodepth=12 
--runtime=3  --ioengine=libaio --loops=1 --direct=1 --exitall --name 
pipi --filename=/dev/mapper/ram|egrep "read|write"
    read: IOPS=354k, BW=10.8GiB/s (11.6GB/s)(32.4GiB/3005msec)
   write: IOPS=354k, BW=10.8GiB/s (11.6GB/s)(32.5GiB/3005msec)

# fio --bs=32k --rw=randrw --numjobs=99 --group_reporting --iodepth=12 
--runtime=3  --ioengine=libaio --loops=1 --direct=1 --exitall --name 
pipi --filename=/dev/nullb0|egrep "read|write"
    read: IOPS=337k, BW=10.3GiB/s (11.0GB/s)(30.9GiB/3007msec)
   write: IOPS=337k, BW=10.3GiB/s (11.0GB/s)(30.9GiB/3007msec)

>
> As for dm-loop, doubling the performance of the loopback driver is quite
> nice (especially with only 1/7 the number of lines of code as
> drives/block/loop.c).

Yes, found this challenging in particular too.
Didn't bother to cover direct io or async io (yet).
Much rather wanted to keep it simple.

Cheers,
Heinz

>
> I'll review both of these closer in the coming days but it is getting
> _really_ close to the 4.16 merge window (likley opens Sunday)  So they
> may have to wait until 4.17.  We'll see.
>
> Thanks,
> Mike
>
> --
> dm-devel mailing list
> dm-devel@redhat.com
> https://www.redhat.com/mailman/listinfo/dm-devel

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel

Re: [PATCH 0/2] dm: add new loop and ram targets

[Mike Snitzer]

On Wed, Jan 17 2018 at  6:21pm -0500,
Heinz Mauelshagen <heinzm@redhat.com> wrote:

> On 01/17/2018 10:29 PM, Mike Snitzer wrote:
>
> >My initial thought for dm-ram was: why?  (considering we have brd and
> >pmem and null_blk).  But for 100 lines of code if nothing else it could
> >serve as yet another example DM target for those interested in learning
> >more about how to implement a DM target.  Would be good to compare its
> >performance with brd, null_blk and pmem though.
> 
> With it we get the dm flexibility to set up ramdisks as opposed to
> brd module parameters.  It's performance is pretty similar
> to brd but it's faster for larger block sizes.
> Yes, the value of its simplicity for beginners is an additonal goody.
> 
> null_blk doesn't quite fit the list lagging backing store support?

Sure, but I was saying null_blk vs dm-ram.  dm-ram doesn't use a backing
store.

> Some numbers in brd, dm-ram, null_blk order:
> # fio --bs=32k --rw=randrw --numjobs=99 --group_reporting
> --iodepth=12 --runtime=3  --ioengine=libaio --loops=1 --direct=1
> --exitall --name pipi --filename=/dev/ram0|egrep "read|write"
>    read: IOPS=334k, BW=10.2GiB/s (10.0GB/s)(30.7GiB/3009msec)
>   write: IOPS=334k, BW=10.2GiB/s (10.0GB/s)(30.7GiB/3009msec)
> 
> # fio --bs=32k --rw=randrw --numjobs=99 --group_reporting
> --iodepth=12 --runtime=3  --ioengine=libaio --loops=1 --direct=1
> --exitall --name pipi --filename=/dev/mapper/ram|egrep "read|write"
>    read: IOPS=354k, BW=10.8GiB/s (11.6GB/s)(32.4GiB/3005msec)
>   write: IOPS=354k, BW=10.8GiB/s (11.6GB/s)(32.5GiB/3005msec)
> 
> # fio --bs=32k --rw=randrw --numjobs=99 --group_reporting
> --iodepth=12 --runtime=3  --ioengine=libaio --loops=1 --direct=1
> --exitall --name pipi --filename=/dev/nullb0|egrep "read|write"
>    read: IOPS=337k, BW=10.3GiB/s (11.0GB/s)(30.9GiB/3007msec)
>   write: IOPS=337k, BW=10.3GiB/s (11.0GB/s)(30.9GiB/3007msec)

Nice, dm-ram is doing best for that test.

> >As for dm-loop, doubling the performance of the loopback driver is quite
> >nice (especially with only 1/7 the number of lines of code as
> >drives/block/loop.c).
> 
> Yes, found this challenging in particular too.
> Didn't bother to cover direct io or async io (yet).
> Much rather wanted to keep it simple.

Ah, OK.  But still, doubling the buffered case isn't insignificant.

Mike

Re: [PATCH 0/2] dm: add new loop and ram targets

[Bryn M. Reeves]

On Wed, Jan 17, 2018 at 04:29:36PM -0500, Mike Snitzer wrote:
> On Wed, Jan 17 2018 at  2:33pm -0500,
> As for dm-loop, doubling the performance of the loopback driver is quite
> nice (especially with only 1/7 the number of lines of code as
> drives/block/loop.c).

Isn't this going to raise the same objection that akpm had years ago,
with the original dm-loop (block mapping) target?

We had an even bigger performance boost with that but it was rejected
on the grounds that a second loop back block device implementation was
not welcome unless the two could share code.

We had out of tree users for years from folks looking for better
performance.

I've been through a few revisions of a patch set that refactors the
loop.c driver into an interface and back end, which both /dev/loopN and
dm-loop could use. This was at the request of various groups who were
interested in a DM loop target but ultimately interest waned each time
and I ended up stopping maintenance of it a few years back.

Regards,
Bryn.
 

Re: [PATCH 0/2] dm: add new loop and ram targets

[Mike Snitzer]

On Thu, Jan 18 2018 at  6:42am -0500,
Bryn M. Reeves <bmr@redhat.com> wrote:

> On Wed, Jan 17, 2018 at 04:29:36PM -0500, Mike Snitzer wrote:
> > On Wed, Jan 17 2018 at  2:33pm -0500,
> > As for dm-loop, doubling the performance of the loopback driver is quite
> > nice (especially with only 1/7 the number of lines of code as
> > drives/block/loop.c).
> 
> Isn't this going to raise the same objection that akpm had years ago,
> with the original dm-loop (block mapping) target?
> 
> We had an even bigger performance boost with that but it was rejected
> on the grounds that a second loop back block device implementation was
> not welcome unless the two could share code.

Could.  But I wasn't around for that particular spat.  It seems quite
misplaced to swoop in with an aire of design purity to defeat a DM
target that shows such clear wins.

This idea that our poor Linux users will lose their heads because they
have multiple options is also idiotic.

But we'll cross that bridge as needed (before burning it down?) ;)

> We had out of tree users for years from folks looking for better
> performance.
>
> I've been through a few revisions of a patch set that refactors the
> loop.c driver into an interface and back end, which both /dev/loopN and
> dm-loop could use. This was at the request of various groups who were
> interested in a DM loop target but ultimately interest waned each time
> and I ended up stopping maintenance of it a few years back.

It is make work.  Yes there is duplicated logic but loop is now
encumbered by blk-mq complexity now.  To think that we have to factor
out a front and back end to stand up a DM bio-based loop device is
_insane_.  Especially when you consider the front-end would need to
differentiate between bio-based vs blk-mq.

It could also easily be that we don't need or want feature compatibility
(which I assume is the intent of a common front-end?).

This isn't rocket science.  If we're willing to maintain ~400 line
dm-loop then nobody else should worry their pretty little heads about
it.  Problems arise when code isn't maintained.

Mike

Re: [PATCH 0/2] dm: add new loop and ram targets

[Mike Snitzer]

On Thu, Jan 18 2018 at  6:56am -0500,
Mike Snitzer <snitzer@redhat.com> wrote:

> On Thu, Jan 18 2018 at  6:42am -0500,
> Bryn M. Reeves <bmr@redhat.com> wrote:
> 
> > On Wed, Jan 17, 2018 at 04:29:36PM -0500, Mike Snitzer wrote:
> > > On Wed, Jan 17 2018 at  2:33pm -0500,
> > > As for dm-loop, doubling the performance of the loopback driver is quite
> > > nice (especially with only 1/7 the number of lines of code as
> > > drives/block/loop.c).
> > 
> > Isn't this going to raise the same objection that akpm had years ago,
> > with the original dm-loop (block mapping) target?
> > 
> > We had an even bigger performance boost with that but it was rejected
> > on the grounds that a second loop back block device implementation was
> > not welcome unless the two could share code.
> 
> Could.  But I wasn't around for that particular spat.  It seems quite
> misplaced to swoop in with an aire of design purity to defeat a DM
> target that shows such clear wins.
> 
> This idea that our poor Linux users will lose their heads because they
> have multiple options is also idiotic.
> 
> But we'll cross that bridge as needed (before burning it down?) ;)

Reflecting on getting spun up about the potential for a fight made me
feel like this guy wanting to use the phone in "Dumb and Dumber":
https://youtu.be/Gue2LvHibpg?t=25

Could easily be in the end someone like akpm or Jens will just
effectively punch me in the face from inside their phone booth ;)

Re: [dm-devel] [PATCH 0/2] dm: add new loop and ram targets

[Christoph Hellwig]

On Wed, Jan 17, 2018 at 08:33:59PM +0100, Heinz Mauelshagen wrote:
> Enhancing IO performance compared to the kernels existing
> loop driver thus better suiting respective requirements in
> test setups, this patch series adds new "loop" and "ram" targets.

For a fair comparism vs loop you'll need to compare it to the
direct I/O mode.  If it is faster than that we have a very serious
bug somewhere.

> 
> For measures see test results below.
> 
> 
> The "loop" target maps segments to backing files.
> Mapping table example:
> 0 4192256 loop /tmp/mapper_loop1
> 4192256 2097152 loop /dev/nvm/mapper_loop0
> 
> 
> The "ram" target accesses RAM directly rather than through
> tmpfs additionally enhancing performance compared to "loop"
> thus avoding filesystem overhead.
> Mapping table example:
> 0 8388608 ram
> 
> "ram" is a singleton target.
> 
> 
> Performance test results for 4K and 32K IOPS comparing the loop driver
> with dm-loop backed by tmpfs and dm-ram (all 2GiB backing size):
> 
> <TESTSCRIPT>
> #!/bin/sh
> for f in /tmp/loop0 /tmp/mapper_loop0
> do
> 	dd if=/dev/zero of=$f bs=256M count=8 iflag=fullblock
> done
> 
> losetup /dev/loop0 /tmp/loop0
> sectors=`du -s /tmp/mapper_loop0|cut -f1`
> dmsetup create loop0 --table "0 $sectors loop /tmp/mapper_loop0"
> dmsetup create ram --table "0 $sectors ram"
> 
> for bs in 4K 32K
> do
> 	for d in /dev/loop0 /dev/mapper/loop0 /dev/mapper/ram
> 	do
> 		echo 3 > /proc/sys/vm/drop_caches
> 		fio --bs=$bs --rw=randrw --numjobs=99 --group_reporting --iodepth=12 --runtime=3 --ioengine=libaio \
> 		    --loops=1 --direct=1 --exitall --name dc --filename=$d | egrep "read|write"
> 	done
> done
> </TESTSCRIPT>
> 
> <4K_RESULTS>
> loop driver:
>    read: IOPS=226k, BW=881MiB/s (924MB/s)(2645MiB/3003msec)
>   write: IOPS=225k, BW=880MiB/s (923MB/s)(2643MiB/3003msec)
> dm-loop target:
>    read: IOPS=425k, BW=1661MiB/s (1742MB/s)(4990MiB/3004msec)
>   write: IOPS=425k, BW=1662MiB/s (1743MB/s)(4992MiB/3004msec)
> dm-ram target:
>    read: IOPS=636k, BW=2484MiB/s (2605MB/s)(7464MiB/3005msec)
>   write: IOPS=636k, BW=2484MiB/s (2605MB/s)(7464MiB/3005msec)
> </4K_RESULTS>
> 
> <32K_RESULTS>
> loop driver:
>   read: IOPS=55.5k, BW=1733MiB/s (1817MB/s)(5215MiB/3009msec)
>   write: IOPS=55.2k, BW=1726MiB/s (1810MB/s)(5195MiB/3009msec)
> dm-loop target:
>    read: IOPS=110k, BW=3452MiB/s (3620MB/s)(10.1GiB/3006msec)
>   write: IOPS=110k, BW=3448MiB/s (3615MB/s)(10.1GiB/3006msec)
> dm-ram target:
>    read: IOPS=355k, BW=10.8GiB/s (11.6GB/s)(32.6GiB/3008msec)
>   write: IOPS=355k, BW=10.8GiB/s (11.6GB/s)(32.6GiB/3008msec)
> </32K_RESULTS>
> 
> 
> Signed-off-by: Heinz Mauelshagen <heinzm@redhat.com>
> 
> Heinz Mauelshagen (2):
>   dm loop: new target redirecting io to backing file(s)
>   dm ram: new target redirecting io to RAM
> 
>  Documentation/device-mapper/loop.txt |  20 ++
>  Documentation/device-mapper/ram.txt  |  15 ++
>  drivers/md/Kconfig                   |  14 ++
>  drivers/md/Makefile                  |   2 +
>  drivers/md/dm-loop.c                 | 352 +++++++++++++++++++++++++++++++++++
>  drivers/md/dm-ram.c                  | 101 ++++++++++
>  6 files changed, 504 insertions(+)
>  create mode 100644 Documentation/device-mapper/loop.txt
>  create mode 100644 Documentation/device-mapper/ram.txt
>  create mode 100644 drivers/md/dm-loop.c
>  create mode 100644 drivers/md/dm-ram.c
> 
> -- 
> 2.14.3
> 
> --
> dm-devel mailing list
> dm-devel@redhat.com
> https://www.redhat.com/mailman/listinfo/dm-devel
---end quoted text---

Re: [dm-devel] [PATCH 0/2] dm: add new loop and ram targets

[Heinz Mauelshagen]

On 22/01/18 21:19, Christoph Hellwig wrote:

> On Wed, Jan 17, 2018 at 08:33:59PM +0100, Heinz Mauelshagen wrote:
>> Enhancing IO performance compared to the kernels existing
>> loop driver thus better suiting respective requirements in
>> test setups, this patch series adds new "loop" and "ram" targets.
> For a fair comparism vs loop you'll need to compare it to the
> direct I/O mode.

True, the tested use cases were based  on backing filesystems
w/o direct I/O support which is valuable to me.

>    If it is faster than that we have a very serious
> bug somewhere.

As a quick test I opened the backing file O_DIRECT in dm-loop on XFS 
backed by
NVMe and it's 10-20% slower than loop because it's bio based and thus 
lags mq.

So, no worries, loop looks ok in regards to direct I/O mode.
>> For measures see test results below.
>>
>>
>> The "loop" target maps segments to backing files.
>> Mapping table example:
>> 0 4192256 loop /tmp/mapper_loop1
>> 4192256 2097152 loop /dev/nvm/mapper_loop0
>>
>>
>> The "ram" target accesses RAM directly rather than through
>> tmpfs additionally enhancing performance compared to "loop"
>> thus avoding filesystem overhead.
>> Mapping table example:
>> 0 8388608 ram
>>
>> "ram" is a singleton target.
>>
>>
>> Performance test results for 4K and 32K IOPS comparing the loop driver
>> with dm-loop backed by tmpfs and dm-ram (all 2GiB backing size):
>>
>> <TESTSCRIPT>
>> #!/bin/sh
>> for f in /tmp/loop0 /tmp/mapper_loop0
>> do
>> 	dd if=/dev/zero of=$f bs=256M count=8 iflag=fullblock
>> done
>>
>> losetup /dev/loop0 /tmp/loop0
>> sectors=`du -s /tmp/mapper_loop0|cut -f1`
>> dmsetup create loop0 --table "0 $sectors loop /tmp/mapper_loop0"
>> dmsetup create ram --table "0 $sectors ram"
>>
>> for bs in 4K 32K
>> do
>> 	for d in /dev/loop0 /dev/mapper/loop0 /dev/mapper/ram
>> 	do
>> 		echo 3 > /proc/sys/vm/drop_caches
>> 		fio --bs=$bs --rw=randrw --numjobs=99 --group_reporting --iodepth=12 --runtime=3 --ioengine=libaio \
>> 		    --loops=1 --direct=1 --exitall --name dc --filename=$d | egrep "read|write"
>> 	done
>> done
>> </TESTSCRIPT>
>>
>> <4K_RESULTS>
>> loop driver:
>>     read: IOPS=226k, BW=881MiB/s (924MB/s)(2645MiB/3003msec)
>>    write: IOPS=225k, BW=880MiB/s (923MB/s)(2643MiB/3003msec)
>> dm-loop target:
>>     read: IOPS=425k, BW=1661MiB/s (1742MB/s)(4990MiB/3004msec)
>>    write: IOPS=425k, BW=1662MiB/s (1743MB/s)(4992MiB/3004msec)
>> dm-ram target:
>>     read: IOPS=636k, BW=2484MiB/s (2605MB/s)(7464MiB/3005msec)
>>    write: IOPS=636k, BW=2484MiB/s (2605MB/s)(7464MiB/3005msec)
>> </4K_RESULTS>
>>
>> <32K_RESULTS>
>> loop driver:
>>    read: IOPS=55.5k, BW=1733MiB/s (1817MB/s)(5215MiB/3009msec)
>>    write: IOPS=55.2k, BW=1726MiB/s (1810MB/s)(5195MiB/3009msec)
>> dm-loop target:
>>     read: IOPS=110k, BW=3452MiB/s (3620MB/s)(10.1GiB/3006msec)
>>    write: IOPS=110k, BW=3448MiB/s (3615MB/s)(10.1GiB/3006msec)
>> dm-ram target:
>>     read: IOPS=355k, BW=10.8GiB/s (11.6GB/s)(32.6GiB/3008msec)
>>    write: IOPS=355k, BW=10.8GiB/s (11.6GB/s)(32.6GiB/3008msec)
>> </32K_RESULTS>
>>
>>
>> Signed-off-by: Heinz Mauelshagen <heinzm@redhat.com>
>>
>> Heinz Mauelshagen (2):
>>    dm loop: new target redirecting io to backing file(s)
>>    dm ram: new target redirecting io to RAM
>>
>>   Documentation/device-mapper/loop.txt |  20 ++
>>   Documentation/device-mapper/ram.txt  |  15 ++
>>   drivers/md/Kconfig                   |  14 ++
>>   drivers/md/Makefile                  |   2 +
>>   drivers/md/dm-loop.c                 | 352 +++++++++++++++++++++++++++++++++++
>>   drivers/md/dm-ram.c                  | 101 ++++++++++
>>   6 files changed, 504 insertions(+)
>>   create mode 100644 Documentation/device-mapper/loop.txt
>>   create mode 100644 Documentation/device-mapper/ram.txt
>>   create mode 100644 drivers/md/dm-loop.c
>>   create mode 100644 drivers/md/dm-ram.c
>>
>> -- 
>> 2.14.3
>>
>> --
>> dm-devel mailing list
>> dm-devel@redhat.com
>> https://www.redhat.com/mailman/listinfo/dm-devel
> ---end quoted text---
>
> --
> dm-devel mailing list
> dm-devel@redhat.com
> https://www.redhat.com/mailman/listinfo/dm-devel