From: Scott Bauer <scott.bauer@intel.com>
To: dm-devel@redhat.com
Cc: snitzer@redhat.com, agk@redhat.com, linux-kernel@vger.kernel.org,
keith.busch@intel.com, jonathan.derrick@intel.com,
Scott Bauer <scott.bauer@intel.com>
Subject: [PATCH v4 2/2] dm-unstripe: Add documentation for unstripe target
Date: Mon, 18 Dec 2017 10:28:09 -0700 [thread overview]
Message-ID: <20171218172809.5826-3-scott.bauer@intel.com> (raw)
In-Reply-To: <20171218172809.5826-1-scott.bauer@intel.com>
Signed-off-by: Scott Bauer <scott.bauer@intel.com>
---
Documentation/device-mapper/dm-unstripe.txt | 130 ++++++++++++++++++++++++++++
1 file changed, 130 insertions(+)
create mode 100644 Documentation/device-mapper/dm-unstripe.txt
diff --git a/Documentation/device-mapper/dm-unstripe.txt b/Documentation/device-mapper/dm-unstripe.txt
new file mode 100644
index 000000000000..9c13e9316d52
--- /dev/null
+++ b/Documentation/device-mapper/dm-unstripe.txt
@@ -0,0 +1,130 @@
+Device-Mapper Unstripe
+=====================
+
+The device-mapper unstripe (dm-unstripe) target provides a transparent
+mechanism to unstripe a device-mapper "striped" target to access the
+underlying disks without having to touch the true backing block-device.
+It can also be used to unstripe a hardware RAID-0 to access backing disks
+as well.
+
+
+Parameters:
+<drive (ex: /dev/nvme0n1)> <drive #> <# of drives> <chunk sectors>
+
+
+<drive>
+ The block device you wish to unstripe.
+
+<drive #>
+ The physical drive you wish to expose via this "virtual" device
+ mapper target. This must be 0 indexed.
+
+<# of drives>
+ The number of drives in the RAID 0.
+
+<chunk sectors>
+ The amount of 512B sectors in the chunk striping, or zero, if you
+ wish you use max_hw_sector_size.
+
+
+Why use this module?
+=====================
+
+ An example of undoing an existing dm-stripe:
+
+ This small bash script will setup 4 loop devices and use the existing
+ dm-stripe target to combine the 4 devices into one. It then will use
+ the unstripe target on the new combined stripe device to access the
+ individual backing loop devices. We write data to the newly exposed
+ unstriped devices and verify the data written matches the correct
+ underlying device on the striped array.
+
+ #!/bin/bash
+
+ MEMBER_SIZE=$((128 * 1024 * 1024))
+ NUM=4
+ SEQ_END=$((${NUM}-1))
+ CHUNK=256
+ BS=4096
+
+ RAID_SIZE=$((${MEMBER_SIZE}*${NUM}/512))
+ DM_PARMS="0 ${RAID_SIZE} striped ${NUM} ${CHUNK}"
+ COUNT=$((${MEMBER_SIZE} / ${BS}))
+
+ for i in $(seq 0 ${SEQ_END}); do
+ dd if=/dev/zero of=member-${i} bs=${MEMBER_SIZE} count=1 oflag=direct
+ losetup /dev/loop${i} member-${i}
+ DM_PARMS+=" /dev/loop${i} 0"
+ done
+
+ echo $DM_PARMS | dmsetup create raid0
+ for i in $(seq 0 ${SEQ_END}); do
+ echo "0 1 unstripe /dev/mapper/raid0 ${i} ${NUM} ${CHUNK}" | dmsetup create set-${i}
+ done;
+
+ for i in $(seq 0 ${SEQ_END}); do
+ dd if=/dev/urandom of=/dev/mapper/set-${i} bs=${BS} count=${COUNT} oflag=direct
+ diff /dev/mapper/set-${i} member-${i}
+ done;
+
+ for i in $(seq 0 ${SEQ_END}); do
+ dmsetup remove set-${i}
+ done
+
+ dmsetup remove raid0
+
+ for i in $(seq 0 ${SEQ_END}); do
+ losetup -d /dev/loop${i}
+ rm -f member-${i}
+ done
+
+==============
+
+
+ Another example:
+
+ Intel NVMe drives contain two cores on the physical device.
+ Each core of the drive has segregated access to its LBA range.
+ The current LBA model has a RAID 0 128k chunk on each core, resulting
+ in a 256k stripe across the two cores:
+
+ Core 0: Core 1:
+ __________ __________
+ | LBA 512| | LBA 768|
+ | LBA 0 | | LBA 256|
+ ---------- ----------
+
+ The purpose of this unstriping is to provide better QoS in noisy
+ neighbor environments. When two partitions are created on the
+ aggregate drive without this unstriping, reads on one partition
+ can affect writes on another partition. This is because the partitions
+ are striped across the two cores. When we unstripe this hardware RAID 0
+ and make partitions on each new exposed device the two partitions are now
+ physically separated.
+
+ With the module we were able to segregate a fio script that has read and
+ write jobs that are independent of each other. Compared to when we run
+ the test on a combined drive with partitions, we were able to get a 92%
+ reduction read latency using this device mapper target.
+
+
+====================
+Example scripts:
+
+
+dmsetup create nvmset1 --table '0 1 unstripe /dev/nvme0n1 1 2 0'
+dmsetup create nvmset0 --table '0 1 unstripe /dev/nvme0n1 0 2 0'
+
+There will now be two mappers:
+/dev/mapper/nvmset1
+/dev/mapper/nvmset0
+
+that will expose core 0 and core 1.
+
+
+# In a dm-stripe with 4 drives of chunk size 128K:
+dmsetup create raid_disk0 --table '0 1 unstripe /dev/mapper/striped 0 4 256'
+dmsetup create raid_disk1 --table '0 1 unstripe /dev/mapper/striped 1 4 256'
+dmsetup create raid_disk2 --table '0 1 unstripe /dev/mapper/striped 2 4 256'
+dmsetup create raid_disk3 --table '0 1 unstripe /dev/mapper/striped 3 4 256'
+
--
2.11.0
prev parent reply other threads:[~2017-12-18 17:51 UTC|newest]
Thread overview: 10+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-12-18 17:28 [PATCH v4 0/2] dm-unstripe Scott Bauer
2017-12-18 17:28 ` [PATCH v4 1/2] dm-unstripe: unstripe RAID 0/dm-striped device Scott Bauer
2017-12-18 19:11 ` Randy Dunlap
2017-12-18 18:52 ` Scott Bauer
2017-12-18 19:32 ` Mike Snitzer
2017-12-18 19:37 ` Mike Snitzer
2017-12-20 10:01 ` kbuild test robot
2017-12-20 10:21 ` kbuild test robot
2017-12-20 16:09 ` Mike Snitzer
2017-12-18 17:28 ` Scott Bauer [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20171218172809.5826-3-scott.bauer@intel.com \
--to=scott.bauer@intel.com \
--cc=agk@redhat.com \
--cc=dm-devel@redhat.com \
--cc=jonathan.derrick@intel.com \
--cc=keith.busch@intel.com \
--cc=linux-kernel@vger.kernel.org \
--cc=snitzer@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox