* [PATCH v5 1/9] fstests: doc: add new raid-stripe-tree group
2023-12-07 9:03 [PATCH v5 0/9] fstests: add tests for btrfs' raid-stripe-tree feature Johannes Thumshirn
@ 2023-12-07 9:03 ` Johannes Thumshirn
2023-12-07 9:03 ` [PATCH v5 2/9] common: add filter for btrfs raid-stripe dump Johannes Thumshirn
` (8 subsequent siblings)
9 siblings, 0 replies; 15+ messages in thread
From: Johannes Thumshirn @ 2023-12-07 9:03 UTC (permalink / raw)
To: Anand Jain, Zorro Lang
Cc: Filipe Manana, fstests, linux-btrfs, Johannes Thumshirn
Add a new test group for testing the raid-stripe-tree feature of btrfs
with fstests.
Signed-off-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
---
doc/group-names.txt | 1 +
1 file changed, 1 insertion(+)
diff --git a/doc/group-names.txt b/doc/group-names.txt
index c3dcca375537..9c1624868518 100644
--- a/doc/group-names.txt
+++ b/doc/group-names.txt
@@ -94,6 +94,7 @@ punch fallocate FALLOC_FL_PUNCH_HOLE
qgroup btrfs qgroup feature
quota filesystem usage quotas
raid btrfs RAID
+raid-stripe-tree btrfs raid-stripe-tree feature
read_repair btrfs error correction on read failure
realtime XFS realtime volumes
recoveryloop crash recovery loops
--
2.43.0
^ permalink raw reply related [flat|nested] 15+ messages in thread* [PATCH v5 2/9] common: add filter for btrfs raid-stripe dump
2023-12-07 9:03 [PATCH v5 0/9] fstests: add tests for btrfs' raid-stripe-tree feature Johannes Thumshirn
2023-12-07 9:03 ` [PATCH v5 1/9] fstests: doc: add new raid-stripe-tree group Johannes Thumshirn
@ 2023-12-07 9:03 ` Johannes Thumshirn
2023-12-07 9:03 ` [PATCH v5 3/9] common: add _require_btrfs_no_nodatacow helper Johannes Thumshirn
` (7 subsequent siblings)
9 siblings, 0 replies; 15+ messages in thread
From: Johannes Thumshirn @ 2023-12-07 9:03 UTC (permalink / raw)
To: Anand Jain, Zorro Lang
Cc: Filipe Manana, fstests, linux-btrfs, Johannes Thumshirn
Signed-off-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
---
common/filter.btrfs | 14 ++++++++++++++
1 file changed, 14 insertions(+)
diff --git a/common/filter.btrfs b/common/filter.btrfs
index 02c6b92dfa94..2003ba7b7015 100644
--- a/common/filter.btrfs
+++ b/common/filter.btrfs
@@ -125,5 +125,19 @@ _filter_btrfs_cloner_error()
sed -e "s/\(clone failed:\) Operation not supported/\1 Invalid argument/g"
}
+# filter output of "btrfs inspect-internal dump-tree -t raid-stripe"
+_filter_stripe_tree()
+{
+ sed -E -e "s/leaf [0-9]+ items [0-9]+ free space [0-9]+ generation [0-9]+ owner RAID_STRIPE_TREE/leaf XXXXXXXXX items X free space XXXXX generation X owner RAID_STRIPE_TREE/" \
+ -e "s/leaf [0-9]+ flags 0x1\(WRITTEN\) backref revision 1/leaf XXXXXXXXX flags 0x1\(WRITTEN\) backref revision 1/" \
+ -e "s/checksum stored [0-9a-f]+/checksum stored <CHECKSUM>/" \
+ -e "s/checksum calced [0-9a-f]+/checksum calced <CHECKSUM>/" \
+ -e "s/[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}/<UUID>/" \
+ -e "s/item ([0-9]+) key \([0-9]+ RAID_STRIPE ([0-9]+)\) itemoff [0-9]+ itemsize ([0-9]+)/item \1 key \(XXXXXX RAID_STRIPE \2\) itemoff XXXXX itemsize \3/" \
+ -e "s/stripe ([0-9]+) devid ([0-9]+) physical [0-9]+/stripe \1 devid \2 physical XXXXXXXXX/" \
+ -e "s/total bytes [0-9]+/total bytes XXXXXXXX/" \
+ -e "s/bytes used [0-9]+/bytes used XXXXXX/"
+}
+
# make sure this script returns success
/bin/true
--
2.43.0
^ permalink raw reply related [flat|nested] 15+ messages in thread* [PATCH v5 3/9] common: add _require_btrfs_no_nodatacow helper
2023-12-07 9:03 [PATCH v5 0/9] fstests: add tests for btrfs' raid-stripe-tree feature Johannes Thumshirn
2023-12-07 9:03 ` [PATCH v5 1/9] fstests: doc: add new raid-stripe-tree group Johannes Thumshirn
2023-12-07 9:03 ` [PATCH v5 2/9] common: add filter for btrfs raid-stripe dump Johannes Thumshirn
@ 2023-12-07 9:03 ` Johannes Thumshirn
2023-12-07 9:03 ` [PATCH v5 4/9] common: add _require_btrfs_free_space_tree Johannes Thumshirn
` (6 subsequent siblings)
9 siblings, 0 replies; 15+ messages in thread
From: Johannes Thumshirn @ 2023-12-07 9:03 UTC (permalink / raw)
To: Anand Jain, Zorro Lang
Cc: Filipe Manana, fstests, linux-btrfs, Johannes Thumshirn
Signed-off-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
---
common/btrfs | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/common/btrfs b/common/btrfs
index e3ccb4fa07d4..bf69bcee158b 100644
--- a/common/btrfs
+++ b/common/btrfs
@@ -120,6 +120,13 @@ _require_btrfs_no_compress()
fi
}
+_require_btrfs_no_nodatacow()
+{
+ if _normalize_mount_options "$MOUNT_OPTIONS" | grep -q "nodatacow"; then
+ _notrun "This test requires no nodatacow enabled"
+ fi
+}
+
_require_btrfs_no_block_group_tree()
{
_scratch_mkfs > /dev/null 2>&1
--
2.43.0
^ permalink raw reply related [flat|nested] 15+ messages in thread* [PATCH v5 4/9] common: add _require_btrfs_free_space_tree
2023-12-07 9:03 [PATCH v5 0/9] fstests: add tests for btrfs' raid-stripe-tree feature Johannes Thumshirn
` (2 preceding siblings ...)
2023-12-07 9:03 ` [PATCH v5 3/9] common: add _require_btrfs_no_nodatacow helper Johannes Thumshirn
@ 2023-12-07 9:03 ` Johannes Thumshirn
2023-12-07 9:03 ` [PATCH v5 5/9] btrfs: add fstest for stripe-tree metadata with 4k write Johannes Thumshirn
` (5 subsequent siblings)
9 siblings, 0 replies; 15+ messages in thread
From: Johannes Thumshirn @ 2023-12-07 9:03 UTC (permalink / raw)
To: Anand Jain, Zorro Lang
Cc: Filipe Manana, fstests, linux-btrfs, Johannes Thumshirn
Signed-off-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
---
common/btrfs | 10 ++++++++++
1 file changed, 10 insertions(+)
diff --git a/common/btrfs b/common/btrfs
index bf69bcee158b..95e46726a801 100644
--- a/common/btrfs
+++ b/common/btrfs
@@ -137,6 +137,16 @@ _require_btrfs_no_block_group_tree()
fi
}
+_require_btrfs_free_space_tree()
+{
+ _scratch_mkfs > /dev/null 2>&1
+ if ! $BTRFS_UTIL_PROG inspect-internal dump-super $SCRATCH_DEV | \
+ grep -q "FREE_SPACE_TREE"
+ then
+ _notrun "This test requires a free-space-tree"
+ fi
+}
+
_check_btrfs_filesystem()
{
device=$1
--
2.43.0
^ permalink raw reply related [flat|nested] 15+ messages in thread* [PATCH v5 5/9] btrfs: add fstest for stripe-tree metadata with 4k write
2023-12-07 9:03 [PATCH v5 0/9] fstests: add tests for btrfs' raid-stripe-tree feature Johannes Thumshirn
` (3 preceding siblings ...)
2023-12-07 9:03 ` [PATCH v5 4/9] common: add _require_btrfs_free_space_tree Johannes Thumshirn
@ 2023-12-07 9:03 ` Johannes Thumshirn
2023-12-07 9:03 ` [PATCH v5 6/9] btrfs: add fstest for 8k write spanning two stripes on raid-stripe-tree Johannes Thumshirn
` (4 subsequent siblings)
9 siblings, 0 replies; 15+ messages in thread
From: Johannes Thumshirn @ 2023-12-07 9:03 UTC (permalink / raw)
To: Anand Jain, Zorro Lang
Cc: Filipe Manana, fstests, linux-btrfs, Johannes Thumshirn
Test a simple 4k write on all RAID profiles currently supported with the
raid-stripe-tree.
Signed-off-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
---
tests/btrfs/304 | 56 +++++++++++++++++++++++++++++++++++++++++++++++++++
tests/btrfs/304.out | 58 +++++++++++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 114 insertions(+)
diff --git a/tests/btrfs/304 b/tests/btrfs/304
new file mode 100755
index 000000000000..7774a1bd74d0
--- /dev/null
+++ b/tests/btrfs/304
@@ -0,0 +1,56 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2023 Western Digital Cooperation. All Rights Reserved.
+#
+# FS QA Test 304
+#
+# Test on-disk layout of RAID Stripe Tree Metadata writing 4k to a new file on
+# a pristine file system.
+#
+. ./common/preamble
+_begin_fstest auto quick raid remount volume raid-stripe-tree
+
+. ./common/filter
+. ./common/filter.btrfs
+
+_supported_fs btrfs
+_require_btrfs_command inspect-internal dump-tree
+_require_btrfs_mkfs_feature "raid-stripe-tree"
+_require_scratch_dev_pool 4
+_require_btrfs_fs_feature "raid_stripe_tree"
+_require_btrfs_fs_feature "free_space_tree"
+_require_btrfs_free_space_tree
+_require_btrfs_no_compress
+
+test_4k_write()
+{
+ local profile=$1
+ local ndevs=$2
+
+ _scratch_dev_pool_get $ndevs
+
+ echo "==== Testing $profile ===="
+ _scratch_pool_mkfs -d $profile -m $profile -O raid-stripe-tree
+ _scratch_mount
+
+ $XFS_IO_PROG -fc "pwrite 0 4k" "$SCRATCH_MNT/foo" | _filter_xfs_io
+
+ _scratch_cycle_mount
+ md5sum "$SCRATCH_MNT/foo" | _filter_scratch
+
+ _scratch_unmount
+
+ $BTRFS_UTIL_PROG inspect-internal dump-tree -t raid_stripe $SCRATCH_DEV_POOL |\
+ _filter_btrfs_version | _filter_stripe_tree
+
+ _scratch_dev_pool_put
+}
+
+echo "= Test basic 4k write ="
+test_4k_write raid0 2
+test_4k_write raid1 2
+test_4k_write raid10 4
+
+# success, all done
+status=0
+exit
diff --git a/tests/btrfs/304.out b/tests/btrfs/304.out
new file mode 100644
index 000000000000..48036efbf0cf
--- /dev/null
+++ b/tests/btrfs/304.out
@@ -0,0 +1,58 @@
+QA output created by 304
+= Test basic 4k write =
+==== Testing raid0 ====
+wrote 4096/4096 bytes at offset 0
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+5fed275e7617a806f94c173746a2a723 SCRATCH_MNT/foo
+
+raid stripe tree key (RAID_STRIPE_TREE ROOT_ITEM 0)
+leaf XXXXXXXXX items X free space XXXXX generation X owner RAID_STRIPE_TREE
+leaf XXXXXXXXX flags 0x1(WRITTEN) backref revision 1
+checksum stored <CHECKSUM>
+checksum calced <CHECKSUM>
+fs uuid <UUID>
+chunk uuid <UUID>
+ item 0 key (XXXXXX RAID_STRIPE 4096) itemoff XXXXX itemsize 24
+ encoding: RAID0
+ stripe 0 devid 1 physical XXXXXXXXX
+total bytes XXXXXXXX
+bytes used XXXXXX
+uuid <UUID>
+==== Testing raid1 ====
+wrote 4096/4096 bytes at offset 0
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+5fed275e7617a806f94c173746a2a723 SCRATCH_MNT/foo
+
+raid stripe tree key (RAID_STRIPE_TREE ROOT_ITEM 0)
+leaf XXXXXXXXX items X free space XXXXX generation X owner RAID_STRIPE_TREE
+leaf XXXXXXXXX flags 0x1(WRITTEN) backref revision 1
+checksum stored <CHECKSUM>
+checksum calced <CHECKSUM>
+fs uuid <UUID>
+chunk uuid <UUID>
+ item 0 key (XXXXXX RAID_STRIPE 4096) itemoff XXXXX itemsize 40
+ encoding: RAID1
+ stripe 0 devid 1 physical XXXXXXXXX
+ stripe 1 devid 2 physical XXXXXXXXX
+total bytes XXXXXXXX
+bytes used XXXXXX
+uuid <UUID>
+==== Testing raid10 ====
+wrote 4096/4096 bytes at offset 0
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+5fed275e7617a806f94c173746a2a723 SCRATCH_MNT/foo
+
+raid stripe tree key (RAID_STRIPE_TREE ROOT_ITEM 0)
+leaf XXXXXXXXX items X free space XXXXX generation X owner RAID_STRIPE_TREE
+leaf XXXXXXXXX flags 0x1(WRITTEN) backref revision 1
+checksum stored <CHECKSUM>
+checksum calced <CHECKSUM>
+fs uuid <UUID>
+chunk uuid <UUID>
+ item 0 key (XXXXXX RAID_STRIPE 4096) itemoff XXXXX itemsize 40
+ encoding: RAID10
+ stripe 0 devid 1 physical XXXXXXXXX
+ stripe 1 devid 2 physical XXXXXXXXX
+total bytes XXXXXXXX
+bytes used XXXXXX
+uuid <UUID>
--
2.43.0
^ permalink raw reply related [flat|nested] 15+ messages in thread* [PATCH v5 6/9] btrfs: add fstest for 8k write spanning two stripes on raid-stripe-tree
2023-12-07 9:03 [PATCH v5 0/9] fstests: add tests for btrfs' raid-stripe-tree feature Johannes Thumshirn
` (4 preceding siblings ...)
2023-12-07 9:03 ` [PATCH v5 5/9] btrfs: add fstest for stripe-tree metadata with 4k write Johannes Thumshirn
@ 2023-12-07 9:03 ` Johannes Thumshirn
2023-12-07 9:03 ` [PATCH v5 7/9] btrfs: add fstest for writing to a file at an offset with RST Johannes Thumshirn
` (3 subsequent siblings)
9 siblings, 0 replies; 15+ messages in thread
From: Johannes Thumshirn @ 2023-12-07 9:03 UTC (permalink / raw)
To: Anand Jain, Zorro Lang
Cc: Filipe Manana, fstests, linux-btrfs, Johannes Thumshirn
Add a test-case writing 8k to a raid-stripe-tree formatted filesystem with
one stripe pre-filled to 60k so the 8k are split into a 4k write finishing
stripe 1 and a 4k write starting the next stripe.
Signed-off-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
---
tests/btrfs/305 | 61 +++++++++++++++++++++++++++++++++++++++
tests/btrfs/305.out | 82 +++++++++++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 143 insertions(+)
diff --git a/tests/btrfs/305 b/tests/btrfs/305
new file mode 100755
index 000000000000..df8a12ab9a6a
--- /dev/null
+++ b/tests/btrfs/305
@@ -0,0 +1,61 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2023 Western Digital Cooperation. All Rights Reserved.
+#
+# FS QA Test 305
+#
+# Test on-disk layout of RAID Stripe Tree Metadata by writing 8k to a new file
+# with a filesystem prepropulated, so that 4k of the write are written to the
+# 1st stripe and 4k start a new stripe.
+#
+. ./common/preamble
+_begin_fstest auto quick raid remount volume raid-stripe-tree
+
+. ./common/filter
+. ./common/filter.btrfs
+
+_supported_fs btrfs
+_require_btrfs_command inspect-internal dump-tree
+_require_btrfs_mkfs_feature "raid-stripe-tree"
+_require_scratch_dev_pool 4
+_require_btrfs_fs_feature "raid_stripe_tree"
+_require_btrfs_fs_feature "free_space_tree"
+_require_btrfs_free_space_tree
+_require_btrfs_no_compress
+
+test_8k_new_stripe()
+{
+ local profile=$1
+ local ndevs=$2
+
+ _scratch_dev_pool_get $ndevs
+
+ echo "==== Testing $profile ===="
+ _scratch_pool_mkfs -d $profile -m $profile -O raid-stripe-tree
+ _scratch_mount
+
+ # Fill the first stripe up to 64k - 4k
+ $XFS_IO_PROG -fc "pwrite 0 60k" -c fsync "$SCRATCH_MNT/bar" | _filter_xfs_io
+
+ # The actual 8k write
+ $XFS_IO_PROG -fc "pwrite 0 8k" "$SCRATCH_MNT/foo" | _filter_xfs_io
+
+ _scratch_cycle_mount
+ md5sum "$SCRATCH_MNT/foo" | _filter_scratch
+
+ _scratch_unmount
+
+ $BTRFS_UTIL_PROG inspect-internal dump-tree -t raid_stripe $SCRATCH_DEV_POOL |\
+ _filter_btrfs_version | _filter_stripe_tree
+
+ _scratch_dev_pool_put
+}
+
+echo "= Test 8k write to a new file so that 4k start a new stripe ="
+test_8k_new_stripe raid0 2
+test_8k_new_stripe raid1 2
+test_8k_new_stripe raid10 4
+
+# success, all done
+status=0
+exit
diff --git a/tests/btrfs/305.out b/tests/btrfs/305.out
new file mode 100644
index 000000000000..7460501ef40b
--- /dev/null
+++ b/tests/btrfs/305.out
@@ -0,0 +1,82 @@
+QA output created by 305
+= Test 8k write to a new file so that 4k start a new stripe =
+==== Testing raid0 ====
+wrote 61440/61440 bytes at offset 0
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+wrote 8192/8192 bytes at offset 0
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+9d3940adb41dd525e008a847e01b15f4 SCRATCH_MNT/foo
+
+raid stripe tree key (RAID_STRIPE_TREE ROOT_ITEM 0)
+leaf XXXXXXXXX items X free space XXXXX generation X owner RAID_STRIPE_TREE
+leaf XXXXXXXXX flags 0x1(WRITTEN) backref revision 1
+checksum stored <CHECKSUM>
+checksum calced <CHECKSUM>
+fs uuid <UUID>
+chunk uuid <UUID>
+ item 0 key (XXXXXX RAID_STRIPE 61440) itemoff XXXXX itemsize 24
+ encoding: RAID0
+ stripe 0 devid 1 physical XXXXXXXXX
+ item 1 key (XXXXXX RAID_STRIPE 4096) itemoff XXXXX itemsize 24
+ encoding: RAID0
+ stripe 0 devid 1 physical XXXXXXXXX
+ item 2 key (XXXXXX RAID_STRIPE 4096) itemoff XXXXX itemsize 24
+ encoding: RAID0
+ stripe 0 devid 2 physical XXXXXXXXX
+total bytes XXXXXXXX
+bytes used XXXXXX
+uuid <UUID>
+==== Testing raid1 ====
+wrote 61440/61440 bytes at offset 0
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+wrote 8192/8192 bytes at offset 0
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+9d3940adb41dd525e008a847e01b15f4 SCRATCH_MNT/foo
+
+raid stripe tree key (RAID_STRIPE_TREE ROOT_ITEM 0)
+leaf XXXXXXXXX items X free space XXXXX generation X owner RAID_STRIPE_TREE
+leaf XXXXXXXXX flags 0x1(WRITTEN) backref revision 1
+checksum stored <CHECKSUM>
+checksum calced <CHECKSUM>
+fs uuid <UUID>
+chunk uuid <UUID>
+ item 0 key (XXXXXX RAID_STRIPE 61440) itemoff XXXXX itemsize 40
+ encoding: RAID1
+ stripe 0 devid 1 physical XXXXXXXXX
+ stripe 1 devid 2 physical XXXXXXXXX
+ item 1 key (XXXXXX RAID_STRIPE 8192) itemoff XXXXX itemsize 40
+ encoding: RAID1
+ stripe 0 devid 1 physical XXXXXXXXX
+ stripe 1 devid 2 physical XXXXXXXXX
+total bytes XXXXXXXX
+bytes used XXXXXX
+uuid <UUID>
+==== Testing raid10 ====
+wrote 61440/61440 bytes at offset 0
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+wrote 8192/8192 bytes at offset 0
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+9d3940adb41dd525e008a847e01b15f4 SCRATCH_MNT/foo
+
+raid stripe tree key (RAID_STRIPE_TREE ROOT_ITEM 0)
+leaf XXXXXXXXX items X free space XXXXX generation X owner RAID_STRIPE_TREE
+leaf XXXXXXXXX flags 0x1(WRITTEN) backref revision 1
+checksum stored <CHECKSUM>
+checksum calced <CHECKSUM>
+fs uuid <UUID>
+chunk uuid <UUID>
+ item 0 key (XXXXXX RAID_STRIPE 61440) itemoff XXXXX itemsize 40
+ encoding: RAID10
+ stripe 0 devid 1 physical XXXXXXXXX
+ stripe 1 devid 2 physical XXXXXXXXX
+ item 1 key (XXXXXX RAID_STRIPE 4096) itemoff XXXXX itemsize 40
+ encoding: RAID10
+ stripe 0 devid 1 physical XXXXXXXXX
+ stripe 1 devid 2 physical XXXXXXXXX
+ item 2 key (XXXXXX RAID_STRIPE 4096) itemoff XXXXX itemsize 40
+ encoding: RAID10
+ stripe 0 devid 3 physical XXXXXXXXX
+ stripe 1 devid 4 physical XXXXXXXXX
+total bytes XXXXXXXX
+bytes used XXXXXX
+uuid <UUID>
--
2.43.0
^ permalink raw reply related [flat|nested] 15+ messages in thread* [PATCH v5 7/9] btrfs: add fstest for writing to a file at an offset with RST
2023-12-07 9:03 [PATCH v5 0/9] fstests: add tests for btrfs' raid-stripe-tree feature Johannes Thumshirn
` (5 preceding siblings ...)
2023-12-07 9:03 ` [PATCH v5 6/9] btrfs: add fstest for 8k write spanning two stripes on raid-stripe-tree Johannes Thumshirn
@ 2023-12-07 9:03 ` Johannes Thumshirn
2023-12-07 9:03 ` [PATCH v5 8/9] btrfs: add fstests to write 128k to a RST filesystem Johannes Thumshirn
` (2 subsequent siblings)
9 siblings, 0 replies; 15+ messages in thread
From: Johannes Thumshirn @ 2023-12-07 9:03 UTC (permalink / raw)
To: Anand Jain, Zorro Lang
Cc: Filipe Manana, fstests, linux-btrfs, Johannes Thumshirn
Add a fstest writing 4k at offset 64k to a file with one RAID tripe
already pre-filled for a raid-stripe-tree formatted file system.
Signed-off-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
---
tests/btrfs/306 | 59 +++++++++++++++++++++++++++++++++++++++++
tests/btrfs/306.out | 75 +++++++++++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 134 insertions(+)
diff --git a/tests/btrfs/306 b/tests/btrfs/306
new file mode 100755
index 000000000000..72c066aa6940
--- /dev/null
+++ b/tests/btrfs/306
@@ -0,0 +1,59 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2023 Western Digital Cooperation. All Rights Reserved.
+#
+# FS QA Test 306
+#
+# Test on-disk layout of RAID Stripe Tree Metadata by writing 4k to an emppty
+# file at offset 64k with one stripe pre-filled on an otherwise pristine
+# filesystem.
+#
+. ./common/preamble
+_begin_fstest auto quick raid remount volume raid-stripe-tree
+
+. ./common/filter
+. ./common/filter.btrfs
+
+_supported_fs btrfs
+_require_btrfs_command inspect-internal dump-tree
+_require_btrfs_mkfs_feature "raid-stripe-tree"
+_require_scratch_dev_pool 4
+_require_btrfs_fs_feature "raid_stripe_tree"
+_require_btrfs_fs_feature "free_space_tree"
+_require_btrfs_free_space_tree
+_require_btrfs_no_compress
+
+test_4k_write_64koff()
+{
+ local profile=$1
+ local ndevs=$2
+
+ _scratch_dev_pool_get $ndevs
+
+ echo "==== Testing $profile ===="
+ _scratch_pool_mkfs -d $profile -m $profile -O raid-stripe-tree
+ _scratch_mount
+
+ # precondition one stripe
+ $XFS_IO_PROG -fc "pwrite 0 64k" "$SCRATCH_MNT/bar" | _filter_xfs_io
+
+ $XFS_IO_PROG -fc "pwrite 64k 4k" "$SCRATCH_MNT/foo" | _filter_xfs_io
+
+ _scratch_cycle_mount
+ md5sum "$SCRATCH_MNT/foo" | _filter_scratch
+
+ _scratch_unmount
+
+ $BTRFS_UTIL_PROG inspect-internal dump-tree -t raid_stripe $SCRATCH_DEV_POOL |\
+ _filter_btrfs_version | _filter_stripe_tree
+
+ _scratch_dev_pool_put
+}
+echo "= Test 4k write to an empty file at offset 64k with one stripe prefilled ="
+test_4k_write_64koff raid0 2
+test_4k_write_64koff raid1 2
+test_4k_write_64koff raid10 4
+
+# success, all done
+status=0
+exit
diff --git a/tests/btrfs/306.out b/tests/btrfs/306.out
new file mode 100644
index 000000000000..8031cddb79f7
--- /dev/null
+++ b/tests/btrfs/306.out
@@ -0,0 +1,75 @@
+QA output created by 306
+= Test 4k write to an empty file at offset 64k with one stripe prefilled =
+==== Testing raid0 ====
+wrote 65536/65536 bytes at offset 0
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+wrote 4096/4096 bytes at offset 65536
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+381b0e7d72cb4f75286fe2b445e8d92a SCRATCH_MNT/foo
+
+raid stripe tree key (RAID_STRIPE_TREE ROOT_ITEM 0)
+leaf XXXXXXXXX items X free space XXXXX generation X owner RAID_STRIPE_TREE
+leaf XXXXXXXXX flags 0x1(WRITTEN) backref revision 1
+checksum stored <CHECKSUM>
+checksum calced <CHECKSUM>
+fs uuid <UUID>
+chunk uuid <UUID>
+ item 0 key (XXXXXX RAID_STRIPE 65536) itemoff XXXXX itemsize 24
+ encoding: RAID0
+ stripe 0 devid 1 physical XXXXXXXXX
+ item 1 key (XXXXXX RAID_STRIPE 4096) itemoff XXXXX itemsize 24
+ encoding: RAID0
+ stripe 0 devid 2 physical XXXXXXXXX
+total bytes XXXXXXXX
+bytes used XXXXXX
+uuid <UUID>
+==== Testing raid1 ====
+wrote 65536/65536 bytes at offset 0
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+wrote 4096/4096 bytes at offset 65536
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+381b0e7d72cb4f75286fe2b445e8d92a SCRATCH_MNT/foo
+
+raid stripe tree key (RAID_STRIPE_TREE ROOT_ITEM 0)
+leaf XXXXXXXXX items X free space XXXXX generation X owner RAID_STRIPE_TREE
+leaf XXXXXXXXX flags 0x1(WRITTEN) backref revision 1
+checksum stored <CHECKSUM>
+checksum calced <CHECKSUM>
+fs uuid <UUID>
+chunk uuid <UUID>
+ item 0 key (XXXXXX RAID_STRIPE 65536) itemoff XXXXX itemsize 40
+ encoding: RAID1
+ stripe 0 devid 1 physical XXXXXXXXX
+ stripe 1 devid 2 physical XXXXXXXXX
+ item 1 key (XXXXXX RAID_STRIPE 4096) itemoff XXXXX itemsize 40
+ encoding: RAID1
+ stripe 0 devid 1 physical XXXXXXXXX
+ stripe 1 devid 2 physical XXXXXXXXX
+total bytes XXXXXXXX
+bytes used XXXXXX
+uuid <UUID>
+==== Testing raid10 ====
+wrote 65536/65536 bytes at offset 0
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+wrote 4096/4096 bytes at offset 65536
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+381b0e7d72cb4f75286fe2b445e8d92a SCRATCH_MNT/foo
+
+raid stripe tree key (RAID_STRIPE_TREE ROOT_ITEM 0)
+leaf XXXXXXXXX items X free space XXXXX generation X owner RAID_STRIPE_TREE
+leaf XXXXXXXXX flags 0x1(WRITTEN) backref revision 1
+checksum stored <CHECKSUM>
+checksum calced <CHECKSUM>
+fs uuid <UUID>
+chunk uuid <UUID>
+ item 0 key (XXXXXX RAID_STRIPE 65536) itemoff XXXXX itemsize 40
+ encoding: RAID10
+ stripe 0 devid 1 physical XXXXXXXXX
+ stripe 1 devid 2 physical XXXXXXXXX
+ item 1 key (XXXXXX RAID_STRIPE 4096) itemoff XXXXX itemsize 40
+ encoding: RAID10
+ stripe 0 devid 3 physical XXXXXXXXX
+ stripe 1 devid 4 physical XXXXXXXXX
+total bytes XXXXXXXX
+bytes used XXXXXX
+uuid <UUID>
--
2.43.0
^ permalink raw reply related [flat|nested] 15+ messages in thread* [PATCH v5 8/9] btrfs: add fstests to write 128k to a RST filesystem
2023-12-07 9:03 [PATCH v5 0/9] fstests: add tests for btrfs' raid-stripe-tree feature Johannes Thumshirn
` (6 preceding siblings ...)
2023-12-07 9:03 ` [PATCH v5 7/9] btrfs: add fstest for writing to a file at an offset with RST Johannes Thumshirn
@ 2023-12-07 9:03 ` Johannes Thumshirn
2023-12-07 9:03 ` [PATCH v5 9/9] btrfs: add fstest for overwriting a file partially with RST Johannes Thumshirn
2023-12-07 12:11 ` [PATCH v5 0/9] fstests: add tests for btrfs' raid-stripe-tree feature Filipe Manana
9 siblings, 0 replies; 15+ messages in thread
From: Johannes Thumshirn @ 2023-12-07 9:03 UTC (permalink / raw)
To: Anand Jain, Zorro Lang
Cc: Filipe Manana, fstests, linux-btrfs, Johannes Thumshirn
Add a test writing 128k to a file on an empty filesystem formatted with a
raid-stripe-tree.
Signed-off-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
---
tests/btrfs/307 | 56 +++++++++++++++++++++++++++++++++++++++++++++
tests/btrfs/307.out | 65 +++++++++++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 121 insertions(+)
diff --git a/tests/btrfs/307 b/tests/btrfs/307
new file mode 100755
index 000000000000..9509019b50cc
--- /dev/null
+++ b/tests/btrfs/307
@@ -0,0 +1,56 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2023 Western Digital Cooperation. All Rights Reserved.
+#
+# FS QA Test 307
+#
+# Test on-disk layout of RAID Stripe Tree Metadata by writing 128k to a new
+# file on a pristine filesystem
+#
+. ./common/preamble
+_begin_fstest auto quick raid remount volume raid-stripe-tree
+
+. ./common/filter
+. ./common/filter.btrfs
+
+_supported_fs btrfs
+_require_btrfs_command inspect-internal dump-tree
+_require_btrfs_mkfs_feature "raid-stripe-tree"
+_require_scratch_dev_pool 4
+_require_btrfs_fs_feature "raid_stripe_tree"
+_require_btrfs_fs_feature "free_space_tree"
+_require_btrfs_free_space_tree
+_require_btrfs_no_compress
+
+test_128k_write()
+{
+ local profile=$1
+ local ndevs=$2
+
+ _scratch_dev_pool_get $ndevs
+
+ echo "==== Testing $profile ===="
+ _scratch_pool_mkfs -d $profile -m $profile -O raid-stripe-tree
+ _scratch_mount
+
+ $XFS_IO_PROG -fc "pwrite 0 128k" "$SCRATCH_MNT/foo" | _filter_xfs_io
+
+ _scratch_cycle_mount
+ md5sum "$SCRATCH_MNT/foo" | _filter_scratch
+
+ _scratch_unmount
+
+ $BTRFS_UTIL_PROG inspect-internal dump-tree -t raid_stripe $SCRATCH_DEV_POOL |\
+ _filter_btrfs_version | _filter_stripe_tree
+
+ _scratch_dev_pool_put
+}
+
+echo "= Test 128k write to empty file ="
+test_128k_write raid0 2
+test_128k_write raid1 2
+test_128k_write raid10 4
+
+# success, all done
+status=0
+exit
diff --git a/tests/btrfs/307.out b/tests/btrfs/307.out
new file mode 100644
index 000000000000..b39f3162bff6
--- /dev/null
+++ b/tests/btrfs/307.out
@@ -0,0 +1,65 @@
+QA output created by 307
+= Test 128k write to empty file =
+==== Testing raid0 ====
+wrote 131072/131072 bytes at offset 0
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+d48858312a922db7eb86377f638dbc9f SCRATCH_MNT/foo
+
+raid stripe tree key (RAID_STRIPE_TREE ROOT_ITEM 0)
+leaf XXXXXXXXX items X free space XXXXX generation X owner RAID_STRIPE_TREE
+leaf XXXXXXXXX flags 0x1(WRITTEN) backref revision 1
+checksum stored <CHECKSUM>
+checksum calced <CHECKSUM>
+fs uuid <UUID>
+chunk uuid <UUID>
+ item 0 key (XXXXXX RAID_STRIPE 65536) itemoff XXXXX itemsize 24
+ encoding: RAID0
+ stripe 0 devid 1 physical XXXXXXXXX
+ item 1 key (XXXXXX RAID_STRIPE 65536) itemoff XXXXX itemsize 24
+ encoding: RAID0
+ stripe 0 devid 2 physical XXXXXXXXX
+total bytes XXXXXXXX
+bytes used XXXXXX
+uuid <UUID>
+==== Testing raid1 ====
+wrote 131072/131072 bytes at offset 0
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+d48858312a922db7eb86377f638dbc9f SCRATCH_MNT/foo
+
+raid stripe tree key (RAID_STRIPE_TREE ROOT_ITEM 0)
+leaf XXXXXXXXX items X free space XXXXX generation X owner RAID_STRIPE_TREE
+leaf XXXXXXXXX flags 0x1(WRITTEN) backref revision 1
+checksum stored <CHECKSUM>
+checksum calced <CHECKSUM>
+fs uuid <UUID>
+chunk uuid <UUID>
+ item 0 key (XXXXXX RAID_STRIPE 131072) itemoff XXXXX itemsize 40
+ encoding: RAID1
+ stripe 0 devid 1 physical XXXXXXXXX
+ stripe 1 devid 2 physical XXXXXXXXX
+total bytes XXXXXXXX
+bytes used XXXXXX
+uuid <UUID>
+==== Testing raid10 ====
+wrote 131072/131072 bytes at offset 0
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+d48858312a922db7eb86377f638dbc9f SCRATCH_MNT/foo
+
+raid stripe tree key (RAID_STRIPE_TREE ROOT_ITEM 0)
+leaf XXXXXXXXX items X free space XXXXX generation X owner RAID_STRIPE_TREE
+leaf XXXXXXXXX flags 0x1(WRITTEN) backref revision 1
+checksum stored <CHECKSUM>
+checksum calced <CHECKSUM>
+fs uuid <UUID>
+chunk uuid <UUID>
+ item 0 key (XXXXXX RAID_STRIPE 65536) itemoff XXXXX itemsize 40
+ encoding: RAID10
+ stripe 0 devid 1 physical XXXXXXXXX
+ stripe 1 devid 2 physical XXXXXXXXX
+ item 1 key (XXXXXX RAID_STRIPE 65536) itemoff XXXXX itemsize 40
+ encoding: RAID10
+ stripe 0 devid 3 physical XXXXXXXXX
+ stripe 1 devid 4 physical XXXXXXXXX
+total bytes XXXXXXXX
+bytes used XXXXXX
+uuid <UUID>
--
2.43.0
^ permalink raw reply related [flat|nested] 15+ messages in thread* [PATCH v5 9/9] btrfs: add fstest for overwriting a file partially with RST
2023-12-07 9:03 [PATCH v5 0/9] fstests: add tests for btrfs' raid-stripe-tree feature Johannes Thumshirn
` (7 preceding siblings ...)
2023-12-07 9:03 ` [PATCH v5 8/9] btrfs: add fstests to write 128k to a RST filesystem Johannes Thumshirn
@ 2023-12-07 9:03 ` Johannes Thumshirn
2023-12-07 12:11 ` [PATCH v5 0/9] fstests: add tests for btrfs' raid-stripe-tree feature Filipe Manana
9 siblings, 0 replies; 15+ messages in thread
From: Johannes Thumshirn @ 2023-12-07 9:03 UTC (permalink / raw)
To: Anand Jain, Zorro Lang
Cc: Filipe Manana, fstests, linux-btrfs, Johannes Thumshirn
Add a test writing 128k to an empty file with one stripe already
pre-filled on-disk. Then overwrite a portion of the file in the middle.
Signed-off-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
---
tests/btrfs/308 | 60 +++++++++++++++++++++++++++++
tests/btrfs/308.out | 106 ++++++++++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 166 insertions(+)
diff --git a/tests/btrfs/308 b/tests/btrfs/308
new file mode 100755
index 000000000000..eb63acf6a9f2
--- /dev/null
+++ b/tests/btrfs/308
@@ -0,0 +1,60 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2023 Western Digital Cooperation. All Rights Reserved.
+#
+# FS QA Test 308
+#
+# Test on-disk layout of RAID Stripe Tree Metadata by writing 128k to an empty
+# file on a filesystem that has one stripe already pre-filled. Afterwards
+# overwrite a portion of the file.
+#
+. ./common/preamble
+_begin_fstest auto quick raid remount volume raid-stripe-tree
+
+. ./common/filter
+. ./common/filter.btrfs
+
+_supported_fs btrfs
+_require_btrfs_command inspect-internal dump-tree
+_require_btrfs_mkfs_feature "raid-stripe-tree"
+_require_scratch_dev_pool 4
+_require_btrfs_fs_feature "raid_stripe_tree"
+_require_btrfs_fs_feature "free_space_tree"
+_require_btrfs_free_space_tree
+_require_btrfs_no_compress
+_require_btrfs_no_nodatacow
+
+test_128k_write_overwrite()
+{
+ local profile=$1
+ local ndevs=$2
+
+ _scratch_dev_pool_get $ndevs
+
+ echo "==== Testing $profile ===="
+ _scratch_pool_mkfs -d $profile -m $profile -O raid-stripe-tree
+ _scratch_mount
+
+ $XFS_IO_PROG -fc "pwrite -W 0 32k" "$SCRATCH_MNT/bar" | _filter_xfs_io
+ $XFS_IO_PROG -fc "pwrite -W 0 128k" "$SCRATCH_MNT/foo" | _filter_xfs_io
+ $XFS_IO_PROG -fc "pwrite -W 64k 8k" "$SCRATCH_MNT/foo" | _filter_xfs_io
+
+ _scratch_cycle_mount
+ md5sum "$SCRATCH_MNT/foo" | _filter_scratch
+
+ _scratch_unmount
+
+ $BTRFS_UTIL_PROG inspect-internal dump-tree -t raid_stripe $SCRATCH_DEV_POOL |\
+ _filter_btrfs_version | _filter_stripe_tree
+
+ _scratch_dev_pool_put
+}
+
+echo "= Test 128k write to empty file with 1st stripe partially prefilled then overwrite ="
+test_128k_write_overwrite raid0 2
+test_128k_write_overwrite raid1 2
+test_128k_write_overwrite raid10 4
+
+# success, all done
+status=0
+exit
diff --git a/tests/btrfs/308.out b/tests/btrfs/308.out
new file mode 100644
index 000000000000..c93a3a43b248
--- /dev/null
+++ b/tests/btrfs/308.out
@@ -0,0 +1,106 @@
+QA output created by 308
+= Test 128k write to empty file with 1st stripe partially prefilled then overwrite =
+==== Testing raid0 ====
+wrote 32768/32768 bytes at offset 0
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+wrote 131072/131072 bytes at offset 0
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+wrote 8192/8192 bytes at offset 65536
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+d48858312a922db7eb86377f638dbc9f SCRATCH_MNT/foo
+
+raid stripe tree key (RAID_STRIPE_TREE ROOT_ITEM 0)
+leaf XXXXXXXXX items X free space XXXXX generation X owner RAID_STRIPE_TREE
+leaf XXXXXXXXX flags 0x1(WRITTEN) backref revision 1
+checksum stored <CHECKSUM>
+checksum calced <CHECKSUM>
+fs uuid <UUID>
+chunk uuid <UUID>
+ item 0 key (XXXXXX RAID_STRIPE 32768) itemoff XXXXX itemsize 24
+ encoding: RAID0
+ stripe 0 devid 1 physical XXXXXXXXX
+ item 1 key (XXXXXX RAID_STRIPE 32768) itemoff XXXXX itemsize 24
+ encoding: RAID0
+ stripe 0 devid 1 physical XXXXXXXXX
+ item 2 key (XXXXXX RAID_STRIPE 65536) itemoff XXXXX itemsize 24
+ encoding: RAID0
+ stripe 0 devid 2 physical XXXXXXXXX
+ item 3 key (XXXXXX RAID_STRIPE 32768) itemoff XXXXX itemsize 24
+ encoding: RAID0
+ stripe 0 devid 1 physical XXXXXXXXX
+ item 4 key (XXXXXX RAID_STRIPE 8192) itemoff XXXXX itemsize 24
+ encoding: RAID0
+ stripe 0 devid 1 physical XXXXXXXXX
+total bytes XXXXXXXX
+bytes used XXXXXX
+uuid <UUID>
+==== Testing raid1 ====
+wrote 32768/32768 bytes at offset 0
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+wrote 131072/131072 bytes at offset 0
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+wrote 8192/8192 bytes at offset 65536
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+d48858312a922db7eb86377f638dbc9f SCRATCH_MNT/foo
+
+raid stripe tree key (RAID_STRIPE_TREE ROOT_ITEM 0)
+leaf XXXXXXXXX items X free space XXXXX generation X owner RAID_STRIPE_TREE
+leaf XXXXXXXXX flags 0x1(WRITTEN) backref revision 1
+checksum stored <CHECKSUM>
+checksum calced <CHECKSUM>
+fs uuid <UUID>
+chunk uuid <UUID>
+ item 0 key (XXXXXX RAID_STRIPE 32768) itemoff XXXXX itemsize 40
+ encoding: RAID1
+ stripe 0 devid 1 physical XXXXXXXXX
+ stripe 1 devid 2 physical XXXXXXXXX
+ item 1 key (XXXXXX RAID_STRIPE 131072) itemoff XXXXX itemsize 40
+ encoding: RAID1
+ stripe 0 devid 1 physical XXXXXXXXX
+ stripe 1 devid 2 physical XXXXXXXXX
+ item 2 key (XXXXXX RAID_STRIPE 8192) itemoff XXXXX itemsize 40
+ encoding: RAID1
+ stripe 0 devid 1 physical XXXXXXXXX
+ stripe 1 devid 2 physical XXXXXXXXX
+total bytes XXXXXXXX
+bytes used XXXXXX
+uuid <UUID>
+==== Testing raid10 ====
+wrote 32768/32768 bytes at offset 0
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+wrote 131072/131072 bytes at offset 0
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+wrote 8192/8192 bytes at offset 65536
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+d48858312a922db7eb86377f638dbc9f SCRATCH_MNT/foo
+
+raid stripe tree key (RAID_STRIPE_TREE ROOT_ITEM 0)
+leaf XXXXXXXXX items X free space XXXXX generation X owner RAID_STRIPE_TREE
+leaf XXXXXXXXX flags 0x1(WRITTEN) backref revision 1
+checksum stored <CHECKSUM>
+checksum calced <CHECKSUM>
+fs uuid <UUID>
+chunk uuid <UUID>
+ item 0 key (XXXXXX RAID_STRIPE 32768) itemoff XXXXX itemsize 40
+ encoding: RAID10
+ stripe 0 devid 1 physical XXXXXXXXX
+ stripe 1 devid 2 physical XXXXXXXXX
+ item 1 key (XXXXXX RAID_STRIPE 32768) itemoff XXXXX itemsize 40
+ encoding: RAID10
+ stripe 0 devid 1 physical XXXXXXXXX
+ stripe 1 devid 2 physical XXXXXXXXX
+ item 2 key (XXXXXX RAID_STRIPE 65536) itemoff XXXXX itemsize 40
+ encoding: RAID10
+ stripe 0 devid 3 physical XXXXXXXXX
+ stripe 1 devid 4 physical XXXXXXXXX
+ item 3 key (XXXXXX RAID_STRIPE 32768) itemoff XXXXX itemsize 40
+ encoding: RAID10
+ stripe 0 devid 1 physical XXXXXXXXX
+ stripe 1 devid 2 physical XXXXXXXXX
+ item 4 key (XXXXXX RAID_STRIPE 8192) itemoff XXXXX itemsize 40
+ encoding: RAID10
+ stripe 0 devid 1 physical XXXXXXXXX
+ stripe 1 devid 2 physical XXXXXXXXX
+total bytes XXXXXXXX
+bytes used XXXXXX
+uuid <UUID>
--
2.43.0
^ permalink raw reply related [flat|nested] 15+ messages in thread* Re: [PATCH v5 0/9] fstests: add tests for btrfs' raid-stripe-tree feature
2023-12-07 9:03 [PATCH v5 0/9] fstests: add tests for btrfs' raid-stripe-tree feature Johannes Thumshirn
` (8 preceding siblings ...)
2023-12-07 9:03 ` [PATCH v5 9/9] btrfs: add fstest for overwriting a file partially with RST Johannes Thumshirn
@ 2023-12-07 12:11 ` Filipe Manana
2023-12-08 1:19 ` Anand Jain
9 siblings, 1 reply; 15+ messages in thread
From: Filipe Manana @ 2023-12-07 12:11 UTC (permalink / raw)
To: Johannes Thumshirn
Cc: Anand Jain, Zorro Lang, Filipe Manana, fstests, linux-btrfs
On Thu, Dec 7, 2023 at 9:03 AM Johannes Thumshirn
<johannes.thumshirn@wdc.com> wrote:
>
> Add tests for btrfs' raid-stripe-tree feature. All of these test work by
> writing a specific pattern to a newly created filesystem and afterwards
> using `btrfs inspect-internal -t raid-stripe $SCRATCH_DEV_POOL` to verify
> the placement and the layout of the metadata.
>
> The md5sum of each file will be compared as well after a re-mount of the
> filesystem.
>
> ---
> Changes in v5:
> - add _require_btrfs_free_space_tree helper and use in tests
> - Link to v4: https://lore.kernel.org/r/20231206-btrfs-raid-v4-0-578284dd3a70@wdc.com
>
> Changes in v4:
> - add _require_btrfs_no_compress to all tests
> - add _require_btrfs_no_nodatacow helper and add to btrfs/308
> - add _require_btrfs_feature "free_space_tree" to all tests
> - Link to v3: https://lore.kernel.org/r/20231205-btrfs-raid-v3-0-0e857a5439a2@wdc.com
>
> Changes in v3:
> - added 'raid-stripe-tree' to mkfs options, as only zoned raid gets it
> automatically
> - Rename test cases as btrfs/302 and btrfs/303 already exist upstream
> - Link to v2: https://lore.kernel.org/r/20231205-btrfs-raid-v2-0-25f80eea345b@wdc.com
>
> Changes in v2:
> - Re-ordered series so the newly introduced group is added before the
> tests
> - Changes Filipe requested to the tests.
> - Link to v1: https://lore.kernel.org/r/20231204-btrfs-raid-v1-0-b254eb1bcff8@wdc.com
>
> ---
> Johannes Thumshirn (9):
> fstests: doc: add new raid-stripe-tree group
> common: add filter for btrfs raid-stripe dump
> common: add _require_btrfs_no_nodatacow helper
> common: add _require_btrfs_free_space_tree
> btrfs: add fstest for stripe-tree metadata with 4k write
> btrfs: add fstest for 8k write spanning two stripes on raid-stripe-tree
> btrfs: add fstest for writing to a file at an offset with RST
> btrfs: add fstests to write 128k to a RST filesystem
> btrfs: add fstest for overwriting a file partially with RST
>
> common/btrfs | 17 +++++++++
> common/filter.btrfs | 14 +++++++
> doc/group-names.txt | 1 +
> tests/btrfs/304 | 56 +++++++++++++++++++++++++++
> tests/btrfs/304.out | 58 ++++++++++++++++++++++++++++
> tests/btrfs/305 | 61 ++++++++++++++++++++++++++++++
> tests/btrfs/305.out | 82 ++++++++++++++++++++++++++++++++++++++++
> tests/btrfs/306 | 59 +++++++++++++++++++++++++++++
> tests/btrfs/306.out | 75 +++++++++++++++++++++++++++++++++++++
> tests/btrfs/307 | 56 +++++++++++++++++++++++++++
> tests/btrfs/307.out | 65 ++++++++++++++++++++++++++++++++
> tests/btrfs/308 | 60 +++++++++++++++++++++++++++++
> tests/btrfs/308.out | 106 ++++++++++++++++++++++++++++++++++++++++++++++++++++
> 13 files changed, 710 insertions(+)
> ---
> base-commit: baca8a2b5cb6e798ce3a07e79a081031370c6cb8
Btw this base commit does not exist in the official fstests repo.
That commit is from the staging branch at https://github.com/kdave/xfstests
A "git am" will fail because the official fstests repo doesn't have
_require_btrfs_no_block_group_tree() at common/btrfs,
so it needs to be manually adjusted when applying the 3rd patch.
I tried the tests and they look good, so:
Reviewed-by: Filipe Manana <fdmanana@suse.com>
One question I missed before. Test 304 for example does a 4K write and
expects in the golden output to get a 4K raid stripe item.
What happens on a machine with 64K page size? There the default sector
size is 64K, will the write result in a 64K raid stripe item or will
it be 4K? In the former case, it will make the test fail.
Thanks.
> change-id: 20231204-btrfs-raid-75975797f97d
>
> Best regards,
> --
> Johannes Thumshirn <johannes.thumshirn@wdc.com>
>
>
^ permalink raw reply [flat|nested] 15+ messages in thread* Re: [PATCH v5 0/9] fstests: add tests for btrfs' raid-stripe-tree feature
2023-12-07 12:11 ` [PATCH v5 0/9] fstests: add tests for btrfs' raid-stripe-tree feature Filipe Manana
@ 2023-12-08 1:19 ` Anand Jain
2023-12-08 9:19 ` Johannes Thumshirn
0 siblings, 1 reply; 15+ messages in thread
From: Anand Jain @ 2023-12-08 1:19 UTC (permalink / raw)
To: Filipe Manana, Johannes Thumshirn
Cc: Zorro Lang, Filipe Manana, fstests, linux-btrfs
On 12/7/23 17:41, Filipe Manana wrote:
> On Thu, Dec 7, 2023 at 9:03 AM Johannes Thumshirn
> <johannes.thumshirn@wdc.com> wrote:
>>
>> Add tests for btrfs' raid-stripe-tree feature. All of these test work by
>> writing a specific pattern to a newly created filesystem and afterwards
>> using `btrfs inspect-internal -t raid-stripe $SCRATCH_DEV_POOL` to verify
>> the placement and the layout of the metadata.
>>
>> The md5sum of each file will be compared as well after a re-mount of the
>> filesystem.
>>
>> ---
>> Changes in v5:
>> - add _require_btrfs_free_space_tree helper and use in tests
>> - Link to v4: https://lore.kernel.org/r/20231206-btrfs-raid-v4-0-578284dd3a70@wdc.com
>>
>> Changes in v4:
>> - add _require_btrfs_no_compress to all tests
>> - add _require_btrfs_no_nodatacow helper and add to btrfs/308
>> - add _require_btrfs_feature "free_space_tree" to all tests
>> - Link to v3: https://lore.kernel.org/r/20231205-btrfs-raid-v3-0-0e857a5439a2@wdc.com
>>
>> Changes in v3:
>> - added 'raid-stripe-tree' to mkfs options, as only zoned raid gets it
>> automatically
>> - Rename test cases as btrfs/302 and btrfs/303 already exist upstream
>> - Link to v2: https://lore.kernel.org/r/20231205-btrfs-raid-v2-0-25f80eea345b@wdc.com
>>
>> Changes in v2:
>> - Re-ordered series so the newly introduced group is added before the
>> tests
>> - Changes Filipe requested to the tests.
>> - Link to v1: https://lore.kernel.org/r/20231204-btrfs-raid-v1-0-b254eb1bcff8@wdc.com
>>
>> ---
>> Johannes Thumshirn (9):
>> fstests: doc: add new raid-stripe-tree group
>> common: add filter for btrfs raid-stripe dump
>> common: add _require_btrfs_no_nodatacow helper
>> common: add _require_btrfs_free_space_tree
>> btrfs: add fstest for stripe-tree metadata with 4k write
>> btrfs: add fstest for 8k write spanning two stripes on raid-stripe-tree
>> btrfs: add fstest for writing to a file at an offset with RST
>> btrfs: add fstests to write 128k to a RST filesystem
>> btrfs: add fstest for overwriting a file partially with RST
>>
>> common/btrfs | 17 +++++++++
>> common/filter.btrfs | 14 +++++++
>> doc/group-names.txt | 1 +
>> tests/btrfs/304 | 56 +++++++++++++++++++++++++++
>> tests/btrfs/304.out | 58 ++++++++++++++++++++++++++++
>> tests/btrfs/305 | 61 ++++++++++++++++++++++++++++++
>> tests/btrfs/305.out | 82 ++++++++++++++++++++++++++++++++++++++++
>> tests/btrfs/306 | 59 +++++++++++++++++++++++++++++
>> tests/btrfs/306.out | 75 +++++++++++++++++++++++++++++++++++++
>> tests/btrfs/307 | 56 +++++++++++++++++++++++++++
>> tests/btrfs/307.out | 65 ++++++++++++++++++++++++++++++++
>> tests/btrfs/308 | 60 +++++++++++++++++++++++++++++
>> tests/btrfs/308.out | 106 ++++++++++++++++++++++++++++++++++++++++++++++++++++
>> 13 files changed, 710 insertions(+)
>> ---
>> base-commit: baca8a2b5cb6e798ce3a07e79a081031370c6cb8
>
> Btw this base commit does not exist in the official fstests repo.
> That commit is from the staging branch at https://github.com/kdave/xfstests
>
> A "git am" will fail because the official fstests repo doesn't have
> _require_btrfs_no_block_group_tree() at common/btrfs,
> so it needs to be manually adjusted when applying the 3rd patch.
>
> I tried the tests and they look good, so:
>
> Reviewed-by: Filipe Manana <fdmanana@suse.com>
>
> One question I missed before. Test 304 for example does a 4K write and
> expects in the golden output to get a 4K raid stripe item.
> What happens on a machine with 64K page size? There the default sector
> size is 64K, will the write result in a 64K raid stripe item or will
> it be 4K? In the former case, it will make the test fail.
>
Testing on a 64K pagesize. Will run it. Apologies for intermittent
responses; OOO until December 21.
Thanks Anand
> Thanks.
>
>
>> change-id: 20231204-btrfs-raid-75975797f97d
>>
>> Best regards,
>> --
>> Johannes Thumshirn <johannes.thumshirn@wdc.com>
>>
>>
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v5 0/9] fstests: add tests for btrfs' raid-stripe-tree feature
2023-12-08 1:19 ` Anand Jain
@ 2023-12-08 9:19 ` Johannes Thumshirn
2023-12-09 19:15 ` Neal Gompa
0 siblings, 1 reply; 15+ messages in thread
From: Johannes Thumshirn @ 2023-12-08 9:19 UTC (permalink / raw)
To: Anand Jain, Filipe Manana
Cc: Zorro Lang, Filipe Manana, fstests@vger.kernel.org,
linux-btrfs@vger.kernel.org
On 08.12.23 02:19, Anand Jain wrote:
>
>
> On 12/7/23 17:41, Filipe Manana wrote:
>> On Thu, Dec 7, 2023 at 9:03 AM Johannes Thumshirn
>> <johannes.thumshirn@wdc.com> wrote:
>>>
>>> Add tests for btrfs' raid-stripe-tree feature. All of these test work by
>>> writing a specific pattern to a newly created filesystem and afterwards
>>> using `btrfs inspect-internal -t raid-stripe $SCRATCH_DEV_POOL` to verify
>>> the placement and the layout of the metadata.
>>>
>>> The md5sum of each file will be compared as well after a re-mount of the
>>> filesystem.
>>>
>>> ---
>>> Changes in v5:
>>> - add _require_btrfs_free_space_tree helper and use in tests
>>> - Link to v4: https://lore.kernel.org/r/20231206-btrfs-raid-v4-0-578284dd3a70@wdc.com
>>>
>>> Changes in v4:
>>> - add _require_btrfs_no_compress to all tests
>>> - add _require_btrfs_no_nodatacow helper and add to btrfs/308
>>> - add _require_btrfs_feature "free_space_tree" to all tests
>>> - Link to v3: https://lore.kernel.org/r/20231205-btrfs-raid-v3-0-0e857a5439a2@wdc.com
>>>
>>> Changes in v3:
>>> - added 'raid-stripe-tree' to mkfs options, as only zoned raid gets it
>>> automatically
>>> - Rename test cases as btrfs/302 and btrfs/303 already exist upstream
>>> - Link to v2: https://lore.kernel.org/r/20231205-btrfs-raid-v2-0-25f80eea345b@wdc.com
>>>
>>> Changes in v2:
>>> - Re-ordered series so the newly introduced group is added before the
>>> tests
>>> - Changes Filipe requested to the tests.
>>> - Link to v1: https://lore.kernel.org/r/20231204-btrfs-raid-v1-0-b254eb1bcff8@wdc.com
>>>
>>> ---
>>> Johannes Thumshirn (9):
>>> fstests: doc: add new raid-stripe-tree group
>>> common: add filter for btrfs raid-stripe dump
>>> common: add _require_btrfs_no_nodatacow helper
>>> common: add _require_btrfs_free_space_tree
>>> btrfs: add fstest for stripe-tree metadata with 4k write
>>> btrfs: add fstest for 8k write spanning two stripes on raid-stripe-tree
>>> btrfs: add fstest for writing to a file at an offset with RST
>>> btrfs: add fstests to write 128k to a RST filesystem
>>> btrfs: add fstest for overwriting a file partially with RST
>>>
>>> common/btrfs | 17 +++++++++
>>> common/filter.btrfs | 14 +++++++
>>> doc/group-names.txt | 1 +
>>> tests/btrfs/304 | 56 +++++++++++++++++++++++++++
>>> tests/btrfs/304.out | 58 ++++++++++++++++++++++++++++
>>> tests/btrfs/305 | 61 ++++++++++++++++++++++++++++++
>>> tests/btrfs/305.out | 82 ++++++++++++++++++++++++++++++++++++++++
>>> tests/btrfs/306 | 59 +++++++++++++++++++++++++++++
>>> tests/btrfs/306.out | 75 +++++++++++++++++++++++++++++++++++++
>>> tests/btrfs/307 | 56 +++++++++++++++++++++++++++
>>> tests/btrfs/307.out | 65 ++++++++++++++++++++++++++++++++
>>> tests/btrfs/308 | 60 +++++++++++++++++++++++++++++
>>> tests/btrfs/308.out | 106 ++++++++++++++++++++++++++++++++++++++++++++++++++++
>>> 13 files changed, 710 insertions(+)
>>> ---
>>> base-commit: baca8a2b5cb6e798ce3a07e79a081031370c6cb8
>>
>> Btw this base commit does not exist in the official fstests repo.
>> That commit is from the staging branch at https://github.com/kdave/xfstests
>>
>> A "git am" will fail because the official fstests repo doesn't have
>> _require_btrfs_no_block_group_tree() at common/btrfs,
>> so it needs to be manually adjusted when applying the 3rd patch.
>>
>> I tried the tests and they look good, so:
>>
>> Reviewed-by: Filipe Manana <fdmanana@suse.com>
>>
>> One question I missed before. Test 304 for example does a 4K write and
>> expects in the golden output to get a 4K raid stripe item.
>> What happens on a machine with 64K page size? There the default sector
>> size is 64K, will the write result in a 64K raid stripe item or will
>> it be 4K? In the former case, it will make the test fail.
>>
>
> Testing on a 64K pagesize. Will run it. Apologies for intermittent
> responses; OOO until December 21.
Thanks Anand!
I don't have a 64k page size system to test, but I _think_ Filipe is
right, that will fail. I think we should skip these tests on non 4k sectors.
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v5 0/9] fstests: add tests for btrfs' raid-stripe-tree feature
2023-12-08 9:19 ` Johannes Thumshirn
@ 2023-12-09 19:15 ` Neal Gompa
2023-12-11 8:49 ` Johannes Thumshirn
0 siblings, 1 reply; 15+ messages in thread
From: Neal Gompa @ 2023-12-09 19:15 UTC (permalink / raw)
To: Johannes Thumshirn
Cc: Anand Jain, Filipe Manana, Zorro Lang, Filipe Manana,
fstests@vger.kernel.org, linux-btrfs@vger.kernel.org
On Fri, Dec 8, 2023 at 4:19 AM Johannes Thumshirn
<Johannes.Thumshirn@wdc.com> wrote:
>
> On 08.12.23 02:19, Anand Jain wrote:
> >
> >
> > On 12/7/23 17:41, Filipe Manana wrote:
> >> On Thu, Dec 7, 2023 at 9:03 AM Johannes Thumshirn
> >> <johannes.thumshirn@wdc.com> wrote:
> >>>
> >>> Add tests for btrfs' raid-stripe-tree feature. All of these test work by
> >>> writing a specific pattern to a newly created filesystem and afterwards
> >>> using `btrfs inspect-internal -t raid-stripe $SCRATCH_DEV_POOL` to verify
> >>> the placement and the layout of the metadata.
> >>>
> >>> The md5sum of each file will be compared as well after a re-mount of the
> >>> filesystem.
> >>>
> >>> ---
> >>> Changes in v5:
> >>> - add _require_btrfs_free_space_tree helper and use in tests
> >>> - Link to v4: https://lore.kernel.org/r/20231206-btrfs-raid-v4-0-578284dd3a70@wdc.com
> >>>
> >>> Changes in v4:
> >>> - add _require_btrfs_no_compress to all tests
> >>> - add _require_btrfs_no_nodatacow helper and add to btrfs/308
> >>> - add _require_btrfs_feature "free_space_tree" to all tests
> >>> - Link to v3: https://lore.kernel.org/r/20231205-btrfs-raid-v3-0-0e857a5439a2@wdc.com
> >>>
> >>> Changes in v3:
> >>> - added 'raid-stripe-tree' to mkfs options, as only zoned raid gets it
> >>> automatically
> >>> - Rename test cases as btrfs/302 and btrfs/303 already exist upstream
> >>> - Link to v2: https://lore.kernel.org/r/20231205-btrfs-raid-v2-0-25f80eea345b@wdc.com
> >>>
> >>> Changes in v2:
> >>> - Re-ordered series so the newly introduced group is added before the
> >>> tests
> >>> - Changes Filipe requested to the tests.
> >>> - Link to v1: https://lore.kernel.org/r/20231204-btrfs-raid-v1-0-b254eb1bcff8@wdc.com
> >>>
> >>> ---
> >>> Johannes Thumshirn (9):
> >>> fstests: doc: add new raid-stripe-tree group
> >>> common: add filter for btrfs raid-stripe dump
> >>> common: add _require_btrfs_no_nodatacow helper
> >>> common: add _require_btrfs_free_space_tree
> >>> btrfs: add fstest for stripe-tree metadata with 4k write
> >>> btrfs: add fstest for 8k write spanning two stripes on raid-stripe-tree
> >>> btrfs: add fstest for writing to a file at an offset with RST
> >>> btrfs: add fstests to write 128k to a RST filesystem
> >>> btrfs: add fstest for overwriting a file partially with RST
> >>>
> >>> common/btrfs | 17 +++++++++
> >>> common/filter.btrfs | 14 +++++++
> >>> doc/group-names.txt | 1 +
> >>> tests/btrfs/304 | 56 +++++++++++++++++++++++++++
> >>> tests/btrfs/304.out | 58 ++++++++++++++++++++++++++++
> >>> tests/btrfs/305 | 61 ++++++++++++++++++++++++++++++
> >>> tests/btrfs/305.out | 82 ++++++++++++++++++++++++++++++++++++++++
> >>> tests/btrfs/306 | 59 +++++++++++++++++++++++++++++
> >>> tests/btrfs/306.out | 75 +++++++++++++++++++++++++++++++++++++
> >>> tests/btrfs/307 | 56 +++++++++++++++++++++++++++
> >>> tests/btrfs/307.out | 65 ++++++++++++++++++++++++++++++++
> >>> tests/btrfs/308 | 60 +++++++++++++++++++++++++++++
> >>> tests/btrfs/308.out | 106 ++++++++++++++++++++++++++++++++++++++++++++++++++++
> >>> 13 files changed, 710 insertions(+)
> >>> ---
> >>> base-commit: baca8a2b5cb6e798ce3a07e79a081031370c6cb8
> >>
> >> Btw this base commit does not exist in the official fstests repo.
> >> That commit is from the staging branch at https://github.com/kdave/xfstests
> >>
> >> A "git am" will fail because the official fstests repo doesn't have
> >> _require_btrfs_no_block_group_tree() at common/btrfs,
> >> so it needs to be manually adjusted when applying the 3rd patch.
> >>
> >> I tried the tests and they look good, so:
> >>
> >> Reviewed-by: Filipe Manana <fdmanana@suse.com>
> >>
> >> One question I missed before. Test 304 for example does a 4K write and
> >> expects in the golden output to get a 4K raid stripe item.
> >> What happens on a machine with 64K page size? There the default sector
> >> size is 64K, will the write result in a 64K raid stripe item or will
> >> it be 4K? In the former case, it will make the test fail.
> >>
> >
> > Testing on a 64K pagesize. Will run it. Apologies for intermittent
> > responses; OOO until December 21.
>
> Thanks Anand!
>
> I don't have a 64k page size system to test, but I _think_ Filipe is
> right, that will fail. I think we should skip these tests on non 4k sectors.
>
Once we land the patch to default to 4k sector size[1] regardless of
page size, this should all work across all architectures, no?
[1]: https://lore.kernel.org/linux-btrfs/20231116160235.2708131-2-neal@gompa.dev/
--
真実はいつも一つ！/ Always, there's only one truth!
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v5 0/9] fstests: add tests for btrfs' raid-stripe-tree feature
2023-12-09 19:15 ` Neal Gompa
@ 2023-12-11 8:49 ` Johannes Thumshirn
0 siblings, 0 replies; 15+ messages in thread
From: Johannes Thumshirn @ 2023-12-11 8:49 UTC (permalink / raw)
To: Neal Gompa
Cc: Anand Jain, Filipe Manana, Zorro Lang, Filipe Manana,
fstests@vger.kernel.org, linux-btrfs@vger.kernel.org
On 09.12.23 20:16, Neal Gompa wrote:
>
> Once we land the patch to default to 4k sector size[1] regardless of
> page size, this should all work across all architectures, no?
>
> [1]: https://lore.kernel.org/linux-btrfs/20231116160235.2708131-2-neal@gompa.dev/
>
Per default, yes. But we can still set 64k sector size manually.
^ permalink raw reply [flat|nested] 15+ messages in thread