* Re: kernel BUG at lib/percpu-refcount.c:192
From: Mike Snitzer @ 2017-06-08 13:38 UTC (permalink / raw)
To: Marian Csontos; +Cc: linux-raid, Heinz Mauelshagen, device-mapper development
In-Reply-To: <9dcf0086-6d8a-29db-2a27-6c63af988388@redhat.com>
On Thu, Jun 08 2017 at 4:29am -0400,
Marian Csontos <mcsontos@redhat.com> wrote:
> Mike, Heinz, help please!
>
> RC4 and still present:
>
> [ 248.815709] CPU: 0 PID: 21 Comm: rcuos/1 Not tainted
> 4.12.0-0.rc4.git2.1.fc27.x86_64 #1
> [ 248.816841] Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2007
>
> ...and it affects RAID1 as well.
>
> Is this md wrong or is there a more serious issue? And if so is it a
> known one?
> I have not found any reference anywhere except this dm-devel thread.
> Is this a problem in LVM RAID only?
Unclear.
> Where to forward this? LKML? Fedora Bugzilla?
Certainly linux-raid (now cc'd).
> Is there anything else I should provide?
The linewrapping makes it hard to read these logs, but I'll defer to
others to weigh-in on any additional info needed.
Mike
> -- Martian
>
> On 05/18/2017 10:33 AM, Marian Csontos wrote:
> >When running `lvm lvcreate --type raid5 --size 4194304B --name
> >meta_r5 LVMTEST12279nqymhamb_vg --yes` there's a warning followed
> >by the bug.
> >
> >Also at [ 192.140142] it looks like there is an uninitialized
> >variable somewhere.
> >
> >env: rawhide, kvm, 2 (V)CPUs
> >lvm2: upstream
> >kernel: 4.12.0-0.rc1.git1.1.fc27.x86_64
> >
> >----
> >
> >[ 192.131076] mdX: bitmap file is out of date, doing full recovery
> >
> >[ 192.140142] percpu ref ( (null)) <= 0
> >(-9223372036854775807) after switching to atomic
> >
> >[ 192.140180] ------------[ cut here ]------------
> >[ 192.143734] WARNING: CPU: 0 PID: 10 at
> >lib/percpu-refcount.c:155
> >percpu_ref_switch_to_atomic_rcu+0x1cd/0x1f0
> >[ 192.146353] Modules linked in: dm_raid raid456
> >async_raid6_recov async_memcpy async_pq async_xor async_tx xor
> >raid6_pq ...|dm_thin_pool dm_cache_smq dm_cache dm_persistent_data
> >dm_bio_prison libcrc32c loop crct10dif_pclmul crc32_pclmul
> >crc32c_intel ...|ppdev ghash_clmulni_intel acpi_cpufreq
> >virtio_net virtio_balloon tpm_tis joydev tpm_tis_core parport_pc
> >parport tpm i2c_piix4 ...|virtio_blk cirrus drm_kms_helper ttm
> >drm serio_raw virtio_pci virtio_ring virtio ata_generic pata_acpi
> >[ 192.154276] CPU: 0 PID: 10 Comm: rcuos/0 Not tainted
> >4.12.0-0.rc1.git1.1.fc27.x86_64 #1
> >[ 192.155684] Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2007
> >[ 192.156741] task: ffff8d5bb6e73100 task.stack: ffffa9a3c0360000
> >[ 192.157830] RIP: 0010:percpu_ref_switch_to_atomic_rcu+0x1cd/0x1f0
> >[ 192.158956] RSP: 0018:ffffa9a3c0363e50 EFLAGS: 00010282
> >[ 192.159899] RAX: 0000000000000053 RBX: 8000000000000000 RCX:
> >0000000000000000
> >[ 192.161192] RDX: 0000000000000000 RSI: ffff8d5bb6e73d90 RDI:
> >0000000000000247
> >[ 192.162483] RBP: ffffa9a3c0363e68 R08: 0000000000000000 R09:
> >0000000000000000
> >[ 192.163762] R10: ffffa9a3c0363e00 R11: 0000000000000000 R12:
> >0000000000000000
> >[ 192.165144] R13: ffff8d5bb5252580 R14: ffff8d5bb5252580 R15:
> >0000000000000000
> >[ 192.166437] FS: 0000000000000000(0000)
> >GS:ffff8d5bb7800000(0000) knlGS:0000000000000000
> >[ 192.169392] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> >[ 192.171893] CR2: 00007ffc83d87d98 CR3: 0000000013e11000 CR4:
> >00000000000006f0
> >[ 192.174692] Call Trace:
> >[ 192.176627] ? percpu_ref_reinit+0x140/0x140
> >[ 192.178875] rcu_nocb_kthread+0x1ae/0x580
> >[ 192.181174] kthread+0x133/0x150
> >[ 192.183305] ? rcu_eqs_enter_common.constprop.68+0x1c0/0x1c0
> >[ 192.185813] ? kthread_create_on_node+0x70/0x70
> >[ 192.188110] ret_from_fork+0x31/0x40
> >[ 192.190338] Code: ff ff ff 80 3d a6 71 b9 00 00 0f 85 c1 fe ff
> >ff 49 8b 55 d8 49 8b 75 e8 48 c7 c7 f0 2b ce 8b c6 05 8a 71 b9 00
> >...|01 e8 89 95 d4 ff <0f> ff e9 9f fe ff ff f0 49 83 6d d8 01 0f
> >85 05 ff ff ff 48 89
> >[ 192.196842] ---[ end trace fbf926294c454ffa ]---
> >
> >[ 192.199425] ------------[ cut here ]------------
> >[ 192.200346] kernel BUG at lib/percpu-refcount.c:192!
> >[ 192.200346] invalid opcode: 0000 [#1] SMP
> >[ 192.200346] Modules linked in: dm_raid raid456
> >async_raid6_recov async_memcpy async_pq async_xor async_tx xor
> >raid6_pq ...|dm_thin_pool dm_cache_smq dm_cache dm_persistent_data
> >dm_bio_prison libcrc32c loop crct10dif_pclmul crc32_pclmul
> >crc32c_intel ...|ppdev ghash_clmulni_intel acpi_cpufreq
> >virtio_net virtio_balloon tpm_tis joydev tpm_tis_core parport_pc
> >parport tpm i2c_piix4 ...|virtio_blk cirrus drm_kms_helper ttm
> >drm serio_raw virtio_pci virtio_ring virtio ata_generic pata_acpi
> >[ 192.211647] CPU: 0 PID: 13767 Comm: mdX_raid5 Tainted: G
> >W 4.12.0-0.rc1.git1.1.fc27.x86_64 #1
> >[ 192.211647] Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2007
> >[ 192.211647] task: ffff8d5bb15f3100 task.stack: ffffa9a3c0d10000
> >[ 192.211647] RIP: 0010:__percpu_ref_switch_mode+0x278/0x280
> >[ 192.211647] RSP: 0018:ffffa9a3c0d13ce8 EFLAGS: 00010046
> >[ 192.211647] RAX: 0000000000000001 RBX: ffff8d5bb5252558 RCX:
> >0000000000000000
> >[ 192.211647] RDX: 0000000000000001 RSI: ffffffff8c05a098 RDI:
> >0000000000000046
> >[ 192.211647] RBP: ffffa9a3c0d13d30 R08: 0000000000000001 R09:
> >0000000000000000
> >[ 192.211647] R10: ffffa9a3c0d13ca0 R11: ffffffff8c05a098 R12:
> >0000000000000000
> >[ 192.211647] R13: 0000000000000000 R14: ffff8d5bb15f3100 R15:
> >ffff8d5bb508f000
> >[ 192.211647] FS: 0000000000000000(0000)
> >GS:ffff8d5bb7800000(0000) knlGS:0000000000000000
> >[ 192.211647] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> >[ 192.211647] CR2: 00007ffc83d87d98 CR3: 0000000013e11000 CR4:
> >00000000000006f0
> >[ 192.211647] Call Trace:
> >[ 192.211647] percpu_ref_switch_to_percpu+0x27/0x40
> >[ 192.211647] set_in_sync+0xd4/0xe0
> >[ 192.211647] md_check_recovery+0x1f5/0x4e0
> >[ 192.211647] raid5d+0x56/0x6a0 [raid456]
> >[ 192.211647] ? finish_wait+0x72/0x90
> >[ 192.211647] ? trace_hardirqs_on_caller+0xf4/0x190
> >[ 192.211647] ? trace_hardirqs_on+0xd/0x10
> >[ 192.211647] md_thread+0x138/0x180
> >[ 192.211647] ? md_thread+0x138/0x180
> >[ 192.211647] ? finish_wait+0x90/0x90
> >[ 192.211647] kthread+0x133/0x150
> >[ 192.211647] ? find_pers+0x70/0x70
> >[ 192.211647] ? kthread_create_on_node+0x70/0x70
> >[ 192.211647] ret_from_fork+0x31/0x40
> >[ 192.211647] Code: f0 4a c8 8b be d5 03 00 00 48 c7 c7 b1 75 c7
> >8b c6 05 ef 75 b9 00 01 e8 97 d2 c4 ff e9 e8 fe ff ff f0 48 83 03
> >...|01 e9 bf fe ff ff <0f> 0b e8 21 d9 be ff 90 55 48 89 e5 41 55
> >41 54 53 48 89 fb 48
> >[ 192.211647] RIP: __percpu_ref_switch_mode+0x278/0x280 RSP:
> >ffffa9a3c0d13ce8
> >[ 192.211647] ---[ end trace fbf926294c454ffb ]---
> >
> >[ 192.211647] BUG: sleeping function called from invalid context
> >at ./include/linux/percpu-rwsem.h:33
> >[ 192.211647] in_atomic(): 1, irqs_disabled(): 1, pid: 13767,
> >name: mdX_raid5
> >[ 192.211647] INFO: lockdep is turned off.
> >[ 192.211647] irq event stamp: 68
> >[ 192.211647] hardirqs last enabled at (67):
> >[<ffffffff8b983586>] _raw_spin_unlock_irqrestore+0x36/0x60
> >[ 192.211647] hardirqs last disabled at (68):
> >[<ffffffff8b983c47>] _raw_spin_lock_irqsave+0x27/0x90
> >[ 192.211647] softirqs last enabled at (0): [<ffffffff8b0ae308>]
> >copy_process.part.26+0x618/0x1ee0
> >[ 192.211647] softirqs last disabled at (0): [< (null)>]
> >(null)
> >[ 192.211647] CPU: 0 PID: 13767 Comm: mdX_raid5 Tainted: G D
> >W 4.12.0-0.rc1.git1.1.fc27.x86_64 #1
> >[ 192.211647] Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2007
> >[ 192.211647] Call Trace:
> >[ 192.211647] dump_stack+0x8e/0xcd
> >[ 192.211647] ___might_sleep+0x144/0x260
> >[ 192.211647] __might_sleep+0x4a/0x80
> >[ 192.211647] exit_signals+0x33/0x240
> >[ 192.211647] do_exit+0xb4/0xd30
> >[ 192.211647] ? kthread+0x133/0x150
> >[ 192.211647] ? find_pers+0x70/0x70
> >[ 192.211647] rewind_stack_do_exit+0x17/0x20
> >
> >[ 192.211647] note: mdX_raid5[13767] exited with preempt_count 2
> >
> >[ 220.071005] NMI watchdog: BUG: soft lockup - CPU#1 stuck for
> >22s! [lvm:13678]
> >[ 220.071079] Modules linked in: dm_raid raid456
> >async_raid6_recov async_memcpy async_pq async_xor async_tx xor
> >raid6_pq ...|dm_thin_pool dm_cache_smq dm_cache dm_persistent_data
> >dm_bio_prison libcrc32c loop crct10dif_pclmul crc32_pclmul
> >crc32c_intel ...|ppdev ghash_clmulni_intel acpi_cpufreq
> >virtio_net virtio_balloon tpm_tis joydev tpm_tis_core parport_pc
> >parport tpm i2c_piix4 ...|virtio_blk cirrus drm_kms_helper ttm
> >drm serio_raw virtio_pci virtio_ring virtio ata_generic pata_acpi
> >[ 220.083131] irq event stamp: 313716
> >[ 220.087117] hardirqs last enabled at (313715):
> >[<ffffffff8b983da7>] entry_SYSCALL_64_fastpath+0x5/0xbe
> >[ 220.091109] hardirqs last disabled at (313716):
> >[<ffffffff8b97afe4>] __schedule+0xc4/0xb40
> >[ 220.091109] softirqs last enabled at (301518):
> >[<ffffffff8b988d52>] __do_softirq+0x382/0x4ed
> >[ 220.095114] softirqs last disabled at (301511):
> >[<ffffffff8b0b8e7f>] irq_exit+0x10f/0x120
> >[ 220.099150] CPU: 1 PID: 13678 Comm: lvm Tainted: G D W
> >4.12.0-0.rc1.git1.1.fc27.x86_64 #1
> >[ 220.103049] Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2007
> >[ 220.103049] task: ffff8d5bb3c03100 task.stack: ffffa9a3c155c000
> >[ 220.107103] RIP: 0010:queued_spin_lock_slowpath+0x25/0x1b0
> >[ 220.107103] RSP: 0018:ffffa9a3c155f720 EFLAGS: 00000202
> >ORIG_RAX: ffffffffffffff10
> >[ 220.111144] RAX: 0000000000000001 RBX: ffff8d5bb5252448 RCX:
> >0000000000000000
> >[ 220.115126] RDX: 0000000000000001 RSI: 0000000000000001 RDI:
> >ffff8d5bb5252448
> >[ 220.115126] RBP: ffffa9a3c155f720 R08: 0000000000000000 R09:
> >0000000000000000
> >[ 220.119130] R10: ffffa9a3c155f6d0 R11: ffff8d5bb3c03100 R12:
> >ffff8d5bb5252448
> >[ 220.123139] R13: ffff8d5bb5252448 R14: ffff8d5bb379ca20 R15:
> >ffff8d5bb5252058
> >[ 220.127118] FS: 00007f1f8253b880(0000)
> >GS:ffff8d5bb7a00000(0000) knlGS:0000000000000000
> >[ 220.127118] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> >[ 220.131158] CR2: 00007fe2715ae000 CR3: 0000000075108000 CR4:
> >00000000000006e0
> >[ 220.135114] Call Trace:
> >[ 220.135114] do_raw_spin_lock+0xb2/0xc0
> >[ 220.139145] _raw_spin_lock+0x55/0x70
> >[ 220.139145] md_write_start+0x143/0x360
> >[ 220.143126] ? probe_sched_wakeup+0x1c/0x20
> >[ 220.143126] ? check_preempt_curr+0x27/0xb0
> >[ 220.147147] raid5_make_request+0xd0/0x1060 [raid456]
> >[ 220.147147] ? _raw_spin_unlock_irqrestore+0x5b/0x60
> >[ 220.151162] ? try_to_wake_up+0x58/0x520
> >[ 220.151162] ? finish_wait+0x90/0x90
> >[ 220.155099] ? mempool_alloc_slab+0x1d/0x30
> >[ 220.155099] ? finish_wait+0x90/0x90
> >[ 220.159136] raid_map+0x2b/0x40 [dm_raid]
> >[ 220.159136] __map_bio+0xc9/0x2b0
> >[ 220.163115] ? dm_put_table_device+0xd0/0xd0
> >[ 220.163115] __split_and_process_bio+0x293/0x590
> >[ 220.167133] dm_make_request+0x7f/0x120
> >[ 220.167133] ? generic_make_request+0xf9/0x3b0
> >[ 220.171131] generic_make_request+0x126/0x3b0
> >[ 220.171131] ? iov_iter_get_pages+0xc9/0x300
> >[ 220.175138] submit_bio+0x73/0x150
> >[ 220.175138] ? submit_bio+0x73/0x150
> >[ 220.179132] ? bio_iov_iter_get_pages+0xe0/0x120
> >[ 220.179132] ? __blkdev_direct_IO_simple+0x168/0x320
> >[ 220.183114] __blkdev_direct_IO_simple+0x184/0x320
> >[ 220.183114] ? lock_acquire+0xa3/0x1f0
> >[ 220.187129] ? bdput+0x20/0x20
> >[ 220.187129] blkdev_direct_IO+0x3b1/0x3e0
> >[ 220.191112] ? blkdev_direct_IO+0x3b1/0x3e0
> >[ 220.191112] ? current_time+0x1a/0x70
> >[ 220.195108] ? trace_hardirqs_on+0xd/0x10
> >[ 220.195108] generic_file_direct_write+0x9c/0x110
> >[ 220.199114] ? generic_file_direct_write+0x9c/0x110
> >[ 220.199114] __generic_file_write_iter+0xbe/0x1d0
> >[ 220.199114] blkdev_write_iter+0x96/0x110
> >[ 220.203115] __vfs_write+0xe8/0x160
> >[ 220.203115] ? __vfs_write+0xe8/0x160
> >[ 220.207211] vfs_write+0xc6/0x1c0
> >[ 220.207211] SyS_write+0x58/0xc0
> >[ 220.207211] entry_SYSCALL_64_fastpath+0x1f/0xbe
> >[ 220.211130] RIP: 0033:0x7f1f81309290
> >[ 220.211130] RSP: 002b:00007fffba453af8 EFLAGS: 00000246
> >ORIG_RAX: 0000000000000001
> >[ 220.215116] RAX: ffffffffffffffda RBX: 0000000000000001 RCX:
> >00007f1f81309290
> >[ 220.215116] RDX: 0000000000001000 RSI: 000055f0c4ab4000 RDI:
> >000000000000000c
> >[ 220.219114] RBP: 00007f1f815d5583 R08: fffffffffffffff0 R09:
> >0000000000000000
> >[ 220.219114] R10: 00007f1f815d4b38 R11: 0000000000000246 R12:
> >0000000000000001
> >[ 220.223148] R13: 0000000000000001 R14: 00007f1f815d5500 R15:
> >00007f1f815d08c0
> >[ 220.223148] Code: 1f 80 00 00 00 00 66 66 66 66 90 55 48 89 e5
> >66 66 90 66 90 ba 01 00 00 00 8b 07 85 c0 75 0a f0 0f b1 17 85 c0
> >...|75 f2 5d c3 f3 90 <eb> ec 81 fe 00 01 00 00 0f 84 1a 01 00 00
> >41 b8 01 01 00 00 b9
> >
> >--
> >dm-devel mailing list
> >dm-devel@redhat.com
> >https://www.redhat.com/mailman/listinfo/dm-devel
>
^ permalink raw reply
* [PATCH 2/3 v1] mdadm/test: Refactor the codes of 'test' script
From: Zhilong Liu @ 2017-06-08 9:20 UTC (permalink / raw)
To: Jes.Sorensen; +Cc: linux-raid, Zhilong Liu
In-Reply-To: <1496827888-17118-3-git-send-email-zlliu@suse.com>
Adding functions:
die()
uniform the abnormal situations that have to abort.
check_env()
do various basic checking before running test suite.
save_log()
collect array infos, include of dmesg, superblock,
bitmap and /proc/mdstat.
main()
the core function of this script.
Improve functions:
cleanup()
clear dmesg and remove the /vat/tmp/mdtest* files.
mdadm()
clear superblock once creating or building arrays
every time, because it's always creating arrays
many times in a test case.
check()
just tidy up with die(), didn't change code meanings.
testdev()
add checking $1 must be a block device, add 'return 0'
in final because this function exists in last line of
test case, such as tests/05r6tor0.
do_test()
add checking abnormal dmesg and changing log management.
do_help()
just recommend a better way to print Usage.
parse_args()
just tidy up.
Signed-off-by: Zhilong Liu <zlliu@suse.com>
---
test | 346 +++++++++++++++++++++++++++++++++++++------------------------------
1 file changed, 191 insertions(+), 155 deletions(-)
diff --git a/test b/test
index 7942d6e..37b3096 100755
--- a/test
+++ b/test
@@ -1,35 +1,17 @@
#!/bin/bash
#
# run test suite for mdadm
-user=`id -un`
-if [ " $user" != " root" ]
-then
- echo >&2 "test: testing can only be done as 'root'."
- exit 1
-fi
-
-prefix='[0-9][0-9]'
-
-dir=`pwd`
+dir=$(pwd)
mdadm=$dir/mdadm
-if [ \! -x $mdadm ]
-then
- echo >&2 "test: $mdadm isn't usable."
-fi
-
testdir="tests"
-logdir="$testdir/logs"
-logsave=0
-exitonerror=1
+targetdir="/var/tmp"
+logdir="$targetdir"
+config=/tmp/mdadm.conf
-echo "Testing on linux-$(uname -r) kernel"
+savelogs=0
+exitonerror=1
+prefix='[0-9][0-9]'
-# Check whether to run multipath tests
-modprobe multipath 2> /dev/null
-if grep -s 'Personalities : .*multipath' > /dev/null /proc/mdstat
-then
- MULTIPATH="yes"
-fi
INTEGRITY=yes
DEVTYPE=loop
LVM_VOLGROUP=mdtest
@@ -46,7 +28,6 @@ mdp1=/dev/md_d1
# We test mdadm on loop-back block devices.
# dir for storing files should be settable by command line maybe
-targetdir=/var/tmp
size=20000
# super0, round down to multiple of 64 and substract 64
mdsize0=19904
@@ -68,7 +49,51 @@ mdsize12=19988
# ddf needs bigger devices as 32Meg is reserved!
ddfsize=65536
-config=/tmp/mdadm.conf
+# $1 is optional parameter, it shows why to save log
+save_log() {
+ status=$1
+ logfile="$status""$_basename".log
+
+ cat $targetdir/stderr >> $targetdir/log
+ cp $targetdir/log $logdir/$_basename.log
+ echo "## $HOSTNAME: saving dmesg." >> $logdir/$logfile
+ dmesg -c >> $logdir/$logfile
+ $mdadm -As 2> /dev/null
+ echo "## $HOSTNAME: saving proc mdstat." >> $logdir/$logfile
+ cat /proc/mdstat >> $logdir/$logfile
+ array=($(mdadm -Ds | cut -d' ' -f2))
+ echo "## $HOSTNAME: mdadm -D ${array[@]}" >> $logdir/$logfile
+ $mdadm -D ${array[@]} >> $logdir/$logfile
+ [ "$1" == "fail" ] &&
+ echo "FAILED - see $logdir/$_basename.log and $logdir/$logfile for details"
+ # ignore saving external(external file, imsm...) bitmap
+ cat /proc/mdstat | grep -q "linear\|external" && return 0
+ if [ $DEVTYPE == 'lvm' ]
+ then
+ # not supported lvm type yet
+ echo
+ elif [ $DEVTYPE == 'loop' ]
+ then
+ if [ ! -z ${array[@]} -a ${#array[@]} -ge 1 ]
+ then
+ md_disks=($($mdadm -D -Y ${array[@]} | grep "/dev/$DEVTYPE" | cut -d'=' -f2))
+ cat /proc/mdstat | grep -q "bitmap"
+ if [ $? -eq 0 ]
+ then
+ echo "## $HOSTNAME: mdadm -X ${md_disks[@]}" >> $logdir/$logfile
+ $mdadm -X ${md_disks[@]} >> $logdir/$logfile
+ fi
+ else
+ echo "## $HOSTNAME: no array assembled!" >> $logdir/$logfile
+ fi
+ fi
+}
+
+die() {
+ echo -e "\n\tERROR: $* \n"
+ save_log fail
+ exit 2
+}
cleanup() {
udevadm settle
@@ -77,8 +102,9 @@ cleanup() {
loop)
for d in 0 1 2 3 4 5 6 7 8 9 10 11 12 13
do
- losetup -d /dev/loop$d # rm -f $targetdir/mdtest$d
+ losetup -d /dev/loop$d
rm -f /dev/disk/by-path/loop*
+ rm -f /var/tmp/mdtest$d
done
;;
lvm)
@@ -88,6 +114,7 @@ cleanup() {
done
;;
esac
+ dmesg -c > /dev/null
}
ctrl_c() {
@@ -98,23 +125,21 @@ do_setup() {
trap cleanup 0 1 3 15
trap ctrl_c 2
+ [ -d $logdir ] || mkdir -p $logdir
# make sure there are no loop devices remaining.
# udev started things can sometimes prevent them being stopped
# immediately
while grep loop /proc/partitions > /dev/null 2>&1
do
- mdadm -Ss
+ $mdadm -Ssq
losetup -d /dev/loop[0-9]* 2> /dev/null
- sleep 1
+ sleep 0.2
done
devlist=
for d in 0 1 2 3 4 5 6 7 8 9 10 11 12 13
do
sz=$size
- if [ $d -gt 7 ]
- then
- sz=$ddfsize
- fi
+ [ $d -gt 7 ] && sz=$ddfsize
case $DEVTYPE in
loop)
[ -f $targetdir/mdtest$d ] ||
@@ -169,7 +194,17 @@ mdadm() {
;;
esac
case $* in
- *-C* )
+ *-C* | *--create* | *-B* | *--build* )
+ # clear superblock every time once creating or
+ # building arrays, because it's always creating
+ # and building array many times in a test case.
+ for args in $*
+ do
+ [[ $args =~ "/dev/" ]] && {
+ [[ $args =~ "md" ]] ||
+ $mdadm --zero $args > /dev/null
+ }
+ done
$mdadm 2> $targetdir/stderr --quiet "$@" --auto=yes
;;
* )
@@ -191,39 +226,28 @@ mdadm() {
check() {
case $1 in
spares )
- spares=`tr '] ' '\012\012' < /proc/mdstat | grep -c '(S)' || exit 0`
- if [ $spares -ne $2 ]
- then
- echo >&2 "ERROR expected $2 spares, found $spares"
- exit 1
- fi
+ spares=$(tr '] ' '\012\012' < /proc/mdstat | grep -c '(S)' || exit 0)
+ [ $spares -ne $2 ] &&
+ die "expected $2 spares, found $spares"
;;
raid* | linear )
- grep -s "active $1 " /proc/mdstat > /dev/null || {
- echo >&2 "ERROR active $1 not found"
- cat /proc/mdstat
- exit 1
- }
+ grep -sq "active $1 " /proc/mdstat ||
+ die "active $1 not found"
;;
algorithm )
- grep -s " algorithm $2 " /proc/mdstat > /dev/null || {
- echo >&2 "ERROR algorithm $2 not found"
- cat /proc/mdstat
- exit 1
- }
+ grep -sq " algorithm $2 " /proc/mdstat ||
+ die "algorithm $2 not found"
;;
resync | recovery | reshape )
cnt=5
- while ! grep -s $1 /proc/mdstat > /dev/null
+ while ! grep -sq $1 /proc/mdstat
do
if [ $cnt -gt 0 ] && grep -v idle /sys/block/md*/md/sync_action > /dev/null
then # Something isn't idle - wait a bit
sleep 0.5
cnt=$[cnt-1]
else
- echo >&2 ERROR no $1 happening
- cat /proc/mdstat
- exit 1
+ die "no $1 happening"
fi
done
;;
@@ -234,22 +258,18 @@ check() {
# to do can still take a little longer than expected.
# add an extra check: is sync_completed shows the end is reached, assume
# there is no recovery.
- if grep -s -E '(resync|recovery|reshape) *=' > /dev/null /proc/mdstat
+ if grep -sq -E '(resync|recovery|reshape) *=' /proc/mdstat
then
incomplete=`grep / /sys/block/md*/md/sync_completed 2> /dev/null | sed '/^ *\([0-9]*\) \/ \1/d'`
- if [ -n "$incomplete" ]
- then
- echo >&2 "ERROR resync or recovery is happening!"
- cat /proc/mdstat
- exit 1
- fi
+ [ -n "$incomplete" ] &&
+ die "resync or recovery is happening!"
fi
;;
wait )
p=`cat /proc/sys/dev/raid/speed_limit_max`
echo 2000000 > /proc/sys/dev/raid/speed_limit_max
sleep 0.1
- while grep -E '(resync|recovery|reshape|check|repair) *=' > /dev/null /proc/mdstat ||
+ while grep -Eq '(resync|recovery|reshape|check|repair) *=' /proc/mdstat ||
grep -v idle > /dev/null /sys/block/md*/md/sync_action
do
sleep 0.5
@@ -257,45 +277,28 @@ check() {
echo $p > /proc/sys/dev/raid/speed_limit_max
;;
state )
- grep -s "blocks.*\[$2\]\$" /proc/mdstat > /dev/null || {
- echo >&2 "ERROR state $2 not found!"
- cat /proc/mdstat
- exit 1
- }
+ grep -sq "blocks.*\[$2\]\$" /proc/mdstat ||
+ die "state $2 not found!"
sleep 0.5
;;
bitmap )
- grep -s bitmap > /dev/null /proc/mdstat || {
- echo >&2 ERROR no bitmap
- cat /proc/mdstat
- exit 1
- }
+ grep -sq bitmap /proc/mdstat ||
+ die "no bitmap"
;;
nobitmap )
- if grep -s "bitmap" > /dev/null /proc/mdstat
- then
- echo >&2 ERROR bitmap present
- cat /proc/mdstat
- exit 1
- fi
+ grep -sq "bitmap" /proc/mdstat &&
+ die "bitmap present"
;;
readonly )
- grep -s "read-only" > /dev/null /proc/mdstat || {
- echo >&2 "ERROR array is not read-only!"
- cat /proc/mdstat
- exit 1
- }
+ grep -sq "read-only" /proc/mdstat ||
+ die "array is not read-only!"
;;
inactive )
- grep -s "inactive" > /dev/null /proc/mdstat || {
- echo >&2 "ERROR array is not inactive!"
- cat /proc/mdstat
- exit 1
- }
+ grep -sq "inactive" /proc/mdstat ||
+ die "array is not inactive!"
;;
* )
- echo >&2 ERROR unknown check $1
- exit 1
+ die "unknown check $1"
;;
esac
}
@@ -311,6 +314,8 @@ no_errors() {
# basic device test
testdev() {
+ # $1 must be a block device
+ [ -b $1 ] || die "$1 isn't a block device."
udevadm settle
dev=$1
cnt=$2
@@ -329,20 +334,11 @@ testdev() {
rasize=$[rasize/DEV_ROUND_K/2]
rasize=$[rasize*DEV_ROUND_K*2]
fi
- if [ `/sbin/blockdev --getsize $dev` -eq 0 ]
- then
- sleep 2
- fi
+ [ `/sbin/blockdev --getsize $dev` -eq 0 ] && sleep 2
_sz=`/sbin/blockdev --getsize $dev`
- if [ $rasize -lt $_sz -o $[rasize*4/5] -gt $_sz ]
- then
- echo "ERROR: size is wrong for $dev: $cnt * $dvsize (chunk=$chunk) = $rasize, not $_sz"
- exit 1
- fi
-}
-
-fast_sync() {
- echo 200000 > /proc/sys/dev/raid/speed_limit_max
+ [ $rasize -lt $_sz -o $[rasize*4/5] -gt $_sz ] &&
+ die "size is wrong for $dev: $cnt * $dvsize (chunk=$chunk) = $rasize, not $_sz"
+ return 0
}
rotest() {
@@ -359,7 +355,6 @@ do_test() {
# stop all arrays, just incase some script left an array active.
$mdadm -Ssq 2> /dev/null
mdadm --zero $devlist 2> /dev/null
- mdadm --zero $devlist 2> /dev/null
# this might have been reset: restore the default.
echo 2000 > /proc/sys/dev/raid/speed_limit_max
# source script in a subshell, so it has access to our
@@ -367,45 +362,40 @@ do_test() {
echo -ne "$_script... "
if ( set -ex ; . $_script ) &> $targetdir/log
then
+ dmesg | grep -iq "error\|call trace\|segfault" &&
+ die "dmesg prints errors when testing $_basename!"
echo "succeeded"
_fail=0
else
- log=log
- cat $targetdir/stderr >> $targetdir/log
- echo "=======================dmesg=================" >> $targetdir/log
- dmesg | tail -n 200 >> $targetdir/log
- if [ $exitonerror == 0 ]; then
- log=log-`basename $_script`
- mv $targetdir/log $logdir/$log
- fi
- echo "FAILED - see $logdir/$log for details"
+ save_log fail
_fail=1
fi
if [ "$savelogs" == "1" ]
then
- cp $targetdir/log $logdir/$_basename.log
- fi
- if [ "$_fail" == "1" -a "$exitonerror" == "1" ]
- then
- exit 1
+ [ "$targetdir" != "$logdir" ] &&
+ mv -f $targetdir/log $logdir/$_basename.log
fi
+ [ "$_fail" == "1" -a "$exitonerror" == "1" ] && exit 1
fi
}
do_help() {
- echo "Usage: $0 [options]"
- echo " Options:"
- echo " --tests=<test1,test2,..> Comma separated list of tests to run"
- echo " --disable-multipath Disable any tests involving multipath"
- echo " --disable-integrity Disable slow tests of RAID[56] consistency"
- echo " --logdir=<directory> Directory to save logfiles in"
- echo " --save-logs Save all logs in <logdir>"
- echo " --keep-going Don't stop on error, ie. run all tests"
- echo " --dev=[loop|lvm|ram] Use loop devices (default), LVM, or RAM disk"
- echo " --volgroup=<name> LVM volume group for LVM test"
- echo " setup Setup test environment and exit"
- echo " cleanup Cleanup test environment"
- echo " <prefix> Run tests with <prefix>"
+ cat <<-EOF
+ Usage: $0 [options]
+ Options:
+ --tests=<test1,test2,..> Comma separated list of tests to run
+ --disable-multipath Disable any tests involving multipath
+ --disable-integrity Disable slow tests of RAID[56] consistency
+ --logdir=<directory> Directory to save all logfiles in
+ --save-logs Usually use with --logdir together
+ --keep-going Don't stop on error, ie. run all tests
+ --dev=[loop|lvm|ram] Use loop devices (default), LVM, or RAM disk
+ --volgroup=<name> LVM volume group for LVM test
+ setup Setup test environment and exit
+ cleanup Cleanup test environment
+ <prefix> Run tests with <prefix>
+ EOF
+ exit 0
}
parse_args() {
@@ -457,38 +447,84 @@ parse_args() {
;;
--help )
do_help
- exit 0
;;
-* )
echo " $0: Unknown argument: $i"
do_help
- exit 0
;;
esac
done
}
-logdir=$targetdir
-parse_args $@
+check_env() {
+ user=$(id -un)
+ [ "X$user" != "Xroot" ] && {
+ echo "test: testing can only be done as 'root'."
+ exit 1
+ }
+ [ -x "raid6check" -a -x $mdadm ] || {
+ echo "test: please run 'make everything' before perform testing."
+ exit 1
+ }
+ for cmd in mdadm lsblk df udevadm losetup mkfs.ext3
+ do
+ which $cmd > /dev/null || {
+ echo "$cmd command not found!"
+ exit 1
+ }
+ done
+ mdadm_src_ver="$($mdadm -V 2>&1)"
+ mdadm_sbin_ver="$($(which mdadm) -V 2>&1)"
+ if [ "$mdadm_src_ver" != "$mdadm_sbin_ver" ]
+ then
+ # it's nessesary to 'make install' mdadm to /SBIN/DIR,
+ # such as systemd/mdadm-grow-continue@.service, would
+ # run as an instance by systemd when reshape happens,
+ # thus ensure that the correct mdadm is in testing.
+ echo "test: please run 'make install' before testing."
+ exit 1
+ fi
+ if ! $(df -T . | grep -iq ext)
+ then
+ # 'external file' bitmap only supports with ext[2-4] file system
+ echo "test: please run test suite with ext[2-4] file system."
+ exit 1
+ fi
+ if $(lsblk -a | grep -iq raid)
+ then
+ # For test purpose, it's better to make environment simple,
+ # donot run mdadm -Ss directly if there are RAIDs working.
+ echo "test: please run test suite without running RAIDs environment."
+ exit 1
+ fi
+ # Check whether to run multipath tests
+ modprobe multipath 2> /dev/null
+ grep -sq 'Personalities : .*multipath' /proc/mdstat &&
+ MULTIPATH="yes"
+}
-do_setup
-mkdir -p $logdir
+main() {
+ check_env
+ do_setup
+
+ echo "Testing on linux-$(uname -r) kernel"
+ [ "$savelogs" == "1" ] &&
+ echo "Saving logs to $logdir"
+ if [ "x$TESTLIST" != "x" ]
+ then
+ for script in $TESTLIST
+ do
+ do_test $testdir/$script
+ done
+ else
+ for script in $testdir/$prefix $testdir/$prefix*[^~]
+ do
+ do_test $script
+ done
+ fi
-if [ "$savelogs" == "1" ]
-then
- echo "Saving logs to $logdir"
-fi
+ exit 0
+}
-if [ "x$TESTLIST" != "x" ]
-then
- for script in $TESTLIST
- do
- do_test $testdir/$script
- done
-else
- for script in $testdir/$prefix $testdir/$prefix*[^~]
- do
- do_test $script
- done
-fi
-exit 0
+parse_args $@
+main
--
2.6.6
^ permalink raw reply related
* Re: [PATCH] md: don't use flush_signals in userspace processes
From: NeilBrown @ 2017-06-08 6:59 UTC (permalink / raw)
To: Mikulas Patocka, Shaohua Li
Cc: linux-raid, linux-kernel, Ingo Molnar, Peter Zijlstra
In-Reply-To: <alpine.LRH.2.02.1706071834250.22850@file01.intranet.prod.int.rdu2.redhat.com>
[-- Attachment #1: Type: text/plain, Size: 2605 bytes --]
On Wed, Jun 07 2017, Mikulas Patocka wrote:
> The function flush_signals clears all pending signals for the process. It
> may be used by kernel threads when we need to prepare a kernel thread for
> responding to signals. However using this function for an userspaces
> processes is incorrect - clearing signals without the program expecting it
> can cause misbehavior.
>
> The raid1 and raid5 code uses flush_signals in its request routine because
> it wants to prepare for an interruptible wait. This patch drops
> flush_signals and uses sigprocmask instead to block all signals (including
> SIGKILL) around the schedule() call. The signals are not lost, but the
> schedule() call won't respond to them.
>
> Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
> Cc: stable@vger.kernel.org
Thanks for catching that!
Acked-by: NeilBrown <neilb@suse.com>
NeilBrown
>
> ---
> drivers/md/raid1.c | 5 ++++-
> drivers/md/raid5.c | 5 ++++-
> 2 files changed, 8 insertions(+), 2 deletions(-)
>
> Index: linux-4.12-rc4/drivers/md/raid1.c
> ===================================================================
> --- linux-4.12-rc4.orig/drivers/md/raid1.c
> +++ linux-4.12-rc4/drivers/md/raid1.c
> @@ -1335,7 +1335,7 @@ static void raid1_write_request(struct m
> */
> DEFINE_WAIT(w);
> for (;;) {
> - flush_signals(current);
> + sigset_t full, old;
> prepare_to_wait(&conf->wait_barrier,
> &w, TASK_INTERRUPTIBLE);
> if (bio_end_sector(bio) <= mddev->suspend_lo ||
> @@ -1345,7 +1345,10 @@ static void raid1_write_request(struct m
> bio->bi_iter.bi_sector,
> bio_end_sector(bio))))
> break;
> + sigfillset(&full);
> + sigprocmask(SIG_BLOCK, &full, &old);
> schedule();
> + sigprocmask(SIG_SETMASK, &old, NULL);
> }
> finish_wait(&conf->wait_barrier, &w);
> }
> Index: linux-4.12-rc4/drivers/md/raid5.c
> ===================================================================
> --- linux-4.12-rc4.orig/drivers/md/raid5.c
> +++ linux-4.12-rc4/drivers/md/raid5.c
> @@ -5693,12 +5693,15 @@ static void raid5_make_request(struct md
> * userspace, we want an interruptible
> * wait.
> */
> - flush_signals(current);
> prepare_to_wait(&conf->wait_for_overlap,
> &w, TASK_INTERRUPTIBLE);
> if (logical_sector >= mddev->suspend_lo &&
> logical_sector < mddev->suspend_hi) {
> + sigset_t full, old;
> + sigfillset(&full);
> + sigprocmask(SIG_BLOCK, &full, &old);
> schedule();
> + sigprocmask(SIG_SETMASK, &old, NULL);
> do_prepare = true;
> }
> goto retry;
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]
^ permalink raw reply
* [PATCH] md: don't use flush_signals in userspace processes
From: Mikulas Patocka @ 2017-06-07 23:05 UTC (permalink / raw)
To: Shaohua Li, NeilBrown
Cc: linux-raid, linux-kernel, Ingo Molnar, Peter Zijlstra
The function flush_signals clears all pending signals for the process. It
may be used by kernel threads when we need to prepare a kernel thread for
responding to signals. However using this function for an userspaces
processes is incorrect - clearing signals without the program expecting it
can cause misbehavior.
The raid1 and raid5 code uses flush_signals in its request routine because
it wants to prepare for an interruptible wait. This patch drops
flush_signals and uses sigprocmask instead to block all signals (including
SIGKILL) around the schedule() call. The signals are not lost, but the
schedule() call won't respond to them.
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
Cc: stable@vger.kernel.org
---
drivers/md/raid1.c | 5 ++++-
drivers/md/raid5.c | 5 ++++-
2 files changed, 8 insertions(+), 2 deletions(-)
Index: linux-4.12-rc4/drivers/md/raid1.c
===================================================================
--- linux-4.12-rc4.orig/drivers/md/raid1.c
+++ linux-4.12-rc4/drivers/md/raid1.c
@@ -1335,7 +1335,7 @@ static void raid1_write_request(struct m
*/
DEFINE_WAIT(w);
for (;;) {
- flush_signals(current);
+ sigset_t full, old;
prepare_to_wait(&conf->wait_barrier,
&w, TASK_INTERRUPTIBLE);
if (bio_end_sector(bio) <= mddev->suspend_lo ||
@@ -1345,7 +1345,10 @@ static void raid1_write_request(struct m
bio->bi_iter.bi_sector,
bio_end_sector(bio))))
break;
+ sigfillset(&full);
+ sigprocmask(SIG_BLOCK, &full, &old);
schedule();
+ sigprocmask(SIG_SETMASK, &old, NULL);
}
finish_wait(&conf->wait_barrier, &w);
}
Index: linux-4.12-rc4/drivers/md/raid5.c
===================================================================
--- linux-4.12-rc4.orig/drivers/md/raid5.c
+++ linux-4.12-rc4/drivers/md/raid5.c
@@ -5693,12 +5693,15 @@ static void raid5_make_request(struct md
* userspace, we want an interruptible
* wait.
*/
- flush_signals(current);
prepare_to_wait(&conf->wait_for_overlap,
&w, TASK_INTERRUPTIBLE);
if (logical_sector >= mddev->suspend_lo &&
logical_sector < mddev->suspend_hi) {
+ sigset_t full, old;
+ sigfillset(&full);
+ sigprocmask(SIG_BLOCK, &full, &old);
schedule();
+ sigprocmask(SIG_SETMASK, &old, NULL);
do_prepare = true;
}
goto retry;
^ permalink raw reply
* Re: [PATCH] mdadm: Send version text to stdout rather than stderr
From: Jes Sorensen @ 2017-06-07 15:21 UTC (permalink / raw)
To: NeilBrown, Zhilong Liu; +Cc: linux-raid
In-Reply-To: <877f0od124.fsf@notabene.neil.brown.name>
On 06/06/2017 09:44 PM, NeilBrown wrote:
> On Tue, Jun 06 2017, Zhilong Liu wrote:
>
>> This helps with piping to a paper.
>
> It might help you, but it is an API change and could cause problems for
> others.
>
> Some programs, such as "mount -V", send version info to stdout.
> Others, like "gcc -v", send it to stderr.
> There is no uniform standard, it is best to leave the behavior
> unchanged.
>
> If you need to capture the output use
> mdadm -V |& ...
> or
> mdadm -V 2>&1
>
> or similar.
>
> NeilBrtown
I agree with mr Brtown here :)
Changing the API is a big issue. We need to avoid this unless it is
absolutely necessary, so we don't break things for people trying to run
newer mdadm on older distros or those relying on third party scripts.
Jes
^ permalink raw reply
* [PATCH 3/3] mdadm/test: Get rid of fast_sync()
From: Zhilong Liu @ 2017-06-07 9:31 UTC (permalink / raw)
To: Jes.Sorensen; +Cc: linux-raid, Zhilong Liu
In-Reply-To: <1496827888-17118-1-git-send-email-zlliu@suse.com>
fast_sync() is no longer used, so remove it.
Signed-off-by: Zhilong Liu <zlliu@suse.com>
---
test | 4 ----
1 file changed, 4 deletions(-)
diff --git a/test b/test
index 8ecfe99..3cb4bc0 100755
--- a/test
+++ b/test
@@ -337,10 +337,6 @@ testdev() {
return 0
}
-fast_sync() {
- echo 200000 > /proc/sys/dev/raid/speed_limit_max
-}
-
rotest() {
dev=$1
fsck -fn $dev >&2
--
2.6.6
^ permalink raw reply related
* [PATCH 2/3] mdadm/test: Refactor the codes of 'test' script
From: Zhilong Liu @ 2017-06-07 9:31 UTC (permalink / raw)
To: Jes.Sorensen; +Cc: linux-raid, Zhilong Liu
In-Reply-To: <1496827888-17118-1-git-send-email-zlliu@suse.com>
Adding functions:
die()
uniform the abnormal situations that have to abort.
check_env()
do various basic checking before running test suite.
save_log()
collect array infos, include of dmesg, superblock,
bitmap and /proc/mdstat.
main()
the core function of this script.
Improve functions:
cleanup()
clear dmesg and remove the /vat/tmp/mdtest* files.
mdadm()
clear superblock once creating or building arrays
every time, because it's always creating arrays
many times in a test case.
check()
just tidy up with die(), didn't change code meanings.
testdev()
add checking $1 must be a block device, add 'return 0'
in final because this function exists in last line of
test case, such as tests/05r6tor0.
do_test()
add checking abnormal dmesg and changing log management.
do_help()
just recommend a better way to print Usage.
parse_args()
just tidy up.
Signed-off-by: Zhilong Liu <zlliu@suse.com>
---
test | 339 +++++++++++++++++++++++++++++++++++++------------------------------
1 file changed, 189 insertions(+), 150 deletions(-)
diff --git a/test b/test
index 7942d6e..8ecfe99 100755
--- a/test
+++ b/test
@@ -1,35 +1,17 @@
#!/bin/bash
#
# run test suite for mdadm
-user=`id -un`
-if [ " $user" != " root" ]
-then
- echo >&2 "test: testing can only be done as 'root'."
- exit 1
-fi
-
-prefix='[0-9][0-9]'
-
-dir=`pwd`
+dir=$(pwd)
mdadm=$dir/mdadm
-if [ \! -x $mdadm ]
-then
- echo >&2 "test: $mdadm isn't usable."
-fi
-
testdir="tests"
-logdir="$testdir/logs"
-logsave=0
-exitonerror=1
+targetdir="/var/tmp"
+logdir="$targetdir"
+config=/tmp/mdadm.conf
-echo "Testing on linux-$(uname -r) kernel"
+savelogs=0
+exitonerror=1
+prefix='[0-9][0-9]'
-# Check whether to run multipath tests
-modprobe multipath 2> /dev/null
-if grep -s 'Personalities : .*multipath' > /dev/null /proc/mdstat
-then
- MULTIPATH="yes"
-fi
INTEGRITY=yes
DEVTYPE=loop
LVM_VOLGROUP=mdtest
@@ -46,7 +28,6 @@ mdp1=/dev/md_d1
# We test mdadm on loop-back block devices.
# dir for storing files should be settable by command line maybe
-targetdir=/var/tmp
size=20000
# super0, round down to multiple of 64 and substract 64
mdsize0=19904
@@ -68,7 +49,47 @@ mdsize12=19988
# ddf needs bigger devices as 32Meg is reserved!
ddfsize=65536
-config=/tmp/mdadm.conf
+# $1 is optional parameter, it shows why to save log
+save_log() {
+ status=$1
+ logfile="$status""$_basename".log
+
+ echo "## $HOSTNAME: saving dmesg." >> $logdir/$logfile
+ dmesg -c >> $logdir/$logfile
+ $mdadm -As 2> /dev/null
+ echo "## $HOSTNAME: saving proc mdstat." >> $logdir/$logfile
+ cat /proc/mdstat >> $logdir/$logfile
+ array=($(mdadm -Ds | cut -d' ' -f2))
+ echo "## $HOSTNAME: mdadm -D ${array[@]}" >> $logdir/$logfile
+ $mdadm -D ${array[@]} >> $logdir/$logfile
+ # ignore saving external(external file, imsm...) bitmap
+ cat /proc/mdstat | grep -q "linear\|external" && return 0
+ if [ $DEVTYPE == 'lvm' ]
+ then
+ # not supported lvm type yet
+ echo
+ elif [ $DEVTYPE == 'loop' ]
+ then
+ if [ ! -z ${array[@]} -a ${#array[@]} -ge 1 ]
+ then
+ md_disks=($($mdadm -D -Y ${array[@]} | grep "/dev/$DEVTYPE" | cut -d'=' -f2))
+ cat /proc/mdstat | grep -q "bitmap"
+ if [ $? -eq 0 ]
+ then
+ echo "## $HOSTNAME: mdadm -X ${md_disks[@]}" >> $logdir/$logfile
+ $mdadm -X ${md_disks[@]} >> $logdir/$logfile
+ fi
+ else
+ echo "## $HOSTNAME: no array assembled!" >> $logdir/$logfile
+ fi
+ fi
+}
+
+die() {
+ echo -e "\n\tERROR: $* \n"
+ save_log fail
+ exit 2
+}
cleanup() {
udevadm settle
@@ -77,8 +98,9 @@ cleanup() {
loop)
for d in 0 1 2 3 4 5 6 7 8 9 10 11 12 13
do
- losetup -d /dev/loop$d # rm -f $targetdir/mdtest$d
+ losetup -d /dev/loop$d
rm -f /dev/disk/by-path/loop*
+ rm -f /var/tmp/mdtest$d
done
;;
lvm)
@@ -88,6 +110,7 @@ cleanup() {
done
;;
esac
+ dmesg -c > /dev/null
}
ctrl_c() {
@@ -98,23 +121,21 @@ do_setup() {
trap cleanup 0 1 3 15
trap ctrl_c 2
+ [ -d $logdir ] || mkdir -p $logdir
# make sure there are no loop devices remaining.
# udev started things can sometimes prevent them being stopped
# immediately
while grep loop /proc/partitions > /dev/null 2>&1
do
- mdadm -Ss
+ $mdadm -Ssq
losetup -d /dev/loop[0-9]* 2> /dev/null
- sleep 1
+ sleep 0.2
done
devlist=
for d in 0 1 2 3 4 5 6 7 8 9 10 11 12 13
do
sz=$size
- if [ $d -gt 7 ]
- then
- sz=$ddfsize
- fi
+ [ $d -gt 7 ] && sz=$ddfsize
case $DEVTYPE in
loop)
[ -f $targetdir/mdtest$d ] ||
@@ -169,7 +190,17 @@ mdadm() {
;;
esac
case $* in
- *-C* )
+ *-C* | *--create* | *-B* | *--build* )
+ # clear superblock every time once creating or
+ # building arrays, because it's always creating
+ # and building array many times in a test case.
+ for args in $*
+ do
+ [[ $args =~ "/dev/" ]] && {
+ [[ $args =~ "md" ]] ||
+ $mdadm --zero $args > /dev/null
+ }
+ done
$mdadm 2> $targetdir/stderr --quiet "$@" --auto=yes
;;
* )
@@ -191,39 +222,28 @@ mdadm() {
check() {
case $1 in
spares )
- spares=`tr '] ' '\012\012' < /proc/mdstat | grep -c '(S)' || exit 0`
- if [ $spares -ne $2 ]
- then
- echo >&2 "ERROR expected $2 spares, found $spares"
- exit 1
- fi
+ spares=$(tr '] ' '\012\012' < /proc/mdstat | grep -c '(S)' || exit 0)
+ [ $spares -ne $2 ] &&
+ die "expected $2 spares, found $spares"
;;
raid* | linear )
- grep -s "active $1 " /proc/mdstat > /dev/null || {
- echo >&2 "ERROR active $1 not found"
- cat /proc/mdstat
- exit 1
- }
+ grep -sq "active $1 " /proc/mdstat ||
+ die "active $1 not found"
;;
algorithm )
- grep -s " algorithm $2 " /proc/mdstat > /dev/null || {
- echo >&2 "ERROR algorithm $2 not found"
- cat /proc/mdstat
- exit 1
- }
+ grep -sq " algorithm $2 " /proc/mdstat ||
+ die "algorithm $2 not found"
;;
resync | recovery | reshape )
cnt=5
- while ! grep -s $1 /proc/mdstat > /dev/null
+ while ! grep -sq $1 /proc/mdstat
do
if [ $cnt -gt 0 ] && grep -v idle /sys/block/md*/md/sync_action > /dev/null
then # Something isn't idle - wait a bit
sleep 0.5
cnt=$[cnt-1]
else
- echo >&2 ERROR no $1 happening
- cat /proc/mdstat
- exit 1
+ die "no $1 happening"
fi
done
;;
@@ -234,22 +254,18 @@ check() {
# to do can still take a little longer than expected.
# add an extra check: is sync_completed shows the end is reached, assume
# there is no recovery.
- if grep -s -E '(resync|recovery|reshape) *=' > /dev/null /proc/mdstat
+ if grep -sq -E '(resync|recovery|reshape) *=' /proc/mdstat
then
incomplete=`grep / /sys/block/md*/md/sync_completed 2> /dev/null | sed '/^ *\([0-9]*\) \/ \1/d'`
- if [ -n "$incomplete" ]
- then
- echo >&2 "ERROR resync or recovery is happening!"
- cat /proc/mdstat
- exit 1
- fi
+ [ -n "$incomplete" ] &&
+ die "resync or recovery is happening!"
fi
;;
wait )
p=`cat /proc/sys/dev/raid/speed_limit_max`
echo 2000000 > /proc/sys/dev/raid/speed_limit_max
sleep 0.1
- while grep -E '(resync|recovery|reshape|check|repair) *=' > /dev/null /proc/mdstat ||
+ while grep -Eq '(resync|recovery|reshape|check|repair) *=' /proc/mdstat ||
grep -v idle > /dev/null /sys/block/md*/md/sync_action
do
sleep 0.5
@@ -257,45 +273,28 @@ check() {
echo $p > /proc/sys/dev/raid/speed_limit_max
;;
state )
- grep -s "blocks.*\[$2\]\$" /proc/mdstat > /dev/null || {
- echo >&2 "ERROR state $2 not found!"
- cat /proc/mdstat
- exit 1
- }
+ grep -sq "blocks.*\[$2\]\$" /proc/mdstat ||
+ die "state $2 not found!"
sleep 0.5
;;
bitmap )
- grep -s bitmap > /dev/null /proc/mdstat || {
- echo >&2 ERROR no bitmap
- cat /proc/mdstat
- exit 1
- }
+ grep -sq bitmap /proc/mdstat ||
+ die "no bitmap"
;;
nobitmap )
- if grep -s "bitmap" > /dev/null /proc/mdstat
- then
- echo >&2 ERROR bitmap present
- cat /proc/mdstat
- exit 1
- fi
+ grep -sq "bitmap" /proc/mdstat &&
+ die "bitmap present"
;;
readonly )
- grep -s "read-only" > /dev/null /proc/mdstat || {
- echo >&2 "ERROR array is not read-only!"
- cat /proc/mdstat
- exit 1
- }
+ grep -sq "read-only" /proc/mdstat ||
+ die "array is not read-only!"
;;
inactive )
- grep -s "inactive" > /dev/null /proc/mdstat || {
- echo >&2 "ERROR array is not inactive!"
- cat /proc/mdstat
- exit 1
- }
+ grep -sq "inactive" /proc/mdstat ||
+ die "array is not inactive!"
;;
* )
- echo >&2 ERROR unknown check $1
- exit 1
+ die "unknown check $1"
;;
esac
}
@@ -311,6 +310,8 @@ no_errors() {
# basic device test
testdev() {
+ # $1 must be a block device
+ [ -b $1 ] || die "$1 isn't a block device."
udevadm settle
dev=$1
cnt=$2
@@ -329,16 +330,11 @@ testdev() {
rasize=$[rasize/DEV_ROUND_K/2]
rasize=$[rasize*DEV_ROUND_K*2]
fi
- if [ `/sbin/blockdev --getsize $dev` -eq 0 ]
- then
- sleep 2
- fi
+ [ `/sbin/blockdev --getsize $dev` -eq 0 ] && sleep 2
_sz=`/sbin/blockdev --getsize $dev`
- if [ $rasize -lt $_sz -o $[rasize*4/5] -gt $_sz ]
- then
- echo "ERROR: size is wrong for $dev: $cnt * $dvsize (chunk=$chunk) = $rasize, not $_sz"
- exit 1
- fi
+ [ $rasize -lt $_sz -o $[rasize*4/5] -gt $_sz ] &&
+ die "size is wrong for $dev: $cnt * $dvsize (chunk=$chunk) = $rasize, not $_sz"
+ return 0
}
fast_sync() {
@@ -359,7 +355,6 @@ do_test() {
# stop all arrays, just incase some script left an array active.
$mdadm -Ssq 2> /dev/null
mdadm --zero $devlist 2> /dev/null
- mdadm --zero $devlist 2> /dev/null
# this might have been reset: restore the default.
echo 2000 > /proc/sys/dev/raid/speed_limit_max
# source script in a subshell, so it has access to our
@@ -367,45 +362,43 @@ do_test() {
echo -ne "$_script... "
if ( set -ex ; . $_script ) &> $targetdir/log
then
+ dmesg | grep -iq "error\|call trace\|segfault" &&
+ die "dmesg prints errors when testing $_basename"
echo "succeeded"
_fail=0
else
- log=log
cat $targetdir/stderr >> $targetdir/log
- echo "=======================dmesg=================" >> $targetdir/log
- dmesg | tail -n 200 >> $targetdir/log
- if [ $exitonerror == 0 ]; then
- log=log-`basename $_script`
- mv $targetdir/log $logdir/$log
- fi
- echo "FAILED - see $logdir/$log for details"
+ save_log fail
+ cp $targetdir/log $logdir/$_basename.log
+ echo "FAILED - see $logdir/$_basename.log and $logdir/$logfile for details"
_fail=1
fi
if [ "$savelogs" == "1" ]
then
- cp $targetdir/log $logdir/$_basename.log
- fi
- if [ "$_fail" == "1" -a "$exitonerror" == "1" ]
- then
- exit 1
+ [ "$targetdir" != "$logdir" ] &&
+ mv $targetdir/log $logdir/$_basename.log
fi
+ [ "$_fail" == "1" -a "$exitonerror" == "1" ] && exit 1
fi
}
do_help() {
- echo "Usage: $0 [options]"
- echo " Options:"
- echo " --tests=<test1,test2,..> Comma separated list of tests to run"
- echo " --disable-multipath Disable any tests involving multipath"
- echo " --disable-integrity Disable slow tests of RAID[56] consistency"
- echo " --logdir=<directory> Directory to save logfiles in"
- echo " --save-logs Save all logs in <logdir>"
- echo " --keep-going Don't stop on error, ie. run all tests"
- echo " --dev=[loop|lvm|ram] Use loop devices (default), LVM, or RAM disk"
- echo " --volgroup=<name> LVM volume group for LVM test"
- echo " setup Setup test environment and exit"
- echo " cleanup Cleanup test environment"
- echo " <prefix> Run tests with <prefix>"
+ cat <<-EOF
+ Usage: $0 [options]
+ Options:
+ --tests=<test1,test2,..> Comma separated list of tests to run
+ --disable-multipath Disable any tests involving multipath
+ --disable-integrity Disable slow tests of RAID[56] consistency
+ --logdir=<directory> Directory to save all logfiles in
+ --save-logs Usually use with --logdir together
+ --keep-going Don't stop on error, ie. run all tests
+ --dev=[loop|lvm|ram] Use loop devices (default), LVM, or RAM disk
+ --volgroup=<name> LVM volume group for LVM test
+ setup Setup test environment and exit
+ cleanup Cleanup test environment
+ <prefix> Run tests with <prefix>
+ EOF
+ exit 0
}
parse_args() {
@@ -457,38 +450,84 @@ parse_args() {
;;
--help )
do_help
- exit 0
;;
-* )
echo " $0: Unknown argument: $i"
do_help
- exit 0
;;
esac
done
}
-logdir=$targetdir
-parse_args $@
+check_env() {
+ user=$(id -un)
+ [ "X$user" != "Xroot" ] && {
+ echo "test: testing can only be done as 'root'."
+ exit 1
+ }
+ [ -x "raid6check" -a -x $mdadm ] || {
+ echo "test: please run 'make everything' before perform testing."
+ exit 1
+ }
+ for cmd in mdadm lsblk df udevadm losetup mkfs.ext3
+ do
+ which $cmd > /dev/null || {
+ echo "$cmd command not found!"
+ exit 1
+ }
+ done
+ mdadm_src_ver="$($mdadm -V 2>&1)"
+ mdadm_sbin_ver="$($(which mdadm) -V 2>&1)"
+ if [ "$mdadm_src_ver" != "$mdadm_sbin_ver" ]
+ then
+ # it's nessesary to 'make install' mdadm to /SBIN/DIR,
+ # such as systemd/mdadm-grow-continue@.service, would
+ # run as an instance by systemd when reshape happens,
+ # thus ensure that the correct mdadm is in testing.
+ echo "test: please run 'make install' before testing."
+ exit 1
+ fi
+ if ! $(df -T . | grep -iq ext)
+ then
+ # 'external file' bitmap only supports with ext[2-4] file system
+ echo "test: please run test suite with ext[2-4] file system."
+ exit 1
+ fi
+ if $(lsblk -a | grep -iq raid)
+ then
+ # For test purpose, it's better to make environment simple,
+ # donot run mdadm -Ss directly if there are RAIDs working.
+ echo "test: please run test suite without running RAIDs environment."
+ exit 1
+ fi
+ # Check whether to run multipath tests
+ modprobe multipath 2> /dev/null
+ grep -sq 'Personalities : .*multipath' /proc/mdstat &&
+ MULTIPATH="yes"
+}
-do_setup
-mkdir -p $logdir
+main() {
+ check_env
+ do_setup
+
+ echo "Testing on linux-$(uname -r) kernel"
+ [ "$savelogs" == "1" ] &&
+ echo "Saving logs to $logdir"
+ if [ "x$TESTLIST" != "x" ]
+ then
+ for script in $TESTLIST
+ do
+ do_test $testdir/$script
+ done
+ else
+ for script in $testdir/$prefix $testdir/$prefix*[^~]
+ do
+ do_test $script
+ done
+ fi
-if [ "$savelogs" == "1" ]
-then
- echo "Saving logs to $logdir"
-fi
+ exit 0
+}
-if [ "x$TESTLIST" != "x" ]
-then
- for script in $TESTLIST
- do
- do_test $testdir/$script
- done
-else
- for script in $testdir/$prefix $testdir/$prefix*[^~]
- do
- do_test $script
- done
-fi
-exit 0
+parse_args $@
+main
--
2.6.6
^ permalink raw reply related
* [PATCH 1/3] mdadm/test: Convert code format to use Tab
From: Zhilong Liu @ 2017-06-07 9:31 UTC (permalink / raw)
To: Jes.Sorensen; +Cc: linux-raid, Zhilong Liu
In-Reply-To: <1496827888-17118-1-git-send-email-zlliu@suse.com>
In case to make codes neat, this commit didn't change
any codes, just tidy up and use Tab as code format.
Signed-off-by: Zhilong Liu <zlliu@suse.com>
---
test | 702 ++++++++++++++++++++++++++++++++++++-------------------------------
1 file changed, 378 insertions(+), 324 deletions(-)
diff --git a/test b/test
index 13f1bda..7942d6e 100755
--- a/test
+++ b/test
@@ -3,8 +3,9 @@
# run test suite for mdadm
user=`id -un`
if [ " $user" != " root" ]
-then echo >&2 "test: testing can only be done as 'root'."
- exit 1;
+then
+ echo >&2 "test: testing can only be done as 'root'."
+ exit 1
fi
prefix='[0-9][0-9]'
@@ -13,7 +14,7 @@ dir=`pwd`
mdadm=$dir/mdadm
if [ \! -x $mdadm ]
then
- echo >&2 "test: $mdadm isn't usable."
+ echo >&2 "test: $mdadm isn't usable."
fi
testdir="tests"
@@ -25,8 +26,9 @@ echo "Testing on linux-$(uname -r) kernel"
# Check whether to run multipath tests
modprobe multipath 2> /dev/null
-if grep -s 'Personalities : .*multipath' > /dev/null /proc/mdstat ; then
- MULTIPATH="yes"
+if grep -s 'Personalities : .*multipath' > /dev/null /proc/mdstat
+then
+ MULTIPATH="yes"
fi
INTEGRITY=yes
DEVTYPE=loop
@@ -36,7 +38,9 @@ LVM_VOLGROUP=mdtest
export MDADM_NO_SYSTEMCTL=1
# assume md0, md1, md2 exist in /dev
-md0=/dev/md0 md1=/dev/md1 md2=/dev/md2
+md0=/dev/md0
+md1=/dev/md1
+md2=/dev/md2
mdp0=/dev/md_d0
mdp1=/dev/md_d1
@@ -69,21 +73,21 @@ config=/tmp/mdadm.conf
cleanup() {
udevadm settle
$mdadm -Ssq 2> /dev/null
- case $DEVTYPE in
- loop)
- for d in 0 1 2 3 4 5 6 7 8 9 10 11 12 13
- do
- losetup -d /dev/loop$d ; # rm -f $targetdir/mdtest$d
- rm -f /dev/disk/by-path/loop*
- done
- ;;
- lvm)
- for d in 0 1 2 3 4 5 6 7 8 9 10 11 12 13
- do
- eval "lvremove --quiet -f \$dev$d"
- done
- ;;
- esac
+ case $DEVTYPE in
+ loop)
+ for d in 0 1 2 3 4 5 6 7 8 9 10 11 12 13
+ do
+ losetup -d /dev/loop$d # rm -f $targetdir/mdtest$d
+ rm -f /dev/disk/by-path/loop*
+ done
+ ;;
+ lvm)
+ for d in 0 1 2 3 4 5 6 7 8 9 10 11 12 13
+ do
+ eval "lvremove --quiet -f \$dev$d"
+ done
+ ;;
+ esac
}
ctrl_c() {
@@ -91,329 +95,377 @@ ctrl_c() {
}
do_setup() {
- trap cleanup 0 1 3 15
- trap ctrl_c 2
+ trap cleanup 0 1 3 15
+ trap ctrl_c 2
- # make sure there are no loop devices remaining.
- # udev started things can sometimes prevent them being stopped
- # immediately
- while grep loop /proc/partitions > /dev/null 2>&1
- do
- mdadm -Ss
- losetup -d /dev/loop[0-9]* 2> /dev/null
- sleep 1
- done
- devlist=
- for d in 0 1 2 3 4 5 6 7 8 9 10 11 12 13
- do
- sz=$size
- if [ $d -gt 7 ]; then sz=$ddfsize ; fi
- case $DEVTYPE in
- loop)
- [ -f $targetdir/mdtest$d ] || dd if=/dev/zero of=$targetdir/mdtest$d count=$sz bs=1K > /dev/null 2>&1
- # make sure udev doesn't touch
- mdadm --zero $targetdir/mdtest$d 2> /dev/null
- [ -b /dev/loop$d ] || mknod /dev/loop$d b 7 $d
- if [ $d -eq 7 ]
- then
- losetup /dev/loop$d $targetdir/mdtest6 # for multipath use
- else
- losetup /dev/loop$d $targetdir/mdtest$d
- fi
- eval dev$d=/dev/loop$d
- eval file$d=$targetdir/mdtest$d
- ;;
- lvm)
- unset MULTIPATH
- eval dev$d=/dev/mapper/${LVM_VOLGROUP}-mdtest$d
- if ! lvcreate --quiet -L ${sz}K -n mdtest$d $LVM_VOLGROUP; then
- trap '' 0 # make sure lvremove is not called
- eval echo error creating \$dev$d
- exit 129
- fi
- ;;
- ram)
- unset MULTIPATH
- eval dev$d=/dev/ram$d
- ;;
- esac
- eval devlist=\"\$devlist \$dev$d\"
- eval devlist$d=\"\$devlist\"
- #" <-- add this quote to un-confuse vim syntax highlighting
- done
- path0=$dev6
- path1=$dev7
-
- ulimit -c unlimited
- [ -f /proc/mdstat ] || modprobe md_mod
- echo 2000 > /proc/sys/dev/raid/speed_limit_max
- echo 0 > /sys/module/md_mod/parameters/start_ro
+ # make sure there are no loop devices remaining.
+ # udev started things can sometimes prevent them being stopped
+ # immediately
+ while grep loop /proc/partitions > /dev/null 2>&1
+ do
+ mdadm -Ss
+ losetup -d /dev/loop[0-9]* 2> /dev/null
+ sleep 1
+ done
+ devlist=
+ for d in 0 1 2 3 4 5 6 7 8 9 10 11 12 13
+ do
+ sz=$size
+ if [ $d -gt 7 ]
+ then
+ sz=$ddfsize
+ fi
+ case $DEVTYPE in
+ loop)
+ [ -f $targetdir/mdtest$d ] ||
+ dd if=/dev/zero of=$targetdir/mdtest$d count=$sz bs=1K > /dev/null 2>&1
+ # make sure udev doesn't touch
+ mdadm --zero $targetdir/mdtest$d 2> /dev/null
+ [ -b /dev/loop$d ] || mknod /dev/loop$d b 7 $d
+ if [ $d -eq 7 ]
+ then
+ losetup /dev/loop$d $targetdir/mdtest6 # for multipath use
+ else
+ losetup /dev/loop$d $targetdir/mdtest$d
+ fi
+ eval dev$d=/dev/loop$d
+ eval file$d=$targetdir/mdtest$d
+ ;;
+ lvm)
+ unset MULTIPATH
+ eval dev$d=/dev/mapper/${LVM_VOLGROUP}-mdtest$d
+ if ! lvcreate --quiet -L ${sz}K -n mdtest$d $LVM_VOLGROUP
+ then
+ trap '' 0 # make sure lvremove is not called
+ eval echo error creating \$dev$d
+ exit 129
+ fi
+ ;;
+ ram)
+ unset MULTIPATH
+ eval dev$d=/dev/ram$d
+ ;;
+ esac
+ eval devlist=\"\$devlist \$dev$d\"
+ eval devlist$d=\"\$devlist\"
+ #" <-- add this quote to un-confuse vim syntax highlighting
+ done
+ path0=$dev6
+ path1=$dev7
+ ulimit -c unlimited
+ [ -f /proc/mdstat ] || modprobe md_mod
+ echo 2000 > /proc/sys/dev/raid/speed_limit_max
+ echo 0 > /sys/module/md_mod/parameters/start_ro
}
# mdadm always adds --quiet, and we want to see any unexpected messages
mdadm() {
- rm -f $targetdir/stderr
- case $* in
- *-S* ) udevadm settle
- p=`cat /proc/sys/dev/raid/speed_limit_max`
- echo 20000 > /proc/sys/dev/raid/speed_limit_max
- esac
- case $* in
- *-C* ) $mdadm 2> $targetdir/stderr --quiet "$@" --auto=yes;;
- * ) $mdadm 2> $targetdir/stderr --quiet "$@"
- esac
- rv=$?
- case $* in
- *-S* ) udevadm settle
- echo $p > /proc/sys/dev/raid/speed_limit_max
- esac
- cat >&2 $targetdir/stderr
- return $rv
+ rm -f $targetdir/stderr
+ case $* in
+ *-S* )
+ udevadm settle
+ p=`cat /proc/sys/dev/raid/speed_limit_max`
+ echo 20000 > /proc/sys/dev/raid/speed_limit_max
+ ;;
+ esac
+ case $* in
+ *-C* )
+ $mdadm 2> $targetdir/stderr --quiet "$@" --auto=yes
+ ;;
+ * )
+ $mdadm 2> $targetdir/stderr --quiet "$@"
+ ;;
+ esac
+ rv=$?
+ case $* in
+ *-S* )
+ udevadm settle
+ echo $p > /proc/sys/dev/raid/speed_limit_max
+ ;;
+ esac
+ cat >&2 $targetdir/stderr
+ return $rv
}
# check various things
check() {
- case $1 in
- spares )
- spares=`tr '] ' '\012\012' < /proc/mdstat | grep -c '(S)' || exit 0`
- if [ $spares -ne $2 ]
- then
- echo >&2 "ERROR expected $2 spares, found $spares"; exit 1;
- fi
- ;;
- raid* | linear )
- grep -s "active $1 " /proc/mdstat > /dev/null || {
- echo >&2 "ERROR active $1 not found" ; cat /proc/mdstat ; exit 1;}
- ;;
- algorithm )
- grep -s " algorithm $2 " /proc/mdstat > /dev/null || {
- echo >&2 "ERROR algorithm $2 not found"; cat /proc/mdstat; exit 1;}
- ;;
- resync | recovery | reshape)
- cnt=5
- while ! grep -s $1 /proc/mdstat > /dev/null
- do
- if [ $cnt -gt 0 ] && grep -v idle /sys/block/md*/md/sync_action > /dev/null
- then # Something isn't idle - wait a bit
+ case $1 in
+ spares )
+ spares=`tr '] ' '\012\012' < /proc/mdstat | grep -c '(S)' || exit 0`
+ if [ $spares -ne $2 ]
+ then
+ echo >&2 "ERROR expected $2 spares, found $spares"
+ exit 1
+ fi
+ ;;
+ raid* | linear )
+ grep -s "active $1 " /proc/mdstat > /dev/null || {
+ echo >&2 "ERROR active $1 not found"
+ cat /proc/mdstat
+ exit 1
+ }
+ ;;
+ algorithm )
+ grep -s " algorithm $2 " /proc/mdstat > /dev/null || {
+ echo >&2 "ERROR algorithm $2 not found"
+ cat /proc/mdstat
+ exit 1
+ }
+ ;;
+ resync | recovery | reshape )
+ cnt=5
+ while ! grep -s $1 /proc/mdstat > /dev/null
+ do
+ if [ $cnt -gt 0 ] && grep -v idle /sys/block/md*/md/sync_action > /dev/null
+ then # Something isn't idle - wait a bit
+ sleep 0.5
+ cnt=$[cnt-1]
+ else
+ echo >&2 ERROR no $1 happening
+ cat /proc/mdstat
+ exit 1
+ fi
+ done
+ ;;
+ nosync )
sleep 0.5
- cnt=$[cnt-1]
- else
- echo >&2 ERROR no $1 happening; cat /proc/mdstat; exit 1
- fi
- done
- ;;
-
- nosync )
- sleep 0.5
- # Since 4.2 we delay the close of recovery until there has been a chance for
- # spares to be activated. That means that a recovery that finds nothing
- # to do can still take a little longer than expected.
- # add an extra check: is sync_completed shows the end is reached, assume
- # there is no recovery.
- if grep -s -E '(resync|recovery|reshape) *=' > /dev/null /proc/mdstat ; then
- incomplete=`grep / /sys/block/md*/md/sync_completed 2> /dev/null | sed '/^ *\([0-9]*\) \/ \1/d'`
- if [ -n "$incomplete" ]; then
- echo >&2 "ERROR resync or recovery is happening!"; cat /proc/mdstat ; exit 1;
- fi
- fi
- ;;
-
- wait )
- p=`cat /proc/sys/dev/raid/speed_limit_max`
- echo 2000000 > /proc/sys/dev/raid/speed_limit_max
- sleep 0.1
- while grep -E '(resync|recovery|reshape|check|repair) *=' > /dev/null /proc/mdstat ||
- grep -v idle > /dev/null /sys/block/md*/md/sync_action
- do sleep 0.5;
- done
- echo $p > /proc/sys/dev/raid/speed_limit_max
- ;;
-
- state )
- grep -s "blocks.*\[$2\]\$" /proc/mdstat > /dev/null || {
- echo >&2 "ERROR state $2 not found!"; cat /proc/mdstat ; exit 1; }
- sleep 0.5
- ;;
-
- bitmap )
- grep -s bitmap > /dev/null /proc/mdstat || {
- echo >&2 ERROR no bitmap ; cat /proc/mdstat ; exit 1; }
- ;;
- nobitmap )
- if grep -s "bitmap" > /dev/null /proc/mdstat
- then
- echo >&2 ERROR bitmap present ; cat /proc/mdstat ; exit 1;
- fi
- ;;
-
- readonly )
- grep -s "read-only" > /dev/null /proc/mdstat || {
- echo >&2 "ERROR array is not read-only!"; cat /proc/mdstat ; exit 1; }
- ;;
-
- inactive )
- grep -s "inactive" > /dev/null /proc/mdstat || {
- echo >&2 "ERROR array is not inactive!"; cat /proc/mdstat ; exit 1; }
- ;;
- * ) echo >&2 ERROR unknown check $1 ; exit 1;
- esac
+ # Since 4.2 we delay the close of recovery until there has been a chance for
+ # spares to be activated. That means that a recovery that finds nothing
+ # to do can still take a little longer than expected.
+ # add an extra check: is sync_completed shows the end is reached, assume
+ # there is no recovery.
+ if grep -s -E '(resync|recovery|reshape) *=' > /dev/null /proc/mdstat
+ then
+ incomplete=`grep / /sys/block/md*/md/sync_completed 2> /dev/null | sed '/^ *\([0-9]*\) \/ \1/d'`
+ if [ -n "$incomplete" ]
+ then
+ echo >&2 "ERROR resync or recovery is happening!"
+ cat /proc/mdstat
+ exit 1
+ fi
+ fi
+ ;;
+ wait )
+ p=`cat /proc/sys/dev/raid/speed_limit_max`
+ echo 2000000 > /proc/sys/dev/raid/speed_limit_max
+ sleep 0.1
+ while grep -E '(resync|recovery|reshape|check|repair) *=' > /dev/null /proc/mdstat ||
+ grep -v idle > /dev/null /sys/block/md*/md/sync_action
+ do
+ sleep 0.5
+ done
+ echo $p > /proc/sys/dev/raid/speed_limit_max
+ ;;
+ state )
+ grep -s "blocks.*\[$2\]\$" /proc/mdstat > /dev/null || {
+ echo >&2 "ERROR state $2 not found!"
+ cat /proc/mdstat
+ exit 1
+ }
+ sleep 0.5
+ ;;
+ bitmap )
+ grep -s bitmap > /dev/null /proc/mdstat || {
+ echo >&2 ERROR no bitmap
+ cat /proc/mdstat
+ exit 1
+ }
+ ;;
+ nobitmap )
+ if grep -s "bitmap" > /dev/null /proc/mdstat
+ then
+ echo >&2 ERROR bitmap present
+ cat /proc/mdstat
+ exit 1
+ fi
+ ;;
+ readonly )
+ grep -s "read-only" > /dev/null /proc/mdstat || {
+ echo >&2 "ERROR array is not read-only!"
+ cat /proc/mdstat
+ exit 1
+ }
+ ;;
+ inactive )
+ grep -s "inactive" > /dev/null /proc/mdstat || {
+ echo >&2 "ERROR array is not inactive!"
+ cat /proc/mdstat
+ exit 1
+ }
+ ;;
+ * )
+ echo >&2 ERROR unknown check $1
+ exit 1
+ ;;
+ esac
}
no_errors() {
- if [ -s $targetdir/stderr ]
- then echo Bad errors from mdadm: ; cat $targetdir/stderr; exit 2;
- fi
+ if [ -s $targetdir/stderr ]
+ then
+ echo Bad errors from mdadm:
+ cat $targetdir/stderr
+ exit 2
+ fi
}
-# basic device test
+# basic device test
testdev() {
- udevadm settle
- dev=$1
- cnt=$2
- dvsize=$3
- chunk=$4
- if [ -z "$5" ]; then
- mkfs.ext3 -F -j $dev > /dev/null 2>&1 && fsck -fn $dev >&2
- fi
- dsize=$[dvsize/chunk]
- dsize=$[dsize*chunk]
- rasize=$[dsize*2*cnt]
- # rasize is in sectors
- if [ -n "$DEV_ROUND_K" ]; then
- rasize=$[rasize/DEV_ROUND_K/2]
- rasize=$[rasize*DEV_ROUND_K*2]
- fi
- if [ `/sbin/blockdev --getsize $dev` -eq 0 ]; then sleep 2 ; fi
- _sz=`/sbin/blockdev --getsize $dev`
- if [ $rasize -lt $_sz -o $[rasize*4/5] -gt $_sz ]
- then
- echo "ERROR: size is wrong for $dev: $cnt * $dvsize (chunk=$chunk) = $rasize, not $_sz"
- exit 1
- fi
+ udevadm settle
+ dev=$1
+ cnt=$2
+ dvsize=$3
+ chunk=$4
+ if [ -z "$5" ]
+ then
+ mkfs.ext3 -F -j $dev > /dev/null 2>&1 && fsck -fn $dev >&2
+ fi
+ dsize=$[dvsize/chunk]
+ dsize=$[dsize*chunk]
+ rasize=$[dsize*2*cnt]
+ # rasize is in sectors
+ if [ -n "$DEV_ROUND_K" ]
+ then
+ rasize=$[rasize/DEV_ROUND_K/2]
+ rasize=$[rasize*DEV_ROUND_K*2]
+ fi
+ if [ `/sbin/blockdev --getsize $dev` -eq 0 ]
+ then
+ sleep 2
+ fi
+ _sz=`/sbin/blockdev --getsize $dev`
+ if [ $rasize -lt $_sz -o $[rasize*4/5] -gt $_sz ]
+ then
+ echo "ERROR: size is wrong for $dev: $cnt * $dvsize (chunk=$chunk) = $rasize, not $_sz"
+ exit 1
+ fi
}
fast_sync() {
- echo 200000 > /proc/sys/dev/raid/speed_limit_max
+ echo 200000 > /proc/sys/dev/raid/speed_limit_max
}
rotest() {
- dev=$1
- fsck -fn $dev >&2
+ dev=$1
+ fsck -fn $dev >&2
}
do_test() {
- _script=$1
- _basename=`basename $_script`
- if [ -f "$_script" ]
- then
- rm -f $targetdir/stderr
- # stop all arrays, just incase some script left an array active.
- $mdadm -Ssq 2> /dev/null
- mdadm --zero $devlist 2> /dev/null
- mdadm --zero $devlist 2> /dev/null
- # this might have been reset: restore the default.
- echo 2000 > /proc/sys/dev/raid/speed_limit_max
- # source script in a subshell, so it has access to our
- # namespace, but cannot change it.
- echo -ne "$_script... "
- if ( set -ex ; . $_script ) &> $targetdir/log
- then
- echo "succeeded"
- _fail=0
- else
- log=log
- cat $targetdir/stderr >> $targetdir/log
- echo "=======================dmesg=================" >> $targetdir/log
- dmesg | tail -n 200 >> $targetdir/log
- if [ $exitonerror == 0 ]; then
- log=log-`basename $_script`
- mv $targetdir/log $logdir/$log
- fi
- echo "FAILED - see $logdir/$log for details"
- _fail=1
- fi
- if [ "$savelogs" == "1" ]; then
- cp $targetdir/log $logdir/$_basename.log
- fi
- if [ "$_fail" == "1" -a "$exitonerror" == "1" ]; then
- exit 1
- fi
- fi
+ _script=$1
+ _basename=`basename $_script`
+ if [ -f "$_script" ]
+ then
+ rm -f $targetdir/stderr
+ # stop all arrays, just incase some script left an array active.
+ $mdadm -Ssq 2> /dev/null
+ mdadm --zero $devlist 2> /dev/null
+ mdadm --zero $devlist 2> /dev/null
+ # this might have been reset: restore the default.
+ echo 2000 > /proc/sys/dev/raid/speed_limit_max
+ # source script in a subshell, so it has access to our
+ # namespace, but cannot change it.
+ echo -ne "$_script... "
+ if ( set -ex ; . $_script ) &> $targetdir/log
+ then
+ echo "succeeded"
+ _fail=0
+ else
+ log=log
+ cat $targetdir/stderr >> $targetdir/log
+ echo "=======================dmesg=================" >> $targetdir/log
+ dmesg | tail -n 200 >> $targetdir/log
+ if [ $exitonerror == 0 ]; then
+ log=log-`basename $_script`
+ mv $targetdir/log $logdir/$log
+ fi
+ echo "FAILED - see $logdir/$log for details"
+ _fail=1
+ fi
+ if [ "$savelogs" == "1" ]
+ then
+ cp $targetdir/log $logdir/$_basename.log
+ fi
+ if [ "$_fail" == "1" -a "$exitonerror" == "1" ]
+ then
+ exit 1
+ fi
+ fi
}
do_help() {
- echo "Usage: $0 [options]"
- echo " Options:"
- echo " --tests=<test1,test2,..> Comma separated list of tests to run"
- echo " --disable-multipath Disable any tests involving multipath"
- echo " --disable-integrity Disable slow tests of RAID[56] consistency"
- echo " --logdir=<directory> Directory to save logfiles in"
- echo " --save-logs Save all logs in <logdir>"
- echo " --keep-going Don't stop on error, ie. run all tests"
- echo " --dev=[loop|lvm|ram] Use loop devices (default), LVM, or RAM disk"
- echo " --volgroup=<name> LVM volume group for LVM test"
- echo " setup Setup test environment and exit"
- echo " cleanup Cleanup test environment"
- echo " <prefix> Run tests with <prefix>"
+ echo "Usage: $0 [options]"
+ echo " Options:"
+ echo " --tests=<test1,test2,..> Comma separated list of tests to run"
+ echo " --disable-multipath Disable any tests involving multipath"
+ echo " --disable-integrity Disable slow tests of RAID[56] consistency"
+ echo " --logdir=<directory> Directory to save logfiles in"
+ echo " --save-logs Save all logs in <logdir>"
+ echo " --keep-going Don't stop on error, ie. run all tests"
+ echo " --dev=[loop|lvm|ram] Use loop devices (default), LVM, or RAM disk"
+ echo " --volgroup=<name> LVM volume group for LVM test"
+ echo " setup Setup test environment and exit"
+ echo " cleanup Cleanup test environment"
+ echo " <prefix> Run tests with <prefix>"
}
parse_args() {
- for i in $*
- do
- case $i in
- [0-9]*)
- prefix=$i
- ;;
- setup)
- echo "mdadm test environment setup"
- do_setup
- trap 0; exit 0
- ;;
- cleanup)
- cleanup
- exit 0
- ;;
- --tests=*)
- TESTLIST=`expr "x$i" : 'x[^=]*=\(.*\)' | sed -e 's/,/ /g'`
- ;;
- --logdir=*)
- logdir=`expr "x$i" : 'x[^=]*=\(.*\)'`
- ;;
- --save-logs)
- savelogs=1
- ;;
- --keep-going | --no-error)
- exitonerror=0
- ;;
- --disable-multipath)
- unset MULTIPATH
- ;;
- --disable-integrity)
- unset INTEGRITY
- ;;
- --dev=loop)
- DEVTYPE=loop
- ;;
- --dev=lvm)
- DEVTYPE=lvm
- ;;
- --dev=ram)
- DEVTYPE=ram
- ;;
- --volgroup=*)
- LVM_VOLGROUP=`expr "x$i" : 'x[^=]*=\(.*\)'`
- ;;
- --help)
- do_help
- exit 0;
- ;;
- -*)
- echo " $0: Unknown argument: $i"
- do_help
- exit 0;
- ;;
- esac
-done
+ for i in $*
+ do
+ case $i in
+ [0-9]* )
+ prefix=$i
+ ;;
+ setup )
+ echo "mdadm test environment setup"
+ do_setup
+ trap 0
+ exit 0
+ ;;
+ cleanup )
+ cleanup
+ exit 0
+ ;;
+ --tests=* )
+ TESTLIST=`expr "x$i" : 'x[^=]*=\(.*\)' | sed -e 's/,/ /g'`
+ ;;
+ --logdir=* )
+ logdir=`expr "x$i" : 'x[^=]*=\(.*\)'`
+ ;;
+ --save-logs )
+ savelogs=1
+ ;;
+ --keep-going | --no-error )
+ exitonerror=0
+ ;;
+ --disable-multipath )
+ unset MULTIPATH
+ ;;
+ --disable-integrity )
+ unset INTEGRITY
+ ;;
+ --dev=loop )
+ DEVTYPE=loop
+ ;;
+ --dev=lvm )
+ DEVTYPE=lvm
+ ;;
+ --dev=ram )
+ DEVTYPE=ram
+ ;;
+ --volgroup=* )
+ LVM_VOLGROUP=`expr "x$i" : 'x[^=]*=\(.*\)'`
+ ;;
+ --help )
+ do_help
+ exit 0
+ ;;
+ -* )
+ echo " $0: Unknown argument: $i"
+ do_help
+ exit 0
+ ;;
+ esac
+ done
}
logdir=$targetdir
@@ -422,19 +474,21 @@ parse_args $@
do_setup
mkdir -p $logdir
-if [ "$savelogs" == "1" ]; then
- echo "Saving logs to $logdir"
+if [ "$savelogs" == "1" ]
+then
+ echo "Saving logs to $logdir"
fi
-if [ "x$TESTLIST" != "x" ]; then
- for script in $TESTLIST
- do
- do_test $testdir/$script
- done
+if [ "x$TESTLIST" != "x" ]
+then
+ for script in $TESTLIST
+ do
+ do_test $testdir/$script
+ done
else
- for script in $testdir/$prefix $testdir/$prefix*[^~]
- do
- do_test $script
- done
+ for script in $testdir/$prefix $testdir/$prefix*[^~]
+ do
+ do_test $script
+ done
fi
exit 0
--
2.6.6
^ permalink raw reply related
* [PATCH 0/3] mdadm/test: Refactor the codes of 'test' script
From: Zhilong Liu @ 2017-06-07 9:31 UTC (permalink / raw)
To: Jes.Sorensen; +Cc: linux-raid, Zhilong Liu
Hi, Jes;
This patchset mainly focus on improving the 'test' script, rip it
apart and put it together in a more coherent form.
Over 3 months passed, the RFC patch of improving 'test' hadn't received
any ideas, thus I post this sample version that I have tested many times.
It's just the beginning to improve the test part of mdadm, any questions
is very welcome.
Sorry for that, I haven't added the function suggested from Guoqing, cut
the words from the email tree of
"ANNOUNCE: mdadm 4.0 - A tool for managing md Soft RAID under Linux":
Guoqing asked:
...
Is current test can run the test for different raid level, say,
"./test --raidtype=raid1"
could execute all the *r1* tests, does it make sense to do it if we don't
support it now.
...
Just list this suggestion here, waiting for ideas.
RFC patch mail-tree:
[PATCH RFC] test: revise 'test' and make it easier to understand
Acked-by: Neil Brown
And suggest to split those changing into smaller patches from Guoqing.
Thanks very much,
Zhilong
Zhilong Liu (3):
mdadm/test: Convert code format to use Tab
mdadm/test: Refactor the codes of 'test' script
mdadm/test: Get rid of fast_sync()
test | 803 +++++++++++++++++++++++++++++++++++++------------------------------
1 file changed, 446 insertions(+), 357 deletions(-)
--
2.6.6
^ permalink raw reply
* Re: 9 second recovery when re-adding a drive that got kicked out?
From: Marc MERLIN @ 2017-06-07 3:03 UTC (permalink / raw)
To: NeilBrown, Phil Turmel; +Cc: linux-raid@vger.kernel.org
In-Reply-To: <87r2yxdazc.fsf@notabene.neil.brown.name>
On Tue, Jun 06, 2017 at 01:57:27PM +1000, NeilBrown wrote:
> > The first bitmap line shows 3 pages totallying 12KB, so each page
> > contains 4KB, or 2048 chunks per page.
> > Did the above say that I had 6144 chunks that needed to be synced?
>
> No. It said that of the 44 pages of space that might be needed to store
> 16-bit counters that each represent 1 bitmap-chunk, only 3 of those
> pages would contain non-zero counters, so only 3 had been allocated.
>
> There could be as few as 3 chunks that need to be recovered, or there
> could be as many a 3*2048 chunks, or any number in between.
Ah, I see. I wasn't clear about that part, thanks.
> Had you run "mdadm --examine-bitmap /dev/sdk1" before the re-add, it
> would have told you how many bits were set at that time.
Noted for next time, thanks.
> > The part I'm not too clear about is 44 pages of intent isn't enough to
> > cover all my data.
>
> 44 pages means 90112 16 bit counters, one for each 64M on each device.
> 90112 * 64M = 5632 GiB or 5905GB.
> That is the size of each device.
Ah, so it's not based on the 512k chunk size, I see.
> One bit in the bitmap (one counter in the internal bitmap) corresponds
> to "a set of data the might be out of sync" which, in your case, is a
> 64MB wide stripe across all devices.
I got that part now, just now where the 64MB came from, or how I ended
up with 44 pages of intent maps if it's not based on my chunk size.
Thanks,
Marc
--
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
Microsoft is to operating systems ....
.... what McDonalds is to gourmet cooking
Home page: http://marc.merlins.org/ | PGP 1024R/763BE901
^ permalink raw reply
* Re: [PATCH] mdadm: Send version text to stdout rather than stderr
From: Zhilong Liu @ 2017-06-07 2:11 UTC (permalink / raw)
To: NeilBrown, Jes.Sorensen; +Cc: linux-raid
In-Reply-To: <877f0od124.fsf@notabene.neil.brown.name>
On 06/07/2017 09:44 AM, NeilBrown wrote:
> On Tue, Jun 06 2017, Zhilong Liu wrote:
>
>> This helps with piping to a paper.
> It might help you, but it is an API change and could cause problems for
> others.
>
> Some programs, such as "mount -V", send version info to stdout.
> Others, like "gcc -v", send it to stderr.
> There is no uniform standard, it is best to leave the behavior
> unchanged.
>
> If you need to capture the output use
> mdadm -V |& ...
> or
> mdadm -V 2>&1
>
> or similar.
Thanks for your detail explanations. Yes, I just encountered this issue
in my progress of improving ./mdadm/test script,
the ./mdadm --version always prints to stderr, thus I post this email,
and I have already used "2>&1" in my test script.
Thanks very much,
-Zhilong
> NeilBrtown
>
>> Signed-off-by: Zhilong Liu <zlliu@suse.com>
>> ---
>> mdadm.c | 2 +-
>> 1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/mdadm.c b/mdadm.c
>> index 70b16f2..0f736d7 100644
>> --- a/mdadm.c
>> +++ b/mdadm.c
>> @@ -139,7 +139,7 @@ int main(int argc, char *argv[])
>> continue;
>>
>> case 'V':
>> - fputs(Version, stderr);
>> + fputs(Version, stdout);
>> exit(0);
>>
>> case 'v': c.verbose++;
>> --
>> 2.6.6
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: mdadm: Apply to revert the commit of a4dcdb23
From: Zhilong Liu @ 2017-06-07 2:03 UTC (permalink / raw)
To: Jes Sorensen; +Cc: linux-raid@vger.kernel.org
In-Reply-To: <0d1a79b0-a7fd-656d-5cfd-a85b4398c284@gmail.com>
On 06/07/2017 12:02 AM, Jes Sorensen wrote:
> On 06/06/2017 05:25 AM, Zhilong Liu wrote:
>> Hi, Jes;
>> Excuse me, there is one problematic commit for Detail.c,
>> a4dcdb23(Detail: determine array state from sysfs).
>> This commit does change the codes meaning and it didn't
>> run to the right path. This commit was result in a bug that
>> Detail() cannot work normally.
>>
>> For example:
>> # ./mdadm -CR /dev/md0 -l1 -n2 /dev/loop[0-1] --assume-clean
>> # ./mdadm -D /dev/md0
>> mdadm: cannot get array detail for /dev/md0: No such file or directory
>>
>> Thanks very much,
>> -Zhilong
>
> Please be more specific and explain what you believe is wrong. I
> prefer a patch fixing the issue to just reverting things.
>
I just re-checked the mailing-list, found the email that post the commit
of 9b8fea914f82
from Tomasz in May.24th, it has already fixed this issue. please ignore
this mail. Sorry
for that.
Thanks,
-Zhilong
> Jes
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
^ permalink raw reply
* Re: [PATCH] md: fix deadlock between mddev_suspend() and md_write_start()
From: NeilBrown @ 2017-06-07 1:45 UTC (permalink / raw)
To: Shaohua Li; +Cc: Nix, linux-raid
In-Reply-To: <20170606000115.jijxe6xvkclm4tsb@kernel.org>
[-- Attachment #1: Type: text/plain, Size: 1201 bytes --]
On Mon, Jun 05 2017, Shaohua Li wrote:
> On Mon, Jun 05, 2017 at 04:49:39PM +1000, Neil Brown wrote:
>>
>> If mddev_suspend() races with md_write_start() we can deadlock
>> with mddev_suspend() waiting for the request that is currently
>> in md_write_start() to complete the ->make_request() call,
>> and md_write_start() waiting for the metadata to be updated
>> to mark the array as 'dirty'.
>> As metadata updates done by md_check_recovery() only happen then
>> the mddev_lock() can be claimed, and as mddev_suspend() is often
>> called with the lock held, these threads wait indefinitely for each
>> other.
>>
>> We fix this by having md_write_start() abort if mddev_suspend()
>> is happening, and ->make_request() aborts if md_write_start()
>> aborted.
>> md_make_request() can detect this abort, decrease the ->active_io
>> count, and wait for mddev_suspend().
>
> Applied the two patches, thanks! For this one, I added md_start_write symbol back. Will
> push these to Linus in the 4.12 cycle.
>
Thanks for fixing the md_start_write export! At one stage in
development I made it "static", but then forgot the export when I put it
back :-(
Thanks,
NeilBrown
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]
^ permalink raw reply
* Re: [PATCH] mdadm: Send version text to stdout rather than stderr
From: NeilBrown @ 2017-06-07 1:44 UTC (permalink / raw)
To: Jes.Sorensen; +Cc: linux-raid, Zhilong Liu
In-Reply-To: <1496735987-10653-1-git-send-email-zlliu@suse.com>
[-- Attachment #1: Type: text/plain, Size: 1126 bytes --]
On Tue, Jun 06 2017, Zhilong Liu wrote:
> This helps with piping to a paper.
It might help you, but it is an API change and could cause problems for
others.
Some programs, such as "mount -V", send version info to stdout.
Others, like "gcc -v", send it to stderr.
There is no uniform standard, it is best to leave the behavior
unchanged.
If you need to capture the output use
mdadm -V |& ...
or
mdadm -V 2>&1
or similar.
NeilBrtown
>
> Signed-off-by: Zhilong Liu <zlliu@suse.com>
> ---
> mdadm.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/mdadm.c b/mdadm.c
> index 70b16f2..0f736d7 100644
> --- a/mdadm.c
> +++ b/mdadm.c
> @@ -139,7 +139,7 @@ int main(int argc, char *argv[])
> continue;
>
> case 'V':
> - fputs(Version, stderr);
> + fputs(Version, stdout);
> exit(0);
>
> case 'v': c.verbose++;
> --
> 2.6.6
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]
^ permalink raw reply
* (unknown),
From: mdavis @ 2017-06-06 23:46 UTC (permalink / raw)
To: linux-raid
[-- Attachment #1: 37913653393087.zip --]
[-- Type: application/zip, Size: 4706 bytes --]
^ permalink raw reply
* Re: internal write-intent bitmap is horribly slow with RAID10 over 20 drives
From: NeilBrown @ 2017-06-06 22:02 UTC (permalink / raw)
To: CoolCold; +Cc: Linux RAID
In-Reply-To: <CAGqmV7qVq8nBmop5h7OZKr9ZjCjGLcoPqMF5xa4d2iHzf2uQdw@mail.gmail.com>
[-- Attachment #1: Type: text/plain, Size: 3630 bytes --]
On Tue, Jun 06 2017, CoolCold wrote:
> Hello!
> Neil, thanks for reply, further inline
>
> On Tue, Jun 6, 2017 at 10:40 AM, NeilBrown <neilb@suse.com> wrote:
>> On Mon, Jun 05 2017, CoolCold wrote:
>>
>>> Hello!
>>> Keep testing the new box and while having not the best sync speed,
>>> it's not the worst thing I found.
>>>
>>> Doing FIO testing, for RAID10 over 20 10k RPM drives, I have very bad
>>> performance, like _45_ iops only.
>>
>> ...
>>>
>>>
>>> Output from fio with internal write-intent bitmap:
>>> Jobs: 1 (f=1): [w(1)] [28.3% done] [0KB/183KB/0KB /s] [0/45/0 iops]
>>> [eta 07m:11s]
>>>
>>> array definition:
>>> [root@spare-a17484327407661 rovchinnikov]# cat /proc/mdstat
>>> Personalities : [raid1] [raid10] [raid6] [raid5] [raid4]
>>> md1 : active raid10 sdx[19] sdw[18] sdv[17] sdu[16] sdt[15] sds[14]
>>> sdr[13] sdq[12] sdp[11] sdo[10] sdn[9] sdm[8] sdl[7] sdk[6] sdj[5]
>>> sdi[4] sdh[3] sdg[2] sdf[1] sde[0]
>>> 17580330880 blocks super 1.2 64K chunks 2 near-copies [20/20]
>>> [UUUUUUUUUUUUUUUUUUUU]
>>> bitmap: 0/66 pages [0KB], 131072KB chunk
>>>
>>> Setting journal to be
>>> 1) on SSD (separate drives), shows
>>> Jobs: 1 (f=1): [w(1)] [5.0% done] [0KB/18783KB/0KB /s] [0/4695/0 iops]
>>> [eta 09m:31s]
>>> 2) to 'none' (disabling) shows
>>> Jobs: 1 (f=1): [w(1)] [14.0% done] [0KB/18504KB/0KB /s] [0/4626/0
>>> iops] [eta 08m:36s]
>>
>> These numbers suggest that the write intent bitmap causes a 100-fold slow
>> down.
>> i.e. 45 iops instead of 4500 iops (roughly).
>>
>> That is certainly more than I would expect, so maybe there is a bug.
> I suppose noone is using raid10 over more than 4 drives then, i can't
> believe i'm the one who hit this problem.
We have customers who use RAID10 will many more than 4 drives, but I
haven't had reports like this. Presumably whatever problem is affecting
you is not affecting them. We cannot know until we drill down to
understand the problem.
>
>>
>> Large RAID10 is a worst-base for bitmap updates as the bitmap is written
>> to all devices instead of just those devices that contain the data which
>> the bit corresponds to. So every bitmap update goes to 10 device.
>>
>> Your bitmap chunk size of 128M is nice and large, but making it larger
>> might help - maybe 1GB.
> Tried that already, wasn't any much difference, but will gather more statistics.
>
>>
>> Still 100-fold ... that's a lot..
>>
>> A potentially useful exercise would be to run a series of tests,
>> changing the number of devices in the array from 2 to 10, changing the
>> RAID chunk size from 64K to 64M, and changing the bitmap chunk size from
>> 64M to 4G.
> Changing chunk size to up to 64M just to gather statistics or you
> suppose it may be some practical usage for this?
I don't have any particular reason to expect this to have an effect.
But it is easy to change, and changing it might show provide some hints.
So probably "just to gather statistics".
NeilBrown
>> In each configuration, run the same test and record the iops.
>> (You don't need to wait for a resync each time, just use
>> --assume-clean).
> This helps, thanks
>> Then graph all this data (or just provide the table and I'll graph it).
>> That might provide an insight into where to start looking for the
>> slowdown.
>>
>> NeilBrown
>
>
>
> --
> Best regards,
> [COOLCOLD-RIPN]
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]
^ permalink raw reply
* Re: mdadm: Apply to revert the commit of a4dcdb23
From: Jes Sorensen @ 2017-06-06 16:02 UTC (permalink / raw)
To: Zhilong Liu; +Cc: linux-raid@vger.kernel.org
In-Reply-To: <0010fa6f-f227-cdd8-31ef-7073ec46546a@suse.com>
On 06/06/2017 05:25 AM, Zhilong Liu wrote:
> Hi, Jes;
> Excuse me, there is one problematic commit for Detail.c,
> a4dcdb23(Detail: determine array state from sysfs).
> This commit does change the codes meaning and it didn't
> run to the right path. This commit was result in a bug that
> Detail() cannot work normally.
>
> For example:
> # ./mdadm -CR /dev/md0 -l1 -n2 /dev/loop[0-1] --assume-clean
> # ./mdadm -D /dev/md0
> mdadm: cannot get array detail for /dev/md0: No such file or directory
>
> Thanks very much,
> -Zhilong
Please be more specific and explain what you believe is wrong. I prefer
a patch fixing the issue to just reverting things.
Jes
^ permalink raw reply
* Re: [PATCH] Grow: set component size prior to array size
From: Jes Sorensen @ 2017-06-06 16:00 UTC (permalink / raw)
To: Tomasz Majchrzak; +Cc: linux-raid
In-Reply-To: <20170606144732.GA1070@proton.igk.intel.com>
On 06/06/2017 10:47 AM, Tomasz Majchrzak wrote:
> On Mon, Jun 05, 2017 at 11:06:48AM -0400, Jes Sorensen wrote:
>> On 06/05/2017 10:09 AM, Tomasz Majchrzak wrote:
>>> It is a partial revert of commit 758b327cf5a7 ("Grow: Remove unnecessary
>>> optimization"). For native metadata component size is set in kernel for
>>> entire disk space. As external metadata supports multiple arrays within
>>> one disk, the component size is set to array size. If component size is
>>> not updated prior to array size update, the grow operation fails.
>>>
>>> Signed-off-by: Tomasz Majchrzak <tomasz.majchrzak@intel.com>
>>> ---
>>> Grow.c | 2 ++
>>> 1 file changed, 2 insertions(+)
>>
>> Applied!
>>
>> Thanks,
>> Jes
>>
>
> Would you mind pushing it to the repository? I'd like to have it ported into
> certain Linux distribution.
Only because it's you :)
Done!
Jes
^ permalink raw reply
* Re: [PATCH] Grow: set component size prior to array size
From: Tomasz Majchrzak @ 2017-06-06 14:47 UTC (permalink / raw)
To: Jes Sorensen; +Cc: linux-raid
In-Reply-To: <611d0b8e-f42f-48de-c881-bd55339d434c@gmail.com>
On Mon, Jun 05, 2017 at 11:06:48AM -0400, Jes Sorensen wrote:
> On 06/05/2017 10:09 AM, Tomasz Majchrzak wrote:
> >It is a partial revert of commit 758b327cf5a7 ("Grow: Remove unnecessary
> >optimization"). For native metadata component size is set in kernel for
> >entire disk space. As external metadata supports multiple arrays within
> >one disk, the component size is set to array size. If component size is
> >not updated prior to array size update, the grow operation fails.
> >
> >Signed-off-by: Tomasz Majchrzak <tomasz.majchrzak@intel.com>
> >---
> > Grow.c | 2 ++
> > 1 file changed, 2 insertions(+)
>
> Applied!
>
> Thanks,
> Jes
>
Would you mind pushing it to the repository? I'd like to have it ported into
certain Linux distribution.
Tomek
^ permalink raw reply
* Re: internal write-intent bitmap is horribly slow with RAID10 over 20 drives
From: CoolCold @ 2017-06-06 11:31 UTC (permalink / raw)
To: NeilBrown; +Cc: Linux RAID
In-Reply-To: <87tw3tdbs1.fsf@notabene.neil.brown.name>
Hello!
Neil, thanks for reply, further inline
On Tue, Jun 6, 2017 at 10:40 AM, NeilBrown <neilb@suse.com> wrote:
> On Mon, Jun 05 2017, CoolCold wrote:
>
>> Hello!
>> Keep testing the new box and while having not the best sync speed,
>> it's not the worst thing I found.
>>
>> Doing FIO testing, for RAID10 over 20 10k RPM drives, I have very bad
>> performance, like _45_ iops only.
>
> ...
>>
>>
>> Output from fio with internal write-intent bitmap:
>> Jobs: 1 (f=1): [w(1)] [28.3% done] [0KB/183KB/0KB /s] [0/45/0 iops]
>> [eta 07m:11s]
>>
>> array definition:
>> [root@spare-a17484327407661 rovchinnikov]# cat /proc/mdstat
>> Personalities : [raid1] [raid10] [raid6] [raid5] [raid4]
>> md1 : active raid10 sdx[19] sdw[18] sdv[17] sdu[16] sdt[15] sds[14]
>> sdr[13] sdq[12] sdp[11] sdo[10] sdn[9] sdm[8] sdl[7] sdk[6] sdj[5]
>> sdi[4] sdh[3] sdg[2] sdf[1] sde[0]
>> 17580330880 blocks super 1.2 64K chunks 2 near-copies [20/20]
>> [UUUUUUUUUUUUUUUUUUUU]
>> bitmap: 0/66 pages [0KB], 131072KB chunk
>>
>> Setting journal to be
>> 1) on SSD (separate drives), shows
>> Jobs: 1 (f=1): [w(1)] [5.0% done] [0KB/18783KB/0KB /s] [0/4695/0 iops]
>> [eta 09m:31s]
>> 2) to 'none' (disabling) shows
>> Jobs: 1 (f=1): [w(1)] [14.0% done] [0KB/18504KB/0KB /s] [0/4626/0
>> iops] [eta 08m:36s]
>
> These numbers suggest that the write intent bitmap causes a 100-fold slow
> down.
> i.e. 45 iops instead of 4500 iops (roughly).
>
> That is certainly more than I would expect, so maybe there is a bug.
I suppose noone is using raid10 over more than 4 drives then, i can't
believe i'm the one who hit this problem.
>
> Large RAID10 is a worst-base for bitmap updates as the bitmap is written
> to all devices instead of just those devices that contain the data which
> the bit corresponds to. So every bitmap update goes to 10 device.
>
> Your bitmap chunk size of 128M is nice and large, but making it larger
> might help - maybe 1GB.
Tried that already, wasn't any much difference, but will gather more statistics.
>
> Still 100-fold ... that's a lot..
>
> A potentially useful exercise would be to run a series of tests,
> changing the number of devices in the array from 2 to 10, changing the
> RAID chunk size from 64K to 64M, and changing the bitmap chunk size from
> 64M to 4G.
Changing chunk size to up to 64M just to gather statistics or you
suppose it may be some practical usage for this?
> In each configuration, run the same test and record the iops.
> (You don't need to wait for a resync each time, just use
> --assume-clean).
This helps, thanks
> Then graph all this data (or just provide the table and I'll graph it).
> That might provide an insight into where to start looking for the
> slowdown.
>
> NeilBrown
--
Best regards,
[COOLCOLD-RIPN]
^ permalink raw reply
* mdadm: Apply to revert the commit of a4dcdb23
From: Zhilong Liu @ 2017-06-06 9:25 UTC (permalink / raw)
To: Jes Sorensen; +Cc: linux-raid@vger.kernel.org
Hi, Jes;
Excuse me, there is one problematic commit for Detail.c,
a4dcdb23(Detail: determine array state from sysfs).
This commit does change the codes meaning and it didn't
run to the right path. This commit was result in a bug that
Detail() cannot work normally.
For example:
# ./mdadm -CR /dev/md0 -l1 -n2 /dev/loop[0-1] --assume-clean
# ./mdadm -D /dev/md0
mdadm: cannot get array detail for /dev/md0: No such file or directory
Thanks very much,
-Zhilong
^ permalink raw reply
* [PATCH] mdadm: Send version text to stdout rather than stderr
From: Zhilong Liu @ 2017-06-06 7:59 UTC (permalink / raw)
To: Jes.Sorensen; +Cc: linux-raid, Zhilong Liu
This helps with piping to a paper.
Signed-off-by: Zhilong Liu <zlliu@suse.com>
---
mdadm.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/mdadm.c b/mdadm.c
index 70b16f2..0f736d7 100644
--- a/mdadm.c
+++ b/mdadm.c
@@ -139,7 +139,7 @@ int main(int argc, char *argv[])
continue;
case 'V':
- fputs(Version, stderr);
+ fputs(Version, stdout);
exit(0);
case 'v': c.verbose++;
--
2.6.6
^ permalink raw reply related
* Re: 9 second recovery when re-adding a drive that got kicked out?
From: NeilBrown @ 2017-06-06 3:57 UTC (permalink / raw)
To: Marc MERLIN, linux-raid@vger.kernel.org
In-Reply-To: <20170604223817.ntxlvafgfa2umww2@merlins.org>
[-- Attachment #1: Type: text/plain, Size: 4975 bytes --]
On Sun, Jun 04 2017, Marc MERLIN wrote:
> Howdy,
>
> Can you confirm that I understand how the write intent bitmap works, and
> that it doesn't cover the entire array, but only a part of it, and once
> you overflow it, syncing reverts to syncing the entire array?
>
> I had a raid5 array with 5 6TB drives.
>
> /dev/sdl1 got kicked out due to a bus disk error of some kind.
> The drive is fine, it was a cabling issue, so I fixed the cabling,
> re-added it, and did
>
> gargamel:~# mdadm -a /dev/md6 /dev/sdl1
>
> Then I saw this:
> [ 1001.728134] md: recovery of RAID array md6
> [ 1010.975255] md: md6: recovery done.
>
> Before the re-add:
> md6 : active raid5 sdk1[5] sdb1[3] sdm1[2] sdj1[1]
> 23441555456 blocks super 1.2 level 5, 512k chunk, algorithm 2 [5/4] [_UUUU]
> bitmap: 3/44 pages [12KB], 65536KB chunk
>
> After the re-add (syncing now just to be safe):
> md6 : active raid5 sdl1[0] sdj1[1] sdk1[5] sdf1[3] sdm1[2]
> 23441555456 blocks super 1.2 level 5, 512k chunk, algorithm 2 [5/5] [UUUUU]
> [>....................] check = 0.8% (49258960/5860388864) finish=569.3min speed=170093K/sec
> bitmap: 0/44 pages [0KB], 65536KB chunk
>
> https://raid.wiki.kernel.org/index.php/Mdstat
> Explains a bit, I don't think it says how big a page is, but it seems to
> be 4KB.
>
> So let's say I have 64MB chuncks, each take 16 bits.
> The whole array is 22,892,144MiB
> That's 357,689 chunks, or about 700KB (16 bits per chunk) to keep all the
> state, but there is 44 pages of 4KB, or 176KB of write intent
> state.
>
> The first bitmap line shows 3 pages totallying 12KB, so each page
> contains 4KB, or 2048 chunks per page.
> Did the above say that I had 6144 chunks that needed to be synced?
No. It said that of the 44 pages of space that might be needed to store
16-bit counters that each represent 1 bitmap-chunk, only 3 of those
pages would contain non-zero counters, so only 3 had been allocated.
There could be as few as 3 chunks that need to be recovered, or there
could be as many a 3*2048 chunks, or any number in between.
Had you run "mdadm --examine-bitmap /dev/sdk1" before the re-add, it
would have told you how many bits were set at that time.
That "x/y pages" information never should have appeared in /proc/mdstat
- it is really just of interest to developers. But it is there now, so
removing it is awkward.
>
> If so it would be 6144 * 65536KB = 393,216 MB to write
> They were written in 9 seconds, so the sync happened at 43MB/s, which is
> believeable.
>
> The part I'm not too clear about is 44 pages of intent isn't enough to
> cover all my data.
44 pages means 90112 16 bit counters, one for each 64M on each device.
90112 * 64M = 5632 GiB or 5905GB.
That is the size of each device.
One bit in the bitmap (one counter in the internal bitmap) corresponds
to "a set of data the might be out of sync" which, in your case, is a
64MB wide stripe across all devices.
So the numbers do add up.
NeilBrown
> Is the idea that once I overflow that write intent bitmap, then it
> reverts to resyncing the entire array?
>
> I looked at https://raid.wiki.kernel.org/index.php/Write-intent_bitmap
> but didn't see anything about that specific bit.
>
>
> Array details if that helps:
> gargamel:~# mdadm --examine /dev/sdl1
> /dev/sdl1:
> Magic : a92b4efc
> Version : 1.2
> Feature Map : 0x1
> Array UUID : 66bccdfb:afbf9683:fcf1f12e:f2af2dcb
> Name : gargamel.svh.merlins.org:6 (local to host gargamel.svh.merlins.org)
> Creation Time : Thu Jan 28 14:38:40 2016
> Raid Level : raid5
> Raid Devices : 5
>
> Avail Dev Size : 11720777728 (5588.90 GiB 6001.04 GB)
> Array Size : 23441555456 (22355.61 GiB 24004.15 GB)
> Data Offset : 262144 sectors
> Super Offset : 8 sectors
> Unused Space : before=262056 sectors, after=0 sectors
> State : clean
> Device UUID : ca4598ba:de585baa:b9935222:e06ac97d
>
> Internal Bitmap : 8 sectors from superblock
> Update Time : Sun Jun 4 15:08:45 2017
> Bad Block Log : 512 entries available at offset 72 sectors
> Checksum : d645f600 - correct
> Events : 84917
>
> Layout : left-symmetric
> Chunk Size : 512K
>
> Device Role : Active device 0
> Array State : AAAAA ('A' == active, '.' == missing, 'R' == replacing)
>
> Thanks,
> Marc
> --
> "A mouse is a device used to point at the xterm you want to type in" - A.S.R.
> Microsoft is to operating systems ....
> .... what McDonalds is to gourmet cooking
> Home page: http://marc.merlins.org/ | PGP 1024R/763BE901
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]
^ permalink raw reply
* Re: internal write-intent bitmap is horribly slow with RAID10 over 20 drives
From: NeilBrown @ 2017-06-06 3:40 UTC (permalink / raw)
To: CoolCold, Linux RAID
In-Reply-To: <CAGqmV7qRh+_BXAr-qf6vUkiZyzi69E8CiX_vA92tZV2LfDxq_A@mail.gmail.com>
[-- Attachment #1: Type: text/plain, Size: 2205 bytes --]
On Mon, Jun 05 2017, CoolCold wrote:
> Hello!
> Keep testing the new box and while having not the best sync speed,
> it's not the worst thing I found.
>
> Doing FIO testing, for RAID10 over 20 10k RPM drives, I have very bad
> performance, like _45_ iops only.
...
>
>
> Output from fio with internal write-intent bitmap:
> Jobs: 1 (f=1): [w(1)] [28.3% done] [0KB/183KB/0KB /s] [0/45/0 iops]
> [eta 07m:11s]
>
> array definition:
> [root@spare-a17484327407661 rovchinnikov]# cat /proc/mdstat
> Personalities : [raid1] [raid10] [raid6] [raid5] [raid4]
> md1 : active raid10 sdx[19] sdw[18] sdv[17] sdu[16] sdt[15] sds[14]
> sdr[13] sdq[12] sdp[11] sdo[10] sdn[9] sdm[8] sdl[7] sdk[6] sdj[5]
> sdi[4] sdh[3] sdg[2] sdf[1] sde[0]
> 17580330880 blocks super 1.2 64K chunks 2 near-copies [20/20]
> [UUUUUUUUUUUUUUUUUUUU]
> bitmap: 0/66 pages [0KB], 131072KB chunk
>
> Setting journal to be
> 1) on SSD (separate drives), shows
> Jobs: 1 (f=1): [w(1)] [5.0% done] [0KB/18783KB/0KB /s] [0/4695/0 iops]
> [eta 09m:31s]
> 2) to 'none' (disabling) shows
> Jobs: 1 (f=1): [w(1)] [14.0% done] [0KB/18504KB/0KB /s] [0/4626/0
> iops] [eta 08m:36s]
These numbers suggest that the write intent bitmap causes a 100-fold slow
down.
i.e. 45 iops instead of 4500 iops (roughly).
That is certainly more than I would expect, so maybe there is a bug.
Large RAID10 is a worst-base for bitmap updates as the bitmap is written
to all devices instead of just those devices that contain the data which
the bit corresponds to. So every bitmap update goes to 10 device.
Your bitmap chunk size of 128M is nice and large, but making it larger
might help - maybe 1GB.
Still 100-fold ... that's a lot..
A potentially useful exercise would be to run a series of tests,
changing the number of devices in the array from 2 to 10, changing the
RAID chunk size from 64K to 64M, and changing the bitmap chunk size from
64M to 4G.
In each configuration, run the same test and record the iops.
(You don't need to wait for a resync each time, just use
--assume-clean).
Then graph all this data (or just provide the table and I'll graph it).
That might provide an insight into where to start looking for the
slowdown.
NeilBrown
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]
^ permalink raw reply
* Re: 9 second recovery when re-adding a drive that got kicked out?
From: Phil Turmel @ 2017-06-06 2:58 UTC (permalink / raw)
To: Marc MERLIN, linux-raid@vger.kernel.org
In-Reply-To: <20170604223817.ntxlvafgfa2umww2@merlins.org>
On 06/04/2017 06:38 PM, Marc MERLIN wrote:
> Howdy,
>
> Can you confirm that I understand how the write intent bitmap works, and
> that it doesn't cover the entire array, but only a part of it, and once
> you overflow it, syncing reverts to syncing the entire array?
There's no overflow. The size correlation between bits in the bitmap
and areas in the array is adjusted so the whole array can be
represented. One bit can be many multiples of a page, iirc.
> I had a raid5 array with 5 6TB drives.
>
> /dev/sdl1 got kicked out due to a bus disk error of some kind.
> The drive is fine, it was a cabling issue, so I fixed the cabling,
> re-added it, and did
>
> gargamel:~# mdadm -a /dev/md6 /dev/sdl1
>
> Then I saw this:
> [ 1001.728134] md: recovery of RAID array md6
> [ 1010.975255] md: md6: recovery done.
> So let's say I have 64MB chuncks, each take 16 bits.
> The whole array is 22,892,144MiB
> That's 357,689 chunks, or about 700KB (16 bits per chunk) to keep all the
> state, but there is 44 pages of 4KB, or 176KB of write intent
> state.
So each bit in the bitmap represents two 4k pages.
Nine seconds to re-add after a short disconnect is perfectly normal.
For lightly loaded arrays, it can be virtually instant.
Phil
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox