* Re: [PATCH 1/8] md-cluster: call md_kick_rdev_from_array once ack failed
From: Shaohua Li @ 2016-08-02 22:17 UTC (permalink / raw)
To: Guoqing Jiang; +Cc: linux-raid
In-Reply-To: <579FF771.4060007@suse.com>
On Mon, Aug 01, 2016 at 09:29:21PM -0400, Guoqing Jiang wrote:
>
>
> On 08/01/2016 05:58 PM, Shaohua Li wrote:
> >On Thu, Jul 28, 2016 at 02:16:45AM -0400, Guoqing Jiang wrote:
> >>The new_disk_ack could return failure if WAITING_FOR_NEWDISK
> >>is not set, so we need to kick the dev from array in case
> >>failure happened.
> >>
> >>Reviewed-by: NeilBrown <neilb@suse.com>
> >>Signed-off-by: Guoqing Jiang <gqjiang@suse.com>
> >>---
> >> drivers/md/md.c | 8 +++++---
> >> 1 file changed, 5 insertions(+), 3 deletions(-)
> >>
> >>diff --git a/drivers/md/md.c b/drivers/md/md.c
> >>index 2ed547f..743cd21 100644
> >>--- a/drivers/md/md.c
> >>+++ b/drivers/md/md.c
> >>@@ -6092,9 +6092,11 @@ static int add_new_disk(struct mddev *mddev, mdu_disk_info_t *info)
> >> export_rdev(rdev);
> >> if (mddev_is_clustered(mddev)) {
> >>- if (info->state & (1 << MD_DISK_CANDIDATE))
> >>- md_cluster_ops->new_disk_ack(mddev, (err == 0));
> >>- else {
> >>+ if (info->state & (1 << MD_DISK_CANDIDATE)) {
> >if err != 0, we already do export_rdev, do we need to do md_kick_rdev_from_array in that case?
>
> I suppose you mean the export_rdev before new_disk_ack, it doesn't need to
> call md_kick_rdev_from_array
> since we don't bind rdev to array successfully.
>
> err = bind_rdev_to_array(rdev, mddev);
>
> if (err)
> *export_rdev*(rdev);
Let's assume bind_rdev_to_array fails, err != 0, we export_rdev
> if (mddev_is_clustered(mddev)) {
> if (info->state & (1 << MD_DISK_CANDIDATE))
> md_cluster_ops->new_disk_ack(mddev, (err == 0));
we don't check err here, so we could call ->new_disk_ack, then if
new_disk_ack() fails, we will call md_kick_rdev_from_array(rdev) again with
your patch. we are kicking out rdev which isn't bound yet in this case.
Thanks,
Shaohua
^ permalink raw reply
* Re: [PATCH 0/4] Add AVX512 optimized gen_syndrome and recovery functions
From: Shaohua Li @ 2016-08-02 22:03 UTC (permalink / raw)
To: Kammela, Gayatri; +Cc: linux-raid@vger.kernel.org, linux-kernel@vger.kernel.org
In-Reply-To: <2824B99DC70CE746926327C52CA42F7613649559@ORSMSX111.amr.corp.intel.com>
On Tue, Aug 02, 2016 at 09:08:39PM +0000, Kammela, Gayatri wrote:
> Hi shaohua,
> I am wondering if there are any comments on this patch series ?
I didn't see the patches in my mailbox. please resend.
> Thanks,
> Gayatri Kammela
>
> -----Original Message-----
> From: Kammela, Gayatri
> Sent: Friday, July 8, 2016 12:16 PM
> To: linux-raid@vger.kernel.org
> Cc: shli@kernel.org; linux-kernel@vger.kernel.org; Kammela, Gayatri <gayatri.kammela@intel.com>
> Subject: [PATCH 0/4] Add AVX512 optimized gen_syndrome and recovery functions
>
> This is the patch set for adding AVX512 optimized gen_syndrome and recovery functions.
>
> Optimization of RAID6 using AVX512 instructions should improve the
> RAID6 performance.These patches are tested and observed the improvement in performance.
>
> Gayatri Kammela (4):
> lib/raid6: Add AVX512 optimized gen_syndrome functions
> lib/raid6: Add AVX512 optimized recovery functions
> lib/raid6/test/Makefile: Add avx512 gen_syndrome and recovery
> functions
> lib/raid6: Add unroll by 8 to AVX512 optimized gen_syndrome functions
>
> arch/x86/Makefile | 5 +-
> include/linux/raid/pq.h | 5 +
> lib/raid6/Makefile | 2 +-
> lib/raid6/algos.c | 13 ++
> lib/raid6/avx512.c | 466 +++++++++++++++++++++++++++++++++++++++++++++++
> lib/raid6/recov_avx512.c | 335 ++++++++++++++++++++++++++++++++++
> lib/raid6/test/Makefile | 5 +-
> lib/raid6/x86.h | 10 +
> 8 files changed, 837 insertions(+), 4 deletions(-) create mode 100644 lib/raid6/avx512.c create mode 100644 lib/raid6/recov_avx512.c
>
> --
> 1.9.1
>
^ permalink raw reply
* nopea Lainat
From: varainhoitoelimen @ 2016-08-02 21:52 UTC (permalink / raw)
To: Recipients
Apply for a loan at 2% reply to this Email for more Info
Hae lainaa 2% vastausta tähän viestiin lisätietoja
^ permalink raw reply
* RE: [PATCH 0/4] Add AVX512 optimized gen_syndrome and recovery functions
From: Kammela, Gayatri @ 2016-08-02 21:08 UTC (permalink / raw)
To: linux-raid@vger.kernel.org; +Cc: shli@kernel.org, linux-kernel@vger.kernel.org
In-Reply-To: <1468005353-26194-1-git-send-email-gayatri.kammela@intel.com>
Hi shaohua,
I am wondering if there are any comments on this patch series ?
Thanks,
Gayatri Kammela
-----Original Message-----
From: Kammela, Gayatri
Sent: Friday, July 8, 2016 12:16 PM
To: linux-raid@vger.kernel.org
Cc: shli@kernel.org; linux-kernel@vger.kernel.org; Kammela, Gayatri <gayatri.kammela@intel.com>
Subject: [PATCH 0/4] Add AVX512 optimized gen_syndrome and recovery functions
This is the patch set for adding AVX512 optimized gen_syndrome and recovery functions.
Optimization of RAID6 using AVX512 instructions should improve the
RAID6 performance.These patches are tested and observed the improvement in performance.
Gayatri Kammela (4):
lib/raid6: Add AVX512 optimized gen_syndrome functions
lib/raid6: Add AVX512 optimized recovery functions
lib/raid6/test/Makefile: Add avx512 gen_syndrome and recovery
functions
lib/raid6: Add unroll by 8 to AVX512 optimized gen_syndrome functions
arch/x86/Makefile | 5 +-
include/linux/raid/pq.h | 5 +
lib/raid6/Makefile | 2 +-
lib/raid6/algos.c | 13 ++
lib/raid6/avx512.c | 466 +++++++++++++++++++++++++++++++++++++++++++++++
lib/raid6/recov_avx512.c | 335 ++++++++++++++++++++++++++++++++++
lib/raid6/test/Makefile | 5 +-
lib/raid6/x86.h | 10 +
8 files changed, 837 insertions(+), 4 deletions(-) create mode 100644 lib/raid6/avx512.c create mode 100644 lib/raid6/recov_avx512.c
--
1.9.1
^ permalink raw reply
* My Email Again
From: Paul Karn @ 2016-08-02 18:07 UTC (permalink / raw)
To: Recipients
I am yet to get a respose from you based on your promise
^ permalink raw reply
* Re: [PATCH] mdadm:add 'clustered' in typo prompt when specify wrong param for bitmap
From: Jes Sorensen @ 2016-08-02 14:08 UTC (permalink / raw)
To: Zhilong Liu; +Cc: linux-raid
In-Reply-To: <1470129520-17502-1-git-send-email-zlliu@suse.com>
Zhilong Liu <zlliu@suse.com> writes:
> mdadm: 'clustered' bitmap has already supported, thus add the
> prompt if users specify wrong value for bitmap param.
>
> Signed-off-by: Zhilong Liu <zlliu@suse.com>
Applied!
Thanks,
Jes
^ permalink raw reply
* Re: WARNING: CPU: 4 PID: 10512 at drivers/md/raid5-cache.c:728 r5l_do_reclaim+0x415/0x430 [raid456]
From: yizhan @ 2016-08-02 13:31 UTC (permalink / raw)
To: Guoqing Jiang, linux-raid
In-Reply-To: <57A05B70.9060500@suse.com>
> I tried with latest Linus's tree
> (731c7d3a205ba89b475b2aa71b5f13dd6ae3de56), but still can't find the
> warning.
>
> linux241:~ # uname -r
> 4.7.0-11-default+
> linux241:~ # mdadm --version
> mdadm - v3.4-41-gf96b130 - 30th March 2016
>
> The biggest difference is that I am using VM machine while you were
> testing with real hardware, though I am not
> sure it was caused by hardware or not. Do you always see the "couldn't
> update array info" info during test? If so,
> pls try to make it disappear.
>
> Thanks,
> Guoqing
May be it's env related, I have tried on another PC which with the same
kernel/mdadm version and cannot reproduce it.
Below is one log I reproduced which without " couldn't update array info"'
[ 504.650453] md: bind<sdb1>
[ 504.650539] md: bind<sdd1>
[ 504.650611] md: bind<sde1>
[ 504.651137] md: bind<sdc1>
[ 504.651211] md: bind<sdf1>
[ 504.655535] md/raid:md0: device sde1 operational as raid disk 2
[ 504.655537] md/raid:md0: device sdd1 operational as raid disk 1
[ 504.655538] md/raid:md0: device sdb1 operational as raid disk 0
[ 504.656040] md/raid:md0: allocated 4374kB
[ 504.656069] md/raid:md0: raid level 4 active with 3 out of 4 devices,
algorithm 0
[ 504.656071] RAID conf printout:
[ 504.656085] --- level:4 rd:4 wd:3
[ 504.656086] disk 0, o:1, dev:sdb1
[ 504.656087] disk 1, o:1, dev:sdd1
[ 504.656088] disk 2, o:1, dev:sde1
[ 504.656107] md/raid:md0: using device sdc1 as journal
[ 504.658052] created bitmap (1 pages) for device md0
[ 504.658124] md0: bitmap initialized from disk: read 1 pages, set 16
of 16 bits
[ 504.667007] md0: detected capacity change from 0 to 3221225472
[ 504.667032] RAID conf printout:
[ 504.667035] --- level:4 rd:4 wd:3
[ 504.667037] disk 0, o:1, dev:sdb1
[ 504.667039] disk 1, o:1, dev:sdd1
[ 504.667041] disk 2, o:1, dev:sde1
[ 504.667042] disk 3, o:1, dev:sdf1
[ 504.667156] md: recovery of RAID array md0
[ 504.667159] md: minimum _guaranteed_ speed: 1000 KB/sec/disk.
[ 504.667160] md: using maximum available idle IO bandwidth (but not
more than 200000 KB/sec) for recovery.
[ 504.667165] md: using 128k window, over a total of 1048576k.
[ 516.581223] md: md0: recovery done.
[ 516.678940] RAID conf printout:
[ 516.678943] --- level:4 rd:4 wd:4
[ 516.678945] disk 0, o:1, dev:sdb1
[ 516.678946] disk 1, o:1, dev:sdd1
[ 516.678947] disk 2, o:1, dev:sde1
[ 516.678948] disk 3, o:1, dev:sdf1
[ 517.694241] md/raid:md0: Disk failure on sdb1, disabling device.
md/raid:md0: Operation continuing on 3 devices.
[ 517.734547] RAID conf printout:
[ 517.734549] --- level:4 rd:4 wd:3
[ 517.734551] disk 0, o:0, dev:sdb1
[ 517.734552] disk 1, o:1, dev:sdd1
[ 517.734553] disk 2, o:1, dev:sde1
[ 517.734554] disk 3, o:1, dev:sdf1
[ 517.737295] RAID conf printout:
[ 517.737298] --- level:4 rd:4 wd:3
[ 517.737299] disk 1, o:1, dev:sdd1
[ 517.737300] disk 2, o:1, dev:sde1
[ 517.737301] disk 3, o:1, dev:sdf1
[ 522.695241] md: unbind<sdb1>
[ 522.698393] md: export_rdev(sdb1)
[ 523.277566] md: bind<sdb1>
[ 523.333573] RAID conf printout:
[ 523.333576] --- level:4 rd:4 wd:3
[ 523.333577] disk 0, o:1, dev:sdb1
[ 523.333578] disk 1, o:1, dev:sdd1
[ 523.333579] disk 2, o:1, dev:sde1
[ 523.333580] disk 3, o:1, dev:sdf1
[ 523.379453] ------------[ cut here ]------------
[ 523.379462] WARNING: CPU: 0 PID: 5186 at drivers/md/raid5-cache.c:728
r5l_do_reclaim+0x415/0x430 [raid456]
[ 523.379463] Modules linked in: raid456 async_raid6_recov async_memcpy
async_pq async_xor xor async_tx raid6_pq xt_CHECKSUM ipt_MASQUERADE
nf_nat_masquerade_ipv4 tun ipt_REJECT nf_reject_ipv4 ip6t_rpfilter
ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat
ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6
nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security
ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4
nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw
ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter
sb_edac edac_core snd_hda_codec_hdmi x86_pkg_temp_thermal
snd_hda_codec_realtek intel_powerclamp coretemp snd_hda_codec_generic
snd_hda_intel kvm_intel snd_hda_codec kvm snd_hda_core snd_hwdep snd_seq
snd_seq_device
[ 523.379488] irqbypass snd_pcm crct10dif_pclmul crc32_pclmul
ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper
cryptd snd_timer iTCO_wdt hp_wmi snd sparse_keymap ioatdma rfkill
iTCO_vendor_support sg pcspkr shpchp soundcore mei_me i2c_i801 dca mei
lpc_ich mfd_core tpm_infineon nfsd auth_rpcgss nfs_acl lockd grace
sunrpc ip_tables xfs libcrc32c sd_mod nouveau video mxm_wmi i2c_algo_bit
drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm isci
drm libsas e1000e ata_generic ahci pata_acpi scsi_transport_sas libahci
libata firewire_ohci firewire_core crc32c_intel serio_raw ptp pps_core
crc_itu_t i2c_core wmi fjes dm_mirror dm_region_hash dm_log dm_mod
[ 523.379520] CPU: 0 PID: 5186 Comm: md0_resync Tainted: G W 4.7.0 #5
[ 523.379521] Hardware name: Hewlett-Packard HP Z620 Workstation/158A,
BIOS J61 v03.69 03/25/2014
[ 523.379522] 0000000000000286 00000000277f1813 ffff880801c5fb40
ffffffff8134cd0c
[ 523.379524] 0000000000000000 0000000000000000 ffff880801c5fb80
ffffffff8108c351
[ 523.379526] 000002d8c7a96d80 0000000023f0d6c6 ffff8800c6dc8000
ffff8807f334be88
[ 523.379528] Call Trace:
[ 523.379533] [<ffffffff8134cd0c>] dump_stack+0x63/0x87
[ 523.379537] [<ffffffff8108c351>] __warn+0xd1/0xf0
[ 523.379539] [<ffffffff8108c48d>] warn_slowpath_null+0x1d/0x20
[ 523.379542] [<ffffffffa0995805>] r5l_do_reclaim+0x415/0x430 [raid456]
[ 523.379546] [<ffffffff811f9d00>] ? kfree+0x120/0x170
[ 523.379548] [<ffffffffa099633b>] r5l_quiesce+0x7b/0xa0 [raid456]
[ 523.379551] [<ffffffffa0988da0>] raid5_quiesce+0x50/0x2a0 [raid456]
[ 523.379553] [<ffffffff810d0240>] ? prepare_to_wait_event+0xf0/0xf0
[ 523.379556] [<ffffffff8156d32e>] md_do_sync+0xe7e/0xf60
[ 523.379560] [<ffffffff81036bf9>] ? sched_clock+0x9/0x10
[ 523.379562] [<ffffffff810bc4b2>] ? sched_clock_cpu+0x72/0xa0
[ 523.379564] [<ffffffff810c3985>] ? put_prev_entity+0x35/0x730
[ 523.379567] [<ffffffff8102c6d9>] ? __switch_to+0x219/0x5c0
[ 523.379570] [<ffffffff81098d43>] ? kernel_sigaction+0x43/0xe0
[ 523.379572] [<ffffffff81566dc6>] md_thread+0x136/0x150
[ 523.379575] [<ffffffff81566c90>] ? find_pers+0x70/0x70
[ 523.379577] [<ffffffff810ab678>] kthread+0xd8/0xf0
[ 523.379580] [<ffffffff816e88bf>] ret_from_fork+0x1f/0x40
[ 523.379582] [<ffffffff810ab5a0>] ? kthread_park+0x60/0x60
[ 523.379583] ---[ end trace 76fe4588f24ee7b5 ]---
[ 523.429860] md: recovery of RAID array md0
[ 523.429862] md: minimum _guaranteed_ speed: 1000 KB/sec/disk.
[ 523.429864] md: using maximum available idle IO bandwidth (but not
more than 200000 KB/sec) for recovery.
[ 523.429876] md: using 128k window, over a total of 1048576k.
[ 523.429882] md: md0: recovery interrupted.
[ 523.501621] md: md0 still in use.
[ 523.501629] RAID conf printout:
[ 523.501632] --- level:4 rd:4 wd:3
[ 523.501634] disk 0, o:1, dev:sdb1
[ 523.501635] disk 1, o:1, dev:sdd1
[ 523.501636] disk 2, o:1, dev:sde1
[ 523.501637] disk 3, o:1, dev:sdf1
[ 523.501638] RAID conf printout:
[ 523.501638] --- level:4 rd:4 wd:3
[ 523.501639] disk 0, o:1, dev:sdb1
[ 523.501640] disk 1, o:1, dev:sdd1
[ 523.501641] disk 2, o:1, dev:sde1
[ 523.501642] disk 3, o:1, dev:sdf1
[ 523.501737] md: recovery of RAID array md0
[ 523.501740] md: minimum _guaranteed_ speed: 1000 KB/sec/disk.
[ 523.501741] md: using maximum available idle IO bandwidth (but not
more than 200000 KB/sec) for recovery.
[ 523.501745] md: using 128k window, over a total of 1048576k.
[ 523.501746] md: resuming recovery of md0 from checkpoint.
[ 523.501768] md: md0: recovery done.
[ 523.534961] RAID conf printout:
[ 523.534964] --- level:4 rd:4 wd:4
[ 523.534966] disk 0, o:1, dev:sdb1
[ 523.534967] disk 1, o:1, dev:sdd1
[ 523.534968] disk 2, o:1, dev:sde1
[ 523.534969] disk 3, o:1, dev:sdf1
[ 523.725191] md0: detected capacity change from 3221225472 to 0
[ 523.725282] md: md0 stopped.
[ 523.725286] md: unbind<sdb1>
[ 523.740217] md: export_rdev(sdb1)
[ 523.740226] md: unbind<sdf1>
[ 523.746268] md: export_rdev(sdf1)
[ 523.746284] md: unbind<sdc1>
[ 523.753263] md: export_rdev(sdc1)
[ 523.753278] md: unbind<sde1>
[ 523.759267] md: export_rdev(sde1)
[ 523.759282] md: unbind<sdd1>
[ 523.765299] md: export_rdev(sdd1)
^ permalink raw reply
* [PATCH 0315/1285] Replace numeric parameter like 0444 with macro
From: Baole Ni @ 2016-08-02 10:59 UTC (permalink / raw)
To: shli, snitzer, dm-devel, hal.rosenstock, dledford, sean.hefty, bp
Cc: linux-raid, linux-kernel, chuansheng.liu, baolex.ni
I find that the developers often just specified the numeric value
when calling a macro which is defined with a parameter for access permission.
As we know, these numeric value for access permission have had the corresponding macro,
and that using macro can improve the robustness and readability of the code,
thus, I suggest replacing the numeric parameter with the macro.
Signed-off-by: Chuansheng Liu <chuansheng.liu@intel.com>
Signed-off-by: Baole Ni <baolex.ni@intel.com>
---
drivers/md/raid5.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c
index 8959e6d..33562bf 100644
--- a/drivers/md/raid5.c
+++ b/drivers/md/raid5.c
@@ -66,7 +66,7 @@
#define ANY_GROUP NUMA_NO_NODE
static bool devices_handle_discard_safely = false;
-module_param(devices_handle_discard_safely, bool, 0644);
+module_param(devices_handle_discard_safely, bool, S_IRUSR | S_IWUSR | S_IRGRP | S_IROTH);
MODULE_PARM_DESC(devices_handle_discard_safely,
"Set to Y if all devices in each array reliably return zeroes on reads from discarded regions");
static struct workqueue_struct *raid5_wq;
--
2.9.2
^ permalink raw reply related
* [PATCH 0314/1285] Replace numeric parameter like 0444 with macro
From: Baole Ni @ 2016-08-02 10:59 UTC (permalink / raw)
To: agk, snitzer, shli, dm-devel, hal.rosenstock, dledford,
sean.hefty, bp
Cc: linux-raid, linux-kernel, chuansheng.liu, baolex.ni
I find that the developers often just specified the numeric value
when calling a macro which is defined with a parameter for access permission.
As we know, these numeric value for access permission have had the corresponding macro,
and that using macro can improve the robustness and readability of the code,
thus, I suggest replacing the numeric parameter with the macro.
Signed-off-by: Chuansheng Liu <chuansheng.liu@intel.com>
Signed-off-by: Baole Ni <baolex.ni@intel.com>
---
drivers/md/dm-raid.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/md/dm-raid.c b/drivers/md/dm-raid.c
index 5253274..b01f1e9 100644
--- a/drivers/md/dm-raid.c
+++ b/drivers/md/dm-raid.c
@@ -1754,7 +1754,7 @@ static void __exit dm_raid_exit(void)
module_init(dm_raid_init);
module_exit(dm_raid_exit);
-module_param(devices_handle_discard_safely, bool, 0644);
+module_param(devices_handle_discard_safely, bool, S_IRUSR | S_IWUSR | S_IRGRP | S_IROTH);
MODULE_PARM_DESC(devices_handle_discard_safely,
"Set to Y if all devices in each array reliably return zeroes on reads from discarded regions");
--
2.9.2
^ permalink raw reply related
* Re: Inactive arrays
From: Daniel Sanabria @ 2016-08-02 10:45 UTC (permalink / raw)
To: Wols Lists; +Cc: linux-raid
In-Reply-To: <57A07345.4040708@youngman.org.uk>
Thanks very much for the response Wol.
It looks like the PSU is dead (server automatically powers off a few
seconds after power on).
I'm planning to order a PSU replacement to resume troubleshooting so
please bear with me; maybe the PSU was degraded and couldn't power
some of drives?
Cheers,
Daniel
On 2 August 2016 at 11:17, Wols Lists <antlists@youngman.org.uk> wrote:
> Just a quick first response. I see md128 and md129 are both down, and
> are both listed as one drive, raid0. Bit odd, that ...
>
> What version of mdadm are you using? One of them had a bug (3.2.3 era?)
> that would split an array in two. Is it possible that you should have
> one raid0 array with sdf1 and sdf2? But that's a bit of a weird setup...
>
> I notice also that md126 is raid10 across two drives. That's odd, too.
>
> How much do you know about what the setup should be, and why it was set
> up that way?
>
> Download lspci by Phil Turmel (it requires python2.7, if your machine is
> python3 a quick fix to the shebang at the start should get it to work).
> Post the output from that here.
>
> Cheers,
> Wol
>
> On 02/08/16 08:36, Daniel Sanabria wrote:
>> Hi All,
>>
>> I have a box that I believe was not powered down correctly and after
>> transporting it to a different location it doesn't boot anymore
>> stopping at BIOS check "Verifying DMI Pool Data".
>>
>> The box have 6 drives and after instructing the BIOS to boot from the
>> first drive I managed to boot the OS (Fedora 23) after commenting out
>> 2 /etc/fstab entries , output for "uname -a; cat /etc/fstab" follows:
>>
>> [root@lamachine ~]# uname -a; cat /etc/fstab
>> Linux lamachine 4.3.3-303.fc23.x86_64 #1 SMP Tue Jan 19 18:31:55 UTC
>> 2016 x86_64 x86_64 x86_64 GNU/Linux
>>
>> #
>> # /etc/fstab
>> # Created by anaconda on Tue Mar 24 19:31:21 2015
>> #
>> # Accessible filesystems, by reference, are maintained under '/dev/disk'
>> # See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
>> #
>> /dev/mapper/vg_bigblackbox-LogVol_root / ext4
>> defaults 1 1
>> UUID=4e51f903-37ca-4479-9197-fac7b2280557 /boot ext4
>> defaults 1 2
>> /dev/mapper/vg_bigblackbox-LogVol_opt /opt ext4
>> defaults 1 2
>> /dev/mapper/vg_bigblackbox-LogVol_tmp /tmp ext4
>> defaults 1 2
>> /dev/mapper/vg_bigblackbox-LogVol_var /var ext4
>> defaults 1 2
>> UUID=9194f492-881a-4fc3-ac09-ca4e1cc2985a swap swap
>> defaults 0 0
>> /dev/md2 /home ext4 defaults 1 2
>> #/dev/vg_media/lv_media /mnt/media ext4 defaults 1 2
>> #/dev/vg_virt_dir/lv_virt_dir1 /mnt/guest_images/ ext4 defaults 1 2
>> [root@lamachine ~]#
>>
>> When checking mdstat I can see that 2 of the arrays are showing up as
>> inactive, but not sure how to safely activate these so looking for
>> some knowledgeable advice on how to proceed here.
>>
>> Thanks in advance,
>>
>> Daniel
>>
>> Below some more relevant outputs:
>>
>> [root@lamachine ~]# cat /proc/mdstat
>> Personalities : [raid10] [raid6] [raid5] [raid4] [raid0]
>> md127 : active raid0 sda5[0] sdc5[2] sdb5[1]
>> 94367232 blocks super 1.2 512k chunks
>>
>> md2 : active raid5 sda3[0] sdc2[2] sdb2[1]
>> 511999872 blocks level 5, 64k chunk, algorithm 2 [3/3] [UUU]
>>
>> md128 : inactive sdf1[3](S)
>> 2147352576 blocks super 1.2
>>
>> md129 : inactive sdf2[2](S)
>> 524156928 blocks super 1.2
>>
>> md126 : active raid10 sda2[0] sdc1[1]
>> 30719936 blocks 2 near-copies [2/2] [UU]
>>
>> unused devices: <none>
>> [root@lamachine ~]# cat /etc/mdadm.conf
>> # mdadm.conf written out by anaconda
>> MAILADDR root
>> AUTO +imsm +1.x -all
>> ARRAY /dev/md2 level=raid5 num-devices=3
>> UUID=2cff15d1:e411447b:fd5d4721:03e44022
>> ARRAY /dev/md126 level=raid10 num-devices=2
>> UUID=9af006ca:8845bbd3:bfe78010:bc810f04
>> ARRAY /dev/md127 level=raid0 num-devices=3
>> UUID=acd5374f:72628c93:6a906c4b:5f675ce5
>> ARRAY /dev/md128 metadata=1.2 spares=1 name=lamachine:128
>> UUID=f2372cb9:d3816fd6:ce86d826:882ec82e
>> ARRAY /dev/md129 metadata=1.2 name=lamachine:129
>> UUID=895dae98:d1a496de:4f590b8b:cb8ac12a
>> [root@lamachine ~]# mdadm --detail /dev/md1*
>> /dev/md126:
>> Version : 0.90
>> Creation Time : Thu Dec 3 22:12:12 2009
>> Raid Level : raid10
>> Array Size : 30719936 (29.30 GiB 31.46 GB)
>> Used Dev Size : 30719936 (29.30 GiB 31.46 GB)
>> Raid Devices : 2
>> Total Devices : 2
>> Preferred Minor : 126
>> Persistence : Superblock is persistent
>>
>> Update Time : Tue Aug 2 07:46:39 2016
>> State : clean
>> Active Devices : 2
>> Working Devices : 2
>> Failed Devices : 0
>> Spare Devices : 0
>>
>> Layout : near=2
>> Chunk Size : 64K
>>
>> UUID : 9af006ca:8845bbd3:bfe78010:bc810f04
>> Events : 0.264152
>>
>> Number Major Minor RaidDevice State
>> 0 8 2 0 active sync set-A /dev/sda2
>> 1 8 33 1 active sync set-B /dev/sdc1
>> /dev/md127:
>> Version : 1.2
>> Creation Time : Tue Jul 26 19:00:28 2011
>> Raid Level : raid0
>> Array Size : 94367232 (90.00 GiB 96.63 GB)
>> Raid Devices : 3
>> Total Devices : 3
>> Persistence : Superblock is persistent
>>
>> Update Time : Tue Jul 26 19:00:28 2011
>> State : clean
>> Active Devices : 3
>> Working Devices : 3
>> Failed Devices : 0
>> Spare Devices : 0
>>
>> Chunk Size : 512K
>>
>> Name : reading.homeunix.com:3
>> UUID : acd5374f:72628c93:6a906c4b:5f675ce5
>> Events : 0
>>
>> Number Major Minor RaidDevice State
>> 0 8 5 0 active sync /dev/sda5
>> 1 8 21 1 active sync /dev/sdb5
>> 2 8 37 2 active sync /dev/sdc5
>> /dev/md128:
>> Version : 1.2
>> Raid Level : raid0
>> Total Devices : 1
>> Persistence : Superblock is persistent
>>
>> State : inactive
>>
>> Name : lamachine:128 (local to host lamachine)
>> UUID : f2372cb9:d3816fd6:ce86d826:882ec82e
>> Events : 4154
>>
>> Number Major Minor RaidDevice
>>
>> - 8 81 - /dev/sdf1
>> /dev/md129:
>> Version : 1.2
>> Raid Level : raid0
>> Total Devices : 1
>> Persistence : Superblock is persistent
>>
>> State : inactive
>>
>> Name : lamachine:129 (local to host lamachine)
>> UUID : 895dae98:d1a496de:4f590b8b:cb8ac12a
>> Events : 0
>>
>> Number Major Minor RaidDevice
>>
>> - 8 82 - /dev/sdf2
>> [root@lamachine ~]# mdadm --detail /dev/md2
>> /dev/md2:
>> Version : 0.90
>> Creation Time : Mon Feb 11 07:54:36 2013
>> Raid Level : raid5
>> Array Size : 511999872 (488.28 GiB 524.29 GB)
>> Used Dev Size : 255999936 (244.14 GiB 262.14 GB)
>> Raid Devices : 3
>> Total Devices : 3
>> Preferred Minor : 2
>> Persistence : Superblock is persistent
>>
>> Update Time : Mon Aug 1 20:24:23 2016
>> State : clean
>> Active Devices : 3
>> Working Devices : 3
>> Failed Devices : 0
>> Spare Devices : 0
>>
>> Layout : left-symmetric
>> Chunk Size : 64K
>>
>> UUID : 2cff15d1:e411447b:fd5d4721:03e44022 (local to host lamachine)
>> Events : 0.611
>>
>> Number Major Minor RaidDevice State
>> 0 8 3 0 active sync /dev/sda3
>> 1 8 18 1 active sync /dev/sdb2
>> 2 8 34 2 active sync /dev/sdc2
>> [root@lamachine ~]#
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>
>
^ permalink raw reply
* Re: Inactive arrays
From: Wols Lists @ 2016-08-02 10:17 UTC (permalink / raw)
To: Daniel Sanabria, linux-raid
In-Reply-To: <CAHscji0ddZT-aq5fEp8aG030+F7UqGzMgRnm4QP1Lr1AsGy61Q@mail.gmail.com>
Just a quick first response. I see md128 and md129 are both down, and
are both listed as one drive, raid0. Bit odd, that ...
What version of mdadm are you using? One of them had a bug (3.2.3 era?)
that would split an array in two. Is it possible that you should have
one raid0 array with sdf1 and sdf2? But that's a bit of a weird setup...
I notice also that md126 is raid10 across two drives. That's odd, too.
How much do you know about what the setup should be, and why it was set
up that way?
Download lspci by Phil Turmel (it requires python2.7, if your machine is
python3 a quick fix to the shebang at the start should get it to work).
Post the output from that here.
Cheers,
Wol
On 02/08/16 08:36, Daniel Sanabria wrote:
> Hi All,
>
> I have a box that I believe was not powered down correctly and after
> transporting it to a different location it doesn't boot anymore
> stopping at BIOS check "Verifying DMI Pool Data".
>
> The box have 6 drives and after instructing the BIOS to boot from the
> first drive I managed to boot the OS (Fedora 23) after commenting out
> 2 /etc/fstab entries , output for "uname -a; cat /etc/fstab" follows:
>
> [root@lamachine ~]# uname -a; cat /etc/fstab
> Linux lamachine 4.3.3-303.fc23.x86_64 #1 SMP Tue Jan 19 18:31:55 UTC
> 2016 x86_64 x86_64 x86_64 GNU/Linux
>
> #
> # /etc/fstab
> # Created by anaconda on Tue Mar 24 19:31:21 2015
> #
> # Accessible filesystems, by reference, are maintained under '/dev/disk'
> # See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
> #
> /dev/mapper/vg_bigblackbox-LogVol_root / ext4
> defaults 1 1
> UUID=4e51f903-37ca-4479-9197-fac7b2280557 /boot ext4
> defaults 1 2
> /dev/mapper/vg_bigblackbox-LogVol_opt /opt ext4
> defaults 1 2
> /dev/mapper/vg_bigblackbox-LogVol_tmp /tmp ext4
> defaults 1 2
> /dev/mapper/vg_bigblackbox-LogVol_var /var ext4
> defaults 1 2
> UUID=9194f492-881a-4fc3-ac09-ca4e1cc2985a swap swap
> defaults 0 0
> /dev/md2 /home ext4 defaults 1 2
> #/dev/vg_media/lv_media /mnt/media ext4 defaults 1 2
> #/dev/vg_virt_dir/lv_virt_dir1 /mnt/guest_images/ ext4 defaults 1 2
> [root@lamachine ~]#
>
> When checking mdstat I can see that 2 of the arrays are showing up as
> inactive, but not sure how to safely activate these so looking for
> some knowledgeable advice on how to proceed here.
>
> Thanks in advance,
>
> Daniel
>
> Below some more relevant outputs:
>
> [root@lamachine ~]# cat /proc/mdstat
> Personalities : [raid10] [raid6] [raid5] [raid4] [raid0]
> md127 : active raid0 sda5[0] sdc5[2] sdb5[1]
> 94367232 blocks super 1.2 512k chunks
>
> md2 : active raid5 sda3[0] sdc2[2] sdb2[1]
> 511999872 blocks level 5, 64k chunk, algorithm 2 [3/3] [UUU]
>
> md128 : inactive sdf1[3](S)
> 2147352576 blocks super 1.2
>
> md129 : inactive sdf2[2](S)
> 524156928 blocks super 1.2
>
> md126 : active raid10 sda2[0] sdc1[1]
> 30719936 blocks 2 near-copies [2/2] [UU]
>
> unused devices: <none>
> [root@lamachine ~]# cat /etc/mdadm.conf
> # mdadm.conf written out by anaconda
> MAILADDR root
> AUTO +imsm +1.x -all
> ARRAY /dev/md2 level=raid5 num-devices=3
> UUID=2cff15d1:e411447b:fd5d4721:03e44022
> ARRAY /dev/md126 level=raid10 num-devices=2
> UUID=9af006ca:8845bbd3:bfe78010:bc810f04
> ARRAY /dev/md127 level=raid0 num-devices=3
> UUID=acd5374f:72628c93:6a906c4b:5f675ce5
> ARRAY /dev/md128 metadata=1.2 spares=1 name=lamachine:128
> UUID=f2372cb9:d3816fd6:ce86d826:882ec82e
> ARRAY /dev/md129 metadata=1.2 name=lamachine:129
> UUID=895dae98:d1a496de:4f590b8b:cb8ac12a
> [root@lamachine ~]# mdadm --detail /dev/md1*
> /dev/md126:
> Version : 0.90
> Creation Time : Thu Dec 3 22:12:12 2009
> Raid Level : raid10
> Array Size : 30719936 (29.30 GiB 31.46 GB)
> Used Dev Size : 30719936 (29.30 GiB 31.46 GB)
> Raid Devices : 2
> Total Devices : 2
> Preferred Minor : 126
> Persistence : Superblock is persistent
>
> Update Time : Tue Aug 2 07:46:39 2016
> State : clean
> Active Devices : 2
> Working Devices : 2
> Failed Devices : 0
> Spare Devices : 0
>
> Layout : near=2
> Chunk Size : 64K
>
> UUID : 9af006ca:8845bbd3:bfe78010:bc810f04
> Events : 0.264152
>
> Number Major Minor RaidDevice State
> 0 8 2 0 active sync set-A /dev/sda2
> 1 8 33 1 active sync set-B /dev/sdc1
> /dev/md127:
> Version : 1.2
> Creation Time : Tue Jul 26 19:00:28 2011
> Raid Level : raid0
> Array Size : 94367232 (90.00 GiB 96.63 GB)
> Raid Devices : 3
> Total Devices : 3
> Persistence : Superblock is persistent
>
> Update Time : Tue Jul 26 19:00:28 2011
> State : clean
> Active Devices : 3
> Working Devices : 3
> Failed Devices : 0
> Spare Devices : 0
>
> Chunk Size : 512K
>
> Name : reading.homeunix.com:3
> UUID : acd5374f:72628c93:6a906c4b:5f675ce5
> Events : 0
>
> Number Major Minor RaidDevice State
> 0 8 5 0 active sync /dev/sda5
> 1 8 21 1 active sync /dev/sdb5
> 2 8 37 2 active sync /dev/sdc5
> /dev/md128:
> Version : 1.2
> Raid Level : raid0
> Total Devices : 1
> Persistence : Superblock is persistent
>
> State : inactive
>
> Name : lamachine:128 (local to host lamachine)
> UUID : f2372cb9:d3816fd6:ce86d826:882ec82e
> Events : 4154
>
> Number Major Minor RaidDevice
>
> - 8 81 - /dev/sdf1
> /dev/md129:
> Version : 1.2
> Raid Level : raid0
> Total Devices : 1
> Persistence : Superblock is persistent
>
> State : inactive
>
> Name : lamachine:129 (local to host lamachine)
> UUID : 895dae98:d1a496de:4f590b8b:cb8ac12a
> Events : 0
>
> Number Major Minor RaidDevice
>
> - 8 82 - /dev/sdf2
> [root@lamachine ~]# mdadm --detail /dev/md2
> /dev/md2:
> Version : 0.90
> Creation Time : Mon Feb 11 07:54:36 2013
> Raid Level : raid5
> Array Size : 511999872 (488.28 GiB 524.29 GB)
> Used Dev Size : 255999936 (244.14 GiB 262.14 GB)
> Raid Devices : 3
> Total Devices : 3
> Preferred Minor : 2
> Persistence : Superblock is persistent
>
> Update Time : Mon Aug 1 20:24:23 2016
> State : clean
> Active Devices : 3
> Working Devices : 3
> Failed Devices : 0
> Spare Devices : 0
>
> Layout : left-symmetric
> Chunk Size : 64K
>
> UUID : 2cff15d1:e411447b:fd5d4721:03e44022 (local to host lamachine)
> Events : 0.611
>
> Number Major Minor RaidDevice State
> 0 8 3 0 active sync /dev/sda3
> 1 8 18 1 active sync /dev/sdb2
> 2 8 34 2 active sync /dev/sdc2
> [root@lamachine ~]#
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
^ permalink raw reply
* Re: [PATCH 3/3] MD: hold mddev lock for md-cluster receive thread
From: Guoqing Jiang @ 2016-08-02 9:52 UTC (permalink / raw)
To: Shaohua Li; +Cc: linux-raid, NeilBrown
In-Reply-To: <20160801214522.GA129828@kernel.org>
On 08/02/2016 05:45 AM, Shaohua Li wrote:
> On Mon, Aug 01, 2016 at 04:38:59PM +0800, Guoqing Jiang wrote:
>> Hi,
>>
>> On 07/31/2016 07:54 AM, shli@kernel.org wrote:
>>> From: Shaohua Li <shli@fb.com>
>>>
>>> md-cluster receive thread calls .quiesce too, let it hold mddev lock.
>> I'd suggest hold on for the patchset, I can find lock problem easily with
>> the patchset applied. Take a resyncing clusteed raid1 as example.
>>
>> md127_raid1 thread held reconfig_mutex then update sb, so it needs dlm
>> token lock. Meanwhile md127_resync thread got token lock and wants
>> EX on ack lock but recv_daemon can't release ack lock since recv_daemon
>> doesn't get reconfig_mutex.
> Thansk, I'll drop this one. Other two patches are still safe for md-cluster,
> right?
From the latest test, I can't find lock issues with the first two patches,
but I doubt it would have side effect for the performance of resync.
> I really hope to have consistent locking for .quiesce. For the
> process_recvd_msg, I'm wondering what's protecting the datas? for example,
> md-cluster uses md_find_rdev_nr_rcu, which access the disks list without
> locking. Is there a race?
Yes, it should be protected by rcu lock, I will post a patch for it, thanks
for reminder.
> Does it work if we move the mddev lock to
> process_recvd_msg?
I tried that, but It still have lock issue, eg, when node B and C have
status
as "resync=PENDING", then try to stop the resyncing array in node A.
Thanks,
Guoqing
^ permalink raw reply
* [PATCH] mdadm:add 'clustered' in typo prompt when specify wrong param for bitmap
From: Zhilong Liu @ 2016-08-02 9:18 UTC (permalink / raw)
To: Jes.Sorensen; +Cc: linux-raid, Zhilong Liu
mdadm: 'clustered' bitmap has already supported, thus add the
prompt if users specify wrong value for bitmap param.
Signed-off-by: Zhilong Liu <zlliu@suse.com>
diff --git a/mdadm.c b/mdadm.c
index 58ac32a..cca0933 100644
--- a/mdadm.c
+++ b/mdadm.c
@@ -1142,7 +1142,7 @@ int main(int argc, char *argv[])
continue;
}
/* probable typo */
- pr_err("bitmap file must contain a '/', or be 'internal', or 'none'\n"
+ pr_err("bitmap file must contain a '/', or be 'internal', or be 'clustered', or 'none'\n"
" not '%s'\n", optarg);
exit(2);
--
2.6.6
^ permalink raw reply related
* Re: WARNING: CPU: 4 PID: 10512 at drivers/md/raid5-cache.c:728 r5l_do_reclaim+0x415/0x430 [raid456]
From: Guoqing Jiang @ 2016-08-02 8:36 UTC (permalink / raw)
To: yizhan, linux-raid
In-Reply-To: <7d0acb42-1209-0722-7740-0fa03d44106b@redhat.com>
On 08/01/2016 08:28 AM, yizhan wrote:
>
>
>>> [ 902.706372] RAID conf printout:
>>> [ 902.706380] --- level:6 rd:4 wd:4
>>> [ 902.706381] disk 0, o:1, dev:sdb1
>>> [ 902.706382] disk 1, o:1, dev:sdd1
>>> [ 902.706383] disk 2, o:1, dev:sde1
>>> [ 902.706384] disk 3, o:1, dev:sdf1
>>> [ 902.706393] md/raid456: discard support disabled due to uncertainty.
>>> [ 902.706394] Set raid456.devices_handle_discard_safely=Y to override.
>>> [ 902.706396] md/raid:md0: using device sdc1 as journal
>>> [ 902.707354] created bitmap (1 pages) for device md0
>>> [ 902.707401] md0: bitmap initialized from disk: read 1 pages, set 8 of 8 bits
>>> [ 902.781331] md0: detected capacity change from 0 to 1071644672
>>> [ 902.781465] md: resync of RAID array md0
>>> [ 902.781468] md: minimum _guaranteed_ speed: 1000 KB/sec/disk.
>>> [ 902.781469] md: using maximum available idle IO bandwidth (but not more than 200000 KB/sec) for resync.
>>> [ 902.781475] md: using 128k window, over a total of 523264k.
>>> [ 902.829618] md: couldn't update array info. -22
>>> [ 902.876797] md: couldn't update array info. -22
>>> [ 902.896926] md: couldn't update array info. -22
>>
>> Errors happened when update array info, not sure it have relationship
>> with
>> the WARNING or not.
>>
[snip]
>>> [ 1548.486067] ------------[ cut here ]------------
>>> [ 1548.486076] WARNING: CPU: 4 PID: 10512 at drivers/md/raid5-cache.c:728 r5l_do_reclaim+0x415/0x430 [raid456]
>>> [ 1548.486077] Modules linked in: fuse btrfs vfat msdos fat ext4 jbd2 mbcache binfmt_misc raid456 async_raid6_recov async_memcpy async_pq async_xor xor async_tx raid6_pq xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun ipt_REJECT nf_reject_ipv4 ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter sb_edac edac_core snd_hda_codec_realtek snd_hda_codec_hdmi snd_hda_codec_generic x86_pkg_temp_thermal snd_hda_intel intel_powerclamp coretemp
snd_hda_codec
>>> [ 1548.486117] kvm_intel kvm snd_hda_core irqbypass crct10dif_pclmul crc32_pclmul snd_hwdep ghash_clmulni_intel snd_seq aesni_intel lrw snd_seq_device gf128mul glue_helper snd_pcm ablk_helper hp_wmi iTCO_wdt snd_timer sparse_keymap rfkill cryptd iTCO_vendor_support mei_me ioatdma snd mei shpchp pcspkr sg soundcore i2c_i801 tpm_infineon lpc_ich mfd_core dca nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sd_mod nouveau video mxm_wmi i2c_algo_bit drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm isci e1000e ata_generic libsas pata_acpi ahci libahci scsi_transport_sas firewire_ohci libata firewire_core ptp crc32c_intel serio_raw pps_core i2c_core crc_itu_t wmi fjes dm_mirror dm_region_hash dm_log dm_mod
>>> [ 1548.486152] CPU: 4 PID: 10512 Comm: md0_resync Tainted: G W 4.7.0-rc7 #1
>>> [ 1548.486153] Hardware name: Hewlett-Packard HP Z620 Workstation/158A, BIOS J61 v03.69 03/25/2014
>>> [ 1548.486155] 0000000000000286 000000002369dfe1 ffff88079d713b40 ffffffff8134caec
>>> [ 1548.486157] 0000000000000000 0000000000000000 ffff88079d713b80 ffffffff8108c351
>>> [ 1548.486159] 000002d8a6ae4d80 000000008f0c904f ffff8807f64b0000 ffff8807c4ab6888
>>> [ 1548.486161] Call Trace:
>>> [ 1548.486166] [<ffffffff8134caec>] dump_stack+0x63/0x87
>>> [ 1548.486169] [<ffffffff8108c351>] __warn+0xd1/0xf0
>>> [ 1548.486172] [<ffffffff8108c48d>] warn_slowpath_null+0x1d/0x20
>>> [ 1548.486175] [<ffffffffa097d805>] r5l_do_reclaim+0x415/0x430 [raid456]
>>> [ 1548.486178] [<ffffffff811f9c50>] ? kfree+0x120/0x170
>>> [ 1548.486181] [<ffffffffa097e33b>] r5l_quiesce+0x7b/0xa0 [raid456]
>>> [ 1548.486183] [<ffffffffa0970da0>] raid5_quiesce+0x50/0x2a0 [raid456]
>>> [ 1548.486186] [<ffffffff810d0250>] ? prepare_to_wait_event+0xf0/0xf0
>>> [ 1548.486189] [<ffffffff8156d00e>] md_do_sync+0xe7e/0xf60
>>> [ 1548.486192] [<ffffffff810c4370>] ? enqueue_entity+0x2a0/0xcc0
>>> [ 1548.486194] [<ffffffff810c201d>] ? update_curr+0xed/0x180
>>> [ 1548.486196] [<ffffffff810c026e>] ? account_entity_dequeue+0xae/0xd0
>>> [ 1548.486197] [<ffffffff810c27f6>] ? dequeue_entity+0x266/0x980
>>> [ 1548.486200] [<ffffffff810b5ef5>] ? check_preempt_curr+0x75/0x90
>>> [ 1548.486202] [<ffffffff810b5f29>] ? ttwu_do_wakeup+0x19/0xe0
>>> [ 1548.486204] [<ffffffff810c2fce>] ? dequeue_task_fair+0xbe/0x830
>>> [ 1548.486206] [<ffffffff810c3995>] ? put_prev_entity+0x35/0x730
>>> [ 1548.486209] [<ffffffff8102c6d9>] ? __switch_to+0x219/0x5c0
>>> [ 1548.486211] [<ffffffff81098d43>] ? kernel_sigaction+0x43/0xe0
>>> [ 1548.486214] [<ffffffff81566aa6>] md_thread+0x136/0x150
>>> [ 1548.486216] [<ffffffff81566970>] ? find_pers+0x70/0x70
>>> [ 1548.486219] [<ffffffff810ab698>] kthread+0xd8/0xf0
>>> [ 1548.486221] [<ffffffff816e873f>] ret_from_fork+0x1f/0x40
>>> [ 1548.486223] [<ffffffff810ab5c0>] ? kthread_park+0x60/0x60
>>> [ 1548.486224] ---[ end trace 242e9516e36acb1e ]---
>>> [ 1548.543855] md: recovery of RAID array md0
>>> [ 1548.543857] md: minimum _guaranteed_ speed: 1000 KB/sec/disk.
>>> [ 1548.543859] md: using maximum available idle IO bandwidth (but not more than 200000 KB/sec) for recovery.
>>> [ 1548.543864] md: using 128k window, over a total of 523264k.
>>> [ 1548.543868] md: md0: recovery done.
>>> [ 1548.574961] RAID conf printout:
>>> [ 1548.574964] --- level:6 rd:4 wd:4
>>> [ 1548.574966] disk 0, o:1, dev:sdb1
>>> [ 1548.574967] disk 1, o:1, dev:sdd1
>>> [ 1548.574968] disk 2, o:1, dev:sde1
>>> [ 1548.574969] disk 3, o:1, dev:sdf1
>>>
>>> Steps I used:
>>>
>>> mdadm --create --run /dev/md0 --level 4 --metadata 1.2 --raid-devices 4
>>> /dev/sdb1 /dev/sd[d-f]1 --write-journal /dev/sdc1 --bitmap=internal
>>> --bitmap-chunk=64M --chunk 512
>>> mdadm --wait /dev/md0
>>> mkfs.ext4 /dev/md0
>>> mdadm /dev/md0 -f /dev/sdb1
>>> mdadm /dev/md0 -r /dev/sdb1
>>> mdadm /dev/md0 -a /dev/sdb1
>>
>> I tried above cmds in my VM machine (with set level as 6), but can't
>> see the calltrace with below versions.
>>
>> linux135:~ # uname -r
>> 4.4.13-46-default
>> linux135:~ # mdadm --version
>> mdadm - v3.4-47-g8718fc3 - 11th May 2016
>>
>> Thanks,
>> Guoqing
>>
> Could you try with 4.7.0, it's very easy to reproduce on my side
> below is my env:
> #lsblk /dev/sd[b-f]
> NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
> sdb 8:16 0 931.5G 0 disk
> └─sdb1 8:17 0 10G 0 part
> └─md0 9:0 0 30G 0 raid4
> sdc 8:32 0 186.3G 0 disk
> └─sdc1 8:33 0 10G 0 part
> └─md0 9:0 0 30G 0 raid4
> sdd 8:48 0 111.8G 0 disk
> └─sdd1 8:49 0 10G 0 part
> └─md0 9:0 0 30G 0 raid4
> sde 8:64 0 931.5G 0 disk
> └─sde1 8:65 0 10G 0 part
> └─md0 9:0 0 30G 0 raid4
> sdf 8:80 0 931.5G 0 disk
> └─sdf1 8:81 0 10G 0 part
> └─md0 9:0 0 30G 0 raid4
> # uname -r
> 4.7.0
> # mdadm --version
> mdadm - v3.4-63-g52209d6 - 21st July 2016
>
I tried with latest Linus's tree
(731c7d3a205ba89b475b2aa71b5f13dd6ae3de56), but still can't find the
warning.
linux241:~ # uname -r
4.7.0-11-default+
linux241:~ # mdadm --version
mdadm - v3.4-41-gf96b130 - 30th March 2016
The biggest difference is that I am using VM machine while you were
testing with real hardware, though I am not
sure it was caused by hardware or not. Do you always see the "couldn't
update array info" info during test? If so,
pls try to make it disappear.
Thanks,
Guoqing
^ permalink raw reply
* Inactive arrays
From: Daniel Sanabria @ 2016-08-02 7:36 UTC (permalink / raw)
To: linux-raid
Hi All,
I have a box that I believe was not powered down correctly and after
transporting it to a different location it doesn't boot anymore
stopping at BIOS check "Verifying DMI Pool Data".
The box have 6 drives and after instructing the BIOS to boot from the
first drive I managed to boot the OS (Fedora 23) after commenting out
2 /etc/fstab entries , output for "uname -a; cat /etc/fstab" follows:
[root@lamachine ~]# uname -a; cat /etc/fstab
Linux lamachine 4.3.3-303.fc23.x86_64 #1 SMP Tue Jan 19 18:31:55 UTC
2016 x86_64 x86_64 x86_64 GNU/Linux
#
# /etc/fstab
# Created by anaconda on Tue Mar 24 19:31:21 2015
#
# Accessible filesystems, by reference, are maintained under '/dev/disk'
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#
/dev/mapper/vg_bigblackbox-LogVol_root / ext4
defaults 1 1
UUID=4e51f903-37ca-4479-9197-fac7b2280557 /boot ext4
defaults 1 2
/dev/mapper/vg_bigblackbox-LogVol_opt /opt ext4
defaults 1 2
/dev/mapper/vg_bigblackbox-LogVol_tmp /tmp ext4
defaults 1 2
/dev/mapper/vg_bigblackbox-LogVol_var /var ext4
defaults 1 2
UUID=9194f492-881a-4fc3-ac09-ca4e1cc2985a swap swap
defaults 0 0
/dev/md2 /home ext4 defaults 1 2
#/dev/vg_media/lv_media /mnt/media ext4 defaults 1 2
#/dev/vg_virt_dir/lv_virt_dir1 /mnt/guest_images/ ext4 defaults 1 2
[root@lamachine ~]#
When checking mdstat I can see that 2 of the arrays are showing up as
inactive, but not sure how to safely activate these so looking for
some knowledgeable advice on how to proceed here.
Thanks in advance,
Daniel
Below some more relevant outputs:
[root@lamachine ~]# cat /proc/mdstat
Personalities : [raid10] [raid6] [raid5] [raid4] [raid0]
md127 : active raid0 sda5[0] sdc5[2] sdb5[1]
94367232 blocks super 1.2 512k chunks
md2 : active raid5 sda3[0] sdc2[2] sdb2[1]
511999872 blocks level 5, 64k chunk, algorithm 2 [3/3] [UUU]
md128 : inactive sdf1[3](S)
2147352576 blocks super 1.2
md129 : inactive sdf2[2](S)
524156928 blocks super 1.2
md126 : active raid10 sda2[0] sdc1[1]
30719936 blocks 2 near-copies [2/2] [UU]
unused devices: <none>
[root@lamachine ~]# cat /etc/mdadm.conf
# mdadm.conf written out by anaconda
MAILADDR root
AUTO +imsm +1.x -all
ARRAY /dev/md2 level=raid5 num-devices=3
UUID=2cff15d1:e411447b:fd5d4721:03e44022
ARRAY /dev/md126 level=raid10 num-devices=2
UUID=9af006ca:8845bbd3:bfe78010:bc810f04
ARRAY /dev/md127 level=raid0 num-devices=3
UUID=acd5374f:72628c93:6a906c4b:5f675ce5
ARRAY /dev/md128 metadata=1.2 spares=1 name=lamachine:128
UUID=f2372cb9:d3816fd6:ce86d826:882ec82e
ARRAY /dev/md129 metadata=1.2 name=lamachine:129
UUID=895dae98:d1a496de:4f590b8b:cb8ac12a
[root@lamachine ~]# mdadm --detail /dev/md1*
/dev/md126:
Version : 0.90
Creation Time : Thu Dec 3 22:12:12 2009
Raid Level : raid10
Array Size : 30719936 (29.30 GiB 31.46 GB)
Used Dev Size : 30719936 (29.30 GiB 31.46 GB)
Raid Devices : 2
Total Devices : 2
Preferred Minor : 126
Persistence : Superblock is persistent
Update Time : Tue Aug 2 07:46:39 2016
State : clean
Active Devices : 2
Working Devices : 2
Failed Devices : 0
Spare Devices : 0
Layout : near=2
Chunk Size : 64K
UUID : 9af006ca:8845bbd3:bfe78010:bc810f04
Events : 0.264152
Number Major Minor RaidDevice State
0 8 2 0 active sync set-A /dev/sda2
1 8 33 1 active sync set-B /dev/sdc1
/dev/md127:
Version : 1.2
Creation Time : Tue Jul 26 19:00:28 2011
Raid Level : raid0
Array Size : 94367232 (90.00 GiB 96.63 GB)
Raid Devices : 3
Total Devices : 3
Persistence : Superblock is persistent
Update Time : Tue Jul 26 19:00:28 2011
State : clean
Active Devices : 3
Working Devices : 3
Failed Devices : 0
Spare Devices : 0
Chunk Size : 512K
Name : reading.homeunix.com:3
UUID : acd5374f:72628c93:6a906c4b:5f675ce5
Events : 0
Number Major Minor RaidDevice State
0 8 5 0 active sync /dev/sda5
1 8 21 1 active sync /dev/sdb5
2 8 37 2 active sync /dev/sdc5
/dev/md128:
Version : 1.2
Raid Level : raid0
Total Devices : 1
Persistence : Superblock is persistent
State : inactive
Name : lamachine:128 (local to host lamachine)
UUID : f2372cb9:d3816fd6:ce86d826:882ec82e
Events : 4154
Number Major Minor RaidDevice
- 8 81 - /dev/sdf1
/dev/md129:
Version : 1.2
Raid Level : raid0
Total Devices : 1
Persistence : Superblock is persistent
State : inactive
Name : lamachine:129 (local to host lamachine)
UUID : 895dae98:d1a496de:4f590b8b:cb8ac12a
Events : 0
Number Major Minor RaidDevice
- 8 82 - /dev/sdf2
[root@lamachine ~]# mdadm --detail /dev/md2
/dev/md2:
Version : 0.90
Creation Time : Mon Feb 11 07:54:36 2013
Raid Level : raid5
Array Size : 511999872 (488.28 GiB 524.29 GB)
Used Dev Size : 255999936 (244.14 GiB 262.14 GB)
Raid Devices : 3
Total Devices : 3
Preferred Minor : 2
Persistence : Superblock is persistent
Update Time : Mon Aug 1 20:24:23 2016
State : clean
Active Devices : 3
Working Devices : 3
Failed Devices : 0
Spare Devices : 0
Layout : left-symmetric
Chunk Size : 64K
UUID : 2cff15d1:e411447b:fd5d4721:03e44022 (local to host lamachine)
Events : 0.611
Number Major Minor RaidDevice State
0 8 3 0 active sync /dev/sda3
1 8 18 1 active sync /dev/sdb2
2 8 34 2 active sync /dev/sdc2
[root@lamachine ~]#
^ permalink raw reply
* Re: RAID5 Performance
From: Adam Goryachev @ 2016-08-02 7:09 UTC (permalink / raw)
To: Peter Grandi, Linux RAID
In-Reply-To: <22426.16080.579831.750402@tree.ty.sabi.co.uk>
On 29/07/16 03:20, Peter Grandi wrote:
> [ ... ]
>
>>> * Replace the flash SSDs with those that are known to deliver
>>> high (at least > 10,000 single threaded) small synchronous
>>> write IOPS.
>> Is there a "known" SSD that you would suggest? My problem is
>> that Intel spec sheets seem to suggest that there is little
>> performance difference across the range of SSD's, so it's
>> really not clear which SSD model I should buy.
> The links I wrote earlier have lists:
Thanks for reminding me of that. I see that the list reflects my
experience (if we assume the 530 model is equivalent to the 535 model on
the list, and my 520 480GB is equivalent to the 520 on the list).
However, I can't get the budget for those really awesome drives up the
top of the list, that would require around $20k... or more.
For now, I've got 16 x 545s TB drives, and have replaced the first half
(ie, all drives in one server). Now I can see that the drives themselves
don't seem to be the bottleneck (the drives don't run at 100% util,
while the DRBD device does run at 100%).
I've written a small script to keep track of the number of seconds each
drive util value fits into each bracket (increments of 10%). Let me know
if you would like a copy (it's just a perl script which reads iostat
output, I'm sure it could be written much nicer).
So far, this is what I get on the secondary (with the new 8 x 845s 1TB
drives):
Drive 10 20 30 40 50 60 70 80
90 100
md1 19265 0 0 0 0 0 0 0
0 0
sda 17029 1579 404 137 49 45 13 4 4 1
sdb 16983 1453 477 179 77 63 22 6 3 2
sdc 16867 1579 492 182 76 40 17 8 1 3
sdd 17043 1499 445 154 59 40 14 6 3 2
sde 17064 1506 415 152 68 32 15 4 6 3
sdf 17138 1467 396 152 46 37 11 10 4 4
sdg 17118 1493 401 139 56 31 14 7 2 4
sdh 16997 1577 407 138 62 45 11 10 7 6
sdi 19236 12 4 4 2 0 2
3 0 1
Hopefully that will line up right !
So, out of the last 19265 seconds, each of the underlying drives was at
100% for only a couple of seconds (sdi is the OS drive). ie, the last
column shows the number of seconds the drive was at 90 to 100% util as
reported by iostat. The 10 column shows number of seconds between 0 and
10%, etc...
Looking at the primary, with all 520 series drives (except sda which is
a 545s series) and the DRBD drives I see this:
Drive 10 20 30 40 50 60 70 80
90 100
drbd0 19971 108 54 36 13 2 0 2 1
1
drbd1 19842 165 77 48 34 4 6 5 3
3
drbd10 19766 279 62 35 23 7 6 4
2 1
drbd11 20081 37 32 21 12 1 3 1
0 0
drbd12 20041 79 38 19 9 1 0 0
1 0
drbd13 16195 2335 758 338 220 131 77 39 32 58
drbd14 19765 230 90 49 30 9 4 6 2
1
drbd15 3473 6323 4136 2250 1390 913 614 443 418 220
drbd17 20175 9 1 0 3 0 0
0 0 0
drbd18 19878 170 65 29 23 10 4 0 6 1
drbd19 19255 368 138 86 87 100 39 35 44 35
drbd2 20188 0 0 0 0 0 0
0 0 0
drbd3 17457 1276 610 316 175 140 66 43 33 56
drbd4 20154 17 6 6 5 0 0
0 0 0
drbd5 19859 141 59 38 26 10 4 5 3 42
drbd6 20112 39 20 9 3 1 1 1
1 0
drbd7 20188 0 0 0 0 0 0
0 0 0
drbd8 19894 136 78 44 22 5 3 2 0
2
drbd9 19476 289 211 123 41 21 9 6
3 7
md1 20188 0 0 0 0 0 0
0 0 0
sda 16948 1696 439 286 213 206 316 81 3 0
sdb 16059 2177 844 402 290 352 50 13 1 0
sdc 16141 2132 852 388 312 328 30 5 0 0
sdd 15914 2182 956 395 300 362 72 6 1 0
sde 16099 2137 801 393 256 366 124 10 1 1
sdf 16000 2169 898 408 322 340 39 9 3
0
sdg 15929 2265 822 418 259 290 195 8 2 0
sdh 16107 2129 822 419 324 337 41 9 0 0
sdi 20155 3 3 7 14 6 0
0 0 0
So on the primary, I see even less of a bottleneck on the underlying
drives, which doesn't make a lot of sense to me. The secondary has less
read load (since all reads are handled by the primary), and should only
need to deal with raid rmw. Also, I'm not sure, but I think the
secondary does less meta data updates for DRBD. So I can only presume
the new drives are much better than the 530 series, but still not as
good as the 520 series. I'll need to run some tests before I put the
drives live next time.
However, the point of note is that DRBD devices are showing high util
levels much more frequently than the underlying devices, so I can only
assume that the current limitation is caused by DRBD rather than the
drives. Though probably solving the DRBD issue will then go back to the
drives being the limit, with not a lot of difference. See below for my
(your) ideas on improving both of those things.....
>>>> https://www.sebastien-han.fr/blog/2014/10/10/ceph-how-to-test-if-your-ssd-is-suitable-as-a-journal-device/
>>>> http://www.spinics.net/lists/ceph-users/msg25928.html
>>>> https://www.redhat.com/en/resources/ceph-pcie-ssd-performance-part-1
> As one of those pages says the Samsung SM863 looks attractive,
> but for historical reasons so far I have only seen Intel DCs in
> similar use. There discussions of other models in various posts
> related to Ceph journal SSD usage.
>
>> Obviously it's not something I can afford to buy one of each
>> and test them either.
> In addition to the lists above I have justed tested my three
> home flash SSDs:
>
> * Micron M4 256GB:
> # dd bs=4k count=100000 oflag=direct,dsync if=/dev/zero of=/var/tmp/TEST
> 100000+0 records in
> 100000+0 records out
> 409600000 bytes (410 MB) copied, 1200.3 s, 341 kB/s
> * Samsung 850 Pro 256GB:
> # dd bs=4k count=100000 oflag=direct,dsync if=/dev/zero of=/var/tmp/TEST
> 100000+0 records in
> 100000+0 records out
> 409600000 bytes (410 MB) copied, 1732.93 s, 236 kB/s
> * Hynix SK SH910 256GB:
> # dd bs=4k count=100000 oflag=direct,dsync if=/dev/zero of=/var/tmp/TEST
> 100000+0 records in
> 100000+0 records out
> 409600000 bytes (410 MB) copied, 644.742 s, 635 kB/s
>
> So I would not recommend any of them for "small sync writes"
> workloads :-), but they are quite good otherwise. I do notice
> they are slow on small sync writes when downloading mail, as
> each message is duly 'fsync'ed.
>
> BTW as bonus material, I have done on the SH910 an abbreviated
> test with block sizes between 4KiB and 1024KiB:
>
> # for N in 4k 16k 64k 128k 256k 512k 1024k; do echo -n "$N: "; dd bs=$N count=1000 oflag=dsync if=/dev/zero of=/var/tmp/TEST |& grep copied; done
> 4k: 4096000 bytes (4.1 MB) copied, 6.23481 s, 657 kB/s
> 16k: 16384000 bytes (16 MB) copied, 6.29379 s, 2.6 MB/s
> 64k: 65536000 bytes (66 MB) copied, 6.09223 s, 10.8 MB/s
> 128k: 131072000 bytes (131 MB) copied, 6.5487 s, 20.0 MB/s
> 256k: 262144000 bytes (262 MB) copied, 6.93361 s, 37.8 MB/s
> 512k: 524288000 bytes (524 MB) copied, 7.73957 s, 67.7 MB/s
> 1024k: 1048576000 bytes (1.0 GB) copied, 12.8671 s, 81.5 MB/s
>
> Note how the time to write 1000 blocks is essentially the same
> betweeen 4KiB and 128KiB, which is quite amusing. Probably the
> flash-page size is around 256KiB.
>
> For additional bonus value the same on a "fastish" consumer 2TB
> disk, a Seagate ST2000DM001:
>
> # for N in 4k 16k 64k 128k 256k 512k 1024k; do echo -n "$N: "; dd bs=$N count=1000 oflag=dsync if=/dev/zero of=/fs/sdb6/tmp/TEST |& grep copied; done
> 4k: 4096000 bytes (4.1 MB) copied, 44.9177 s, 91.2 kB/s
> 16k: 16384000 bytes (16 MB) copied, 38.131 s, 430 kB/s
> 64k: 65536000 bytes (66 MB) copied, 35.8263 s, 1.8 MB/s
> 128k: 131072000 bytes (131 MB) copied, 35.8188 s, 3.7 MB/s
> 256k: 262144000 bytes (262 MB) copied, 36.6838 s, 7.1 MB/s
> 512k: 524288000 bytes (524 MB) copied, 37.0612 s, 14.1 MB/s
> 1024k: 1048576000 bytes (1.0 GB) copied, 42.0844 s, 24.9 MB/s
>
Yep, definitely won't be going backwards to spinning disks :)
>>> * Relax the requirement for synchronous writes on *both* the
>>> primary and secondary DRBD servers, if feeling lucky.
>> I have the following entries for DRBD which were suggested by
>> linbit (which previously lifted performance from abysmal to
>> more than sufficient around 2+ years ago). [ ... ]
> That's an inappropriate use of "performance" here:
>
>> disk-barrier no;
>> disk-flushes no;
>> md-flushes no;
> That "feeling lucky" list seems to me to have made performance
> lower (in the sense that the performance of writing to
> '/dev/null' is zero, even if the speed is really good :->).
>
> With those settings the data sync policy is "disk-drain", which
> also involves some waiting, but somewhat dangerous, except "In
> case your backing storage device has battery-backed write cache"
> (and "device" here means system and host adapter and disk); it
> is not clear to me for metadata what "md-flushes no" gives.
>
> BTW if you have battery-backed everything on the secondary side
> you could use protocol "B".
From my understanding, the times these settings can cause a problem:
1) When both servers hard power off - possibly all the latest data is
not written to disk that the VM's expect. If this is the case, all the
VM's were also hard powered off, and so the VM has no idea about what it
expects to be written/not. The end user may need to redo some work/etc,
but that is acceptable. Worst case scenario, a DB file is corrupted and
needs to be restored from the previous night backup, and users must redo
all work, which is also "acceptable" (from a risk point of view).
2) One server hard power off, perhaps power supply failure/etc - When it
powers on again, it should re-sync with the DRBD primary, and
potentially we do a DRBD verify to confirm everything is good. As long
as there is no failure on the primary, then everything is good. Worst
case, catastrophic failure of the primary before the verify is complete,
or before the secondary comes on-line again, and basically we treat it
as above.
We can't deal with every possible scenario, as the cost is prohibitive,
we can only deal with the more common scenarios, and those that are
cheaper to deal with. eg, all equipment is protected by UPS, using
redundancy RAID instead of linear/striping, and using DRBD for
replication. Most likely failures are disk, power supply, or network
cables (ie, unplugged by accident/etc), and this setup protects well for
all three of those.
> However given those it looks likely that the bottleneck is also
> on the primary DRBD side.
>
>> Do you have any other suggestions or ideas that might assist?
> * Smaller RAID5 stripes, as in 4+1 or 2+1, are cheaper in space
> than RAID10 and enormously raise the chances that a full
> stripe-write can happen (it still has the write-hole problem
> of parity RAID).
I was planning to upgrade to the 4.4.x kernel, which would kind of solve
this, since it will only read from 2 drives anyway, but it turns out
that is more difficult than I expected. (iscsitarget kernel module
doesn't compile cleanly with the new kernel, and it doesn't seem to be
well supported into such recent kernel versions. I'll probably wait
until debian testing becomes stable, or at least a lot closer, before
going down that path).
I could potentially move to 2 x RAID5 with 3+1 and then linear or stripe
those, which means I only lose one extra disk of capacity.... Will need
to think about that further...
> * Make sure the DRBD journal is also on a separate device that
> allows fast small sync writes.
I think this would be the next option to investigate. Currently the DRBD
journal is on the same devices.
Reading from:
http://www.drbd.org/en/doc/users-guide-84/ch-internals#s-internal-meta-data
>
> *Advantage. *For some write operations, using external meta data
> produces a somewhat improved latency behavior.
>
Do you have any more knowledge on the expected performance advantage?
ie, would half the writes move from the data drive to the meta data drive?
I'm thinking it might be plausible to purchase 2 x Intel P3700 400GB and
put one in each DRBD server for the meta data updates. Although if this
isn't going to make much difference (eg, only 20%) then it is less
likely to be worthwhile...
Can anyone suggest what kind of performance improvement might I see by
doing this?
The alternative (for double the cost + a bit more) would be to migrate
from RAID5 to RAID10, is that likely to produce a better/worse result?
2 x P3700 400GB is probably around $2500, while 12 x 545s 1000GB is
around $4800, but would need to add another SATA controller card, which
probably means changing motherboard/CPU/etc as well, so that becomes a
lot more....
> Also, I have appended a sample DRBD configuration I have used:
>
> ----------------------------------------------------------------
>
> # http://article.gmane.org/gmane.linux.network.drbd/18348
> # http://www.drbd.org/users-guide-8.3/s-throughput-tuning.html
> # https://alteeve.ca/w/AN!Cluster_Tutorial_2_-_Performance_Tuning
> # http://fghaas.wordpress.com/2007/06/22/performance-tuning-drbd-setups/
> sndbuf-size 0;
> rcvbuf-size 0;
> max-buffers 16384;
> unplug-watermark 16384;
> max-epoch-size 16384;
I have similar values, but will need to investigate the above options
further. rcvbuf-size doesn't seem to be well documented, at least in the
DRBD 8.4 manual, but will research these some more. Then will also need
to check how to modify the values without causing a system meltdown....
Thanks again for your advice/information, it is very helpful.
Regards,
Adam
--
Adam Goryachev Website Managers www.websitemanagers.com.au
^ permalink raw reply
* Re: [PATCH 4/8] md-cluster: introduce dlm_lock_sync_interruptible to fix tasks hang
From: Guoqing Jiang @ 2016-08-02 3:24 UTC (permalink / raw)
To: Shaohua Li; +Cc: linux-raid
In-Reply-To: <20160801222042.GB18810@kernel.org>
On 08/01/2016 06:20 PM, Shaohua Li wrote:
> On Thu, Jul 28, 2016 at 02:16:48AM -0400, Guoqing Jiang wrote:
>> When some node leaves cluster, then it's bitmap need to be
>> synced by another node, so "md*_recover" thread is triggered
>> for the purpose. However, with below steps. we can find tasks
>> hang happened either in B or C.
>>
>> 1. Node A create a resyncing cluster raid1, assemble it in
>> other two nodes (B and C).
>> 2. stop array in B and C.
>> 3. stop array in A.
>>
>> linux44:~ # ps aux|grep md|grep D
>> root 5938 0.0 0.1 19852 1964 pts/0 D+ 14:52 0:00 mdadm -S md0
>> root 5939 0.0 0.0 0 0 ? D 14:52 0:00 [md0_recover]
>>
>> linux44:~ # cat /proc/5939/stack
>> [<ffffffffa04cf321>] dlm_lock_sync+0x71/0x90 [md_cluster]
>> [<ffffffffa04d0705>] recover_bitmaps+0x125/0x220 [md_cluster]
>> [<ffffffffa052105d>] md_thread+0x16d/0x180 [md_mod]
>> [<ffffffff8107ad94>] kthread+0xb4/0xc0
>> [<ffffffff8152a518>] ret_from_fork+0x58/0x90
>>
>> linux44:~ # cat /proc/5938/stack
>> [<ffffffff8107afde>] kthread_stop+0x6e/0x120
>> [<ffffffffa0519da0>] md_unregister_thread+0x40/0x80 [md_mod]
>> [<ffffffffa04cfd20>] leave+0x70/0x120 [md_cluster]
>> [<ffffffffa0525e24>] md_cluster_stop+0x14/0x30 [md_mod]
>> [<ffffffffa05269ab>] bitmap_free+0x14b/0x150 [md_mod]
>> [<ffffffffa0523f3b>] do_md_stop+0x35b/0x5a0 [md_mod]
>> [<ffffffffa0524e83>] md_ioctl+0x873/0x1590 [md_mod]
>> [<ffffffff81288464>] blkdev_ioctl+0x214/0x7d0
>> [<ffffffff811dd3dd>] block_ioctl+0x3d/0x40
>> [<ffffffff811b92d4>] do_vfs_ioctl+0x2d4/0x4b0
>> [<ffffffff811b9538>] SyS_ioctl+0x88/0xa0
>> [<ffffffff8152a5c9>] system_call_fastpath+0x16/0x1b
>>
>> The problem is caused by recover_bitmaps can't reliably abort
>> when the thread is unregistered. So dlm_lock_sync_interruptible
>> is introduced to detect the thread's situation to fix the problem.
>>
>> Reviewed-by: NeilBrown <neilb@suse.com>
>> Signed-off-by: Guoqing Jiang <gqjiang@suse.com>
>> ---
>> drivers/md/md-cluster.c | 38 +++++++++++++++++++++++++++++++++++++-
>> 1 file changed, 37 insertions(+), 1 deletion(-)
>>
>> diff --git a/drivers/md/md-cluster.c b/drivers/md/md-cluster.c
>> index ea2699e..f3d584e 100644
>> --- a/drivers/md/md-cluster.c
>> +++ b/drivers/md/md-cluster.c
>> @@ -10,6 +10,8 @@
>>
>>
>> #include <linux/module.h>
>> +#include <linux/completion.h>
>> +#include <linux/kthread.h>
>> #include <linux/dlm.h>
>> #include <linux/sched.h>
>> #include <linux/raid/md_p.h>
>> @@ -141,6 +143,40 @@ static int dlm_unlock_sync(struct dlm_lock_resource *res)
>> return dlm_lock_sync(res, DLM_LOCK_NL);
>> }
>>
>> +/* An variation of dlm_lock_sync, which make lock request could
>> + * be interrupted */
>> +static int dlm_lock_sync_interruptible(struct dlm_lock_resource *res, int mode,
>> + struct mddev *mddev)
>> +{
>> + int ret = 0;
>> +
>> + ret = dlm_lock(res->ls, mode, &res->lksb,
>> + res->flags, res->name, strlen(res->name),
>> + 0, sync_ast, res, res->bast);
>> + if (ret)
>> + return ret;
>> +
>> + wait_event(res->completion.wait,
>> + res->completion.done || kthread_should_stop());
> can you convert it to a waitq? Directly using the .wait/.done of completion is
> really intrusive.
Maybe, but we still need completion for dlm_lock_resource otherwise there
are different types of dlm_lock_resource, we also need to keep align with
sync_ast as dlm_lock_sync did.
Regards,
Guoqing
>> + if (!res->completion.done) {
>> + /*
>> + * the convert queue contains the lock request when request is
>> + * interrupted, and sync_ast could still be run, so need to
>> + * cancel the request and reset completion
>> + */
>> + ret = dlm_unlock(res->ls, res->lksb.sb_lkid, DLM_LKF_CANCEL, &res->lksb, res);
>> + reinit_completion(&res->completion);
>> + if (unlikely(ret != 0))
>> + pr_info("failed to cancel previous lock request "
>> + "%s return %d\n", res->name, ret);
>> + return -EPERM;
>> + }
>> + wait_for_completion(&res->completion);
>> + if (res->lksb.sb_status == 0)
>> + res->mode = mode;
>> + return res->lksb.sb_status;
>> +}
>> +
>> static struct dlm_lock_resource *lockres_init(struct mddev *mddev,
>> char *name, void (*bastfn)(void *arg, int mode), int with_lvb)
>> {
>> @@ -272,7 +308,7 @@ static void recover_bitmaps(struct md_thread *thread)
>> goto clear_bit;
>> }
>>
>> - ret = dlm_lock_sync(bm_lockres, DLM_LOCK_PW);
>> + ret = dlm_lock_sync_interruptible(bm_lockres, DLM_LOCK_PW, mddev);
>> if (ret) {
>> pr_err("md-cluster: Could not DLM lock %s: %d\n",
>> str, ret);
>> --
>> 2.6.2
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
^ permalink raw reply
* Re: [PATCH 1/8] md-cluster: call md_kick_rdev_from_array once ack failed
From: Guoqing Jiang @ 2016-08-02 3:21 UTC (permalink / raw)
To: Shaohua Li; +Cc: linux-raid
In-Reply-To: <20160801215828.GA18810@kernel.org>
On 08/01/2016 05:58 PM, Shaohua Li wrote:
> On Thu, Jul 28, 2016 at 02:16:45AM -0400, Guoqing Jiang wrote:
>> The new_disk_ack could return failure if WAITING_FOR_NEWDISK
>> is not set, so we need to kick the dev from array in case
>> failure happened.
>>
>> Reviewed-by: NeilBrown <neilb@suse.com>
>> Signed-off-by: Guoqing Jiang <gqjiang@suse.com>
>> ---
>> drivers/md/md.c | 8 +++++---
>> 1 file changed, 5 insertions(+), 3 deletions(-)
>>
>> diff --git a/drivers/md/md.c b/drivers/md/md.c
>> index 2ed547f..743cd21 100644
>> --- a/drivers/md/md.c
>> +++ b/drivers/md/md.c
>> @@ -6092,9 +6092,11 @@ static int add_new_disk(struct mddev *mddev, mdu_disk_info_t *info)
>> export_rdev(rdev);
>>
>> if (mddev_is_clustered(mddev)) {
>> - if (info->state & (1 << MD_DISK_CANDIDATE))
>> - md_cluster_ops->new_disk_ack(mddev, (err == 0));
>> - else {
>> + if (info->state & (1 << MD_DISK_CANDIDATE)) {
> if err != 0, we already do export_rdev, do we need to do md_kick_rdev_from_array in that case?
I suppose you mean the export_rdev before new_disk_ack, it doesn't need
to call md_kick_rdev_from_array
since we don't bind rdev to array successfully.
err = bind_rdev_to_array(rdev, mddev);
if (err)
*export_rdev*(rdev);
if (mddev_is_clustered(mddev)) {
if (info->state & (1 << MD_DISK_CANDIDATE))
md_cluster_ops->new_disk_ack(mddev, (err == 0));
But it is possible that new_disk_ack returns -EINVAL, and the rdev is
binded with array, so we should call
md_kick_rdev_from_array for the case.
Thanks,
Guoqing
^ permalink raw reply
* Re: [PATCH 6/8] md-cluster: make resync lock also could be interruptted
From: Guoqing Jiang @ 2016-08-02 1:38 UTC (permalink / raw)
To: Shaohua Li; +Cc: linux-raid
In-Reply-To: <20160801222946.GC18810@kernel.org>
On 08/01/2016 06:29 PM, Shaohua Li wrote:
> On Thu, Jul 28, 2016 at 02:16:50AM -0400, Guoqing Jiang wrote:
>> When one node is perform resync or recovery, other nodes
>> can't get resync lock and could block for a while before
>> it holds the lock, so we can't stop array immediately for
>> this scenario.
>>
>> To make array could be stop quickly, we check MD_CLOSING
>> in dlm_lock_sync_interruptible to make us can interrupt
>> the lock request.
> if the thread is waiting for the resync lock and you set MD_CLOSING, where do
> you wake up the thread?
If array needs to do resync, it would call md_cluster_ops->resync_start
within md_do_sync. And since MD_CLOSING is set when stop md, we
can see do_md_stop would wake up sync_thread.
Thanks,
Guoqing
^ permalink raw reply
* Re: [PATCH 6/8] md-cluster: make resync lock also could be interruptted
From: Shaohua Li @ 2016-08-01 22:29 UTC (permalink / raw)
To: Guoqing Jiang; +Cc: linux-raid
In-Reply-To: <1469686612-16126-6-git-send-email-gqjiang@suse.com>
On Thu, Jul 28, 2016 at 02:16:50AM -0400, Guoqing Jiang wrote:
> When one node is perform resync or recovery, other nodes
> can't get resync lock and could block for a while before
> it holds the lock, so we can't stop array immediately for
> this scenario.
>
> To make array could be stop quickly, we check MD_CLOSING
> in dlm_lock_sync_interruptible to make us can interrupt
> the lock request.
if the thread is waiting for the resync lock and you set MD_CLOSING, where do
you wake up the thread?
> Reviewed-by: NeilBrown <neilb@suse.com>
> Signed-off-by: Guoqing Jiang <gqjiang@suse.com>
> ---
> drivers/md/md-cluster.c | 5 +++--
> 1 file changed, 3 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/md/md-cluster.c b/drivers/md/md-cluster.c
> index f3d584e..0984771 100644
> --- a/drivers/md/md-cluster.c
> +++ b/drivers/md/md-cluster.c
> @@ -157,7 +157,8 @@ static int dlm_lock_sync_interruptible(struct dlm_lock_resource *res, int mode,
> return ret;
>
> wait_event(res->completion.wait,
> - res->completion.done || kthread_should_stop());
> + res->completion.done || kthread_should_stop()
> + || test_bit(MD_CLOSING, &mddev->flags));
> if (!res->completion.done) {
> /*
> * the convert queue contains the lock request when request is
> @@ -1026,7 +1027,7 @@ static void metadata_update_cancel(struct mddev *mddev)
> static int resync_start(struct mddev *mddev)
> {
> struct md_cluster_info *cinfo = mddev->cluster_info;
> - return dlm_lock_sync(cinfo->resync_lockres, DLM_LOCK_EX);
> + return dlm_lock_sync_interruptible(cinfo->resync_lockres, DLM_LOCK_EX, mddev);
> }
>
> static int resync_info_update(struct mddev *mddev, sector_t lo, sector_t hi)
> --
> 2.6.2
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: [PATCH 4/8] md-cluster: introduce dlm_lock_sync_interruptible to fix tasks hang
From: Shaohua Li @ 2016-08-01 22:20 UTC (permalink / raw)
To: Guoqing Jiang; +Cc: linux-raid
In-Reply-To: <1469686612-16126-4-git-send-email-gqjiang@suse.com>
On Thu, Jul 28, 2016 at 02:16:48AM -0400, Guoqing Jiang wrote:
> When some node leaves cluster, then it's bitmap need to be
> synced by another node, so "md*_recover" thread is triggered
> for the purpose. However, with below steps. we can find tasks
> hang happened either in B or C.
>
> 1. Node A create a resyncing cluster raid1, assemble it in
> other two nodes (B and C).
> 2. stop array in B and C.
> 3. stop array in A.
>
> linux44:~ # ps aux|grep md|grep D
> root 5938 0.0 0.1 19852 1964 pts/0 D+ 14:52 0:00 mdadm -S md0
> root 5939 0.0 0.0 0 0 ? D 14:52 0:00 [md0_recover]
>
> linux44:~ # cat /proc/5939/stack
> [<ffffffffa04cf321>] dlm_lock_sync+0x71/0x90 [md_cluster]
> [<ffffffffa04d0705>] recover_bitmaps+0x125/0x220 [md_cluster]
> [<ffffffffa052105d>] md_thread+0x16d/0x180 [md_mod]
> [<ffffffff8107ad94>] kthread+0xb4/0xc0
> [<ffffffff8152a518>] ret_from_fork+0x58/0x90
>
> linux44:~ # cat /proc/5938/stack
> [<ffffffff8107afde>] kthread_stop+0x6e/0x120
> [<ffffffffa0519da0>] md_unregister_thread+0x40/0x80 [md_mod]
> [<ffffffffa04cfd20>] leave+0x70/0x120 [md_cluster]
> [<ffffffffa0525e24>] md_cluster_stop+0x14/0x30 [md_mod]
> [<ffffffffa05269ab>] bitmap_free+0x14b/0x150 [md_mod]
> [<ffffffffa0523f3b>] do_md_stop+0x35b/0x5a0 [md_mod]
> [<ffffffffa0524e83>] md_ioctl+0x873/0x1590 [md_mod]
> [<ffffffff81288464>] blkdev_ioctl+0x214/0x7d0
> [<ffffffff811dd3dd>] block_ioctl+0x3d/0x40
> [<ffffffff811b92d4>] do_vfs_ioctl+0x2d4/0x4b0
> [<ffffffff811b9538>] SyS_ioctl+0x88/0xa0
> [<ffffffff8152a5c9>] system_call_fastpath+0x16/0x1b
>
> The problem is caused by recover_bitmaps can't reliably abort
> when the thread is unregistered. So dlm_lock_sync_interruptible
> is introduced to detect the thread's situation to fix the problem.
>
> Reviewed-by: NeilBrown <neilb@suse.com>
> Signed-off-by: Guoqing Jiang <gqjiang@suse.com>
> ---
> drivers/md/md-cluster.c | 38 +++++++++++++++++++++++++++++++++++++-
> 1 file changed, 37 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/md/md-cluster.c b/drivers/md/md-cluster.c
> index ea2699e..f3d584e 100644
> --- a/drivers/md/md-cluster.c
> +++ b/drivers/md/md-cluster.c
> @@ -10,6 +10,8 @@
>
>
> #include <linux/module.h>
> +#include <linux/completion.h>
> +#include <linux/kthread.h>
> #include <linux/dlm.h>
> #include <linux/sched.h>
> #include <linux/raid/md_p.h>
> @@ -141,6 +143,40 @@ static int dlm_unlock_sync(struct dlm_lock_resource *res)
> return dlm_lock_sync(res, DLM_LOCK_NL);
> }
>
> +/* An variation of dlm_lock_sync, which make lock request could
> + * be interrupted */
> +static int dlm_lock_sync_interruptible(struct dlm_lock_resource *res, int mode,
> + struct mddev *mddev)
> +{
> + int ret = 0;
> +
> + ret = dlm_lock(res->ls, mode, &res->lksb,
> + res->flags, res->name, strlen(res->name),
> + 0, sync_ast, res, res->bast);
> + if (ret)
> + return ret;
> +
> + wait_event(res->completion.wait,
> + res->completion.done || kthread_should_stop());
can you convert it to a waitq? Directly using the .wait/.done of completion is
really intrusive.
> + if (!res->completion.done) {
> + /*
> + * the convert queue contains the lock request when request is
> + * interrupted, and sync_ast could still be run, so need to
> + * cancel the request and reset completion
> + */
> + ret = dlm_unlock(res->ls, res->lksb.sb_lkid, DLM_LKF_CANCEL, &res->lksb, res);
> + reinit_completion(&res->completion);
> + if (unlikely(ret != 0))
> + pr_info("failed to cancel previous lock request "
> + "%s return %d\n", res->name, ret);
> + return -EPERM;
> + }
> + wait_for_completion(&res->completion);
> + if (res->lksb.sb_status == 0)
> + res->mode = mode;
> + return res->lksb.sb_status;
> +}
> +
> static struct dlm_lock_resource *lockres_init(struct mddev *mddev,
> char *name, void (*bastfn)(void *arg, int mode), int with_lvb)
> {
> @@ -272,7 +308,7 @@ static void recover_bitmaps(struct md_thread *thread)
> goto clear_bit;
> }
>
> - ret = dlm_lock_sync(bm_lockres, DLM_LOCK_PW);
> + ret = dlm_lock_sync_interruptible(bm_lockres, DLM_LOCK_PW, mddev);
> if (ret) {
> pr_err("md-cluster: Could not DLM lock %s: %d\n",
> str, ret);
> --
> 2.6.2
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: [PATCH 1/8] md-cluster: call md_kick_rdev_from_array once ack failed
From: Shaohua Li @ 2016-08-01 21:58 UTC (permalink / raw)
To: Guoqing Jiang; +Cc: linux-raid
In-Reply-To: <1469686612-16126-1-git-send-email-gqjiang@suse.com>
On Thu, Jul 28, 2016 at 02:16:45AM -0400, Guoqing Jiang wrote:
> The new_disk_ack could return failure if WAITING_FOR_NEWDISK
> is not set, so we need to kick the dev from array in case
> failure happened.
>
> Reviewed-by: NeilBrown <neilb@suse.com>
> Signed-off-by: Guoqing Jiang <gqjiang@suse.com>
> ---
> drivers/md/md.c | 8 +++++---
> 1 file changed, 5 insertions(+), 3 deletions(-)
>
> diff --git a/drivers/md/md.c b/drivers/md/md.c
> index 2ed547f..743cd21 100644
> --- a/drivers/md/md.c
> +++ b/drivers/md/md.c
> @@ -6092,9 +6092,11 @@ static int add_new_disk(struct mddev *mddev, mdu_disk_info_t *info)
> export_rdev(rdev);
>
> if (mddev_is_clustered(mddev)) {
> - if (info->state & (1 << MD_DISK_CANDIDATE))
> - md_cluster_ops->new_disk_ack(mddev, (err == 0));
> - else {
> + if (info->state & (1 << MD_DISK_CANDIDATE)) {
if err != 0, we already do export_rdev, do we need to do md_kick_rdev_from_array in that case?
> + err = md_cluster_ops->new_disk_ack(mddev, (err == 0));
> + if (err)
> + md_kick_rdev_from_array(rdev);
> + } else {
> if (err)
> md_cluster_ops->add_new_disk_cancel(mddev);
> else
> --
> 2.6.2
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: [PATCH 3/3] MD: hold mddev lock for md-cluster receive thread
From: Shaohua Li @ 2016-08-01 21:45 UTC (permalink / raw)
To: Guoqing Jiang; +Cc: linux-raid, NeilBrown
In-Reply-To: <579F0AA3.5090806@suse.com>
On Mon, Aug 01, 2016 at 04:38:59PM +0800, Guoqing Jiang wrote:
> Hi,
>
> On 07/31/2016 07:54 AM, shli@kernel.org wrote:
> >From: Shaohua Li <shli@fb.com>
> >
> >md-cluster receive thread calls .quiesce too, let it hold mddev lock.
>
> I'd suggest hold on for the patchset, I can find lock problem easily with
> the patchset applied. Take a resyncing clusteed raid1 as example.
>
> md127_raid1 thread held reconfig_mutex then update sb, so it needs dlm
> token lock. Meanwhile md127_resync thread got token lock and wants
> EX on ack lock but recv_daemon can't release ack lock since recv_daemon
> doesn't get reconfig_mutex.
Thansk, I'll drop this one. Other two patches are still safe for md-cluster,
right? I really hope to have consistent locking for .quiesce. For the
process_recvd_msg, I'm wondering what's protecting the datas? for example,
md-cluster uses md_find_rdev_nr_rcu, which access the disks list without
locking. Is there a race? Does it work if we move the mddev lock to
process_recvd_msg?
Thanks,
Shaohua
^ permalink raw reply
* Re: xosview
From: Wols Lists @ 2016-08-01 19:25 UTC (permalink / raw)
To: mdraid
In-Reply-To: <579F9A00.6000906@gmail.com>
Thanks very much to all who replied :-)
The author is planning to rewrite based on /sys, however, so mdstat's
are likely no longer required ... the original raid code was donated,
and seems to have bit-rotted :-(
What I've said I'd like to see is the raid name and status - healthy,
rebuilding or degraded on the left; with the constituent
drives/partitions listed on the right, over colour bars indicating their
status (live, rebuilding, failed or spare). The author would also like
to add a status bar indicating the status of any rebuild.
So a healthy raid would be indicated by (default colours) green and
maybe blue for spare drives, while a degraded array would have red
around, and a rebuilding array would have yellow.
Anybody have any other ideas?
I know I'm bad at checking my raid status, and a lot of people probably
let the default install set up raid on a desktop without configuring
notification etc. It's just lovely to have a little utility like xosview
that can sit in the background on your desktop keeping an eye on things.
And that shows up instantly when things start going wrong. It keeps an
eye on my cpus, memory, swap space, and i/o. It could probably keep an
eye on more ...
Cheers,
Wol
On 01/08/16 19:50, Bill Hudacek wrote:
> Anthony Youngman wrote on 07/29/2016 12:52 PM:
>> So if people wouldn't mind, could you email your mdstat files?
>> Preferably on the list so people can see what has and has not been sent
>> - obviously I'd like standard setups like raid10, raid5, raid6, both
>> named and numbered. And if people have them, mdstats showing broken
>> arrays, rebuilds, complicated setups with lvm, etc.
>>
>
> RAID 6 across 5 disks, of which 1 is a spare (in external cabinet), and
> two disks in RAID-1 (for OS, inside the tower):
>
>> cat /proc/mdstat
> Personalities : [raid1] [raid6] [raid5] [raid4]
> md0 : active raid6 sdc1[0] sdf1[3] sdd1[1] sdg1[4](S) sde1[2]
> 3071737856 blocks super 1.2 level 6, 1024k chunk, algorithm 2
> [4/4] [UUUU]
> bitmap: 0/12 pages [0KB], 65536KB chunk
>
> md126 : active raid1 sdb1[1] sda1[0]
> 2099136 blocks super 1.0 [2/2] [UU]
> bitmap: 0/1 pages [0KB], 65536KB chunk
>
> md127 : active raid1 sdb2[1] sda2[0]
> 234921984 blocks super 1.2 [2/2] [UU]
> bitmap: 1/2 pages [4KB], 65536KB chunk
>
> unused devices: <none>
>
> I don't have any failure mdstat output saved, sorry...
>
^ permalink raw reply
* Re: xosview
From: Bill Hudacek @ 2016-08-01 18:50 UTC (permalink / raw)
To: mdraid
In-Reply-To: <ffd06b83-0692-d5a4-1daa-126abf53edfa@youngman.org.uk>
Anthony Youngman wrote on 07/29/2016 12:52 PM:
> So if people wouldn't mind, could you email your mdstat files?
> Preferably on the list so people can see what has and has not been sent
> - obviously I'd like standard setups like raid10, raid5, raid6, both
> named and numbered. And if people have them, mdstats showing broken
> arrays, rebuilds, complicated setups with lvm, etc.
>
RAID 6 across 5 disks, of which 1 is a spare (in external cabinet),
and two disks in RAID-1 (for OS, inside the tower):
> cat /proc/mdstat
Personalities : [raid1] [raid6] [raid5] [raid4]
md0 : active raid6 sdc1[0] sdf1[3] sdd1[1] sdg1[4](S) sde1[2]
3071737856 blocks super 1.2 level 6, 1024k chunk, algorithm 2
[4/4] [UUUU]
bitmap: 0/12 pages [0KB], 65536KB chunk
md126 : active raid1 sdb1[1] sda1[0]
2099136 blocks super 1.0 [2/2] [UU]
bitmap: 0/1 pages [0KB], 65536KB chunk
md127 : active raid1 sdb2[1] sda2[0]
234921984 blocks super 1.2 [2/2] [UU]
bitmap: 1/2 pages [4KB], 65536KB chunk
unused devices: <none>
I don't have any failure mdstat output saved, sorry...
--
/Bill
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox