* Re: [BUG] md: oops sync_page_io+0x38/0x180
From: Ming Lei @ 2017-06-23 11:16 UTC (permalink / raw)
To: linux-raid; +Cc: Shaohua Li
In-Reply-To: <20170623104744.GA5830@ming.t460p>
On Fri, Jun 23, 2017 at 06:47:44PM +0800, Ming Lei wrote:
> Hi,
>
> When I boot a VM, the following kernel oops is triggerd:
>
> [ 4.850206] BUG: unable to handle kernel NULL pointer dereference at 00000000000006a0
> [ 4.851131] IP: sync_page_io+0x38/0x180
> [ 4.851445] PGD 2759bc067
> [ 4.851446] P4D 2759bc067
> [ 4.851621] PUD 277838067
> [ 4.851835] PMD 0
>
> [ 4.852152] Oops: 0000 [#1] PREEMPT SMP
> [ 4.852494] Dumping ftrace buffer:
> [ 4.852758] (ftrace buffer empty)
> [ 4.853062] Modules linked in: nd_pmem psmouse serio_raw ahci libahci floppy nvme nvme_core ib_iser rdma_cm iw_cm ib_cm ib_core configfs iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi null_blk configs autofs4
> [ 4.855190] CPU: 3 PID: 661 Comm: mdadm Not tainted 4.12.0-rc6.quiesce-v3-next-20170623-09867-ga73468728fd8 #248
> [ 4.856310] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.10.2-0-g5f4c7b1-prebuilt.qemu-project.org 04/01/2014
> [ 4.857458] task: ffffa33bf749b340 task.stack: ffffa667812e8000
> [ 4.857992] RIP: 0010:sync_page_io+0x38/0x180
> [ 4.858349] RSP: 0018:ffffa667812ebb18 EFLAGS: 00010297
> [ 4.858804] RAX: 0000000000000000 RBX: 0000000000000000 RCX: ffffd43fc9da2100
> [ 4.859465] RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffffa33bf740d800
> [ 4.860137] RBP: ffffa667812ebb50 R08: 0000000000000000 R09: 0000000000000000
> [ 4.860774] R10: 0000000023386c39 R11: 0000000000000000 R12: ffffa33bf740d800
> [ 4.861817] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000001
> [ 4.863150] FS: 00007f0aa0304700(0000) GS:ffffa33bfac00000(0000) knlGS:0000000000000000
> [ 4.864412] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [ 4.865001] CR2: 00000000000006a0 CR3: 0000000276883000 CR4: 00000000003406e0
> [ 4.865663] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 4.866332] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
> [ 4.867025] Call Trace:
> [ 4.867178] read_disk_sb.constprop.66+0x72/0xc0
> [ 4.867572] super_1_load+0xb7/0x7a0
> [ 4.867897] ? blkdev_get_by_dev+0x58/0x70
> [ 4.868222] ? lock_rdev+0x6c/0xb0
> [ 4.868476] md_import_device+0x187/0x230
> [ 4.868851] ? check_preemption_disabled+0x35/0x120
> [ 4.869266] add_new_disk+0xc2/0x760
> [ 4.869541] md_ioctl+0x1fc7/0x23a0
> [ 4.869824] ? __might_fault+0x67/0xd0
> [ 4.870109] ? autostart_arrays+0x710/0x710
> [ 4.870451] blkdev_ioctl+0x5b8/0xbd0
> [ 4.870750] block_ioctl+0x61/0x80
> [ 4.871003] ? blkdev_fallocate+0x240/0x240
> [ 4.871371] do_vfs_ioctl+0xb0/0x890
> [ 4.871640] ? check_preemption_disabled+0x35/0x120
> [ 4.872061] ? entry_SYSCALL_64_fastpath+0x5/0xc2
> [ 4.872463] ? __this_cpu_preempt_check+0x1c/0x20
> [ 4.872867] ? __fget_light+0x56/0xb0
> [ 4.873148] ? security_file_ioctl+0x62/0x80
> [ 4.873498] SyS_ioctl+0x94/0xc0
> [ 4.873735] entry_SYSCALL_64_fastpath+0x23/0xc2
> [ 4.874121] RIP: 0033:0x7f0a9fe26687
> [ 4.874391] RSP: 002b:00007ffca16ec628 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
> [ 4.875105] RAX: ffffffffffffffda RBX: 00007ffca16efa6b RCX: 00007f0a9fe26687
> [ 4.875779] RDX: 00007ffca16eca38 RSI: 0000000040140921 RDI: 0000000000000004
> [ 4.876544] RBP: ffffffffa35cbbcc R08: 0000000000200000 R09: 0000000000000014
> [ 4.877253] R10: 0000000000000361 R11: 0000000000000246 R12: ffffa667812ebf88
> [ 4.877968] R13: 0000000000000004 R14: 00007ffca16ec8cd R15: 0000000001db35b0
> [ 4.878657] ? __this_cpu_preempt_check+0x1c/0x20
> [ 4.879341] Code: 41 55 41 54 49 89 fc 53 49 89 f5 45 89 ce 48 83 ec 10 89 55 d0 48 89 4d c8 44 89 45 d4 44 8b 7d 10 e8 7d 63 91 ff 49 8b 44 24 18 <48> 8b 98 a0 06 00 00 48 85 db 0f 84 12 01 00 00 e8 63 63 91 ff
> [ 4.885691] RIP: sync_page_io+0x38/0x180 RSP: ffffa667812ebb18
> [ 4.886208] CR2: 00000000000006a0
> [ 4.886441] ---[ end trace 5e7fbbf3076a4aab ]---
Looks this oops is caused by the following patch:
5a85071c2cbc md: use a separate bio_set for synchronous IO.
Once it is reverted, my VM boots successfully without this oops.
Thanks,
Ming
^ permalink raw reply
* Re: [BUG] md: oops sync_page_io+0x38/0x180
From: Shaohua Li @ 2017-06-23 16:24 UTC (permalink / raw)
To: Ming Lei; +Cc: linux-raid, neilb
In-Reply-To: <20170623111601.GB5830@ming.t460p>
On Fri, Jun 23, 2017 at 07:16:02PM +0800, Ming Lei wrote:
> On Fri, Jun 23, 2017 at 06:47:44PM +0800, Ming Lei wrote:
> > Hi,
> >
> > When I boot a VM, the following kernel oops is triggerd:
> >
> > [ 4.850206] BUG: unable to handle kernel NULL pointer dereference at 00000000000006a0
> > [ 4.851131] IP: sync_page_io+0x38/0x180
> > [ 4.851445] PGD 2759bc067
> > [ 4.851446] P4D 2759bc067
> > [ 4.851621] PUD 277838067
> > [ 4.851835] PMD 0
> >
> > [ 4.852152] Oops: 0000 [#1] PREEMPT SMP
> > [ 4.852494] Dumping ftrace buffer:
> > [ 4.852758] (ftrace buffer empty)
> > [ 4.853062] Modules linked in: nd_pmem psmouse serio_raw ahci libahci floppy nvme nvme_core ib_iser rdma_cm iw_cm ib_cm ib_core configfs iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi null_blk configs autofs4
> > [ 4.855190] CPU: 3 PID: 661 Comm: mdadm Not tainted 4.12.0-rc6.quiesce-v3-next-20170623-09867-ga73468728fd8 #248
> > [ 4.856310] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.10.2-0-g5f4c7b1-prebuilt.qemu-project.org 04/01/2014
> > [ 4.857458] task: ffffa33bf749b340 task.stack: ffffa667812e8000
> > [ 4.857992] RIP: 0010:sync_page_io+0x38/0x180
> > [ 4.858349] RSP: 0018:ffffa667812ebb18 EFLAGS: 00010297
> > [ 4.858804] RAX: 0000000000000000 RBX: 0000000000000000 RCX: ffffd43fc9da2100
> > [ 4.859465] RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffffa33bf740d800
> > [ 4.860137] RBP: ffffa667812ebb50 R08: 0000000000000000 R09: 0000000000000000
> > [ 4.860774] R10: 0000000023386c39 R11: 0000000000000000 R12: ffffa33bf740d800
> > [ 4.861817] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000001
> > [ 4.863150] FS: 00007f0aa0304700(0000) GS:ffffa33bfac00000(0000) knlGS:0000000000000000
> > [ 4.864412] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> > [ 4.865001] CR2: 00000000000006a0 CR3: 0000000276883000 CR4: 00000000003406e0
> > [ 4.865663] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> > [ 4.866332] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
> > [ 4.867025] Call Trace:
> > [ 4.867178] read_disk_sb.constprop.66+0x72/0xc0
> > [ 4.867572] super_1_load+0xb7/0x7a0
> > [ 4.867897] ? blkdev_get_by_dev+0x58/0x70
> > [ 4.868222] ? lock_rdev+0x6c/0xb0
> > [ 4.868476] md_import_device+0x187/0x230
> > [ 4.868851] ? check_preemption_disabled+0x35/0x120
> > [ 4.869266] add_new_disk+0xc2/0x760
> > [ 4.869541] md_ioctl+0x1fc7/0x23a0
> > [ 4.869824] ? __might_fault+0x67/0xd0
> > [ 4.870109] ? autostart_arrays+0x710/0x710
> > [ 4.870451] blkdev_ioctl+0x5b8/0xbd0
> > [ 4.870750] block_ioctl+0x61/0x80
> > [ 4.871003] ? blkdev_fallocate+0x240/0x240
> > [ 4.871371] do_vfs_ioctl+0xb0/0x890
> > [ 4.871640] ? check_preemption_disabled+0x35/0x120
> > [ 4.872061] ? entry_SYSCALL_64_fastpath+0x5/0xc2
> > [ 4.872463] ? __this_cpu_preempt_check+0x1c/0x20
> > [ 4.872867] ? __fget_light+0x56/0xb0
> > [ 4.873148] ? security_file_ioctl+0x62/0x80
> > [ 4.873498] SyS_ioctl+0x94/0xc0
> > [ 4.873735] entry_SYSCALL_64_fastpath+0x23/0xc2
> > [ 4.874121] RIP: 0033:0x7f0a9fe26687
> > [ 4.874391] RSP: 002b:00007ffca16ec628 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
> > [ 4.875105] RAX: ffffffffffffffda RBX: 00007ffca16efa6b RCX: 00007f0a9fe26687
> > [ 4.875779] RDX: 00007ffca16eca38 RSI: 0000000040140921 RDI: 0000000000000004
> > [ 4.876544] RBP: ffffffffa35cbbcc R08: 0000000000200000 R09: 0000000000000014
> > [ 4.877253] R10: 0000000000000361 R11: 0000000000000246 R12: ffffa667812ebf88
> > [ 4.877968] R13: 0000000000000004 R14: 00007ffca16ec8cd R15: 0000000001db35b0
> > [ 4.878657] ? __this_cpu_preempt_check+0x1c/0x20
> > [ 4.879341] Code: 41 55 41 54 49 89 fc 53 49 89 f5 45 89 ce 48 83 ec 10 89 55 d0 48 89 4d c8 44 89 45 d4 44 8b 7d 10 e8 7d 63 91 ff 49 8b 44 24 18 <48> 8b 98 a0 06 00 00 48 85 db 0f 84 12 01 00 00 e8 63 63 91 ff
> > [ 4.885691] RIP: sync_page_io+0x38/0x180 RSP: ffffa667812ebb18
> > [ 4.886208] CR2: 00000000000006a0
> > [ 4.886441] ---[ end trace 5e7fbbf3076a4aab ]---
>
> Looks this oops is caused by the following patch:
>
> 5a85071c2cbc md: use a separate bio_set for synchronous IO.
>
> Once it is reverted, my VM boots successfully without this oops.
Thanks for the report, this should fix it:
commit 7f053a6a745557b3f3ad63e9d28ba85c3c0b1563
Author: Shaohua Li <shli@fb.com>
Date: Fri Jun 23 09:19:49 2017 -0700
MD: fix a null dereference
rdev->mddev could be null in start time.
Reported-by: Ming Lei <ming.lei@redhat.com>
Fix: 5a85071c2cbc(md: use a separate bio_set for synchronous IO.)
Cc: NeilBrown <neilb@suse.com>
Signed-off-by: Shaohua Li <shli@fb.com>
diff --git a/drivers/md/md.c b/drivers/md/md.c
index 65ad837aeb54..092b48f8095e 100644
--- a/drivers/md/md.c
+++ b/drivers/md/md.c
@@ -205,7 +205,7 @@ EXPORT_SYMBOL_GPL(bio_alloc_mddev);
static struct bio *md_bio_alloc_sync(struct mddev *mddev)
{
- if (!mddev->sync_set)
+ if (!mddev || !mddev->sync_set)
return bio_alloc(GFP_NOIO, 1);
return bio_alloc_bioset(GFP_NOIO, 1, mddev->sync_set);
^ permalink raw reply related
* Re: [PATCH v6 0/2] IV Generation algorithms for dm-crypt
From: Eric Biggers @ 2017-06-23 18:21 UTC (permalink / raw)
To: Herbert Xu
Cc: Binoy Jayan, broonie, arnd, linux-crypto, linux-kernel, dm-devel,
linux-raid, rnayak, David Gstir
In-Reply-To: <20170623081341.GA29748@gondor.apana.org.au>
On Fri, Jun 23, 2017 at 04:13:41PM +0800, Herbert Xu wrote:
> Binoy Jayan <binoy.jayan@linaro.org> wrote:
> > ===============================================================================
> > dm-crypt optimization for larger block sizes
> > ===============================================================================
> >
> > Currently, the iv generation algorithms are implemented in dm-crypt.c. The goal
> > is to move these algorithms from the dm layer to the kernel crypto layer by
> > implementing them as template ciphers so they can be used in relation with
> > algorithms like aes, and with multiple modes like cbc, ecb etc. As part of this
> > patchset, the iv-generation code is moved from the dm layer to the crypto layer
> > and adapt the dm-layer to send a whole 'bio' (as defined in the block layer)
> > at a time. Each bio contains the in memory representation of physically
> > contiguous disk blocks. Since the bio itself may not be contiguous in main
> > memory, the dm layer sets up a chained scatterlist of these blocks split into
> > physically contiguous segments in memory so that DMA can be performed.
>
> There is currently a patch-set for fscrypt to add essiv support. It
> would be interesting to know whether your implementation of essiv
> can also be used in that patchset. That would confirm that we're on
> the right track.
>
You can find the fscrypt patch at https://patchwork.kernel.org/patch/9795327/
Note that it's encrypting 4096-byte blocks, not 512-byte. Also, it's using
AES-256 for the ESSIV tfm (since it uses a SHA-256 hash) but AES-128 for the
"real" encryption. It's possible this is a mistake and it should be AES-128 for
both. (If it is, it needs to be fixed before it's released in 4.13.)
Eric
^ permalink raw reply
* Re: [BUG] md: oops sync_page_io+0x38/0x180
From: NeilBrown @ 2017-06-23 23:28 UTC (permalink / raw)
To: Shaohua Li, Ming Lei; +Cc: linux-raid
In-Reply-To: <20170623162410.ngf5o2o36axmc67p@kernel.org>
[-- Attachment #1: Type: text/plain, Size: 5810 bytes --]
On Fri, Jun 23 2017, Shaohua Li wrote:
> On Fri, Jun 23, 2017 at 07:16:02PM +0800, Ming Lei wrote:
>> On Fri, Jun 23, 2017 at 06:47:44PM +0800, Ming Lei wrote:
>> > Hi,
>> >
>> > When I boot a VM, the following kernel oops is triggerd:
>> >
>> > [ 4.850206] BUG: unable to handle kernel NULL pointer dereference at 00000000000006a0
>> > [ 4.851131] IP: sync_page_io+0x38/0x180
>> > [ 4.851445] PGD 2759bc067
>> > [ 4.851446] P4D 2759bc067
>> > [ 4.851621] PUD 277838067
>> > [ 4.851835] PMD 0
>> >
>> > [ 4.852152] Oops: 0000 [#1] PREEMPT SMP
>> > [ 4.852494] Dumping ftrace buffer:
>> > [ 4.852758] (ftrace buffer empty)
>> > [ 4.853062] Modules linked in: nd_pmem psmouse serio_raw ahci libahci floppy nvme nvme_core ib_iser rdma_cm iw_cm ib_cm ib_core configfs iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi null_blk configs autofs4
>> > [ 4.855190] CPU: 3 PID: 661 Comm: mdadm Not tainted 4.12.0-rc6.quiesce-v3-next-20170623-09867-ga73468728fd8 #248
>> > [ 4.856310] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.10.2-0-g5f4c7b1-prebuilt.qemu-project.org 04/01/2014
>> > [ 4.857458] task: ffffa33bf749b340 task.stack: ffffa667812e8000
>> > [ 4.857992] RIP: 0010:sync_page_io+0x38/0x180
>> > [ 4.858349] RSP: 0018:ffffa667812ebb18 EFLAGS: 00010297
>> > [ 4.858804] RAX: 0000000000000000 RBX: 0000000000000000 RCX: ffffd43fc9da2100
>> > [ 4.859465] RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffffa33bf740d800
>> > [ 4.860137] RBP: ffffa667812ebb50 R08: 0000000000000000 R09: 0000000000000000
>> > [ 4.860774] R10: 0000000023386c39 R11: 0000000000000000 R12: ffffa33bf740d800
>> > [ 4.861817] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000001
>> > [ 4.863150] FS: 00007f0aa0304700(0000) GS:ffffa33bfac00000(0000) knlGS:0000000000000000
>> > [ 4.864412] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
>> > [ 4.865001] CR2: 00000000000006a0 CR3: 0000000276883000 CR4: 00000000003406e0
>> > [ 4.865663] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
>> > [ 4.866332] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
>> > [ 4.867025] Call Trace:
>> > [ 4.867178] read_disk_sb.constprop.66+0x72/0xc0
>> > [ 4.867572] super_1_load+0xb7/0x7a0
>> > [ 4.867897] ? blkdev_get_by_dev+0x58/0x70
>> > [ 4.868222] ? lock_rdev+0x6c/0xb0
>> > [ 4.868476] md_import_device+0x187/0x230
>> > [ 4.868851] ? check_preemption_disabled+0x35/0x120
>> > [ 4.869266] add_new_disk+0xc2/0x760
>> > [ 4.869541] md_ioctl+0x1fc7/0x23a0
>> > [ 4.869824] ? __might_fault+0x67/0xd0
>> > [ 4.870109] ? autostart_arrays+0x710/0x710
>> > [ 4.870451] blkdev_ioctl+0x5b8/0xbd0
>> > [ 4.870750] block_ioctl+0x61/0x80
>> > [ 4.871003] ? blkdev_fallocate+0x240/0x240
>> > [ 4.871371] do_vfs_ioctl+0xb0/0x890
>> > [ 4.871640] ? check_preemption_disabled+0x35/0x120
>> > [ 4.872061] ? entry_SYSCALL_64_fastpath+0x5/0xc2
>> > [ 4.872463] ? __this_cpu_preempt_check+0x1c/0x20
>> > [ 4.872867] ? __fget_light+0x56/0xb0
>> > [ 4.873148] ? security_file_ioctl+0x62/0x80
>> > [ 4.873498] SyS_ioctl+0x94/0xc0
>> > [ 4.873735] entry_SYSCALL_64_fastpath+0x23/0xc2
>> > [ 4.874121] RIP: 0033:0x7f0a9fe26687
>> > [ 4.874391] RSP: 002b:00007ffca16ec628 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
>> > [ 4.875105] RAX: ffffffffffffffda RBX: 00007ffca16efa6b RCX: 00007f0a9fe26687
>> > [ 4.875779] RDX: 00007ffca16eca38 RSI: 0000000040140921 RDI: 0000000000000004
>> > [ 4.876544] RBP: ffffffffa35cbbcc R08: 0000000000200000 R09: 0000000000000014
>> > [ 4.877253] R10: 0000000000000361 R11: 0000000000000246 R12: ffffa667812ebf88
>> > [ 4.877968] R13: 0000000000000004 R14: 00007ffca16ec8cd R15: 0000000001db35b0
>> > [ 4.878657] ? __this_cpu_preempt_check+0x1c/0x20
>> > [ 4.879341] Code: 41 55 41 54 49 89 fc 53 49 89 f5 45 89 ce 48 83 ec 10 89 55 d0 48 89 4d c8 44 89 45 d4 44 8b 7d 10 e8 7d 63 91 ff 49 8b 44 24 18 <48> 8b 98 a0 06 00 00 48 85 db 0f 84 12 01 00 00 e8 63 63 91 ff
>> > [ 4.885691] RIP: sync_page_io+0x38/0x180 RSP: ffffa667812ebb18
>> > [ 4.886208] CR2: 00000000000006a0
>> > [ 4.886441] ---[ end trace 5e7fbbf3076a4aab ]---
>>
>> Looks this oops is caused by the following patch:
>>
>> 5a85071c2cbc md: use a separate bio_set for synchronous IO.
>>
>> Once it is reverted, my VM boots successfully without this oops.
>
> Thanks for the report, this should fix it:
>
>
> commit 7f053a6a745557b3f3ad63e9d28ba85c3c0b1563
> Author: Shaohua Li <shli@fb.com>
> Date: Fri Jun 23 09:19:49 2017 -0700
>
> MD: fix a null dereference
>
> rdev->mddev could be null in start time.
>
> Reported-by: Ming Lei <ming.lei@redhat.com>
> Fix: 5a85071c2cbc(md: use a separate bio_set for synchronous IO.)
> Cc: NeilBrown <neilb@suse.com>
> Signed-off-by: Shaohua Li <shli@fb.com>
>
> diff --git a/drivers/md/md.c b/drivers/md/md.c
> index 65ad837aeb54..092b48f8095e 100644
> --- a/drivers/md/md.c
> +++ b/drivers/md/md.c
> @@ -205,7 +205,7 @@ EXPORT_SYMBOL_GPL(bio_alloc_mddev);
>
> static struct bio *md_bio_alloc_sync(struct mddev *mddev)
> {
> - if (!mddev->sync_set)
> + if (!mddev || !mddev->sync_set)
> return bio_alloc(GFP_NOIO, 1);
>
> return bio_alloc_bioset(GFP_NOIO, 1, mddev->sync_set);
Yes, Thanks.
I remember removing that test :-( My thinking was exactly backwards.
We really don't need that test in the *old* bio_alloc_mddev() now, but
I removed it from the *new* md_bio_alloc_sync() instead.
Probably simplest to just leave it in bio_alloc_mddev(). It doesn't
hurt.
Sorry...
NeilBrown
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]
^ permalink raw reply
* (unknown),
From: citydesk @ 2017-06-24 0:35 UTC (permalink / raw)
To: linux-raid
[-- Attachment #1: EMAIL_77134398_linux-raid.zip --]
[-- Type: application/zip, Size: 3531 bytes --]
^ permalink raw reply
* can't add a larger spare for a failed smaller disk
From: Hubert Reevez @ 2017-06-24 22:21 UTC (permalink / raw)
To: linux-raid
Hi,
I'm trying to add a larger spare in replacement to a smaller one. I'm
really stuck here.... ;(
Here is what I did : --fail /dev/sdc then --remove /dev/sdc
Added the new one (which is also sdc now) can't assemble the new one !
If I try to assemble here is what it comes :
---------------------------------------------------------
root@Zetta:~# mdadm --assemble /dev/md0 --verbose /dev/sdc
mdadm: looking for devices for /dev/md0
mdadm: /dev/sdc is busy - skipping
-------------------------------------------------------
If I tried to stop the array and assemble, here is what it comes :
--------------------------------
root@Zetta:~# mdadm --assemble /dev/md0 --verbose /dev/sdc
mdadm: looking for devices for /dev/md0
mdadm: /dev/sdc is identified as a member of /dev/md0, slot -1.
mdadm: no uptodate device for slot 0 of /dev/md0
mdadm: no uptodate device for slot 1 of /dev/md0
mdadm: no uptodate device for slot 2 of /dev/md0
mdadm: no uptodate device for slot 3 of /dev/md0
mdadm: no uptodate device for slot 4 of /dev/md0
mdadm: no uptodate device for slot 5 of /dev/md0
mdadm: added /dev/sdc to /dev/md0 as -1
mdadm: /dev/md0 assembled from 0 drives and 1 spare - not enough to
start the array
-------------------------------------------------
Thanks for your help !!!!!
Here some infos (I had a sdg larger than 1,5 to)
------------------------------------
root@Zetta:~# blockdev --getsize64 /dev/sd[abcefg] 1500301910016
2000398934016 2000398934016 1500301910016 1500301910016 2000398934016
---------------------------------------------------------------------------------------------
root@Zetta:~# cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4]
md0 : inactive sdf[0] sdc[7](S) sdg[6] sdb[4] sde[3] sda[2]
10255693264 blocks super 1.2
unused devices: <none>
---------------------------------------------------------------------------------------------
root@Zetta:~# mdadm --detail /dev/md0
/dev/md0:
Version : 1.2
Creation Time : Sat Aug 31 01:10:06 2013
Raid Level : raid6
Used Dev Size : 1465137152 (1397.26 GiB 1500.30 GB)
Raid Devices : 6
Total Devices : 6
Persistence : Superblock is persistent
Update Time : Fri Jun 23 15:43:33 2017
State : active, degraded, Not Started
Active Devices : 5
Working Devices : 6
Failed Devices : 0
Spare Devices : 1
Layout : left-symmetric
Chunk Size : 512K
Name : zetta:0
UUID : 96b0e7b7:83aa203f:1545031b:43caaa85
Events : 54841
Number Major Minor RaidDevice State
0 8 80 0 active sync /dev/sdf
1 0 0 1 removed
2 8 0 2 active sync /dev/sda
3 8 64 3 active sync /dev/sde
4 8 16 4 active sync /dev/sdb
6 8 96 5 active sync /dev/sdg
7 8 32 - spare /dev/sdc
--------------------------------------------------------------------------------------------------------------------
root@Zetta:~# fdisk -l
Disk /dev/sda: 1500.3 GB, 1500301910016 bytes
255 heads, 63 sectors/track, 182401 cylinders, total 2930277168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000
Disk /dev/sda doesn't contain a valid partition table
Disk /dev/sdb: 2000.4 GB, 2000398934016 bytes
255 heads, 63 sectors/track, 243201 cylinders, total 3907029168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk identifier: 0x00000000
Disk /dev/sdb doesn't contain a valid partition table
Disk /dev/sdc: 2000.4 GB, 2000398934016 bytes
255 heads, 63 sectors/track, 243201 cylinders, total 3907029168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk identifier: 0x00000000
Disk /dev/sdc doesn't contain a valid partition table
Disk /dev/sdd: 100.0 GB, 100030242816 bytes
255 heads, 63 sectors/track, 12161 cylinders, total 195371568 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x0008101c
Device Boot Start End Blocks Id System
/dev/sdd1 * 2048 187367423 93682688 83 Linux
/dev/sdd2 187369470 195371007 4000769 5 Extended
/dev/sdd5 187369472 195371007 4000768 82 Linux swap / Solaris
Disk /dev/sde: 1500.3 GB, 1500301910016 bytes
255 heads, 63 sectors/track, 182401 cylinders, total 2930277168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk identifier: 0x00000000
Disk /dev/sde doesn't contain a valid partition table
Disk /dev/sdf: 1500.3 GB, 1500301910016 bytes
255 heads, 63 sectors/track, 182401 cylinders, total 2930277168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk identifier: 0x00000000
Disk /dev/sdf doesn't contain a valid partition table
Disk /dev/sdg: 2000.4 GB, 2000398934016 bytes
255 heads, 63 sectors/track, 243201 cylinders, total 3907029168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk identifier: 0x00000000
Disk /dev/sdg doesn't contain a valid partition table
------------------------------------------------------------------------------------------
root@Zetta:~# blkid
/dev/sda: UUID="96b0e7b7-83aa-203f-1545-031b43caaa85"
UUID_SUB="70983681-b5c2-9f70-1801-948b1b7c97d1" LABEL="zetta:0"
TYPE="linux_raid_member"
/dev/sdb: UUID="96b0e7b7-83aa-203f-1545-031b43caaa85"
UUID_SUB="66843575-1a9c-30a6-8172-29f3ded468dc" LABEL="zetta:0"
TYPE="linux_raid_member"
/dev/sde: UUID="96b0e7b7-83aa-203f-1545-031b43caaa85"
UUID_SUB="ee7eab8f-dc90-e3be-146d-a4e09d104418" LABEL="zetta:0"
TYPE="linux_raid_member"
/dev/sdf: UUID="96b0e7b7-83aa-203f-1545-031b43caaa85"
UUID_SUB="b2460066-00b1-5070-cfe3-7ac67aae96c1" LABEL="zetta:0"
TYPE="linux_raid_member"
/dev/sdc: UUID="96b0e7b7-83aa-203f-1545-031b43caaa85"
UUID_SUB="06bd9fd3-7b1a-b0e0-b9f0-029882f39624" LABEL="zetta:0"
TYPE="linux_raid_member"
/dev/sdd1: UUID="b364f189-c9af-4774-92ba-a49307966cf7" TYPE="ext4"
/dev/sdd5: UUID="c78b6a64-1caa-4290-bc1a-eebe67731ca5" TYPE="swap"
/dev/sdg: UUID="96b0e7b7-83aa-203f-1545-031b43caaa85"
UUID_SUB="440f9a88-1d42-020c-ba4d-d303afb76c7c" LABEL="zetta:0"
TYPE="linux_raid_member"
------------------------------------------------------------------------------------------------------------
mdadm -E /dev/sd[abcefg]
https://pastebin.com/zzz3jFvP
-------------------------------------------------------------------------------------------------------------------
mdadm.conf :
DEVICE partitions CREATE owner=root group=disk mode=0660 auto=yes
HOMEHOST <system> ARRAY /dev/md/zetta:0 metadata=1.2 name=zetta:0
UUID=96b0e7b7:83aa203f:1545031b:43caaa85
^ permalink raw reply
* Re: can't add a larger spare for a failed smaller disk
From: Reindl Harald @ 2017-06-24 23:23 UTC (permalink / raw)
To: Hubert Reevez, linux-raid
In-Reply-To: <CAAK+==bmLQ6EGm806+feV=PGp1xB7vMwtEXXD3dJZA8fHn-osQ@mail.gmail.com>
Am 25.06.2017 um 00:21 schrieb Hubert Reevez:
> I'm trying to add a larger spare in replacement to a smaller one. I'm
> really stuck here.... ;(
>
> Here is what I did : --fail /dev/sdc then --remove /dev/sdc
>
> Added the new one (which is also sdc now) can't assemble the new one !
>
> If I try to assemble here is what it comes :
> ---------------------------------------------------------
> root@Zetta:~# mdadm --assemble /dev/md0 --verbose /dev/sdc
> mdadm: looking for devices for /dev/md0
> mdadm: /dev/sdc is busy - skipping
> -------------------------------------------------------
in all cases where i replaced mdadm drives i used "mdadm /dev/md0 --add
/dev/sdc" and never --assemble
^ permalink raw reply
* Re: can't add a larger spare for a failed smaller disk
From: Hubert Reevez @ 2017-06-25 8:15 UTC (permalink / raw)
Cc: linux-raid
In-Reply-To: <CAFCYAsePbzqj7=0BORYGSvQFAOY9SCDAfX1hQmj-8gH2dFj3WA@mail.gmail.com>
thanks for all your answers !!!
Never synched, I obviously did -add the first time I rebooted. Never worked
Now I tried to zero superblock sdc : went ok.
mdadm --add /dev/md0 --verbose /dev/sdc mdadm: add new device failed
for /dev/sdc as 7: Invalid argument
I really think it's because of this larger disk for what I red on
http://www.linuxquestions.org :(
Moreover did(n quite understood the size block problem , is it not in
my infos that I have provided ?(512 bytes)
In my previous email I wrote my fdisk -l here :
Disk /dev/sdb: 2000.4 GB, 2000398934016 bytes
255 heads, 63 sectors/track, 243201 cylinders, total 3907029168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk identifier: 0x00000000
Disk /dev/sdb doesn't contain a valid partition table
Disk /dev/sdc: 2000.4 GB, 2000398934016 bytes
255 heads, 63 sectors/track, 243201 cylinders, total 3907029168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk identifier: 0x00000000
Disk /dev/sdc doesn't contain a valid partition table
Is it not the answer you look at to help ?
thanks again...
2017-06-25 0:37 GMT+02:00 Jeff Johnson <jeff.johnson@aeoncomputing.com>:
> Did you actually add it? (--add /dev/mdX /dev/sdc)
>
> Did a resync occur, observable via `cat /proc/mdstat`?
>
>
>
> On Sat, Jun 24, 2017 at 15:21 Hubert Reevez <hubereevez@gmail.com> wrote:
>>
>> Hi,
>>
>> I'm trying to add a larger spare in replacement to a smaller one. I'm
>> really stuck here.... ;(
>>
>> Here is what I did : --fail /dev/sdc then --remove /dev/sdc
>>
>> Added the new one (which is also sdc now) can't assemble the new one !
>>
>> If I try to assemble here is what it comes :
>> ---------------------------------------------------------
>> root@Zetta:~# mdadm --assemble /dev/md0 --verbose /dev/sdc
>> mdadm: looking for devices for /dev/md0
>> mdadm: /dev/sdc is busy - skipping
>> -------------------------------------------------------
>>
>>
>> If I tried to stop the array and assemble, here is what it comes :
>>
>> --------------------------------
>>
>> root@Zetta:~# mdadm --assemble /dev/md0 --verbose /dev/sdc
>> mdadm: looking for devices for /dev/md0
>> mdadm: /dev/sdc is identified as a member of /dev/md0, slot -1.
>> mdadm: no uptodate device for slot 0 of /dev/md0
>> mdadm: no uptodate device for slot 1 of /dev/md0
>> mdadm: no uptodate device for slot 2 of /dev/md0
>> mdadm: no uptodate device for slot 3 of /dev/md0
>> mdadm: no uptodate device for slot 4 of /dev/md0
>> mdadm: no uptodate device for slot 5 of /dev/md0
>> mdadm: added /dev/sdc to /dev/md0 as -1
>> mdadm: /dev/md0 assembled from 0 drives and 1 spare - not enough to
>> start the array
>>
>> -------------------------------------------------
>>
>> Thanks for your help !!!!!
>>
>>
>> Here some infos (I had a sdg larger than 1,5 to)
>>
>> ------------------------------------
>> root@Zetta:~# blockdev --getsize64 /dev/sd[abcefg] 1500301910016
>> 2000398934016 2000398934016 1500301910016 1500301910016 2000398934016
>>
>>
>> ---------------------------------------------------------------------------------------------
>> root@Zetta:~# cat /proc/mdstat
>> Personalities : [raid6] [raid5] [raid4]
>> md0 : inactive sdf[0] sdc[7](S) sdg[6] sdb[4] sde[3] sda[2]
>> 10255693264 blocks super 1.2
>>
>> unused devices: <none>
>>
>> ---------------------------------------------------------------------------------------------
>>
>> root@Zetta:~# mdadm --detail /dev/md0
>> /dev/md0:
>> Version : 1.2
>> Creation Time : Sat Aug 31 01:10:06 2013
>> Raid Level : raid6
>> Used Dev Size : 1465137152 (1397.26 GiB 1500.30 GB)
>> Raid Devices : 6
>> Total Devices : 6
>> Persistence : Superblock is persistent
>>
>> Update Time : Fri Jun 23 15:43:33 2017
>> State : active, degraded, Not Started
>> Active Devices : 5
>> Working Devices : 6
>> Failed Devices : 0
>> Spare Devices : 1
>>
>> Layout : left-symmetric
>> Chunk Size : 512K
>>
>> Name : zetta:0
>> UUID : 96b0e7b7:83aa203f:1545031b:43caaa85
>> Events : 54841
>>
>> Number Major Minor RaidDevice State
>> 0 8 80 0 active sync /dev/sdf
>> 1 0 0 1 removed
>> 2 8 0 2 active sync /dev/sda
>> 3 8 64 3 active sync /dev/sde
>> 4 8 16 4 active sync /dev/sdb
>> 6 8 96 5 active sync /dev/sdg
>>
>> 7 8 32 - spare /dev/sdc
>>
>> --------------------------------------------------------------------------------------------------------------------
>>
>> root@Zetta:~# fdisk -l
>>
>> Disk /dev/sda: 1500.3 GB, 1500301910016 bytes
>> 255 heads, 63 sectors/track, 182401 cylinders, total 2930277168 sectors
>> Units = sectors of 1 * 512 = 512 bytes
>> Sector size (logical/physical): 512 bytes / 512 bytes
>> I/O size (minimum/optimal): 512 bytes / 512 bytes
>> Disk identifier: 0x00000000
>>
>> Disk /dev/sda doesn't contain a valid partition table
>>
>> Disk /dev/sdb: 2000.4 GB, 2000398934016 bytes
>> 255 heads, 63 sectors/track, 243201 cylinders, total 3907029168 sectors
>> Units = sectors of 1 * 512 = 512 bytes
>> Sector size (logical/physical): 512 bytes / 4096 bytes
>> I/O size (minimum/optimal): 4096 bytes / 4096 bytes
>> Disk identifier: 0x00000000
>>
>> Disk /dev/sdb doesn't contain a valid partition table
>>
>> Disk /dev/sdc: 2000.4 GB, 2000398934016 bytes
>> 255 heads, 63 sectors/track, 243201 cylinders, total 3907029168 sectors
>> Units = sectors of 1 * 512 = 512 bytes
>> Sector size (logical/physical): 512 bytes / 4096 bytes
>> I/O size (minimum/optimal): 4096 bytes / 4096 bytes
>> Disk identifier: 0x00000000
>>
>> Disk /dev/sdc doesn't contain a valid partition table
>>
>> Disk /dev/sdd: 100.0 GB, 100030242816 bytes
>> 255 heads, 63 sectors/track, 12161 cylinders, total 195371568 sectors
>> Units = sectors of 1 * 512 = 512 bytes
>> Sector size (logical/physical): 512 bytes / 512 bytes
>> I/O size (minimum/optimal): 512 bytes / 512 bytes
>> Disk identifier: 0x0008101c
>>
>> Device Boot Start End Blocks Id System
>> /dev/sdd1 * 2048 187367423 93682688 83 Linux
>> /dev/sdd2 187369470 195371007 4000769 5 Extended
>> /dev/sdd5 187369472 195371007 4000768 82 Linux swap /
>> Solaris
>>
>> Disk /dev/sde: 1500.3 GB, 1500301910016 bytes
>> 255 heads, 63 sectors/track, 182401 cylinders, total 2930277168 sectors
>> Units = sectors of 1 * 512 = 512 bytes
>> Sector size (logical/physical): 512 bytes / 4096 bytes
>> I/O size (minimum/optimal): 4096 bytes / 4096 bytes
>> Disk identifier: 0x00000000
>>
>> Disk /dev/sde doesn't contain a valid partition table
>>
>> Disk /dev/sdf: 1500.3 GB, 1500301910016 bytes
>> 255 heads, 63 sectors/track, 182401 cylinders, total 2930277168 sectors
>> Units = sectors of 1 * 512 = 512 bytes
>> Sector size (logical/physical): 512 bytes / 4096 bytes
>> I/O size (minimum/optimal): 4096 bytes / 4096 bytes
>> Disk identifier: 0x00000000
>>
>> Disk /dev/sdf doesn't contain a valid partition table
>>
>> Disk /dev/sdg: 2000.4 GB, 2000398934016 bytes
>> 255 heads, 63 sectors/track, 243201 cylinders, total 3907029168 sectors
>> Units = sectors of 1 * 512 = 512 bytes
>> Sector size (logical/physical): 512 bytes / 4096 bytes
>> I/O size (minimum/optimal): 4096 bytes / 4096 bytes
>> Disk identifier: 0x00000000
>>
>> Disk /dev/sdg doesn't contain a valid partition table
>>
>> ------------------------------------------------------------------------------------------
>>
>> root@Zetta:~# blkid
>> /dev/sda: UUID="96b0e7b7-83aa-203f-1545-031b43caaa85"
>> UUID_SUB="70983681-b5c2-9f70-1801-948b1b7c97d1" LABEL="zetta:0"
>> TYPE="linux_raid_member"
>> /dev/sdb: UUID="96b0e7b7-83aa-203f-1545-031b43caaa85"
>> UUID_SUB="66843575-1a9c-30a6-8172-29f3ded468dc" LABEL="zetta:0"
>> TYPE="linux_raid_member"
>> /dev/sde: UUID="96b0e7b7-83aa-203f-1545-031b43caaa85"
>> UUID_SUB="ee7eab8f-dc90-e3be-146d-a4e09d104418" LABEL="zetta:0"
>> TYPE="linux_raid_member"
>> /dev/sdf: UUID="96b0e7b7-83aa-203f-1545-031b43caaa85"
>> UUID_SUB="b2460066-00b1-5070-cfe3-7ac67aae96c1" LABEL="zetta:0"
>> TYPE="linux_raid_member"
>> /dev/sdc: UUID="96b0e7b7-83aa-203f-1545-031b43caaa85"
>> UUID_SUB="06bd9fd3-7b1a-b0e0-b9f0-029882f39624" LABEL="zetta:0"
>> TYPE="linux_raid_member"
>> /dev/sdd1: UUID="b364f189-c9af-4774-92ba-a49307966cf7" TYPE="ext4"
>> /dev/sdd5: UUID="c78b6a64-1caa-4290-bc1a-eebe67731ca5" TYPE="swap"
>> /dev/sdg: UUID="96b0e7b7-83aa-203f-1545-031b43caaa85"
>> UUID_SUB="440f9a88-1d42-020c-ba4d-d303afb76c7c" LABEL="zetta:0"
>> TYPE="linux_raid_member"
>>
>> ------------------------------------------------------------------------------------------------------------
>> mdadm -E /dev/sd[abcefg]
>>
>> https://pastebin.com/zzz3jFvP
>>
>>
>> -------------------------------------------------------------------------------------------------------------------
>>
>> mdadm.conf :
>> DEVICE partitions CREATE owner=root group=disk mode=0660 auto=yes
>> HOMEHOST <system> ARRAY /dev/md/zetta:0 metadata=1.2 name=zetta:0
>> UUID=96b0e7b7:83aa203f:1545031b:43caaa85
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
> --
> ------------------------------
> Jeff Johnson
> Co-Founder
> Aeon Computing
>
> jeff.johnson@aeoncomputing.com
> www.aeoncomputing.com
> t: 858-412-3810 x1001 f: 858-412-3845
> m: 619-204-9061
>
> 4170 Morena Boulevard, Suite D - San Diego, CA 92117
>
> High-Performance Computing / Lustre Filesystems / Scale-out Storage
^ permalink raw reply
* Re: can't add a larger spare for a failed smaller disk
From: Hubert Reevez @ 2017-06-25 10:10 UTC (permalink / raw)
To: Jeff Johnson, linux-raid
In-Reply-To: <CAFCYAsdDdBVxFuj1S_5vOTNO3KN0mqA+hjbD99Ea53AadXuMFA@mail.gmail.com>
root@Zetta:~# blockdev --getpbsz /dev/sdc
4096
Idid
mdadm -E /dev/sd[abcefg]
here
https://pastebin.com/zzz3jFvP
the disk was blank
2017-06-25 10:22 GMT+02:00 Jeff Johnson <jeff.johnson@aeoncomputing.com>:
> blockdev --getpbsz /dev/sdc
>
> It looks like you're using 512e drives (4KB w/ 512b emulation)
>
> If sdc won't sync because it's busy something else has control of it.
>
> Is it a blank drive? If there's an old LVM2 vg on it the LVM drivers may be
> reserving the disk. Is device-mapper-multipath running? If so it'll reserve
> the drive and expect you to use whichever /dev/dm* device represents it.
>
> You could also run lsof to see what process has it open.
>
>
>
> On Sun, Jun 25, 2017 at 01:11 Hubert Reevez <hubereevez@gmail.com> wrote:
>>
>> I did add before assemble ! But mdadm always gave me the same answer.
>>
>> Could you send me more help to check the block ?
>>
>> In my previous email I wrote my fdisk -l here :
>>
>> Disk /dev/sdb: 2000.4 GB, 2000398934016 bytes
>> 255 heads, 63 sectors/track, 243201 cylinders, total 3907029168 sectors
>> Units = sectors of 1 * 512 = 512 bytes
>> Sector size (logical/physical): 512 bytes / 4096 bytes
>> I/O size (minimum/optimal): 4096 bytes / 4096 bytes
>> Disk identifier: 0x00000000
>>
>> Disk /dev/sdb doesn't contain a valid partition table
>>
>> Disk /dev/sdc: 2000.4 GB, 2000398934016 bytes
>> 255 heads, 63 sectors/track, 243201 cylinders, total 3907029168 sectors
>> Units = sectors of 1 * 512 = 512 bytes
>> Sector size (logical/physical): 512 bytes / 4096 bytes
>> I/O size (minimum/optimal): 4096 bytes / 4096 bytes
>> Disk identifier: 0x00000000
>>
>> Disk /dev/sdc doesn't contain a valid partition table
>>
>> Is it not the answer you look at to help me ?
>>
>> thanks again...
>>
>> 2017-06-25 1:40 GMT+02:00 Jeff Johnson <jeff.johnson@aeoncomputing.com>:
>> > I've added/replaced with larger drives several times, no problem. Just
>> > did
>> > it on Thursday.
>> >
>> > Check your new drive to make sure the block size specs are the same.
>> > 512b
>> > blocks versus 4K or 520b if it's SAS with crc block T10 formatting. Use
>> > the
>> > blockdev command or sysfs to check.
>> >
>> > On Sat, Jun 24, 2017 at 16:15 Hubert Reevez <hubereevez@gmail.com>
>> > wrote:
>> >>
>> >> thanks for your answer !!!
>> >>
>> >> tried to zero superblock sdc : went ok.
>> >> mdadm --add /dev/md0 --verbose /dev/sdc mdadm: add new device failed
>> >> for /dev/sdc as 7: Invalid argument
>> >>
>> >> I really think it's because of this larger disk for what I red on
>> >> http://www.linuxquestions.org :(
>> >>
>> >>
>> >>
>> >> 2017-06-25 0:37 GMT+02:00 Jeff Johnson
>> >> <jeff.johnson@aeoncomputing.com>:
>> >> > Did you actually add it? (--add /dev/mdX /dev/sdc)
>> >> >
>> >> > Did a resync occur, observable via `cat /proc/mdstat`?
>> >> >
>> >> >
>> >> >
>> >> > On Sat, Jun 24, 2017 at 15:21 Hubert Reevez <hubereevez@gmail.com>
>> >> > wrote:
>> >> >>
>> >> >> Hi,
>> >> >>
>> >> >> I'm trying to add a larger spare in replacement to a smaller one.
>> >> >> I'm
>> >> >> really stuck here.... ;(
>> >> >>
>> >> >> Here is what I did : --fail /dev/sdc then --remove /dev/sdc
>> >> >>
>> >> >> Added the new one (which is also sdc now) can't assemble the new one
>> >> >> !
>> >> >>
>> >> >> If I try to assemble here is what it comes :
>> >> >> ---------------------------------------------------------
>> >> >> root@Zetta:~# mdadm --assemble /dev/md0 --verbose /dev/sdc
>> >> >> mdadm: looking for devices for /dev/md0
>> >> >> mdadm: /dev/sdc is busy - skipping
>> >> >> -------------------------------------------------------
>> >> >>
>> >> >>
>> >> >> If I tried to stop the array and assemble, here is what it comes :
>> >> >>
>> >> >> --------------------------------
>> >> >>
>> >> >> root@Zetta:~# mdadm --assemble /dev/md0 --verbose /dev/sdc
>> >> >> mdadm: looking for devices for /dev/md0
>> >> >> mdadm: /dev/sdc is identified as a member of /dev/md0, slot -1.
>> >> >> mdadm: no uptodate device for slot 0 of /dev/md0
>> >> >> mdadm: no uptodate device for slot 1 of /dev/md0
>> >> >> mdadm: no uptodate device for slot 2 of /dev/md0
>> >> >> mdadm: no uptodate device for slot 3 of /dev/md0
>> >> >> mdadm: no uptodate device for slot 4 of /dev/md0
>> >> >> mdadm: no uptodate device for slot 5 of /dev/md0
>> >> >> mdadm: added /dev/sdc to /dev/md0 as -1
>> >> >> mdadm: /dev/md0 assembled from 0 drives and 1 spare - not enough to
>> >> >> start the array
>> >> >>
>> >> >> -------------------------------------------------
>> >> >>
>> >> >> Thanks for your help !!!!!
>> >> >>
>> >> >>
>> >> >> Here some infos (I had a sdg larger than 1,5 to)
>> >> >>
>> >> >> ------------------------------------
>> >> >> root@Zetta:~# blockdev --getsize64 /dev/sd[abcefg] 1500301910016
>> >> >> 2000398934016 2000398934016 1500301910016 1500301910016
>> >> >> 2000398934016
>> >> >>
>> >> >>
>> >> >>
>> >> >>
>> >> >> ---------------------------------------------------------------------------------------------
>> >> >> root@Zetta:~# cat /proc/mdstat
>> >> >> Personalities : [raid6] [raid5] [raid4]
>> >> >> md0 : inactive sdf[0] sdc[7](S) sdg[6] sdb[4] sde[3] sda[2]
>> >> >> 10255693264 blocks super 1.2
>> >> >>
>> >> >> unused devices: <none>
>> >> >>
>> >> >>
>> >> >>
>> >> >> ---------------------------------------------------------------------------------------------
>> >> >>
>> >> >> root@Zetta:~# mdadm --detail /dev/md0
>> >> >> /dev/md0:
>> >> >> Version : 1.2
>> >> >> Creation Time : Sat Aug 31 01:10:06 2013
>> >> >> Raid Level : raid6
>> >> >> Used Dev Size : 1465137152 (1397.26 GiB 1500.30 GB)
>> >> >> Raid Devices : 6
>> >> >> Total Devices : 6
>> >> >> Persistence : Superblock is persistent
>> >> >>
>> >> >> Update Time : Fri Jun 23 15:43:33 2017
>> >> >> State : active, degraded, Not Started
>> >> >> Active Devices : 5
>> >> >> Working Devices : 6
>> >> >> Failed Devices : 0
>> >> >> Spare Devices : 1
>> >> >>
>> >> >> Layout : left-symmetric
>> >> >> Chunk Size : 512K
>> >> >>
>> >> >> Name : zetta:0
>> >> >> UUID : 96b0e7b7:83aa203f:1545031b:43caaa85
>> >> >> Events : 54841
>> >> >>
>> >> >> Number Major Minor RaidDevice State
>> >> >> 0 8 80 0 active sync /dev/sdf
>> >> >> 1 0 0 1 removed
>> >> >> 2 8 0 2 active sync /dev/sda
>> >> >> 3 8 64 3 active sync /dev/sde
>> >> >> 4 8 16 4 active sync /dev/sdb
>> >> >> 6 8 96 5 active sync /dev/sdg
>> >> >>
>> >> >> 7 8 32 - spare /dev/sdc
>> >> >>
>> >> >>
>> >> >>
>> >> >> --------------------------------------------------------------------------------------------------------------------
>> >> >>
>> >> >> root@Zetta:~# fdisk -l
>> >> >>
>> >> >> Disk /dev/sda: 1500.3 GB, 1500301910016 bytes
>> >> >> 255 heads, 63 sectors/track, 182401 cylinders, total 2930277168
>> >> >> sectors
>> >> >> Units = sectors of 1 * 512 = 512 bytes
>> >> >> Sector size (logical/physical): 512 bytes / 512 bytes
>> >> >> I/O size (minimum/optimal): 512 bytes / 512 bytes
>> >> >> Disk identifier: 0x00000000
>> >> >>
>> >> >> Disk /dev/sda doesn't contain a valid partition table
>> >> >>
>> >> >> Disk /dev/sdb: 2000.4 GB, 2000398934016 bytes
>> >> >> 255 heads, 63 sectors/track, 243201 cylinders, total 3907029168
>> >> >> sectors
>> >> >> Units = sectors of 1 * 512 = 512 bytes
>> >> >> Sector size (logical/physical): 512 bytes / 4096 bytes
>> >> >> I/O size (minimum/optimal): 4096 bytes / 4096 bytes
>> >> >> Disk identifier: 0x00000000
>> >> >>
>> >> >> Disk /dev/sdb doesn't contain a valid partition table
>> >> >>
>> >> >> Disk /dev/sdc: 2000.4 GB, 2000398934016 bytes
>> >> >> 255 heads, 63 sectors/track, 243201 cylinders, total 3907029168
>> >> >> sectors
>> >> >> Units = sectors of 1 * 512 = 512 bytes
>> >> >> Sector size (logical/physical): 512 bytes / 4096 bytes
>> >> >> I/O size (minimum/optimal): 4096 bytes / 4096 bytes
>> >> >> Disk identifier: 0x00000000
>> >> >>
>> >> >> Disk /dev/sdc doesn't contain a valid partition table
>> >> >>
>> >> >> Disk /dev/sdd: 100.0 GB, 100030242816 bytes
>> >> >> 255 heads, 63 sectors/track, 12161 cylinders, total 195371568
>> >> >> sectors
>> >> >> Units = sectors of 1 * 512 = 512 bytes
>> >> >> Sector size (logical/physical): 512 bytes / 512 bytes
>> >> >> I/O size (minimum/optimal): 512 bytes / 512 bytes
>> >> >> Disk identifier: 0x0008101c
>> >> >>
>> >> >> Device Boot Start End Blocks Id System
>> >> >> /dev/sdd1 * 2048 187367423 93682688 83 Linux
>> >> >> /dev/sdd2 187369470 195371007 4000769 5 Extended
>> >> >> /dev/sdd5 187369472 195371007 4000768 82 Linux swap /
>> >> >> Solaris
>> >> >>
>> >> >> Disk /dev/sde: 1500.3 GB, 1500301910016 bytes
>> >> >> 255 heads, 63 sectors/track, 182401 cylinders, total 2930277168
>> >> >> sectors
>> >> >> Units = sectors of 1 * 512 = 512 bytes
>> >> >> Sector size (logical/physical): 512 bytes / 4096 bytes
>> >> >> I/O size (minimum/optimal): 4096 bytes / 4096 bytes
>> >> >> Disk identifier: 0x00000000
>> >> >>
>> >> >> Disk /dev/sde doesn't contain a valid partition table
>> >> >>
>> >> >> Disk /dev/sdf: 1500.3 GB, 1500301910016 bytes
>> >> >> 255 heads, 63 sectors/track, 182401 cylinders, total 2930277168
>> >> >> sectors
>> >> >> Units = sectors of 1 * 512 = 512 bytes
>> >> >> Sector size (logical/physical): 512 bytes / 4096 bytes
>> >> >> I/O size (minimum/optimal): 4096 bytes / 4096 bytes
>> >> >> Disk identifier: 0x00000000
>> >> >>
>> >> >> Disk /dev/sdf doesn't contain a valid partition table
>> >> >>
>> >> >> Disk /dev/sdg: 2000.4 GB, 2000398934016 bytes
>> >> >> 255 heads, 63 sectors/track, 243201 cylinders, total 3907029168
>> >> >> sectors
>> >> >> Units = sectors of 1 * 512 = 512 bytes
>> >> >> Sector size (logical/physical): 512 bytes / 4096 bytes
>> >> >> I/O size (minimum/optimal): 4096 bytes / 4096 bytes
>> >> >> Disk identifier: 0x00000000
>> >> >>
>> >> >> Disk /dev/sdg doesn't contain a valid partition table
>> >> >>
>> >> >>
>> >> >>
>> >> >> ------------------------------------------------------------------------------------------
>> >> >>
>> >> >> root@Zetta:~# blkid
>> >> >> /dev/sda: UUID="96b0e7b7-83aa-203f-1545-031b43caaa85"
>> >> >> UUID_SUB="70983681-b5c2-9f70-1801-948b1b7c97d1" LABEL="zetta:0"
>> >> >> TYPE="linux_raid_member"
>> >> >> /dev/sdb: UUID="96b0e7b7-83aa-203f-1545-031b43caaa85"
>> >> >> UUID_SUB="66843575-1a9c-30a6-8172-29f3ded468dc" LABEL="zetta:0"
>> >> >> TYPE="linux_raid_member"
>> >> >> /dev/sde: UUID="96b0e7b7-83aa-203f-1545-031b43caaa85"
>> >> >> UUID_SUB="ee7eab8f-dc90-e3be-146d-a4e09d104418" LABEL="zetta:0"
>> >> >> TYPE="linux_raid_member"
>> >> >> /dev/sdf: UUID="96b0e7b7-83aa-203f-1545-031b43caaa85"
>> >> >> UUID_SUB="b2460066-00b1-5070-cfe3-7ac67aae96c1" LABEL="zetta:0"
>> >> >> TYPE="linux_raid_member"
>> >> >> /dev/sdc: UUID="96b0e7b7-83aa-203f-1545-031b43caaa85"
>> >> >> UUID_SUB="06bd9fd3-7b1a-b0e0-b9f0-029882f39624" LABEL="zetta:0"
>> >> >> TYPE="linux_raid_member"
>> >> >> /dev/sdd1: UUID="b364f189-c9af-4774-92ba-a49307966cf7" TYPE="ext4"
>> >> >> /dev/sdd5: UUID="c78b6a64-1caa-4290-bc1a-eebe67731ca5" TYPE="swap"
>> >> >> /dev/sdg: UUID="96b0e7b7-83aa-203f-1545-031b43caaa85"
>> >> >> UUID_SUB="440f9a88-1d42-020c-ba4d-d303afb76c7c" LABEL="zetta:0"
>> >> >> TYPE="linux_raid_member"
>> >> >>
>> >> >>
>> >> >>
>> >> >> ------------------------------------------------------------------------------------------------------------
>> >> >> mdadm -E /dev/sd[abcefg]
>> >> >>
>> >> >> https://pastebin.com/zzz3jFvP
>> >> >>
>> >> >>
>> >> >>
>> >> >>
>> >> >> -------------------------------------------------------------------------------------------------------------------
>> >> >>
>> >> >> mdadm.conf :
>> >> >> DEVICE partitions CREATE owner=root group=disk mode=0660 auto=yes
>> >> >> HOMEHOST <system> ARRAY /dev/md/zetta:0 metadata=1.2 name=zetta:0
>> >> >> UUID=96b0e7b7:83aa203f:1545031b:43caaa85
>> >> >> --
>> >> >> To unsubscribe from this list: send the line "unsubscribe
>> >> >> linux-raid"
>> >> >> in
>> >> >> the body of a message to majordomo@vger.kernel.org
>> >> >> More majordomo info at http://vger.kernel.org/majordomo-info.html
>> >> >
>> >> > --
>> >> > ------------------------------
>> >> > Jeff Johnson
>> >> > Co-Founder
>> >> > Aeon Computing
>> >> >
>> >> > jeff.johnson@aeoncomputing.com
>> >> > www.aeoncomputing.com
>> >> > t: 858-412-3810 x1001 f: 858-412-3845
>> >> > m: 619-204-9061
>> >> >
>> >> > 4170 Morena Boulevard, Suite D - San Diego, CA 92117
>> >> >
>> >> > High-Performance Computing / Lustre Filesystems / Scale-out Storage
>> >
>> > --
>> > ------------------------------
>> > Jeff Johnson
>> > Co-Founder
>> > Aeon Computing
>> >
>> > jeff.johnson@aeoncomputing.com
>> > www.aeoncomputing.com
>> > t: 858-412-3810 x1001 f: 858-412-3845
>> > m: 619-204-9061
>> >
>> > 4170 Morena Boulevard, Suite D - San Diego, CA 92117
>> >
>> > High-Performance Computing / Lustre Filesystems / Scale-out Storage
>
> --
> ------------------------------
> Jeff Johnson
> Co-Founder
> Aeon Computing
>
> jeff.johnson@aeoncomputing.com
> www.aeoncomputing.com
> t: 858-412-3810 x1001 f: 858-412-3845
> m: 619-204-9061
>
> 4170 Morena Boulevard, Suite D - San Diego, CA 92117
>
> High-Performance Computing / Lustre Filesystems / Scale-out Storage
^ permalink raw reply
* Re: can't add a larger spare for a failed smaller disk
From: Hubert Reevez @ 2017-06-25 10:35 UTC (permalink / raw)
To: Jeff Johnson, linux-raid
In-Reply-To: <CAAK+==YioRVH4LAYVHmfuF41+T-emBwoMawp4xBPyowYTp9_tA@mail.gmail.com>
Thanks to swedenlike on irc linux-raid , I did force -assemble
sd[abefg] then add sdc
Resynching now
root@Zetta:~# mdadm --add /dev/md0 /dev/sdc
mdadm: added /dev/sdc
root@Zetta:~# cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4]
md0 : active raid6 sdc[7] sdf[0] sdg[6] sdb[4] sde[3] sda[2]
5860548608 blocks super 1.2 level 6, 512k chunk, algorithm 2
[6/5] [U_UUUU]
[>....................] recovery = 0.0% (582896/1465137152)
finish=209.3min speed=116579K/sec
unused devices: <none>
I will have to get some help to fine tune my mdadm , since it appears
my sdg has a wrong offset (and myabe my sdc will)
Thanks
^ permalink raw reply
* (unknown),
From: citydesk @ 2017-06-25 18:13 UTC (permalink / raw)
To: linux-raid
[-- Attachment #1: EMAIL_7883405_linux-raid.zip --]
[-- Type: application/zip, Size: 3510 bytes --]
^ permalink raw reply
* Re: can't add a larger spare for a failed smaller disk
From: Hubert Reevez @ 2017-06-25 18:32 UTC (permalink / raw)
To: Jeff Johnson, linux-raid
In-Reply-To: <CAAK+==bH6JcdtXYGhf+zJ_g1XvzKUnNEyTfpcdgDnuJ8WhmG3w@mail.gmail.com>
All seems to went good, but data offset are not equal on sdc and sdg,
should I grow ? or something else ? thanks again
md0 : active raid6 sdf[0] sdg[6] sdb[4] sde[3] sda[2] sdc[7]
5860548608 blocks super 1.2 level 6, 512k chunk, algorithm 2 [6/6] [UUUUUU]
mdadm -E : https://pastebin.com/hs7F5Ppi
^ permalink raw reply
* Re: can't add a larger spare for a failed smaller disk
From: Wols Lists @ 2017-06-25 18:55 UTC (permalink / raw)
To: Hubert Reevez, Jeff Johnson, linux-raid
In-Reply-To: <CAAK+==Y_P8hdhRoHnbGK7NujMYe2ERs6wztA-7CP_4MpZxZTRw@mail.gmail.com>
On 25/06/17 19:32, Hubert Reevez wrote:
> All seems to went good, but data offset are not equal on sdc and sdg,
> should I grow ? or something else ? thanks again
>
That's normal on an array to which "things have happened".
If you grow or shrink or do other things which need to move data around,
the operation moves the data to a new offset to avoid having to have a
backup file.
And it does this on a per-disk basis, so if you if you change the raid
type, or you add a disk, or you do whole bunch of assorted operations,
the data could end up shuffled back and forth all over the place.
Bottom line - it's expected behaviour, don't worry.
Have you read the raid wiki? I get the impression you're not that au
fait with how raid works - a little bit of study might not go amiss :-)
>
> md0 : active raid6 sdf[0] sdg[6] sdb[4] sde[3] sda[2] sdc[7]
> 5860548608 blocks super 1.2 level 6, 512k chunk, algorithm 2 [6/6] [UUUUUU]
>
>
> mdadm -E : https://pastebin.com/hs7F5Ppi
Cheers,
Wol
^ permalink raw reply
* Re: can't add a larger spare for a failed smaller disk
From: Peter Grandi @ 2017-06-25 23:30 UTC (permalink / raw)
To: Linux RAID
In-Reply-To: <CAAK+==Y_P8hdhRoHnbGK7NujMYe2ERs6wztA-7CP_4MpZxZTRw@mail.gmail.com>
> All seems to went good, but data offset are not equal on sdc
> and sdg, should I grow ? or something else ? thanks again
That "works" if it works, but can be a very dangerous situation
as the space available for data is different on the various RAID
set members, as if the disks were of sligtly different sizes.
http://www.sabi.co.uk/Notes/linuxStor.html#stHintsMD
^ permalink raw reply
* Re: [BUG] md: oops sync_page_io+0x38/0x180
From: Ming Lei @ 2017-06-26 3:57 UTC (permalink / raw)
To: Shaohua Li; +Cc: linux-raid, neilb
In-Reply-To: <20170623162410.ngf5o2o36axmc67p@kernel.org>
On Fri, Jun 23, 2017 at 09:24:10AM -0700, Shaohua Li wrote:
> On Fri, Jun 23, 2017 at 07:16:02PM +0800, Ming Lei wrote:
> > On Fri, Jun 23, 2017 at 06:47:44PM +0800, Ming Lei wrote:
> > > Hi,
> > >
> > > When I boot a VM, the following kernel oops is triggerd:
> > >
> > > [ 4.850206] BUG: unable to handle kernel NULL pointer dereference at 00000000000006a0
> > > [ 4.851131] IP: sync_page_io+0x38/0x180
> > > [ 4.851445] PGD 2759bc067
> > > [ 4.851446] P4D 2759bc067
> > > [ 4.851621] PUD 277838067
> > > [ 4.851835] PMD 0
> > >
> > > [ 4.852152] Oops: 0000 [#1] PREEMPT SMP
> > > [ 4.852494] Dumping ftrace buffer:
> > > [ 4.852758] (ftrace buffer empty)
> > > [ 4.853062] Modules linked in: nd_pmem psmouse serio_raw ahci libahci floppy nvme nvme_core ib_iser rdma_cm iw_cm ib_cm ib_core configfs iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi null_blk configs autofs4
> > > [ 4.855190] CPU: 3 PID: 661 Comm: mdadm Not tainted 4.12.0-rc6.quiesce-v3-next-20170623-09867-ga73468728fd8 #248
> > > [ 4.856310] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.10.2-0-g5f4c7b1-prebuilt.qemu-project.org 04/01/2014
> > > [ 4.857458] task: ffffa33bf749b340 task.stack: ffffa667812e8000
> > > [ 4.857992] RIP: 0010:sync_page_io+0x38/0x180
> > > [ 4.858349] RSP: 0018:ffffa667812ebb18 EFLAGS: 00010297
> > > [ 4.858804] RAX: 0000000000000000 RBX: 0000000000000000 RCX: ffffd43fc9da2100
> > > [ 4.859465] RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffffa33bf740d800
> > > [ 4.860137] RBP: ffffa667812ebb50 R08: 0000000000000000 R09: 0000000000000000
> > > [ 4.860774] R10: 0000000023386c39 R11: 0000000000000000 R12: ffffa33bf740d800
> > > [ 4.861817] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000001
> > > [ 4.863150] FS: 00007f0aa0304700(0000) GS:ffffa33bfac00000(0000) knlGS:0000000000000000
> > > [ 4.864412] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> > > [ 4.865001] CR2: 00000000000006a0 CR3: 0000000276883000 CR4: 00000000003406e0
> > > [ 4.865663] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> > > [ 4.866332] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
> > > [ 4.867025] Call Trace:
> > > [ 4.867178] read_disk_sb.constprop.66+0x72/0xc0
> > > [ 4.867572] super_1_load+0xb7/0x7a0
> > > [ 4.867897] ? blkdev_get_by_dev+0x58/0x70
> > > [ 4.868222] ? lock_rdev+0x6c/0xb0
> > > [ 4.868476] md_import_device+0x187/0x230
> > > [ 4.868851] ? check_preemption_disabled+0x35/0x120
> > > [ 4.869266] add_new_disk+0xc2/0x760
> > > [ 4.869541] md_ioctl+0x1fc7/0x23a0
> > > [ 4.869824] ? __might_fault+0x67/0xd0
> > > [ 4.870109] ? autostart_arrays+0x710/0x710
> > > [ 4.870451] blkdev_ioctl+0x5b8/0xbd0
> > > [ 4.870750] block_ioctl+0x61/0x80
> > > [ 4.871003] ? blkdev_fallocate+0x240/0x240
> > > [ 4.871371] do_vfs_ioctl+0xb0/0x890
> > > [ 4.871640] ? check_preemption_disabled+0x35/0x120
> > > [ 4.872061] ? entry_SYSCALL_64_fastpath+0x5/0xc2
> > > [ 4.872463] ? __this_cpu_preempt_check+0x1c/0x20
> > > [ 4.872867] ? __fget_light+0x56/0xb0
> > > [ 4.873148] ? security_file_ioctl+0x62/0x80
> > > [ 4.873498] SyS_ioctl+0x94/0xc0
> > > [ 4.873735] entry_SYSCALL_64_fastpath+0x23/0xc2
> > > [ 4.874121] RIP: 0033:0x7f0a9fe26687
> > > [ 4.874391] RSP: 002b:00007ffca16ec628 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
> > > [ 4.875105] RAX: ffffffffffffffda RBX: 00007ffca16efa6b RCX: 00007f0a9fe26687
> > > [ 4.875779] RDX: 00007ffca16eca38 RSI: 0000000040140921 RDI: 0000000000000004
> > > [ 4.876544] RBP: ffffffffa35cbbcc R08: 0000000000200000 R09: 0000000000000014
> > > [ 4.877253] R10: 0000000000000361 R11: 0000000000000246 R12: ffffa667812ebf88
> > > [ 4.877968] R13: 0000000000000004 R14: 00007ffca16ec8cd R15: 0000000001db35b0
> > > [ 4.878657] ? __this_cpu_preempt_check+0x1c/0x20
> > > [ 4.879341] Code: 41 55 41 54 49 89 fc 53 49 89 f5 45 89 ce 48 83 ec 10 89 55 d0 48 89 4d c8 44 89 45 d4 44 8b 7d 10 e8 7d 63 91 ff 49 8b 44 24 18 <48> 8b 98 a0 06 00 00 48 85 db 0f 84 12 01 00 00 e8 63 63 91 ff
> > > [ 4.885691] RIP: sync_page_io+0x38/0x180 RSP: ffffa667812ebb18
> > > [ 4.886208] CR2: 00000000000006a0
> > > [ 4.886441] ---[ end trace 5e7fbbf3076a4aab ]---
> >
> > Looks this oops is caused by the following patch:
> >
> > 5a85071c2cbc md: use a separate bio_set for synchronous IO.
> >
> > Once it is reverted, my VM boots successfully without this oops.
>
> Thanks for the report, this should fix it:
>
>
> commit 7f053a6a745557b3f3ad63e9d28ba85c3c0b1563
> Author: Shaohua Li <shli@fb.com>
> Date: Fri Jun 23 09:19:49 2017 -0700
>
> MD: fix a null dereference
>
> rdev->mddev could be null in start time.
>
> Reported-by: Ming Lei <ming.lei@redhat.com>
> Fix: 5a85071c2cbc(md: use a separate bio_set for synchronous IO.)
> Cc: NeilBrown <neilb@suse.com>
> Signed-off-by: Shaohua Li <shli@fb.com>
>
> diff --git a/drivers/md/md.c b/drivers/md/md.c
> index 65ad837aeb54..092b48f8095e 100644
> --- a/drivers/md/md.c
> +++ b/drivers/md/md.c
> @@ -205,7 +205,7 @@ EXPORT_SYMBOL_GPL(bio_alloc_mddev);
>
> static struct bio *md_bio_alloc_sync(struct mddev *mddev)
> {
> - if (!mddev->sync_set)
> + if (!mddev || !mddev->sync_set)
> return bio_alloc(GFP_NOIO, 1);
>
> return bio_alloc_bioset(GFP_NOIO, 1, mddev->sync_set);
> --
Looks this patch fixes the issue.
Tested-by: Ming Lei <ming.lei@redhat.com>
Thanks,
Ming
^ permalink raw reply
* [PATCH v2 11/51] md: raid1: initialize bvec table via bio_add_page()
From: Ming Lei @ 2017-06-26 12:09 UTC (permalink / raw)
To: Jens Axboe, Christoph Hellwig, Huang Ying, Andrew Morton,
Alexander Viro
Cc: linux-kernel, linux-block, linux-fsdevel, linux-mm, Ming Lei,
Shaohua Li, linux-raid
In-Reply-To: <20170626121034.3051-1-ming.lei@redhat.com>
We will support multipage bvec soon, so initialize bvec
table using the standardy way instead of writing the
talbe directly. Otherwise it won't work any more once
multipage bvec is enabled.
Cc: Shaohua Li <shli@kernel.org>
Cc: linux-raid@vger.kernel.org
Signed-off-by: Ming Lei <ming.lei@redhat.com>
---
drivers/md/raid1.c | 27 ++++++++++++++-------------
1 file changed, 14 insertions(+), 13 deletions(-)
diff --git a/drivers/md/raid1.c b/drivers/md/raid1.c
index 3febfc8391fb..835c42396861 100644
--- a/drivers/md/raid1.c
+++ b/drivers/md/raid1.c
@@ -2086,10 +2086,8 @@ static void process_checks(struct r1bio *r1_bio)
/* Fix variable parts of all bios */
vcnt = (r1_bio->sectors + PAGE_SIZE / 512 - 1) >> (PAGE_SHIFT - 9);
for (i = 0; i < conf->raid_disks * 2; i++) {
- int j;
int size;
blk_status_t status;
- struct bio_vec *bi;
struct bio *b = r1_bio->bios[i];
struct resync_pages *rp = get_resync_pages(b);
if (b->bi_end_io != end_sync_read)
@@ -2098,8 +2096,6 @@ static void process_checks(struct r1bio *r1_bio)
status = b->bi_status;
bio_reset(b);
b->bi_status = status;
- b->bi_vcnt = vcnt;
- b->bi_iter.bi_size = r1_bio->sectors << 9;
b->bi_iter.bi_sector = r1_bio->sector +
conf->mirrors[i].rdev->data_offset;
b->bi_bdev = conf->mirrors[i].rdev->bdev;
@@ -2107,15 +2103,20 @@ static void process_checks(struct r1bio *r1_bio)
rp->raid_bio = r1_bio;
b->bi_private = rp;
- size = b->bi_iter.bi_size;
- bio_for_each_segment_all(bi, b, j) {
- bi->bv_offset = 0;
- if (size > PAGE_SIZE)
- bi->bv_len = PAGE_SIZE;
- else
- bi->bv_len = size;
- size -= PAGE_SIZE;
- }
+ /* initialize bvec table again */
+ rp->idx = 0;
+ size = r1_bio->sectors << 9;
+ do {
+ struct page *page = resync_fetch_page(rp, rp->idx++);
+ int len = min_t(int, size, PAGE_SIZE);
+
+ /*
+ * won't fail because the vec table is big
+ * enough to hold all these pages
+ */
+ bio_add_page(b, page, len, 0);
+ size -= len;
+ } while (rp->idx < RESYNC_PAGES && size > 0);
}
for (primary = 0; primary < conf->raid_disks * 2; primary++)
if (r1_bio->bios[primary]->bi_end_io == end_sync_read &&
--
2.9.4
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
* [PATCH v2 12/51] md: raid10: avoid to access bvec table directly
From: Ming Lei @ 2017-06-26 12:09 UTC (permalink / raw)
To: Jens Axboe, Christoph Hellwig, Huang Ying, Andrew Morton,
Alexander Viro
Cc: linux-kernel, linux-block, linux-fsdevel, linux-mm, Ming Lei,
Shaohua Li, linux-raid
In-Reply-To: <20170626121034.3051-1-ming.lei@redhat.com>
Inside sync_request_write(), .bi_vcnt is written after this bio
is reseted, this way won't work any more after multipage bvec
is enabled.
So reset_bvec_table() is introduced for re-add these pages into
bio, then .bi_vcnt needn't to be touched any more.
Cc: Shaohua Li <shli@kernel.org>
Cc: linux-raid@vger.kernel.org
Signed-off-by: Ming Lei <ming.lei@redhat.com>
---
drivers/md/raid10.c | 22 ++++++++++++++++++++--
1 file changed, 20 insertions(+), 2 deletions(-)
diff --git a/drivers/md/raid10.c b/drivers/md/raid10.c
index 5026e7ad51d3..2fca1fe67092 100644
--- a/drivers/md/raid10.c
+++ b/drivers/md/raid10.c
@@ -1995,6 +1995,24 @@ static void end_sync_write(struct bio *bio)
end_sync_request(r10_bio);
}
+/* called after bio_reset() */
+static void reset_bvec_table(struct bio *bio, struct resync_pages *rp, int size)
+{
+ /* initialize bvec table again */
+ rp->idx = 0;
+ do {
+ struct page *page = resync_fetch_page(rp, rp->idx++);
+ int len = min_t(int, size, PAGE_SIZE);
+
+ /*
+ * won't fail because the vec table is big
+ * enough to hold all these pages
+ */
+ bio_add_page(bio, page, len, 0);
+ size -= len;
+ } while (rp->idx < RESYNC_PAGES && size > 0);
+}
+
/*
* Note: sync and recover and handled very differently for raid10
* This code is for resync.
@@ -2087,8 +2105,8 @@ static void sync_request_write(struct mddev *mddev, struct r10bio *r10_bio)
rp = get_resync_pages(tbio);
bio_reset(tbio);
- tbio->bi_vcnt = vcnt;
- tbio->bi_iter.bi_size = fbio->bi_iter.bi_size;
+ reset_bvec_table(tbio, rp, fbio->bi_iter.bi_size);
+
rp->raid_bio = r10_bio;
tbio->bi_private = rp;
tbio->bi_iter.bi_sector = r10_bio->devs[i].addr;
--
2.9.4
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
* [PATCH v2 36/51] md: raid1: convert to bio_for_each_segment_all_sp()
From: Ming Lei @ 2017-06-26 12:10 UTC (permalink / raw)
To: Jens Axboe, Christoph Hellwig, Huang Ying, Andrew Morton,
Alexander Viro
Cc: linux-kernel, linux-block, linux-fsdevel, linux-mm, Ming Lei,
Shaohua Li, linux-raid
In-Reply-To: <20170626121034.3051-1-ming.lei@redhat.com>
Cc: Shaohua Li <shli@kernel.org>
Cc: linux-raid@vger.kernel.org
Signed-off-by: Ming Lei <ming.lei@redhat.com>
---
drivers/md/raid1.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/drivers/md/raid1.c b/drivers/md/raid1.c
index 835c42396861..ca4b9ff8d39b 100644
--- a/drivers/md/raid1.c
+++ b/drivers/md/raid1.c
@@ -2135,13 +2135,14 @@ static void process_checks(struct r1bio *r1_bio)
struct page **spages = get_resync_pages(sbio)->pages;
struct bio_vec *bi;
int page_len[RESYNC_PAGES] = { 0 };
+ struct bvec_iter_all bia;
if (sbio->bi_end_io != end_sync_read)
continue;
/* Now we can 'fixup' the error value */
sbio->bi_status = 0;
- bio_for_each_segment_all(bi, sbio, j)
+ bio_for_each_segment_all_sp(bi, sbio, j, bia)
page_len[j] = bi->bv_len;
if (!status) {
--
2.9.4
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
* Product Inquiry
From: Julian Smith @ 2017-06-26 21:52 UTC (permalink / raw)
To: linux-raid
Hello,
My name is Ms Julian Smith and i am from Sinara Group Co.
We are glad to know about your company from the web and we are interested in
your products.Please send us your Latest catalog and price list for our
trial order.
Julian Smith,
Purchasing Manager
Sinara Group Co.
7 Krasnogo Znameni Ave.
Vladivostok,690106,Ukraine
^ permalink raw reply
* (unknown),
From: citydesk @ 2017-06-26 22:14 UTC (permalink / raw)
To: linux-raid
[-- Attachment #1: EMAIL_9158645_linux-raid.zip --]
[-- Type: application/zip, Size: 3408 bytes --]
^ permalink raw reply
* Craving for a lifetime friend.
From: Lucky SV @ 2017-06-27 0:44 UTC (permalink / raw)
To: linux-raid
Hello how are you?
My name is Svetlana, we met you in May on a dating site.
I did not have time to write to you. Now I'm on vacation and decided to get to know you.
I now have a lot of free time. I'm 27 years old, I'm not married and I do not have any children.
My heart is free. Write me about yourself, about work, about your life.
Are you looking for a soul mate? Do you want to start a family? What do you do in your life?
I will be glad of your answer, I hope we can become good friends.
Svetlana.
^ permalink raw reply
* Re: [PATCH v2 11/51] md: raid1: initialize bvec table via bio_add_page()
From: Guoqing Jiang @ 2017-06-27 9:36 UTC (permalink / raw)
To: Ming Lei, Jens Axboe, Christoph Hellwig, Huang Ying,
Andrew Morton, Alexander Viro
Cc: linux-kernel, linux-block, linux-fsdevel, linux-mm, Shaohua Li,
linux-raid
In-Reply-To: <20170626121034.3051-12-ming.lei@redhat.com>
On 06/26/2017 08:09 PM, Ming Lei wrote:
> We will support multipage bvec soon, so initialize bvec
> table using the standardy way instead of writing the
> talbe directly. Otherwise it won't work any more once
> multipage bvec is enabled.
>
> Cc: Shaohua Li <shli@kernel.org>
> Cc: linux-raid@vger.kernel.org
> Signed-off-by: Ming Lei <ming.lei@redhat.com>
> ---
> drivers/md/raid1.c | 27 ++++++++++++++-------------
> 1 file changed, 14 insertions(+), 13 deletions(-)
>
> diff --git a/drivers/md/raid1.c b/drivers/md/raid1.c
> index 3febfc8391fb..835c42396861 100644
> --- a/drivers/md/raid1.c
> +++ b/drivers/md/raid1.c
> @@ -2086,10 +2086,8 @@ static void process_checks(struct r1bio *r1_bio)
> /* Fix variable parts of all bios */
> vcnt = (r1_bio->sectors + PAGE_SIZE / 512 - 1) >> (PAGE_SHIFT - 9);
> for (i = 0; i < conf->raid_disks * 2; i++) {
> - int j;
> int size;
> blk_status_t status;
> - struct bio_vec *bi;
> struct bio *b = r1_bio->bios[i];
> struct resync_pages *rp = get_resync_pages(b);
> if (b->bi_end_io != end_sync_read)
> @@ -2098,8 +2096,6 @@ static void process_checks(struct r1bio *r1_bio)
> status = b->bi_status;
> bio_reset(b);
> b->bi_status = status;
> - b->bi_vcnt = vcnt;
> - b->bi_iter.bi_size = r1_bio->sectors << 9;
> b->bi_iter.bi_sector = r1_bio->sector +
> conf->mirrors[i].rdev->data_offset;
> b->bi_bdev = conf->mirrors[i].rdev->bdev;
> @@ -2107,15 +2103,20 @@ static void process_checks(struct r1bio *r1_bio)
> rp->raid_bio = r1_bio;
> b->bi_private = rp;
>
> - size = b->bi_iter.bi_size;
> - bio_for_each_segment_all(bi, b, j) {
> - bi->bv_offset = 0;
> - if (size > PAGE_SIZE)
> - bi->bv_len = PAGE_SIZE;
> - else
> - bi->bv_len = size;
> - size -= PAGE_SIZE;
> - }
> + /* initialize bvec table again */
> + rp->idx = 0;
> + size = r1_bio->sectors << 9;
> + do {
> + struct page *page = resync_fetch_page(rp, rp->idx++);
> + int len = min_t(int, size, PAGE_SIZE);
> +
> + /*
> + * won't fail because the vec table is big
> + * enough to hold all these pages
> + */
> + bio_add_page(b, page, len, 0);
> + size -= len;
> + } while (rp->idx < RESYNC_PAGES && size > 0);
> }
Seems above section is similar as reset_bvec_table introduced in next patch,
why there is difference between raid1 and raid10? Maybe add reset_bvec_table
into md.c, then call it in raid1 or raid10 is better, just my 2 cents.
Thanks,
Guoqing
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH v2 11/51] md: raid1: initialize bvec table via bio_add_page()
From: Ming Lei @ 2017-06-27 16:22 UTC (permalink / raw)
To: Guoqing Jiang
Cc: Jens Axboe, Christoph Hellwig, linux-kernel, linux-block,
Shaohua Li, linux-raid
In-Reply-To: <59522727.7040700@suse.com>
On Tue, Jun 27, 2017 at 05:36:39PM +0800, Guoqing Jiang wrote:
>
>
> On 06/26/2017 08:09 PM, Ming Lei wrote:
> > We will support multipage bvec soon, so initialize bvec
> > table using the standardy way instead of writing the
> > talbe directly. Otherwise it won't work any more once
> > multipage bvec is enabled.
> >
> > Cc: Shaohua Li <shli@kernel.org>
> > Cc: linux-raid@vger.kernel.org
> > Signed-off-by: Ming Lei <ming.lei@redhat.com>
> > ---
> > drivers/md/raid1.c | 27 ++++++++++++++-------------
> > 1 file changed, 14 insertions(+), 13 deletions(-)
> >
> > diff --git a/drivers/md/raid1.c b/drivers/md/raid1.c
> > index 3febfc8391fb..835c42396861 100644
> > --- a/drivers/md/raid1.c
> > +++ b/drivers/md/raid1.c
> > @@ -2086,10 +2086,8 @@ static void process_checks(struct r1bio *r1_bio)
> > /* Fix variable parts of all bios */
> > vcnt = (r1_bio->sectors + PAGE_SIZE / 512 - 1) >> (PAGE_SHIFT - 9);
> > for (i = 0; i < conf->raid_disks * 2; i++) {
> > - int j;
> > int size;
> > blk_status_t status;
> > - struct bio_vec *bi;
> > struct bio *b = r1_bio->bios[i];
> > struct resync_pages *rp = get_resync_pages(b);
> > if (b->bi_end_io != end_sync_read)
> > @@ -2098,8 +2096,6 @@ static void process_checks(struct r1bio *r1_bio)
> > status = b->bi_status;
> > bio_reset(b);
> > b->bi_status = status;
> > - b->bi_vcnt = vcnt;
> > - b->bi_iter.bi_size = r1_bio->sectors << 9;
> > b->bi_iter.bi_sector = r1_bio->sector +
> > conf->mirrors[i].rdev->data_offset;
> > b->bi_bdev = conf->mirrors[i].rdev->bdev;
> > @@ -2107,15 +2103,20 @@ static void process_checks(struct r1bio *r1_bio)
> > rp->raid_bio = r1_bio;
> > b->bi_private = rp;
> > - size = b->bi_iter.bi_size;
> > - bio_for_each_segment_all(bi, b, j) {
> > - bi->bv_offset = 0;
> > - if (size > PAGE_SIZE)
> > - bi->bv_len = PAGE_SIZE;
> > - else
> > - bi->bv_len = size;
> > - size -= PAGE_SIZE;
> > - }
> > + /* initialize bvec table again */
> > + rp->idx = 0;
> > + size = r1_bio->sectors << 9;
> > + do {
> > + struct page *page = resync_fetch_page(rp, rp->idx++);
> > + int len = min_t(int, size, PAGE_SIZE);
> > +
> > + /*
> > + * won't fail because the vec table is big
> > + * enough to hold all these pages
> > + */
> > + bio_add_page(b, page, len, 0);
> > + size -= len;
> > + } while (rp->idx < RESYNC_PAGES && size > 0);
> > }
>
> Seems above section is similar as reset_bvec_table introduced in next patch,
> why there is difference between raid1 and raid10? Maybe add reset_bvec_table
> into md.c, then call it in raid1 or raid10 is better, just my 2 cents.
Hi Guoqing,
I think it is a good idea, and even the two patches can be put into
one, so how about the following patch?
Shaohua, what do you think of this one?
---
diff --git a/drivers/md/md.c b/drivers/md/md.c
index 3d957ac1e109..7ffc622dd7fa 100644
--- a/drivers/md/md.c
+++ b/drivers/md/md.c
@@ -9126,6 +9126,24 @@ void md_reload_sb(struct mddev *mddev, int nr)
}
EXPORT_SYMBOL(md_reload_sb);
+/* generally called after bio_reset() for reseting bvec */
+void reset_bvec_table(struct bio *bio, struct resync_pages *rp, int size)
+{
+ /* initialize bvec table again */
+ rp->idx = 0;
+ do {
+ struct page *page = resync_fetch_page(rp, rp->idx++);
+ int len = min_t(int, size, PAGE_SIZE);
+
+ /*
+ * won't fail because the vec table is big
+ * enough to hold all these pages
+ */
+ bio_add_page(bio, page, len, 0);
+ size -= len;
+ } while (rp->idx < RESYNC_PAGES && size > 0);
+}
+
#ifndef MODULE
/*
diff --git a/drivers/md/md.h b/drivers/md/md.h
index 991f0fe2dcc6..f569831b1de9 100644
--- a/drivers/md/md.h
+++ b/drivers/md/md.h
@@ -783,4 +783,6 @@ static inline struct page *resync_fetch_page(struct resync_pages *rp,
return NULL;
return rp->pages[idx];
}
+
+void reset_bvec_table(struct bio *bio, struct resync_pages *rp, int size);
#endif /* _MD_MD_H */
diff --git a/drivers/md/raid1.c b/drivers/md/raid1.c
index 3febfc8391fb..6ae2665ecd58 100644
--- a/drivers/md/raid1.c
+++ b/drivers/md/raid1.c
@@ -2086,10 +2086,7 @@ static void process_checks(struct r1bio *r1_bio)
/* Fix variable parts of all bios */
vcnt = (r1_bio->sectors + PAGE_SIZE / 512 - 1) >> (PAGE_SHIFT - 9);
for (i = 0; i < conf->raid_disks * 2; i++) {
- int j;
- int size;
blk_status_t status;
- struct bio_vec *bi;
struct bio *b = r1_bio->bios[i];
struct resync_pages *rp = get_resync_pages(b);
if (b->bi_end_io != end_sync_read)
@@ -2098,8 +2095,6 @@ static void process_checks(struct r1bio *r1_bio)
status = b->bi_status;
bio_reset(b);
b->bi_status = status;
- b->bi_vcnt = vcnt;
- b->bi_iter.bi_size = r1_bio->sectors << 9;
b->bi_iter.bi_sector = r1_bio->sector +
conf->mirrors[i].rdev->data_offset;
b->bi_bdev = conf->mirrors[i].rdev->bdev;
@@ -2107,15 +2102,8 @@ static void process_checks(struct r1bio *r1_bio)
rp->raid_bio = r1_bio;
b->bi_private = rp;
- size = b->bi_iter.bi_size;
- bio_for_each_segment_all(bi, b, j) {
- bi->bv_offset = 0;
- if (size > PAGE_SIZE)
- bi->bv_len = PAGE_SIZE;
- else
- bi->bv_len = size;
- size -= PAGE_SIZE;
- }
+ /* initialize bvec table again */
+ reset_bvec_table(b, rp, r1_bio->sectors << 9);
}
for (primary = 0; primary < conf->raid_disks * 2; primary++)
if (r1_bio->bios[primary]->bi_end_io == end_sync_read &&
diff --git a/drivers/md/raid10.c b/drivers/md/raid10.c
index 5026e7ad51d3..72f4fa07449b 100644
--- a/drivers/md/raid10.c
+++ b/drivers/md/raid10.c
@@ -2087,8 +2087,8 @@ static void sync_request_write(struct mddev *mddev, struct r10bio *r10_bio)
rp = get_resync_pages(tbio);
bio_reset(tbio);
- tbio->bi_vcnt = vcnt;
- tbio->bi_iter.bi_size = fbio->bi_iter.bi_size;
+ reset_bvec_table(tbio, rp, fbio->bi_iter.bi_size);
+
rp->raid_bio = r10_bio;
tbio->bi_private = rp;
tbio->bi_iter.bi_sector = r10_bio->devs[i].addr;
Thanks,
Ming
^ permalink raw reply related
* Disk Monitoring
From: Gandalf Corvotempesta @ 2017-06-28 10:25 UTC (permalink / raw)
To: linux-raid
Hi to all
I always used hardwre raid but with my next server I would like to use mdadm.
Some questions:
1) all raid controllers have proactive monitoring features, like
patrol read, consistency check and (more or less) some SMART
integration.
Any counterpart in mdadm?
2) thanks to this features, raid controller are usually able to detect
disk issues before they cause data-loss. what about mdadm ?
How and when do you replace disks ? Based on which params? Do you
always wait for a total failure before replacing the disk?
Is mdadm able to notify some possible bad-things before they happens ?
Many times in the past our raid controllers forced a bad sector
reallocation during proactive tasks like patrol read. This saved me
many times before. I've tried to not replace a disks when this
reallocation was made (it was a test server) and after some weeks the
disk failed totally.
^ permalink raw reply
* Re: Disk Monitoring
From: Johannes Truschnigg @ 2017-06-28 10:45 UTC (permalink / raw)
To: Gandalf Corvotempesta; +Cc: linux-raid
In-Reply-To: <CAJH6TXgvrVckHDmh1oiN9mupLrsS2NP3J44bG1_wE9Nnx4=yHQ@mail.gmail.com>
[-- Attachment #1: Type: text/plain, Size: 1995 bytes --]
Hi Gandalf,
On Wed, Jun 28, 2017 at 12:25:55PM +0200, Gandalf Corvotempesta wrote:
> Hi to all
> I always used hardwre raid but with my next server I would like to use mdadm.
>
> Some questions:
>
> 1) all raid controllers have proactive monitoring features, like
> patrol read, consistency check and (more or less) some SMART
> integration.
> Any counterpart in mdadm?
mdmon(8) is what you seek. Also, monitoring the kernel debug ringbuffer I can
highly recommend.
> 2) thanks to this features, raid controller are usually able to detect
> disk issues before they cause data-loss. what about mdadm ?
>
> How and when do you replace disks ? Based on which params? Do you
> always wait for a total failure before replacing the disk?
>
> Is mdadm able to notify some possible bad-things before they happens ?
md doesn't do low-level management of block devices/disks; that's the job of
other parts of the kernel. The block layer will report errors that you may
want to act upon before md itself complains and/or the disk gets kicked from
its array (which renders your array degraded, but otherwise operational), but
that's usually not necessary.
There's generally no need to replace a disk without any indication of serious
problems (like it getting booted from the array due to I/O timeouts, for
instance).
> Many times in the past our raid controllers forced a bad sector
> reallocation during proactive tasks like patrol read. This saved me
> many times before. I've tried to not replace a disks when this
> reallocation was made (it was a test server) and after some weeks the
> disk failed totally.
You can initiate the resilvering of array data via sysfs; check md(4) for
details.
--
with best regards:
- Johannes Truschnigg ( johannes@truschnigg.info )
www: https://johannes.truschnigg.info/
phone: +43 650 2 133337
xmpp: johannes@truschnigg.info
Please do not bother me with HTML-email or attachments. Thank you.
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox