* Re: [PATCH 07/10] block: kill merge_bvec_fn() completely
From: Christoph Hellwig @ 2015-05-06 7:26 UTC (permalink / raw)
To: Ming Lin
Cc: NeilBrown, linux-kernel, Christoph Hellwig, Jens Axboe,
Kent Overstreet, Dongsu Park, Lars Ellenberg, drbd-user,
Jiri Kosina, Yehuda Sadeh, Sage Weil, Alex Elder, ceph-devel,
Alasdair Kergon, Mike Snitzer, dm-devel, linux-raid,
Christoph Hellwig, Martin K. Petersen
In-Reply-To: <5547241B.1040903@kernel.org>
> -static int chunk_aligned_read(struct mddev *mddev, struct bio * raid_bio)
> +static int __chunk_aligned_read(struct mddev *mddev, struct bio *raid_bio)
Call it raid5_read_one_chunk or something similar descriptive?
> {
> struct r5conf *conf = mddev->private;
> int dd_idx;
> @@ -4718,7 +4718,7 @@ static int chunk_aligned_read(struct mddev *mddev, struct bio * raid_bio)
> sector_t end_sector;
>
> if (!in_chunk_boundary(mddev, raid_bio)) {
> - pr_debug("chunk_aligned_read : non aligned\n");
> + pr_debug("__chunk_aligned_read : non aligned\n");
Switch to __func__?
> +static struct bio *chunk_aligned_read(struct mddev *mddev, struct bio *raid_bio)
> +{
> + struct bio *split;
> +
> + do {
> + sector_t sector = raid_bio->bi_iter.bi_sector;
> + unsigned chunk_sects = mddev->chunk_sectors;
> +
> + unsigned sectors = chunk_sects -
> + (likely(is_power_of_2(chunk_sects))
> + ? (sector & (chunk_sects-1))
> + : sector_div(sector, chunk_sects));
This would be a lot more readable with a good old if.
> if (rw == READ && mddev->degraded == 0 &&
> mddev->reshape_position == MaxSector &&
> - chunk_aligned_read(mddev,bi))
> + (!(bi = chunk_aligned_read(mddev, bi))))
> return;
if (rw == READ && mddev->degraded == 0 &&
mddev->reshape_position == MaxSector) {
bi = chunk_aligned_read(mddev, bi);
if (!bi)
return;
}
^ permalink raw reply
* Re: [PATCH 07/10] block: kill merge_bvec_fn() completely
From: Ming Lin @ 2015-05-06 7:10 UTC (permalink / raw)
To: Ming Lin
Cc: NeilBrown, lkml, Christoph Hellwig, Jens Axboe, Kent Overstreet,
Dongsu Park, Lars Ellenberg, drbd-user, Jiri Kosina, Yehuda Sadeh,
Sage Weil, Alex Elder, ceph-devel, Alasdair Kergon, Mike Snitzer,
dm-devel, linux-raid, Christoph Hellwig, Martin K. Petersen
In-Reply-To: <5547241B.1040903@kernel.org>
On Mon, May 4, 2015 at 12:47 AM, Ming Lin <mlin@kernel.org> wrote:
> On 04/28/2015 03:09 PM, NeilBrown wrote:
>> On Mon, 27 Apr 2015 23:48:34 -0700 Ming Lin <mlin@kernel.org> wrote:
>>
>>> From: Kent Overstreet <kent.overstreet@gmail.com>
>>>
>>> As generic_make_request() is now able to handle arbitrarily sized bios,
>>> it's no longer necessary for each individual block driver to define its
>>> own ->merge_bvec_fn() callback. Remove every invocation completely.
>>
>> This patch it just a little premature I think.
>>
>> md/raid5 still assumes read requests will mostly fit within a single chunk
>> (which merge_bvec_fn encourages) so they can be serviced without using the
>> stripe-cache.
>> You've just broken that assumption.
>>
>> I think 'chunk_aligned_read' needs to get a loop using bio_split, a bit like
>> raid0, first.
>
> How about below?
Hi NeilBrown,
Are you OK with below fix?
Then I'll include it in the next version.
Thanks.
>
> drivers/md/raid5.c | 35 ++++++++++++++++++++++++++++++++---
> 1 file changed, 32 insertions(+), 3 deletions(-)
>
> diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c
> index e42b624..2ddfa1e 100644
> --- a/drivers/md/raid5.c
> +++ b/drivers/md/raid5.c
> @@ -4709,7 +4709,7 @@ static void raid5_align_endio(struct bio *bi, int error)
> add_bio_to_retry(raid_bi, conf);
> }
>
> -static int chunk_aligned_read(struct mddev *mddev, struct bio * raid_bio)
> +static int __chunk_aligned_read(struct mddev *mddev, struct bio *raid_bio)
> {
> struct r5conf *conf = mddev->private;
> int dd_idx;
> @@ -4718,7 +4718,7 @@ static int chunk_aligned_read(struct mddev *mddev, struct bio * raid_bio)
> sector_t end_sector;
>
> if (!in_chunk_boundary(mddev, raid_bio)) {
> - pr_debug("chunk_aligned_read : non aligned\n");
> + pr_debug("__chunk_aligned_read : non aligned\n");
> return 0;
> }
> /*
> @@ -4793,6 +4793,35 @@ static int chunk_aligned_read(struct mddev *mddev, struct bio * raid_bio)
> }
> }
>
> +static struct bio *chunk_aligned_read(struct mddev *mddev, struct bio *raid_bio)
> +{
> + struct bio *split;
> +
> + do {
> + sector_t sector = raid_bio->bi_iter.bi_sector;
> + unsigned chunk_sects = mddev->chunk_sectors;
> +
> + unsigned sectors = chunk_sects -
> + (likely(is_power_of_2(chunk_sects))
> + ? (sector & (chunk_sects-1))
> + : sector_div(sector, chunk_sects));
> +
> + if (sectors < bio_sectors(raid_bio)) {
> + split = bio_split(raid_bio, sectors, GFP_NOIO, fs_bio_set);
> + bio_chain(split, raid_bio);
> + } else
> + split = raid_bio;
> +
> + if (!__chunk_aligned_read(mddev, split)) {
> + if (split != raid_bio)
> + generic_make_request(raid_bio);
> + return split;
> + }
> + } while (split != raid_bio);
> +
> + return NULL;
> +}
> +
> /* __get_priority_stripe - get the next stripe to process
> *
> * Full stripe writes are allowed to pass preread active stripes up until
> @@ -5071,7 +5100,7 @@ static void make_request(struct mddev *mddev, struct bio * bi)
> */
> if (rw == READ && mddev->degraded == 0 &&
> mddev->reshape_position == MaxSector &&
> - chunk_aligned_read(mddev,bi))
> + (!(bi = chunk_aligned_read(mddev, bi))))
> return;
>
> if (unlikely(bi->bi_rw & REQ_DISCARD)) {
^ permalink raw reply
* Re: raid6: general protection fault in async_copy_data
From: NeilBrown @ 2015-05-06 4:21 UTC (permalink / raw)
To: Alexander Lyakas; +Cc: linux-raid
In-Reply-To: <CAGRgLy439G+AtYkxzPh=2jBZ=GRtFggYE1kMyLAfadaeghz0xA@mail.gmail.com>
[-- Attachment #1: Type: text/plain, Size: 3640 bytes --]
On Tue, 5 May 2015 10:14:18 +0200 Alexander Lyakas <alex.bolshoy@gmail.com>
wrote:
> Hi Neil,
> we had the following crash:
>
> [86399.862150] general protection fault: 0000 [#1] SMP
> [86399.881970] CPU 1
> [86399.882264] Pid: 17989, comm: md4_raid6 Tainted: GF W O
> 3.8.13-030813-generic #201305111843 Bochs Bochs
> [86399.883681] RIP: 0010:[<ffffffff8135d446>] [<ffffffff8135d446>]
> memcpy+0x6/0x110
> [86399.884886] RSP: 0018:ffff8800a78e5a80 EFLAGS: 00010286
> [86399.885629] RAX: 4588966d912cea06 RBX: ffff8800a78e4000 RCX: 0000000000001000
> [86399.886605] RDX: 0000000000001000 RSI: ffff8800a7ed2000 RDI: 4588966d912cea06
> [86399.887586] RBP: ffff8800a78e5ae8 R08: 0000000000001000 R09: ffff8800a78e5b20
> [86399.888603] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
> [86399.889593] R13: ffff8800a78e5b20 R14: 0000000000001000 R15: 0000000000000000
> [86399.890551] FS: 0000000000000000(0000) GS:ffff88011fd00000(0000)
> knlGS:0000000000000000
> [86399.891648] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [86399.892504] CR2: 00007f10cb8ae966 CR3: 0000000113bfc000 CR4: 00000000001406e0
> [86399.893493] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [86399.894458] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [86399.895426] Process md4_raid6 (pid: 17989, threadinfo
> ffff8800a78e4000, task ffff8800a7dc0000)
> [86399.896629] Stack:
> [86399.896930] ffffffffa05061c5 ffff88000ab6fa06 ffffffff816ed725
> ffffea00029fb480
> [86399.898005] 51160e39b619d7e4 0000000000000000 000000000ab6fa06
> ffff8800c696b938
> [86399.899082] 000000003eca7624 ffff880084dc1ac0 0000000000001000
> 0000000000000002
> [86399.900293] Call Trace:
> [86399.900660] [<ffffffffa05061c5>] ? async_memcpy+0x1c5/0x1000 [async_memcpy]
> [86399.901653] [<ffffffff816ed725>] ? _raw_spin_lock_irq+0x15/0x20
> [86399.902655] [<ffffffffa05a5090>] async_copy_data+0x100/0x140 [raid456]
> [86399.903557] [<ffffffffa05abe20>] handle_stripe+0x13e0/0x2380 [raid456]
> [86399.904531] [<ffffffff815739de>] ? dm_dispatch_request+0x3e/0x70
> [86399.905388] [<ffffffff81097c33>] ? update_curr+0x143/0x1f0
> [86399.906151] [<ffffffff816eb03d>] ? mutex_lock+0x1d/0x50
> [86399.906888] [<ffffffffa05adea5>] handle_active_stripes+0x165/0x200 [raid456]
> [86399.907857] [<ffffffff8156ab8e>] ? md_check_recovery.part.49+0x3e/0x530
> [86399.908811] [<ffffffffa05ae28a>] raid5d+0x34a/0x570 [raid456]
> [86399.909614] [<ffffffff8156344d>] md_thread+0x10d/0x140
> [86399.910356] [<ffffffff8107fc10>] ? add_wait_queue+0x60/0x60
> [86399.911149] [<ffffffff81563340>] ? md_rdev_init+0x140/0x140
> [86399.911955] [<ffffffff8107f050>] kthread+0xc0/0xd0
> [86399.912668] [<ffffffff8107ef90>] ? flush_kthread_worker+0xb0/0xb0
> [86399.913528] [<ffffffff816f61ec>] ret_from_fork+0x7c/0xb0
> [86399.914267] [<ffffffff8107ef90>] ? flush_kthread_worker+0xb0/0xb0
> [86399.915109] Code: 74 13 48 8b 43 58 48 2b 43 50 88 43 4e 48 83 c4
> 08 5b 5d c3 90 e8 fb fd ff ff eb e6 90 90 90 90 90 90 90 90 90 48 89
> f8 48 89 d1 <f3> a4 c3 03 83 e2 07 f3 48 a5 89 d1 f3 a4 c3 20 4c 8b 06
> 4c 8b
> [86399.919028] RIP [<ffffffff8135d446>] memcpy+0x6/0x110
>
> Can you maybe advise what is happening here? Our kernel is 3.8.13.
>
Not really.
It appears that %RDI is the destination for the memcpy, and it contains a
garbage address.
I cannot easily tell if this is a read or a write, but I'd guess a read as it
is hard to get the address of the page in the stripe_cache wrong.
Maybe something has corrupted the bio??
NeilBrown
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 811 bytes --]
^ permalink raw reply
* Removed disk vs. failed disk?
From: Hans Malissa @ 2015-05-05 16:42 UTC (permalink / raw)
To: linux-raid
I’m somewhat new to using RAID, but I may be looking at a broken hard drive already. I’ve been looking at the documentation of mdadm and Linux software RAID, but I’m not sure if I understand everything correctly. So I apologize if some of the questions have already been answered elsewhere, but I need to get this thing running again as soon as possible.
I cannot mount the RAID (/dev/md0) anymore. /proc/mdstat looks like this:
# cat /proc/mdstat
Personalities : [raid1]
md0 : active raid1 sdb1[0]
976760640 blocks super 1.0 [2/1] [U_]
bitmap: 3/8 pages [12KB], 65536KB chunk
unused devices: <none>
and mdadm —detail looks like this:
# mdadm —detail /dev/md0
/dev/md0:
Version : 1.0
Creation Time : Sun Dec 15 16:03:28 2013
Raid Level : raid1
Array Size : 976760640 (931.51 GiB 1000.20 GB)
Used Dev Size : 976760640 (931.51 GiB 1000.20 GB)
Raid Devices : 2
Total Devices : 1
Persistence : Superblock is persistent
Intent Bitmap : Internal
Update Time : Tue May 5 10:17:03 2015
State : active, degraded
Active Devices : 1
Working Devices : 1
Failed Devices : 0
Spare Devices : 0
Name : eprb21:0 (local to host eprb21)
UUID : 34d12cbd:eef71d8d:14dcf224:dfe6c013
Events : 3509
Number Major Minor RaidDevice State
0 8 17 0 active sync /dev/sdb1
1 0 0 1 removed
So it looks like there is a problem with the second disc in the array has some problem. First, I’m not sure what the difference between ‘removed’ and ‘failed’ is, because the disk is physically still present. How does mdadm differentiate between both states?
I understand that the next step would be to put in a new hard drive and rebuild the array. Is there a way to figure out right away if the data on the intact disk is uncompromised?
Best regards, and thanks for your help,
Hans Malissa--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: Potential race in dlm based messaging md-cluster.c
From: Abhijit Bhopatkar @ 2015-05-05 12:10 UTC (permalink / raw)
To: Lidong Zhong, Goldwyn Rodrigues; +Cc: linux-raid
In-Reply-To: <5548911B.1080702@cisco.com>
On 05/05/15 3:14 pm, Abhijit Bhopatkar wrote:
> On 05/05/15 2:52 pm, Lidong Zhong wrote:
>>>>> On 5/1/2015 at 02:36 AM, in message <5542763C.90202@cisco.com>, Abhijit
>> Bhopatkar <abhopatk@cisco.com> wrote:
<snip>
>>>
>>> To illustrate the problem consider timeline for two senders and one
>>> receiver (we will ignore receive part for Sender2 node)
>>>
>>> Sender1 Sender2 Receiver
>>> Get EX on TOKEN Get EX on TOKEN
>>> <Granted> <Wait till granted>
>>>
>>> Get EX on MSG
>>> write LVB
>>> down MSG to CR
>>> Get EX of ACK
>>> <wait till granted>
>>> BAST for ACK
>>> Get CR on MSG
>>> read LVB
>>> process
>>> release ACK
>>> AST for ACK
>>> down ACK to CR
>>> release MSG
>>> release TOKEN
>>> <granted>
>>> Get EX on MSG
>>
>> I am afraid this corner case could not be achieved ever. Sender2 will be blocked on getting
>> EX lock on MSG resource until the receivers release the lock. The receivers' request on
>> upconverting CR to EX on MSG should be put into the convert queue before Sender2's
>> request being put into the wait queue, because sender2 has to wait until the EX on TOKEN
>> is released.
>>
> Yes my initial though of losing a message is not correct. The EX on message won't be granted
> immediately to Sender2 However there is still a deadlock.
>
> Perhaps i am missing something, but according to me nothing prevents Sender2 from acquiring
> EX on TOKEN _and_ MESSAGE __before__ up convert from reciever is queued. Consider adding
> unusual delay right after ACK is released on receiver. The Sender1 will immediately release
> MESSAGE and TOKEN. The receiver is still delayed for whatever reason. Sender2 gets TOKEN grant
> and immediately queues EX for MESSAGE (note this is before EX for MESSAGE is queued by receiver).
>
> DLM will (should?) return error for the up convert saying there is deadlock (-EDEADLK ??)
>
On further investigation in dlm code. Since we do not set DLM_LKF_CONVDEADLK flag on our locks,
in above deadlock case receiver's request to up convert will be simply canceled. And the code
will proceed as expected since receiver still holds CR on MESSAGE. And then after the processing
we will release the CR.
So now my question is changed to;
Why do we up convert the MESSAGE to EX in the first place?
Was receiver EX on MESSAGE intended to serialize all receivers before taking CR on ACK?
Since there is a possibility that we might lose out on this up convert in a race condition, can
we simply eliminate this up conversion? (since CR is preventing the next Sender from taking
EX on MESSAGE anyway).
Regards,
Abhijit
^ permalink raw reply
* Re: Potential race in dlm based messaging md-cluster.c
From: Abhijit Bhopatkar @ 2015-05-05 9:44 UTC (permalink / raw)
To: Lidong Zhong, Goldwyn Rodrigues; +Cc: linux-raid
In-Reply-To: <5548FC6C020000E100022FA0@relay2.provo.novell.com>
On 05/05/15 2:52 pm, Lidong Zhong wrote:
>>>> On 5/1/2015 at 02:36 AM, in message <5542763C.90202@cisco.com>, Abhijit
> Bhopatkar <abhopatk@cisco.com> wrote:
>> There is a possibility of a receiver losing out on messages in certain
>> corner conditions. One of the buggy case is if there is are two sender
>> ready with messages to be sent. Sender 1 initially gets the TOKEN lock
>> and proceeds.
>> After initial processing the sender of message 1 _will_ release TOKEN as
>> soon as receiver releases ACK, it does not wait till ACK CR is
>> re-acquired by receiver.
>>
>> To illustrate the problem consider timeline for two senders and one
>> receiver (we will ignore receive part for Sender2 node)
>>
>> Sender1 Sender2 Receiver
>> Get EX on TOKEN Get EX on TOKEN
>> <Granted> <Wait till granted>
>>
>> Get EX on MSG
>> write LVB
>> down MSG to CR
>> Get EX of ACK
>> <wait till granted>
>> BAST for ACK
>> Get CR on MSG
>> read LVB
>> process
>> release ACK
>> AST for ACK
>> down ACK to CR
>> release MSG
>> release TOKEN
>> <granted>
>> Get EX on MSG
>
> I am afraid this corner case could not be achieved ever. Sender2 will be blocked on getting
> EX lock on MSG resource until the receivers release the lock. The receivers' request on
> upconverting CR to EX on MSG should be put into the convert queue before Sender2's
> request being put into the wait queue, because sender2 has to wait until the EX on TOKEN
> is released.
>
Yes my initial though of losing a message is not correct. The EX on message won't be granted
immediately to Sender2 However there is still a deadlock.
Perhaps i am missing something, but according to me nothing prevents Sender2 from acquiring
EX on TOKEN _and_ MESSAGE __before__ up convert from reciever is queued. Consider adding
unusual delay right after ACK is released on receiver. The Sender1 will immediately release
MESSAGE and TOKEN. The receiver is still delayed for whatever reason. Sender2 gets TOKEN grant
and immediately queues EX for MESSAGE (note this is before EX for MESSAGE is queued by receiver).
DLM will (should?) return error for the up convert saying there is deadlock (-EDEADLK ??)
This also assumes BAST on MESSAGE is NOP and receiver does not let go of MESSAGE CR.
Abhijit
> Regards,
> Lidong
^ permalink raw reply
* Re: Potential race in dlm based messaging md-cluster.c
From: Lidong Zhong @ 2015-05-05 9:22 UTC (permalink / raw)
To: Abhijit Bhopatkar, Goldwyn Rodrigues; +Cc: linux-raid
In-Reply-To: <5542763C.90202@cisco.com>
>>> On 5/1/2015 at 02:36 AM, in message <5542763C.90202@cisco.com>, Abhijit
Bhopatkar <abhopatk@cisco.com> wrote:
> There is a possibility of a receiver losing out on messages in certain
> corner conditions. One of the buggy case is if there is are two sender
> ready with messages to be sent. Sender 1 initially gets the TOKEN lock
> and proceeds.
> After initial processing the sender of message 1 _will_ release TOKEN as
> soon as receiver releases ACK, it does not wait till ACK CR is
> re-acquired by receiver.
>
> To illustrate the problem consider timeline for two senders and one
> receiver (we will ignore receive part for Sender2 node)
>
> Sender1 Sender2 Receiver
> Get EX on TOKEN Get EX on TOKEN
> <Granted> <Wait till granted>
>
> Get EX on MSG
> write LVB
> down MSG to CR
> Get EX of ACK
> <wait till granted>
> BAST for ACK
> Get CR on MSG
> read LVB
> process
> release ACK
> AST for ACK
> down ACK to CR
> release MSG
> release TOKEN
> <granted>
> Get EX on MSG
I am afraid this corner case could not be achieved ever. Sender2 will be blocked on getting
EX lock on MSG resource until the receivers release the lock. The receivers' request on
upconverting CR to EX on MSG should be put into the convert queue before Sender2's
request being put into the wait queue, because sender2 has to wait until the EX on TOKEN
is released.
Regards,
Lidong
> <... proceed ...>
> release TOKEN
> <lost one message>
> ^^^^^^^^^^^^^^^^^
> Get EX on MSG
> Get CR on ACK
> release MSG
>
>
> Abhijit
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
>
^ permalink raw reply
* raid6: general protection fault in async_copy_data
From: Alexander Lyakas @ 2015-05-05 8:14 UTC (permalink / raw)
To: linux-raid; +Cc: Neil Brown
Hi Neil,
we had the following crash:
[86399.862150] general protection fault: 0000 [#1] SMP
[86399.881970] CPU 1
[86399.882264] Pid: 17989, comm: md4_raid6 Tainted: GF W O
3.8.13-030813-generic #201305111843 Bochs Bochs
[86399.883681] RIP: 0010:[<ffffffff8135d446>] [<ffffffff8135d446>]
memcpy+0x6/0x110
[86399.884886] RSP: 0018:ffff8800a78e5a80 EFLAGS: 00010286
[86399.885629] RAX: 4588966d912cea06 RBX: ffff8800a78e4000 RCX: 0000000000001000
[86399.886605] RDX: 0000000000001000 RSI: ffff8800a7ed2000 RDI: 4588966d912cea06
[86399.887586] RBP: ffff8800a78e5ae8 R08: 0000000000001000 R09: ffff8800a78e5b20
[86399.888603] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
[86399.889593] R13: ffff8800a78e5b20 R14: 0000000000001000 R15: 0000000000000000
[86399.890551] FS: 0000000000000000(0000) GS:ffff88011fd00000(0000)
knlGS:0000000000000000
[86399.891648] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[86399.892504] CR2: 00007f10cb8ae966 CR3: 0000000113bfc000 CR4: 00000000001406e0
[86399.893493] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[86399.894458] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[86399.895426] Process md4_raid6 (pid: 17989, threadinfo
ffff8800a78e4000, task ffff8800a7dc0000)
[86399.896629] Stack:
[86399.896930] ffffffffa05061c5 ffff88000ab6fa06 ffffffff816ed725
ffffea00029fb480
[86399.898005] 51160e39b619d7e4 0000000000000000 000000000ab6fa06
ffff8800c696b938
[86399.899082] 000000003eca7624 ffff880084dc1ac0 0000000000001000
0000000000000002
[86399.900293] Call Trace:
[86399.900660] [<ffffffffa05061c5>] ? async_memcpy+0x1c5/0x1000 [async_memcpy]
[86399.901653] [<ffffffff816ed725>] ? _raw_spin_lock_irq+0x15/0x20
[86399.902655] [<ffffffffa05a5090>] async_copy_data+0x100/0x140 [raid456]
[86399.903557] [<ffffffffa05abe20>] handle_stripe+0x13e0/0x2380 [raid456]
[86399.904531] [<ffffffff815739de>] ? dm_dispatch_request+0x3e/0x70
[86399.905388] [<ffffffff81097c33>] ? update_curr+0x143/0x1f0
[86399.906151] [<ffffffff816eb03d>] ? mutex_lock+0x1d/0x50
[86399.906888] [<ffffffffa05adea5>] handle_active_stripes+0x165/0x200 [raid456]
[86399.907857] [<ffffffff8156ab8e>] ? md_check_recovery.part.49+0x3e/0x530
[86399.908811] [<ffffffffa05ae28a>] raid5d+0x34a/0x570 [raid456]
[86399.909614] [<ffffffff8156344d>] md_thread+0x10d/0x140
[86399.910356] [<ffffffff8107fc10>] ? add_wait_queue+0x60/0x60
[86399.911149] [<ffffffff81563340>] ? md_rdev_init+0x140/0x140
[86399.911955] [<ffffffff8107f050>] kthread+0xc0/0xd0
[86399.912668] [<ffffffff8107ef90>] ? flush_kthread_worker+0xb0/0xb0
[86399.913528] [<ffffffff816f61ec>] ret_from_fork+0x7c/0xb0
[86399.914267] [<ffffffff8107ef90>] ? flush_kthread_worker+0xb0/0xb0
[86399.915109] Code: 74 13 48 8b 43 58 48 2b 43 50 88 43 4e 48 83 c4
08 5b 5d c3 90 e8 fb fd ff ff eb e6 90 90 90 90 90 90 90 90 90 48 89
f8 48 89 d1 <f3> a4 c3 03 83 e2 07 f3 48 a5 89 d1 f3 a4 c3 20 4c 8b 06
4c 8b
[86399.919028] RIP [<ffffffff8135d446>] memcpy+0x6/0x110
Can you maybe advise what is happening here? Our kernel is 3.8.13.
Thanks,
Alex.
^ permalink raw reply
* Re: upgrade to jessie/newer kernel and mdadm problems
From: Jérôme Tytgat @ 2015-05-04 13:07 UTC (permalink / raw)
To: linux-raid
In-Reply-To: <5547691F.9060908@turmel.org>
> I was leaving your case for people who know IMSM to pipe up, as I don't
> have any experience with it. But the silence is deafening :-(
That's OK, a good guy (PascalHambourg) in the french debian forum was
able to help me a lot.
> However, if you've been using the system in this degraded state, you
> will need to do the manual assembly with only the good partitions, then
> add the other partitions to rebuild each.
Yes, it was in use, but this what we done :
1. restoring the original mdadm.conf
2. modifying the DEVICE lines with this : DEVICE /dev/sdb?* /dev/sdc?*
3. updating initram: update-initramfs -u (got some errors but we ignored
them, however I made two initrd to be failsafe)
4. rebooted, md126 was gone and md9 back. However all arrays had a
partition marked as fail
5. rebuilded each partiton with mdadm /dev/mdX --add /dev/sdcX or mdadm
/dev/mdX --add /dev/sdbX accordingly (sometimes, the failed partition
was on sdb and sometimes on sdc, this scarried me as I thought I would
loose everything if one drive failed).
6. rebuild was ok, I needed to remove the superblock on /dev/sdb and
/dev/sdc because it looked like this (it contains Intel RAID data but my
disk are softraid and we thought that was one origin of the problem):
<--------------------------------------------------------------------->
# mdadm -E /dev/sdb
mdmon: /dev/sdb is not attached to Intel(R) RAID controller.
mdmon: /dev/sdb is not attached to Intel(R) RAID controller.
/dev/sdb:
Magic : Intel Raid ISM Cfg Sig.
Version : 1.1.00
Orig Family : 26b5a9e0
Family : 26b5a9e0
Generation : 00004db7
Attributes : All supported
UUID : d9cfa6d9:2a715e4f:1fbc2095:be342429
Checksum : 261d2aed correct
MPB Sectors : 1
Disks : 2
RAID Devices : 1
Disk01 Serial : VFC100R10BE79D
State : active
Id : 00010000
Usable Size : 488390862 (232.88 GiB 250.06 GB)
[raidlin]:
UUID : 91449a9d:9242bfe9:d99bceb0:a59f9314
RAID Level : 1
Members : 2
Slots : [UU]
Failed disk : none
This Slot : 1
Array Size : 488390656 (232.88 GiB 250.06 GB)
Per Dev Size : 488390656 (232.88 GiB 250.06 GB)
Sector Offset : 0
Num Stripes : 1907776
Chunk Size : 64 KiB
Reserved : 0
Migrate State : idle
Map State : normal
Dirty State : dirty
Disk00 Serial : VFC100R10BRKMD
State : active
Id : 00000000
Usable Size : 488390862 (232.88 GiB 250.06 GB)
<--------------------------------------------------------------------->
# mdadm -E /dev/sdc
mdmon: /dev/sdc is not attached to Intel(R) RAID controller.
mdmon: /dev/sdc is not attached to Intel(R) RAID controller.
/dev/sdc:
Magic : Intel Raid ISM Cfg Sig.
Version : 1.1.00
Orig Family : 26b5a9e0
Family : 26b5a9e0
Generation : 00004dbc
Attributes : All supported
UUID : d9cfa6d9:2a715e4f:1fbc2095:be342429
Checksum : 261c2af2 correct
MPB Sectors : 1
Disks : 2
RAID Devices : 1
Disk00 Serial : VFC100R10BRKMD
State : active
Id : 00000000
Usable Size : 488390862 (232.88 GiB 250.06 GB)
[raidlin]:
UUID : 91449a9d:9242bfe9:d99bceb0:a59f9314
RAID Level : 1
Members : 2
Slots : [UU]
Failed disk : none
This Slot : 0
Array Size : 488390656 (232.88 GiB 250.06 GB)
Per Dev Size : 488390656 (232.88 GiB 250.06 GB)
Sector Offset : 0
Num Stripes : 1907776
Chunk Size : 64 KiB
Reserved : 0
Migrate State : idle
Map State : normal
Dirty State : clean
Disk01 Serial : VFC100R10BE79D
State : active
Id : 00010000
Usable Size : 488390862 (232.88 GiB 250.06 GB)
<--------------------------------------------------------------------->
7. So I rebooted into initramfs shell by editing GRUB command line to
add "break" at the end of kernel line
8. stopped the raid (they was mounted accordingly to /proc/mdstat) :
mdadm --stop --scan
9. removed the superblock on /dev/sdb and /dev/sdc : mdadm
--zero-superblock --metadata=imsm /dev/sdb ; mdadm --zero-superblock
--metadata=imsm /dev/sdc
This is what they look now :
<--------------------------------------------------------------------->
# mdadm -E /dev/sdb
/dev/sdb:
MBR Magic : aa55
Partition[0] : 979902 sectors at 63 (type fd)
Partition[1] : 9767520 sectors at 979965 (type fd)
Partition[2] : 3903795 sectors at 10747485 (type fd)
Partition[3] : 473740785 sectors at 14651280 (type 05)
<--------------------------------------------------------------------->
# mdadm -E /dev/sdc
/dev/sdc:
MBR Magic : aa55
Partition[0] : 979902 sectors at 63 (type fd)
Partition[1] : 9767520 sectors at 979965 (type fd)
Partition[2] : 3903795 sectors at 10747485 (type fd)
Partition[3] : 473740785 sectors at 14651280 (type 05)
<--------------------------------------------------------------------->
10. rebooted, changed back DEVICE line in mdadm.conf to DEVICE
partitions
The array looks like OK now.
Anyway, Should I upgrade to superblock 1.0 (or 1.2) ? If so, can I use
your method do it in initramfs shell (because my system is live with
active raid arrays) ?
Full thread there (in french, sorry):
https://www.debian-fr.org/mise-a-jour-vers-jessie-et-mdadm-t51945.html
^ permalink raw reply
* Re: upgrade to jessie/newer kernel and mdadm problems
From: Phil Turmel @ 2015-05-04 12:42 UTC (permalink / raw)
To: Jérôme Tytgat, linux-raid
In-Reply-To: <55434A08.3050505@sioban.net>
Good morning Jérôme,
On 05/01/2015 05:40 AM, Jérôme Tytgat wrote:
> Hello list,
>
> Sorry for the long post, but I wanted to be as much informative as I can
> be.
I was leaving your case for people who know IMSM to pipe up, as I don't
have any experience with it. But the silence is deafening :-(
> Forgive my lack of knowlegde in mdadm, I know how to create it using the
> debian installer and few things to get information but that's all.
> Forgive also my english, I'm not a native in this language.
It's ok. Your report was thorough.
> My system has been installed in 2007, and I've upgraded it several times
> until this week to Debian Jessie (the latest version).
That explains the use of v0.90 metadata.
> So, I've upgraded my system to jessie today (only partially with apt-get
> upgrade + kernel upgrade) and I faced with a problem with my RAID 1 soft.
>
> I have two disk (/dev/sdb and /dev/sdc) which are members of the raid array
> There's 10 partitions on these disks, each one is a array (ie sdb1 and
> sdc1).
> All of these form my raids array : md0 to md9
Understood.
> Today , one of my md partition was missing.
> Before upgrade I had partitions from md0 to md9, after reboot I'm
> missing md9.
>
> my mdadm.conf before reboot looked like that:
[trim /]
Very good. Although the level= and num-devices= clauses aren't
necessary, and sometimes troublesome.
> After reboot and after a mdadm -Es, I got this:
[trim /]
> # definitions of existing MD arrays
> ARRAY metadata=imsm UUID=d9cfa6d9:2a715e4f:1fbc2095:be342429
> ARRAY /dev/md/raidlin container=d9cfa6d9:2a715e4f:1fbc2095:be342429
> member=0 UUID=91449a9d:9242bfe9:d99bceb0:a59f9314
So here your system misidentified your drives as members of an
Intel-based MD-compatible hardware raid, then found raid members inside it.
This is almost certainly a side effect of using v0.90 metadata. It has
always had a design problem distinguishing between a raid partition at
the end of a disk and a raid occupying an entire device. It's one of
the reasons that metadata was deprecated long ago.
If you haven't used the system in this weird state, fixing it should be
relatively simple:
1) use mdadm --stop on all the arrays, in numerical order. /proc/mdstat
should then be empty.
2) manually assemble your arrays, one by one, using the
--update=metadata clause to converted them to v1.0 metadata.
3) If md9 refuses to assemble (possibly damaged by the usage as IMSM),
re-create it with metadata v1.0.
4) Replace your mdadm.conf with a new scan, then update your initramfs.
However, if you've been using the system in this degraded state, you
will need to do the manual assembly with only the good partitions, then
add the other partitions to rebuild each.
Hope this helps.
Phil
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: [PATCH] md/raid5: init batch_xxx for new sh at resize_stripes
From: Yuanhan Liu @ 2015-05-04 7:51 UTC (permalink / raw)
To: NeilBrown; +Cc: linux-raid, linux-kernel, Shaohua Li, Fengguang Wu
In-Reply-To: <20150504172424.283b7727@notabene.brown>
On Mon, May 04, 2015 at 05:24:24PM +1000, NeilBrown wrote:
> On Mon, 4 May 2015 13:50:24 +0800 Yuanhan Liu <yuanhan.liu@linux.intel.com>
> wrote:
>
> > This is to fix a kernel NULL dereference oops introduced by commit
> > 59fc630b("RAID5: batch adjacent full stripe write"), which introduced
> > several batch_xxx fields, and did initiation for them at grow_one_stripes(),
> > but forgot to do same at resize_stripes().
> >
> > This oops can be easily triggered by following steps:
> >
> > __create RAID5 /dev/md0
> > __grow /dev/md0
> > mdadm --wait /dev/md0
> > dd if=/dev/zero of=/dev/md0
> >
> > Here is the detailed oops log:
...
> >
> > Cc: Shaohua Li <shli@kernel.org>
> > Signed-off-by: Yuanhan Liu <yuanhan.liu@linux.intel.com>
> > ---
> > drivers/md/raid5.c | 4 ++++
> > 1 file changed, 4 insertions(+)
> >
> > diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c
> > index 697d77a..7b074f7 100644
> > --- a/drivers/md/raid5.c
> > +++ b/drivers/md/raid5.c
> > @@ -2217,6 +2217,10 @@ static int resize_stripes(struct r5conf *conf, int newsize)
> > if (!p)
> > err = -ENOMEM;
> > }
> > +
> > + spin_lock_init(&nsh->batch_lock);
> > + INIT_LIST_HEAD(&nsh->batch_list);
> > + nsh->batch_head = NULL;
> > release_stripe(nsh);
> > }
> > /* critical section pass, GFP_NOIO no longer needed */
>
> Thanks!
>
> However I already have the following fix queued - though not pushed out
Yeah, much cleaner.
> you. I probably would have got it into -rc2 except that I was chasing
> another raid5 bug. The
> BUG_ON(sh->batch_head);
>
> in handle_stripe_fill() fires when I run the mdadm selftests. I got caught
> up chasing that and didn't push the other fix.
I am not aware of there is a selftests for raid. I'd like to add it to our 0day
kernel testing in near future so that we could catch bugs and bisect it down in
first time ;)
--yliu
>
>
> From 3dd8ba734349e602fe17d647ce3da5f4a13748aa Mon Sep 17 00:00:00 2001
> From: NeilBrown <neilb@suse.de>
> Date: Thu, 30 Apr 2015 11:24:28 +1000
> Subject: [PATCH] md/raid5 new alloc_stripe function.
>
>
> diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c
> index 77dfd720aaa0..91a1e8b26b52 100644
> --- a/drivers/md/raid5.c
> +++ b/drivers/md/raid5.c
> @@ -1971,17 +1971,30 @@ static void raid_run_ops(struct stripe_head *sh, unsigned long ops_request)
> put_cpu();
> }
>
> +static struct stripe_head *alloc_stripe(struct kmem_cache *sc, gfp_t gfp)
> +{
> + struct stripe_head *sh;
> +
> + sh = kmem_cache_zalloc(sc, gfp);
> + if (sh) {
> + spin_lock_init(&sh->stripe_lock);
> + spin_lock_init(&sh->batch_lock);
> + INIT_LIST_HEAD(&sh->batch_list);
> + INIT_LIST_HEAD(&sh->lru);
> + atomic_set(&sh->count, 1);
> + }
> + return sh;
> +}
> static int grow_one_stripe(struct r5conf *conf, gfp_t gfp)
> {
> struct stripe_head *sh;
> - sh = kmem_cache_zalloc(conf->slab_cache, gfp);
> +
> + sh = alloc_stripe(conf->slab_cache, gfp);
> if (!sh)
> return 0;
>
> sh->raid_conf = conf;
>
> - spin_lock_init(&sh->stripe_lock);
> -
> if (grow_buffers(sh, gfp)) {
> shrink_buffers(sh);
> kmem_cache_free(conf->slab_cache, sh);
> @@ -1990,13 +2003,8 @@ static int grow_one_stripe(struct r5conf *conf, gfp_t gfp)
> sh->hash_lock_index =
> conf->max_nr_stripes % NR_STRIPE_HASH_LOCKS;
> /* we just created an active stripe so... */
> - atomic_set(&sh->count, 1);
> atomic_inc(&conf->active_stripes);
> - INIT_LIST_HEAD(&sh->lru);
>
> - spin_lock_init(&sh->batch_lock);
> - INIT_LIST_HEAD(&sh->batch_list);
> - sh->batch_head = NULL;
> release_stripe(sh);
> conf->max_nr_stripes++;
> return 1;
> @@ -2109,13 +2117,11 @@ static int resize_stripes(struct r5conf *conf, int newsize)
> return -ENOMEM;
>
> for (i = conf->max_nr_stripes; i; i--) {
> - nsh = kmem_cache_zalloc(sc, GFP_KERNEL);
> + nsh = alloc_stripe(sc, GFP_KERNEL);
> if (!nsh)
> break;
>
> nsh->raid_conf = conf;
> - spin_lock_init(&nsh->stripe_lock);
> -
> list_add(&nsh->lru, &newstripes);
> }
> if (i) {
> @@ -2142,13 +2148,11 @@ static int resize_stripes(struct r5conf *conf, int newsize)
> lock_device_hash_lock(conf, hash));
> osh = get_free_stripe(conf, hash);
> unlock_device_hash_lock(conf, hash);
> - atomic_set(&nsh->count, 1);
> +
> for(i=0; i<conf->pool_size; i++) {
> nsh->dev[i].page = osh->dev[i].page;
> nsh->dev[i].orig_page = osh->dev[i].page;
> }
> - for( ; i<newsize; i++)
> - nsh->dev[i].page = NULL;
> nsh->hash_lock_index = hash;
> kmem_cache_free(conf->slab_cache, osh);
> cnt++;
>
^ permalink raw reply
* Re: [PATCH 07/10] block: kill merge_bvec_fn() completely
From: Ming Lin @ 2015-05-04 7:47 UTC (permalink / raw)
To: NeilBrown
Cc: linux-kernel, Christoph Hellwig, Jens Axboe, Kent Overstreet,
Dongsu Park, Lars Ellenberg, drbd-user, Jiri Kosina, Yehuda Sadeh,
Sage Weil, Alex Elder, ceph-devel, Alasdair Kergon, Mike Snitzer,
dm-devel, linux-raid, Christoph Hellwig, Martin K. Petersen
In-Reply-To: <20150429080919.342fddfd@notabene.brown>
On 04/28/2015 03:09 PM, NeilBrown wrote:
> On Mon, 27 Apr 2015 23:48:34 -0700 Ming Lin <mlin@kernel.org> wrote:
>
>> From: Kent Overstreet <kent.overstreet@gmail.com>
>>
>> As generic_make_request() is now able to handle arbitrarily sized bios,
>> it's no longer necessary for each individual block driver to define its
>> own ->merge_bvec_fn() callback. Remove every invocation completely.
>
> This patch it just a little premature I think.
>
> md/raid5 still assumes read requests will mostly fit within a single chunk
> (which merge_bvec_fn encourages) so they can be serviced without using the
> stripe-cache.
> You've just broken that assumption.
>
> I think 'chunk_aligned_read' needs to get a loop using bio_split, a bit like
> raid0, first.
How about below?
drivers/md/raid5.c | 35 ++++++++++++++++++++++++++++++++---
1 file changed, 32 insertions(+), 3 deletions(-)
diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c
index e42b624..2ddfa1e 100644
--- a/drivers/md/raid5.c
+++ b/drivers/md/raid5.c
@@ -4709,7 +4709,7 @@ static void raid5_align_endio(struct bio *bi, int error)
add_bio_to_retry(raid_bi, conf);
}
-static int chunk_aligned_read(struct mddev *mddev, struct bio * raid_bio)
+static int __chunk_aligned_read(struct mddev *mddev, struct bio *raid_bio)
{
struct r5conf *conf = mddev->private;
int dd_idx;
@@ -4718,7 +4718,7 @@ static int chunk_aligned_read(struct mddev *mddev, struct bio * raid_bio)
sector_t end_sector;
if (!in_chunk_boundary(mddev, raid_bio)) {
- pr_debug("chunk_aligned_read : non aligned\n");
+ pr_debug("__chunk_aligned_read : non aligned\n");
return 0;
}
/*
@@ -4793,6 +4793,35 @@ static int chunk_aligned_read(struct mddev *mddev, struct bio * raid_bio)
}
}
+static struct bio *chunk_aligned_read(struct mddev *mddev, struct bio *raid_bio)
+{
+ struct bio *split;
+
+ do {
+ sector_t sector = raid_bio->bi_iter.bi_sector;
+ unsigned chunk_sects = mddev->chunk_sectors;
+
+ unsigned sectors = chunk_sects -
+ (likely(is_power_of_2(chunk_sects))
+ ? (sector & (chunk_sects-1))
+ : sector_div(sector, chunk_sects));
+
+ if (sectors < bio_sectors(raid_bio)) {
+ split = bio_split(raid_bio, sectors, GFP_NOIO, fs_bio_set);
+ bio_chain(split, raid_bio);
+ } else
+ split = raid_bio;
+
+ if (!__chunk_aligned_read(mddev, split)) {
+ if (split != raid_bio)
+ generic_make_request(raid_bio);
+ return split;
+ }
+ } while (split != raid_bio);
+
+ return NULL;
+}
+
/* __get_priority_stripe - get the next stripe to process
*
* Full stripe writes are allowed to pass preread active stripes up until
@@ -5071,7 +5100,7 @@ static void make_request(struct mddev *mddev, struct bio * bi)
*/
if (rw == READ && mddev->degraded == 0 &&
mddev->reshape_position == MaxSector &&
- chunk_aligned_read(mddev,bi))
+ (!(bi = chunk_aligned_read(mddev, bi))))
return;
if (unlikely(bi->bi_rw & REQ_DISCARD)) {
^ permalink raw reply related
* Re: [PATCH] md/raid5: init batch_xxx for new sh at resize_stripes
From: NeilBrown @ 2015-05-04 7:24 UTC (permalink / raw)
To: Yuanhan Liu; +Cc: linux-raid, linux-kernel, Shaohua Li
In-Reply-To: <1430718624-8988-1-git-send-email-yuanhan.liu@linux.intel.com>
[-- Attachment #1: Type: text/plain, Size: 8590 bytes --]
On Mon, 4 May 2015 13:50:24 +0800 Yuanhan Liu <yuanhan.liu@linux.intel.com>
wrote:
> This is to fix a kernel NULL dereference oops introduced by commit
> 59fc630b("RAID5: batch adjacent full stripe write"), which introduced
> several batch_xxx fields, and did initiation for them at grow_one_stripes(),
> but forgot to do same at resize_stripes().
>
> This oops can be easily triggered by following steps:
>
> __create RAID5 /dev/md0
> __grow /dev/md0
> mdadm --wait /dev/md0
> dd if=/dev/zero of=/dev/md0
>
> Here is the detailed oops log:
>
> [ 32.384499] BUG: unable to handle kernel NULL pointer dereference at (null)
> [ 32.385366] IP: [<ffffffff81844082>] add_stripe_bio+0x48d/0x544
> [ 32.385955] PGD 373f3067 PUD 36e34067 PMD 0
> [ 32.386404] Oops: 0002 [#1] SMP
> [ 32.386740] Modules linked in:
> [ 32.387040] CPU: 0 PID: 1059 Comm: kworker/u2:2 Not tainted 4.0.0-next-20150427+ #107
> [ 32.387762] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.7.5-0-ge51488c-20140602_164612-nilsson.home.kraxel.org 04/01/2014
> [ 32.388044] Workqueue: writeback bdi_writeback_workfn (flush-9:0)
> [ 32.388044] task: ffff88003d038000 ti: ffff88003d40c000 task.ti: ffff88003d40c000
> [ 32.388044] RIP: 0010:[<ffffffff81844082>] [<ffffffff81844082>] add_stripe_bio+0x48d/0x544
> [ 32.388044] RSP: 0000:ffff88003d40f6f8 EFLAGS: 00010046
> [ 32.388044] RAX: 0000000000000000 RBX: ffff880037168cd0 RCX: ffff880037179a28
> [ 32.388044] RDX: ffff880037168d58 RSI: 0000000000000000 RDI: ffff880037179a20
> [ 32.388044] RBP: ffff88003d40f738 R08: 0000000000000410 R09: 0000000000000410
> [ 32.388044] R10: 0000000000000410 R11: 0000000000000002 R12: ffff8800371799a0
> [ 32.388044] R13: ffff88003c3d0800 R14: 0000000000000001 R15: ffff880037179a08
> [ 32.388044] FS: 0000000000000000(0000) GS:ffff88003fc00000(0000) knlGS:0000000000000000
> [ 32.388044] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 32.388044] CR2: 0000000000000000 CR3: 0000000036e33000 CR4: 00000000000006f0
> [ 32.388044] Stack:
> [ 32.388044] 0000000200000000 ffff880037168d38 ffff88003d40f738 ffff88003c3abd00
> [ 32.388044] ffff88003c2df800 ffff88003c3d0800 0000000000000408 ffff88003c3d0b54
> [ 32.388044] ffff88003d40f828 ffffffff8184b9ea ffffffff3d40f7e8 0000000000000292
> [ 32.388044] Call Trace:
> [ 32.388044] [<ffffffff8184b9ea>] make_request+0x7a8/0xaee
> [ 32.388044] [<ffffffff81120387>] ? wait_woken+0x79/0x79
> [ 32.388044] [<ffffffff811e9a85>] ? kmem_cache_alloc+0x95/0x1b6
> [ 32.388044] [<ffffffff8186b944>] md_make_request+0xeb/0x1c3
> [ 32.388044] [<ffffffff811a3025>] ? mempool_alloc+0x64/0x127
> [ 32.388044] [<ffffffff81481575>] generic_make_request+0x9c/0xdb
> [ 32.388044] [<ffffffff814816aa>] submit_bio+0xf6/0x134
> [ 32.388044] [<ffffffff8122a1f7>] _submit_bh+0x119/0x141
> [ 32.388044] [<ffffffff8122a22f>] submit_bh+0x10/0x12
> [ 32.388044] [<ffffffff8122bbb9>] __block_write_full_page.constprop.30+0x1a3/0x2a4
> [ 32.388044] [<ffffffff8122bead>] ? I_BDEV+0xd/0xd
> [ 32.388044] [<ffffffff8122bd65>] block_write_full_page+0xab/0xaf
> [ 32.388044] [<ffffffff8122c657>] blkdev_writepage+0x18/0x1a
> [ 32.388044] [<ffffffff811a9853>] __writepage+0x14/0x2d
> [ 32.388044] [<ffffffff811a9ef3>] write_cache_pages+0x29a/0x3a7
> [ 32.388044] [<ffffffff811a983f>] ? mapping_tagged+0x14/0x14
> [ 32.388044] [<ffffffff811aa03e>] generic_writepages+0x3e/0x56
> [ 32.388044] [<ffffffff811ab638>] do_writepages+0x1e/0x2c
> [ 32.388044] [<ffffffff812229ed>] __writeback_single_inode+0x5b/0x27e
> [ 32.388044] [<ffffffff81222ec7>] writeback_sb_inodes+0x1dc/0x358
> [ 32.388044] [<ffffffff812230c2>] __writeback_inodes_wb+0x7f/0xb8
> [ 32.388044] [<ffffffff812232b9>] wb_writeback+0x11a/0x271
> [ 32.388044] [<ffffffff811aa483>] ? global_dirty_limits+0x1b/0xfd
> [ 32.388044] [<ffffffff8122399c>] bdi_writeback_workfn+0x1ae/0x360
> [ 32.388044] [<ffffffff81101bab>] process_one_work+0x1c2/0x340
> [ 32.388044] [<ffffffff81102571>] worker_thread+0x28b/0x389
> [ 32.388044] [<ffffffff811022e6>] ? cancel_delayed_work_sync+0x15/0x15
> [ 32.388044] [<ffffffff81106936>] kthread+0xd2/0xda
> [ 32.388044] [<ffffffff81106864>] ? kthread_create_on_node+0x17c/0x17c
> [ 32.388044] [<ffffffff81a16682>] ret_from_fork+0x42/0x70
> [ 32.388044] [<ffffffff81106864>] ? kthread_create_on_node+0x17c/0x17c
> [ 32.388044] Code: 84 24 90 00 00 00 48 8d 93 88 00 00 00 49 8d 8c 24 88 00 00 00 49 89 94 24 90 00 00 00 48 89 8b 88 00 00 00 48 89 83 90 00 00 00 <48> 89 10 66 41 83 84 24 80 00 00 00 01 3e 0f ba 73 48 06 72 02
> [ 32.388044] RIP [<ffffffff81844082>] add_stripe_bio+0x48d/0x544
> [ 32.388044] RSP <ffff88003d40f6f8>
> [ 32.388044] CR2: 0000000000000000
> [ 32.388044] ---[ end trace 2b255d3f55be9eb3 ]---
>
> Cc: Shaohua Li <shli@kernel.org>
> Signed-off-by: Yuanhan Liu <yuanhan.liu@linux.intel.com>
> ---
> drivers/md/raid5.c | 4 ++++
> 1 file changed, 4 insertions(+)
>
> diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c
> index 697d77a..7b074f7 100644
> --- a/drivers/md/raid5.c
> +++ b/drivers/md/raid5.c
> @@ -2217,6 +2217,10 @@ static int resize_stripes(struct r5conf *conf, int newsize)
> if (!p)
> err = -ENOMEM;
> }
> +
> + spin_lock_init(&nsh->batch_lock);
> + INIT_LIST_HEAD(&nsh->batch_list);
> + nsh->batch_head = NULL;
> release_stripe(nsh);
> }
> /* critical section pass, GFP_NOIO no longer needed */
Thanks!
However I already have the following fix queued - though not pushed out
you. I probably would have got it into -rc2 except that I was chasing
another raid5 bug. The
BUG_ON(sh->batch_head);
in handle_stripe_fill() fires when I run the mdadm selftests. I got caught
up chasing that and didn't push the other fix.
Thanks,
NeilBrown
From 3dd8ba734349e602fe17d647ce3da5f4a13748aa Mon Sep 17 00:00:00 2001
From: NeilBrown <neilb@suse.de>
Date: Thu, 30 Apr 2015 11:24:28 +1000
Subject: [PATCH] md/raid5 new alloc_stripe function.
diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c
index 77dfd720aaa0..91a1e8b26b52 100644
--- a/drivers/md/raid5.c
+++ b/drivers/md/raid5.c
@@ -1971,17 +1971,30 @@ static void raid_run_ops(struct stripe_head *sh, unsigned long ops_request)
put_cpu();
}
+static struct stripe_head *alloc_stripe(struct kmem_cache *sc, gfp_t gfp)
+{
+ struct stripe_head *sh;
+
+ sh = kmem_cache_zalloc(sc, gfp);
+ if (sh) {
+ spin_lock_init(&sh->stripe_lock);
+ spin_lock_init(&sh->batch_lock);
+ INIT_LIST_HEAD(&sh->batch_list);
+ INIT_LIST_HEAD(&sh->lru);
+ atomic_set(&sh->count, 1);
+ }
+ return sh;
+}
static int grow_one_stripe(struct r5conf *conf, gfp_t gfp)
{
struct stripe_head *sh;
- sh = kmem_cache_zalloc(conf->slab_cache, gfp);
+
+ sh = alloc_stripe(conf->slab_cache, gfp);
if (!sh)
return 0;
sh->raid_conf = conf;
- spin_lock_init(&sh->stripe_lock);
-
if (grow_buffers(sh, gfp)) {
shrink_buffers(sh);
kmem_cache_free(conf->slab_cache, sh);
@@ -1990,13 +2003,8 @@ static int grow_one_stripe(struct r5conf *conf, gfp_t gfp)
sh->hash_lock_index =
conf->max_nr_stripes % NR_STRIPE_HASH_LOCKS;
/* we just created an active stripe so... */
- atomic_set(&sh->count, 1);
atomic_inc(&conf->active_stripes);
- INIT_LIST_HEAD(&sh->lru);
- spin_lock_init(&sh->batch_lock);
- INIT_LIST_HEAD(&sh->batch_list);
- sh->batch_head = NULL;
release_stripe(sh);
conf->max_nr_stripes++;
return 1;
@@ -2109,13 +2117,11 @@ static int resize_stripes(struct r5conf *conf, int newsize)
return -ENOMEM;
for (i = conf->max_nr_stripes; i; i--) {
- nsh = kmem_cache_zalloc(sc, GFP_KERNEL);
+ nsh = alloc_stripe(sc, GFP_KERNEL);
if (!nsh)
break;
nsh->raid_conf = conf;
- spin_lock_init(&nsh->stripe_lock);
-
list_add(&nsh->lru, &newstripes);
}
if (i) {
@@ -2142,13 +2148,11 @@ static int resize_stripes(struct r5conf *conf, int newsize)
lock_device_hash_lock(conf, hash));
osh = get_free_stripe(conf, hash);
unlock_device_hash_lock(conf, hash);
- atomic_set(&nsh->count, 1);
+
for(i=0; i<conf->pool_size; i++) {
nsh->dev[i].page = osh->dev[i].page;
nsh->dev[i].orig_page = osh->dev[i].page;
}
- for( ; i<newsize; i++)
- nsh->dev[i].page = NULL;
nsh->hash_lock_index = hash;
kmem_cache_free(conf->slab_cache, osh);
cnt++;
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 811 bytes --]
^ permalink raw reply related
* [PATCH] md/raid5: init batch_xxx for new sh at resize_stripes
From: Yuanhan Liu @ 2015-05-04 5:50 UTC (permalink / raw)
To: neilb; +Cc: linux-raid, linux-kernel, Yuanhan Liu, Shaohua Li
This is to fix a kernel NULL dereference oops introduced by commit
59fc630b("RAID5: batch adjacent full stripe write"), which introduced
several batch_xxx fields, and did initiation for them at grow_one_stripes(),
but forgot to do same at resize_stripes().
This oops can be easily triggered by following steps:
__create RAID5 /dev/md0
__grow /dev/md0
mdadm --wait /dev/md0
dd if=/dev/zero of=/dev/md0
Here is the detailed oops log:
[ 32.384499] BUG: unable to handle kernel NULL pointer dereference at (null)
[ 32.385366] IP: [<ffffffff81844082>] add_stripe_bio+0x48d/0x544
[ 32.385955] PGD 373f3067 PUD 36e34067 PMD 0
[ 32.386404] Oops: 0002 [#1] SMP
[ 32.386740] Modules linked in:
[ 32.387040] CPU: 0 PID: 1059 Comm: kworker/u2:2 Not tainted 4.0.0-next-20150427+ #107
[ 32.387762] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.7.5-0-ge51488c-20140602_164612-nilsson.home.kraxel.org 04/01/2014
[ 32.388044] Workqueue: writeback bdi_writeback_workfn (flush-9:0)
[ 32.388044] task: ffff88003d038000 ti: ffff88003d40c000 task.ti: ffff88003d40c000
[ 32.388044] RIP: 0010:[<ffffffff81844082>] [<ffffffff81844082>] add_stripe_bio+0x48d/0x544
[ 32.388044] RSP: 0000:ffff88003d40f6f8 EFLAGS: 00010046
[ 32.388044] RAX: 0000000000000000 RBX: ffff880037168cd0 RCX: ffff880037179a28
[ 32.388044] RDX: ffff880037168d58 RSI: 0000000000000000 RDI: ffff880037179a20
[ 32.388044] RBP: ffff88003d40f738 R08: 0000000000000410 R09: 0000000000000410
[ 32.388044] R10: 0000000000000410 R11: 0000000000000002 R12: ffff8800371799a0
[ 32.388044] R13: ffff88003c3d0800 R14: 0000000000000001 R15: ffff880037179a08
[ 32.388044] FS: 0000000000000000(0000) GS:ffff88003fc00000(0000) knlGS:0000000000000000
[ 32.388044] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 32.388044] CR2: 0000000000000000 CR3: 0000000036e33000 CR4: 00000000000006f0
[ 32.388044] Stack:
[ 32.388044] 0000000200000000 ffff880037168d38 ffff88003d40f738 ffff88003c3abd00
[ 32.388044] ffff88003c2df800 ffff88003c3d0800 0000000000000408 ffff88003c3d0b54
[ 32.388044] ffff88003d40f828 ffffffff8184b9ea ffffffff3d40f7e8 0000000000000292
[ 32.388044] Call Trace:
[ 32.388044] [<ffffffff8184b9ea>] make_request+0x7a8/0xaee
[ 32.388044] [<ffffffff81120387>] ? wait_woken+0x79/0x79
[ 32.388044] [<ffffffff811e9a85>] ? kmem_cache_alloc+0x95/0x1b6
[ 32.388044] [<ffffffff8186b944>] md_make_request+0xeb/0x1c3
[ 32.388044] [<ffffffff811a3025>] ? mempool_alloc+0x64/0x127
[ 32.388044] [<ffffffff81481575>] generic_make_request+0x9c/0xdb
[ 32.388044] [<ffffffff814816aa>] submit_bio+0xf6/0x134
[ 32.388044] [<ffffffff8122a1f7>] _submit_bh+0x119/0x141
[ 32.388044] [<ffffffff8122a22f>] submit_bh+0x10/0x12
[ 32.388044] [<ffffffff8122bbb9>] __block_write_full_page.constprop.30+0x1a3/0x2a4
[ 32.388044] [<ffffffff8122bead>] ? I_BDEV+0xd/0xd
[ 32.388044] [<ffffffff8122bd65>] block_write_full_page+0xab/0xaf
[ 32.388044] [<ffffffff8122c657>] blkdev_writepage+0x18/0x1a
[ 32.388044] [<ffffffff811a9853>] __writepage+0x14/0x2d
[ 32.388044] [<ffffffff811a9ef3>] write_cache_pages+0x29a/0x3a7
[ 32.388044] [<ffffffff811a983f>] ? mapping_tagged+0x14/0x14
[ 32.388044] [<ffffffff811aa03e>] generic_writepages+0x3e/0x56
[ 32.388044] [<ffffffff811ab638>] do_writepages+0x1e/0x2c
[ 32.388044] [<ffffffff812229ed>] __writeback_single_inode+0x5b/0x27e
[ 32.388044] [<ffffffff81222ec7>] writeback_sb_inodes+0x1dc/0x358
[ 32.388044] [<ffffffff812230c2>] __writeback_inodes_wb+0x7f/0xb8
[ 32.388044] [<ffffffff812232b9>] wb_writeback+0x11a/0x271
[ 32.388044] [<ffffffff811aa483>] ? global_dirty_limits+0x1b/0xfd
[ 32.388044] [<ffffffff8122399c>] bdi_writeback_workfn+0x1ae/0x360
[ 32.388044] [<ffffffff81101bab>] process_one_work+0x1c2/0x340
[ 32.388044] [<ffffffff81102571>] worker_thread+0x28b/0x389
[ 32.388044] [<ffffffff811022e6>] ? cancel_delayed_work_sync+0x15/0x15
[ 32.388044] [<ffffffff81106936>] kthread+0xd2/0xda
[ 32.388044] [<ffffffff81106864>] ? kthread_create_on_node+0x17c/0x17c
[ 32.388044] [<ffffffff81a16682>] ret_from_fork+0x42/0x70
[ 32.388044] [<ffffffff81106864>] ? kthread_create_on_node+0x17c/0x17c
[ 32.388044] Code: 84 24 90 00 00 00 48 8d 93 88 00 00 00 49 8d 8c 24 88 00 00 00 49 89 94 24 90 00 00 00 48 89 8b 88 00 00 00 48 89 83 90 00 00 00 <48> 89 10 66 41 83 84 24 80 00 00 00 01 3e 0f ba 73 48 06 72 02
[ 32.388044] RIP [<ffffffff81844082>] add_stripe_bio+0x48d/0x544
[ 32.388044] RSP <ffff88003d40f6f8>
[ 32.388044] CR2: 0000000000000000
[ 32.388044] ---[ end trace 2b255d3f55be9eb3 ]---
Cc: Shaohua Li <shli@kernel.org>
Signed-off-by: Yuanhan Liu <yuanhan.liu@linux.intel.com>
---
drivers/md/raid5.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c
index 697d77a..7b074f7 100644
--- a/drivers/md/raid5.c
+++ b/drivers/md/raid5.c
@@ -2217,6 +2217,10 @@ static int resize_stripes(struct r5conf *conf, int newsize)
if (!p)
err = -ENOMEM;
}
+
+ spin_lock_init(&nsh->batch_lock);
+ INIT_LIST_HEAD(&nsh->batch_list);
+ nsh->batch_head = NULL;
release_stripe(nsh);
}
/* critical section pass, GFP_NOIO no longer needed */
--
1.9.0
^ permalink raw reply related
* Re: drive failing on same bus every couple weeks
From: Mikael Abrahamsson @ 2015-05-04 5:34 UTC (permalink / raw)
To: Stephen Burke; +Cc: linux-raid
In-Reply-To: <CAAugdn6aaXZ45pbODiUp+3E74fXsgxWHKd6Dc7tuy77dqxhMNQ@mail.gmail.com>
On Sun, 3 May 2015, Stephen Burke wrote:
> Any ideas as to what could be going wrong? The only other thing I can
> think of hardware wise is I have a small power supply from a different
> computer. It seems to run everything fine. I'm wondering if that
> doesn't have enough power would random drive failures occur like this or
> would they not power on?
The errors you pasted doesn't indicate a read or write error. Use
"smartctl -a /dev/sdX" to check the drive hardware status, that'll give
you a better indication if you're actually seeing read/write error or not.
The errors could indicate anything from bad cable, bad power or anything
else, but moving the drive to another power socket, change the sata cable
and move it to another controller port, are all very valid next steps, so
is swapping out the PSU if you feel that that's close to the limit.
For instance, there are single and multi "rail" PSUs. If they're not
single rail, then they have internal "partitions" that will each only give
for instance half of the rated power output. If you put all your drives on
one of the rails, then you're effectively just using half of the rated
capacity. So that might be another thing to investigate. Personally, I
would buy a single rail PSU and not have to worry about this.
http://www.overclock.net/t/761202/single-rail-vs-multi-rail-explained
--
Mikael Abrahamsson email: swmike@swm.pp.se
^ permalink raw reply
* drive failing on same bus every couple weeks
From: Stephen Burke @ 2015-05-03 23:46 UTC (permalink / raw)
To: linux-raid
So I have had 3 faulty drives so far in the last 6 mo. For the first
2 I have swapped them out and rebuilt fine, figured I was unlucky and
didn't think about it. The latest one I noticed that it's the same
bus that is failing every time. /dev/sdb I am suspecting that the
drives themselves haven't been faulty & something else is going wrong.
I'm thinking that I should hook up the drive to another bus & rebuild
& see if that fails again. Will I be able to just swap the cable to a
new bus & rebuild just like if I was swapping out a failed drive?
What are the commands for that?
Any ideas as to what could be going wrong? The only other thing I can
think of hardware wise is I have a small power supply from a different
computer. It seems to run everything fine. I'm wondering if that
doesn't have enough power would random drive failures occur like this
or would they not power on?
Any help would be appreciated. Here's some info on the raid and what
I could find in the log.
root@ht-pc:/home/sburke# mdadm --detail /dev/md0
/dev/md0:
Version : 1.2
Creation Time : Fri Dec 13 01:18:13 2013
Raid Level : raid5
Array Size : 3906763776 (3725.78 GiB 4000.53 GB)
Used Dev Size : 1953381888 (1862.89 GiB 2000.26 GB)
Raid Devices : 3
Total Devices : 3
Persistence : Superblock is persistent
Update Time : Sun May 3 19:09:01 2015
State : clean, degraded
Active Devices : 2
Working Devices : 2
Failed Devices : 1
Spare Devices : 0
Layout : left-symmetric
Chunk Size : 512K
Name : ht-pc:0 (local to host ht-pc)
UUID : 508cb42f:d2c1ea9c:e62b4121:c3d9cbc3
Events : 2506
Number Major Minor RaidDevice State
0 0 0 0 removed
1 8 33 1 active sync /dev/sdc1
3 8 65 2 active sync /dev/sde1
4 8 16 - faulty spare /dev/sdb
/var/log/syslog
May 3 00:02:12 ht-pc kernel: [533073.515788] ata2.00: exception Emask
0x50 SAct 0x40 SErr 0x280900 action 0x6 frozen
May 3 00:02:12 ht-pc kernel: [533073.515799] ata2.00: irq_stat
0x08000000, interface fatal error
May 3 00:02:12 ht-pc kernel: [533073.515809] ata2: SError: {
UnrecovData HostInt 10B8B BadCRC }
May 3 00:02:12 ht-pc kernel: [533073.515819] ata2.00: failed command:
READ FPDMA QUEUED
May 3 00:02:12 ht-pc kernel: [533073.515835] ata2.00: cmd
60/08:30:58:18:c7/00:00:04:00:00/40 tag 6 ncq 4096 in
May 3 00:02:12 ht-pc kernel: [533073.515838] res
40/00:34:58:18:c7/00:00:04:00:00/40 Emask 0x50 (ATA bus error)
May 3 00:02:12 ht-pc kernel: [533073.515846] ata2.00: status: { DRDY }
May 3 00:02:12 ht-pc kernel: [533073.515856] ata2: hard resetting link
May 3 00:02:13 ht-pc kernel: [533074.004085] ata2: SATA link up 3.0
Gbps (SStatus 123 SControl 300)
May 3 00:02:13 ht-pc kernel: [533074.005165] ata2.00: configured for UDMA/133
May 3 00:02:13 ht-pc kernel: [533074.020088] ata2: EH complete
May 3 00:02:29 ht-pc kernel: [533090.498230] ata2.00: exception Emask
0x50 SAct 0x20000000 SErr 0x280900 action 0x6 frozen
May 3 00:02:29 ht-pc kernel: [533090.498242] ata2.00: irq_stat
0x08000000, interface fatal error
May 3 00:02:29 ht-pc kernel: [533090.498252] ata2: SError: {
UnrecovData HostInt 10B8B BadCRC }
May 3 00:02:29 ht-pc kernel: [533090.498262] ata2.00: failed command:
READ FPDMA QUEUED
May 3 00:02:29 ht-pc kernel: [533090.498277] ata2.00: cmd
60/08:e8:b0:52:ae/00:00:02:00:00/40 tag 29 ncq 4096 in
May 3 00:02:29 ht-pc kernel: [533090.498281] res
40/00:ec:b0:52:ae/00:00:02:00:00/40 Emask 0x50 (ATA bus error)
May 3 00:02:29 ht-pc kernel: [533090.498289] ata2.00: status: { DRDY }
May 3 00:02:29 ht-pc kernel: [533090.498298] ata2: hard resetting link
May 3 00:02:30 ht-pc kernel: [533090.988061] ata2: SATA link up 3.0
Gbps (SStatus 123 SControl 300)
May 3 00:02:30 ht-pc kernel: [533090.989184] ata2.00: configured for UDMA/133
May 3 00:02:30 ht-pc kernel: [533091.004093] ata2: EH complete
--
Steve
www.stayathomedevs.com
Game Data Editor Unity Plugin
^ permalink raw reply
* [SOLVED] Re: md-cluster documentation for testing?
From: Abhijit Bhopatkar @ 2015-05-01 16:18 UTC (permalink / raw)
To: linux-raid
In-Reply-To: <55439997.6050002@cisco.com>
> Any pointers will be helpful.
Never mind I found http://www.spinics.net/lists/raid/msg47863.html
However if there is something better or updated please let me know.
Abhijit
^ permalink raw reply
* md-cluster documentation for testing?
From: Abhijit Bhopatkar @ 2015-05-01 15:19 UTC (permalink / raw)
To: linux-raid
Is there some documentation that will let me know how to bring up the cluster RAID? The kernel documentation is limited to design.
Are there special user space binaires required? How to setup the dlm?
Any pointers will be helpful.
As of now i have a shared disk exposed over iscsi to three VM's running 4.1-rc1 kernel.
Abhijit
^ permalink raw reply
* upgrade to jessie/newer kernel and mdadm problems
From: Jérôme Tytgat @ 2015-05-01 9:40 UTC (permalink / raw)
To: linux-raid
Hello list,
Sorry for the long post, but I wanted to be as much informative as I can be.
Forgive my lack of knowlegde in mdadm, I know how to create it using the
debian installer and few things to get information but that's all.
Forgive also my english, I'm not a native in this language.
My system has been installed in 2007, and I've upgraded it several times
until this week to Debian Jessie (the latest version).
So, I've upgraded my system to jessie today (only partially with apt-get
upgrade + kernel upgrade) and I faced with a problem with my RAID 1 soft.
I have two disk (/dev/sdb and /dev/sdc) which are members of the raid array
There's 10 partitions on these disks, each one is a array (ie sdb1 and
sdc1).
All of these form my raids array : md0 to md9
Today , one of my md partition was missing.
Before upgrade I had partitions from md0 to md9, after reboot I'm
missing md9.
my mdadm.conf before reboot looked like that:
>-------------------------------------------------------------------------------------------------------------<
# mdadm.conf
#
# Please refer to mdadm.conf(5) for information about this file.
#
# by default, scan all partitions (/proc/partitions) for MD superblocks.
# alternatively, specify devices to scan, using wildcards if desired.
DEVICE partitions
# auto-create devices with Debian standard permissions
CREATE owner=root group=disk mode=0660 auto=yes
# automatically tag new arrays as belonging to the local system
HOMEHOST <system>
# instruct the monitoring daemon where to send mail alerts
MAILADDR root
# definitions of existing MD arrays
ARRAY /dev/md0 level=raid1 num-devices=2
UUID=350e253f:863b7b04:b1617c47:b213a024
ARRAY /dev/md1 level=raid1 num-devices=2
UUID=086e68ed:3607317f:60b56e23:6bae62bc
ARRAY /dev/md2 level=raid1 num-devices=2
UUID=0f6e3ed5:aeee975a:c3647deb:763d68ce
ARRAY /dev/md3 level=raid1 num-devices=2
UUID=6b560fe8:f24d6f2e:8942bd3a:1903abbc
ARRAY /dev/md4 level=raid1 num-devices=2
UUID=462dab96:b7ca2a17:7c4aebf1:d4d7ec3b
ARRAY /dev/md5 level=raid1 num-devices=2
UUID=ea511351:3abc7b12:4c81e838:93dbd21a
ARRAY /dev/md6 level=raid1 num-devices=2
UUID=da0d76c6:91422584:dc3d6162:37ced53b
ARRAY /dev/md7 level=raid1 num-devices=2
UUID=387c831c:8a6d05e3:b649696c:0870b930
ARRAY /dev/md8 level=raid1 num-devices=2
UUID=b07c4ab4:39d0ba53:9913afa9:fd9cc323
ARRAY /dev/md9 level=raid1 num-devices=2
UUID=36c4edd0:a0492cc9:0cd2fce0:2745e358
# This file was auto-generated on Wed, 07 Feb 2007 17:15:33 +0000
# by mkconf $Id: mkconf 261 2006-11-09 13:32:35Z madduck $
>-------------------------------------------------------------------------------------------------------------<
After reboot and after a mdadm -Es, I got this:
>-------------------------------------------------------------------------------------------------------------<
# mdadm.conf
#
# Please refer to mdadm.conf(5) for information about this file.
#
# by default, scan all partitions (/proc/partitions) for MD superblocks.
# alternatively, specify devices to scan, using wildcards if desired.
DEVICE partitions
# auto-create devices with Debian standard permissions
CREATE owner=root group=disk mode=0660 auto=yes
# automatically tag new arrays as belonging to the local system
HOMEHOST <system>
# instruct the monitoring daemon where to send mail alerts
MAILADDR root
# definitions of existing MD arrays
ARRAY metadata=imsm UUID=d9cfa6d9:2a715e4f:1fbc2095:be342429
ARRAY /dev/md/raidlin container=d9cfa6d9:2a715e4f:1fbc2095:be342429
member=0 UUID=91449a9d:9242bfe9:d99bceb0:a59f9314
ARRAY /dev/md0 UUID=350e253f:863b7b04:b1617c47:b213a024
ARRAY /dev/md1 UUID=086e68ed:3607317f:60b56e23:6bae62bc
ARRAY /dev/md2 UUID=0f6e3ed5:aeee975a:c3647deb:763d68ce
ARRAY /dev/md3 UUID=6b560fe8:f24d6f2e:8942bd3a:1903abbc
ARRAY /dev/md4 UUID=462dab96:b7ca2a17:7c4aebf1:d4d7ec3b
ARRAY /dev/md5 UUID=ea511351:3abc7b12:4c81e838:93dbd21a
ARRAY /dev/md6 UUID=da0d76c6:91422584:dc3d6162:37ced53b
ARRAY /dev/md7 UUID=387c831c:8a6d05e3:b649696c:0870b930
ARRAY /dev/md8 UUID=b07c4ab4:39d0ba53:9913afa9:fd9cc323
>-------------------------------------------------------------------------------------------------------------<
This is what is looking my fdisk extract and /proc/mdstat:
>-------------------------------------------------------------------------------------------------------------<
# fdisk -l /dev/sdb
Disk /dev/sdb: 250.1 GB, 250059350016 bytes
255 heads, 63 sectors/track, 30401 cylinders, total 488397168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x0001edee
Device Boot Start End Blocks Id System
/dev/sdb1 * 63 979964 489951 fd Linux raid
autodetect
/dev/sdb2 979965 10747484 4883760 fd Linux raid
autodetect
/dev/sdb3 10747485 14651279 1951897+ fd Linux raid
autodetect
/dev/sdb4 14651280 488392064 236870392+ 5 Extended
/dev/sdb5 14651343 24418799 4883728+ fd Linux raid
autodetect
/dev/sdb6 24418863 43953839 9767488+ fd Linux raid
autodetect
/dev/sdb7 43953903 53721359 4883728+ fd Linux raid
autodetect
/dev/sdb8 53721423 63488879 4883728+ fd Linux raid
autodetect
/dev/sdb9 63488943 73256399 4883728+ fd Linux raid
autodetect
/dev/sdb10 73256463 83023919 4883728+ fd Linux raid
autodetect
/dev/sdb11 83023983 488392064 202684041 fd Linux raid
autodetect
>-------------------------------------------------------------------------------------------------------------<
# fdisk -l /dev/sdc
Disk /dev/sdc: 250.1 GB, 250059350016 bytes
255 heads, 63 sectors/track, 30401 cylinders, total 488397168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000c352c
Device Boot Start End Blocks Id System
/dev/sdc1 * 63 979964 489951 fd Linux raid
autodetect
/dev/sdc2 979965 10747484 4883760 fd Linux raid
autodetect
/dev/sdc3 10747485 14651279 1951897+ fd Linux raid
autodetect
/dev/sdc4 14651280 488392064 236870392+ 5 Extended
/dev/sdc5 14651343 24418799 4883728+ fd Linux raid
autodetect
/dev/sdc6 24418863 43953839 9767488+ fd Linux raid
autodetect
/dev/sdc7 43953903 53721359 4883728+ fd Linux raid
autodetect
/dev/sdc8 53721423 63488879 4883728+ fd Linux raid
autodetect
/dev/sdc9 63488943 73256399 4883728+ fd Linux raid
autodetect
/dev/sdc10 73256463 83023919 4883728+ fd Linux raid
autodetect
/dev/sdc11 83023983 488392064 202684041 fd Linux raid
autodetect
>-------------------------------------------------------------------------------------------------------------<
# cat /proc/mdstat
Personalities : [raid1]
md8 : active raid1 md126p10[0]
4883648 blocks [2/1] [U_]
md7 : active raid1 md126p9[0]
4883648 blocks [2/1] [U_]
md6 : active raid1 md126p8[0]
4883648 blocks [2/1] [U_]
md5 : active raid1 md126p7[0]
4883648 blocks [2/1] [U_]
md4 : active raid1 md126p6[0]
9767424 blocks [2/1] [U_]
md3 : active raid1 md126p5[0]
4883648 blocks [2/1] [U_]
md2 : active (auto-read-only) raid1 md126p3[0]
1951808 blocks [2/1] [U_]
md1 : active raid1 md126p2[0]
4883648 blocks [2/1] [U_]
md0 : active raid1 md126p1[0]
489856 blocks [2/1] [U_]
md126 : active raid1 sdc[1] sdb[0]
244195328 blocks super external:/md127/0 [2/2] [UU]
md127 : inactive sdc[1](S) sdb[0](S)
6306 blocks super external:imsm
unused devices: <none>
>-------------------------------------------------------------------------------------------------------------<
I'm not an expert at md at all, so I called a friend and we were able to
find my lost md9 partition, it was md126p11.
This is what we saw in dmesg:
>-------------------------------------------------------------------------------------------------------------<
[ 1.593297] md: bind<sdb>
[ 1.598582] md: bind<sdc>
[ 1.599902] md: bind<sdb>
[ 1.600045] md: bind<sdc>
[ 1.606550] md: raid1 personality registered for level 1
[ 1.607396] md/raid1:md126: active with 2 out of 2 mirrors
[ 1.607433] md126: detected capacity change from 0 to 250056015872
[ 1.632396] md: md126 switched to read-write mode.
[ 1.669910] md126: p1 p2 p3 p4 < p5 p6 p7 p8 p9 p10 p11 >
[ 1.670501] md126: p11 size 405368082 extends beyond EOD, truncated
[ 4.100396] md: md0 stopped.
[ 4.100919] md: bind<md126p1>
[ 4.101708] md/raid1:md0: active with 1 out of 2 mirrors
[ 4.101734] md0: detected capacity change from 0 to 501612544
[ 4.102179] md0: unknown partition table
[ 4.224625] md: md1 stopped.
[ 4.225109] md: bind<md126p2>
[ 4.225886] md/raid1:md1: active with 1 out of 2 mirrors
[ 4.225911] md1: detected capacity change from 0 to 5000855552
[ 4.226658] md1: unknown partition table
[ 4.420746] md: md2 stopped.
[ 4.421441] md: bind<md126p3>
[ 4.422216] md/raid1:md2: active with 1 out of 2 mirrors
[ 4.422241] md2: detected capacity change from 0 to 1998651392
[ 4.422677] md2: unknown partition table
[ 4.595729] md: md3 stopped.
[ 4.596410] md: bind<md126p5>
[ 4.597189] md/raid1:md3: active with 1 out of 2 mirrors
[ 4.597215] md3: detected capacity change from 0 to 5000855552
[ 4.597638] md3: unknown partition table
[ 4.668224] md: md4 stopped.
[ 4.668693] md: bind<md126p6>
[ 4.669446] md/raid1:md4: active with 1 out of 2 mirrors
[ 4.669474] md4: detected capacity change from 0 to 10001842176
[ 4.669909] md4: unknown partition table
[ 4.783732] md: md5 stopped.
[ 4.784236] md: bind<md126p7>
[ 4.785024] md/raid1:md5: active with 1 out of 2 mirrors
[ 4.785049] md5: detected capacity change from 0 to 5000855552
[ 4.785479] md5: unknown partition table
[ 4.970769] md: md6 stopped.
[ 4.971366] md: bind<md126p8>
[ 4.972129] md/raid1:md6: active with 1 out of 2 mirrors
[ 4.972158] md6: detected capacity change from 0 to 5000855552
[ 4.972594] md6: unknown partition table
[ 5.137394] md: md7 stopped.
[ 5.138011] md: bind<md126p9>
[ 5.138754] md/raid1:md7: active with 1 out of 2 mirrors
[ 5.138779] md7: detected capacity change from 0 to 5000855552
[ 5.139232] md7: unknown partition table
[ 5.329093] md: md8 stopped.
[ 5.330228] md: bind<md126p10>
[ 5.330977] md/raid1:md8: active with 1 out of 2 mirrors
[ 5.331003] md8: detected capacity change from 0 to 5000855552
[ 5.350896] md8: unknown partition table
>-------------------------------------------------------------------------------------------------------------<
note the line "[ 1.670501] md126: p11 size 405368082 extends beyond EOD,
truncated"
After force mounting /dev/md126p11, I noted these messages in dmesg:
>-------------------------------------------------------------------------------------------------------------<
[ 4361.248369] md126p11: rw=32, want=405367936, limit=405366673
[ 4361.248382] XFS (md126p11): Mounting V4 Filesystem
[ 4361.657842] XFS (md126p11): Ending clean mount
[ 4609.295281] md126p11: rw=32, want=405367936, limit=405366673
[ 4609.295300] XFS (md126p11): Mounting V4 Filesystem
[ 4609.513109] XFS (md126p11): Ending clean mount
>-------------------------------------------------------------------------------------------------------------<
and in daemon.log (lines correspond to the reboot after the upgrade, the
time I lost my md9):
>-------------------------------------------------------------------------------------------------------------<
Apr 29 20:26:26 shax mdadm[4735]: DeviceDisappeared event detected
on md device /dev/md9
Apr 29 20:26:29 shax mdadm[4735]: DegradedArray event detected on md
device /dev/md8
Apr 29 20:26:30 shax mdadm[4735]: DegradedArray event detected on md
device /dev/md7
Apr 29 20:26:31 shax mdadm[4735]: DegradedArray event detected on md
device /dev/md6
Apr 29 20:26:33 shax mdadm[4735]: DegradedArray event detected on md
device /dev/md5
Apr 29 20:26:34 shax mdadm[4735]: DegradedArray event detected on md
device /dev/md4
Apr 29 20:26:35 shax mdadm[4735]: DegradedArray event detected on md
device /dev/md3
Apr 29 20:26:35 shax mdadm[4735]: DegradedArray event detected on md
device /dev/md2
Apr 29 20:26:36 shax mdadm[4735]: DegradedArray event detected on md
device /dev/md1
Apr 29 20:26:37 shax mdadm[4735]: DegradedArray event detected on md
device /dev/md0
Apr 29 20:26:37 shax mdadm[4735]: NewArray event detected on md
device /dev/md127
Apr 29 20:26:37 shax mdadm[4735]: NewArray event detected on md
device /dev/md126
>-------------------------------------------------------------------------------------------------------------<
output of fdisk -l /dev/md126 :
>-------------------------------------------------------------------------------------------------------------<
fdisk -l /dev/md126
Disk /dev/md126: 250.1 GB, 250056015872 bytes
255 heads, 63 sectors/track, 30400 cylinders, total 488390656 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000c352c
Device Boot Start End Blocks Id System
/dev/md126p1 * 63 979964 489951 fd Linux
raid autodetect
/dev/md126p2 979965 10747484 4883760 fd Linux
raid autodetect
/dev/md126p3 10747485 14651279 1951897+ fd Linux
raid autodetect
/dev/md126p4 14651280 488392064 236870392+ 5 Extended
/dev/md126p5 14651343 24418799 4883728+ fd Linux
raid autodetect
/dev/md126p6 24418863 43953839 9767488+ fd Linux
raid autodetect
/dev/md126p7 43953903 53721359 4883728+ fd Linux
raid autodetect
/dev/md126p8 53721423 63488879 4883728+ fd Linux
raid autodetect
/dev/md126p9 63488943 73256399 4883728+ fd Linux
raid autodetect
/dev/md126p10 73256463 83023919 4883728+ fd Linux
raid autodetect
/dev/md126p11 83023983 488392064 202684041 fd Linux
raid autodetect
>-------------------------------------------------------------------------------------------------------------<
mdadm -E /dev/sdbX ou mdadm -E /dev/sdcX doesn't give anything :
>-------------------------------------------------------------------------------------------------------------<
# mdadm -E /dev/sdb1
mdadm: cannot open /dev/sdb1: No such device or address
# mdadm -E /dev/sdb2
mdadm: cannot open /dev/sdb2: No such device or address
# mdadm -E /dev/sdb3
mdadm: cannot open /dev/sdb3: No such device or address
# mdadm -E /dev/sdb4
mdadm: cannot open /dev/sdb4: No such device or address
# mdadm -E /dev/sdb5
mdadm: cannot open /dev/sdb5: No such device or address
# mdadm -E /dev/sdb6
mdadm: cannot open /dev/sdb6: No such device or address
# mdadm -E /dev/sdb7
mdadm: cannot open /dev/sdb7: No such device or address
# mdadm -E /dev/sdb8
mdadm: cannot open /dev/sdb8: No such device or address
# mdadm -E /dev/sdb9
mdadm: cannot open /dev/sdb9: No such device or address
# mdadm -E /dev/sdb10
mdadm: cannot open /dev/sdb10: No such device or address
# mdadm -E /dev/sdb11
mdadm: cannot open /dev/sdb11: No such device or address
>-------------------------------------------------------------------------------------------------------------<
mdadm -E /dev/md126pX do give something (except for md126p11):
>-------------------------------------------------------------------------------------------------------------<
# mdadm -E /dev/md126p1
/dev/md126p1:
Magic : a92b4efc
Version : 0.90.00
UUID : 350e253f:863b7b04:b1617c47:b213a024
Creation Time : Mon May 7 20:29:35 2007
Raid Level : raid1
Used Dev Size : 489856 (478.46 MiB 501.61 MB)
Array Size : 489856 (478.46 MiB 501.61 MB)
Raid Devices : 2
Total Devices : 1
Preferred Minor : 0
Update Time : Thu Apr 30 08:00:08 2015
State : clean
Active Devices : 1
Working Devices : 1
Failed Devices : 1
Spare Devices : 0
Checksum : 6372ca75 - correct
Events : 1769
Number Major Minor RaidDevice State
this 0 259 0 0 active sync
/dev/md/raidlin_0p1
0 0 259 0 0 active sync
/dev/md/raidlin_0p1
1 1 0 0 1 faulty removed
# mdadm -E /dev/md126p2
/dev/md126p2:
Magic : a92b4efc
Version : 0.90.00
UUID : 086e68ed:3607317f:60b56e23:6bae62bc
Creation Time : Mon May 7 20:29:41 2007
Raid Level : raid1
Used Dev Size : 4883648 (4.66 GiB 5.00 GB)
Array Size : 4883648 (4.66 GiB 5.00 GB)
Raid Devices : 2
Total Devices : 1
Preferred Minor : 1
Update Time : Thu Apr 30 20:57:21 2015
State : clean
Active Devices : 1
Working Devices : 1
Failed Devices : 1
Spare Devices : 0
Checksum : 4fd14bc9 - correct
Events : 3980
Number Major Minor RaidDevice State
this 0 259 1 0 active sync
/dev/md/raidlin_0p2
0 0 259 1 0 active sync
/dev/md/raidlin_0p2
1 1 0 0 1 faulty removed
# mdadm -E /dev/md126p3
/dev/md126p3:
Magic : a92b4efc
Version : 0.90.00
UUID : 0f6e3ed5:aeee975a:c3647deb:763d68ce
Creation Time : Mon May 7 20:29:48 2007
Raid Level : raid1
Used Dev Size : 1951808 (1906.38 MiB 1998.65 MB)
Array Size : 1951808 (1906.38 MiB 1998.65 MB)
Raid Devices : 2
Total Devices : 1
Preferred Minor : 2
Update Time : Thu Apr 30 18:27:58 2015
State : clean
Active Devices : 1
Working Devices : 1
Failed Devices : 1
Spare Devices : 0
Checksum : 3cc9ac24 - correct
Events : 1703
Number Major Minor RaidDevice State
this 0 259 2 0 active sync
/dev/md/raidlin_0p3
0 0 259 2 0 active sync
/dev/md/raidlin_0p3
1 1 0 0 1 faulty removed
# mdadm -E /dev/md126p4
/dev/md126p4:
MBR Magic : aa55
Partition[0] : 9767457 sectors at 63 (type fd)
Partition[1] : 19535040 sectors at 9767520 (type 05)
# mdadm -E /dev/md126p5
/dev/md126p5:
Magic : a92b4efc
Version : 0.90.00
UUID : 6b560fe8:f24d6f2e:8942bd3a:1903abbc
Creation Time : Mon May 7 20:29:55 2007
Raid Level : raid1
Used Dev Size : 4883648 (4.66 GiB 5.00 GB)
Array Size : 4883648 (4.66 GiB 5.00 GB)
Raid Devices : 2
Total Devices : 1
Preferred Minor : 3
Update Time : Thu Apr 30 20:57:18 2015
State : clean
Active Devices : 1
Working Devices : 1
Failed Devices : 1
Spare Devices : 0
Checksum : 44e1e6e6 - correct
Events : 7856
Number Major Minor RaidDevice State
this 0 259 4 0 active sync
/dev/md/raidlin_0p5
0 0 259 4 0 active sync
/dev/md/raidlin_0p5
1 1 0 0 1 faulty removed
# mdadm -E /dev/md126p6
/dev/md126p6:
Magic : a92b4efc
Version : 0.90.00
UUID : 462dab96:b7ca2a17:7c4aebf1:d4d7ec3b
Creation Time : Mon May 7 20:30:02 2007
Raid Level : raid1
Used Dev Size : 9767424 (9.31 GiB 10.00 GB)
Array Size : 9767424 (9.31 GiB 10.00 GB)
Raid Devices : 2
Total Devices : 1
Preferred Minor : 4
Update Time : Thu Apr 30 20:57:14 2015
State : clean
Active Devices : 1
Working Devices : 1
Failed Devices : 1
Spare Devices : 0
Checksum : 945d0c07 - correct
Events : 2999
Number Major Minor RaidDevice State
this 0 259 5 0 active sync
/dev/md/raidlin_0p6
0 0 259 5 0 active sync
/dev/md/raidlin_0p6
1 1 0 0 1 faulty removed
# mdadm -E /dev/md126p7
/dev/md126p7:
Magic : a92b4efc
Version : 0.90.00
UUID : ea511351:3abc7b12:4c81e838:93dbd21a
Creation Time : Mon May 7 20:30:09 2007
Raid Level : raid1
Used Dev Size : 4883648 (4.66 GiB 5.00 GB)
Array Size : 4883648 (4.66 GiB 5.00 GB)
Raid Devices : 2
Total Devices : 1
Preferred Minor : 5
Update Time : Thu Apr 30 20:57:18 2015
State : clean
Active Devices : 1
Working Devices : 1
Failed Devices : 1
Spare Devices : 0
Checksum : 4a634da7 - correct
Events : 8626
Number Major Minor RaidDevice State
this 0 259 6 0 active sync
/dev/md/raidlin_0p7
0 0 259 6 0 active sync
/dev/md/raidlin_0p7
1 1 0 0 1 faulty removed
# mdadm -E /dev/md126p8
/dev/md126p8:
Magic : a92b4efc
Version : 0.90.00
UUID : da0d76c6:91422584:dc3d6162:37ced53b
Creation Time : Mon May 7 20:30:15 2007
Raid Level : raid1
Used Dev Size : 4883648 (4.66 GiB 5.00 GB)
Array Size : 4883648 (4.66 GiB 5.00 GB)
Raid Devices : 2
Total Devices : 1
Preferred Minor : 6
Update Time : Thu Apr 30 20:57:31 2015
State : active
Active Devices : 1
Working Devices : 1
Failed Devices : 1
Spare Devices : 0
Checksum : c4540a0c - correct
Events : 30082
Number Major Minor RaidDevice State
this 0 259 7 0 active sync
/dev/md/raidlin_0p8
0 0 259 7 0 active sync
/dev/md/raidlin_0p8
1 1 0 0 1 faulty removed
# mdadm -E /dev/md126p9
/dev/md126p9:
Magic : a92b4efc
Version : 0.90.00
UUID : 387c831c:8a6d05e3:b649696c:0870b930
Creation Time : Mon May 7 20:30:21 2007
Raid Level : raid1
Used Dev Size : 4883648 (4.66 GiB 5.00 GB)
Array Size : 4883648 (4.66 GiB 5.00 GB)
Raid Devices : 2
Total Devices : 1
Preferred Minor : 7
Update Time : Thu Apr 30 20:57:10 2015
State : clean
Active Devices : 1
Working Devices : 1
Failed Devices : 1
Spare Devices : 0
Checksum : c69b931a - correct
Events : 4852
Number Major Minor RaidDevice State
this 0 259 8 0 active sync
/dev/md/raidlin_0p9
0 0 259 8 0 active sync
/dev/md/raidlin_0p9
1 1 0 0 1 faulty removed
# mdadm -E /dev/md126p10
/dev/md126p10:
Magic : a92b4efc
Version : 0.90.00
UUID : b07c4ab4:39d0ba53:9913afa9:fd9cc323
Creation Time : Mon May 7 20:30:28 2007
Raid Level : raid1
Used Dev Size : 4883648 (4.66 GiB 5.00 GB)
Array Size : 4883648 (4.66 GiB 5.00 GB)
Raid Devices : 2
Total Devices : 1
Preferred Minor : 8
Update Time : Thu Apr 30 20:57:28 2015
State : clean
Active Devices : 1
Working Devices : 1
Failed Devices : 1
Spare Devices : 0
Checksum : c5f5d015 - correct
Events : 19271
Number Major Minor RaidDevice State
this 0 259 9 0 active sync
/dev/md/raidlin_0p10
0 0 259 9 0 active sync
/dev/md/raidlin_0p10
1 1 0 0 1 faulty removed
# mdadm -E /dev/md126p11
mdadm: No md superblock detected on /dev/md126p11.
>-------------------------------------------------------------------------------------------------------------<
Some more commands:
>-------------------------------------------------------------------------------------------------------------<
# mdadm -D /dev/md126
/dev/md126:
Container : /dev/md/imsm0, member 0
Raid Level : raid1
Array Size : 244195328 (232.88 GiB 250.06 GB)
Used Dev Size : 244195328 (232.88 GiB 250.06 GB)
Raid Devices : 2
Total Devices : 2
State : active
Active Devices : 2
Working Devices : 2
Failed Devices : 0
Spare Devices : 0
UUID : 91449a9d:9242bfe9:d99bceb0:a59f9314
Number Major Minor RaidDevice State
1 8 32 0 active sync /dev/sdc
0 8 16 1 active sync /dev/sdb
>-------------------------------------------------------------------------------------------------------------<
# mdadm -E /dev/sdb
mdmon: /dev/sdb is not attached to Intel(R) RAID controller.
mdmon: /dev/sdb is not attached to Intel(R) RAID controller.
/dev/sdb:
Magic : Intel Raid ISM Cfg Sig.
Version : 1.1.00
Orig Family : 26b5a9e0
Family : 26b5a9e0
Generation : 00004db7
Attributes : All supported
UUID : d9cfa6d9:2a715e4f:1fbc2095:be342429
Checksum : 261d2aed correct
MPB Sectors : 1
Disks : 2
RAID Devices : 1
Disk01 Serial : VFC100R10BE79D
State : active
Id : 00010000
Usable Size : 488390862 (232.88 GiB 250.06 GB)
[raidlin]:
UUID : 91449a9d:9242bfe9:d99bceb0:a59f9314
RAID Level : 1
Members : 2
Slots : [UU]
Failed disk : none
This Slot : 1
Array Size : 488390656 (232.88 GiB 250.06 GB)
Per Dev Size : 488390656 (232.88 GiB 250.06 GB)
Sector Offset : 0
Num Stripes : 1907776
Chunk Size : 64 KiB
Reserved : 0
Migrate State : idle
Map State : normal
Dirty State : dirty
Disk00 Serial : VFC100R10BRKMD
State : active
Id : 00000000
Usable Size : 488390862 (232.88 GiB 250.06 GB)
>-------------------------------------------------------------------------------------------------------------<
# mdadm -E /dev/sdc
mdmon: /dev/sdc is not attached to Intel(R) RAID controller.
mdmon: /dev/sdc is not attached to Intel(R) RAID controller.
/dev/sdc:
Magic : Intel Raid ISM Cfg Sig.
Version : 1.1.00
Orig Family : 26b5a9e0
Family : 26b5a9e0
Generation : 00004dbc
Attributes : All supported
UUID : d9cfa6d9:2a715e4f:1fbc2095:be342429
Checksum : 261c2af2 correct
MPB Sectors : 1
Disks : 2
RAID Devices : 1
Disk00 Serial : VFC100R10BRKMD
State : active
Id : 00000000
Usable Size : 488390862 (232.88 GiB 250.06 GB)
[raidlin]:
UUID : 91449a9d:9242bfe9:d99bceb0:a59f9314
RAID Level : 1
Members : 2
Slots : [UU]
Failed disk : none
This Slot : 0
Array Size : 488390656 (232.88 GiB 250.06 GB)
Per Dev Size : 488390656 (232.88 GiB 250.06 GB)
Sector Offset : 0
Num Stripes : 1907776
Chunk Size : 64 KiB
Reserved : 0
Migrate State : idle
Map State : normal
Dirty State : clean
Disk01 Serial : VFC100R10BE79D
State : active
Id : 00010000
Usable Size : 488390862 (232.88 GiB 250.06 GB)
>-------------------------------------------------------------------------------------------------------------<
I see something wicked in there and I don't know how to correct it...
I see at least two problems :
- mdadm is not able to detect md9
- my array does not seems fully operationa
Can you help ?
If you need more logs / command output, tell me which one.
I hope I'm not too much confusing...
kernel : Linux shax 3.16.0-4-686-pae #1 SMP Debian 3.16.7-ckt9-3~deb8u1
(2015-04-24) i686 GNU/Linux
^ permalink raw reply
* Re: Help needed recovering from raid failure
From: NeilBrown @ 2015-05-01 2:31 UTC (permalink / raw)
To: Peter van Es; +Cc: linux-raid
In-Reply-To: <75230D05-6BFF-4906-921A-15175D6567EB@gmail.com>
[-- Attachment #1: Type: text/plain, Size: 1549 bytes --]
On Thu, 30 Apr 2015 21:25:04 +0200 Peter van Es <vanes.peter@gmail.com> wrote:
> Neil,
>
> thanks. I followed your instructions (slightly modified as my version of mdadm did not support the --data-offset stanza). /dev/sdd was the 3rd drive and I had physically removed the 4th drive from my server.
>
> I managed to restart the array. Then I replaced the failing drive, created partitions the same as on /dev/sda and added it to the two arrays.
>
> It is now rebuilding for the data array, and will be done in 440 minutes.... It appears that I've lost nothing important...
Excellent.
>
> One question: I did spot that the Array UUID has changed on the Create command. Is there any way of getting it back to the old value ?
Why would you want to?
But I think you can. Firstly stop the array (so you need to be booted from a
USB or similar) and then
mdadm --assemble /dev/mdWHATEVER --update=uuid --uuid=your:favo:rite:nums ..list.of.devices..
NeilBrown
>
> Peter
>
>
> >
> > Before doing this, double check that the names have changed, so check that
> > mdadm --examine /dev/sda2
> > shows
> >> Array UUID : 1f28f7bb:7b3ecd41:ca0fa5d1:ccd008df
> >> Device Role : Active device 0
> >
> > (among other info) and that
> > mdadm --exmaine /dev/sdb2
> > show the same Array UUID and
> >> Device Role : Active device 1
> >
> >
> > Then run
> >
> > mdadm -C /dev/md1 -l5 -n4 --data-offset=262144s --metadata=1.2 --assume-clean \
> > /dev/sda2 /dev/sdb2 missing /dev/sde2
>
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 811 bytes --]
^ permalink raw reply
* Re: Help needed recovering from raid failure
From: Peter van Es @ 2015-04-30 19:25 UTC (permalink / raw)
To: NeilBrown; +Cc: linux-raid
In-Reply-To: <20150430092741.0dc24c39@notabene.brown>
Neil,
thanks. I followed your instructions (slightly modified as my version of mdadm did not support the --data-offset stanza). /dev/sdd was the 3rd drive and I had physically removed the 4th drive from my server.
I managed to restart the array. Then I replaced the failing drive, created partitions the same as on /dev/sda and added it to the two arrays.
It is now rebuilding for the data array, and will be done in 440 minutes.... It appears that I've lost nothing important...
One question: I did spot that the Array UUID has changed on the Create command. Is there any way of getting it back to the old value ?
Peter
>
> Before doing this, double check that the names have changed, so check that
> mdadm --examine /dev/sda2
> shows
>> Array UUID : 1f28f7bb:7b3ecd41:ca0fa5d1:ccd008df
>> Device Role : Active device 0
>
> (among other info) and that
> mdadm --exmaine /dev/sdb2
> show the same Array UUID and
>> Device Role : Active device 1
>
>
> Then run
>
> mdadm -C /dev/md1 -l5 -n4 --data-offset=262144s --metadata=1.2 --assume-clean \
> /dev/sda2 /dev/sdb2 missing /dev/sde2
^ permalink raw reply
* Re: Potential race in dlm based messaging md-cluster.c
From: Abhijit Bhopatkar @ 2015-04-30 18:51 UTC (permalink / raw)
To: Goldwyn Rodrigues; +Cc: linux-raid
In-Reply-To: <554278CF.4000401@cisco.com>
On 01/05/15 12:17 am, Abhijit Bhopatkar wrote:
> On 01/05/15 12:06 am, Abhijit Bhopatkar wrote:
>> There is a possibility of a receiver losing out on messages in certain
>> corner conditions. One of the buggy case is if there is are two sender
>> ready with messages to be sent. Sender 1 initially gets the TOKEN lock
>> and proceeds.
>> After initial processing the sender of message 1 _will_ release TOKEN as
>> soon as receiver releases ACK, it does not wait till ACK CR is
>> re-acquired by receiver.
>>
> I could not come up with any solution except to add one more lock
> resource for now we will call it "SYNC"
>
Here is POC patch (completely untested only as an RFC not even compiled)
If the solution is agreed upon I will go ahead and test it.
Abhijit
---
diff --git a/drivers/md/md-cluster.c b/drivers/md/md-cluster.c
index fcfc4b9..addbbb4 100644
--- a/drivers/md/md-cluster.c
+++ b/drivers/md/md-cluster.c
@@ -62,6 +62,7 @@ struct md_cluster_info {
struct dlm_lock_resource *ack_lockres;
struct dlm_lock_resource *message_lockres;
struct dlm_lock_resource *token_lockres;
+ struct dlm_lock_resource *sync_lockres;
struct dlm_lock_resource *no_new_dev_lockres;
struct md_thread *recv_thread;
struct completion newdisk_completion;
@@ -94,7 +95,7 @@ static void sync_ast(void *arg)
complete(&res->completion);
}
-static int dlm_lock_sync(struct dlm_lock_resource *res, int mode)
+static int dlm_lock_queue(struct dlm_lock_resource *res, int mode)
{
int ret = 0;
@@ -102,12 +103,26 @@ static int dlm_lock_sync(struct dlm_lock_resource *res, int mode)
ret = dlm_lock(res->ls, mode, &res->lksb,
res->flags, res->name, strlen(res->name),
0, sync_ast, res, res->bast);
- if (ret)
- return ret;
+ return ret;
+}
+
+static int dlm_wait_for_lock_grant(struct dlm_lock_resource *res)
+{
wait_for_completion(&res->completion);
return res->lksb.sb_status;
}
+static int dlm_lock_sync(struct dlm_lock_resource *res, int mode)
+{
+ int ret = 0;
+ ret = dlm_lock_queue(res,mode);
+
+ if (ret)
+ return ret;
+ ret = dlm_wait_for_lock_grant(res);
+ return ret;
+}
+
static int dlm_unlock_sync(struct dlm_lock_resource *res)
{
return dlm_lock_sync(res, DLM_LOCK_NL);
@@ -466,6 +481,7 @@ static void recv_daemon(struct md_thread *thread)
struct md_cluster_info *cinfo = thread->mddev->cluster_info;
struct dlm_lock_resource *ack_lockres = cinfo->ack_lockres;
struct dlm_lock_resource *message_lockres = cinfo->message_lockres;
+ struct dlm_lock_resource *sync_lockres = cinfo->sync_lockres;
struct cluster_msg msg;
/*get CR on Message*/
@@ -478,6 +494,9 @@ static void recv_daemon(struct md_thread *thread)
memcpy(&msg, message_lockres->lksb.sb_lvbptr, sizeof(struct cluster_msg));
process_recvd_msg(thread->mddev, &msg);
+ /*queue EX on TOKEN blocks new senders till we acquire CR on ACK */
+ dlm_lock_queue(sync_lockres,DLM_LOCK_EX);
+
/*release CR on ack_lockres*/
dlm_unlock_sync(ack_lockres);
/*up-convert to EX on message_lockres*/
@@ -486,6 +505,11 @@ static void recv_daemon(struct md_thread *thread)
dlm_lock_sync(ack_lockres, DLM_LOCK_CR);
/*release CR on message_lockres*/
dlm_unlock_sync(message_lockres);
+
+ /*wait till EX on token is granted */
+ dlm_wait_for_lock_grant(token_lockres);
+ /*release EX on token_lockres*/
+ dlm_unlock_sync(sync_lockres);
}
/* lock_comm()
@@ -500,11 +524,16 @@ static int lock_comm(struct md_cluster_info *cinfo)
if (error)
pr_err("md-cluster(%s:%d): failed to get EX on TOKEN (%d)\n",
__func__, __LINE__, error);
+ error = dlm_lock_sync(cinfo->sync_lockres, DLM_LOCK_EX);
+ if (error)
+ pr_err("md-cluster(%s:%d): failed to get EX on SYNC (%d)\n",
+ __func__, __LINE__, error);
return error;
}
static void unlock_comm(struct md_cluster_info *cinfo)
{
+ dlm_unlock_sync(cinfo->sync_lockres);
dlm_unlock_sync(cinfo->token_lockres);
}
@@ -673,6 +702,9 @@ static int join(struct mddev *mddev, int nodes)
cinfo->token_lockres = lockres_init(mddev, "token", NULL, 0);
if (!cinfo->token_lockres)
goto err;
+ cinfo->sync_lockres = lockres_init(mddev, "sync", NULL, 0);
+ if (!cinfo->sync_lockres)
+ goto err;
cinfo->ack_lockres = lockres_init(mddev, "ack", ack_bast, 0);
if (!cinfo->ack_lockres)
goto err;
@@ -711,6 +743,7 @@ static int join(struct mddev *mddev, int nodes)
err:
lockres_free(cinfo->message_lockres);
lockres_free(cinfo->token_lockres);
+ lockres_free(cinfo->sync_lockres);
lockres_free(cinfo->ack_lockres);
lockres_free(cinfo->no_new_dev_lockres);
lockres_free(cinfo->bitmap_lockres);
@@ -733,6 +766,7 @@ static int leave(struct mddev *mddev)
md_unregister_thread(&cinfo->recv_thread);
lockres_free(cinfo->message_lockres);
lockres_free(cinfo->token_lockres);
+ lockres_free(cinfo->sync_lockres);
lockres_free(cinfo->ack_lockres);
lockres_free(cinfo->no_new_dev_lockres);
lockres_free(cinfo->sb_lock);
^ permalink raw reply related
* Re: Potential race in dlm based messaging md-cluster.c
From: Abhijit Bhopatkar @ 2015-04-30 18:47 UTC (permalink / raw)
To: Goldwyn Rodrigues; +Cc: linux-raid
In-Reply-To: <5542763C.90202@cisco.com>
On 01/05/15 12:06 am, Abhijit Bhopatkar wrote:
> There is a possibility of a receiver losing out on messages in certain
> corner conditions. One of the buggy case is if there is are two sender
> ready with messages to be sent. Sender 1 initially gets the TOKEN lock
> and proceeds.
> After initial processing the sender of message 1 _will_ release TOKEN as
> soon as receiver releases ACK, it does not wait till ACK CR is
> re-acquired by receiver.
>
I could not come up with any solution except to add one more lock
resource for now we will call it "SYNC"
Sender 1 Sender2 Receiver
Get EX on TOKEN Get EX on TOKEN
Get EX on SYNC <Wait till granted>
<Granted>
Get EX on MSG
write LVB
down MSG to CR
Get EX of ACK
<wait till granted> BAST for ACK
Get CR on MSG
read LVB
<process>
Queue EX on SYNC
release ACK
AST for ACK
down ACK to CR
release MSG
release SYNC
release TOKEN
SYNC granted
<granted>
Get EX on SYNC
<wait till grant>
Get EX on MSG
Get CR on ACK
release MSG
release SYNC
Get EX on MSG
<....proceed rest>
release TOKEN
The key thing to note here is that the SYNC lock request is only queued
in receiver path. Having worked in dlm before I know for sure this will
work as expected.
Abhijit
^ permalink raw reply
* Potential race in dlm based messaging md-cluster.c
From: Abhijit Bhopatkar @ 2015-04-30 18:36 UTC (permalink / raw)
To: Goldwyn Rodrigues; +Cc: linux-raid
In-Reply-To: <CAE3Hb8pJ=0MB6EX5jVch28gj-gnf0Mp1wyzxBfWjzLf=SuV4sQ@mail.gmail.com>
There is a possibility of a receiver losing out on messages in certain
corner conditions. One of the buggy case is if there is are two sender
ready with messages to be sent. Sender 1 initially gets the TOKEN lock
and proceeds.
After initial processing the sender of message 1 _will_ release TOKEN as
soon as receiver releases ACK, it does not wait till ACK CR is
re-acquired by receiver.
To illustrate the problem consider timeline for two senders and one
receiver (we will ignore receive part for Sender2 node)
Sender1 Sender2 Receiver
Get EX on TOKEN Get EX on TOKEN
<Granted> <Wait till granted>
Get EX on MSG
write LVB
down MSG to CR
Get EX of ACK
<wait till granted>
BAST for ACK
Get CR on MSG
read LVB
process
release ACK
AST for ACK
down ACK to CR
release MSG
release TOKEN
<granted>
Get EX on MSG
<... proceed ...>
release TOKEN
<lost one message>
^^^^^^^^^^^^^^^^^
Get EX on MSG
Get CR on ACK
release MSG
Abhijit
^ permalink raw reply
* Re: [PATCH 03/10] Create n bitmaps for clustered mode
From: Goldwyn Rodrigues @ 2015-04-30 12:44 UTC (permalink / raw)
To: NeilBrown; +Cc: gqjiang, linux-raid
In-Reply-To: <20150430125153.428f4884@notabene.brown>
On 04/29/2015 09:51 PM, NeilBrown wrote:
> On Tue, 28 Apr 2015 21:41:47 -0500 Goldwyn Rodrigues <rgoldwyn@suse.de> wrote:
>
>>
>>
>> On 04/28/2015 08:36 PM, NeilBrown wrote:
>>> On Fri, 24 Apr 2015 15:30:34 +0800 gqjiang@suse.com wrote:
>>>
>>>> From: Guoqing Jiang <gqjiang@suse.com>
>>>>
>>>> For a clustered MD, create bitmaps equal to number of nodes so
>>>> each node has an independent bitmap.
>>>>
>>>> Only the first bitmap is has the bits set so that the first node
>>>> that assembles the device also performs the sync.
>>>>
>>>> The bitmaps are aligned to 4k boundaries.
>>>>
>>>> On-disk format:
>>>>
>>>> 0 4k 8k 12k
>>>> -------------------------------------------------------------------
>>>> | idle | md super | bm super [0] + bits |
>>>> | bm bits[0, contd] | bm super[1] + bits | bm bits[1, contd] |
>>>> | bm super[2] + bits | bm bits [2, contd] | bm super[3] + bits |
>>>> | bm bits [3, contd] | | |
>>>>
>>>> Signed-off-by: Goldwyn Rodrigues <rgoldwyn@suse.com>
>>>> Signed-off-by: Guoqing Jiang <gqjiang@suse.com>
>>>> ---
>>>> Create.c | 3 ++-
>>>> bitmap.h | 7 +++++--
>>>> mdadm.8.in | 7 ++++++-
>>>> mdadm.c | 17 ++++++++++++++++-
>>>> super1.c | 59 +++++++++++++++++++++++++++++++++++++++++------------------
>>>> 5 files changed, 70 insertions(+), 23 deletions(-)
>>>>
>>>> diff --git a/Create.c b/Create.c
>>>> index cd5485b..9663dc4 100644
>>>> --- a/Create.c
>>>> +++ b/Create.c
>>>> @@ -752,7 +752,8 @@ int Create(struct supertype *st, char *mddev,
>>>> #endif
>>>> }
>>>>
>>>> - if (s->bitmap_file && strcmp(s->bitmap_file, "internal")==0) {
>>>> + if (s->bitmap_file && (strcmp(s->bitmap_file, "internal")==0
>>>> + || strcmp(s->bitmap_file, "clustered")==0)) {
>>>> if ((vers%100) < 2) {
>>>> pr_err("internal bitmaps not supported by this kernel.\n");
>>>> goto abort_locked;
>>>> diff --git a/bitmap.h b/bitmap.h
>>>> index c8725a3..adbf0b4 100644
>>>> --- a/bitmap.h
>>>> +++ b/bitmap.h
>>>> @@ -154,8 +154,11 @@ typedef struct bitmap_super_s {
>>>> __u32 chunksize; /* 52 the bitmap chunk size in bytes */
>>>> __u32 daemon_sleep; /* 56 seconds between disk flushes */
>>>> __u32 write_behind; /* 60 number of outstanding write-behind writes */
>>>> -
>>>> - __u8 pad[256 - 64]; /* set to zero */
>>>> + __u32 sectors_reserved; /* 64 number of 512-byte sectors that are
>>>> + * reserved for the bitmap. */
>>>> + __u32 nodes; /* 68 the maximum number of nodes in cluster. */
>>>> + __u8 cluster_name[64]; /* 72 cluster name to which this md belongs */
>>>> + __u8 pad[256 - 136]; /* set to zero */
>>>> } bitmap_super_t;
>>>>
>>>> /* notes:
>>>> diff --git a/mdadm.8.in b/mdadm.8.in
>>>> index a0e8288..c015cbf 100644
>>>> --- a/mdadm.8.in
>>>> +++ b/mdadm.8.in
>>>> @@ -700,7 +700,12 @@ and so is replicated on all devices. If the word
>>>> .B "none"
>>>> is given with
>>>> .B \-\-grow
>>>> -mode, then any bitmap that is present is removed.
>>>> +mode, then any bitmap that is present is removed. If the word
>>>> +.B "clustered"
>>>> +is given, the array is created for a clustered environment. One bitmap
>>>> +is created for each node as defined by the
>>>> +.B \-\-nodes
>>>> +parameter and are stored internally.
>>>>
>>>> To help catch typing errors, the filename must contain at least one
>>>> slash ('/') if it is a real file (not 'internal' or 'none').
>>>> diff --git a/mdadm.c b/mdadm.c
>>>> index e4f8568..6963a09 100644
>>>> --- a/mdadm.c
>>>> +++ b/mdadm.c
>>>> @@ -1111,6 +1111,15 @@ int main(int argc, char *argv[])
>>>> s.bitmap_file = optarg;
>>>> continue;
>>>> }
>>>> + if (strcmp(optarg, "clustered")== 0) {
>>>> + s.bitmap_file = optarg;
>>>> + /* Set the default number of cluster nodes
>>>> + * to 4 if not already set by user
>>>> + */
>>>> + if (c.nodes < 1)
>>>> + c.nodes = 4;
>>>> + continue;
>>>> + }
>>>> /* probable typo */
>>>> pr_err("bitmap file must contain a '/', or be 'internal', or 'none'\n"
>>>> " not '%s'\n", optarg);
>>>> @@ -1404,7 +1413,13 @@ int main(int argc, char *argv[])
>>>> if (c.delay == 0)
>>>> c.delay = DEFAULT_BITMAP_DELAY;
>>>>
>>>> - if (!strncmp(s.bitmap_file, "internal", 9) ||
>>>> + if (!strncmp(s.bitmap_file, "clustered", 9)) {
>>>> + if (s.level != 1) {
>>>> + pr_err("--bitmap=clustered is currently supported with RAID mirror only\n");
>>>> + rv = 1;
>>>> + break;
>>>> + }
>>>> + } else if (!strncmp(s.bitmap_file, "internal", 9) ||
>>>> !strncmp(s.bitmap_file,"none", 4)) {
>>>> if (c.nodes) {
>>>> pr_err("--nodes argument is incompatible with --bitmap=%s.\n",
>>>> diff --git a/super1.c b/super1.c
>>>> index f0508fe..ac1b011 100644
>>>> --- a/super1.c
>>>> +++ b/super1.c
>>>> @@ -2144,6 +2144,10 @@ add_internal_bitmap1(struct supertype *st,
>>>> bms->daemon_sleep = __cpu_to_le32(delay);
>>>> bms->sync_size = __cpu_to_le64(size);
>>>> bms->write_behind = __cpu_to_le32(write_behind);
>>>> + bms->nodes = __cpu_to_le32(st->nodes);
>>>> + if (st->cluster_name)
>>>> + strncpy((char *)bms->cluster_name,
>>>> + st->cluster_name, strlen(st->cluster_name));
>>>>
>>>> *chunkp = chunk;
>>>> return 1;
>>>> @@ -2177,6 +2181,7 @@ static int write_bitmap1(struct supertype *st, int fd)
>>>> void *buf;
>>>> int towrite, n;
>>>> struct align_fd afd;
>>>> + unsigned int i;
>>>>
>>>> init_afd(&afd, fd);
>>>>
>>>> @@ -2185,27 +2190,45 @@ static int write_bitmap1(struct supertype *st, int fd)
>>>> if (posix_memalign(&buf, 4096, 4096))
>>>> return -ENOMEM;
>>>>
>>>> - memset(buf, 0xff, 4096);
>>>> - memcpy(buf, (char *)bms, sizeof(bitmap_super_t));
>>>> -
>>>> - towrite = __le64_to_cpu(bms->sync_size) / (__le32_to_cpu(bms->chunksize)>>9);
>>>> - towrite = (towrite+7) >> 3; /* bits to bytes */
>>>> - towrite += sizeof(bitmap_super_t);
>>>> - towrite = ROUND_UP(towrite, 512);
>>>> - while (towrite > 0) {
>>>> - n = towrite;
>>>> - if (n > 4096)
>>>> - n = 4096;
>>>> - n = awrite(&afd, buf, n);
>>>> - if (n > 0)
>>>> - towrite -= n;
>>>> + /* We use bms->nodes as opposed to st->nodes to
>>>> + * be compatible with write-after-reads such as
>>>> + * the GROW operation.
>>>> + */
>>>> + for (i = 0; i < __le32_to_cpu(bms->nodes); i++) {
>>>> + /* Only the first bitmap should resync
>>>> + * the whole device
>>>> + */
>>>> + if (i)
>>>> + memset(buf, 0x00, 4096);
>>>> else
>>>> + memset(buf, 0xff, 4096);
>>>
>>> Why is the first bitmap initialised to 0x00 and the others to 0xff?
>>> If there is a good reason it should be documented either in a comment in the
>>> code or in the changelog entry.
>>
>>
>> Rather, it is the reverse. The first one is initialized to 0xff and the
>> rest are set to 0x00.
>>
>> The reason is only the first node to assemble the device should perform
>> the resync (if --assume-clean is not provided). The comment is right
>> above the code. Perhaps I should be more elaborate with the comment.
>>
>>
>
> Hmmm... Perhaps I should read code with my eyes open!
>
> Not sure I agree though. Why should the first node be special?
> What if node '0' doesn't get activated?
> I guess it always well because of the way numbers are assigned, but I'm not
> feeling very comfortable...
Yes, the first (zero'th) one will get activated first. The cluster
communication will guarantee that.
In any case, we do have fallback scenarios:
- In case of failure of the first node, the "alive" node will take over
- All bitmaps are checked by the kernel while assembling. This works for
a total cluster failure as well.
> Thinking a bit more ... why do we set any bits to '1'?
This is how the original non-clustered code is, I just moved it to the
zeroth node :)
> Why not just set BITMAP_STALE, and let the kernel figure things out.
>
> For the single-node case, BITMAP_STALE is the same as setting all the bits to
> one.
> For the cluster case, we can get BITMAP_STALE to do whatever we want. and we
> should make sure we handle it correctly anyway.
>
> So maybe mdadm should set BITMAP_STALE, and leave all the bits as 0.
>
> Thoughts?
Should it be set for all bitmaps? If yes, how should the second node
behave on observing that BITMAP_STALE is set while assembling? I suppose
we can clear the flag when the (first) node is reading all bitmaps.
I suppose with --assume-clean, we just not set the BITMAP_STALE. Right?
--
Goldwyn
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox