* [PATCH v4 05/12] crypto: LLVMLinux: Remove VLAIS from crypto/n2_core.c
From: behanw @ 2014-09-23 4:42 UTC (permalink / raw)
To: agk, clm, davem, dm-devel, fabf, herbert, jbacik, snitzer,
tadeusz.struk
Cc: thomas.lendacky, linux-ima-user, qat-linux, d.kasatkin,
bruce.w.allan, linux-kernel, john.griffin, linux-raid,
linux-security-module, Behan Webster, linux-crypto,
james.l.morris, torvalds, linux-ima-devel, akpm, zohar,
linux-btrfs, serge
In-Reply-To: <1411447337-22362-1-git-send-email-behanw@converseincode.com>
From: Behan Webster <behanw@converseincode.com>
Replaced the use of a Variable Length Array In Struct (VLAIS) with a C99
compliant equivalent. This patch allocates the appropriate amount of memory
using a char array using the SHASH_DESC_ON_STACK macro.
The new code can be compiled with both gcc and clang.
Signed-off-by: Behan Webster <behanw@converseincode.com>
Reviewed-by: Mark Charlebois <charlebm@gmail.com>
Reviewed-by: Jan-Simon Möller <dl9pf@gmx.de>
Acked-by: Herbert Xu <herbert@gondor.apana.org.au>
---
drivers/crypto/n2_core.c | 11 ++++-------
1 file changed, 4 insertions(+), 7 deletions(-)
diff --git a/drivers/crypto/n2_core.c b/drivers/crypto/n2_core.c
index 7263c10..f8e3207 100644
--- a/drivers/crypto/n2_core.c
+++ b/drivers/crypto/n2_core.c
@@ -445,10 +445,7 @@ static int n2_hmac_async_setkey(struct crypto_ahash *tfm, const u8 *key,
struct n2_hmac_ctx *ctx = crypto_ahash_ctx(tfm);
struct crypto_shash *child_shash = ctx->child_shash;
struct crypto_ahash *fallback_tfm;
- struct {
- struct shash_desc shash;
- char ctx[crypto_shash_descsize(child_shash)];
- } desc;
+ SHASH_DESC_ON_STACK(shash, child_shash);
int err, bs, ds;
fallback_tfm = ctx->base.fallback_tfm;
@@ -456,15 +453,15 @@ static int n2_hmac_async_setkey(struct crypto_ahash *tfm, const u8 *key,
if (err)
return err;
- desc.shash.tfm = child_shash;
- desc.shash.flags = crypto_ahash_get_flags(tfm) &
+ shash->tfm = child_shash;
+ shash->flags = crypto_ahash_get_flags(tfm) &
CRYPTO_TFM_REQ_MAY_SLEEP;
bs = crypto_shash_blocksize(child_shash);
ds = crypto_shash_digestsize(child_shash);
BUG_ON(ds > N2_HASH_KEY_MAX);
if (keylen > bs) {
- err = crypto_shash_digest(&desc.shash, key, keylen,
+ err = crypto_shash_digest(shash, key, keylen,
ctx->hash_key);
if (err)
return err;
--
1.9.1
--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
^ permalink raw reply related
* [PATCH v4 04/12] crypto: LLVMLinux: Remove VLAIS from crypto/mv_cesa.c
From: behanw @ 2014-09-23 4:42 UTC (permalink / raw)
To: agk, clm, davem, dm-devel, fabf, herbert, jbacik, snitzer,
tadeusz.struk
Cc: akpm, bruce.w.allan, d.kasatkin, james.l.morris, john.griffin,
linux-btrfs, linux-crypto, linux-ima-devel, linux-ima-user,
linux-kernel, linux-raid, linux-security-module, neilb, qat-linux,
serge, thomas.lendacky, zohar, torvalds, Behan Webster
In-Reply-To: <1411447337-22362-1-git-send-email-behanw@converseincode.com>
From: Behan Webster <behanw@converseincode.com>
Replaced the use of a Variable Length Array In Struct (VLAIS) with a C99
compliant equivalent. This patch allocates the appropriate amount of memory
using a char array using the SHASH_DESC_ON_STACK macro.
The new code can be compiled with both gcc and clang.
Signed-off-by: Behan Webster <behanw@converseincode.com>
Reviewed-by: Mark Charlebois <charlebm@gmail.com>
Reviewed-by: Jan-Simon Möller <dl9pf@gmx.de>
Acked-by: Herbert Xu <herbert@gondor.apana.org.au>
---
drivers/crypto/mv_cesa.c | 41 ++++++++++++++++++-----------------------
1 file changed, 18 insertions(+), 23 deletions(-)
diff --git a/drivers/crypto/mv_cesa.c b/drivers/crypto/mv_cesa.c
index 29d0ee5..032c72c 100644
--- a/drivers/crypto/mv_cesa.c
+++ b/drivers/crypto/mv_cesa.c
@@ -402,26 +402,23 @@ static int mv_hash_final_fallback(struct ahash_request *req)
{
const struct mv_tfm_hash_ctx *tfm_ctx = crypto_tfm_ctx(req->base.tfm);
struct mv_req_hash_ctx *req_ctx = ahash_request_ctx(req);
- struct {
- struct shash_desc shash;
- char ctx[crypto_shash_descsize(tfm_ctx->fallback)];
- } desc;
+ SHASH_DESC_ON_STACK(shash, tfm_ctx->fallback);
int rc;
- desc.shash.tfm = tfm_ctx->fallback;
- desc.shash.flags = CRYPTO_TFM_REQ_MAY_SLEEP;
+ shash->tfm = tfm_ctx->fallback;
+ shash->flags = CRYPTO_TFM_REQ_MAY_SLEEP;
if (unlikely(req_ctx->first_hash)) {
- crypto_shash_init(&desc.shash);
- crypto_shash_update(&desc.shash, req_ctx->buffer,
+ crypto_shash_init(shash);
+ crypto_shash_update(shash, req_ctx->buffer,
req_ctx->extra_bytes);
} else {
/* only SHA1 for now....
*/
- rc = mv_hash_import_sha1_ctx(req_ctx, &desc.shash);
+ rc = mv_hash_import_sha1_ctx(req_ctx, shash);
if (rc)
goto out;
}
- rc = crypto_shash_final(&desc.shash, req->result);
+ rc = crypto_shash_final(shash, req->result);
out:
return rc;
}
@@ -794,23 +791,21 @@ static int mv_hash_setkey(struct crypto_ahash *tfm, const u8 * key,
ss = crypto_shash_statesize(ctx->base_hash);
{
- struct {
- struct shash_desc shash;
- char ctx[crypto_shash_descsize(ctx->base_hash)];
- } desc;
+ SHASH_DESC_ON_STACK(shash, ctx->base_hash);
+
unsigned int i;
char ipad[ss];
char opad[ss];
- desc.shash.tfm = ctx->base_hash;
- desc.shash.flags = crypto_shash_get_flags(ctx->base_hash) &
+ shash->tfm = ctx->base_hash;
+ shash->flags = crypto_shash_get_flags(ctx->base_hash) &
CRYPTO_TFM_REQ_MAY_SLEEP;
if (keylen > bs) {
int err;
err =
- crypto_shash_digest(&desc.shash, key, keylen, ipad);
+ crypto_shash_digest(shash, key, keylen, ipad);
if (err)
return err;
@@ -826,12 +821,12 @@ static int mv_hash_setkey(struct crypto_ahash *tfm, const u8 * key,
opad[i] ^= 0x5c;
}
- rc = crypto_shash_init(&desc.shash) ? :
- crypto_shash_update(&desc.shash, ipad, bs) ? :
- crypto_shash_export(&desc.shash, ipad) ? :
- crypto_shash_init(&desc.shash) ? :
- crypto_shash_update(&desc.shash, opad, bs) ? :
- crypto_shash_export(&desc.shash, opad);
+ rc = crypto_shash_init(shash) ? :
+ crypto_shash_update(shash, ipad, bs) ? :
+ crypto_shash_export(shash, ipad) ? :
+ crypto_shash_init(shash) ? :
+ crypto_shash_update(shash, opad, bs) ? :
+ crypto_shash_export(shash, opad);
if (rc == 0)
mv_hash_init_ivs(ctx, ipad, opad);
--
1.9.1
^ permalink raw reply related
* [PATCH v4 03/12] crypto: LLVMLinux: Remove VLAIS from crypto/ccp/ccp-crypto-sha.c
From: behanw @ 2014-09-23 4:42 UTC (permalink / raw)
To: agk, clm, davem, dm-devel, fabf, herbert, jbacik, snitzer,
tadeusz.struk
Cc: akpm, bruce.w.allan, d.kasatkin, james.l.morris, john.griffin,
linux-btrfs, linux-crypto, linux-ima-devel, linux-ima-user,
linux-kernel, linux-raid, linux-security-module, neilb, qat-linux,
serge, thomas.lendacky, zohar, torvalds, Jan-Simon Möller,
Behan Webster
In-Reply-To: <1411447337-22362-1-git-send-email-behanw@converseincode.com>
From: Jan-Simon Möller <dl9pf@gmx.de>
Replaced the use of a Variable Length Array In Struct (VLAIS) with a C99
compliant equivalent. This patch allocates the appropriate amount of memory
using a char array using the SHASH_DESC_ON_STACK macro.
The new code can be compiled with both gcc and clang.
Signed-off-by: Jan-Simon Möller <dl9pf@gmx.de>
Signed-off-by: Behan Webster <behanw@converseincode.com>
Reviewed-by: Mark Charlebois <charlebm@gmail.com>
Acked-by: Herbert Xu <herbert@gondor.apana.org.au>
---
drivers/crypto/ccp/ccp-crypto-sha.c | 13 ++++++-------
1 file changed, 6 insertions(+), 7 deletions(-)
diff --git a/drivers/crypto/ccp/ccp-crypto-sha.c b/drivers/crypto/ccp/ccp-crypto-sha.c
index 873f234..9653157 100644
--- a/drivers/crypto/ccp/ccp-crypto-sha.c
+++ b/drivers/crypto/ccp/ccp-crypto-sha.c
@@ -198,10 +198,9 @@ static int ccp_sha_setkey(struct crypto_ahash *tfm, const u8 *key,
{
struct ccp_ctx *ctx = crypto_tfm_ctx(crypto_ahash_tfm(tfm));
struct crypto_shash *shash = ctx->u.sha.hmac_tfm;
- struct {
- struct shash_desc sdesc;
- char ctx[crypto_shash_descsize(shash)];
- } desc;
+
+ SHASH_DESC_ON_STACK(sdesc, shash);
+
unsigned int block_size = crypto_shash_blocksize(shash);
unsigned int digest_size = crypto_shash_digestsize(shash);
int i, ret;
@@ -216,11 +215,11 @@ static int ccp_sha_setkey(struct crypto_ahash *tfm, const u8 *key,
if (key_len > block_size) {
/* Must hash the input key */
- desc.sdesc.tfm = shash;
- desc.sdesc.flags = crypto_ahash_get_flags(tfm) &
+ sdesc->tfm = shash;
+ sdesc->flags = crypto_ahash_get_flags(tfm) &
CRYPTO_TFM_REQ_MAY_SLEEP;
- ret = crypto_shash_digest(&desc.sdesc, key, key_len,
+ ret = crypto_shash_digest(sdesc, key, key_len,
ctx->u.sha.key);
if (ret) {
crypto_ahash_set_flags(tfm, CRYPTO_TFM_RES_BAD_KEY_LEN);
--
1.9.1
^ permalink raw reply related
* [PATCH v4 02/12] btrfs: LLVMLinux: Remove VLAIS
From: behanw @ 2014-09-23 4:42 UTC (permalink / raw)
To: agk, clm, davem, dm-devel, fabf, herbert, jbacik, snitzer,
tadeusz.struk
Cc: akpm, bruce.w.allan, d.kasatkin, james.l.morris, john.griffin,
linux-btrfs, linux-crypto, linux-ima-devel, linux-ima-user,
linux-kernel, linux-raid, linux-security-module, neilb, qat-linux,
serge, thomas.lendacky, zohar, torvalds, Vinícius Tinti,
Behan Webster
In-Reply-To: <1411447337-22362-1-git-send-email-behanw@converseincode.com>
From: Vinícius Tinti <viniciustinti@gmail.com>
Replaced the use of a Variable Length Array In Struct (VLAIS) with a C99
compliant equivalent. This patch instead allocates the appropriate amount of
memory using a char array using the SHASH_DESC_ON_STACK macro.
The new code can be compiled with both gcc and clang.
Signed-off-by: Vinícius Tinti <viniciustinti@gmail.com>
Reviewed-by: Jan-Simon Möller <dl9pf@gmx.de>
Reviewed-by: Mark Charlebois <charlebm@gmail.com>
Signed-off-by: Behan Webster <behanw@converseincode.com>
Acked-by: Chris Mason <clm@fb.com>
Acked-by: Herbert Xu <herbert@gondor.apana.org.au>
Cc: "David S. Miller" <davem@davemloft.net>
---
fs/btrfs/hash.c | 16 +++++++---------
1 file changed, 7 insertions(+), 9 deletions(-)
diff --git a/fs/btrfs/hash.c b/fs/btrfs/hash.c
index 85889aa..4bf4d3a 100644
--- a/fs/btrfs/hash.c
+++ b/fs/btrfs/hash.c
@@ -33,18 +33,16 @@ void btrfs_hash_exit(void)
u32 btrfs_crc32c(u32 crc, const void *address, unsigned int length)
{
- struct {
- struct shash_desc shash;
- char ctx[crypto_shash_descsize(tfm)];
- } desc;
+ SHASH_DESC_ON_STACK(shash, tfm);
+ u32 *ctx = (u32 *)shash_desc_ctx(shash);
int err;
- desc.shash.tfm = tfm;
- desc.shash.flags = 0;
- *(u32 *)desc.ctx = crc;
+ shash->tfm = tfm;
+ shash->flags = 0;
+ *ctx = crc;
- err = crypto_shash_update(&desc.shash, address, length);
+ err = crypto_shash_update(shash, address, length);
BUG_ON(err);
- return *(u32 *)desc.ctx;
+ return *ctx;
}
--
1.9.1
^ permalink raw reply related
* [PATCH v4 01/12] crypto: LLVMLinux: Add macro to remove use of VLAIS in crypto code
From: behanw @ 2014-09-23 4:42 UTC (permalink / raw)
To: agk, clm, davem, dm-devel, fabf, herbert, jbacik, snitzer,
tadeusz.struk
Cc: akpm, bruce.w.allan, d.kasatkin, james.l.morris, john.griffin,
linux-btrfs, linux-crypto, linux-ima-devel, linux-ima-user,
linux-kernel, linux-raid, linux-security-module, neilb, qat-linux,
serge, thomas.lendacky, zohar, torvalds, Behan Webster,
Michał Mirosław
In-Reply-To: <1411447337-22362-1-git-send-email-behanw@converseincode.com>
From: Behan Webster <behanw@converseincode.com>
Add a macro which replaces the use of a Variable Length Array In Struct (VLAIS)
with a C99 compliant equivalent. This macro instead allocates the appropriate
amount of memory using an char array.
The new code can be compiled with both gcc and clang.
struct shash_desc contains a flexible array member member ctx declared with
CRYPTO_MINALIGN_ATTR, so sizeof(struct shash_desc) aligns the beginning
of the array declared after struct shash_desc with long long.
No trailing padding is required because it is not a struct type that can
be used in an array.
The CRYPTO_MINALIGN_ATTR is required so that desc is aligned with long long
as would be the case for a struct containing a member with
CRYPTO_MINALIGN_ATTR.
If you want to get to the ctx at the end of the shash_desc as before you can do
so using shash_desc_ctx(shash)
Signed-off-by: Behan Webster <behanw@converseincode.com>
Reviewed-by: Mark Charlebois <charlebm@gmail.com>
Acked-by: Herbert Xu <herbert@gondor.apana.org.au>
Cc: Michał Mirosław <mirqus@gmail.com>
---
include/crypto/hash.h | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/include/crypto/hash.h b/include/crypto/hash.h
index a391955..74b13ec 100644
--- a/include/crypto/hash.h
+++ b/include/crypto/hash.h
@@ -58,6 +58,11 @@ struct shash_desc {
void *__ctx[] CRYPTO_MINALIGN_ATTR;
};
+#define SHASH_DESC_ON_STACK(shash, ctx) \
+ char __##shash##_desc[sizeof(struct shash_desc) + \
+ crypto_shash_descsize(ctx)] CRYPTO_MINALIGN_ATTR; \
+ struct shash_desc *shash = (struct shash_desc *)__##shash##_desc
+
struct shash_alg {
int (*init)(struct shash_desc *desc);
int (*update)(struct shash_desc *desc, const u8 *data,
--
1.9.1
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related
* [PATCH v4 00/12] LLVMLinux: Patches to enable the kernel to be compiled with clang/LLVM
From: behanw @ 2014-09-23 4:42 UTC (permalink / raw)
To: agk, clm, davem, dm-devel, fabf, herbert, jbacik, snitzer,
tadeusz.struk
Cc: akpm, bruce.w.allan, d.kasatkin, james.l.morris, john.griffin,
linux-btrfs, linux-crypto, linux-ima-devel, linux-ima-user,
linux-kernel, linux-raid, linux-security-module, neilb, qat-linux,
serge, thomas.lendacky, zohar, torvalds, Behan Webster
From: Behan Webster <behanw@converseincode.com>
Replaced the use of a Variable Length Array In Struct (VLAIS) with a C99
compliant equivalent. These patches allocate the appropriate amount of memory
using a char array using the SHASH_DESC_ON_STACK macro.
There are places in the kernel whose maintainers have previously taken our
patches to remove VLAIS from their crypto code. Once this patch set is accepted
into mainline, I'll go back and resubmit patches to these maintainers to use
this new macro so the same approach is used consistently in all places in the
kernel.
The LLVMLinux project aims to fully build the Linux kernel using both gcc and
clang (the C front end for the LLVM compiler infrastructure project).
Behan Webster (6):
crypto: LLVMLinux: Add macro to remove use of VLAIS in crypto code
crypto: LLVMLinux: Remove VLAIS from crypto/mv_cesa.c
crypto: LLVMLinux: Remove VLAIS from crypto/n2_core.c
crypto: LLVMLinux: Remove VLAIS from crypto/omap_sham.c
crypto: LLVMLinux: Remove VLAIS from crypto/.../qat_algs.c
security, crypto: LLVMLinux: Remove VLAIS from ima_crypto.c
Jan-Simon Möller (5):
crypto: LLVMLinux: Remove VLAIS from crypto/ccp/ccp-crypto-sha.c
crypto, dm: LLVMLinux: Remove VLAIS usage from dm-crypt
crypto: LLVMLinux: Remove VLAIS usage from crypto/hmac.c
crypto: LLVMLinux: Remove VLAIS usage from libcrc32c.c
crypto: LLVMLinux: Remove VLAIS usage from crypto/testmgr.c
Vinícius Tinti (1):
btrfs: LLVMLinux: Remove VLAIS
crypto/hmac.c | 25 ++++++++---------
crypto/testmgr.c | 14 ++++------
drivers/crypto/ccp/ccp-crypto-sha.c | 13 ++++-----
drivers/crypto/mv_cesa.c | 41 ++++++++++++----------------
drivers/crypto/n2_core.c | 11 +++-----
drivers/crypto/omap-sham.c | 28 ++++++++-----------
drivers/crypto/qat/qat_common/qat_algs.c | 31 ++++++++++-----------
drivers/md/dm-crypt.c | 34 ++++++++++-------------
fs/btrfs/hash.c | 16 +++++------
include/crypto/hash.h | 5 ++++
lib/libcrc32c.c | 16 +++++------
security/integrity/ima/ima_crypto.c | 47 +++++++++++++-------------------
12 files changed, 122 insertions(+), 159 deletions(-)
--
1.9.1
^ permalink raw reply
* raid sync speed
From: lilofile @ 2014-09-23 3:34 UTC (permalink / raw)
To: stan, Linux RAID, lilofile
In-Reply-To: <36ffd6f7-bfb0-4298-a18c-f45b07cab326@aliyun.com>
when I read raid sync speed control code, I found it is very difficult for me to understand.
such as calculation of currspeed，the setting of SYNC_MARK_STEP, any suggestions will be Welcome.
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: not enough operational mirrors
From: Ian Young @ 2014-09-23 0:55 UTC (permalink / raw)
To: NeilBrown; +Cc: linux-raid
In-Reply-To: <20140923095328.0e6ed347@notabene.brown>
It's XFS. I'm running:
xfs_repair -n /dev/mapper/vg_raid10-srv
I expect it will take hours or days as this volume is 8.15 TiB.
On Mon, Sep 22, 2014 at 4:53 PM, NeilBrown <neilb@suse.de> wrote:
> On Mon, 22 Sep 2014 10:17:46 -0700 Ian Young <ian@duffrecords.com> wrote:
>
>> I forced the three good disks and the one that was behind by two
>> events to assemble:
>>
>> mdadm --assemble --force /dev/md0 /dev/sda2 /dev/sdb2 /dev/sdc2 /dev/sde2
>>
>> Then I added the other two disks and let it sync overnight:
>>
>> mdadm --add --force /dev/md0 /dev/sdd2
>> mdadm --add --force /dev/md0 /dev/sdf2
>>
>> I rebooted the system in recovery mode and the root filesystem is
>> back! However, / is read-only and my /srv partition, which is the
>> largest and has most of my data, can't mount. When I try to examine
>> the array, it says "no md superblock detected on /dev/md0." On top of
>> the software RAID, I have four logical volumes. Here is the full LVM
>> configuration:
>>
>> http://pastebin.com/gzdZq5DL
>>
>> How do I recover the superblock?
>
> What sort of filesystem is it? ext4??
>
> Try "fsck -n" and see if it finds anything.
>
> The fact that LVM found everything suggests that the array is mostly
> working. Maybe just one superblock got corrupted somehow. If 'fsck' doesn't
> get you anywhere you might need to ask on a forum dedicated to the particular
> filesystem.
>
> NeilBrown
>
>
>>
>> On Sun, Sep 21, 2014 at 10:47 PM, NeilBrown <neilb@suse.de> wrote:
>> > On Sun, 21 Sep 2014 22:32:19 -0700 Ian Young <ian@duffrecords.com> wrote:
>> >
>> >> My 6-drive software RAID 10 array failed. The individual drives
>> >> failed one at a time over the past few months but it's been an
>> >> extremely busy summer and I didn't have the free time to RMA the
>> >> drives and rebuild the array. Now I'm wishing I had acted sooner
>> >> because three of the drives are marked as removed and the array
>> >> doesn't have enough mirrors to start. I followed the recovery
>> >> instructions at raid.wiki.kernel.org and, before making things any
>> >> worse, saved the status using mdadm --examine and consulted this
>> >> mailing list. Here's the status:
>> >>
>> >> http://pastebin.com/KkV8e8Gq
>> >>
>> >> I can see that the event counts on sdd2 and sdf2 are significantly far
>> >> behind, so we can consider that data too old. sdc2 is only behind by
>> >> two events, so any data loss there should be minimal. If I can make
>> >> the array start with sd[abce]2 I think that will be enough to mount
>> >> the filesystem, back up my data, and start replacing drives. How do I
>> >> do that?
>> >
>> > Use the "--force" option with "--assemble".
>> >
>> > NeilBrown
>
^ permalink raw reply
* Re: Moving root to raid, weird raid behaviour
From: NeilBrown @ 2014-09-23 0:37 UTC (permalink / raw)
To: Wols Lists; +Cc: linux-raid
In-Reply-To: <54206A9A.8010308@youngman.org.uk>
[-- Attachment #1: Type: text/plain, Size: 2059 bytes --]
On Mon, 22 Sep 2014 19:29:46 +0100 Wols Lists <antlists@youngman.org.uk>
wrote:
> I'm not sure if this is weird raid, stupid grub2 or incompetent newbie,
> but whatever ... and the system is my development/test system so if it
> gets trashed it's no disaster, but I would like to understand what is
> going on ...
>
> Old setup
> / = sdb2
> /var = sdc2
> /home = mdX (sdb3, sdc3), mirror
>
> I added sdd and made a new
> / = mdY (sdd3, missing) mirror
> and added sdd4 to mdX
> /home = mdX (sdb3 sdc3 sdd4)
>
> Weirdo one - sdd4 mirrored and synced fine. Then I rebooted ...
> mdX = (sdb3 sdc3 missing)
> mdZ = (sdd4 missing missing)
You probably have something silly in your initrd which is making invalid
assumptions.
What does /etc/mdadm.conf in your initrd/initramfs contain?
>
> wtf?!?!? - oh and both of them share the same uuid. The obvious (but it
> shouldn't make any difference?) possibility is that sdb and sdc are
> 500Gb, partitioned identically. sdd is 1Tb, so sdd4 is twice the size of
> the other two.
>
> The other weirdo is I'm trying to migrate from grub/mbr to grub2/gpt. Of
> course that's causing me fun, but I've managed to get the system booting
> fine from mdY. Only snag is, when I list /dev, mdY isn't there! mount
> shows it as mounted on / but that's the only place I can find it!
udev should create it. 'udevadm trigger' should cause udev to create device
files for all devices. That should be done as part of the boot sequence.
If you run "udevadm trigger" does /dev/mdX get created?
>
> mdadm v3.2.6
> kernel 3.14.14-gentoo
>
> Any ideas what's going on? I've googled, but everything I find looks out
> of date or not relevant.
"out of date or not relevant"?? You must have been looking on the
Internet ?!?! :-)
NeilBrown
>
> Cheers,
> Wol
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 828 bytes --]
^ permalink raw reply
* Re: raid5 missing disks during chunk size grow
From: NeilBrown @ 2014-09-23 0:32 UTC (permalink / raw)
To: Martin Senebald; +Cc: linux-raid
In-Reply-To: <719BBDB5-4E57-49D9-AFA2-0552141D7C41@senebald.de>
[-- Attachment #1: Type: text/plain, Size: 3188 bytes --]
On Mon, 22 Sep 2014 13:55:24 +0200 Martin Senebald <martin@senebald.de> wrote:
> Am 22.09.2014 um 13:44 schrieb Mikael Abrahamsson <swmike@swm.pp.se>:
>
> > On Mon, 22 Sep 2014, Martin Senebald wrote:
> >
> >> My idea was, first bringing back the disks in the array then continue the grow (using the backup file)
> >> add / re-add disk was not working, so i assumed recreate the array with —assume-clean would bring me closer.
> >
> > Was this an idea you had that you didn't do, or did you actually execute on it?
>
> I did ..
oops. Though maybe I should say OOOPS.
Part of your array had one chunk size, part of the array had another. By
using "create" you had to choose one chunk size or the other. Obviously
neither is correct of the whole device.
You are now in a situation where you have made a mess and you need to somehow
recover your data. It is all there, but how patient and careful can you be?
By far the safest approach would be to find some other storage solution into
which you can restore all the data. Then you can try to restore the data
there and see if it look OK.
There are three sections to the data:
1/ the early part of the array which has been reshaped to the new chunk size.
2/ the part of the array which is stored in the backup file.
3/ the late part of the array which has not been reshaped yet.
Depending on which chunksize you used when you created the array (and
assuming the the newly created array has the same data-offset as the old
array), then either '1' or '3' should be available directly in the newly
created array. Calculating the exact start and size requires care. I
suggest you try to work it out and I can check your calculations.
If you copy that out, then 'create' the array with the other chunk size you
should be able to copy the other large section.
Getting the data out of the backup file might require careful reading of a
hex dump of that file to read the 'superblock' to find out exactly what is
store there and where. It shouldn't be difficult but does need care.
If you do go down this path, please feel free to ask for more specifics and
ask me to check your calculations.
For future reference "--assemble --force" is your friend.
NeilBrown
>
> >
> >> What would be the best way to tackle this problem?
> >
> > Send mdadm --examine from all 6 component drives to the list and let's take it from there.
>
> the current state of the disks:
>
> https://gist.github.com/daquan/94239614fc3b67789c9a#file-current-state
>
> the state before the —create
>
> https://gist.github.com/daquan/94239614fc3b67789c9a#file-before-create-assume-clean
>
>
> > Under no circumstances do --create on the components.
>
> that sounds not so promising anymore :-/
>
> >
> > What kernel version and mdadm version do you have?
> >
> > --
> > Mikael Abrahamsson email: swmike@swm.pp.se
> >
>
>
> BR Martin
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 828 bytes --]
^ permalink raw reply
* Re: not enough operational mirrors
From: NeilBrown @ 2014-09-22 23:53 UTC (permalink / raw)
To: Ian Young; +Cc: linux-raid
In-Reply-To: <CANs+QMxHe3SBAk=VaUQuwOSGyv9p8dJjFsxPT+hvf+c20T23=g@mail.gmail.com>
[-- Attachment #1: Type: text/plain, Size: 2490 bytes --]
On Mon, 22 Sep 2014 10:17:46 -0700 Ian Young <ian@duffrecords.com> wrote:
> I forced the three good disks and the one that was behind by two
> events to assemble:
>
> mdadm --assemble --force /dev/md0 /dev/sda2 /dev/sdb2 /dev/sdc2 /dev/sde2
>
> Then I added the other two disks and let it sync overnight:
>
> mdadm --add --force /dev/md0 /dev/sdd2
> mdadm --add --force /dev/md0 /dev/sdf2
>
> I rebooted the system in recovery mode and the root filesystem is
> back! However, / is read-only and my /srv partition, which is the
> largest and has most of my data, can't mount. When I try to examine
> the array, it says "no md superblock detected on /dev/md0." On top of
> the software RAID, I have four logical volumes. Here is the full LVM
> configuration:
>
> http://pastebin.com/gzdZq5DL
>
> How do I recover the superblock?
What sort of filesystem is it? ext4??
Try "fsck -n" and see if it finds anything.
The fact that LVM found everything suggests that the array is mostly
working. Maybe just one superblock got corrupted somehow. If 'fsck' doesn't
get you anywhere you might need to ask on a forum dedicated to the particular
filesystem.
NeilBrown
>
> On Sun, Sep 21, 2014 at 10:47 PM, NeilBrown <neilb@suse.de> wrote:
> > On Sun, 21 Sep 2014 22:32:19 -0700 Ian Young <ian@duffrecords.com> wrote:
> >
> >> My 6-drive software RAID 10 array failed. The individual drives
> >> failed one at a time over the past few months but it's been an
> >> extremely busy summer and I didn't have the free time to RMA the
> >> drives and rebuild the array. Now I'm wishing I had acted sooner
> >> because three of the drives are marked as removed and the array
> >> doesn't have enough mirrors to start. I followed the recovery
> >> instructions at raid.wiki.kernel.org and, before making things any
> >> worse, saved the status using mdadm --examine and consulted this
> >> mailing list. Here's the status:
> >>
> >> http://pastebin.com/KkV8e8Gq
> >>
> >> I can see that the event counts on sdd2 and sdf2 are significantly far
> >> behind, so we can consider that data too old. sdc2 is only behind by
> >> two events, so any data loss there should be minimal. If I can make
> >> the array start with sd[abce]2 I think that will be enough to mount
> >> the filesystem, back up my data, and start replacing drives. How do I
> >> do that?
> >
> > Use the "--force" option with "--assemble".
> >
> > NeilBrown
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 828 bytes --]
^ permalink raw reply
* Re: not enough operational mirrors
From: Ian Young @ 2014-09-22 23:09 UTC (permalink / raw)
To: NeilBrown; +Cc: linux-raid
In-Reply-To: <CANs+QMwUWZ+z0Kk-voHRLZaherOh8K8o_gCXVvw7nnXYT_goUg@mail.gmail.com>
Oops, I meant to say the error I get when trying to mount /srv is this:
root@dtla:~# mount /srv
mount: /dev/mapper/vg_raid10-srv: can't read superblock
Aren't there other copies of the superblock? I'm not sure how it
works with LVM.
On Mon, Sep 22, 2014 at 10:17 AM, Ian Young <ian@duffrecords.com> wrote:
> I forced the three good disks and the one that was behind by two events to
> assemble:
>
> mdadm --assemble --force /dev/md0 /dev/sda2 /dev/sdb2 /dev/sdc2 /dev/sde2
>
> Then I added the other two disks and let it sync overnight:
>
> mdadm --add --force /dev/md0 /dev/sdd2
> mdadm --add --force /dev/md0 /dev/sdf2
>
> I rebooted the system in recovery mode and the root filesystem is back!
> However, / is read-only and my /srv partition, which is the largest and has
> most of my data, can't mount. When I try to examine the array, it says "no
> md superblock detected on /dev/md0." On top of the software RAID, I have
> four logical volumes. Here is the full LVM configuration:
>
> http://pastebin.com/gzdZq5DL
>
> How do I recover the superblock?
>
> On Sun, Sep 21, 2014 at 10:47 PM, NeilBrown <neilb@suse.de> wrote:
>>
>> On Sun, 21 Sep 2014 22:32:19 -0700 Ian Young <ian@duffrecords.com> wrote:
>>
>> > My 6-drive software RAID 10 array failed. The individual drives
>> > failed one at a time over the past few months but it's been an
>> > extremely busy summer and I didn't have the free time to RMA the
>> > drives and rebuild the array. Now I'm wishing I had acted sooner
>> > because three of the drives are marked as removed and the array
>> > doesn't have enough mirrors to start. I followed the recovery
>> > instructions at raid.wiki.kernel.org and, before making things any
>> > worse, saved the status using mdadm --examine and consulted this
>> > mailing list. Here's the status:
>> >
>> > http://pastebin.com/KkV8e8Gq
>> >
>> > I can see that the event counts on sdd2 and sdf2 are significantly far
>> > behind, so we can consider that data too old. sdc2 is only behind by
>> > two events, so any data loss there should be minimal. If I can make
>> > the array start with sd[abce]2 I think that will be enough to mount
>> > the filesystem, back up my data, and start replacing drives. How do I
>> > do that?
>>
>> Use the "--force" option with "--assemble".
>>
>> NeilBrown
>
>
^ permalink raw reply
* Moving root to raid, weird raid behaviour
From: Wols Lists @ 2014-09-22 18:29 UTC (permalink / raw)
To: linux-raid
I'm not sure if this is weird raid, stupid grub2 or incompetent newbie,
but whatever ... and the system is my development/test system so if it
gets trashed it's no disaster, but I would like to understand what is
going on ...
Old setup
/ = sdb2
/var = sdc2
/home = mdX (sdb3, sdc3), mirror
I added sdd and made a new
/ = mdY (sdd3, missing) mirror
and added sdd4 to mdX
/home = mdX (sdb3 sdc3 sdd4)
Weirdo one - sdd4 mirrored and synced fine. Then I rebooted ...
mdX = (sdb3 sdc3 missing)
mdZ = (sdd4 missing missing)
wtf?!?!? - oh and both of them share the same uuid. The obvious (but it
shouldn't make any difference?) possibility is that sdb and sdc are
500Gb, partitioned identically. sdd is 1Tb, so sdd4 is twice the size of
the other two.
The other weirdo is I'm trying to migrate from grub/mbr to grub2/gpt. Of
course that's causing me fun, but I've managed to get the system booting
fine from mdY. Only snag is, when I list /dev, mdY isn't there! mount
shows it as mounted on / but that's the only place I can find it!
mdadm v3.2.6
kernel 3.14.14-gentoo
Any ideas what's going on? I've googled, but everything I find looks out
of date or not relevant.
Cheers,
Wol
^ permalink raw reply
* Re: not enough operational mirrors
From: Ian Young @ 2014-09-22 17:17 UTC (permalink / raw)
To: NeilBrown; +Cc: linux-raid
In-Reply-To: <20140922154717.6cd3cab2@notabene.brown>
I forced the three good disks and the one that was behind by two
events to assemble:
mdadm --assemble --force /dev/md0 /dev/sda2 /dev/sdb2 /dev/sdc2 /dev/sde2
Then I added the other two disks and let it sync overnight:
mdadm --add --force /dev/md0 /dev/sdd2
mdadm --add --force /dev/md0 /dev/sdf2
I rebooted the system in recovery mode and the root filesystem is
back! However, / is read-only and my /srv partition, which is the
largest and has most of my data, can't mount. When I try to examine
the array, it says "no md superblock detected on /dev/md0." On top of
the software RAID, I have four logical volumes. Here is the full LVM
configuration:
http://pastebin.com/gzdZq5DL
How do I recover the superblock?
On Sun, Sep 21, 2014 at 10:47 PM, NeilBrown <neilb@suse.de> wrote:
> On Sun, 21 Sep 2014 22:32:19 -0700 Ian Young <ian@duffrecords.com> wrote:
>
>> My 6-drive software RAID 10 array failed. The individual drives
>> failed one at a time over the past few months but it's been an
>> extremely busy summer and I didn't have the free time to RMA the
>> drives and rebuild the array. Now I'm wishing I had acted sooner
>> because three of the drives are marked as removed and the array
>> doesn't have enough mirrors to start. I followed the recovery
>> instructions at raid.wiki.kernel.org and, before making things any
>> worse, saved the status using mdadm --examine and consulted this
>> mailing list. Here's the status:
>>
>> http://pastebin.com/KkV8e8Gq
>>
>> I can see that the event counts on sdd2 and sdf2 are significantly far
>> behind, so we can consider that data too old. sdc2 is only behind by
>> two events, so any data loss there should be minimal. If I can make
>> the array start with sd[abce]2 I think that will be enough to mount
>> the filesystem, back up my data, and start replacing drives. How do I
>> do that?
>
> Use the "--force" option with "--assemble".
>
> NeilBrown
^ permalink raw reply
* Re: raid5 missing disks during chunk size grow
From: Mikael Abrahamsson @ 2014-09-22 12:59 UTC (permalink / raw)
To: Martin Senebald; +Cc: linux-raid
In-Reply-To: <2146127B-50ED-46E6-AE63-BAC9DA35DBDC@senebald.de>
On Mon, 22 Sep 2014, Martin Senebald wrote:
> I just forgot :)
>
> Debian 3.2.60-1+deb7u3 x86_64
You still didn't provide mdadm version.
I would suggest you look into getting a more recent kernel version and
mdadm version, because I would guess you have a non-recent mdadm with your
debian dist.
--
Mikael Abrahamsson email: swmike@swm.pp.se
^ permalink raw reply
* Re: raid5 missing disks during chunk size grow
From: Martin Senebald @ 2014-09-22 12:50 UTC (permalink / raw)
To: Mikael Abrahamsson; +Cc: linux-raid
In-Reply-To: <alpine.DEB.2.02.1409221400070.14735@uplift.swm.pp.se>
[-- Attachment #1: Type: text/plain, Size: 1803 bytes --]
Am 22.09.2014 um 14:02 schrieb Mikael Abrahamsson <swmike@swm.pp.se>:
> On Mon, 22 Sep 2014, Martin Senebald wrote:
>
>>> Was this an idea you had that you didn't do, or did you actually execute on it?
>>
>> I did ..
>
> What made you think that was a good idea?
Good question, i can’t point to a specific source. I guess this was more the "out of options“ decision that time. (at least it looked that way)
> I have been re-writing text on the linux-raid wiki to discourage people from doing that. Wherefrom did you get the information?
And obviously i didn’t find that either. I just quickly looked over it and didn’t find it.
>
>> the state before the —create
>>
>> https://gist.github.com/daquan/94239614fc3b67789c9a#file-before-create-assume-clean
>
> I have no idea how to get you from there.
That was maybe what lead me to believe —create might help. I understand that recreating the array was maybe the worst thing (destroying the reshape state information of the array)
but generally speaking i think the data itself is not yet lost. The positions of the reshape is available for each disk(what i see from the examine). The Data on the md2 didn’t change while reshape was ongoing (is a lvm pvdisk and lvm was not active). But i have no clue how to bring the raid to the state of grow when the raid failed. I don’t understand to much how the grow process for chunk size works.
>
>>> Under no circumstances do --create on the components.
>>
>> that sounds not so promising anymore :-/
>
> Indeed.
>>>
>>> What kernel version and mdadm version do you have?
>
> Why didn't you answer this crucial question?
I just forgot :)
Debian 3.2.60-1+deb7u3 x86_64
>
> --
> Mikael Abrahamsson email: swmike@swm.pp.se
>
[-- Attachment #2: Message signed with OpenPGP using GPGMail --]
[-- Type: application/pgp-signature, Size: 842 bytes --]
^ permalink raw reply
* Re: raid5 missing disks during chunk size grow
From: Mikael Abrahamsson @ 2014-09-22 12:02 UTC (permalink / raw)
To: Martin Senebald; +Cc: linux-raid
In-Reply-To: <719BBDB5-4E57-49D9-AFA2-0552141D7C41@senebald.de>
[-- Attachment #1: Type: TEXT/PLAIN, Size: 743 bytes --]
On Mon, 22 Sep 2014, Martin Senebald wrote:
>> Was this an idea you had that you didn't do, or did you actually execute on it?
>
> I did ..
What made you think that was a good idea? I have been re-writing text on
the linux-raid wiki to discourage people from doing that. Wherefrom did
you get the information?
> the state before the —create
>
> https://gist.github.com/daquan/94239614fc3b67789c9a#file-before-create-assume-clean
I have no idea how to get you from there.
>> Under no circumstances do --create on the components.
>
> that sounds not so promising anymore :-/
Indeed.
>>
>> What kernel version and mdadm version do you have?
Why didn't you answer this crucial question?
--
Mikael Abrahamsson email: swmike@swm.pp.se
^ permalink raw reply
* Re: raid5 missing disks during chunk size grow
From: Martin Senebald @ 2014-09-22 11:55 UTC (permalink / raw)
To: linux-raid
In-Reply-To: <alpine.DEB.2.02.1409221342090.14735@uplift.swm.pp.se>
Am 22.09.2014 um 13:44 schrieb Mikael Abrahamsson <swmike@swm.pp.se>:
> On Mon, 22 Sep 2014, Martin Senebald wrote:
>
>> My idea was, first bringing back the disks in the array then continue the grow (using the backup file)
>> add / re-add disk was not working, so i assumed recreate the array with —assume-clean would bring me closer.
>
> Was this an idea you had that you didn't do, or did you actually execute on it?
I did ..
>
>> What would be the best way to tackle this problem?
>
> Send mdadm --examine from all 6 component drives to the list and let's take it from there.
the current state of the disks:
https://gist.github.com/daquan/94239614fc3b67789c9a#file-current-state
the state before the —create
https://gist.github.com/daquan/94239614fc3b67789c9a#file-before-create-assume-clean
> Under no circumstances do --create on the components.
that sounds not so promising anymore :-/
>
> What kernel version and mdadm version do you have?
>
> --
> Mikael Abrahamsson email: swmike@swm.pp.se
>
BR Martin
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: raid5 missing disks during chunk size grow
From: Mikael Abrahamsson @ 2014-09-22 11:44 UTC (permalink / raw)
To: Martin Senebald; +Cc: linux-raid
In-Reply-To: <374D34B3-E7F5-48AD-9A22-23C76AA0A60F@senebald.de>
[-- Attachment #1: Type: TEXT/PLAIN, Size: 648 bytes --]
On Mon, 22 Sep 2014, Martin Senebald wrote:
> My idea was, first bringing back the disks in the array then continue the grow (using the backup file)
> add / re-add disk was not working, so i assumed recreate the array with —assume-clean would bring me closer.
Was this an idea you had that you didn't do, or did you actually execute
on it?
> What would be the best way to tackle this problem?
Send mdadm --examine from all 6 component drives to the list and let's
take it from there. Under no circumstances do --create on the components.
What kernel version and mdadm version do you have?
--
Mikael Abrahamsson email: swmike@swm.pp.se
^ permalink raw reply
* raid5 missing disks during chunk size grow
From: Martin Senebald @ 2014-09-22 11:21 UTC (permalink / raw)
To: linux-raid
[-- Attachment #1: Type: text/plain, Size: 2641 bytes --]
Hi,
i have a tricky problem. I have a 6disk raid5 setup and did a mdadm grow chunk size.
Apparently 3 disk went missing (controller/kernel crash) and the raid failed. Disks are all ok. Backupfile for grow is there and raid config available including status(examine) of all disks.
Problem = 3 disks where in a state like
/dev/sdc1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x4
Array UUID : 84bea3d7:7b697819:b8c1aa5a:a615e3b4
Name : chenbro.han.daquan.eu:2 (local to host chenbro.han.daquan.eu)
Creation Time : Sat Sep 20 12:28:57 2014
Raid Level : raid5
Raid Devices : 6
Avail Dev Size : 5856270336 (2792.49 GiB 2998.41 GB)
Array Size : 14640675840 (13962.44 GiB 14992.05 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
State : active
Device UUID : 67cc689a:2cdf3d4d:0c7d03d4:c12c7475
Reshape pos'n : 557547520 (531.72 GiB 570.93 GB)
New Chunksize : 512K
Update Time : Mon Sep 22 09:39:35 2014
Checksum : 25acf25d - correct
Events : 109099
Layout : left-symmetric
Chunk Size : 64K
Device Role : Active device 3
Array State : ...AAA ('A' == active, '.' == missing)
and 3 in
/dev/sdf1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x4
Array UUID : 84bea3d7:7b697819:b8c1aa5a:a615e3b4
Name : chenbro.han.daquan.eu:2 (local to host chenbro.han.daquan.eu)
Creation Time : Sat Sep 20 12:28:57 2014
Raid Level : raid5
Raid Devices : 6
Avail Dev Size : 5856270336 (2792.49 GiB 2998.41 GB)
Array Size : 14640675840 (13962.44 GiB 14992.05 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
State : clean
Device UUID : f74a1585:93fb850b:64667d57:42c34e52
Reshape pos'n : 557547520 (531.72 GiB 570.93 GB)
New Chunksize : 512K
Update Time : Mon Sep 22 09:02:59 2014
Checksum : 2ef3049b - correct
Events : 109096
Layout : left-symmetric
Chunk Size : 64K
Device Role : Active device 0
Array State : AAAAAA ('A' == active, '.' == missing)
My idea was, first bringing back the disks in the array then continue the grow (using the backup file)
add / re-add disk was not working, so i assumed recreate the array with —assume-clean would bring me closer.
Question now ist, how to i get the array back to the grow process at the state of the failure?
I didnt find anything regarding this. (if even possible)
Someone any idea?
What would be the best way to tackle this problem?
Thanks
BR Martin
[-- Attachment #2: Message signed with OpenPGP using GPGMail --]
[-- Type: application/pgp-signature, Size: 842 bytes --]
^ permalink raw reply
* Re: Please advise, strange "not enough to start the array while not clean"
From: NeilBrown @ 2014-09-22 10:17 UTC (permalink / raw)
To: Patrik Horník; +Cc: linux-raid
In-Reply-To: <CAAOsTSmC7YNzguxF+e978wLmUR_4J1Xb+EfdqbpCj8FCquUrAg@mail.gmail.com>
[-- Attachment #1: Type: text/plain, Size: 8858 bytes --]
On Mon, 22 Sep 2014 09:20:35 +0200 Patrik Horník <patrik@dsl.sk> wrote:
> Well I browsed through the sources of latest mdadm version at night
> instead of sleeping :) and was searching how it got clean flag set to
> 0. And was not sure exactly about that line there and from which
> device it gets clean state. So the bug was that it can get it from not
> current device? It makes sense because first device identified by
> mdadm is old md101 device.
Correct.
>
> So will it work then if I use 3.3 and somehow dont give it md101
> device? By stopping it before -A call or by manually specifying other
> drives? Or you really recommend to build latest version of mdadm?
mdadm -S /dev/mdXX list-of-devices-that-are-working
should start the array for you using 3.3.
Then
mdadm /dev/mdXX --re-add /dev/md101
will re-add the device and hopefully do a quick bitmap-based rebuild.
>
> What is expected behaviour with 3.3.1+? Can it be started with all
> devices and it should automatically start to recover md101? If so what
> is best way, to start it without md101 and then use -re-add to add it,
> start it without md101 and use -add or start it also with md101?
--add should have the same effect as --re-add.
I think mdadm 3.3.1 will just assemble the array without md101 and you then
have to --re-add that yourself. To get it re-added automatically you needed
to have "policy action=re-add" or similar in mdadm.conf, and then use
mdadm -I /dev/devicename
on each device. That should put the whole array together and re-add anything
that needs it.
I think.
NeilBrown
>
> Thank you very much.
>
> 2014-09-22 8:56 GMT+02:00 NeilBrown <neilb@suse.de>:
> > On Mon, 22 Sep 2014 08:34:21 +0200 Patrik Horník <patrik@dsl.sk> wrote:
> >
> >> - Well what is exact meaning of --no-degraded then? Because I am using
> >> it also on RAID6 arrays that are missing one drive and mdadm starts
> >> them. I thought until today that it is against assembling for example
> >> RAID6 array with missing more than two drives or to be more precise
> >> array with number of drives it used last time. (I did not look at the
> >> code what does it exactly. It is mdadm 3.3 on Debian.)
> >
> > Sorry, I confused myself.
> > "--no-degraded" means "Only start the array if all expected devices are
> > present".
> > So if the array "knows" that one device is missing, it will start if all
> > other devices are present. But if it "thinks" that all devices are working,
> > then it will only start if all the devices ar there.
> >
> >>
> >> - Well array was shutdown cleanly manually by mdadm -S. Cant the not
> >> clean classification be result of md101 device between find devices or
> >> result of first two assemble tries?
> >
> > If the state still says "Clean" (which it does, thanks), the mdadm should
> > treat it as 'clean'.
> >
> > I think you are probably hitting the bug fixed by
> >
> > http://git.neil.brown.name/?p=mdadm.git;a=commitdiff;h=56bbc588f7f0f3bdd3ec23f02109b427c1d3b8f1
> >
> > which is in 3.3.1.
> >
> > So a new version of mdadm should fix it.
> >
> > NeilBrown
> >
> >
> >
> >>
> >> - Anyway as I mentioned superblock on all five devices has clean state. Example:
> >> /dev/sdk1:
> >> Magic : XXXXXXX
> >> Version : 1.2
> >> Feature Map : 0x1
> >> Array UUID : XXXXXXXXXXXXXXXXXXXXX
> >> Name :
> >> Creation Time : Thu Aug XXXXXXXX
> >> Raid Level : raid6
> >> Raid Devices : 6
> >>
> >> Avail Dev Size : 5860268943 (2794.39 GiB 3000.46 GB)
> >> Array Size : 11720536064 (11177.57 GiB 12001.83 GB)
> >> Used Dev Size : 5860268032 (2794.39 GiB 3000.46 GB)
> >> Data Offset : 262144 sectors
> >> Super Offset : 8 sectors
> >> Unused Space : before=262056 sectors, after=911 sectors
> >> State : clean
> >> Device UUID : YYYYYYYYYYYYYYYYYYYY
> >>
> >> Internal Bitmap : 8 sectors from superblock
> >> Update Time : Mon Sep 22 02:23:45 2014
> >> Bad Block Log : 512 entries available at offset 72 sectors
> >> Checksum : ZZZZZZZZ - correct
> >> Events : EEEEEE
> >>
> >> Layout : left-symmetric
> >> Chunk Size : 512K
> >>
> >> Device Role : Active device 4
> >> Array State : AAAAA. ('A' == active, '.' == missing, 'R' == replacing)
> >>
> >> - md101 has Events count lower by 16 than others devices.
> >>
> >> - Please I need little more assurance what is exact state of array and
> >> explain why it is behaving as it is behaving, so I can be sure what
> >> steps are needed and what happens. The data on array is important.
> >> Patrik Horník
> >> šéfredaktor www.DSL.sk
> >> Tel.: +421 905 385 666
> >> Email: patrik@dsl.sk
> >>
> >>
> >> 2014-09-22 5:19 GMT+02:00 NeilBrown <neilb@suse.de>:
> >> > On Mon, 22 Sep 2014 04:11:20 +0200 Patrik Horník <patrik@dsl.sk> wrote:
> >> >
> >> >> Hello Neil,
> >> >>
> >> >> I've got this situation unfamiliar to me on RAID6 array md1 with important data.
> >> >>
> >> >> - It is RAID6 with 6 devices, 5 are partitions and 1 is another RAID0
> >> >> array md101 from two smaller drives. One of the smaller drives froze,
> >> >> so md101 got kicked out from md1 and marked as faulty in md1. After
> >> >> while I've stopped md1 without removing md101 from it first. Then I
> >> >> rebooted and assembled md101.
> >> >>
> >> >> - First I tried mdadm -A --no-degraded -u UUID /dev/md1 but got
> >> >> "mdadm: /dev/md1 assembled from 5 drives (out of 6), but not started."
> >> >> so I stopped the md1.
> >> >>
> >> >> - Second time I started it with -v and got:
> >> >>
> >> >> mdadm: /dev/md101 is identified as a member of /dev/md1, slot 5.
> >> >> mdadm: /dev/sdk1 is identified as a member of /dev/md1, slot 4.
> >> >> mdadm: /dev/sdi1 is identified as a member of /dev/md1, slot 1.
> >> >> mdadm: /dev/sdh1 is identified as a member of /dev/md1, slot 2.
> >> >> mdadm: /dev/sdg1 is identified as a member of /dev/md1, slot 0.
> >> >> mdadm: /dev/sde1 is identified as a member of /dev/md1, slot 3.
> >> >> mdadm: added /dev/sdi1 to /dev/md1 as 1
> >> >> mdadm: added /dev/sdh1 to /dev/md1 as 2
> >> >> mdadm: added /dev/sde1 to /dev/md1 as 3
> >> >> mdadm: added /dev/sdk1 to /dev/md1 as 4
> >> >> mdadm: added /dev/md101 to /dev/md1 as 5 (possibly out of date)
> >> >> mdadm: added /dev/sdg1 to /dev/md1 as 0
> >> >> mdadm: /dev/md1 assembled from 5 drives (out of 6), but not started.
> >> >>
> >> >> - On third time I tried without --nodegraded with mdadm -A -v -u UUID
> >> >> /dev/md1. This is what I've got:
> >> >>
> >> >> mdadm: /dev/md101 is identified as a member of /dev/md1, slot 5.
> >> >> mdadm: /dev/sdk1 is identified as a member of /dev/md1, slot 4.
> >> >> mdadm: /dev/sdi1 is identified as a member of /dev/md1, slot 1.
> >> >> mdadm: /dev/sdh1 is identified as a member of /dev/md1, slot 2.
> >> >> mdadm: /dev/sdg1 is identified as a member of /dev/md1, slot 0.
> >> >> mdadm: /dev/sde1 is identified as a member of /dev/md1, slot 3.
> >> >> mdadm: added /dev/sdi1 to /dev/md1 as 1
> >> >> mdadm: added /dev/sdh1 to /dev/md1 as 2
> >> >> mdadm: added /dev/sde1 to /dev/md1 as 3
> >> >> mdadm: added /dev/sdk1 to /dev/md1 as 4
> >> >> mdadm: added /dev/md101 to /dev/md1 as 5 (possibly out of date)
> >> >> mdadm: added /dev/sdg1 to /dev/md1 as 0
> >> >> mdadm: /dev/md1 assembled from 5 drives - not enough to start the
> >> >> array while not clean - consider --force.
> >> >>
> >> >> Array md1 has bitmap. All drive devices have all same Events, their
> >> >> state is clean and Device Role is Active device. md101 has active
> >> >> state and lower Events.
> >> >>
> >> >> Is this expected behavior? My theory is that it is caused by md101 and
> >> >> I should start array md1 without it (by for example stopping md101)
> >> >> and then re-add it. Is that a case or is it something else?
> >> >>
> >> >> Thanks.
> >> >>
> >> >> Best regards,
> >> >>
> >> >> Patrik
> >> >
> >> >
> >> > The array is clearly degraded as one of the devices failed and hasn't been
> >> > recovered yet, so using --nodegraded is counter productive, as you
> >> > discovered.
> >> >
> >> > It appears that the array is also marked as 'dirty'. That suggests that it
> >> > wasn't shut down cleanly.
> >> > What does "mdadm --examine" of some device show?
> >> >
> >> > You probably need to re-assemble the array with --force like it suggests,
> >> > then add the failed device and let it recover.
> >> >
> >> > NeilBrown
> >> >
> >> --
> >> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> >> the body of a message to majordomo@vger.kernel.org
> >> More majordomo info at http://vger.kernel.org/majordomo-info.html
> >
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 828 bytes --]
^ permalink raw reply
* Re: Please advise, strange "not enough to start the array while not clean"
From: Patrik Horník @ 2014-09-22 7:20 UTC (permalink / raw)
To: NeilBrown; +Cc: linux-raid
In-Reply-To: <20140922165602.43330805@notabene.brown>
Well I browsed through the sources of latest mdadm version at night
instead of sleeping :) and was searching how it got clean flag set to
0. And was not sure exactly about that line there and from which
device it gets clean state. So the bug was that it can get it from not
current device? It makes sense because first device identified by
mdadm is old md101 device.
So will it work then if I use 3.3 and somehow dont give it md101
device? By stopping it before -A call or by manually specifying other
drives? Or you really recommend to build latest version of mdadm?
What is expected behaviour with 3.3.1+? Can it be started with all
devices and it should automatically start to recover md101? If so what
is best way, to start it without md101 and then use -re-add to add it,
start it without md101 and use -add or start it also with md101?
Thank you very much.
2014-09-22 8:56 GMT+02:00 NeilBrown <neilb@suse.de>:
> On Mon, 22 Sep 2014 08:34:21 +0200 Patrik Horník <patrik@dsl.sk> wrote:
>
>> - Well what is exact meaning of --no-degraded then? Because I am using
>> it also on RAID6 arrays that are missing one drive and mdadm starts
>> them. I thought until today that it is against assembling for example
>> RAID6 array with missing more than two drives or to be more precise
>> array with number of drives it used last time. (I did not look at the
>> code what does it exactly. It is mdadm 3.3 on Debian.)
>
> Sorry, I confused myself.
> "--no-degraded" means "Only start the array if all expected devices are
> present".
> So if the array "knows" that one device is missing, it will start if all
> other devices are present. But if it "thinks" that all devices are working,
> then it will only start if all the devices ar there.
>
>>
>> - Well array was shutdown cleanly manually by mdadm -S. Cant the not
>> clean classification be result of md101 device between find devices or
>> result of first two assemble tries?
>
> If the state still says "Clean" (which it does, thanks), the mdadm should
> treat it as 'clean'.
>
> I think you are probably hitting the bug fixed by
>
> http://git.neil.brown.name/?p=mdadm.git;a=commitdiff;h=56bbc588f7f0f3bdd3ec23f02109b427c1d3b8f1
>
> which is in 3.3.1.
>
> So a new version of mdadm should fix it.
>
> NeilBrown
>
>
>
>>
>> - Anyway as I mentioned superblock on all five devices has clean state. Example:
>> /dev/sdk1:
>> Magic : XXXXXXX
>> Version : 1.2
>> Feature Map : 0x1
>> Array UUID : XXXXXXXXXXXXXXXXXXXXX
>> Name :
>> Creation Time : Thu Aug XXXXXXXX
>> Raid Level : raid6
>> Raid Devices : 6
>>
>> Avail Dev Size : 5860268943 (2794.39 GiB 3000.46 GB)
>> Array Size : 11720536064 (11177.57 GiB 12001.83 GB)
>> Used Dev Size : 5860268032 (2794.39 GiB 3000.46 GB)
>> Data Offset : 262144 sectors
>> Super Offset : 8 sectors
>> Unused Space : before=262056 sectors, after=911 sectors
>> State : clean
>> Device UUID : YYYYYYYYYYYYYYYYYYYY
>>
>> Internal Bitmap : 8 sectors from superblock
>> Update Time : Mon Sep 22 02:23:45 2014
>> Bad Block Log : 512 entries available at offset 72 sectors
>> Checksum : ZZZZZZZZ - correct
>> Events : EEEEEE
>>
>> Layout : left-symmetric
>> Chunk Size : 512K
>>
>> Device Role : Active device 4
>> Array State : AAAAA. ('A' == active, '.' == missing, 'R' == replacing)
>>
>> - md101 has Events count lower by 16 than others devices.
>>
>> - Please I need little more assurance what is exact state of array and
>> explain why it is behaving as it is behaving, so I can be sure what
>> steps are needed and what happens. The data on array is important.
>> Patrik Horník
>> šéfredaktor www.DSL.sk
>> Tel.: +421 905 385 666
>> Email: patrik@dsl.sk
>>
>>
>> 2014-09-22 5:19 GMT+02:00 NeilBrown <neilb@suse.de>:
>> > On Mon, 22 Sep 2014 04:11:20 +0200 Patrik Horník <patrik@dsl.sk> wrote:
>> >
>> >> Hello Neil,
>> >>
>> >> I've got this situation unfamiliar to me on RAID6 array md1 with important data.
>> >>
>> >> - It is RAID6 with 6 devices, 5 are partitions and 1 is another RAID0
>> >> array md101 from two smaller drives. One of the smaller drives froze,
>> >> so md101 got kicked out from md1 and marked as faulty in md1. After
>> >> while I've stopped md1 without removing md101 from it first. Then I
>> >> rebooted and assembled md101.
>> >>
>> >> - First I tried mdadm -A --no-degraded -u UUID /dev/md1 but got
>> >> "mdadm: /dev/md1 assembled from 5 drives (out of 6), but not started."
>> >> so I stopped the md1.
>> >>
>> >> - Second time I started it with -v and got:
>> >>
>> >> mdadm: /dev/md101 is identified as a member of /dev/md1, slot 5.
>> >> mdadm: /dev/sdk1 is identified as a member of /dev/md1, slot 4.
>> >> mdadm: /dev/sdi1 is identified as a member of /dev/md1, slot 1.
>> >> mdadm: /dev/sdh1 is identified as a member of /dev/md1, slot 2.
>> >> mdadm: /dev/sdg1 is identified as a member of /dev/md1, slot 0.
>> >> mdadm: /dev/sde1 is identified as a member of /dev/md1, slot 3.
>> >> mdadm: added /dev/sdi1 to /dev/md1 as 1
>> >> mdadm: added /dev/sdh1 to /dev/md1 as 2
>> >> mdadm: added /dev/sde1 to /dev/md1 as 3
>> >> mdadm: added /dev/sdk1 to /dev/md1 as 4
>> >> mdadm: added /dev/md101 to /dev/md1 as 5 (possibly out of date)
>> >> mdadm: added /dev/sdg1 to /dev/md1 as 0
>> >> mdadm: /dev/md1 assembled from 5 drives (out of 6), but not started.
>> >>
>> >> - On third time I tried without --nodegraded with mdadm -A -v -u UUID
>> >> /dev/md1. This is what I've got:
>> >>
>> >> mdadm: /dev/md101 is identified as a member of /dev/md1, slot 5.
>> >> mdadm: /dev/sdk1 is identified as a member of /dev/md1, slot 4.
>> >> mdadm: /dev/sdi1 is identified as a member of /dev/md1, slot 1.
>> >> mdadm: /dev/sdh1 is identified as a member of /dev/md1, slot 2.
>> >> mdadm: /dev/sdg1 is identified as a member of /dev/md1, slot 0.
>> >> mdadm: /dev/sde1 is identified as a member of /dev/md1, slot 3.
>> >> mdadm: added /dev/sdi1 to /dev/md1 as 1
>> >> mdadm: added /dev/sdh1 to /dev/md1 as 2
>> >> mdadm: added /dev/sde1 to /dev/md1 as 3
>> >> mdadm: added /dev/sdk1 to /dev/md1 as 4
>> >> mdadm: added /dev/md101 to /dev/md1 as 5 (possibly out of date)
>> >> mdadm: added /dev/sdg1 to /dev/md1 as 0
>> >> mdadm: /dev/md1 assembled from 5 drives - not enough to start the
>> >> array while not clean - consider --force.
>> >>
>> >> Array md1 has bitmap. All drive devices have all same Events, their
>> >> state is clean and Device Role is Active device. md101 has active
>> >> state and lower Events.
>> >>
>> >> Is this expected behavior? My theory is that it is caused by md101 and
>> >> I should start array md1 without it (by for example stopping md101)
>> >> and then re-add it. Is that a case or is it something else?
>> >>
>> >> Thanks.
>> >>
>> >> Best regards,
>> >>
>> >> Patrik
>> >
>> >
>> > The array is clearly degraded as one of the devices failed and hasn't been
>> > recovered yet, so using --nodegraded is counter productive, as you
>> > discovered.
>> >
>> > It appears that the array is also marked as 'dirty'. That suggests that it
>> > wasn't shut down cleanly.
>> > What does "mdadm --examine" of some device show?
>> >
>> > You probably need to re-assemble the array with --force like it suggests,
>> > then add the failed device and let it recover.
>> >
>> > NeilBrown
>> >
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: Please advise, strange "not enough to start the array while not clean"
From: NeilBrown @ 2014-09-22 6:56 UTC (permalink / raw)
To: Patrik Horník; +Cc: linux-raid
In-Reply-To: <CAAOsTSncnYC=hTMtQ8O-dOAdkqueqP3=kJJi6jfoAb1CwQtvHA@mail.gmail.com>
[-- Attachment #1: Type: text/plain, Size: 6640 bytes --]
On Mon, 22 Sep 2014 08:34:21 +0200 Patrik Horník <patrik@dsl.sk> wrote:
> - Well what is exact meaning of --no-degraded then? Because I am using
> it also on RAID6 arrays that are missing one drive and mdadm starts
> them. I thought until today that it is against assembling for example
> RAID6 array with missing more than two drives or to be more precise
> array with number of drives it used last time. (I did not look at the
> code what does it exactly. It is mdadm 3.3 on Debian.)
Sorry, I confused myself.
"--no-degraded" means "Only start the array if all expected devices are
present".
So if the array "knows" that one device is missing, it will start if all
other devices are present. But if it "thinks" that all devices are working,
then it will only start if all the devices ar there.
>
> - Well array was shutdown cleanly manually by mdadm -S. Cant the not
> clean classification be result of md101 device between find devices or
> result of first two assemble tries?
If the state still says "Clean" (which it does, thanks), the mdadm should
treat it as 'clean'.
I think you are probably hitting the bug fixed by
http://git.neil.brown.name/?p=mdadm.git;a=commitdiff;h=56bbc588f7f0f3bdd3ec23f02109b427c1d3b8f1
which is in 3.3.1.
So a new version of mdadm should fix it.
NeilBrown
>
> - Anyway as I mentioned superblock on all five devices has clean state. Example:
> /dev/sdk1:
> Magic : XXXXXXX
> Version : 1.2
> Feature Map : 0x1
> Array UUID : XXXXXXXXXXXXXXXXXXXXX
> Name :
> Creation Time : Thu Aug XXXXXXXX
> Raid Level : raid6
> Raid Devices : 6
>
> Avail Dev Size : 5860268943 (2794.39 GiB 3000.46 GB)
> Array Size : 11720536064 (11177.57 GiB 12001.83 GB)
> Used Dev Size : 5860268032 (2794.39 GiB 3000.46 GB)
> Data Offset : 262144 sectors
> Super Offset : 8 sectors
> Unused Space : before=262056 sectors, after=911 sectors
> State : clean
> Device UUID : YYYYYYYYYYYYYYYYYYYY
>
> Internal Bitmap : 8 sectors from superblock
> Update Time : Mon Sep 22 02:23:45 2014
> Bad Block Log : 512 entries available at offset 72 sectors
> Checksum : ZZZZZZZZ - correct
> Events : EEEEEE
>
> Layout : left-symmetric
> Chunk Size : 512K
>
> Device Role : Active device 4
> Array State : AAAAA. ('A' == active, '.' == missing, 'R' == replacing)
>
> - md101 has Events count lower by 16 than others devices.
>
> - Please I need little more assurance what is exact state of array and
> explain why it is behaving as it is behaving, so I can be sure what
> steps are needed and what happens. The data on array is important.
> Patrik Horník
> šéfredaktor www.DSL.sk
> Tel.: +421 905 385 666
> Email: patrik@dsl.sk
>
>
> 2014-09-22 5:19 GMT+02:00 NeilBrown <neilb@suse.de>:
> > On Mon, 22 Sep 2014 04:11:20 +0200 Patrik Horník <patrik@dsl.sk> wrote:
> >
> >> Hello Neil,
> >>
> >> I've got this situation unfamiliar to me on RAID6 array md1 with important data.
> >>
> >> - It is RAID6 with 6 devices, 5 are partitions and 1 is another RAID0
> >> array md101 from two smaller drives. One of the smaller drives froze,
> >> so md101 got kicked out from md1 and marked as faulty in md1. After
> >> while I've stopped md1 without removing md101 from it first. Then I
> >> rebooted and assembled md101.
> >>
> >> - First I tried mdadm -A --no-degraded -u UUID /dev/md1 but got
> >> "mdadm: /dev/md1 assembled from 5 drives (out of 6), but not started."
> >> so I stopped the md1.
> >>
> >> - Second time I started it with -v and got:
> >>
> >> mdadm: /dev/md101 is identified as a member of /dev/md1, slot 5.
> >> mdadm: /dev/sdk1 is identified as a member of /dev/md1, slot 4.
> >> mdadm: /dev/sdi1 is identified as a member of /dev/md1, slot 1.
> >> mdadm: /dev/sdh1 is identified as a member of /dev/md1, slot 2.
> >> mdadm: /dev/sdg1 is identified as a member of /dev/md1, slot 0.
> >> mdadm: /dev/sde1 is identified as a member of /dev/md1, slot 3.
> >> mdadm: added /dev/sdi1 to /dev/md1 as 1
> >> mdadm: added /dev/sdh1 to /dev/md1 as 2
> >> mdadm: added /dev/sde1 to /dev/md1 as 3
> >> mdadm: added /dev/sdk1 to /dev/md1 as 4
> >> mdadm: added /dev/md101 to /dev/md1 as 5 (possibly out of date)
> >> mdadm: added /dev/sdg1 to /dev/md1 as 0
> >> mdadm: /dev/md1 assembled from 5 drives (out of 6), but not started.
> >>
> >> - On third time I tried without --nodegraded with mdadm -A -v -u UUID
> >> /dev/md1. This is what I've got:
> >>
> >> mdadm: /dev/md101 is identified as a member of /dev/md1, slot 5.
> >> mdadm: /dev/sdk1 is identified as a member of /dev/md1, slot 4.
> >> mdadm: /dev/sdi1 is identified as a member of /dev/md1, slot 1.
> >> mdadm: /dev/sdh1 is identified as a member of /dev/md1, slot 2.
> >> mdadm: /dev/sdg1 is identified as a member of /dev/md1, slot 0.
> >> mdadm: /dev/sde1 is identified as a member of /dev/md1, slot 3.
> >> mdadm: added /dev/sdi1 to /dev/md1 as 1
> >> mdadm: added /dev/sdh1 to /dev/md1 as 2
> >> mdadm: added /dev/sde1 to /dev/md1 as 3
> >> mdadm: added /dev/sdk1 to /dev/md1 as 4
> >> mdadm: added /dev/md101 to /dev/md1 as 5 (possibly out of date)
> >> mdadm: added /dev/sdg1 to /dev/md1 as 0
> >> mdadm: /dev/md1 assembled from 5 drives - not enough to start the
> >> array while not clean - consider --force.
> >>
> >> Array md1 has bitmap. All drive devices have all same Events, their
> >> state is clean and Device Role is Active device. md101 has active
> >> state and lower Events.
> >>
> >> Is this expected behavior? My theory is that it is caused by md101 and
> >> I should start array md1 without it (by for example stopping md101)
> >> and then re-add it. Is that a case or is it something else?
> >>
> >> Thanks.
> >>
> >> Best regards,
> >>
> >> Patrik
> >
> >
> > The array is clearly degraded as one of the devices failed and hasn't been
> > recovered yet, so using --nodegraded is counter productive, as you
> > discovered.
> >
> > It appears that the array is also marked as 'dirty'. That suggests that it
> > wasn't shut down cleanly.
> > What does "mdadm --examine" of some device show?
> >
> > You probably need to re-assemble the array with --force like it suggests,
> > then add the failed device and let it recover.
> >
> > NeilBrown
> >
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 828 bytes --]
^ permalink raw reply
* Re: Please advise, strange "not enough to start the array while not clean"
From: Patrik Horník @ 2014-09-22 6:34 UTC (permalink / raw)
To: NeilBrown; +Cc: linux-raid
In-Reply-To: <20140922131940.58f97093@notabene.brown>
- Well what is exact meaning of --no-degraded then? Because I am using
it also on RAID6 arrays that are missing one drive and mdadm starts
them. I thought until today that it is against assembling for example
RAID6 array with missing more than two drives or to be more precise
array with number of drives it used last time. (I did not look at the
code what does it exactly. It is mdadm 3.3 on Debian.)
- Well array was shutdown cleanly manually by mdadm -S. Cant the not
clean classification be result of md101 device between find devices or
result of first two assemble tries?
- Anyway as I mentioned superblock on all five devices has clean state. Example:
/dev/sdk1:
Magic : XXXXXXX
Version : 1.2
Feature Map : 0x1
Array UUID : XXXXXXXXXXXXXXXXXXXXX
Name :
Creation Time : Thu Aug XXXXXXXX
Raid Level : raid6
Raid Devices : 6
Avail Dev Size : 5860268943 (2794.39 GiB 3000.46 GB)
Array Size : 11720536064 (11177.57 GiB 12001.83 GB)
Used Dev Size : 5860268032 (2794.39 GiB 3000.46 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Unused Space : before=262056 sectors, after=911 sectors
State : clean
Device UUID : YYYYYYYYYYYYYYYYYYYY
Internal Bitmap : 8 sectors from superblock
Update Time : Mon Sep 22 02:23:45 2014
Bad Block Log : 512 entries available at offset 72 sectors
Checksum : ZZZZZZZZ - correct
Events : EEEEEE
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 4
Array State : AAAAA. ('A' == active, '.' == missing, 'R' == replacing)
- md101 has Events count lower by 16 than others devices.
- Please I need little more assurance what is exact state of array and
explain why it is behaving as it is behaving, so I can be sure what
steps are needed and what happens. The data on array is important.
Patrik Horník
šéfredaktor www.DSL.sk
Tel.: +421 905 385 666
Email: patrik@dsl.sk
2014-09-22 5:19 GMT+02:00 NeilBrown <neilb@suse.de>:
> On Mon, 22 Sep 2014 04:11:20 +0200 Patrik Horník <patrik@dsl.sk> wrote:
>
>> Hello Neil,
>>
>> I've got this situation unfamiliar to me on RAID6 array md1 with important data.
>>
>> - It is RAID6 with 6 devices, 5 are partitions and 1 is another RAID0
>> array md101 from two smaller drives. One of the smaller drives froze,
>> so md101 got kicked out from md1 and marked as faulty in md1. After
>> while I've stopped md1 without removing md101 from it first. Then I
>> rebooted and assembled md101.
>>
>> - First I tried mdadm -A --no-degraded -u UUID /dev/md1 but got
>> "mdadm: /dev/md1 assembled from 5 drives (out of 6), but not started."
>> so I stopped the md1.
>>
>> - Second time I started it with -v and got:
>>
>> mdadm: /dev/md101 is identified as a member of /dev/md1, slot 5.
>> mdadm: /dev/sdk1 is identified as a member of /dev/md1, slot 4.
>> mdadm: /dev/sdi1 is identified as a member of /dev/md1, slot 1.
>> mdadm: /dev/sdh1 is identified as a member of /dev/md1, slot 2.
>> mdadm: /dev/sdg1 is identified as a member of /dev/md1, slot 0.
>> mdadm: /dev/sde1 is identified as a member of /dev/md1, slot 3.
>> mdadm: added /dev/sdi1 to /dev/md1 as 1
>> mdadm: added /dev/sdh1 to /dev/md1 as 2
>> mdadm: added /dev/sde1 to /dev/md1 as 3
>> mdadm: added /dev/sdk1 to /dev/md1 as 4
>> mdadm: added /dev/md101 to /dev/md1 as 5 (possibly out of date)
>> mdadm: added /dev/sdg1 to /dev/md1 as 0
>> mdadm: /dev/md1 assembled from 5 drives (out of 6), but not started.
>>
>> - On third time I tried without --nodegraded with mdadm -A -v -u UUID
>> /dev/md1. This is what I've got:
>>
>> mdadm: /dev/md101 is identified as a member of /dev/md1, slot 5.
>> mdadm: /dev/sdk1 is identified as a member of /dev/md1, slot 4.
>> mdadm: /dev/sdi1 is identified as a member of /dev/md1, slot 1.
>> mdadm: /dev/sdh1 is identified as a member of /dev/md1, slot 2.
>> mdadm: /dev/sdg1 is identified as a member of /dev/md1, slot 0.
>> mdadm: /dev/sde1 is identified as a member of /dev/md1, slot 3.
>> mdadm: added /dev/sdi1 to /dev/md1 as 1
>> mdadm: added /dev/sdh1 to /dev/md1 as 2
>> mdadm: added /dev/sde1 to /dev/md1 as 3
>> mdadm: added /dev/sdk1 to /dev/md1 as 4
>> mdadm: added /dev/md101 to /dev/md1 as 5 (possibly out of date)
>> mdadm: added /dev/sdg1 to /dev/md1 as 0
>> mdadm: /dev/md1 assembled from 5 drives - not enough to start the
>> array while not clean - consider --force.
>>
>> Array md1 has bitmap. All drive devices have all same Events, their
>> state is clean and Device Role is Active device. md101 has active
>> state and lower Events.
>>
>> Is this expected behavior? My theory is that it is caused by md101 and
>> I should start array md1 without it (by for example stopping md101)
>> and then re-add it. Is that a case or is it something else?
>>
>> Thanks.
>>
>> Best regards,
>>
>> Patrik
>
>
> The array is clearly degraded as one of the devices failed and hasn't been
> recovered yet, so using --nodegraded is counter productive, as you
> discovered.
>
> It appears that the array is also marked as 'dirty'. That suggests that it
> wasn't shut down cleanly.
> What does "mdadm --examine" of some device show?
>
> You probably need to re-assemble the array with --force like it suggests,
> then add the failed device and let it recover.
>
> NeilBrown
>
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: not enough operational mirrors
From: NeilBrown @ 2014-09-22 5:47 UTC (permalink / raw)
To: Ian Young; +Cc: linux-raid
In-Reply-To: <CANs+QMzWWqh-5MemOrbguaq7fNzJPDaYUH66K0rgGrnaXY7FNA@mail.gmail.com>
[-- Attachment #1: Type: text/plain, Size: 1132 bytes --]
On Sun, 21 Sep 2014 22:32:19 -0700 Ian Young <ian@duffrecords.com> wrote:
> My 6-drive software RAID 10 array failed. The individual drives
> failed one at a time over the past few months but it's been an
> extremely busy summer and I didn't have the free time to RMA the
> drives and rebuild the array. Now I'm wishing I had acted sooner
> because three of the drives are marked as removed and the array
> doesn't have enough mirrors to start. I followed the recovery
> instructions at raid.wiki.kernel.org and, before making things any
> worse, saved the status using mdadm --examine and consulted this
> mailing list. Here's the status:
>
> http://pastebin.com/KkV8e8Gq
>
> I can see that the event counts on sdd2 and sdf2 are significantly far
> behind, so we can consider that data too old. sdc2 is only behind by
> two events, so any data loss there should be minimal. If I can make
> the array start with sd[abce]2 I think that will be enough to mount
> the filesystem, back up my data, and start replacing drives. How do I
> do that?
Use the "--force" option with "--assemble".
NeilBrown
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 828 bytes --]
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox