* [PATCH v3 2/3] bcache: update document info
From: Yijing Wang @ 2016-07-04 1:23 UTC (permalink / raw)
To: axboe, Kent Overstreet
Cc: Eric Wheeler, Coly Li, linux-bcache, linux-raid, linux-kernel,
Yijing Wang
There is no return in continue_at(), update the documentation.
Signed-off-by: Yijing Wang <wangyijing@huawei.com>
---
drivers/md/bcache/closure.c | 2 +-
drivers/md/bcache/closure.h | 3 ++-
2 files changed, 3 insertions(+), 2 deletions(-)
diff --git a/drivers/md/bcache/closure.c b/drivers/md/bcache/closure.c
index 9eaf1d6..864e673 100644
--- a/drivers/md/bcache/closure.c
+++ b/drivers/md/bcache/closure.c
@@ -112,7 +112,7 @@ bool closure_wait(struct closure_waitlist *waitlist, struct closure *cl)
EXPORT_SYMBOL(closure_wait);
/**
- * closure_sync - sleep until a closure a closure has nothing left to wait on
+ * closure_sync - sleep until a closure has nothing left to wait on
*
* Sleeps until the refcount hits 1 - the thread that's running the closure owns
* the last refcount.
diff --git a/drivers/md/bcache/closure.h b/drivers/md/bcache/closure.h
index 782cc2c..9b2fe2d 100644
--- a/drivers/md/bcache/closure.h
+++ b/drivers/md/bcache/closure.h
@@ -31,7 +31,8 @@
* passing it, as you might expect, the function to run when nothing is pending
* and the workqueue to run that function out of.
*
- * continue_at() also, critically, is a macro that returns the calling function.
+ * continue_at() also, critically, requires a 'return' immediately following the
+ * location where this macro is referenced, to return to the calling function.
* There's good reason for this.
*
* To use safely closures asynchronously, they must always have a refcount while
--
1.7.1
^ permalink raw reply related
* [PATCH v3 1/3] bcache: Remove redundant parameter for cache_alloc()
From: Yijing Wang @ 2016-07-04 1:23 UTC (permalink / raw)
To: axboe, Kent Overstreet
Cc: Eric Wheeler, Coly Li, linux-bcache, linux-raid, linux-kernel,
Yijing Wang
Cache_sb is not used in cache_alloc, and we have copied
sb info to cache->sb already, remove it.
Reviewed-by: Coly Li <colyli@suse.de>
Signed-off-by: Yijing Wang <wangyijing@huawei.com>
---
drivers/md/bcache/super.c | 4 ++--
1 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/drivers/md/bcache/super.c b/drivers/md/bcache/super.c
index f5dbb4e..aecaace 100644
--- a/drivers/md/bcache/super.c
+++ b/drivers/md/bcache/super.c
@@ -1803,7 +1803,7 @@ void bch_cache_release(struct kobject *kobj)
module_put(THIS_MODULE);
}
-static int cache_alloc(struct cache_sb *sb, struct cache *ca)
+static int cache_alloc(struct cache *ca)
{
size_t free;
struct bucket *b;
@@ -1858,7 +1858,7 @@ static int register_cache(struct cache_sb *sb, struct page *sb_page,
if (blk_queue_discard(bdev_get_queue(ca->bdev)))
ca->discard = CACHE_DISCARD(&ca->sb);
- ret = cache_alloc(sb, ca);
+ ret = cache_alloc(ca);
if (ret != 0)
goto err;
--
1.7.1
^ permalink raw reply related
* Re: [RFC] block: fix blk_queue_split() resource exhaustion
From: Ming Lei @ 2016-07-02 10:28 UTC (permalink / raw)
To: Lars Ellenberg
Cc: linux-block, Roland Kammerer, Jens Axboe, NeilBrown,
Kent Overstreet, Shaohua Li, Alasdair Kergon, Mike Snitzer,
open list:DEVICE-MAPPER (LVM), Ingo Molnar, Peter Zijlstra,
Takashi Iwai, Jiri Kosina, Zheng Liu, Keith Busch,
Martin K. Petersen, Kirill A. Shutemov, Linux Kernel Mailing List,
open list:BCACHE (BLOCK LAYER CACHE),
open list:SOFTWARE RAID (Multiple Disks) SUPPORT
In-Reply-To: <1466583730-28595-1-git-send-email-lars.ellenberg@linbit.com>
On Wed, Jun 22, 2016 at 4:22 PM, Lars Ellenberg
<lars.ellenberg@linbit.com> wrote:
> For a long time, generic_make_request() converts recursion into
> iteration by queuing recursive arguments on current->bio_list.
>
> This is convenient for stacking drivers,
> the top-most driver would take the originally submitted bio,
> and re-submit a re-mapped version of it, or one or more clones,
> or one or more new allocated bios to its backend(s). Which
> are then simply processed in turn, and each can again queue
> more "backend-bios" until we reach the bottom of the driver stack,
> and actually dispatch to the real backend device.
>
> Any stacking driver ->make_request_fn() could expect that,
> once it returns, any backend-bios it submitted via recursive calls
> to generic_make_request() would now be processed and dispatched, before
> the current task would call into this driver again.
>
> This is changed by commit
> 54efd50 block: make generic_make_request handle arbitrarily sized bios
>
> Drivers may call blk_queue_split() inside their ->make_request_fn(),
> which may split the current bio into a front-part to be dealt with
> immediately, and a remainder-part, which may need to be split even
> further. That remainder-part will simply also be pushed to
> current->bio_list, and would end up being head-of-queue, in front
> of any backend-bios the current make_request_fn() might submit during
> processing of the fron-part.
>
> Which means the current task would immediately end up back in the same
> make_request_fn() of the same driver again, before any of its backend
> bios have even been processed.
>
> This can lead to resource starvation deadlock.
> Drivers could avoid this by learning to not need blk_queue_split(),
> or by submitting their backend bios in a different context (dedicated
> kernel thread, work_queue context, ...). Or by playing funny re-ordering
> games with entries on current->bio_list.
>
> Instead, I suggest to distinguish between recursive calls to
> generic_make_request(), and pushing back the remainder part in
> blk_queue_split(), by pointing current->bio_lists to a
> struct recursion_to_iteration_bio_lists {
> struct bio_list recursion;
> struct bio_list remainder;
> }
>
> To have all bios targeted to drivers lower in the stack processed before
> processing the next piece of a bio targeted at the higher levels,
> as long as queued bios resulting from recursion are available,
> they will continue to be processed in FIFO order.
> Pushed back bio-parts resulting from blk_queue_split() will be processed
> in LIFO order, one-by-one, whenever the recursion list becomes empty.
>
> Signed-off-by: Lars Ellenberg <lars.ellenberg@linbit.com>
> Signed-off-by: Roland Kammerer <roland.kammerer@linbit.com>
> ---
>
> This is not a theoretical problem.
> At least int DRBD, and an unfortunately high IO concurrency wrt. the
> "max-buffers" setting, without this patch we have a reproducible deadlock.
>
> I'm unsure if it could cause problems in md raid in some corner cases
> or when a rebuild/scrub/... starts in a "bad" moment.
>
> It also may increase "stress" on any involved bio_set,
> because it may increase the number of bios that are allocated
> before the first of them is actually processed, causing more
> frequent calls of punt_bios_to_rescuer().
>
> Alternatively,
> a simple one-line change to generic_make_request() would also "fix" it,
> by processing all recursive bios in LIFO order.
> But that may change the order in which bios reach the "physical" layer,
> in case they are in fact split into many smaller pieces, for example in
> a striping setup, which may have other performance implications.
>
> | diff --git a/block/blk-core.c b/block/blk-core.c
> | index 2475b1c7..a5623f6 100644
> | --- a/block/blk-core.c
> | +++ b/block/blk-core.c
> | @@ -2048,7 +2048,7 @@ blk_qc_t generic_make_request(struct bio *bio)
> | * should be added at the tail
> | */
> | if (current->bio_list) {
> | - bio_list_add(current->bio_list, bio);
> | + bio_list_add_head(current->bio_list, bio);
> | goto out;
> | }
>
> Any fix would need to go to stable 4.3.x and up.
> This patch does apply as is to 4.5 and up,
> with a trivial fixup (or a cherry-pick of cda2264) to 4.4,
> and with some more (also trivial) fixup to 4.3 because of
> GFP_WAIT -> GFP_RECLAIM and comment rewrap.
>
> Any input?
The idea looks good, but not sure it can cover all cases of
dm over brbd or brbd over dm and maintaining two lists
becomes too complicated.
One clean solution may be to convert the loop of generic_make_request()
into the following way:
do {
struct bio *splitted, *remainder = NULL;
struct request_queue *q = bdev_get_queue(bio->bi_bdev);
blk_queue_split(q, &bio, &remainder, q->bio_split);
ret = q->make_request_fn(q, bio);
if (remainder)
bio_list_add(current->bio_list, remainder);
bio = bio_list_pop(current->bio_list);
} while (bio)
And blk_queue_split() need a bit change and all its external calling
have to be removed.
Of cource we also need to fix blk_queue_bounce() first, but that can
be done easily by handling bounced bio via generic_make_request().
> How should I proceed?
I think the issue need to be fixed.
Thanks,
Ming
>
> ---
> block/bio.c | 14 +++++++-------
> block/blk-core.c | 32 +++++++++++++++++---------------
> block/blk-merge.c | 3 ++-
> drivers/md/bcache/btree.c | 12 ++++++------
> drivers/md/dm-bufio.c | 2 +-
> drivers/md/md.h | 7 +++++++
> drivers/md/raid1.c | 5 ++---
> drivers/md/raid10.c | 5 ++---
> include/linux/bio.h | 11 +++++++++++
> include/linux/sched.h | 4 ++--
> 10 files changed, 57 insertions(+), 38 deletions(-)
>
> diff --git a/block/bio.c b/block/bio.c
> index 0e4aa42..2ffcea0 100644
> --- a/block/bio.c
> +++ b/block/bio.c
> @@ -368,10 +368,10 @@ static void punt_bios_to_rescuer(struct bio_set *bs)
> bio_list_init(&punt);
> bio_list_init(&nopunt);
>
> - while ((bio = bio_list_pop(current->bio_list)))
> + while ((bio = bio_list_pop(¤t->bio_lists->recursion)))
> bio_list_add(bio->bi_pool == bs ? &punt : &nopunt, bio);
>
> - *current->bio_list = nopunt;
> + current->bio_lists->recursion = nopunt;
>
> spin_lock(&bs->rescue_lock);
> bio_list_merge(&bs->rescue_list, &punt);
> @@ -453,13 +453,13 @@ struct bio *bio_alloc_bioset(gfp_t gfp_mask, int nr_iovecs, struct bio_set *bs)
> *
> * We solve this, and guarantee forward progress, with a rescuer
> * workqueue per bio_set. If we go to allocate and there are
> - * bios on current->bio_list, we first try the allocation
> - * without __GFP_DIRECT_RECLAIM; if that fails, we punt those
> - * bios we would be blocking to the rescuer workqueue before
> - * we retry with the original gfp_flags.
> + * bios on current->bio_lists->recursion, we first try the
> + * allocation without __GFP_DIRECT_RECLAIM; if that fails, we
> + * punt those bios we would be blocking to the rescuer
> + * workqueue before we retry with the original gfp_flags.
> */
>
> - if (current->bio_list && !bio_list_empty(current->bio_list))
> + if (current->bio_lists && !bio_list_empty(¤t->bio_lists->recursion))
> gfp_mask &= ~__GFP_DIRECT_RECLAIM;
>
> p = mempool_alloc(bs->bio_pool, gfp_mask);
> diff --git a/block/blk-core.c b/block/blk-core.c
> index 2475b1c7..f03ff4c 100644
> --- a/block/blk-core.c
> +++ b/block/blk-core.c
> @@ -2031,7 +2031,7 @@ end_io:
> */
> blk_qc_t generic_make_request(struct bio *bio)
> {
> - struct bio_list bio_list_on_stack;
> + struct recursion_to_iteration_bio_lists bio_lists_on_stack;
> blk_qc_t ret = BLK_QC_T_NONE;
>
> if (!generic_make_request_checks(bio))
> @@ -2040,15 +2040,18 @@ blk_qc_t generic_make_request(struct bio *bio)
> /*
> * We only want one ->make_request_fn to be active at a time, else
> * stack usage with stacked devices could be a problem. So use
> - * current->bio_list to keep a list of requests submited by a
> - * make_request_fn function. current->bio_list is also used as a
> + * current->bio_lists to keep a list of requests submited by a
> + * make_request_fn function. current->bio_lists is also used as a
> * flag to say if generic_make_request is currently active in this
> * task or not. If it is NULL, then no make_request is active. If
> * it is non-NULL, then a make_request is active, and new requests
> - * should be added at the tail
> + * should be added at the tail of current->bio_lists->recursion;
> + * remainder bios resulting from a call to bio_queue_split() from
> + * within ->make_request_fn() should be added to the head of
> + * current->bio_lists->remainder.
> */
> - if (current->bio_list) {
> - bio_list_add(current->bio_list, bio);
> + if (current->bio_lists) {
> + bio_list_add(¤t->bio_lists->recursion, bio);
> goto out;
> }
>
> @@ -2057,7 +2060,7 @@ blk_qc_t generic_make_request(struct bio *bio)
> * Before entering the loop, bio->bi_next is NULL (as all callers
> * ensure that) so we have a list with a single bio.
> * We pretend that we have just taken it off a longer list, so
> - * we assign bio_list to a pointer to the bio_list_on_stack,
> + * we assign bio_list to a pointer to the bio_lists_on_stack,
> * thus initialising the bio_list of new bios to be
> * added. ->make_request() may indeed add some more bios
> * through a recursive call to generic_make_request. If it
> @@ -2067,8 +2070,9 @@ blk_qc_t generic_make_request(struct bio *bio)
> * bio_list, and call into ->make_request() again.
> */
> BUG_ON(bio->bi_next);
> - bio_list_init(&bio_list_on_stack);
> - current->bio_list = &bio_list_on_stack;
> + bio_list_init(&bio_lists_on_stack.recursion);
> + bio_list_init(&bio_lists_on_stack.remainder);
> + current->bio_lists = &bio_lists_on_stack;
> do {
> struct request_queue *q = bdev_get_queue(bio->bi_bdev);
>
> @@ -2076,16 +2080,14 @@ blk_qc_t generic_make_request(struct bio *bio)
> ret = q->make_request_fn(q, bio);
>
> blk_queue_exit(q);
> -
> - bio = bio_list_pop(current->bio_list);
> } else {
> - struct bio *bio_next = bio_list_pop(current->bio_list);
> -
> bio_io_error(bio);
> - bio = bio_next;
> }
> + bio = bio_list_pop(¤t->bio_lists->recursion);
> + if (!bio)
> + bio = bio_list_pop(¤t->bio_lists->remainder);
> } while (bio);
> - current->bio_list = NULL; /* deactivate */
> + current->bio_lists = NULL; /* deactivate */
>
> out:
> return ret;
> diff --git a/block/blk-merge.c b/block/blk-merge.c
> index 2613531..8da0c22 100644
> --- a/block/blk-merge.c
> +++ b/block/blk-merge.c
> @@ -172,6 +172,7 @@ void blk_queue_split(struct request_queue *q, struct bio **bio,
> struct bio *split, *res;
> unsigned nsegs;
>
> + BUG_ON(!current->bio_lists);
> if ((*bio)->bi_rw & REQ_DISCARD)
> split = blk_bio_discard_split(q, *bio, bs, &nsegs);
> else if ((*bio)->bi_rw & REQ_WRITE_SAME)
> @@ -190,7 +191,7 @@ void blk_queue_split(struct request_queue *q, struct bio **bio,
>
> bio_chain(split, *bio);
> trace_block_split(q, split, (*bio)->bi_iter.bi_sector);
> - generic_make_request(*bio);
> + bio_list_add_head(¤t->bio_lists->remainder, *bio);
> *bio = split;
> }
> }
> diff --git a/drivers/md/bcache/btree.c b/drivers/md/bcache/btree.c
> index eab505e..eb0b41a 100644
> --- a/drivers/md/bcache/btree.c
> +++ b/drivers/md/bcache/btree.c
> @@ -450,7 +450,7 @@ void __bch_btree_node_write(struct btree *b, struct closure *parent)
>
> trace_bcache_btree_write(b);
>
> - BUG_ON(current->bio_list);
> + BUG_ON(current->bio_lists);
> BUG_ON(b->written >= btree_blocks(b));
> BUG_ON(b->written && !i->keys);
> BUG_ON(btree_bset_first(b)->seq != i->seq);
> @@ -544,7 +544,7 @@ static void bch_btree_leaf_dirty(struct btree *b, atomic_t *journal_ref)
>
> /* Force write if set is too big */
> if (set_bytes(i) > PAGE_SIZE - 48 &&
> - !current->bio_list)
> + !current->bio_lists)
> bch_btree_node_write(b, NULL);
> }
>
> @@ -889,7 +889,7 @@ static struct btree *mca_alloc(struct cache_set *c, struct btree_op *op,
> {
> struct btree *b;
>
> - BUG_ON(current->bio_list);
> + BUG_ON(current->bio_lists);
>
> lockdep_assert_held(&c->bucket_lock);
>
> @@ -976,7 +976,7 @@ retry:
> b = mca_find(c, k);
>
> if (!b) {
> - if (current->bio_list)
> + if (current->bio_lists)
> return ERR_PTR(-EAGAIN);
>
> mutex_lock(&c->bucket_lock);
> @@ -2127,7 +2127,7 @@ static int bch_btree_insert_node(struct btree *b, struct btree_op *op,
>
> return 0;
> split:
> - if (current->bio_list) {
> + if (current->bio_lists) {
> op->lock = b->c->root->level + 1;
> return -EAGAIN;
> } else if (op->lock <= b->c->root->level) {
> @@ -2209,7 +2209,7 @@ int bch_btree_insert(struct cache_set *c, struct keylist *keys,
> struct btree_insert_op op;
> int ret = 0;
>
> - BUG_ON(current->bio_list);
> + BUG_ON(current->bio_lists);
> BUG_ON(bch_keylist_empty(keys));
>
> bch_btree_op_init(&op.op, 0);
> diff --git a/drivers/md/dm-bufio.c b/drivers/md/dm-bufio.c
> index cd77216..b64d4f0 100644
> --- a/drivers/md/dm-bufio.c
> +++ b/drivers/md/dm-bufio.c
> @@ -174,7 +174,7 @@ static inline int dm_bufio_cache_index(struct dm_bufio_client *c)
> #define DM_BUFIO_CACHE(c) (dm_bufio_caches[dm_bufio_cache_index(c)])
> #define DM_BUFIO_CACHE_NAME(c) (dm_bufio_cache_names[dm_bufio_cache_index(c)])
>
> -#define dm_bufio_in_request() (!!current->bio_list)
> +#define dm_bufio_in_request() (!!current->bio_lists)
>
> static void dm_bufio_lock(struct dm_bufio_client *c)
> {
> diff --git a/drivers/md/md.h b/drivers/md/md.h
> index b5c4be7..7afc71c 100644
> --- a/drivers/md/md.h
> +++ b/drivers/md/md.h
> @@ -663,6 +663,13 @@ static inline void rdev_dec_pending(struct md_rdev *rdev, struct mddev *mddev)
> }
> }
>
> +static inline bool current_has_pending_bios(void)
> +{
> + return current->bio_lists && (
> + !bio_list_empty(¤t->bio_lists->recursion) ||
> + !bio_list_empty(¤t->bio_lists->remainder));
> +}
> +
> extern struct md_cluster_operations *md_cluster_ops;
> static inline int mddev_is_clustered(struct mddev *mddev)
> {
> diff --git a/drivers/md/raid1.c b/drivers/md/raid1.c
> index c7c8cde..811f2c0 100644
> --- a/drivers/md/raid1.c
> +++ b/drivers/md/raid1.c
> @@ -876,8 +876,7 @@ static sector_t wait_barrier(struct r1conf *conf, struct bio *bio)
> (!conf->barrier ||
> ((conf->start_next_window <
> conf->next_resync + RESYNC_SECTORS) &&
> - current->bio_list &&
> - !bio_list_empty(current->bio_list))),
> + current_has_pending_bios())),
> conf->resync_lock);
> conf->nr_waiting--;
> }
> @@ -1014,7 +1013,7 @@ static void raid1_unplug(struct blk_plug_cb *cb, bool from_schedule)
> struct r1conf *conf = mddev->private;
> struct bio *bio;
>
> - if (from_schedule || current->bio_list) {
> + if (from_schedule || current->bio_lists) {
> spin_lock_irq(&conf->device_lock);
> bio_list_merge(&conf->pending_bio_list, &plug->pending);
> conf->pending_count += plug->pending_cnt;
> diff --git a/drivers/md/raid10.c b/drivers/md/raid10.c
> index c7de2a5..2c70717 100644
> --- a/drivers/md/raid10.c
> +++ b/drivers/md/raid10.c
> @@ -945,8 +945,7 @@ static void wait_barrier(struct r10conf *conf)
> wait_event_lock_irq(conf->wait_barrier,
> !conf->barrier ||
> (conf->nr_pending &&
> - current->bio_list &&
> - !bio_list_empty(current->bio_list)),
> + current_has_pending_bios()),
> conf->resync_lock);
> conf->nr_waiting--;
> }
> @@ -1022,7 +1021,7 @@ static void raid10_unplug(struct blk_plug_cb *cb, bool from_schedule)
> struct r10conf *conf = mddev->private;
> struct bio *bio;
>
> - if (from_schedule || current->bio_list) {
> + if (from_schedule || current->bio_lists) {
> spin_lock_irq(&conf->device_lock);
> bio_list_merge(&conf->pending_bio_list, &plug->pending);
> conf->pending_count += plug->pending_cnt;
> diff --git a/include/linux/bio.h b/include/linux/bio.h
> index 9faebf7..57e45a0 100644
> --- a/include/linux/bio.h
> +++ b/include/linux/bio.h
> @@ -598,6 +598,17 @@ struct bio_list {
> struct bio *tail;
> };
>
> +/* for generic_make_request() */
> +struct recursion_to_iteration_bio_lists {
> + /* For stacking drivers submitting to their respective backend.
> + * Added to tail, processed in FIFO order. */
> + struct bio_list recursion;
> + /* For pushing back the "remainder" part resulting from calling
> + * blk_queue_split(). Added to head, processed in LIFO order,
> + * once the "recursion" list has been drained. */
> + struct bio_list remainder;
> +};
> +
> static inline int bio_list_empty(const struct bio_list *bl)
> {
> return bl->head == NULL;
> diff --git a/include/linux/sched.h b/include/linux/sched.h
> index 6e42ada..146eedc 100644
> --- a/include/linux/sched.h
> +++ b/include/linux/sched.h
> @@ -128,7 +128,7 @@ struct sched_attr {
>
> struct futex_pi_state;
> struct robust_list_head;
> -struct bio_list;
> +struct recursion_to_iteration_bio_lists;
> struct fs_struct;
> struct perf_event_context;
> struct blk_plug;
> @@ -1727,7 +1727,7 @@ struct task_struct {
> void *journal_info;
>
> /* stacked block device info */
> - struct bio_list *bio_list;
> + struct recursion_to_iteration_bio_lists *bio_lists;
>
> #ifdef CONFIG_BLOCK
> /* stack plugging */
> --
> 1.9.1
>
^ permalink raw reply
* Re: [RFC] block: fix blk_queue_split() resource exhaustion
From: Ming Lei @ 2016-07-02 10:03 UTC (permalink / raw)
To: Lars Ellenberg
Cc: linux-block, Roland Kammerer, Jens Axboe, NeilBrown,
Kent Overstreet, Shaohua Li, Alasdair Kergon, Mike Snitzer,
open list:DEVICE-MAPPER (LVM), Ingo Molnar, Peter Zijlstra,
Takashi Iwai, Jiri Kosina, Zheng Liu, Keith Busch,
Martin K. Petersen, Kirill A. Shutemov, Linux Kernel Mailing List,
open list:BCACHE (BLOCK LAYER CACHE),
open list:SOFTWARE RAID (Multiple Disks) SUPPORT
In-Reply-To: <20160628084534.GI3239@soda.linbit>
On Tue, Jun 28, 2016 at 4:45 PM, Lars Ellenberg
<lars.ellenberg@linbit.com> wrote:
> On Sat, Jun 25, 2016 at 05:30:29PM +0800, Ming Lei wrote:
>> On Fri, Jun 24, 2016 at 10:27 PM, Lars Ellenberg
>> <lars.ellenberg@linbit.com> wrote:
>> > On Fri, Jun 24, 2016 at 07:36:57PM +0800, Ming Lei wrote:
>> >> >
>> >> > This is not a theoretical problem.
>> >> > At least int DRBD, and an unfortunately high IO concurrency wrt. the
>> >> > "max-buffers" setting, without this patch we have a reproducible deadlock.
>> >>
>> >> Is there any log about the deadlock? And is there any lockdep warning
>> >> if it is enabled?
>> >
>> > In DRBD, to avoid potentially very long internal queues as we wait for
>> > our replication peer device and local backend, we limit the number of
>> > in-flight bios we accept, and block in our ->make_request_fn() if that
>> > number exceeds a configured watermark ("max-buffers").
>> >
>> > Works fine, as long as we could assume that once our make_request_fn()
>> > returns, any bios we "recursively" submitted against the local backend
>> > would be dispatched. Which used to be the case.
>> >
>> > commit 54efd50 block: make generic_make_request handle arbitrarily sized bios
>> > introduced blk_queue_split(), which will split any bio that is
>> > violating the queue limits into a smaller piece to be processed
>> > right away, and queue the "rest" on current->bio_list to be
>> > processed by the iteration in generic_make_request() once the
>> > current ->make_request_fn() returns.
>> >
>> > Before that, any user was supposed to go through bio_add_page(),
>> > which would call our merge bvec function, and that should already
>> > be sufficient to not violate queue limits.
>> >
>> > Previously, typically the next in line bio to be processed would
>> > be the cloned one we passed down to our backend device, which in
>> > case of further stacking devices (e.g. logical volume, raid1 or
>> > similar) may again push further bios on that list.
>> > All of which would simply be processed in turn.
>>
>> Could you clarify if the issue is triggered in drbd without dm/md involved?
>> Or the issue is always triggered with dm/md over drbd?
>>
>> As Mike mentioned, there is one known issue with dm-snapshot.
>
>
> The issue can always be triggered, even if only DRBD is involved.
>
>> If your ->make_request_fn() is called several times, each time
>> at least you should dispatch one bio returnd from blk_queue_split(),
>> so I don't understand why even one bio isn't dispatched out.
>
> I'll try to "visualize" the mechanics of "my" deadlock here.
>
> Just to clarify the effect, assume we had a driver that
> for $reasons would limit the number of in-flight IO to
> one single bio.
>
> === before bio_queue_split()
>
> Previously, if something would want to read some range of blocks,
> it would allocate a bio, call bio_add_page() a number of times,
> and once the bio was "full", call generic_make_request(),
> and fill the next bio, then submit that one.
>
> Stacking: "single_in_flight" (q1) -> "sda" (q2)
>
> generic_make_request(bio) current->bio_list in-flight
> B_orig_1 NULL 0
> q1->make_request_fn(B_orig_1) empty
> 1
> recursive call, queues: B_1_remapped
> iterative call:
> q2->make_request_fn(B_1_remapped) empty
> -> actually dispatched to hardware
> return of generic_make_request(B_orig_1).
> B_orig_2
> q1->make_request_fn(B_orig_1)
> 1
> blocks, waits for in-flight to drop
> ...
> completion of B_orig_1 0
>
> recursive call, queues: B_2_remapped
> iterative call:
> q2->make_request_fn(B_2_remapped) empty
> -> actually dispatched to hardware
> return of generic_make_request(B_orig_2).
>
>
> === with bio_queue_split()
>
> Now, uppser layers buils one large bio.
>
> generic_make_request(bio) current->bio_list in-flight
> B_orig NULL 0
> q1->make_request_fn(B_orig) empty
> blk_queue_split(B_orig) splits into
> B_orig_r1
> B_orig_s1
> 1
> recursive call, queues: B_orig_r1, B_s1_remapped
> iterative call:
> q1->make_request_fn(B_orig_r1) B_s1_remapped
> blocks, waits for in-flight to drop
> ...
> which never happens,
> because B_s1_remapped has not even been submitted to q2 yet,
> let alone dispatched to hardware.
>
>
> Obviously we don't limit ourselves to just one request, but with larger
> incoming bios, with the number of times we split them, with the number
> of stacking layers below us, or even layers below us that *also*
> call blk_queue_split (or equivalent open coded clone and split)
> themselves to split even further, things get worse.
Thanks for your so detailed explanation!
Now I believe the issue is really from blk_queue_split(), and I will comment
on your patch later.
thanks,
^ permalink raw reply
* Re: URE, link resets, user hostile defaults
From: Chris Murphy @ 2016-07-01 20:43 UTC (permalink / raw)
To: Edward Kuns; +Cc: Zygo Blaxell, Chris Murphy, Hannes Reinecke, Linux-RAID
In-Reply-To: <CACsGCyRd388u4XXCSnoK4TpNoPibjhdgEJsHS=vR6aztDc=+2A@mail.gmail.com>
Here's a fun one of these I just got off the Fedora users mailing list
with a laptop drive that's apparently hanging on *write*. This I would
not expect to take a long time for a drive to figure out, but... there
are more resets than there are write errors, and in fact there's no
discrete write error from the drive, all we know is the failed command
is a WRITE command.
What seems to happen is, everything in the queue gets obliterated in
the reset, and when ext4 finds out everything failed, not just one
write, it barfs and goes read only.
http://pastebin.com/3JAL297z
How might this turn out differently if the drive reported a single
discrete write error? I don't know how any file system tolerates this
because it's so rare. Would ext4 just try to write again? Would it try
to write to the same sector or another one? Or maybe the write finally
succeeds by resulting in a remap (?) But this sure is dang slow to
recover from a bad write. I don't understand the engineering rational
for this. Maybe it's a firmware bug?
Chris Murphy
^ permalink raw reply
* [PATCH v2 3/3] bcache: Remove redundant block_size assignment
From: Yijing Wang @ 2016-07-01 9:38 UTC (permalink / raw)
To: axboe, Kent Overstreet
Cc: Eric Wheeler, Coly Li, linux-bcache, linux-raid, linux-kernel,
Yijing Wang
We have assigned sb->block_size before the switch,
so remove the redundant one.
Reviewed-by: Coly Li <colyli@suse.de>
Signed-off-by: Yijing Wang <wangyijing@huawei.com>
Acked-by: Eric Wheeler <bcache@lists.ewheeler.net>
---
drivers/md/bcache/super.c | 1 -
1 files changed, 0 insertions(+), 1 deletions(-)
diff --git a/drivers/md/bcache/super.c b/drivers/md/bcache/super.c
index aecaace..bf4b100 100644
--- a/drivers/md/bcache/super.c
+++ b/drivers/md/bcache/super.c
@@ -134,7 +134,6 @@ static const char *read_super(struct cache_sb *sb, struct block_device *bdev,
case BCACHE_SB_VERSION_CDEV:
case BCACHE_SB_VERSION_CDEV_WITH_UUID:
sb->nbuckets = le64_to_cpu(s->nbuckets);
- sb->block_size = le16_to_cpu(s->block_size);
sb->bucket_size = le16_to_cpu(s->bucket_size);
sb->nr_in_set = le16_to_cpu(s->nr_in_set);
--
1.7.1
^ permalink raw reply related
* [PATCH v2 2/3] bcache: update document info
From: Yijing Wang @ 2016-07-01 9:37 UTC (permalink / raw)
To: axboe, Kent Overstreet
Cc: Eric Wheeler, Coly Li, linux-bcache, linux-raid, linux-kernel,
Yijing Wang
There is no return in continue_at(), update the documentation.
Signed-off-by: Yijing Wang <wangyijing@huawei.com>
---
drivers/md/bcache/closure.c | 2 +-
drivers/md/bcache/closure.h | 3 ++-
2 files changed, 3 insertions(+), 2 deletions(-)
diff --git a/drivers/md/bcache/closure.c b/drivers/md/bcache/closure.c
index 9eaf1d6..864e673 100644
--- a/drivers/md/bcache/closure.c
+++ b/drivers/md/bcache/closure.c
@@ -112,7 +112,7 @@ bool closure_wait(struct closure_waitlist *waitlist, struct closure *cl)
EXPORT_SYMBOL(closure_wait);
/**
- * closure_sync - sleep until a closure a closure has nothing left to wait on
+ * closure_sync - sleep until a closure has nothing left to wait on
*
* Sleeps until the refcount hits 1 - the thread that's running the closure owns
* the last refcount.
diff --git a/drivers/md/bcache/closure.h b/drivers/md/bcache/closure.h
index 782cc2c..5935c3c 100644
--- a/drivers/md/bcache/closure.h
+++ b/drivers/md/bcache/closure.h
@@ -31,7 +31,8 @@
* passing it, as you might expect, the function to run when nothing is pending
* and the workqueue to run that function out of.
*
- * continue_at() also, critically, is a macro that returns the calling function.
+ * continue_at() also, critically, is a macro that needs a return followed to
+ * return the calling function.
* There's good reason for this.
*
* To use safely closures asynchronously, they must always have a refcount while
--
1.7.1
^ permalink raw reply related
* [PATCH v2 1/3] bcache: Remove redundant parameter for cache_alloc()
From: Yijing Wang @ 2016-07-01 9:37 UTC (permalink / raw)
To: axboe, Kent Overstreet
Cc: Eric Wheeler, Coly Li, linux-bcache, linux-raid, linux-kernel,
Yijing Wang
Cache_sb is not used in cache_alloc, and we have copied
sb info to cache->sb already, remove it.
Reviewed-by: Coly Li <colyli@suse.de>
Signed-off-by: Yijing Wang <wangyijing@huawei.com>
---
drivers/md/bcache/super.c | 4 ++--
1 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/drivers/md/bcache/super.c b/drivers/md/bcache/super.c
index f5dbb4e..aecaace 100644
--- a/drivers/md/bcache/super.c
+++ b/drivers/md/bcache/super.c
@@ -1803,7 +1803,7 @@ void bch_cache_release(struct kobject *kobj)
module_put(THIS_MODULE);
}
-static int cache_alloc(struct cache_sb *sb, struct cache *ca)
+static int cache_alloc(struct cache *ca)
{
size_t free;
struct bucket *b;
@@ -1858,7 +1858,7 @@ static int register_cache(struct cache_sb *sb, struct page *sb_page,
if (blk_queue_discard(bdev_get_queue(ca->bdev)))
ca->discard = CACHE_DISCARD(&ca->sb);
- ret = cache_alloc(sb, ca);
+ ret = cache_alloc(ca);
if (ret != 0)
goto err;
--
1.7.1
^ permalink raw reply related
* Re: Assembly of RAID6 with 48 disk fails
From: Soeren Grunewald @ 2016-07-01 7:24 UTC (permalink / raw)
To: Phil Turmel, linux-raid mailinglist
In-Reply-To: <70dae6c2-0137-8436-0625-7a056ddf2779@turmel.org>
Hi Phil,
Thanks for your advice, that actually helped. I just build git head,
used '--assemble --force' and it was working. All filesystems on the
array are still valid, no data loss :) But this is the end for this
machine, since it was clearly a controller fault.
So thank you very very much, also from the users.
--
Cheers,
Soeren
On 06/30/2016 11:06 PM, Phil Turmel wrote:
> On 06/30/2016 11:33 AM, Grunewald, Soeren wrote:
>> Hi All,
>>
>> After a crash, probably caused by some raid issue, the array can't be
>> assembled again. It looks like 8 disk were disappearing form the raid at
>> the same time, which then lead to the system crash. Now I'm unable to
>> get the array back to live again. Because the whole system was on the
>> failing array, I can't access any additional information to check what
>> happened. The system is a 8 year old SUN Fire X4540 system running under
>> Ubuntu 12.04.5 (kernel-3.13.0-91 and mdadm-3.2.5-1ubuntu0.3), which is
>> mainly used for data conversion (converting very large files from one
>> format into another). The boot partition is placed on a 2GB ssd and the
>> rest is running on the 48disk raid6 array. I have booted the system from
>> a usb stick with clonezilla (kernel 4.4.x + mdadm 3.3) and started
>> digging...
>>
>> Since this is the first time, that a I face such a large array and such
>> an issue, I better follow the advice from the Linux-Raid-Wiki and
>> request for help.
>>
>> As one can see in the attached log, the 'Events' count of 40 drives is
>> 4391 and 4385 for the 8 others. We have the same picture for the
>> 'State', 40 drives state 'clean' and 8 drives 'active'. So I tried
>> 'mdadm --assemble --force ...'. This fixed the event count and faulty
>> flag on 4 of the 8 disks. But still the array can't be created.
>>
>> Except one smart error (1 Currently unreadable (pending) sectors) do all
>> other drives pass the extended smart test. I know, this does not mean
>> that the drives are not damaged or broken.
>>
>> Anyway, how should I process to get the array working again?
>>
> You're the second person in the last month to run into this... There
> was a bug in mdadm that makes it fail with --assemble --force when there
> are several out-of-date devices.
>
> The fix was to clone the mdadm git tree, compile the latest, and run
> that stand-alone binary to perform the forced assembly. I haven't tried
> to determine which released version has the fix.
>
> Phil
>
^ permalink raw reply
* Re: [RESEND PATCH 2/3] bcache: update document info
From: wangyijing @ 2016-07-01 6:25 UTC (permalink / raw)
To: Coly Li, Coly Li, axboe, Kent Overstreet
Cc: Eric Wheeler, linux-bcache, linux-raid, linux-kernel
In-Reply-To: <e57c4b4d-cc9f-3ad3-0b2d-d08a4e93cc2a@suse.de>
在 2016/7/1 12:21, Coly Li 写道:
> 在 16/7/1 上午9:51, wangyijing 写道:
>> Hi Coly, thanks to your review and comments.
>>
>> Commit 77b5a08427e875 ("bcache: don't embed 'return' statements in closure macros")
>> remove the return in continue_at(), so I think we should update the document info
>> about continue_at().
>>
>> Thanks!
>> Yijing.
>
> Hi Yijing,
>
> The original version of continue_at() returns to caller function inside
> the macro, Jens thinks this macro breaks code execution flow implicitly,
> so he moves 'return' out of continue_at() and to follow continue_at() at
> the location where continue_at() is referenced.
>
> So as I suggested, the original author means the code should return to
> the calling function.
>
> But YES, I agree that the comments should be updated, because there is
> no 'return' inside macro continue_at(). We should explicitly point out
> that there should be a 'return' immediately following macro continue_at().
Yes, you are right, it's better to explicitly point out a return needed to follow continue_at()
than remove this document info, I will update this patch, thanks very much!
>
> Thanks.
>
> Coly
>
>
>> 在 2016/6/29 18:16, Coly Li 写道:
>>> 在 16/6/22 上午10:12, Yijing Wang 写道:
>>>> There is no return in continue_at(), update the documentation.
>>>>
>>>
>>> There are 2 modification of this patch. The first one is about a typo,
>>> it is correct.
>>>
>>> But I doubt your second modification is proper. The line removed in your
>>> patch is,
>>>> - * continue_at() also, critically, is a macro that returns the
>>> calling function.
>>>> - * There's good reason for this.
>>>> - *
>>>
>>> I think this is exactly what original author wants to say. It does not
>>> mean return a value, it means return to the calling function. And the
>>> bellowed lines explains the reason.
>>>
>>>> Signed-off-by: Yijing Wang <wangyijing@huawei.com>
>>>> ---
>>>> drivers/md/bcache/closure.c | 2 +-
>>>> drivers/md/bcache/closure.h | 3 ---
>>>> 2 files changed, 1 insertions(+), 4 deletions(-)
>>>>
>>>> diff --git a/drivers/md/bcache/closure.c b/drivers/md/bcache/closure.c
>>>> index 9eaf1d6..864e673 100644
>>>> --- a/drivers/md/bcache/closure.c
>>>> +++ b/drivers/md/bcache/closure.c
>>>> @@ -112,7 +112,7 @@ bool closure_wait(struct closure_waitlist *waitlist, struct closure *cl)
>>>> EXPORT_SYMBOL(closure_wait);
>>>>
>>>> /**
>>>> - * closure_sync - sleep until a closure a closure has nothing left to wait on
>>>> + * closure_sync - sleep until a closure has nothing left to wait on
>>>
>>> Yes, this modification is good.
>>>
>>>> *
>>>> * Sleeps until the refcount hits 1 - the thread that's running the closure owns
>>>> * the last refcount.
>>>> diff --git a/drivers/md/bcache/closure.h b/drivers/md/bcache/closure.h
>>>> index 782cc2c..f51188d 100644
>>>> --- a/drivers/md/bcache/closure.h
>>>> +++ b/drivers/md/bcache/closure.h
>>>> @@ -31,9 +31,6 @@
>>>> * passing it, as you might expect, the function to run when nothing is pending
>>>> * and the workqueue to run that function out of.
>>>> *
>>>> - * continue_at() also, critically, is a macro that returns the calling function.
>>>> - * There's good reason for this.
>>>> - *
>>>> * To use safely closures asynchronously, they must always have a refcount while
>>>> * they are running owned by the thread that is running them. Otherwise, suppose
>>>> * you submit some bios and wish to have a function run when they all complete:
>>>>
>>>
>>>
>>
>
>
> .
>
^ permalink raw reply
* Re: [RESEND PATCH 1/3] bcache: Remove redundant parameter for cache_alloc()
From: wangyijing @ 2016-07-01 6:18 UTC (permalink / raw)
To: Coly Li, Coly Li, axboe, Kent Overstreet
Cc: Eric Wheeler, linux-bcache, linux-raid, linux-kernel
In-Reply-To: <f97f9fd3-de1d-5de0-3593-c3fd1901d455@suse.de>
在 2016/7/1 12:24, Coly Li 写道:
> 在 16/7/1 上午10:09, wangyijing 写道:
>>
>>
>> 在 2016/6/29 18:20, Coly Li 写道:
>>> 在 16/6/22 上午10:10, Yijing Wang 写道:
>>>> Cache_sb is not used in cache_alloc, and we have copied
>>>> sb info to cache->sb already, remove it.
>>>>
>>>> Signed-off-by: Yijing Wang <wangyijing@huawei.com>
>>>> ---
>>>> drivers/md/bcache/super.c | 4 ++--
>>>> 1 files changed, 2 insertions(+), 2 deletions(-)
>>>>
>>>> diff --git a/drivers/md/bcache/super.c b/drivers/md/bcache/super.c
>>>> index f5dbb4e..aecaace 100644
>>>> --- a/drivers/md/bcache/super.c
>>>> +++ b/drivers/md/bcache/super.c
>>>> @@ -1803,7 +1803,7 @@ void bch_cache_release(struct kobject *kobj)
>>>> module_put(THIS_MODULE);
>>>> }
>>>>
>>>> -static int cache_alloc(struct cache_sb *sb, struct cache *ca)
>>>> +static int cache_alloc(struct cache *ca)
>>>> {
>>>> size_t free;
>>>> struct bucket *b;
>>>> @@ -1858,7 +1858,7 @@ static int register_cache(struct cache_sb *sb, struct page *sb_page,
>>>> if (blk_queue_discard(bdev_get_queue(ca->bdev)))
>>>> ca->discard = CACHE_DISCARD(&ca->sb);
>>>>
>>>> - ret = cache_alloc(sb, ca);
>>>> + ret = cache_alloc(ca);
>>>
>>> I am not sure whether struct cache_b *sb will be used in future, I
>>> suggest to let it be for now.
>>>
>>
>> We have copied sb to ca->sb, so if we need sb info in future, we could get info from ca->sb.
>>
>
> Hi Yijing,
>
> Your point makes sense, please add
> Reviewed-by: Coly Li <colyli@suse.de>
> in your patch, if you don't mind :-)
>
> Thanks for your interpretation.
>
> Coly
Thanks for your review very much!
Thanks!
Yijing.
>
> .
>
^ permalink raw reply
* Re: [RESEND PATCH 1/3] bcache: Remove redundant parameter for cache_alloc()
From: Coly Li @ 2016-07-01 4:24 UTC (permalink / raw)
To: wangyijing, Coly Li, axboe, Kent Overstreet
Cc: Eric Wheeler, linux-bcache, linux-raid, linux-kernel
In-Reply-To: <5775D0E8.4060201@huawei.com>
在 16/7/1 上午10:09, wangyijing 写道:
>
>
> 在 2016/6/29 18:20, Coly Li 写道:
>> 在 16/6/22 上午10:10, Yijing Wang 写道:
>>> Cache_sb is not used in cache_alloc, and we have copied
>>> sb info to cache->sb already, remove it.
>>>
>>> Signed-off-by: Yijing Wang <wangyijing@huawei.com>
>>> ---
>>> drivers/md/bcache/super.c | 4 ++--
>>> 1 files changed, 2 insertions(+), 2 deletions(-)
>>>
>>> diff --git a/drivers/md/bcache/super.c b/drivers/md/bcache/super.c
>>> index f5dbb4e..aecaace 100644
>>> --- a/drivers/md/bcache/super.c
>>> +++ b/drivers/md/bcache/super.c
>>> @@ -1803,7 +1803,7 @@ void bch_cache_release(struct kobject *kobj)
>>> module_put(THIS_MODULE);
>>> }
>>>
>>> -static int cache_alloc(struct cache_sb *sb, struct cache *ca)
>>> +static int cache_alloc(struct cache *ca)
>>> {
>>> size_t free;
>>> struct bucket *b;
>>> @@ -1858,7 +1858,7 @@ static int register_cache(struct cache_sb *sb, struct page *sb_page,
>>> if (blk_queue_discard(bdev_get_queue(ca->bdev)))
>>> ca->discard = CACHE_DISCARD(&ca->sb);
>>>
>>> - ret = cache_alloc(sb, ca);
>>> + ret = cache_alloc(ca);
>>
>> I am not sure whether struct cache_b *sb will be used in future, I
>> suggest to let it be for now.
>>
>
> We have copied sb to ca->sb, so if we need sb info in future, we could get info from ca->sb.
>
Hi Yijing,
Your point makes sense, please add
Reviewed-by: Coly Li <colyli@suse.de>
in your patch, if you don't mind :-)
Thanks for your interpretation.
Coly
^ permalink raw reply
* Re: [RESEND PATCH 2/3] bcache: update document info
From: Coly Li @ 2016-07-01 4:21 UTC (permalink / raw)
To: wangyijing, Coly Li, axboe, Kent Overstreet
Cc: Eric Wheeler, linux-bcache, linux-raid, linux-kernel
In-Reply-To: <5775CCA4.9070805@huawei.com>
在 16/7/1 上午9:51, wangyijing 写道:
> Hi Coly, thanks to your review and comments.
>
> Commit 77b5a08427e875 ("bcache: don't embed 'return' statements in closure macros")
> remove the return in continue_at(), so I think we should update the document info
> about continue_at().
>
> Thanks!
> Yijing.
Hi Yijing,
The original version of continue_at() returns to caller function inside
the macro, Jens thinks this macro breaks code execution flow implicitly,
so he moves 'return' out of continue_at() and to follow continue_at() at
the location where continue_at() is referenced.
So as I suggested, the original author means the code should return to
the calling function.
But YES, I agree that the comments should be updated, because there is
no 'return' inside macro continue_at(). We should explicitly point out
that there should be a 'return' immediately following macro continue_at().
Thanks.
Coly
> 在 2016/6/29 18:16, Coly Li 写道:
>> 在 16/6/22 上午10:12, Yijing Wang 写道:
>>> There is no return in continue_at(), update the documentation.
>>>
>>
>> There are 2 modification of this patch. The first one is about a typo,
>> it is correct.
>>
>> But I doubt your second modification is proper. The line removed in your
>> patch is,
>>> - * continue_at() also, critically, is a macro that returns the
>> calling function.
>>> - * There's good reason for this.
>>> - *
>>
>> I think this is exactly what original author wants to say. It does not
>> mean return a value, it means return to the calling function. And the
>> bellowed lines explains the reason.
>>
>>> Signed-off-by: Yijing Wang <wangyijing@huawei.com>
>>> ---
>>> drivers/md/bcache/closure.c | 2 +-
>>> drivers/md/bcache/closure.h | 3 ---
>>> 2 files changed, 1 insertions(+), 4 deletions(-)
>>>
>>> diff --git a/drivers/md/bcache/closure.c b/drivers/md/bcache/closure.c
>>> index 9eaf1d6..864e673 100644
>>> --- a/drivers/md/bcache/closure.c
>>> +++ b/drivers/md/bcache/closure.c
>>> @@ -112,7 +112,7 @@ bool closure_wait(struct closure_waitlist *waitlist, struct closure *cl)
>>> EXPORT_SYMBOL(closure_wait);
>>>
>>> /**
>>> - * closure_sync - sleep until a closure a closure has nothing left to wait on
>>> + * closure_sync - sleep until a closure has nothing left to wait on
>>
>> Yes, this modification is good.
>>
>>> *
>>> * Sleeps until the refcount hits 1 - the thread that's running the closure owns
>>> * the last refcount.
>>> diff --git a/drivers/md/bcache/closure.h b/drivers/md/bcache/closure.h
>>> index 782cc2c..f51188d 100644
>>> --- a/drivers/md/bcache/closure.h
>>> +++ b/drivers/md/bcache/closure.h
>>> @@ -31,9 +31,6 @@
>>> * passing it, as you might expect, the function to run when nothing is pending
>>> * and the workqueue to run that function out of.
>>> *
>>> - * continue_at() also, critically, is a macro that returns the calling function.
>>> - * There's good reason for this.
>>> - *
>>> * To use safely closures asynchronously, they must always have a refcount while
>>> * they are running owned by the thread that is running them. Otherwise, suppose
>>> * you submit some bios and wish to have a function run when they all complete:
>>>
>>
>>
>
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: [RESEND PATCH 1/3] bcache: Remove redundant parameter for cache_alloc()
From: wangyijing @ 2016-07-01 2:09 UTC (permalink / raw)
To: Coly Li, axboe, Kent Overstreet
Cc: Eric Wheeler, Coly Li, linux-bcache, linux-raid, linux-kernel
In-Reply-To: <bb659bb1-b2ff-6b47-c47e-1b89fe9395b5@coly.li>
在 2016/6/29 18:20, Coly Li 写道:
> 在 16/6/22 上午10:10, Yijing Wang 写道:
>> Cache_sb is not used in cache_alloc, and we have copied
>> sb info to cache->sb already, remove it.
>>
>> Signed-off-by: Yijing Wang <wangyijing@huawei.com>
>> ---
>> drivers/md/bcache/super.c | 4 ++--
>> 1 files changed, 2 insertions(+), 2 deletions(-)
>>
>> diff --git a/drivers/md/bcache/super.c b/drivers/md/bcache/super.c
>> index f5dbb4e..aecaace 100644
>> --- a/drivers/md/bcache/super.c
>> +++ b/drivers/md/bcache/super.c
>> @@ -1803,7 +1803,7 @@ void bch_cache_release(struct kobject *kobj)
>> module_put(THIS_MODULE);
>> }
>>
>> -static int cache_alloc(struct cache_sb *sb, struct cache *ca)
>> +static int cache_alloc(struct cache *ca)
>> {
>> size_t free;
>> struct bucket *b;
>> @@ -1858,7 +1858,7 @@ static int register_cache(struct cache_sb *sb, struct page *sb_page,
>> if (blk_queue_discard(bdev_get_queue(ca->bdev)))
>> ca->discard = CACHE_DISCARD(&ca->sb);
>>
>> - ret = cache_alloc(sb, ca);
>> + ret = cache_alloc(ca);
>
> I am not sure whether struct cache_b *sb will be used in future, I
> suggest to let it be for now.
>
We have copied sb to ca->sb, so if we need sb info in future, we could get info from ca->sb.
Thanks!
Yijing.
>
>
>> if (ret != 0)
>> goto err;
>>
>>
>
>
^ permalink raw reply
* Re: [RESEND PATCH 2/3] bcache: update document info
From: wangyijing @ 2016-07-01 1:51 UTC (permalink / raw)
To: Coly Li, axboe, Kent Overstreet
Cc: Eric Wheeler, Coly Li, linux-bcache, linux-raid, linux-kernel
In-Reply-To: <b66c1795-3201-e571-75da-979893338d85@coly.li>
Hi Coly, thanks to your review and comments.
Commit 77b5a08427e875 ("bcache: don't embed 'return' statements in closure macros")
remove the return in continue_at(), so I think we should update the document info
about continue_at().
Thanks!
Yijing.
在 2016/6/29 18:16, Coly Li 写道:
> 在 16/6/22 上午10:12, Yijing Wang 写道:
>> There is no return in continue_at(), update the documentation.
>>
>
> There are 2 modification of this patch. The first one is about a typo,
> it is correct.
>
> But I doubt your second modification is proper. The line removed in your
> patch is,
>> - * continue_at() also, critically, is a macro that returns the
> calling function.
>> - * There's good reason for this.
>> - *
>
> I think this is exactly what original author wants to say. It does not
> mean return a value, it means return to the calling function. And the
> bellowed lines explains the reason.
>
>> Signed-off-by: Yijing Wang <wangyijing@huawei.com>
>> ---
>> drivers/md/bcache/closure.c | 2 +-
>> drivers/md/bcache/closure.h | 3 ---
>> 2 files changed, 1 insertions(+), 4 deletions(-)
>>
>> diff --git a/drivers/md/bcache/closure.c b/drivers/md/bcache/closure.c
>> index 9eaf1d6..864e673 100644
>> --- a/drivers/md/bcache/closure.c
>> +++ b/drivers/md/bcache/closure.c
>> @@ -112,7 +112,7 @@ bool closure_wait(struct closure_waitlist *waitlist, struct closure *cl)
>> EXPORT_SYMBOL(closure_wait);
>>
>> /**
>> - * closure_sync - sleep until a closure a closure has nothing left to wait on
>> + * closure_sync - sleep until a closure has nothing left to wait on
>
> Yes, this modification is good.
>
>> *
>> * Sleeps until the refcount hits 1 - the thread that's running the closure owns
>> * the last refcount.
>> diff --git a/drivers/md/bcache/closure.h b/drivers/md/bcache/closure.h
>> index 782cc2c..f51188d 100644
>> --- a/drivers/md/bcache/closure.h
>> +++ b/drivers/md/bcache/closure.h
>> @@ -31,9 +31,6 @@
>> * passing it, as you might expect, the function to run when nothing is pending
>> * and the workqueue to run that function out of.
>> *
>> - * continue_at() also, critically, is a macro that returns the calling function.
>> - * There's good reason for this.
>> - *
>> * To use safely closures asynchronously, they must always have a refcount while
>> * they are running owned by the thread that is running them. Otherwise, suppose
>> * you submit some bios and wish to have a function run when they all complete:
>>
>
>
^ permalink raw reply
* Re: [PATCH v2 00/12] gendisk: Generate uevent after attribute available
From: Dan Williams @ 2016-07-01 1:29 UTC (permalink / raw)
To: Fam Zheng
Cc: Christoph Hellwig, Linux Kernel Mailing List,
Benjamin Herrenschmidt, Paul Mackerras, Michael Ellerman,
Jens Axboe, Ed L. Cashin, Jiri Kosina, Michael S. Tsirkin,
Minchan Kim, Nitin Gupta, Sergey Senozhatsky, Shaohua Li,
Ulf Hansson, David Woodhouse, Brian Norris, Keith Busch,
linuxppc-dev, linux-block, virtualization, linux-raid,
linux-mmc@vger.kernel.org
In-Reply-To: <20160701010113.GB10122@ad.usersys.redhat.com>
On Thu, Jun 30, 2016 at 6:01 PM, Fam Zheng <famz@redhat.com> wrote:
> On Wed, 06/29 23:38, Christoph Hellwig wrote:
>> On Thu, Jun 30, 2016 at 02:35:54PM +0800, Fam Zheng wrote:
>> > also more code and less flexible IMO. For example, we need at least two
>> > variants, for attribute_group and device_attribute separately, right?
>>
>> Yes, or maybe just a calling convention that just passes both.
>
> OK, I can look into that, but I'm not sure about the error handling. Currently
> add_disk returns void, do you have any plan on that too? should I change it in
> v3 (to at least return the attribute creation failure)?
I think we should only support a "groups" interface to
device_add_disk() and convert all the drivers that currently do
device_create_file() after add_disk() to pass in a group list instead.
That way we follow the expectation that the only way to get an
attribute for a device to show up before KOBJ_ADD, is to define a
group:
From Documentation/driver-model/device.txt:
As explained in Documentation/kobject.txt, device attributes must be be
created before the KOBJ_ADD uevent is generated. The only way to realize
that is by defining an attribute group.
Let's defer the return value fixing for now.
You can find the pending device_add_disk() patches in the nvdimm
patchwork starting with "block: introduce device_add_disk()"
https://patchwork.kernel.org/project/linux-nvdimm/list/
^ permalink raw reply
* Re: [PATCH v2 04/12] axonrom: Generate uevent after attribute available
From: Fam Zheng @ 2016-07-01 1:03 UTC (permalink / raw)
To: Dan Williams
Cc: Sergey Senozhatsky, Michael S. Tsirkin, Benjamin Herrenschmidt,
linux-nvme, virtualization, Keith Busch, Paul Mackerras,
Michael Ellerman, Christoph Hellwig, Shaohua Li, Nitin Gupta,
Jiri Kosina, linux-block, Ed L. Cashin, Jens Axboe, linux-raid,
David Woodhouse, linux-mmc@vger.kernel.org,
Linux Kernel Mailing List, Minchan Kim, linux-mtd, Brian Norris
In-Reply-To: <CAA9_cmf0cKZXdQ2N3iSbwVqgLDGJ7TMvVYuZTqjVE-7EqHHcdg@mail.gmail.com>
On Thu, 06/30 15:10, Dan Williams wrote:
> On Wed, Jun 29, 2016 at 6:59 PM, Fam Zheng <famz@redhat.com> wrote:
> > It is documented that KOBJ_ADD should be generated after the object's
> > attributes and children are ready. We can achieve this with the new
> > disk_gen_uevents interface.
> >
> > Signed-off-by: Fam Zheng <famz@redhat.com>
> > ---
> > arch/powerpc/sysdev/axonram.c | 3 ++-
> > 1 file changed, 2 insertions(+), 1 deletion(-)
> >
> > diff --git a/arch/powerpc/sysdev/axonram.c b/arch/powerpc/sysdev/axonram.c
> > index 4efd69b..27e7175 100644
> > --- a/arch/powerpc/sysdev/axonram.c
> > +++ b/arch/powerpc/sysdev/axonram.c
> > @@ -238,7 +238,7 @@ static int axon_ram_probe(struct platform_device *device)
> > set_capacity(bank->disk, bank->size >> AXON_RAM_SECTOR_SHIFT);
> > blk_queue_make_request(bank->disk->queue, axon_ram_make_request);
> > blk_queue_logical_block_size(bank->disk->queue, AXON_RAM_SECTOR_SIZE);
> > - add_disk(bank->disk, true);
> > + add_disk(bank->disk, false);
> >
> > bank->irq_id = irq_of_parse_and_map(device->dev.of_node, 0);
> > if (bank->irq_id == NO_IRQ) {
> > @@ -262,6 +262,7 @@ static int axon_ram_probe(struct platform_device *device)
> > rc = -EFAULT;
> > goto failed;
> > }
> > + disk_gen_uevents(bank->disk);
>
> I assume you are doing this after:
>
> rc = device_create_file(&device->dev, &dev_attr_ecc);
>
> ...so that userspace gets notified of the new attribute, but this
> attribute is on the parent device, not the disk itself. Instead I
> think this attribute should simply be registered before the call to
> add_disk(). Then the KOBJ_ADD event for the disk comes after the
> attribute is available. It's still not a clean fit, because userspace
> should not be expecting a child device uevent to signal new attributes
> available on the parent.
Yes you are right, this patch is a mistake. Moving to before add_disk makes
sense to me.
Thanks for taking a look!
Fam
^ permalink raw reply
* Re: [PATCH v2 00/12] gendisk: Generate uevent after attribute available
From: Fam Zheng @ 2016-07-01 1:01 UTC (permalink / raw)
To: Christoph Hellwig
Cc: Sergey Senozhatsky, Michael S. Tsirkin, Benjamin Herrenschmidt,
linux-nvme, virtualization, Keith Busch, Paul Mackerras,
Michael Ellerman, Shaohua Li, Nitin Gupta, famz, Jiri Kosina,
linux-block, dan.j.williams, Ed L. Cashin, Jens Axboe, linux-raid,
David Woodhouse, linux-mmc, linux-kernel, Minchan Kim, linux-mtd,
Brian Norris, linuxppc-dev
In-Reply-To: <20160630063839.GA17205@infradead.org>
On Wed, 06/29 23:38, Christoph Hellwig wrote:
> On Thu, Jun 30, 2016 at 02:35:54PM +0800, Fam Zheng wrote:
> > also more code and less flexible IMO. For example, we need at least two
> > variants, for attribute_group and device_attribute separately, right?
>
> Yes, or maybe just a calling convention that just passes both.
OK, I can look into that, but I'm not sure about the error handling. Currently
add_disk returns void, do you have any plan on that too? should I change it in
v3 (to at least return the attribute creation failure)?
Fam
^ permalink raw reply
* Re: [PATCH v2 05/12] aoeblk: Generate uevent after attribute available
From: Ed Cashin @ 2016-07-01 0:57 UTC (permalink / raw)
To: Fam Zheng, linux-kernel
Cc: Ulf Hansson, Sergey Senozhatsky, Michael S. Tsirkin,
Benjamin Herrenschmidt, linux-nvme, virtualization, Keith Busch,
Paul Mackerras, Michael Ellerman, Christoph Hellwig, Shaohua Li,
Nitin Gupta, Jiri Kosina, linux-block, Jens Axboe, linux-raid,
David Woodhouse, linux-mmc, Minchan Kim, linux-mtd, Brian Norris,
linuxppc-dev
In-Reply-To: <20160630015953.6888-6-famz@redhat.com>
On 06/29/2016 09:59 PM, Fam Zheng wrote:
> It is documented that KOBJ_ADD should be generated after the object's
> attributes and children are ready. We can achieve this with the new
> disk_gen_uevents interface.
Looks like an improvement, thanks!
--
Ed
______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/
^ permalink raw reply
* Re: [PATCH v2 04/12] axonrom: Generate uevent after attribute available
From: Dan Williams @ 2016-06-30 22:10 UTC (permalink / raw)
To: Fam Zheng
Cc: Linux Kernel Mailing List, Benjamin Herrenschmidt, Paul Mackerras,
Michael Ellerman, Jens Axboe, Ed L. Cashin, Jiri Kosina,
Michael S. Tsirkin, Minchan Kim, Nitin Gupta, Sergey Senozhatsky,
Shaohua Li, Ulf Hansson, David Woodhouse, Brian Norris,
Keith Busch, linuxppc-dev, linux-block, virtualization,
linux-raid, linux-mmc@vger.kernel.org, linux-mtd
In-Reply-To: <20160630015953.6888-5-famz@redhat.com>
On Wed, Jun 29, 2016 at 6:59 PM, Fam Zheng <famz@redhat.com> wrote:
> It is documented that KOBJ_ADD should be generated after the object's
> attributes and children are ready. We can achieve this with the new
> disk_gen_uevents interface.
>
> Signed-off-by: Fam Zheng <famz@redhat.com>
> ---
> arch/powerpc/sysdev/axonram.c | 3 ++-
> 1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/arch/powerpc/sysdev/axonram.c b/arch/powerpc/sysdev/axonram.c
> index 4efd69b..27e7175 100644
> --- a/arch/powerpc/sysdev/axonram.c
> +++ b/arch/powerpc/sysdev/axonram.c
> @@ -238,7 +238,7 @@ static int axon_ram_probe(struct platform_device *device)
> set_capacity(bank->disk, bank->size >> AXON_RAM_SECTOR_SHIFT);
> blk_queue_make_request(bank->disk->queue, axon_ram_make_request);
> blk_queue_logical_block_size(bank->disk->queue, AXON_RAM_SECTOR_SIZE);
> - add_disk(bank->disk, true);
> + add_disk(bank->disk, false);
>
> bank->irq_id = irq_of_parse_and_map(device->dev.of_node, 0);
> if (bank->irq_id == NO_IRQ) {
> @@ -262,6 +262,7 @@ static int axon_ram_probe(struct platform_device *device)
> rc = -EFAULT;
> goto failed;
> }
> + disk_gen_uevents(bank->disk);
I assume you are doing this after:
rc = device_create_file(&device->dev, &dev_attr_ecc);
...so that userspace gets notified of the new attribute, but this
attribute is on the parent device, not the disk itself. Instead I
think this attribute should simply be registered before the call to
add_disk(). Then the KOBJ_ADD event for the disk comes after the
attribute is available. It's still not a clean fit, because userspace
should not be expecting a child device uevent to signal new attributes
available on the parent.
^ permalink raw reply
* Re: Assembly of RAID6 with 48 disk fails
From: Phil Turmel @ 2016-06-30 21:06 UTC (permalink / raw)
To: Grunewald, Soeren, linux-raid mailinglist
In-Reply-To: <1407524886.20708095.1467300814680.JavaMail.zimbra@desy.de>
On 06/30/2016 11:33 AM, Grunewald, Soeren wrote:
> Hi All,
>
> After a crash, probably caused by some raid issue, the array can't be
> assembled again. It looks like 8 disk were disappearing form the raid at
> the same time, which then lead to the system crash. Now I'm unable to
> get the array back to live again. Because the whole system was on the
> failing array, I can't access any additional information to check what
> happened. The system is a 8 year old SUN Fire X4540 system running under
> Ubuntu 12.04.5 (kernel-3.13.0-91 and mdadm-3.2.5-1ubuntu0.3), which is
> mainly used for data conversion (converting very large files from one
> format into another). The boot partition is placed on a 2GB ssd and the
> rest is running on the 48disk raid6 array. I have booted the system from
> a usb stick with clonezilla (kernel 4.4.x + mdadm 3.3) and started
> digging...
>
> Since this is the first time, that a I face such a large array and such
> an issue, I better follow the advice from the Linux-Raid-Wiki and
> request for help.
>
> As one can see in the attached log, the 'Events' count of 40 drives is
> 4391 and 4385 for the 8 others. We have the same picture for the
> 'State', 40 drives state 'clean' and 8 drives 'active'. So I tried
> 'mdadm --assemble --force ...'. This fixed the event count and faulty
> flag on 4 of the 8 disks. But still the array can't be created.
>
> Except one smart error (1 Currently unreadable (pending) sectors) do all
> other drives pass the extended smart test. I know, this does not mean
> that the drives are not damaged or broken.
>
> Anyway, how should I process to get the array working again?
>
You're the second person in the last month to run into this... There
was a bug in mdadm that makes it fail with --assemble --force when there
are several out-of-date devices.
The fix was to clone the mdadm git tree, compile the latest, and run
that stand-alone binary to perform the forced assembly. I haven't tried
to determine which released version has the fix.
Phil
^ permalink raw reply
* Assembly of RAID6 with 48 disk fails
From: Grunewald, Soeren @ 2016-06-30 15:33 UTC (permalink / raw)
To: linux-raid mailinglist
[-- Attachment #1: Type: text/plain, Size: 1595 bytes --]
Hi All,
After a crash, probably caused by some raid issue, the array can't be
assembled again. It looks like 8 disk were disappearing form the raid at
the same time, which then lead to the system crash. Now I'm unable to
get the array back to live again. Because the whole system was on the
failing array, I can't access any additional information to check what
happened. The system is a 8 year old SUN Fire X4540 system running under
Ubuntu 12.04.5 (kernel-3.13.0-91 and mdadm-3.2.5-1ubuntu0.3), which is
mainly used for data conversion (converting very large files from one
format into another). The boot partition is placed on a 2GB ssd and the
rest is running on the 48disk raid6 array. I have booted the system from
a usb stick with clonezilla (kernel 4.4.x + mdadm 3.3) and started
digging...
Since this is the first time, that a I face such a large array and such
an issue, I better follow the advice from the Linux-Raid-Wiki and
request for help.
As one can see in the attached log, the 'Events' count of 40 drives is
4391 and 4385 for the 8 others. We have the same picture for the
'State', 40 drives state 'clean' and 8 drives 'active'. So I tried
'mdadm --assemble --force ...'. This fixed the event count and faulty
flag on 4 of the 8 disks. But still the array can't be created.
Except one smart error (1 Currently unreadable (pending) sectors) do all
other drives pass the extended smart test. I know, this does not mean
that the drives are not damaged or broken.
Anyway, how should I process to get the array working again?
--
Thanks and regards,
Soeren
[-- Attachment #2: flashdaqsvr1-md-recovery.txt --]
[-- Type: text/plain, Size: 56086 bytes --]
user@flashdaqsvr1:~$ uname -a
Linux flashdaqsvr1 4.4.0-22-generic #40-Ubuntu SMP Thu May 12 22:03:46 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
user@flashdaqsvr1:~$ lspci | grep SCSI
02:00.0 SCSI storage controller: LSI Logic / Symbios Logic SAS1068E PCI-Express Fusion-MPT SAS (rev 04)
03:00.0 SCSI storage controller: LSI Logic / Symbios Logic SAS1068E PCI-Express Fusion-MPT SAS (rev 04)
04:00.0 SCSI storage controller: LSI Logic / Symbios Logic SAS1068E PCI-Express Fusion-MPT SAS (rev 04)
41:00.0 SCSI storage controller: LSI Logic / Symbios Logic SAS1068E PCI-Express Fusion-MPT SAS (rev 04)
42:00.0 SCSI storage controller: LSI Logic / Symbios Logic SAS1068E PCI-Express Fusion-MPT SAS (rev 04)
43:00.0 SCSI storage controller: LSI Logic / Symbios Logic SAS1068E PCI-Express Fusion-MPT SAS (rev 04)
user@flashdaqsvr1:~$ lspci -s '02:00.0' -vvvv
02:00.0 SCSI storage controller: LSI Logic / Symbios Logic SAS1068E PCI-Express Fusion-MPT SAS (rev 04)
Subsystem: LSI Logic / Symbios Logic SAS1068E PCI-Express Fusion-MPT SAS
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR- FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 64 bytes
Interrupt: pin A routed to IRQ 19
Region 0: I/O ports at b800 [size=256]
Region 1: Memory at df7fc000 (64-bit, non-prefetchable) [size=16K]
Region 3: Memory at df7e0000 (64-bit, non-prefetchable) [size=64K]
Expansion ROM at df400000 [disabled] [size=2M]
Capabilities: <access denied>
Kernel driver in use: mptsas
Kernel modules: mptsas
user@flashdaqsvr1:~$ mdadm --version
mdadm - v3.3 - 3rd September 2013
user@flashdaqsvr1:~$ cat mdadm.conf
ARRAY /dev/md/0 level=raid6 metadata=1.2 num-devices=47 UUID=9b5294f9:a73d4576:301ab56d:af23c26d name=flashdaqsvr1:0
spares=1 devices=/dev/sdax1,/dev/sdaw1,/dev/sdau1,/dev/sdav1,/dev/sdat1,/dev/sdar1,/dev/sdas1,/dev/sdaq1,/dev/sdap1,/dev/sdao1,/dev/sdam1,/dev/sdan1,/dev/sdal1,/dev/sdak1,/dev/sdaj1,/dev/sdai1,/dev/sdah1,/dev/sdaf1,/dev/sdag1,/dev/sdae1,/dev/sdad1,/dev/sdac1,/dev/sdab1,/dev/sdaa1,/dev/sdy1,/dev/sdz1,/dev/sdx1,/dev/sdw1,/dev/sdu1,/dev/sdv1,/dev/sdt1,/dev/sds1,/dev/sdq1,/dev/sdr1,/dev/sdp1,/dev/sdo1,/dev/sdn1,/dev/sdm1,/dev/sdk1,/dev/sdl1,/dev/sdi1,/dev/sdj1,/dev/sdg1,/dev/sdh1,/dev/sde1,/dev/sdf1,/dev/sdd1,/dev/sdc1
user@flashdaqsvr1:~$ sudo mdadm --examine /dev/sdc1 /dev/sdd1 /dev/sde1 /dev/sdf1 /dev/sdg1 /dev/sdh1 /dev/sdi1 /dev/sdj1 /dev/sdk1 /dev/sdl1 /dev/sdm1 /dev/sdn1 /dev/sdo1 /dev/sdp1 /dev/sdq1 /dev/sdr1 /dev/sds1 /dev/sdt1 /dev/sdu1 /dev/sdv1 /dev/sdw1 /dev/sdx1 /dev/sdy1 /dev/sdz1 /dev/sdaa1 /dev/sdab1 /dev/sdac1 /dev/sdad1 /dev/sdae1 /dev/sdaf1 /dev/sdag1 /dev/sdah1 /dev/sdai1 /dev/sdaj1 /dev/sdak1 /dev/sdal1 /dev/sdam1 /dev/sdan1 /dev/sdao1 /dev/sdap1 /dev/sdaq1 /dev/sdar1 /dev/sdas1 /dev/sdat1 /dev/sdau1 /dev/sdav1 /dev/sdaw1 /dev/sdax1
/dev/sdc1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 9b5294f9:a73d4576:301ab56d:af23c26d
Name : flashdaqsvr1:0 (local to host flashdaqsvr1)
Creation Time : Wed Aug 19 13:06:06 2015
Raid Level : raid6
Raid Devices : 47
Avail Dev Size : 1952186368 (930.88 GiB 999.52 GB)
Array Size : 43924170240 (41889.35 GiB 44978.35 GB)
Used Dev Size : 1952185344 (930.87 GiB 999.52 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Unused Space : before=262064 sectors, after=1024 sectors
State : clean
Device UUID : 9508f7a6:29c6051d:6d32cb05:804fb915
Update Time : Wed Jun 29 12:40:45 2016
Checksum : 81bd39e3 - correct
Events : 4391
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 11
Array State : AAAAAA..AAAAAAAAA..AAAAAAAAAA..AAAAAAAAAA..AAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdd1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 9b5294f9:a73d4576:301ab56d:af23c26d
Name : flashdaqsvr1:0 (local to host flashdaqsvr1)
Creation Time : Wed Aug 19 13:06:06 2015
Raid Level : raid6
Raid Devices : 47
Avail Dev Size : 1952186368 (930.88 GiB 999.52 GB)
Array Size : 43924170240 (41889.35 GiB 44978.35 GB)
Used Dev Size : 1952185344 (930.87 GiB 999.52 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Unused Space : before=262064 sectors, after=1024 sectors
State : clean
Device UUID : ebc3c9ba:d55fd8ee:fd085a43:b88bfef5
Update Time : Wed Jun 29 12:40:45 2016
Checksum : 8536a1bb - correct
Events : 4391
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 23
Array State : AAAAAA..AAAAAAAAA..AAAAAAAAAA..AAAAAAAAAA..AAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sde1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 9b5294f9:a73d4576:301ab56d:af23c26d
Name : flashdaqsvr1:0 (local to host flashdaqsvr1)
Creation Time : Wed Aug 19 13:06:06 2015
Raid Level : raid6
Raid Devices : 47
Avail Dev Size : 1952186368 (930.88 GiB 999.52 GB)
Array Size : 43924170240 (41889.35 GiB 44978.35 GB)
Used Dev Size : 1952185344 (930.87 GiB 999.52 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Unused Space : before=262064 sectors, after=1024 sectors
State : clean
Device UUID : b16e707c:e7865b2e:fc53747c:8dfcf2c0
Update Time : Wed Jun 29 12:40:45 2016
Checksum : 8a6f2f72 - correct
Events : 4391
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 35
Array State : AAAAAA..AAAAAAAAA..AAAAAAAAAA..AAAAAAAAAA..AAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdf1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 9b5294f9:a73d4576:301ab56d:af23c26d
Name : flashdaqsvr1:0 (local to host flashdaqsvr1)
Creation Time : Wed Aug 19 13:06:06 2015
Raid Level : raid6
Raid Devices : 47
Avail Dev Size : 1952186368 (930.88 GiB 999.52 GB)
Array Size : 43924170240 (41889.35 GiB 44978.35 GB)
Used Dev Size : 1952185344 (930.87 GiB 999.52 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Unused Space : before=262064 sectors, after=1024 sectors
State : clean
Device UUID : 1435b6f5:9274bdb6:353f17d4:4fcc3e31
Update Time : Wed Jun 29 12:40:45 2016
Checksum : 54059e59 - correct
Events : 4391
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 0
Array State : AAAAAA..AAAAAAAAA..AAAAAAAAAA..AAAAAAAAAA..AAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdg1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 9b5294f9:a73d4576:301ab56d:af23c26d
Name : flashdaqsvr1:0 (local to host flashdaqsvr1)
Creation Time : Wed Aug 19 13:06:06 2015
Raid Level : raid6
Raid Devices : 47
Avail Dev Size : 1952186368 (930.88 GiB 999.52 GB)
Array Size : 43924170240 (41889.35 GiB 44978.35 GB)
Used Dev Size : 1952185344 (930.87 GiB 999.52 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Unused Space : before=262064 sectors, after=1024 sectors
State : clean
Device UUID : 080ceba1:4679ae77:990323d0:ef7cd0bd
Update Time : Wed Jun 29 12:40:45 2016
Checksum : 49c8ef11 - correct
Events : 4391
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 12
Array State : AAAAAA..AAAAAAAAA..AAAAAAAAAA..AAAAAAAAAA..AAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdh1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 9b5294f9:a73d4576:301ab56d:af23c26d
Name : flashdaqsvr1:0 (local to host flashdaqsvr1)
Creation Time : Wed Aug 19 13:06:06 2015
Raid Level : raid6
Raid Devices : 47
Avail Dev Size : 1952186368 (930.88 GiB 999.52 GB)
Array Size : 43924170240 (41889.35 GiB 44978.35 GB)
Used Dev Size : 1952185344 (930.87 GiB 999.52 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Unused Space : before=262064 sectors, after=1024 sectors
State : clean
Device UUID : fa4206a0:7c1474fa:ea17f433:0f4501fe
Update Time : Wed Jun 29 12:40:45 2016
Checksum : 6eab9db6 - correct
Events : 4391
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 24
Array State : AAAAAA..AAAAAAAAA..AAAAAAAAAA..AAAAAAAAAA..AAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdi1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 9b5294f9:a73d4576:301ab56d:af23c26d
Name : flashdaqsvr1:0 (local to host flashdaqsvr1)
Creation Time : Wed Aug 19 13:06:06 2015
Raid Level : raid6
Raid Devices : 47
Avail Dev Size : 1952186368 (930.88 GiB 999.52 GB)
Array Size : 43924170240 (41889.35 GiB 44978.35 GB)
Used Dev Size : 1952185344 (930.87 GiB 999.52 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Unused Space : before=262064 sectors, after=1024 sectors
State : clean
Device UUID : 0cc73360:996fbd4c:9af2b019:c794a4f9
Update Time : Wed Jun 29 12:40:45 2016
Checksum : 6282a758 - correct
Events : 4391
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 36
Array State : AAAAAA..AAAAAAAAA..AAAAAAAAAA..AAAAAAAAAA..AAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdj1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 9b5294f9:a73d4576:301ab56d:af23c26d
Name : flashdaqsvr1:0 (local to host flashdaqsvr1)
Creation Time : Wed Aug 19 13:06:06 2015
Raid Level : raid6
Raid Devices : 47
Avail Dev Size : 1952186368 (930.88 GiB 999.52 GB)
Array Size : 43924170240 (41889.35 GiB 44978.35 GB)
Used Dev Size : 1952185344 (930.87 GiB 999.52 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Unused Space : before=262064 sectors, after=1024 sectors
State : clean
Device UUID : 721dbdd6:f0160888:240be7c0:20a8cd94
Update Time : Wed Jun 29 12:40:45 2016
Checksum : 56b5d0d6 - correct
Events : 4391
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 1
Array State : AAAAAA..AAAAAAAAA..AAAAAAAAAA..AAAAAAAAAA..AAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdk1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 9b5294f9:a73d4576:301ab56d:af23c26d
Name : flashdaqsvr1:0 (local to host flashdaqsvr1)
Creation Time : Wed Aug 19 13:06:06 2015
Raid Level : raid6
Raid Devices : 47
Avail Dev Size : 1952186368 (930.88 GiB 999.52 GB)
Array Size : 43924170240 (41889.35 GiB 44978.35 GB)
Used Dev Size : 1952185344 (930.87 GiB 999.52 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Unused Space : before=262064 sectors, after=1024 sectors
State : clean
Device UUID : 89cc9294:29bdd61f:b51fa68e:e26557fb
Update Time : Wed Jun 29 12:40:45 2016
Checksum : e0a2f884 - correct
Events : 4391
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 13
Array State : AAAAAA..AAAAAAAAA..AAAAAAAAAA..AAAAAAAAAA..AAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdl1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 9b5294f9:a73d4576:301ab56d:af23c26d
Name : flashdaqsvr1:0 (local to host flashdaqsvr1)
Creation Time : Wed Aug 19 13:06:06 2015
Raid Level : raid6
Raid Devices : 47
Avail Dev Size : 1952186368 (930.88 GiB 999.52 GB)
Array Size : 43924170240 (41889.35 GiB 44978.35 GB)
Used Dev Size : 1952185344 (930.87 GiB 999.52 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Unused Space : before=262064 sectors, after=1024 sectors
State : clean
Device UUID : e3ab47d2:792cbbed:8ff2680a:dc25ce6c
Update Time : Wed Jun 29 12:40:45 2016
Checksum : d975da0e - correct
Events : 4391
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 25
Array State : AAAAAA..AAAAAAAAA..AAAAAAAAAA..AAAAAAAAAA..AAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdm1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 9b5294f9:a73d4576:301ab56d:af23c26d
Name : flashdaqsvr1:0 (local to host flashdaqsvr1)
Creation Time : Wed Aug 19 13:06:06 2015
Raid Level : raid6
Raid Devices : 47
Avail Dev Size : 1952186368 (930.88 GiB 999.52 GB)
Array Size : 43924170240 (41889.35 GiB 44978.35 GB)
Used Dev Size : 1952185344 (930.87 GiB 999.52 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Unused Space : before=262064 sectors, after=1024 sectors
State : clean
Device UUID : 10a32d77:7e569ae1:8fcf5c26:fd815ae4
Update Time : Wed Jun 29 12:40:45 2016
Checksum : 5bb346e - correct
Events : 4391
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 37
Array State : AAAAAA..AAAAAAAAA..AAAAAAAAAA..AAAAAAAAAA..AAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdn1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 9b5294f9:a73d4576:301ab56d:af23c26d
Name : flashdaqsvr1:0 (local to host flashdaqsvr1)
Creation Time : Wed Aug 19 13:06:06 2015
Raid Level : raid6
Raid Devices : 47
Avail Dev Size : 1952186368 (930.88 GiB 999.52 GB)
Array Size : 43924170240 (41889.35 GiB 44978.35 GB)
Used Dev Size : 1952185344 (930.87 GiB 999.52 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Unused Space : before=262064 sectors, after=1024 sectors
State : clean
Device UUID : f68e8f68:7b067eb5:8c83a14e:0c0540cf
Update Time : Wed Jun 29 12:40:45 2016
Checksum : de2b0739 - correct
Events : 4391
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 2
Array State : AAAAAA..AAAAAAAAA..AAAAAAAAAA..AAAAAAAAAA..AAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdo1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 9b5294f9:a73d4576:301ab56d:af23c26d
Name : flashdaqsvr1:0 (local to host flashdaqsvr1)
Creation Time : Wed Aug 19 13:06:06 2015
Raid Level : raid6
Raid Devices : 47
Avail Dev Size : 1952186368 (930.88 GiB 999.52 GB)
Array Size : 43924170240 (41889.35 GiB 44978.35 GB)
Used Dev Size : 1952185344 (930.87 GiB 999.52 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Unused Space : before=262064 sectors, after=1024 sectors
State : clean
Device UUID : e3003a59:407e6a95:69a702a1:88c97df1
Update Time : Wed Jun 29 12:40:45 2016
Checksum : 2360d951 - correct
Events : 4391
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 14
Array State : AAAAAA..AAAAAAAAA..AAAAAAAAAA..AAAAAAAAAA..AAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdp1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 9b5294f9:a73d4576:301ab56d:af23c26d
Name : flashdaqsvr1:0 (local to host flashdaqsvr1)
Creation Time : Wed Aug 19 13:06:06 2015
Raid Level : raid6
Raid Devices : 47
Avail Dev Size : 1952186368 (930.88 GiB 999.52 GB)
Array Size : 43924170240 (41889.35 GiB 44978.35 GB)
Used Dev Size : 1952185344 (930.87 GiB 999.52 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Unused Space : before=262064 sectors, after=1024 sectors
State : clean
Device UUID : 0148ea90:cfe79ab4:854e469f:e47f0605
Update Time : Wed Jun 29 12:40:45 2016
Checksum : 8c0de781 - correct
Events : 4391
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 26
Array State : AAAAAA..AAAAAAAAA..AAAAAAAAAA..AAAAAAAAAA..AAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdq1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 9b5294f9:a73d4576:301ab56d:af23c26d
Name : flashdaqsvr1:0 (local to host flashdaqsvr1)
Creation Time : Wed Aug 19 13:06:06 2015
Raid Level : raid6
Raid Devices : 47
Avail Dev Size : 1952186368 (930.88 GiB 999.52 GB)
Array Size : 43924170240 (41889.35 GiB 44978.35 GB)
Used Dev Size : 1952185344 (930.87 GiB 999.52 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Unused Space : before=262064 sectors, after=1024 sectors
State : clean
Device UUID : 7f727449:05b1824c:8ebb0ec1:89710cf1
Update Time : Wed Jun 29 12:40:45 2016
Checksum : ea4e39ef - correct
Events : 4391
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 38
Array State : AAAAAA..AAAAAAAAA..AAAAAAAAAA..AAAAAAAAAA..AAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdr1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 9b5294f9:a73d4576:301ab56d:af23c26d
Name : flashdaqsvr1:0 (local to host flashdaqsvr1)
Creation Time : Wed Aug 19 13:06:06 2015
Raid Level : raid6
Raid Devices : 47
Avail Dev Size : 1952186368 (930.88 GiB 999.52 GB)
Array Size : 43924170240 (41889.35 GiB 44978.35 GB)
Used Dev Size : 1952185344 (930.87 GiB 999.52 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Unused Space : before=262064 sectors, after=1024 sectors
State : clean
Device UUID : a822d82e:9e04abf9:b83cbe1c:7d1cfc61
Update Time : Wed Jun 29 12:40:45 2016
Checksum : 497969ac - correct
Events : 4391
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 3
Array State : AAAAAA..AAAAAAAAA..AAAAAAAAAA..AAAAAAAAAA..AAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sds1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 9b5294f9:a73d4576:301ab56d:af23c26d
Name : flashdaqsvr1:0 (local to host flashdaqsvr1)
Creation Time : Wed Aug 19 13:06:06 2015
Raid Level : raid6
Raid Devices : 47
Avail Dev Size : 1952186368 (930.88 GiB 999.52 GB)
Array Size : 43924170240 (41889.35 GiB 44978.35 GB)
Used Dev Size : 1952185344 (930.87 GiB 999.52 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Unused Space : before=262064 sectors, after=1024 sectors
State : clean
Device UUID : 344e6cad:e71c7530:e4b73a21:9e80b87d
Update Time : Wed Jun 29 12:40:45 2016
Checksum : 1f108cda - correct
Events : 4391
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 15
Array State : AAAAAA..AAAAAAAAA..AAAAAAAAAA..AAAAAAAAAA..AAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdt1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 9b5294f9:a73d4576:301ab56d:af23c26d
Name : flashdaqsvr1:0 (local to host flashdaqsvr1)
Creation Time : Wed Aug 19 13:06:06 2015
Raid Level : raid6
Raid Devices : 47
Avail Dev Size : 1952186368 (930.88 GiB 999.52 GB)
Array Size : 43924170240 (41889.35 GiB 44978.35 GB)
Used Dev Size : 1952185344 (930.87 GiB 999.52 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Unused Space : before=262064 sectors, after=1024 sectors
State : clean
Device UUID : d5b1212e:8d53d328:afd861ea:d097d428
Update Time : Wed Jun 29 12:40:45 2016
Checksum : c675f2a - correct
Events : 4391
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 27
Array State : AAAAAA..AAAAAAAAA..AAAAAAAAAA..AAAAAAAAAA..AAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdu1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 9b5294f9:a73d4576:301ab56d:af23c26d
Name : flashdaqsvr1:0 (local to host flashdaqsvr1)
Creation Time : Wed Aug 19 13:06:06 2015
Raid Level : raid6
Raid Devices : 47
Avail Dev Size : 1952186368 (930.88 GiB 999.52 GB)
Array Size : 43924170240 (41889.35 GiB 44978.35 GB)
Used Dev Size : 1952185344 (930.87 GiB 999.52 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Unused Space : before=262064 sectors, after=1024 sectors
State : clean
Device UUID : d18d6fea:ed2cfc6d:59b0c5a0:8618566a
Update Time : Wed Jun 29 12:40:45 2016
Checksum : 5c36cf3 - correct
Events : 4391
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 39
Array State : AAAAAA..AAAAAAAAA..AAAAAAAAAA..AAAAAAAAAA..AAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdv1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 9b5294f9:a73d4576:301ab56d:af23c26d
Name : flashdaqsvr1:0 (local to host flashdaqsvr1)
Creation Time : Wed Aug 19 13:06:06 2015
Raid Level : raid6
Raid Devices : 47
Avail Dev Size : 1952186368 (930.88 GiB 999.52 GB)
Array Size : 43924170240 (41889.35 GiB 44978.35 GB)
Used Dev Size : 1952185344 (930.87 GiB 999.52 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Unused Space : before=262064 sectors, after=1024 sectors
State : clean
Device UUID : 2ffb0541:74b517d2:58841b77:2dba0fb3
Update Time : Wed Jun 29 12:40:45 2016
Checksum : df84d85a - correct
Events : 4391
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 4
Array State : AAAAAA..AAAAAAAAA..AAAAAAAAAA..AAAAAAAAAA..AAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdw1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 9b5294f9:a73d4576:301ab56d:af23c26d
Name : flashdaqsvr1:0 (local to host flashdaqsvr1)
Creation Time : Wed Aug 19 13:06:06 2015
Raid Level : raid6
Raid Devices : 47
Avail Dev Size : 1952186368 (930.88 GiB 999.52 GB)
Array Size : 43924170240 (41889.35 GiB 44978.35 GB)
Used Dev Size : 1952185344 (930.87 GiB 999.52 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Unused Space : before=262064 sectors, after=1024 sectors
State : clean
Device UUID : 3d5b5e68:9a367a8e:3c5a61a0:db732447
Update Time : Wed Jun 29 12:40:45 2016
Checksum : 809a492c - correct
Events : 4391
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 16
Array State : AAAAAA..AAAAAAAAA..AAAAAAAAAA..AAAAAAAAAA..AAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdx1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 9b5294f9:a73d4576:301ab56d:af23c26d
Name : flashdaqsvr1:0 (local to host flashdaqsvr1)
Creation Time : Wed Aug 19 13:06:06 2015
Raid Level : raid6
Raid Devices : 47
Avail Dev Size : 1952186368 (930.88 GiB 999.52 GB)
Array Size : 43924170240 (41889.35 GiB 44978.35 GB)
Used Dev Size : 1952185344 (930.87 GiB 999.52 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Unused Space : before=262064 sectors, after=1024 sectors
State : clean
Device UUID : c24f63e8:0ee12608:c5aa1f99:1d67b324
Update Time : Wed Jun 29 12:40:45 2016
Checksum : 50992bfc - correct
Events : 4391
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 28
Array State : AAAAAA..AAAAAAAAA..AAAAAAAAAA..AAAAAAAAAA..AAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdy1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 9b5294f9:a73d4576:301ab56d:af23c26d
Name : flashdaqsvr1:0 (local to host flashdaqsvr1)
Creation Time : Wed Aug 19 13:06:06 2015
Raid Level : raid6
Raid Devices : 47
Avail Dev Size : 1952186368 (930.88 GiB 999.52 GB)
Array Size : 43924170240 (41889.35 GiB 44978.35 GB)
Used Dev Size : 1952185344 (930.87 GiB 999.52 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Unused Space : before=262064 sectors, after=1024 sectors
State : clean
Device UUID : c91efeb9:7cc23027:5acc0e10:2cb46cae
Update Time : Wed Jun 29 12:40:45 2016
Checksum : 41e64b21 - correct
Events : 4391
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 40
Array State : AAAAAA..AAAAAAAAA..AAAAAAAAAA..AAAAAAAAAA..AAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdz1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 9b5294f9:a73d4576:301ab56d:af23c26d
Name : flashdaqsvr1:0 (local to host flashdaqsvr1)
Creation Time : Wed Aug 19 13:06:06 2015
Raid Level : raid6
Raid Devices : 47
Avail Dev Size : 1952186368 (930.88 GiB 999.52 GB)
Array Size : 43924170240 (41889.35 GiB 44978.35 GB)
Used Dev Size : 1952185344 (930.87 GiB 999.52 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Unused Space : before=262064 sectors, after=1024 sectors
State : clean
Device UUID : 5394ef42:3f50fc37:ef211747:00b4dd06
Update Time : Wed Jun 29 12:40:45 2016
Checksum : 6b1ca3b3 - correct
Events : 4391
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 5
Array State : AAAAAA..AAAAAAAAA..AAAAAAAAAA..AAAAAAAAAA..AAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdaa1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 9b5294f9:a73d4576:301ab56d:af23c26d
Name : flashdaqsvr1:0 (local to host flashdaqsvr1)
Creation Time : Wed Aug 19 13:06:06 2015
Raid Level : raid6
Raid Devices : 47
Avail Dev Size : 1952186368 (930.88 GiB 999.52 GB)
Array Size : 43924170240 (41889.35 GiB 44978.35 GB)
Used Dev Size : 1952185344 (930.87 GiB 999.52 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Unused Space : before=262064 sectors, after=1024 sectors
State : active
Device UUID : 7ce483aa:1a0c43f6:587f52cc:6801b9dd
Update Time : Wed Jun 29 12:40:12 2016
Checksum : ed705ad3 - correct
Events : 4385
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 17
Array State : AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdab1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 9b5294f9:a73d4576:301ab56d:af23c26d
Name : flashdaqsvr1:0 (local to host flashdaqsvr1)
Creation Time : Wed Aug 19 13:06:06 2015
Raid Level : raid6
Raid Devices : 47
Avail Dev Size : 1952186368 (930.88 GiB 999.52 GB)
Array Size : 43924170240 (41889.35 GiB 44978.35 GB)
Used Dev Size : 1952185344 (930.87 GiB 999.52 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Unused Space : before=262064 sectors, after=1024 sectors
State : active
Device UUID : 4935ac0e:ab67256b:720418af:ac15ee90
Update Time : Wed Jun 29 12:40:12 2016
Checksum : 5c75a09a - correct
Events : 4385
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 29
Array State : AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdac1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 9b5294f9:a73d4576:301ab56d:af23c26d
Name : flashdaqsvr1:0 (local to host flashdaqsvr1)
Creation Time : Wed Aug 19 13:06:06 2015
Raid Level : raid6
Raid Devices : 47
Avail Dev Size : 1952186368 (930.88 GiB 999.52 GB)
Array Size : 43924170240 (41889.35 GiB 44978.35 GB)
Used Dev Size : 1952185344 (930.87 GiB 999.52 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Unused Space : before=262064 sectors, after=1024 sectors
State : active
Device UUID : 3ef8ee26:344c9fbe:ec657996:ed634c32
Update Time : Wed Jun 29 12:40:12 2016
Checksum : 50f1f7df - correct
Events : 4385
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 41
Array State : AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdad1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 9b5294f9:a73d4576:301ab56d:af23c26d
Name : flashdaqsvr1:0 (local to host flashdaqsvr1)
Creation Time : Wed Aug 19 13:06:06 2015
Raid Level : raid6
Raid Devices : 47
Avail Dev Size : 1952186368 (930.88 GiB 999.52 GB)
Array Size : 43924170240 (41889.35 GiB 44978.35 GB)
Used Dev Size : 1952185344 (930.87 GiB 999.52 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Unused Space : before=262064 sectors, after=1024 sectors
State : active
Device UUID : 04be66f0:b5a64ae6:cbd6d6a4:12221d11
Update Time : Wed Jun 29 12:40:12 2016
Checksum : 2f434708 - correct
Events : 4385
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 6
Array State : AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdae1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 9b5294f9:a73d4576:301ab56d:af23c26d
Name : flashdaqsvr1:0 (local to host flashdaqsvr1)
Creation Time : Wed Aug 19 13:06:06 2015
Raid Level : raid6
Raid Devices : 47
Avail Dev Size : 1952186368 (930.88 GiB 999.52 GB)
Array Size : 43924170240 (41889.35 GiB 44978.35 GB)
Used Dev Size : 1952185344 (930.87 GiB 999.52 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Unused Space : before=262064 sectors, after=1024 sectors
State : active
Device UUID : 69ded561:3a8ce15d:5c12f354:9f24407b
Update Time : Wed Jun 29 12:40:12 2016
Checksum : 32888b1b - correct
Events : 4385
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 18
Array State : AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdaf1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 9b5294f9:a73d4576:301ab56d:af23c26d
Name : flashdaqsvr1:0 (local to host flashdaqsvr1)
Creation Time : Wed Aug 19 13:06:06 2015
Raid Level : raid6
Raid Devices : 47
Avail Dev Size : 1952186368 (930.88 GiB 999.52 GB)
Array Size : 43924170240 (41889.35 GiB 44978.35 GB)
Used Dev Size : 1952185344 (930.87 GiB 999.52 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Unused Space : before=262064 sectors, after=1024 sectors
State : active
Device UUID : 8fccfecf:d732f70f:3b74565a:67d351d3
Update Time : Wed Jun 29 12:40:12 2016
Checksum : b03c3091 - correct
Events : 4385
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 30
Array State : AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdag1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 9b5294f9:a73d4576:301ab56d:af23c26d
Name : flashdaqsvr1:0 (local to host flashdaqsvr1)
Creation Time : Wed Aug 19 13:06:06 2015
Raid Level : raid6
Raid Devices : 47
Avail Dev Size : 1952186368 (930.88 GiB 999.52 GB)
Array Size : 43924170240 (41889.35 GiB 44978.35 GB)
Used Dev Size : 1952185344 (930.87 GiB 999.52 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Unused Space : before=262064 sectors, after=1024 sectors
State : active
Device UUID : 1d14348b:cb328952:3fb558b9:10dfbc0d
Update Time : Wed Jun 29 12:40:12 2016
Checksum : 4770c4cc - correct
Events : 4385
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 42
Array State : AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdah1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 9b5294f9:a73d4576:301ab56d:af23c26d
Name : flashdaqsvr1:0 (local to host flashdaqsvr1)
Creation Time : Wed Aug 19 13:06:06 2015
Raid Level : raid6
Raid Devices : 47
Avail Dev Size : 1952186368 (930.88 GiB 999.52 GB)
Array Size : 43924170240 (41889.35 GiB 44978.35 GB)
Used Dev Size : 1952185344 (930.87 GiB 999.52 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Unused Space : before=262064 sectors, after=1024 sectors
State : active
Device UUID : 6edecf6f:8ba7caff:b9257ae4:0861f7b1
Update Time : Wed Jun 29 12:40:12 2016
Checksum : a8a9f6fd - correct
Events : 4385
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 7
Array State : AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdai1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 9b5294f9:a73d4576:301ab56d:af23c26d
Name : flashdaqsvr1:0 (local to host flashdaqsvr1)
Creation Time : Wed Aug 19 13:06:06 2015
Raid Level : raid6
Raid Devices : 47
Avail Dev Size : 1952186368 (930.88 GiB 999.52 GB)
Array Size : 43924170240 (41889.35 GiB 44978.35 GB)
Used Dev Size : 1952185344 (930.87 GiB 999.52 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Unused Space : before=262064 sectors, after=1024 sectors
State : clean
Device UUID : 4b7c0175:40aedae5:7179c5e0:2c868606
Update Time : Wed Jun 29 12:40:45 2016
Checksum : e4641369 - correct
Events : 4391
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 19
Array State : AAAAAA..AAAAAAAAA..AAAAAAAAAA..AAAAAAAAAA..AAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdaj1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 9b5294f9:a73d4576:301ab56d:af23c26d
Name : flashdaqsvr1:0 (local to host flashdaqsvr1)
Creation Time : Wed Aug 19 13:06:06 2015
Raid Level : raid6
Raid Devices : 47
Avail Dev Size : 1952186368 (930.88 GiB 999.52 GB)
Array Size : 43924170240 (41889.35 GiB 44978.35 GB)
Used Dev Size : 1952185344 (930.87 GiB 999.52 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Unused Space : before=262064 sectors, after=1024 sectors
State : clean
Device UUID : 936f6d2d:000f5a85:712eee6e:e2607a10
Update Time : Wed Jun 29 12:40:45 2016
Checksum : d46bf732 - correct
Events : 4391
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 31
Array State : AAAAAA..AAAAAAAAA..AAAAAAAAAA..AAAAAAAAAA..AAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdak1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 9b5294f9:a73d4576:301ab56d:af23c26d
Name : flashdaqsvr1:0 (local to host flashdaqsvr1)
Creation Time : Wed Aug 19 13:06:06 2015
Raid Level : raid6
Raid Devices : 47
Avail Dev Size : 1952186368 (930.88 GiB 999.52 GB)
Array Size : 43924170240 (41889.35 GiB 44978.35 GB)
Used Dev Size : 1952185344 (930.87 GiB 999.52 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Unused Space : before=262064 sectors, after=1024 sectors
State : clean
Device UUID : 61bd6e77:40164fbb:2fbcf802:0326ae71
Update Time : Wed Jun 29 12:40:45 2016
Checksum : 49a09f2c - correct
Events : 4391
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 43
Array State : AAAAAA..AAAAAAAAA..AAAAAAAAAA..AAAAAAAAAA..AAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdal1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 9b5294f9:a73d4576:301ab56d:af23c26d
Name : flashdaqsvr1:0 (local to host flashdaqsvr1)
Creation Time : Wed Aug 19 13:06:06 2015
Raid Level : raid6
Raid Devices : 47
Avail Dev Size : 1952186368 (930.88 GiB 999.52 GB)
Array Size : 43924170240 (41889.35 GiB 44978.35 GB)
Used Dev Size : 1952185344 (930.87 GiB 999.52 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Unused Space : before=262064 sectors, after=1024 sectors
State : clean
Device UUID : 91fdf97e:fcfe2c7d:7c3203d4:65099c00
Update Time : Wed Jun 29 12:40:45 2016
Checksum : 730221a4 - correct
Events : 4391
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 8
Array State : AAAAAA..AAAAAAAAA..AAAAAAAAAA..AAAAAAAAAA..AAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdam1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 9b5294f9:a73d4576:301ab56d:af23c26d
Name : flashdaqsvr1:0 (local to host flashdaqsvr1)
Creation Time : Wed Aug 19 13:06:06 2015
Raid Level : raid6
Raid Devices : 47
Avail Dev Size : 1952186368 (930.88 GiB 999.52 GB)
Array Size : 43924170240 (41889.35 GiB 44978.35 GB)
Used Dev Size : 1952185344 (930.87 GiB 999.52 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Unused Space : before=262064 sectors, after=1024 sectors
State : clean
Device UUID : ce720042:1015321e:f235f593:2b1d6f5f
Update Time : Wed Jun 29 12:40:45 2016
Checksum : f5d2c43c - correct
Events : 4391
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 20
Array State : AAAAAA..AAAAAAAAA..AAAAAAAAAA..AAAAAAAAAA..AAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdan1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 9b5294f9:a73d4576:301ab56d:af23c26d
Name : flashdaqsvr1:0 (local to host flashdaqsvr1)
Creation Time : Wed Aug 19 13:06:06 2015
Raid Level : raid6
Raid Devices : 47
Avail Dev Size : 1952186368 (930.88 GiB 999.52 GB)
Array Size : 43924170240 (41889.35 GiB 44978.35 GB)
Used Dev Size : 1952185344 (930.87 GiB 999.52 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Unused Space : before=262064 sectors, after=1024 sectors
State : clean
Device UUID : 5686c434:bbfa820a:1ca35c82:82b6fd68
Update Time : Wed Jun 29 12:40:45 2016
Checksum : ccddc3fc - correct
Events : 4391
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 32
Array State : AAAAAA..AAAAAAAAA..AAAAAAAAAA..AAAAAAAAAA..AAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdao1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 9b5294f9:a73d4576:301ab56d:af23c26d
Name : flashdaqsvr1:0 (local to host flashdaqsvr1)
Creation Time : Wed Aug 19 13:06:06 2015
Raid Level : raid6
Raid Devices : 47
Avail Dev Size : 1952186368 (930.88 GiB 999.52 GB)
Array Size : 43924170240 (41889.35 GiB 44978.35 GB)
Used Dev Size : 1952185344 (930.87 GiB 999.52 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Unused Space : before=262064 sectors, after=1024 sectors
State : clean
Device UUID : f82b9076:a49f0cbc:eb2f81d1:1ce3ca47
Update Time : Wed Jun 29 12:40:45 2016
Checksum : ee24c7fd - correct
Events : 4391
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 44
Array State : AAAAAA..AAAAAAAAA..AAAAAAAAAA..AAAAAAAAAA..AAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdap1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 9b5294f9:a73d4576:301ab56d:af23c26d
Name : flashdaqsvr1:0 (local to host flashdaqsvr1)
Creation Time : Wed Aug 19 13:06:06 2015
Raid Level : raid6
Raid Devices : 47
Avail Dev Size : 1952186368 (930.88 GiB 999.52 GB)
Array Size : 43924170240 (41889.35 GiB 44978.35 GB)
Used Dev Size : 1952185344 (930.87 GiB 999.52 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Unused Space : before=262064 sectors, after=1024 sectors
State : clean
Device UUID : cb756108:807a30a6:a4b786f3:3b48c6a3
Update Time : Wed Jun 29 12:40:45 2016
Checksum : e81ad961 - correct
Events : 4391
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 9
Array State : AAAAAA..AAAAAAAAA..AAAAAAAAAA..AAAAAAAAAA..AAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdaq1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 9b5294f9:a73d4576:301ab56d:af23c26d
Name : flashdaqsvr1:0 (local to host flashdaqsvr1)
Creation Time : Wed Aug 19 13:06:06 2015
Raid Level : raid6
Raid Devices : 47
Avail Dev Size : 1952186368 (930.88 GiB 999.52 GB)
Array Size : 43924170240 (41889.35 GiB 44978.35 GB)
Used Dev Size : 1952185344 (930.87 GiB 999.52 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Unused Space : before=262064 sectors, after=1024 sectors
State : clean
Device UUID : 3ad8f9c2:41d45cf1:d54898d0:93e2ed32
Update Time : Wed Jun 29 12:40:45 2016
Checksum : 5a18c127 - correct
Events : 4391
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 21
Array State : AAAAAA..AAAAAAAAA..AAAAAAAAAA..AAAAAAAAAA..AAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdar1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 9b5294f9:a73d4576:301ab56d:af23c26d
Name : flashdaqsvr1:0 (local to host flashdaqsvr1)
Creation Time : Wed Aug 19 13:06:06 2015
Raid Level : raid6
Raid Devices : 47
Avail Dev Size : 1952186368 (930.88 GiB 999.52 GB)
Array Size : 43924170240 (41889.35 GiB 44978.35 GB)
Used Dev Size : 1952185344 (930.87 GiB 999.52 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Unused Space : before=262064 sectors, after=1024 sectors
State : clean
Device UUID : 0a2d2823:cc58caf8:55e29aba:6128b494
Update Time : Wed Jun 29 12:40:45 2016
Checksum : d7d79dc - correct
Events : 4391
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 33
Array State : AAAAAA..AAAAAAAAA..AAAAAAAAAA..AAAAAAAAAA..AAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdas1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 9b5294f9:a73d4576:301ab56d:af23c26d
Name : flashdaqsvr1:0 (local to host flashdaqsvr1)
Creation Time : Wed Aug 19 13:06:06 2015
Raid Level : raid6
Raid Devices : 47
Avail Dev Size : 1952186368 (930.88 GiB 999.52 GB)
Array Size : 43924170240 (41889.35 GiB 44978.35 GB)
Used Dev Size : 1952185344 (930.87 GiB 999.52 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Unused Space : before=262064 sectors, after=1024 sectors
State : clean
Device UUID : b208b8c0:e76f210c:a91e7b4c:bf40ee6d
Update Time : Wed Jun 29 12:40:45 2016
Checksum : 297ec15c - correct
Events : 4391
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 45
Array State : AAAAAA..AAAAAAAAA..AAAAAAAAAA..AAAAAAAAAA..AAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdat1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 9b5294f9:a73d4576:301ab56d:af23c26d
Name : flashdaqsvr1:0 (local to host flashdaqsvr1)
Creation Time : Wed Aug 19 13:06:06 2015
Raid Level : raid6
Raid Devices : 47
Avail Dev Size : 1952186368 (930.88 GiB 999.52 GB)
Array Size : 43924170240 (41889.35 GiB 44978.35 GB)
Used Dev Size : 1952185344 (930.87 GiB 999.52 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Unused Space : before=262064 sectors, after=1024 sectors
State : clean
Device UUID : 0318e575:036b453c:d50db9aa:0854fb1f
Update Time : Wed Jun 29 12:40:45 2016
Checksum : 1f1ace1b - correct
Events : 4391
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 10
Array State : AAAAAA..AAAAAAAAA..AAAAAAAAAA..AAAAAAAAAA..AAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdau1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 9b5294f9:a73d4576:301ab56d:af23c26d
Name : flashdaqsvr1:0 (local to host flashdaqsvr1)
Creation Time : Wed Aug 19 13:06:06 2015
Raid Level : raid6
Raid Devices : 47
Avail Dev Size : 1952186368 (930.88 GiB 999.52 GB)
Array Size : 43924170240 (41889.35 GiB 44978.35 GB)
Used Dev Size : 1952185344 (930.87 GiB 999.52 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Unused Space : before=262064 sectors, after=1024 sectors
State : clean
Device UUID : fed64cdb:a9dc3590:64790b84:44ca46a0
Update Time : Wed Jun 29 12:40:45 2016
Checksum : 3210e094 - correct
Events : 4391
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 22
Array State : AAAAAA..AAAAAAAAA..AAAAAAAAAA..AAAAAAAAAA..AAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdav1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 9b5294f9:a73d4576:301ab56d:af23c26d
Name : flashdaqsvr1:0 (local to host flashdaqsvr1)
Creation Time : Wed Aug 19 13:06:06 2015
Raid Level : raid6
Raid Devices : 47
Avail Dev Size : 1952186368 (930.88 GiB 999.52 GB)
Array Size : 43924170240 (41889.35 GiB 44978.35 GB)
Used Dev Size : 1952185344 (930.87 GiB 999.52 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Unused Space : before=262064 sectors, after=1024 sectors
State : clean
Device UUID : deb5d15a:1ec3ac10:90cff353:cd412c08
Update Time : Wed Jun 29 12:40:45 2016
Checksum : 69da73a8 - correct
Events : 4391
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 34
Array State : AAAAAA..AAAAAAAAA..AAAAAAAAAA..AAAAAAAAAA..AAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdaw1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 9b5294f9:a73d4576:301ab56d:af23c26d
Name : flashdaqsvr1:0 (local to host flashdaqsvr1)
Creation Time : Wed Aug 19 13:06:06 2015
Raid Level : raid6
Raid Devices : 47
Avail Dev Size : 1952186368 (930.88 GiB 999.52 GB)
Array Size : 43924170240 (41889.35 GiB 44978.35 GB)
Used Dev Size : 1952185344 (930.87 GiB 999.52 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Unused Space : before=262064 sectors, after=1024 sectors
State : clean
Device UUID : 32ce415e:b11e7100:1ed7e7fb:d5c2d398
Update Time : Wed Jun 29 12:40:45 2016
Checksum : 95aa7032 - correct
Events : 4391
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 46
Array State : AAAAAA..AAAAAAAAA..AAAAAAAAAA..AAAAAAAAAA..AAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdax1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 9b5294f9:a73d4576:301ab56d:af23c26d
Name : flashdaqsvr1:0 (local to host flashdaqsvr1)
Creation Time : Wed Aug 19 13:06:06 2015
Raid Level : raid6
Raid Devices : 47
Avail Dev Size : 1952186368 (930.88 GiB 999.52 GB)
Array Size : 43924170240 (41889.35 GiB 44978.35 GB)
Used Dev Size : 1952185344 (930.87 GiB 999.52 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Unused Space : before=262064 sectors, after=1024 sectors
State : clean
Device UUID : ae6b229f:9501ad27:0ddb9259:f901670a
Update Time : Wed Jun 29 12:40:45 2016
Checksum : cd0533a5 - correct
Events : 4391
Layout : left-symmetric
Chunk Size : 512K
Device Role : spare
Array State : AAAAAA..AAAAAAAAA..AAAAAAAAAA..AAAAAAAAAA..AAAA ('A' == active, '.' == missing, 'R' == replacing)
user@flashdaqsvr1:~$ sudo mdadm --assemble --force /dev/md0 /dev/sdc1 /dev/sdd1 /dev/sde1 /dev/sdf1 /dev/sdg1 /dev/sdh1 /dev/sdi1 /dev/sdj1 /dev/sdk1 /dev/sdl1 /dev/sdm1 /dev/sdn1 /dev/sdo1 /dev/sdp1 /dev/sdq1 /dev/sdr1 /dev/sds1 /dev/sdt1 /dev/sdu1 /dev/sdv1 /dev/sdw1 /dev/sdx1 /dev/sdy1 /dev/sdz1 /dev/sdaa1 /dev/sdab1 /dev/sdac1 /dev/sdad1 /dev/sdae1 /dev/sdaf1 /dev/sdag1 /dev/sdah1 /dev/sdai1 /dev/sdaj1 /dev/sdak1 /dev/sdal1 /dev/sdam1 /dev/sdan1 /dev/sdao1 /dev/sdap1 /dev/sdaq1 /dev/sdar1 /dev/sdas1 /dev/sdat1 /dev/sdau1 /dev/sdav1 /dev/sdaw1 /dev/sdax1
mdadm: forcing event count in /dev/sdad1(6) from 4385 upto 4391
mdadm: forcing event count in /dev/sdah1(7) from 4385 upto 4391
mdadm: forcing event count in /dev/sdaa1(17) from 4385 upto 4391
mdadm: forcing event count in /dev/sdae1(18) from 4385 upto 4391
mdadm: clearing FAULTY flag for device 27 in /dev/md0 for /dev/sdad1
mdadm: clearing FAULTY flag for device 31 in /dev/md0 for /dev/sdah1
mdadm: clearing FAULTY flag for device 24 in /dev/md0 for /dev/sdaa1
mdadm: clearing FAULTY flag for device 28 in /dev/md0 for /dev/sdae1
mdadm: Marking array /dev/md0 as 'clean'
mdadm: /dev/md0 assembled from 43 drives and 1 spare - not enough to start the array.
user@flashdaqsvr1:~$ sudo mdadm --assemble --verbose /dev/md0 /dev/sdc1 /dev/sdd1 /dev/sde1 /dev/sdf1 /dev/sdg1 /dev/sdh1 /dev/sdi1 /dev/sdj1 /dev/sdk1 /dev/sdl1 /dev/sdm1 /dev/sdn1 /dev/sdo1 /dev/sdp1 /dev/sdq1 /dev/sdr1 /dev/sds1 /dev/sdt1 /dev/sdu1 /dev/sdv1 /dev/sdw1 /dev/sdx1 /dev/sdy1 /dev/sdz1 /dev/sdaa1 /dev/sdab1 /dev/sdac1 /dev/sdad1 /dev/sdae1 /dev/sdaf1 /dev/sdag1 /dev/sdah1 /dev/sdai1 /dev/sdaj1 /dev/sdak1 /dev/sdal1 /dev/sdam1 /dev/sdan1 /dev/sdao1 /dev/sdap1 /dev/sdaq1 /dev/sdar1 /dev/sdas1 /dev/sdat1 /dev/sdau1 /dev/sdav1 /dev/sdaw1 /dev/sdax1
mdadm: looking for devices for /dev/md0
mdadm: /dev/sdc1 is identified as a member of /dev/md0, slot 11.
mdadm: /dev/sdd1 is identified as a member of /dev/md0, slot 23.
mdadm: /dev/sde1 is identified as a member of /dev/md0, slot 35.
mdadm: /dev/sdf1 is identified as a member of /dev/md0, slot 0.
mdadm: /dev/sdg1 is identified as a member of /dev/md0, slot 12.
mdadm: /dev/sdh1 is identified as a member of /dev/md0, slot 24.
mdadm: /dev/sdi1 is identified as a member of /dev/md0, slot 36.
mdadm: /dev/sdj1 is identified as a member of /dev/md0, slot 1.
mdadm: /dev/sdk1 is identified as a member of /dev/md0, slot 13.
mdadm: /dev/sdl1 is identified as a member of /dev/md0, slot 25.
mdadm: /dev/sdm1 is identified as a member of /dev/md0, slot 37.
mdadm: /dev/sdn1 is identified as a member of /dev/md0, slot 2.
mdadm: /dev/sdo1 is identified as a member of /dev/md0, slot 14.
mdadm: /dev/sdp1 is identified as a member of /dev/md0, slot 26.
mdadm: /dev/sdq1 is identified as a member of /dev/md0, slot 38.
mdadm: /dev/sdr1 is identified as a member of /dev/md0, slot 3.
mdadm: /dev/sds1 is identified as a member of /dev/md0, slot 15.
mdadm: /dev/sdt1 is identified as a member of /dev/md0, slot 27.
mdadm: /dev/sdu1 is identified as a member of /dev/md0, slot 39.
mdadm: /dev/sdv1 is identified as a member of /dev/md0, slot 4.
mdadm: /dev/sdw1 is identified as a member of /dev/md0, slot 16.
mdadm: /dev/sdx1 is identified as a member of /dev/md0, slot 28.
mdadm: /dev/sdy1 is identified as a member of /dev/md0, slot 40.
mdadm: /dev/sdz1 is identified as a member of /dev/md0, slot 5.
mdadm: /dev/sdaa1 is identified as a member of /dev/md0, slot 17.
mdadm: /dev/sdab1 is identified as a member of /dev/md0, slot 29.
mdadm: /dev/sdac1 is identified as a member of /dev/md0, slot 41.
mdadm: /dev/sdad1 is identified as a member of /dev/md0, slot 6.
mdadm: /dev/sdae1 is identified as a member of /dev/md0, slot 18.
mdadm: /dev/sdaf1 is identified as a member of /dev/md0, slot 30.
mdadm: /dev/sdag1 is identified as a member of /dev/md0, slot 42.
mdadm: /dev/sdah1 is identified as a member of /dev/md0, slot 7.
mdadm: /dev/sdai1 is identified as a member of /dev/md0, slot 19.
mdadm: /dev/sdaj1 is identified as a member of /dev/md0, slot 31.
mdadm: /dev/sdak1 is identified as a member of /dev/md0, slot 43.
mdadm: /dev/sdal1 is identified as a member of /dev/md0, slot 8.
mdadm: /dev/sdam1 is identified as a member of /dev/md0, slot 20.
mdadm: /dev/sdan1 is identified as a member of /dev/md0, slot 32.
mdadm: /dev/sdao1 is identified as a member of /dev/md0, slot 44.
mdadm: /dev/sdap1 is identified as a member of /dev/md0, slot 9.
mdadm: /dev/sdaq1 is identified as a member of /dev/md0, slot 21.
mdadm: /dev/sdar1 is identified as a member of /dev/md0, slot 33.
mdadm: /dev/sdas1 is identified as a member of /dev/md0, slot 45.
mdadm: /dev/sdat1 is identified as a member of /dev/md0, slot 10.
mdadm: /dev/sdau1 is identified as a member of /dev/md0, slot 22.
mdadm: /dev/sdav1 is identified as a member of /dev/md0, slot 34.
mdadm: /dev/sdaw1 is identified as a member of /dev/md0, slot 46.
mdadm: /dev/sdax1 is identified as a member of /dev/md0, slot -1.
mdadm: added /dev/sdj1 to /dev/md0 as 1
mdadm: added /dev/sdn1 to /dev/md0 as 2
mdadm: added /dev/sdr1 to /dev/md0 as 3
mdadm: added /dev/sdv1 to /dev/md0 as 4
mdadm: added /dev/sdz1 to /dev/md0 as 5
mdadm: added /dev/sdad1 to /dev/md0 as 6
mdadm: added /dev/sdah1 to /dev/md0 as 7
mdadm: added /dev/sdal1 to /dev/md0 as 8
mdadm: added /dev/sdap1 to /dev/md0 as 9
mdadm: added /dev/sdat1 to /dev/md0 as 10
mdadm: added /dev/sdc1 to /dev/md0 as 11
mdadm: added /dev/sdg1 to /dev/md0 as 12
mdadm: added /dev/sdk1 to /dev/md0 as 13
mdadm: added /dev/sdo1 to /dev/md0 as 14
mdadm: added /dev/sds1 to /dev/md0 as 15
mdadm: added /dev/sdw1 to /dev/md0 as 16
mdadm: added /dev/sdaa1 to /dev/md0 as 17
mdadm: added /dev/sdae1 to /dev/md0 as 18
mdadm: added /dev/sdai1 to /dev/md0 as 19
mdadm: added /dev/sdam1 to /dev/md0 as 20
mdadm: added /dev/sdaq1 to /dev/md0 as 21
mdadm: added /dev/sdau1 to /dev/md0 as 22
mdadm: added /dev/sdd1 to /dev/md0 as 23
mdadm: added /dev/sdh1 to /dev/md0 as 24
mdadm: added /dev/sdl1 to /dev/md0 as 25
mdadm: added /dev/sdp1 to /dev/md0 as 26
mdadm: added /dev/sdt1 to /dev/md0 as 27
mdadm: added /dev/sdx1 to /dev/md0 as 28
mdadm: added /dev/sdab1 to /dev/md0 as 29 (possibly out of date)
mdadm: added /dev/sdaf1 to /dev/md0 as 30 (possibly out of date)
mdadm: added /dev/sdaj1 to /dev/md0 as 31
mdadm: added /dev/sdan1 to /dev/md0 as 32
mdadm: added /dev/sdar1 to /dev/md0 as 33
mdadm: added /dev/sdav1 to /dev/md0 as 34
mdadm: added /dev/sde1 to /dev/md0 as 35
mdadm: added /dev/sdi1 to /dev/md0 as 36
mdadm: added /dev/sdm1 to /dev/md0 as 37
mdadm: added /dev/sdq1 to /dev/md0 as 38
mdadm: added /dev/sdu1 to /dev/md0 as 39
mdadm: added /dev/sdy1 to /dev/md0 as 40
mdadm: added /dev/sdac1 to /dev/md0 as 41 (possibly out of date)
mdadm: added /dev/sdag1 to /dev/md0 as 42 (possibly out of date)
mdadm: added /dev/sdak1 to /dev/md0 as 43
mdadm: added /dev/sdao1 to /dev/md0 as 44
mdadm: added /dev/sdas1 to /dev/md0 as 45
mdadm: added /dev/sdaw1 to /dev/md0 as 46
mdadm: added /dev/sdax1 to /dev/md0 as -1
mdadm: added /dev/sdf1 to /dev/md0 as 0
mdadm: /dev/md0 assembled from 43 drives and 1 spare - not enough to start the array.
^ permalink raw reply
* Re: [PATCH] bcache: Use setup_timer and mod_timer
From: Coly Li @ 2016-06-30 12:11 UTC (permalink / raw)
To: Amitoj Kaur Chawla, kent.overstreet, shli, linux-bcache,
linux-raid, linux-kernel
Cc: julia.lawall
In-Reply-To: <20160630114520.GA8025@amitoj-Inspiron-3542>
在 16/6/30 下午7:45, Amitoj Kaur Chawla 写道:
> Convert a call to init_timer and accompanying intializations of
> the timer's data and function fields to a call to setup_timer.
>
> The Coccinelle semantic patch that fixes one part of this issue
> is as follows:
> @@
> expression t,d,f,e1;
> identifier x1;
> statement S1;
> @@
>
> (
> -t.data = d;
> |
> -t.function = f;
> |
> -init_timer(&t);
> +setup_timer(&t,f,d);
> |
> -init_timer_on_stack(&t);
> +setup_timer_on_stack(&t,f,d);
> )
> <... when != S1
> t.x1 = e1;
> ...>
>
> Also convert calls to add_timer and intialisation of timer's expire
> field to a call to mod_timer which is more compact.
>
> Signed-off-by: Amitoj Kaur Chawla <amitoj1606@gmail.com>
> ---
> drivers/md/bcache/stats.c | 7 ++-----
> 1 file changed, 2 insertions(+), 5 deletions(-)
>
> diff --git a/drivers/md/bcache/stats.c b/drivers/md/bcache/stats.c
> index 0ca072c..094d6d6 100644
> --- a/drivers/md/bcache/stats.c
> +++ b/drivers/md/bcache/stats.c
> @@ -233,9 +233,6 @@ void bch_cache_accounting_init(struct cache_accounting *acc,
> kobject_init(&acc->day.kobj, &bch_stats_ktype);
>
> closure_init(&acc->cl, parent);
> - init_timer(&acc->timer);
> - acc->timer.expires = jiffies + accounting_delay;
> - acc->timer.data = (unsigned long) acc;
> - acc->timer.function = scale_accounting;
> - add_timer(&acc->timer);
> + setup_timer(&acc->timer, scale_accounting, (unsigned long) acc);
> + mod_timer(&acc->timer, jiffies + accounting_delay);
> }
>
This patch uses new setup_timer() and mod_timer() interface to
initialize acc->timer. No change in functionality. It is OK to me.
Reviewed-by: Coly Li <colyli@suse.de>
Coly
^ permalink raw reply
* [PATCH] bcache: Use setup_timer and mod_timer
From: Amitoj Kaur Chawla @ 2016-06-30 11:45 UTC (permalink / raw)
To: kent.overstreet, shli, linux-bcache, linux-raid, linux-kernel
Cc: julia.lawall
Convert a call to init_timer and accompanying intializations of
the timer's data and function fields to a call to setup_timer.
The Coccinelle semantic patch that fixes one part of this issue
is as follows:
@@
expression t,d,f,e1;
identifier x1;
statement S1;
@@
(
-t.data = d;
|
-t.function = f;
|
-init_timer(&t);
+setup_timer(&t,f,d);
|
-init_timer_on_stack(&t);
+setup_timer_on_stack(&t,f,d);
)
<... when != S1
t.x1 = e1;
...>
Also convert calls to add_timer and intialisation of timer's expire
field to a call to mod_timer which is more compact.
Signed-off-by: Amitoj Kaur Chawla <amitoj1606@gmail.com>
---
drivers/md/bcache/stats.c | 7 ++-----
1 file changed, 2 insertions(+), 5 deletions(-)
diff --git a/drivers/md/bcache/stats.c b/drivers/md/bcache/stats.c
index 0ca072c..094d6d6 100644
--- a/drivers/md/bcache/stats.c
+++ b/drivers/md/bcache/stats.c
@@ -233,9 +233,6 @@ void bch_cache_accounting_init(struct cache_accounting *acc,
kobject_init(&acc->day.kobj, &bch_stats_ktype);
closure_init(&acc->cl, parent);
- init_timer(&acc->timer);
- acc->timer.expires = jiffies + accounting_delay;
- acc->timer.data = (unsigned long) acc;
- acc->timer.function = scale_accounting;
- add_timer(&acc->timer);
+ setup_timer(&acc->timer, scale_accounting, (unsigned long) acc);
+ mod_timer(&acc->timer, jiffies + accounting_delay);
}
--
1.9.1
^ permalink raw reply related
* [PATCH] md: add missing sysfs_notify on array_state update
From: Tomasz Majchrzak @ 2016-06-30 8:47 UTC (permalink / raw)
To: linux-raid
Changeset 6791875e2e53 has added early return from a function so there is no
sysfs notification for 'active' and 'clean' state change.
Signed-off-by: Tomasz Majchrzak <tomasz.majchrzak@intel.com>
---
drivers/md/md.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/drivers/md/md.c b/drivers/md/md.c
index c068f17..8d52f44 100644
--- a/drivers/md/md.c
+++ b/drivers/md/md.c
@@ -3930,6 +3930,8 @@ array_state_store(struct mddev *mddev, const char *buf, size_t len)
} else
err = -EBUSY;
}
+ if (!err)
+ sysfs_notify_dirent_safe(mddev->sysfs_state);
spin_unlock(&mddev->lock);
return err ?: len;
}
--
1.8.3.1
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox