* Re: Setting up fakeraid with mdadm / dmraid
From: John Stoffel @ 2016-04-05 13:58 UTC (permalink / raw)
To: Christoph Pleger; +Cc: linux-raid
In-Reply-To: <b6ced30ee931f9a6faf2541db2462401.squirrel@postweb.cs.tu-dortmund.de>
Christoph> I have a machine with an LSI Megaraid Sofware RAID
Christoph> fakeraid, which uses ddf format. About two weeks ago, when
Christoph> I wanted to install the machine, I configured a RAID 1
Christoph> array in the BIOS config utility and then booted the
Christoph> machine from PXE with an NFSROOT. mdadm and dmraid are
Christoph> installed in the NFSROOT, dracut is used for initrd
Christoph> generation. As dracut prefers mdadm over dmraid, mdadm was
Christoph> chosen for RAID management.
First off, I would strongly suggest that you turn off the FakeRAID
entirely, just expose both disks to the OS and have the OS do the
mirroring. FakeRAID was designed when CPUs were much slower, but RAID
HW ASICs were also expensive. So you got the worst of both worlds!
*grin*
It's also almost entirely black magic to support these things, since
the vendors don't generally share the details on the on-disk format
that I'm aware of. I'm probably wrong in the details.
But again, please just use the controller in JBOD (Just a Bunch of
Disks) mode, and let mdadm mirror the disks for you.
Christoph> But though mdadm detected the RAID (it created container
Christoph> device /dev/md127 and raid device /dev/md126), it destroyed
Christoph> it - that is, though I did not perform any action on the
Christoph> raid disks, at the next boot, the BIOS RAID utility had
Christoph> "forgotten" about the RAID configuration it had created
Christoph> before.
Christoph> Why are the possible reasons for that?
Crappy implementation of semi-secret on-disk format? It could be
that mdadm over-wrote something on the disk, or you partitioned it and
then overwrote something the FakeRAID wanted kept around.
In general, just don't go the fake RAID route, it's not worth the
hassle.
John
^ permalink raw reply
* [PATCH] [RFC] fix potential access after free: return value of blk_check_plugged() must be used schedule() safe
From: Lars Ellenberg @ 2016-04-05 13:36 UTC (permalink / raw)
To: Neil Brown, Jens Axboe
Cc: Chris Mason, Josef Bacik, David Sterba, linux-raid, linux-kernel,
linux-btrfs
blk_check_plugged() will return a pointer
to an object linked on current->plug->cb_list.
That list may "at any time" be implicitly cleared by
blk_flush_plug_list()
flush_plug_callbacks()
either as a result of blk_finish_plug(),
or implicitly by schedule() [and maybe other implicit mechanisms?]
If there is no protection against an implicit unplug
between the call to blk_check_plug() and using its return value,
that implicit unplug may have already happened,
even before the plug is actually initialized or populated,
and we may be using a pointer to already free()d data.
I suggest that both raid1 and raid10 can easily be fixed
by moving the call to blk_check_plugged() inside the spinlock.
For md/raid5 and btrfs/raid56,
I'm unsure how (if) this needs to be fixed.
The other current in-tree users of blk_check_plugged()
are mm_check_plugged(), and mddev_check_plugged().
mm_check_plugged() is already used safely inside a spinlock.
with mddev_check_plugged() I'm unsure, at least on a preempt kernel.
Did I overlook any magic that protects against such implicit unplug?
Also, why pretend that a custom plug struct (such as raid1_plug_cb)
may have its member "struct blk_plug_cb cb" at an arbitrary offset?
As it is, raid1_check_plugged() below is actually just a cast.
Signed-off-by: Lars Ellenberg <lars.ellenberg@linbit.com>
---
drivers/md/raid1.c | 19 +++++++++++++------
drivers/md/raid10.c | 21 +++++++++++++--------
drivers/md/raid5.c | 5 +++++
fs/btrfs/raid56.c | 5 +++++
4 files changed, 36 insertions(+), 14 deletions(-)
diff --git a/drivers/md/raid1.c b/drivers/md/raid1.c
index 39fb21e..55dc960 100644
--- a/drivers/md/raid1.c
+++ b/drivers/md/raid1.c
@@ -1044,6 +1044,18 @@ static void raid1_unplug(struct blk_plug_cb *cb, bool from_schedule)
kfree(plug);
}
+static struct raid1_plug_cb *raid1_check_plugged(struct mddev *mddev)
+{
+ /* return (struct raid1_plug_cb*)blk_check_plugged(...); */
+ struct blk_plug_cb *cb;
+ struct raid1_plug_cb *plug = NULL;
+
+ cb = blk_check_plugged(raid1_unplug, mddev, sizeof(*plug));
+ if (cb)
+ plug = container_of(cb, struct raid1_plug_cb, cb);
+ return plug;
+}
+
static void raid1_make_request(struct mddev *mddev, struct bio * bio)
{
struct r1conf *conf = mddev->private;
@@ -1060,7 +1072,6 @@ static void raid1_make_request(struct mddev *mddev, struct bio * bio)
& (REQ_DISCARD | REQ_SECURE));
const unsigned long do_same = (bio->bi_rw & REQ_WRITE_SAME);
struct md_rdev *blocked_rdev;
- struct blk_plug_cb *cb;
struct raid1_plug_cb *plug = NULL;
int first_clone;
int sectors_handled;
@@ -1382,12 +1393,8 @@ read_again:
atomic_inc(&r1_bio->remaining);
- cb = blk_check_plugged(raid1_unplug, mddev, sizeof(*plug));
- if (cb)
- plug = container_of(cb, struct raid1_plug_cb, cb);
- else
- plug = NULL;
spin_lock_irqsave(&conf->device_lock, flags);
+ plug = raid1_check_plugged(mddev);
if (plug) {
bio_list_add(&plug->pending, mbio);
plug->pending_cnt++;
diff --git a/drivers/md/raid10.c b/drivers/md/raid10.c
index e3fd725..d7d4397 100644
--- a/drivers/md/raid10.c
+++ b/drivers/md/raid10.c
@@ -1052,6 +1052,18 @@ static void raid10_unplug(struct blk_plug_cb *cb, bool from_schedule)
kfree(plug);
}
+static struct raid10_plug_cb *raid10_check_plugged(struct mddev *mddev)
+{
+ /* return (struct raid1_plug_cb*)blk_check_plugged(...); */
+ struct blk_plug_cb *cb;
+ struct raid10_plug_cb *plug = NULL;
+
+ cb = blk_check_plugged(raid10_unplug, mddev, sizeof(*plug));
+ if (cb)
+ plug = container_of(cb, struct raid10_plug_cb, cb);
+ return plug;
+}
+
static void __make_request(struct mddev *mddev, struct bio *bio)
{
struct r10conf *conf = mddev->private;
@@ -1066,7 +1078,6 @@ static void __make_request(struct mddev *mddev, struct bio *bio)
const unsigned long do_same = (bio->bi_rw & REQ_WRITE_SAME);
unsigned long flags;
struct md_rdev *blocked_rdev;
- struct blk_plug_cb *cb;
struct raid10_plug_cb *plug = NULL;
int sectors_handled;
int max_sectors;
@@ -1369,14 +1380,8 @@ retry_write:
atomic_inc(&r10_bio->remaining);
- cb = blk_check_plugged(raid10_unplug, mddev,
- sizeof(*plug));
- if (cb)
- plug = container_of(cb, struct raid10_plug_cb,
- cb);
- else
- plug = NULL;
spin_lock_irqsave(&conf->device_lock, flags);
+ plug = raid10_check_plugged(mddev);
if (plug) {
bio_list_add(&plug->pending, mbio);
plug->pending_cnt++;
diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c
index 8ab8b65..4e3b02b 100644
--- a/drivers/md/raid5.c
+++ b/drivers/md/raid5.c
@@ -5034,6 +5034,11 @@ static void release_stripe_plug(struct mddev *mddev,
}
cb = container_of(blk_cb, struct raid5_plug_cb, cb);
+/* FIXME
+ * Nothing protects current from being scheduled, which means cb, aka plug,
+ * may implicitly be "unplugged" any time now, before it even is initialized,
+ * and will then be a pointer to free()d space.
+ */
if (cb->list.next == NULL) {
int i;
diff --git a/fs/btrfs/raid56.c b/fs/btrfs/raid56.c
index 0b7792e..17757d4 100644
--- a/fs/btrfs/raid56.c
+++ b/fs/btrfs/raid56.c
@@ -1774,6 +1774,11 @@ int raid56_parity_write(struct btrfs_root *root, struct bio *bio,
cb = blk_check_plugged(btrfs_raid_unplug, root->fs_info,
sizeof(*plug));
if (cb) {
+/* FIXME
+ * Nothing protects current from being scheduled, which means cb, aka plug,
+ * may implicitly be "unplugged" any time now, before it even is initialized,
+ * and will then be a pointer to free()d space.
+ */
plug = container_of(cb, struct btrfs_plug_cb, cb);
if (!plug->info) {
plug->info = root->fs_info;
--
1.9.1
^ permalink raw reply related
* Re: [PATCH 20/27] dm: dm-bufio.c: use bio_set_vec_table()
From: Christoph Hellwig @ 2016-04-05 13:04 UTC (permalink / raw)
To: Ming Lei
Cc: Jens Axboe, linux-kernel, linux-block, Christoph Hellwig,
Boaz Harrosh, Alasdair Kergon, Mike Snitzer,
maintainer:DEVICE-MAPPER (LVM), Shaohua Li,
open list:SOFTWARE RAID (Multiple Disks) SUPPORT
In-Reply-To: <1459858062-21075-6-git-send-email-tom.leiming@gmail.com>
On Tue, Apr 05, 2016 at 08:07:35PM +0800, Ming Lei wrote:
> Signed-off-by: Ming Lei <tom.leiming@gmail.com>
> ---
> drivers/md/dm-bufio.c | 3 +--
> 1 file changed, 1 insertion(+), 2 deletions(-)
>
> diff --git a/drivers/md/dm-bufio.c b/drivers/md/dm-bufio.c
> index cd77216..0e48ad7 100644
> --- a/drivers/md/dm-bufio.c
> +++ b/drivers/md/dm-bufio.c
> @@ -624,8 +624,7 @@ static void use_inline_bio(struct dm_buffer *b, int rw, sector_t block,
> int len;
>
> bio_init(&b->bio);
> - b->bio.bi_io_vec = b->bio_vec;
> - b->bio.bi_max_vecs = DM_BUFIO_INLINE_VECS;
> + bio_set_vec_table(&b->bio, b->bio_vec, DM_BUFIO_INLINE_VECS);
> b->bio.bi_iter.bi_sector = block << b->c->sectors_per_block_bits;
> b->bio.bi_bdev = b->c->bdev;
> b->bio.bi_end_io = inline_endio;
Should be switched to use bio_alloc instead.
^ permalink raw reply
* Setting up fakeraid with mdadm / dmraid
From: Christoph Pleger @ 2016-04-05 13:03 UTC (permalink / raw)
To: linux-raid
Hello,
I have a machine with an LSI Megaraid Sofware RAID fakeraid, which uses
ddf format. About two weeks ago, when I wanted to install the machine, I
configured a RAID 1 array in the BIOS config utility and then booted the
machine from PXE with an NFSROOT. mdadm and dmraid are installed in the
NFSROOT, dracut is used for initrd generation. As dracut prefers mdadm
over dmraid, mdadm was chosen for RAID management.
But though mdadm detected the RAID (it created container device /dev/md127
and raid device /dev/md126), it destroyed it - that is, though I did not
perform any action on the raid disks, at the next boot, the BIOS RAID
utility had "forgotten" about the RAID configuration it had created
before.
Why are the possible reasons for that?
Regards
Christoph
^ permalink raw reply
* [PATCH] badblocks: fix wrong return value when badblocks are disabled
From: Artur Paszkiewicz @ 2016-04-05 12:59 UTC (permalink / raw)
To: axboe; +Cc: linux-block, vishal.l.verma, linux-raid, Artur Paszkiewicz
The return value of md_set_badblocks() was inverted when the code was
taken out of md, but the case when badblocks are disabled was left
unchanged. This causes silent ignoring of I/O errors and other
unpredictable behavior on md arrays that do not support badblocks (any
array not using v1.x metadata).
Fixes: 9e0e252a048b ("badblocks: Add core badblock management code")
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
---
block/badblocks.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/block/badblocks.c b/block/badblocks.c
index 7be53cb..252f24e 100644
--- a/block/badblocks.c
+++ b/block/badblocks.c
@@ -147,6 +147,7 @@ EXPORT_SYMBOL_GPL(badblocks_check);
* Return:
* 0: success
* 1: failed to set badblocks (out of space)
+ * -1: failed to set badblocks (badblocks are disabled)
*/
int badblocks_set(struct badblocks *bb, sector_t s, int sectors,
int acknowledged)
@@ -158,7 +159,7 @@ int badblocks_set(struct badblocks *bb, sector_t s, int sectors,
if (bb->shift < 0)
/* badblocks are disabled */
- return 0;
+ return -1;
if (bb->shift) {
/* round the start down, and the end up */
--
2.6.2
^ permalink raw reply related
* Re: [PATCH 17/27] dm: crypt: use bio_add_page()
From: Christoph Hellwig @ 2016-04-05 12:49 UTC (permalink / raw)
To: Ming Lei
Cc: Jens Axboe, linux-kernel, linux-block, Christoph Hellwig,
Boaz Harrosh, Alasdair Kergon, Mike Snitzer,
maintainer:DEVICE-MAPPER (LVM), Shaohua Li,
open list:SOFTWARE RAID (Multiple Disks) SUPPORT
In-Reply-To: <1459858062-21075-3-git-send-email-tom.leiming@gmail.com>
Looks fine,
Reviewed-by: Christoph Hellwig <hch@lst.de>
^ permalink raw reply
* Re: [PATCH 11/27] bcache: io.c: use bio_set_vec_table
From: Christoph Hellwig @ 2016-04-05 12:49 UTC (permalink / raw)
To: Ming Lei
Cc: Jens Axboe, linux-kernel, linux-block, Christoph Hellwig,
Boaz Harrosh, Kent Overstreet, Shaohua Li,
open list:BCACHE (BLOCK LAYER CACHE),
open list:SOFTWARE RAID (Multiple Disks) SUPPORT
In-Reply-To: <1459857443-20611-12-git-send-email-tom.leiming@gmail.com>
On Tue, Apr 05, 2016 at 07:56:56PM +0800, Ming Lei wrote:
> diff --git a/drivers/md/bcache/io.c b/drivers/md/bcache/io.c
> index 86a0bb8..1c48462 100644
> --- a/drivers/md/bcache/io.c
> +++ b/drivers/md/bcache/io.c
> @@ -26,8 +26,7 @@ struct bio *bch_bbio_alloc(struct cache_set *c)
>
> bio_init(bio);
> bio->bi_flags |= BIO_POOL_NONE << BIO_POOL_OFFSET;
> - bio->bi_max_vecs = bucket_pages(c);
> - bio->bi_io_vec = bio->bi_inline_vecs;
> + bio_set_vec_table(bio, bio->bi_inline_vecs, bucket_pages(c));
All this bcache code needs to move away from bio_init on a bio
embedded in a driver private structure toward properly using
bio_alloc / bio_alloc_bioset. That will also fix the crash
with bcache over md that Shaohua reported, so I'd suggest to fast
track this part of the series.
^ permalink raw reply
* Re: Repairing R1: Part tabl, & precise command
From: Ron Leach @ 2016-04-05 12:22 UTC (permalink / raw)
Cc: Linux RAID Mailing List
In-Reply-To: <57039BCC.6040103@tesco.net>
On 05/04/2016 12:04, Ron Leach wrote:
>
> I'll look at parted and gdisk.
>
gdisk reports /dev/sdb has a damaged main GPT partition table, and a
reasonable backup GPT partition table - but with an invalid header.
(/dev/sda and /dev/sdc are fine.) So the immediate problem here is
fixing the GPT partition table.
Strictly, that isn't a RAID1 problem and I don't think it's
appropriate to ask for comment on that issue on the raid list; people
here are truly helpful so I'll fix the partition but I could use a
comment on whether I should then just use the
mdadm -- manage .. --add ..
commands and whether mdadm needs to be inhibited from taking any
automatic remedial action.
regards, Ron
^ permalink raw reply
* [PATCH 20/27] dm: dm-bufio.c: use bio_set_vec_table()
From: Ming Lei @ 2016-04-05 12:07 UTC (permalink / raw)
To: Jens Axboe, linux-kernel
Cc: linux-block, Christoph Hellwig, Boaz Harrosh, Ming Lei,
Alasdair Kergon, Mike Snitzer, maintainer:DEVICE-MAPPER LVM,
Shaohua Li, open list:SOFTWARE RAID Multiple Disks SUPPORT
In-Reply-To: <1459858062-21075-1-git-send-email-tom.leiming@gmail.com>
Signed-off-by: Ming Lei <tom.leiming@gmail.com>
---
drivers/md/dm-bufio.c | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/drivers/md/dm-bufio.c b/drivers/md/dm-bufio.c
index cd77216..0e48ad7 100644
--- a/drivers/md/dm-bufio.c
+++ b/drivers/md/dm-bufio.c
@@ -624,8 +624,7 @@ static void use_inline_bio(struct dm_buffer *b, int rw, sector_t block,
int len;
bio_init(&b->bio);
- b->bio.bi_io_vec = b->bio_vec;
- b->bio.bi_max_vecs = DM_BUFIO_INLINE_VECS;
+ bio_set_vec_table(&b->bio, b->bio_vec, DM_BUFIO_INLINE_VECS);
b->bio.bi_iter.bi_sector = block << b->c->sectors_per_block_bits;
b->bio.bi_bdev = b->c->bdev;
b->bio.bi_end_io = inline_endio;
--
1.9.1
^ permalink raw reply related
* [PATCH 19/27] dm: dm.c: replace 'bio->bi_vcnt == 1' with !bio_multiple_segments
From: Ming Lei @ 2016-04-05 12:07 UTC (permalink / raw)
To: Jens Axboe, linux-kernel
Cc: linux-block, Christoph Hellwig, Boaz Harrosh, Ming Lei,
Alasdair Kergon, Mike Snitzer, maintainer:DEVICE-MAPPER LVM,
Shaohua Li, open list:SOFTWARE RAID Multiple Disks SUPPORT
In-Reply-To: <1459858062-21075-1-git-send-email-tom.leiming@gmail.com>
Signed-off-by: Ming Lei <tom.leiming@gmail.com>
---
drivers/md/dm.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/drivers/md/dm.c b/drivers/md/dm.c
index be49057..4c34f88 100644
--- a/drivers/md/dm.c
+++ b/drivers/md/dm.c
@@ -2172,7 +2172,8 @@ static void dm_request_fn(struct request_queue *q)
pos = blk_rq_pos(rq);
if ((dm_request_peeked_before_merge_deadline(md) &&
- md_in_flight(md) && rq->bio && rq->bio->bi_vcnt == 1 &&
+ md_in_flight(md) && rq->bio &&
+ !bio_multiple_segments(rq->bio) &&
md->last_rq_pos == pos && md->last_rq_rw == rq_data_dir(rq)) ||
(ti->type->busy && ti->type->busy(ti))) {
blk_delay_queue(q, HZ / 100);
--
1.9.1
^ permalink raw reply related
* [PATCH 18/27] dm: dm-io.c: use bio_get_base_vec()
From: Ming Lei @ 2016-04-05 12:07 UTC (permalink / raw)
To: Jens Axboe, linux-kernel
Cc: linux-block, Christoph Hellwig, Boaz Harrosh, Ming Lei,
Alasdair Kergon, Mike Snitzer, maintainer:DEVICE-MAPPER LVM,
Shaohua Li, open list:SOFTWARE RAID Multiple Disks SUPPORT
In-Reply-To: <1459858062-21075-1-git-send-email-tom.leiming@gmail.com>
Signed-off-by: Ming Lei <tom.leiming@gmail.com>
---
drivers/md/dm-io.c | 7 ++++++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/drivers/md/dm-io.c b/drivers/md/dm-io.c
index 06d426e..6b0e466 100644
--- a/drivers/md/dm-io.c
+++ b/drivers/md/dm-io.c
@@ -221,7 +221,12 @@ static void bio_dp_init(struct dpages *dp, struct bio *bio)
{
dp->get_page = bio_get_page;
dp->next_page = bio_next_page;
- dp->context_ptr = __bvec_iter_bvec(bio->bi_io_vec, bio->bi_iter);
+
+ /*
+ * need to fix both bio_get_page() and bio_next_page()
+ * before multipage bvecs
+ */
+ dp->context_ptr = bio_get_base_vec(bio);
dp->context_u = bio->bi_iter.bi_bvec_done;
}
--
1.9.1
^ permalink raw reply related
* [PATCH 17/27] dm: crypt: use bio_add_page()
From: Ming Lei @ 2016-04-05 12:07 UTC (permalink / raw)
To: Jens Axboe, linux-kernel
Cc: linux-block, Christoph Hellwig, Boaz Harrosh, Ming Lei,
Alasdair Kergon, Mike Snitzer, maintainer:DEVICE-MAPPER LVM,
Shaohua Li, open list:SOFTWARE RAID Multiple Disks SUPPORT
In-Reply-To: <1459858062-21075-1-git-send-email-tom.leiming@gmail.com>
Signed-off-by: Ming Lei <tom.leiming@gmail.com>
---
drivers/md/dm-crypt.c | 8 +-------
1 file changed, 1 insertion(+), 7 deletions(-)
diff --git a/drivers/md/dm-crypt.c b/drivers/md/dm-crypt.c
index 4f3cb35..a2805c1 100644
--- a/drivers/md/dm-crypt.c
+++ b/drivers/md/dm-crypt.c
@@ -995,7 +995,6 @@ static struct bio *crypt_alloc_buffer(struct dm_crypt_io *io, unsigned size)
gfp_t gfp_mask = GFP_NOWAIT | __GFP_HIGHMEM;
unsigned i, len, remaining_size;
struct page *page;
- struct bio_vec *bvec;
retry:
if (unlikely(gfp_mask & __GFP_DIRECT_RECLAIM))
@@ -1020,12 +1019,7 @@ retry:
len = (remaining_size > PAGE_SIZE) ? PAGE_SIZE : remaining_size;
- bvec = &clone->bi_io_vec[clone->bi_vcnt++];
- bvec->bv_page = page;
- bvec->bv_len = len;
- bvec->bv_offset = 0;
-
- clone->bi_iter.bi_size += len;
+ bio_add_page(clone, page, len, 0);
remaining_size -= len;
}
--
1.9.1
^ permalink raw reply related
* [PATCH 16/27] bcache: super: use bio_get_base_vec
From: Ming Lei @ 2016-04-05 12:07 UTC (permalink / raw)
To: Jens Axboe, linux-kernel
Cc: linux-block, Christoph Hellwig, Boaz Harrosh, Ming Lei,
Kent Overstreet, Shaohua Li, open list:BCACHE BLOCK LAYER CACHE,
open list:SOFTWARE RAID Multiple Disks SUPPORT
In-Reply-To: <1459858062-21075-1-git-send-email-tom.leiming@gmail.com>
Signed-off-by: Ming Lei <tom.leiming@gmail.com>
---
drivers/md/bcache/super.c | 18 ++++++++++++++----
1 file changed, 14 insertions(+), 4 deletions(-)
diff --git a/drivers/md/bcache/super.c b/drivers/md/bcache/super.c
index 3f649c9..56ad797 100644
--- a/drivers/md/bcache/super.c
+++ b/drivers/md/bcache/super.c
@@ -208,7 +208,12 @@ static void write_bdev_super_endio(struct bio *bio)
static void __write_super(struct cache_sb *sb, struct bio *bio)
{
- struct cache_sb *out = page_address(bio->bi_io_vec[0].bv_page);
+ /*
+ * For accessing page pointed to by the 1st bvec, it
+ * works too after multipage bvecs.
+ */
+ struct bio_vec *bvec = bio_get_base_vec(bio);
+ struct cache_sb *out = page_address(bvec->bv_page);
unsigned i;
bio->bi_iter.bi_sector = SB_SECTOR;
@@ -1145,6 +1150,7 @@ static void register_bdev(struct cache_sb *sb, struct page *sb_page,
char name[BDEVNAME_SIZE];
const char *err = "cannot allocate memory";
struct cache_set *c;
+ struct bio_vec *bvec;
memcpy(&dc->sb, sb, sizeof(struct cache_sb));
dc->bdev = bdev;
@@ -1152,7 +1158,8 @@ static void register_bdev(struct cache_sb *sb, struct page *sb_page,
bio_init(&dc->sb_bio);
bio_set_vec_table(&dc->sb_bio, dc->sb_bio.bi_inline_vecs, 1);
- dc->sb_bio.bi_io_vec[0].bv_page = sb_page;
+ bvec = bio_get_base_vec(&dc->sb_bio);
+ bvec->bv_page = sb_page;
get_page(sb_page);
if (cached_dev_init(dc, sb->block_size << 9))
@@ -1776,6 +1783,7 @@ void bch_cache_release(struct kobject *kobj)
{
struct cache *ca = container_of(kobj, struct cache, kobj);
unsigned i;
+ struct bio_vec *bvec = bio_get_base_vec(&ca->sb_bio);
if (ca->set) {
BUG_ON(ca->set->cache[ca->sb.nr_this_dev] != ca);
@@ -1793,7 +1801,7 @@ void bch_cache_release(struct kobject *kobj)
free_fifo(&ca->free[i]);
if (ca->sb_bio.bi_inline_vecs[0].bv_page)
- put_page(ca->sb_bio.bi_io_vec[0].bv_page);
+ put_page(bvec->bv_page);
if (!IS_ERR_OR_NULL(ca->bdev))
blkdev_put(ca->bdev, FMODE_READ|FMODE_WRITE|FMODE_EXCL);
@@ -1843,6 +1851,7 @@ static int register_cache(struct cache_sb *sb, struct page *sb_page,
char name[BDEVNAME_SIZE];
const char *err = NULL;
int ret = 0;
+ struct bio_vec *bvec;
memcpy(&ca->sb, sb, sizeof(struct cache_sb));
ca->bdev = bdev;
@@ -1850,7 +1859,8 @@ static int register_cache(struct cache_sb *sb, struct page *sb_page,
bio_init(&ca->sb_bio);
bio_set_vec_table(&ca->sb_bio, ca->sb_bio.bi_inline_vecs, 1);
- ca->sb_bio.bi_io_vec[0].bv_page = sb_page;
+ bvec = bio_get_base_vec(&ca->sb_bio);
+ bvec->bv_page = sb_page;
get_page(sb_page);
if (blk_queue_discard(bdev_get_queue(ca->bdev)))
--
1.9.1
^ permalink raw reply related
* [PATCH 15/27] bcache: super: use bio_set_vec_table()
From: Ming Lei @ 2016-04-05 12:07 UTC (permalink / raw)
To: Jens Axboe, linux-kernel
Cc: linux-block, Christoph Hellwig, Boaz Harrosh, Ming Lei,
Kent Overstreet, Shaohua Li, open list:BCACHE BLOCK LAYER CACHE,
open list:SOFTWARE RAID Multiple Disks SUPPORT
Signed-off-by: Ming Lei <tom.leiming@gmail.com>
---
drivers/md/bcache/super.c | 10 ++++------
1 file changed, 4 insertions(+), 6 deletions(-)
diff --git a/drivers/md/bcache/super.c b/drivers/md/bcache/super.c
index a296425..3f649c9 100644
--- a/drivers/md/bcache/super.c
+++ b/drivers/md/bcache/super.c
@@ -1151,8 +1151,7 @@ static void register_bdev(struct cache_sb *sb, struct page *sb_page,
dc->bdev->bd_holder = dc;
bio_init(&dc->sb_bio);
- dc->sb_bio.bi_max_vecs = 1;
- dc->sb_bio.bi_io_vec = dc->sb_bio.bi_inline_vecs;
+ bio_set_vec_table(&dc->sb_bio, dc->sb_bio.bi_inline_vecs, 1);
dc->sb_bio.bi_io_vec[0].bv_page = sb_page;
get_page(sb_page);
@@ -1812,8 +1811,8 @@ static int cache_alloc(struct cache_sb *sb, struct cache *ca)
kobject_init(&ca->kobj, &bch_cache_ktype);
bio_init(&ca->journal.bio);
- ca->journal.bio.bi_max_vecs = 8;
- ca->journal.bio.bi_io_vec = ca->journal.bio.bi_inline_vecs;
+ bio_set_vec_table(&ca->journal.bio,
+ ca->journal.bio.bi_inline_vecs, 8);
free = roundup_pow_of_two(ca->sb.nbuckets) >> 10;
@@ -1850,8 +1849,7 @@ static int register_cache(struct cache_sb *sb, struct page *sb_page,
ca->bdev->bd_holder = ca;
bio_init(&ca->sb_bio);
- ca->sb_bio.bi_max_vecs = 1;
- ca->sb_bio.bi_io_vec = ca->sb_bio.bi_inline_vecs;
+ bio_set_vec_table(&ca->sb_bio, ca->sb_bio.bi_inline_vecs, 1);
ca->sb_bio.bi_io_vec[0].bv_page = sb_page;
get_page(sb_page);
--
1.9.1
^ permalink raw reply related
* [PATCH 14/27] bcache: writeback: use bio_set_vec_table()
From: Ming Lei @ 2016-04-05 11:56 UTC (permalink / raw)
To: Jens Axboe, linux-kernel
Cc: linux-block, Christoph Hellwig, Boaz Harrosh, Ming Lei,
Kent Overstreet, Shaohua Li, open list:BCACHE BLOCK LAYER CACHE,
open list:SOFTWARE RAID Multiple Disks SUPPORT
In-Reply-To: <1459857443-20611-1-git-send-email-tom.leiming@gmail.com>
Signed-off-by: Ming Lei <tom.leiming@gmail.com>
---
drivers/md/bcache/writeback.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/drivers/md/bcache/writeback.c b/drivers/md/bcache/writeback.c
index b9346cd..49a8f8a 100644
--- a/drivers/md/bcache/writeback.c
+++ b/drivers/md/bcache/writeback.c
@@ -112,9 +112,9 @@ static void dirty_init(struct keybuf_key *w)
bio_set_prio(bio, IOPRIO_PRIO_VALUE(IOPRIO_CLASS_IDLE, 0));
bio->bi_iter.bi_size = KEY_SIZE(&w->key) << 9;
- bio->bi_max_vecs = DIV_ROUND_UP(KEY_SIZE(&w->key), PAGE_SECTORS);
bio->bi_private = w;
- bio->bi_io_vec = bio->bi_inline_vecs;
+ bio_set_vec_table(bio, bio->bi_inline_vecs,
+ DIV_ROUND_UP(KEY_SIZE(&w->key), PAGE_SECTORS));
bch_bio_map(bio, NULL);
}
--
1.9.1
^ permalink raw reply related
* [PATCH 13/27] bcache: movinggc: use bio_set_vec_table()
From: Ming Lei @ 2016-04-05 11:56 UTC (permalink / raw)
To: Jens Axboe, linux-kernel
Cc: linux-block, Christoph Hellwig, Boaz Harrosh, Ming Lei,
Kent Overstreet, Shaohua Li, open list:BCACHE BLOCK LAYER CACHE,
open list:SOFTWARE RAID Multiple Disks SUPPORT
In-Reply-To: <1459857443-20611-1-git-send-email-tom.leiming@gmail.com>
Signed-off-by: Ming Lei <tom.leiming@gmail.com>
---
drivers/md/bcache/movinggc.c | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/drivers/md/bcache/movinggc.c b/drivers/md/bcache/movinggc.c
index b929fc9..dbe5af2 100644
--- a/drivers/md/bcache/movinggc.c
+++ b/drivers/md/bcache/movinggc.c
@@ -85,10 +85,10 @@ static void moving_init(struct moving_io *io)
bio_set_prio(bio, IOPRIO_PRIO_VALUE(IOPRIO_CLASS_IDLE, 0));
bio->bi_iter.bi_size = KEY_SIZE(&io->w->key) << 9;
- bio->bi_max_vecs = DIV_ROUND_UP(KEY_SIZE(&io->w->key),
- PAGE_SECTORS);
bio->bi_private = &io->cl;
- bio->bi_io_vec = bio->bi_inline_vecs;
+ bio_set_vec_table(bio, bio->bi_inline_vecs,
+ DIV_ROUND_UP(KEY_SIZE(&io->w->key),
+ PAGE_SECTORS));
bch_bio_map(bio, NULL);
}
--
1.9.1
^ permalink raw reply related
* [PATCH 12/27] bcache: journal.c: use bio_set_vec_table()
From: Ming Lei @ 2016-04-05 11:56 UTC (permalink / raw)
To: Jens Axboe, linux-kernel
Cc: linux-block, Christoph Hellwig, Boaz Harrosh, Ming Lei,
Kent Overstreet, Shaohua Li, open list:BCACHE BLOCK LAYER CACHE,
open list:SOFTWARE RAID Multiple Disks SUPPORT
In-Reply-To: <1459857443-20611-1-git-send-email-tom.leiming@gmail.com>
Signed-off-by: Ming Lei <tom.leiming@gmail.com>
---
drivers/md/bcache/journal.c | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/drivers/md/bcache/journal.c b/drivers/md/bcache/journal.c
index 29eba72..bf8924f 100644
--- a/drivers/md/bcache/journal.c
+++ b/drivers/md/bcache/journal.c
@@ -453,8 +453,7 @@ static void do_journal_discard(struct cache *ca)
ca->sb.d[ja->discard_idx]);
bio->bi_bdev = ca->bdev;
bio->bi_rw = REQ_WRITE|REQ_DISCARD;
- bio->bi_max_vecs = 1;
- bio->bi_io_vec = bio->bi_inline_vecs;
+ bio_set_vec_table(bio, bio->bi_inline_vecs, 1);
bio->bi_iter.bi_size = bucket_bytes(ca);
bio->bi_end_io = journal_discard_endio;
--
1.9.1
^ permalink raw reply related
* [PATCH 11/27] bcache: io.c: use bio_set_vec_table
From: Ming Lei @ 2016-04-05 11:56 UTC (permalink / raw)
To: Jens Axboe, linux-kernel
Cc: linux-block, Christoph Hellwig, Boaz Harrosh, Ming Lei,
Kent Overstreet, Shaohua Li, open list:BCACHE BLOCK LAYER CACHE,
open list:SOFTWARE RAID Multiple Disks SUPPORT
In-Reply-To: <1459857443-20611-1-git-send-email-tom.leiming@gmail.com>
Signed-off-by: Ming Lei <tom.leiming@gmail.com>
---
drivers/md/bcache/io.c | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/drivers/md/bcache/io.c b/drivers/md/bcache/io.c
index 86a0bb8..1c48462 100644
--- a/drivers/md/bcache/io.c
+++ b/drivers/md/bcache/io.c
@@ -26,8 +26,7 @@ struct bio *bch_bbio_alloc(struct cache_set *c)
bio_init(bio);
bio->bi_flags |= BIO_POOL_NONE << BIO_POOL_OFFSET;
- bio->bi_max_vecs = bucket_pages(c);
- bio->bi_io_vec = bio->bi_inline_vecs;
+ bio_set_vec_table(bio, bio->bi_inline_vecs, bucket_pages(c));
return bio;
}
--
1.9.1
^ permalink raw reply related
* [PATCH 10/27] bcache: debug: avoid to access .bi_io_vec directly
From: Ming Lei @ 2016-04-05 11:56 UTC (permalink / raw)
To: Jens Axboe, linux-kernel
Cc: linux-block, Christoph Hellwig, Boaz Harrosh, Ming Lei,
Kent Overstreet, Shaohua Li, open list:BCACHE BLOCK LAYER CACHE,
open list:SOFTWARE RAID Multiple Disks SUPPORT
In-Reply-To: <1459857443-20611-1-git-send-email-tom.leiming@gmail.com>
Instead we use standard iterator way to do that.
Signed-off-by: Ming Lei <tom.leiming@gmail.com>
---
drivers/md/bcache/debug.c | 11 ++++++++---
1 file changed, 8 insertions(+), 3 deletions(-)
diff --git a/drivers/md/bcache/debug.c b/drivers/md/bcache/debug.c
index 8b1f1d5..d1ad49d 100644
--- a/drivers/md/bcache/debug.c
+++ b/drivers/md/bcache/debug.c
@@ -106,8 +106,8 @@ void bch_data_verify(struct cached_dev *dc, struct bio *bio)
{
char name[BDEVNAME_SIZE];
struct bio *check;
- struct bio_vec bv, *bv2;
- struct bvec_iter iter;
+ struct bio_vec bv, cbv, *bv2;
+ struct bvec_iter iter, citer = { 0 };
int i;
check = bio_clone(bio, GFP_NOIO);
@@ -119,9 +119,13 @@ void bch_data_verify(struct cached_dev *dc, struct bio *bio)
submit_bio_wait(READ_SYNC, check);
+ citer.bi_size = UINT_MAX;
bio_for_each_segment(bv, bio, iter) {
void *p1 = kmap_atomic(bv.bv_page);
- void *p2 = page_address(check->bi_io_vec[iter.bi_idx].bv_page);
+ void *p2;
+
+ cbv = bio_iter_iovec(check, citer);
+ p2 = page_address(cbv.bv_page);
cache_set_err_on(memcmp(p1 + bv.bv_offset,
p2 + bv.bv_offset,
@@ -132,6 +136,7 @@ void bch_data_verify(struct cached_dev *dc, struct bio *bio)
(uint64_t) bio->bi_iter.bi_sector);
kunmap_atomic(p1);
+ bio_advance_iter(check, &citer, bv.bv_len);
}
bio_for_each_segment_all(bv2, check, i)
--
1.9.1
^ permalink raw reply related
* [PATCH 00/27] block: cleanup direct access on .bi_vcnt & .bi_io_vec
From: Ming Lei @ 2016-04-05 11:56 UTC (permalink / raw)
To: Jens Axboe, linux-kernel
Cc: linux-block, Christoph Hellwig, Boaz Harrosh, Ming Lei, Al Viro,
Andreas Dilger, Andrew Morton, open list:STAGING SUBSYSTEM,
open list:DEVICE-MAPPER LVM, open list:DRBD DRIVER, Frank Zago,
Greg Kroah-Hartman, Hannes Reinecke, James Simmons, Jan Kara,
Jarod Wilson, Jiri Kosina, Joe Perches, John L. Hammond,
Julia Lawall, Keith Busch, Kent Overstreet, linux-bcache
Hi Guys,
It is always not a good practice to access bio->bi_vcnt and
bio->bi_io_vec from drivers directly. Also this kind of direct
access will cause trouble when converting to multipage bvecs.
The 1st patch introduces the following 4 bio helpers which can be
used inside drivers for avoiding direct access to .bi_vcnt and .bi_io_vec.
bio_pages()
bio_is_full()
bio_get_base_vec()
bio_set_vec_table()
Both bio_pages() and bio_is_full() can be easy to convert to
multipage bvecs.
For bio_get_base_vec() and bio_set_vec_table(), they are often used
during initializing a new bio or in case of single bvec bio. With the
two new helpers, it becomes quite easy to audit access to .bi_io_vec
and .bi_vcnt.
Most of the other patches use the 4 helpers to clean up most of direct
access to .bi_vcnt and .bi_io_vec from drivers, except for MD and btrfs,
which two subsystems will be done in the future.
Also bio_add_page() is used in floppy, dm-crypt and fs/logfs to
avoiding direct access to .bi_vcnt & .bi_io_vec.
Thanks,
Ming
Ming Lei (27):
block: bio: introduce 4 helpers for cleanup
block: drbd: use bio_get_base_vec() to retrieve the 1st bvec
block: drbd: remove impossible failure handling
block: loop: use bio_get_base_vec() to retrive bvec table
block: pktcdvd: use bio_get_base_vec() to retrive bvec table
block: floppy: use bio_set_vec_table()
block: floppy: use bio_add_page()
staging: lustre: avoid to use bio->bi_vcnt directly
target: use bio_is_full()
bcache: debug: avoid to access .bi_io_vec directly
bcache: io.c: use bio_set_vec_table
bcache: journal.c: use bio_set_vec_table()
bcache: movinggc: use bio_set_vec_table()
bcache: writeback: use bio_set_vec_table()
bcache: super: use bio_set_vec_table()
bcache: super: use bio_get_base_vec
dm: crypt: use bio_add_page()
dm: dm-io.c: use bio_get_base_vec()
dm: dm.c: replace 'bio->bi_vcnt == 1' with !bio_multiple_segments
dm: dm-bufio.c: use bio_set_vec_table()
fs: logfs: use bio_set_vec_table()
fs: logfs: convert to bio_add_page() in sync_request()
fs: logfs: use bio_add_page() in __bdev_writeseg()
fs: logfs: use bio_add_page() in do_erase()
fs: logfs: remove unnecesary check
kernel/power/swap.c: use bio_get_base_vec()
mm: page_io.c: use bio_get_base_vec()
drivers/block/drbd/drbd_bitmap.c | 4 +-
drivers/block/drbd/drbd_receiver.c | 14 +---
drivers/block/floppy.c | 9 +--
drivers/block/loop.c | 5 +-
drivers/block/pktcdvd.c | 3 +-
drivers/md/bcache/debug.c | 11 ++-
drivers/md/bcache/io.c | 3 +-
drivers/md/bcache/journal.c | 3 +-
drivers/md/bcache/movinggc.c | 6 +-
drivers/md/bcache/super.c | 28 +++++---
drivers/md/bcache/writeback.c | 4 +-
drivers/md/dm-bufio.c | 3 +-
drivers/md/dm-crypt.c | 8 +--
drivers/md/dm-io.c | 7 +-
drivers/md/dm.c | 3 +-
drivers/staging/lustre/lustre/llite/lloop.c | 9 +--
drivers/target/target_core_pscsi.c | 2 +-
fs/logfs/dev_bdev.c | 107 +++++++++++-----------------
include/linux/bio.h | 28 ++++++++
kernel/power/swap.c | 10 ++-
mm/page_io.c | 18 ++++-
21 files changed, 156 insertions(+), 129 deletions(-)
--
1.9.1
^ permalink raw reply
* Re: recovery from selinux blocking --backup-file during RAID5->6
From: Wols Lists @ 2016-04-05 11:14 UTC (permalink / raw)
To: Noah Beck, linux-raid
In-Reply-To: <CAFAW0zxisX2Db_YdJxtNpF4YKayy=AnKnWcGdAgw6piy1Jjyng@mail.gmail.com>
DO YOU HAVE A BACKUP :-)
Actually, I think this doesn't sound too serious, if it's just the spare
that's failed, you've still got redundancy. If you'd lost an active
drive I'd be screaming "backups!!! backups!!! backups!!!".
My reaction is simply to replace the failed drive and then carry on the
conversion, but I'm not an expert.
Thing is, when one drive fails, it should be ringing alarm bells that
another one is on its last legs - these things have an annoying habit of
failing in bunches. Which says that you really need a *second* spare
drive handy - is the rebuild going to tip one of your live drives over
the edge? I'd say the chances of you ending up with a 5-device raid-6
with one device failed is a lot higher than you'd like :-(
Cheers,
Wol
On 05/04/16 04:16, Noah Beck wrote:
> I see a similar problem has been discussed at least once before at
> https://marc.info/?t=144970286700004&r=1&w=2
>
> In my case, this was a RAID5 array with 4 active devices and one
> spare. I wanted to switch this to a 5-device RAID6 instead. Ran the
> following:
>
> mdadm --grow /dev/md127 --raid-devices 5 --level 6 --backup-file
> /root/raid_migration_file
>
> Two things went wrong:
>
> 1) selinux jumped in and blocked access to the --backup-file. From journalctl:
>
> SELinux is preventing mdadm from getattr access on the file
> /root/raid_migration_file
>
> This can be fixed with a "setenforce 0" in the future. The
> /root/raid_migration_file did get created (25MB) but hexdump says it
> is all zeros so I believe no useful data was placed in this file.
>
> 2) Turns out my spare device in the old RAID5 was actually ready to
> die. This corresponds to what was previously the spare in my RAID5:
>
> ata4.00: revalidation failed (errno=-2)
> ata4.00: disabled
> ata4: EH complete
> blk_update_request: I/O error, dev sdb, sector 0
> blk_update_request: I/O error, dev sdb, sector 3907023935
> md: super_written gets error=-5
> md/raid:md127: Disk failure on sdb1, disabling device.
> md/raid:md127: Operation continuing on 4 devices.
>
> Since /dev/sdb1 was marked as failed in the array I removed it. I
> tried zeroing it out with dd if=/dev/zero of=/dev/sdb1 to see what it
> would do and then that disk completely died. So I'll get a new disk
> tomorrow. In the meantime the system still seems to be running fine.
> /proc/mdstat shows this now:
>
> md127 : active raid6 sde1[3] sda1[2] sdd1[0] sdf1[1]
> 5860535808 blocks super 0.91 level 6, 64k chunk, algorithm 18
> [5/4] [UUUU_]
> [>....................] reshape = 0.0% (1/1953511936)
> finish=722.0min speed=43680K/sec
>
> The previous thread resulted in a patch (in
> https://marc.info/?l=linux-raid&m=145187378405337&w=2 ). If I want to
> go back to having a 4-device RAID5 array before I shut this system
> down to replace the bad disk, is the right thing to do still to apply
> that patch to mdadm, stop /dev/md127, and assemble again with
> --update=revert-reshape? Or does the info above indicate I should use
> any different solution?
>
> Thanks,
> Noah
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
^ permalink raw reply
* Re: Repairing R1: Part tabl, & precise command
From: Ron Leach @ 2016-04-05 11:04 UTC (permalink / raw)
To: Linux RAID Mailing List
In-Reply-To: <57038211.6040101@tesco.net>
On 05/04/2016 10:14, Ron Leach wrote:
>
> Presumably, I need to set up the partition table again. These are
> identical discs from the same manufacturer. Elsewhere, people have
> suggested using:
>
> sfdisk -d /dev/sdc | sfdisk /dev/sdb
>
(Apologies for replying to my own post.)
Everybody will have noticed how ill-prepared I am;
man sfdisk
explains, right at the start, that sfdisk will NOT work with gpt
partitions.
I'll look at parted and gdisk.
Ron
^ permalink raw reply
* Repairing R1: Part tabl, & precise command
From: Ron Leach @ 2016-04-05 9:14 UTC (permalink / raw)
To: Linux-RAID
List, good morning,
We use a 2 x 3TB Raid 1 configuration in a Debian Oldstable (Wheezy)
machine employed for backup and one of the Raid pair has dropped off
the array. An lsdrv report [1] is pasted below; sdb has failed.
Oddly, the offending disc seems to not have a partition table, now,
either. The backup space occupied an LVM space on a 2.5TB R1 array;
the discs also have other R1 arrays. I haven't altered any of the
information on the surviving element of the raid1-lvm, which seems to
be (read) functioning and with its data complete.
I've re-seated cables and power connectors and read-tested the failed
disc using
dd if=/dev/sdb of=/dev/null bs=1M
which, after running overnight, reported the whole (suspect) disc read
without any errors.
I'd like to try to restore the array using the previously-failed disk,
temporarily, while another drive arrives from suppliers. I'm not sure
of the precise mechanism for restoring the array. I think the same
process will be needed when the replacement disk - which will be
blank, too - arrives.
Presumably, I need to set up the partition table again. These are
identical discs from the same manufacturer. Elsewhere, people have
suggested using:
sfdisk -d /dev/sdc | sfdisk /dev/sdb
Will this be ok for mdadm or will this command also replicate some
UUIDs or headers or partition content that mdadm prefers should be
kept unique?
Having prepared the partition table on sdb, is the next step a simple
mdadm --manage /dev/md(x) --add /dev/sdb(y)
sequence of commands? Do I need to disable any attempt by mdadm to
rebuild itself?
Would be grateful for any pointers to anything incorrect I'm
proposing. Though it is 'only' backup material on the disks, to us it
is pretty important because it was incremental and also therefore is
the repository of anything accidentally deleted since.
regards, Ron
[1] lsdrv output
root@D7bak:/home/user# ./lsdrv
PCI [ata_piix] 00:1f.2 IDE interface: Intel Corporation NM10/ICH7
Family SATA Controller [IDE mode] (rev 01)
-scsi 0:0:1:0 ATA WDC WD2500AAKX-2 {WD-WCC2ED752256}
-:sda 232.88g [8:0] Partitioned (dos)
- -sda1 17.27g [8:1] ext2 {ac6943f4-24dd-4605-ab97-9633bdf4aa0f}
- -sda2 1.00k [8:2] Partitioned (dos)
- -sda5 1.86g [8:5] ext4 {d0f0831b-b67b-43c7-869a-74ccd2b82f0a}
- -:Mounted as /dev/disk/by-uuid/d0f0831b-b67b-43c7-869a-74ccd2b82f0a @ /
- -sda6 13.97g [8:6] ext4 {777a0299-5756-4bb4-ae86-4b968a25ed20}
- -:Mounted as /dev/sda6 @ /usr
- -sda7 23.28g [8:7] ext4 {519b7c6e-cc14-4b4e-b8cb-8d9573da48fb}
- -:Mounted as /dev/sda7 @ /var
- -sda8 2.05g [8:8] swap {c1527154-1cfe-42c9-bc00-780fda199508}
- -sda9 2.79g [8:9] ext4 {ca294bd6-4c95-4008-8531-1f467d2a3d7b}
- -:Mounted as /dev/sda9 @ /tmp
- :sda10 79.16g [8:10] ext4 {1a4793f3-cb46-4256-88f6-f065b4c49d88}
- :Mounted as /dev/sda10 @ /home
-scsi 1:0:0:0 ATA WDC WD30EZRX-00D {WD-WMC1T4003426}
-:sdb 2.73t [8:16] Empty/Unknown
:scsi 1:0:1:0 ATA WDC WD30EZRX-00D {WD-WMC1T3894559}
:sdc 2.73t [8:32] Partitioned (gpt)
-sdc1 1.00m [8:33] Empty/Unknown
-sdc2 100.00m [8:34] MD raid1 (1/2) in_sync
{cc72a049-f9e6-7085-4a8f-b478c8ee9588}
-:md2 99.94m [9:2] MD v0.90 raid1 (2) read-auto DEGRADED
{cc72a049:f9e67085:4a8fb478:c8ee9588}
- ext2 'boot' {e211fdf8-4179-4d52-b994-61e958789b6c}
-sdc3 2.00g [8:35] Empty/Unknown
-sdc4 150.00g [8:36] MD raid1 (1/2) in_sync 'D7bak:4'
{741fb6b3-491b-f823-ba5d-bb377101ce96}
-:md4 149.87g [9:4] MD v1.2 raid1 (2) read-auto DEGRADED
{741fb6b3:491bf823:ba5dbb37:7101ce96}
- ext4 'OS' {618eae7a-2a6b-44fd-90d4-74595d3f24bd}
:sdc5 2.58t [8:37] MD raid1 (1/2) in_sync 'D7bak:5'
{0a1bd77e-6f0d-4fba-3260-932c021ed347}
:md5 2.58t [9:5] MD v1.2 raid1 (2) clean DEGRADED
{0a1bd77e:6f0d4fba:3260932c:021ed347}
- PV LVM2_member 2.58t used, 0 free
{5b0KRp-rFJ3-WiBR-JW3i-U01v-SITm-fNcVbr}
:VG bkp100vg 2.58t 0 free {zWgmjF-zYiv-X9fp-0XCU-RFIf-kT8E-7bropr}
:dm-0 2.58t [253:0] LV bkp100lv ext4
{709a00ef-9306-4617-b464-4f30a4790f60}
:Mounted as /dev/mapper/bkp100vg-bkp100lv @ /mnt/bkp
Other Block Devices
-loop0 0.00k [7:0] Empty/Unknown
-loop1 0.00k [7:1] Empty/Unknown
-loop2 0.00k [7:2] Empty/Unknown
-loop3 0.00k [7:3] Empty/Unknown
-loop4 0.00k [7:4] Empty/Unknown
-loop5 0.00k [7:5] Empty/Unknown
-loop6 0.00k [7:6] Empty/Unknown
-loop7 0.00k [7:7] Empty/Unknown
root@D7bak:/home/user#
^ permalink raw reply
* Re: [PATCH v2 1/4] scatterlist: Introduce some helper functions
From: Baolin Wang @ 2016-04-05 7:10 UTC (permalink / raw)
To: Robert Jarzmik
Cc: Herbert Xu, David Miller, Alasdair G Kergon, Mike Snitzer,
Jens Axboe, dm-devel, Andrew Morton, david.s.gordon, Tom Lendacky,
Masahiro Yamada, smueller, tadeusz.struk, Masanari Iida, shli,
Mark Brown, Linus Walleij, Arnd Bergmann, LKML, linux-crypto,
linux-raid
In-Reply-To: <87r3eoukb9.fsf@belgarion.home>
Hi Robert,
Sorry for the late reply.
On 2 April 2016 at 23:00, Robert Jarzmik <robert.jarzmik@free.fr> wrote:
> Baolin Wang <baolin.wang@linaro.org> writes:
>
>> +/**
>> + * sg_is_contiguous - Check if the scatterlists are contiguous
>> + * @sga: SG entry
>> + * @sgb: SG entry
>> + *
>> + * Description:
>> + * If the sga scatterlist is contiguous with the sgb scatterlist,
>> + * that means they can be merged together.
>> + *
>> + **/
>> +static inline bool sg_is_contiguous(struct scatterlist *sga,
>> + struct scatterlist *sgb)
>> +{
>> + return *(unsigned long *)sg_virt(sga) + sga->length ==
>> + *(unsigned long *)sg_virt(sgb);
> As I already said, I don't like casts.
OK. Could you give me a good example. Thanks.
>
> But let's take some height : you're needing this function to decide to merge
> scatterlists. That means that you think the probability of having 2 scatterlist
> mergeable is not meaningless, ie. 50% or more.
>
> I suppose your scatterlists are allocated through kmalloc(). I'd like to know,
> through your testing, what is the success rate of sg_is_contiguous(), ie. I'd
> like to know how many times sg_is_contiguous() was called, and amongst these
> calls how many times it returned true.
>
> That will tell me how "worth" is this optimization.
I think this is just one useful API for users, If the rate is only 1%,
we also need to check if they are contiguous to decide if they can be
coalesced.
>
>> + * sg_add_sg_to_table - Add one scatterlist into sg table
>> + * @sgt: The sg table header to use
>> + * @src: The sg need to be added into sg table
>> + *
>> + * Description:
>> + * The 'nents' member indicates how many mapped scatterlists has been added
>> + * in the dynamic sg table. The 'orig_nents' member indicates the size of the
>> + * dynamic sg table.
>> + *
>> + * Copy one mapped @src@ scatterlist into the dynamic sg table and increase
>> + * 'nents' member.
>> + *
>> + **/
> Okay, I still believe this one is wrong, because we don't understand each other.
> So let's take an example :
> sg_table = {
> .sgl = {
> {
> .page_link = PAGE_48,
> .offset = 0,
> .length = 2048,
> .dma_address = 0x30000,
> .dma_length = 4096,
> },
> {
> .page_link = PAGE_48 | 0x02,
> .offset = 2048,
> .length = 2048,
> .dma_address = 0,
> .dma_length = 0,
> },
> },
> .nents = 1,
> .orig_nents = 2,
> };
>
> In this example, by sheer luck the 2 scatterlist entries were physically
> contiguous, and the mapping function coallesced the 2 entries into only one
> (dma_address, dma_length) entry. That could also happen with an IOMMU by the
> way. Therefore, sg_table.nents = 1.
>
> If I understand your function correctly, it will erase sg_table.sgl[1], and that
> looks incorrect to me. This is why I can't understand how your code can be
> correct, and why I say you add a new "meaning" to sg_table->nents, which is not
> consistent with the meaning I understand.
I think you misunderstood my point. The 'sg_add_sg_to_table()'
function is used to add one mapped scatterlist into the dynamic sg
table. For example:
1. How to add one mapped sg into dynamic sg table
(1) If we allocate one dynamic sg table with 3 (small size can be
showed easily) empty scatterlists.:
sg_table = {
.sgl = {
{
.page_link = 0,
.offset = 0,
.length = 0,
},
{
.page_link = 0,
.offset = 0,
.length = 0,
},
{
.page_link = 0 | 0x02,
.offset = 0,
.length = 0,
},
},
.nents = 0,
.orig_nents = 3,
};
(2) If some module (like dm-crypt module) always sends one mapped
scatterlist (size is always 512bytes) to another module (crypto
driver) to handle the mapped scatterlist at one time. But we want to
collect the mapped scatterlist into one dynamic sg table to manage
them together, means we can send multiple mapped scatterlists (from
sg_table->sgl) to the crypto driver to handle them together to improve
its efficiency. So we add one mapped scatterlists into the dynamic sg
table.
mapped sg = {
.page_link = PAGE_20,
.offset = 0,
.length = 512,
},
Add into synamic sg table ------->
sg_table = {
.sgl = {
{
.page_link = PAGE_20 | 0x02,
.offset = 0,
.length = 512,
},
{
.page_link = 0,
.offset = 0,
.length = 0,
},
{
.page_link = 0,
.offset = 0,
.length = 0,
},
},
.nents = 1,
.orig_nents = 3,
};
Another two mapped scatterlists are added into synamic sg table ------->
sg_table = {
.sgl = {
{
.page_link = PAGE_20,
.offset = 0,
.length = 512,
},
{
.page_link = PAGE_25,
.offset = 0,
.length = 512,
},
{
.page_link = PAGE_28 | 0x20,
.offset = 0,
.length = 512,
},
},
.nents = 3,
.orig_nents = 3,
};
Then we can send the dynamic sg table to the crypto engine driver to
handle them together at one time. (If the dynamic sg table size is
512, then the crypto engine driver can handle 512 scatterlists at one
time, which can improve much efficiency). That's the reason why we
want to introduce the dynamic sg table.
2. How to coalesce
(1) If one mapped scatterlsit already has been added into dynamic sg table:
sg_table = {
.sgl = {
{
.page_link = PAGE_20 | 0x02,
.offset = 0,
.length = 512,
},
{
.page_link = 0,
.offset = 0,
.length = 0,
},
{
.page_link = 0,
.offset = 0,
.length = 0,
},
},
.nents = 1,
.orig_nents = 3,
};
(2) Another mapped scatterlist comes.
mapped sg = {
.page_link = PAGE_20,
.offset = 512,
.length = 512,
},
So we check the new mapped scatterlist can be coalesced into previous
one in dynamic sg table like this:
sg_table = {
.sgl = {
{
.page_link = PAGE_20 | 0x02,
.offset = 0,
.length = 1024,
},
{
.page_link = 0,
.offset = 0,
.length = 0,
},
{
.page_link = 0,
.offset = 0,
.length = 0,
},
},
.nents = 1,
.orig_nents = 3,
};
It's done. We coalesced one scatterlist into antoher one to expand the
scatterlist's length.
Thanks for your comments.
>
> Cheers.
>
> --
> Robert
--
Baolin.wang
Best Regards
^ permalink raw reply
* recovery from selinux blocking --backup-file during RAID5->6
From: Noah Beck @ 2016-04-05 3:16 UTC (permalink / raw)
To: linux-raid
I see a similar problem has been discussed at least once before at
https://marc.info/?t=144970286700004&r=1&w=2
In my case, this was a RAID5 array with 4 active devices and one
spare. I wanted to switch this to a 5-device RAID6 instead. Ran the
following:
mdadm --grow /dev/md127 --raid-devices 5 --level 6 --backup-file
/root/raid_migration_file
Two things went wrong:
1) selinux jumped in and blocked access to the --backup-file. From journalctl:
SELinux is preventing mdadm from getattr access on the file
/root/raid_migration_file
This can be fixed with a "setenforce 0" in the future. The
/root/raid_migration_file did get created (25MB) but hexdump says it
is all zeros so I believe no useful data was placed in this file.
2) Turns out my spare device in the old RAID5 was actually ready to
die. This corresponds to what was previously the spare in my RAID5:
ata4.00: revalidation failed (errno=-2)
ata4.00: disabled
ata4: EH complete
blk_update_request: I/O error, dev sdb, sector 0
blk_update_request: I/O error, dev sdb, sector 3907023935
md: super_written gets error=-5
md/raid:md127: Disk failure on sdb1, disabling device.
md/raid:md127: Operation continuing on 4 devices.
Since /dev/sdb1 was marked as failed in the array I removed it. I
tried zeroing it out with dd if=/dev/zero of=/dev/sdb1 to see what it
would do and then that disk completely died. So I'll get a new disk
tomorrow. In the meantime the system still seems to be running fine.
/proc/mdstat shows this now:
md127 : active raid6 sde1[3] sda1[2] sdd1[0] sdf1[1]
5860535808 blocks super 0.91 level 6, 64k chunk, algorithm 18
[5/4] [UUUU_]
[>....................] reshape = 0.0% (1/1953511936)
finish=722.0min speed=43680K/sec
The previous thread resulted in a patch (in
https://marc.info/?l=linux-raid&m=145187378405337&w=2 ). If I want to
go back to having a 4-device RAID5 array before I shut this system
down to replace the bad disk, is the right thing to do still to apply
that patch to mdadm, stop /dev/md127, and assemble again with
--update=revert-reshape? Or does the info above indicate I should use
any different solution?
Thanks,
Noah
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox