* [GIT PULL REQUEST] md bugfix
From: Neil Brown @ 2015-10-11 5:30 UTC (permalink / raw)
To: Linus Torvalds; +Cc: linux-raid, LKML, Mikulas Patocka
[-- Attachment #1: Type: text/plain, Size: 840 bytes --]
The following changes since commit da6fb7a9e5bd6f04f7e15070f630bdf1ea502841:
md/bitmap: don't pass -1 to bitmap_storage_alloc. (2015-10-02 17:24:13 +1000)
are available in the git repository at:
git://neil.brown.name/md tags/md/4.3-rc4-fix
for you to fetch changes up to a452744bcbf706eac65abb4c98496a366820c60a:
crash in md-raid1 and md-raid10 due to incorrect list manipulation (2015-10-09 08:33:46 +1100)
----------------------------------------------------------------
One bug fix for raid1/raid10.
Very careless bug earler in 4.3-rc, now fixed :-)
----------------------------------------------------------------
Mikulas Patocka (1):
crash in md-raid1 and md-raid10 due to incorrect list manipulation
drivers/md/raid1.c | 4 ++--
drivers/md/raid10.c | 4 ++--
2 files changed, 4 insertions(+), 4 deletions(-)
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 818 bytes --]
^ permalink raw reply
* Re: 30c2a1faaeb3db94fc92f79553cc72634aa3b218 broke cryptsetup on my machine
From: Mike Snitzer @ 2015-10-10 13:00 UTC (permalink / raw)
To: Giedrius Statkevičius; +Cc: Geliang Tang, linux-raid, linux-kernel
In-Reply-To: <alpine.LNX.2.20.1510101543410.1998@localhost.localdomain>
On Sat, Oct 10 2015 at 8:44P -0400,
Giedrius Statkevičius <giedrius.statkevicius@gmail.com> wrote:
> On Thu, 8 Oct 2015, Geliang Tang wrote:
>
> > On Wed, Oct 07, 2015 at 10:27:27PM +0300, Giedrius Statkevičius wrote:
> > > Hello,
> > > I have a LUKS on LVM setup: /boot is unencrypted and everything is "hidden" in
> > > /dev/sda2. After booting on linux-next and just after entering my password
> > > cryptsetup segfaults and a stack trace is printed from the kernel. Since writing
> > > down all those numbers is hard I've made a picture where it is shown:
> > > https://i.imgur.com/6PHNUdv.jpg
> > >
> > > I figured it had something to do with changes to memory management and thus find
> > > out that commit 30c2a1faaeb3db94fc92f79553cc72634aa3b218 broke my system.
> > > Reverting it there is no such error anymore and system boots fine past that
> > > point.
> > >
> > > Could someone look into this? Thanks, Giedrius
> >
> > Bugfix for this has been sended out.
> > https://lkml.org/lkml/2015/10/8/342.
> >
> > Thanks.
> I've tested and that fixed this issue for me. I would've replied on that thread
> but I can't find its Message-Id. Anyway, you can add:
> Tested-By: Giedrius Statkevičius <giedrius.statkevicius@gmail.com>
I dropped the original commit because I'm really not seeing the value of
using kvfree (but it was motivated by reports of crashes from ktest,
etc). I don't have any intention of picking it back up at this point
(so Geliang's fix will go unused at this point).
Mike
^ permalink raw reply
* Re: 30c2a1faaeb3db94fc92f79553cc72634aa3b218 broke cryptsetup on my machine
From: Giedrius Statkevičius @ 2015-10-10 12:44 UTC (permalink / raw)
To: Geliang Tang
Cc: Giedrius Statkevičius, snitzer, linux-raid, linux-kernel
In-Reply-To: <20151008123849.GA4279@ubuntu>
[-- Attachment #1: Type: text/plain, Size: 1112 bytes --]
On Thu, 8 Oct 2015, Geliang Tang wrote:
> On Wed, Oct 07, 2015 at 10:27:27PM +0300, Giedrius Statkevičius wrote:
> > Hello,
> > I have a LUKS on LVM setup: /boot is unencrypted and everything is "hidden" in
> > /dev/sda2. After booting on linux-next and just after entering my password
> > cryptsetup segfaults and a stack trace is printed from the kernel. Since writing
> > down all those numbers is hard I've made a picture where it is shown:
> > https://i.imgur.com/6PHNUdv.jpg
> >
> > I figured it had something to do with changes to memory management and thus find
> > out that commit 30c2a1faaeb3db94fc92f79553cc72634aa3b218 broke my system.
> > Reverting it there is no such error anymore and system boots fine past that
> > point.
> >
> > Could someone look into this? Thanks, Giedrius
>
> Bugfix for this has been sended out.
> https://lkml.org/lkml/2015/10/8/342.
>
> Thanks.
I've tested and that fixed this issue for me. I would've replied on that thread
but I can't find its Message-Id. Anyway, you can add:
Tested-By: Giedrius Statkevičius <giedrius.statkevicius@gmail.com>
Thanks,
Giedrius
^ permalink raw reply
* Re: [parted-devel] [PATCH 1/2] Add nonfs partition type 0xda (#1263835)
From: Brian C. Lane @ 2015-10-09 18:35 UTC (permalink / raw)
To: Phil Susi; +Cc: parted-devel, linux-raid
In-Reply-To: <5617F80C.5050407@ubuntu.com>
On Fri, Oct 09, 2015 at 01:23:24PM -0400, Phil Susi wrote:
> On 10/9/2015 12:21 PM, Brian C. Lane wrote:
> > I think their thinking is that 0x80 implies a filesystem.
> >
> > I really don't see any harm in adding it, if you don't want to use it,
> > don't.
>
> I don't like adding things that serve no purpose and only confuse users.
> People end up running around with their hair on fire asking which type
> they should use, and if it just doesn't bloody matter, why confuse them
> and make the manual longer by giving them the choice? Keep it simple.
>
> To put it another way, seeing the option there makes people think they
> need to use it for some reason, after all, why else would it be there?
> And if that isn't true, then you have just created FUD.
>
It does have purpose, as described in the bug. And most users won't pay
any attention to it. The vast majority of them care about 'boot' and
that's it.
--
Brian C. Lane | Anaconda Team | IRC: bcl #anaconda | Port Orchard, WA (PST8PDT)
^ permalink raw reply
* Re: [parted-devel] [PATCH 1/2] Add nonfs partition type 0xda (#1263835)
From: Phil Susi @ 2015-10-09 17:23 UTC (permalink / raw)
To: Brian C. Lane; +Cc: parted-devel, linux-raid
In-Reply-To: <20151009162111.GR2984@lister.brianlane.com>
On 10/9/2015 12:21 PM, Brian C. Lane wrote:
> I think their thinking is that 0x80 implies a filesystem.
>
> I really don't see any harm in adding it, if you don't want to use it,
> don't.
I don't like adding things that serve no purpose and only confuse users.
People end up running around with their hair on fire asking which type
they should use, and if it just doesn't bloody matter, why confuse them
and make the manual longer by giving them the choice? Keep it simple.
To put it another way, seeing the option there makes people think they
need to use it for some reason, after all, why else would it be there?
And if that isn't true, then you have just created FUD.
^ permalink raw reply
* Re: [PATCH 1/2] Add nonfs partition type 0xda (#1263835)
From: Brian C. Lane @ 2015-10-09 16:21 UTC (permalink / raw)
To: Phil Susi
Cc: linux-raid-u79uwXL29TY76Z2rM5mHXA,
parted-devel-XbBxUvOt3X2LieD7tvxI8l/i77bcL1HB
In-Reply-To: <5617D701.1010109-GeWIH/nMZzLQT0dZR+AlfA@public.gmane.org>
On Fri, Oct 09, 2015 at 11:02:25AM -0400, Phil Susi wrote:
> On 10/8/2015 5:49 PM, Brian C. Lane wrote:
> > https://bugzilla.redhat.com/show_bug.cgi?id=1118065 has a pile of
> > discussion on it.
> >
> > What it amounts to is that on msdos partitioned disks mdadm only wants
> > 0xfd to be used for version 0.90 arrays which the kernel can
> > autoassemble, for 1.x arrays some other type needs to be used.
>
> So use 0x80? That's what I always have done and it has never been a
> problem. Likewise, 0xfd won't cause a problem either: it will trigger
> kernel auto activation *if* you boot with no initramfs, *and* the
> partition does contain 0.90 metadata. If it has 1.x, then it won't
> trigger auto activation, but has no other negative consequence.
>
> I see from the discussion in that bug report that the mdadm man page
> recommends this type code. Is there any reason for that recommendation?
> Are there any consequences to not following it? Cc'ing mdadm list to
> see if they can answer this.
>
I think their thinking is that 0x80 implies a filesystem.
I really don't see any harm in adding it, if you don't want to use it,
don't.
--
Brian C. Lane | Anaconda Team | IRC: bcl #anaconda | Port Orchard, WA (PST8PDT)
^ permalink raw reply
* Re: [parted-devel] [PATCH 1/2] Add nonfs partition type 0xda (#1263835)
From: Phil Susi @ 2015-10-09 15:02 UTC (permalink / raw)
To: Brian C. Lane; +Cc: parted-devel, linux-raid
In-Reply-To: <20151008214916.GK2984@lister.brianlane.com>
On 10/8/2015 5:49 PM, Brian C. Lane wrote:
> https://bugzilla.redhat.com/show_bug.cgi?id=1118065 has a pile of
> discussion on it.
>
> What it amounts to is that on msdos partitioned disks mdadm only wants
> 0xfd to be used for version 0.90 arrays which the kernel can
> autoassemble, for 1.x arrays some other type needs to be used.
So use 0x80? That's what I always have done and it has never been a
problem. Likewise, 0xfd won't cause a problem either: it will trigger
kernel auto activation *if* you boot with no initramfs, *and* the
partition does contain 0.90 metadata. If it has 1.x, then it won't
trigger auto activation, but has no other negative consequence.
I see from the discussion in that bug report that the mdadm man page
recommends this type code. Is there any reason for that recommendation?
Are there any consequences to not following it? Cc'ing mdadm list to
see if they can answer this.
^ permalink raw reply
* [PATCH] md: fix 32-bit build warning
From: Arnd Bergmann @ 2015-10-09 12:12 UTC (permalink / raw)
To: Neil Brown; +Cc: linux-raid, linux-kernel, Goldwyn Rodrigues, linux-arm-kernel
On 32-bit architectures, the md code produces this warning when CONFIG_LDAF
is set:
drivers/md/md.c: In function 'check_sb_changes':
drivers/md/md.c:8990:10: warning: format '%lu' expects argument of type 'long unsigned int', but argument 4 has type 'sector_t {aka long long unsigned int}' [-Wformat=]
pr_info("%s:%d recovery_cp changed from %lu to %lu\n", __func__,
The code was only recently introduced, and uses the wrong format string
for sector_t. As a workaround, this patch adds an explicit cast to 'u64'
so we can use the %llu format string on all architectures.
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Fixes: e0212320066e ("md-cluster: Improve md_reload_sb to be less error prone")
Cc: Goldwyn Rodrigues <rgoldwyn@suse.com>
---
I also noticed that some commmits in md/for-next including the one causing
the problem lack a Signed-off-by line. It might make sense to just fold this
patch and add the lines at the same time.
diff --git a/drivers/md/md.c b/drivers/md/md.c
index 7fff1e6884d6..e13f72a3b561 100644
--- a/drivers/md/md.c
+++ b/drivers/md/md.c
@@ -8987,9 +8987,9 @@ static void check_sb_changes(struct mddev *mddev, struct md_rdev *rdev)
/* recovery_cp changed */
if (le64_to_cpu(sb->resync_offset) != mddev->recovery_cp) {
- pr_info("%s:%d recovery_cp changed from %lu to %lu\n", __func__,
- __LINE__, mddev->recovery_cp,
- (unsigned long) le64_to_cpu(sb->resync_offset));
+ pr_info("%s:%d recovery_cp changed from %llu to %llu\n", __func__,
+ __LINE__, (u64)mddev->recovery_cp,
+ (u64) le64_to_cpu(sb->resync_offset));
mddev->recovery_cp = le64_to_cpu(sb->resync_offset);
}
^ permalink raw reply related
* [PATCH v2 6/6] Add help message and man entry for --write-journal
From: Song Liu @ 2015-10-09 5:51 UTC (permalink / raw)
To: linux-raid; +Cc: neilb, shli, hch, dan.j.williams, kernel-team, Song Liu
In-Reply-To: <1444369906-1804157-1-git-send-email-songliubraving@fb.com>
Signed-off-by: Song Liu <songliubraving@fb.com>
Signed-off-by: Shaohua Li <shli@fb.com>
---
ReadMe.c | 1 +
mdadm.8.in | 6 ++++++
2 files changed, 7 insertions(+)
diff --git a/ReadMe.c b/ReadMe.c
index 10921e3..fb5a671 100644
--- a/ReadMe.c
+++ b/ReadMe.c
@@ -376,6 +376,7 @@ char Help_create[] =
" --name= -N : Textual name for array - max 32 characters\n"
" --bitmap-chunk= : bitmap chunksize in Kilobytes.\n"
" --delay= -d : bitmap update delay in seconds.\n"
+" --write-journal= : Specify journal device for RAID-4/5/6 array\n"
"\n"
;
diff --git a/mdadm.8.in b/mdadm.8.in
index bf3e131..2844039 100644
--- a/mdadm.8.in
+++ b/mdadm.8.in
@@ -990,6 +990,12 @@ Only works when the array is for clustered environment. It specifies
the maximum number of nodes in the cluster that will use this device
simultaneously. If not specified, this defaults to 4.
+.TP
+.BR \-\-write-journal
+Specify journal device for the RAID-4/5/6 array. The journal device
+should be a SSD with reasonable lifetime.
+
+
.SH For assemble:
.TP
--
2.4.6
^ permalink raw reply related
* [PATCH v2 5/6] Check write journal in incremental
From: Song Liu @ 2015-10-09 5:51 UTC (permalink / raw)
To: linux-raid; +Cc: neilb, shli, hch, dan.j.williams, kernel-team, Song Liu
In-Reply-To: <1444369906-1804157-1-git-send-email-songliubraving@fb.com>
If journal device is missing, do not start the array, and shows:
./mdadm -I /dev/sdf
mdadm: journal device is missing, not safe to start yet.
The array will be started when the journal device is attached with -I
./mdadm -I /dev/sdb1
mdadm: /dev/sdb1 attached to /dev/md/0_0, which has been started.
To force start without journal device:
./mdadm -I /dev/sdf --run
mdadm: Trying to run with missing journal device
mdadm: /dev/sdf attached to /dev/md/0_0, which has been started.
Signed-off-by: Song Liu <songliubraving@fb.com>
Signed-off-by: Shaohua Li <shli@fb.com>
---
Incremental.c | 31 +++++++++++++++++++++++++++----
1 file changed, 27 insertions(+), 4 deletions(-)
diff --git a/Incremental.c b/Incremental.c
index 43fddfd..5b2974c 100644
--- a/Incremental.c
+++ b/Incremental.c
@@ -35,7 +35,7 @@
static int count_active(struct supertype *st, struct mdinfo *sra,
int mdfd, char **availp,
- struct mdinfo *info);
+ struct mdinfo *info, int *journal_device_missing);
static void find_reject(int mdfd, struct supertype *st, struct mdinfo *sra,
int number, __u64 events, int verbose,
char *array_name);
@@ -104,6 +104,7 @@ int Incremental(struct mddev_dev *devlist, struct context *c,
struct map_ent target_array;
int have_target;
char *devname = devlist->devname;
+ int journal_device_missing = 0;
struct createinfo *ci = conf_get_create_info();
@@ -519,7 +520,7 @@ int Incremental(struct mddev_dev *devlist, struct context *c,
sysfs_free(sra);
sra = sysfs_read(mdfd, NULL, (GET_DEVS | GET_STATE |
GET_OFFSET | GET_SIZE));
- active_disks = count_active(st, sra, mdfd, &avail, &info);
+ active_disks = count_active(st, sra, mdfd, &avail, &info, &journal_device_missing);
if (enough(info.array.level, info.array.raid_disks,
info.array.layout, info.array.state & 1,
avail) == 0) {
@@ -549,10 +550,12 @@ int Incremental(struct mddev_dev *devlist, struct context *c,
}
map_unlock(&map);
- if (c->runstop > 0 || active_disks >= info.array.working_disks) {
+ if (c->runstop > 0 || (!journal_device_missing && active_disks >= info.array.working_disks)) {
struct mdinfo *dsk;
/* Let's try to start it */
+ if (journal_device_missing)
+ pr_err("Trying to run with missing journal device\n");
if (info.reshape_active && !(info.reshape_active & RESHAPE_NO_BACKUP)) {
pr_err("%s: This array is being reshaped and cannot be started\n",
chosen_name);
@@ -619,6 +622,8 @@ int Incremental(struct mddev_dev *devlist, struct context *c,
} else {
if (c->export) {
printf("MD_STARTED=unsafe\n");
+ } else if (journal_device_missing) {
+ pr_err("Journal device is missing, not safe to start yet.\n");
} else if (c->verbose >= 0)
pr_err("%s attached to %s, not enough to start safely.\n",
devname, chosen_name);
@@ -685,7 +690,8 @@ static void find_reject(int mdfd, struct supertype *st, struct mdinfo *sra,
static int count_active(struct supertype *st, struct mdinfo *sra,
int mdfd, char **availp,
- struct mdinfo *bestinfo)
+ struct mdinfo *bestinfo,
+ int *journal_device_missing)
{
/* count how many devices in sra think they are active */
struct mdinfo *d;
@@ -699,6 +705,8 @@ static int count_active(struct supertype *st, struct mdinfo *sra,
int devnum;
int b, i;
int raid_disks = 0;
+ int require_journal_dev = 0;
+ int has_journal_dev = 0;
if (!sra)
return 0;
@@ -719,8 +727,19 @@ static int count_active(struct supertype *st, struct mdinfo *sra,
close(dfd);
if (ok != 0)
continue;
+
+ if (st->ss->require_journal) {
+ require_journal_dev = st->ss->require_journal(st);
+ if (require_journal_dev == 2) {
+ pr_err("BUG: Superblock not loaded in Incremental.c:count_active\n");
+ return 0;
+ }
+ }
+
info.array.raid_disks = raid_disks;
st->ss->getinfo_super(st, &info, devmap + raid_disks * devnum);
+ if (info.disk.raid_disk == MD_DISK_ROLE_JOURNAL)
+ has_journal_dev = 1;
if (!avail) {
raid_disks = info.array.raid_disks;
avail = xcalloc(raid_disks, 1);
@@ -770,6 +789,10 @@ static int count_active(struct supertype *st, struct mdinfo *sra,
replcnt++;
st->ss->free_super(st);
}
+
+ if (require_journal_dev && !has_journal_dev)
+ *journal_device_missing = 1;
+
if (!avail)
return 0;
/* We need to reject any device that thinks the best device is
--
2.4.6
^ permalink raw reply related
* [PATCH v2 4/6] Assemble array with write journal
From: Song Liu @ 2015-10-09 5:51 UTC (permalink / raw)
To: linux-raid; +Cc: neilb, shli, hch, dan.j.williams, kernel-team, Song Liu
In-Reply-To: <1444369906-1804157-1-git-send-email-songliubraving@fb.com>
Example output:
./mdadm --assemble /dev/md0 /dev/sd[c-f] /dev/sdb1
mdadm: /dev/md0 has been started with 4 drives and 1 journal.
mdadm checks superblock for journal devices. If the journal device
is missing or faulty, mdadm will show warning
./mdadm --assemble /dev/md0 /dev/sd[c-q] /dev/sdb1
mdadm: Not safe to assemble with missing or stale journal device, consider --force.
User can insist to start the array (read only) with --force
./mdadm --assemble /dev/md0 /dev/sd[c-q] /dev/sdb1 --force
mdadm: Journal is missing or stale, starting array read only.
mdadm: /dev/md0 has been started with 15 drives.
Signed-off-by: Song Liu <songliubraving@fb.com>
Signed-off-by: Shaohua Li <shli@fb.com>
---
Assemble.c | 57 ++++++++++++++++++++++++++++++++++++++++++++++-----------
mdadm.h | 3 +++
super1.c | 37 ++++++++++++++++++++++++++++++++-----
3 files changed, 81 insertions(+), 16 deletions(-)
diff --git a/Assemble.c b/Assemble.c
index d9e9001..0661e8d 100644
--- a/Assemble.c
+++ b/Assemble.c
@@ -735,7 +735,7 @@ static int load_devices(struct devs *devices, char *devmap,
i = devcnt;
else
i = devices[devcnt].i.disk.raid_disk;
- if (i+1 == 0) {
+ if (i+1 == 0 || i == MD_DISK_ROLE_JOURNAL) {
if (nextspare < content->array.raid_disks*2)
nextspare = content->array.raid_disks*2;
i = nextspare++;
@@ -913,7 +913,6 @@ static int force_array(struct mdinfo *content,
avail[chosen_drive] = 1;
okcnt++;
tst->ss->free_super(tst);
-
/* If there are any other drives of the same vintage,
* add them in as well. We can't lose and we might gain
*/
@@ -944,17 +943,29 @@ static int start_array(int mdfd,
unsigned int okcnt,
unsigned int sparecnt,
unsigned int rebuilding_cnt,
+ unsigned int journalcnt,
struct context *c,
int clean, char *avail,
int start_partial_ok,
int err_ok,
- int was_forced
+ int was_forced,
+ int expect_journal,
+ int journal_clean
)
{
int rv;
int i;
unsigned int req_cnt;
+ if (expect_journal && (journal_clean == 0)) {
+ if (!c->force) {
+ pr_err("Not safe to assemble with missing or stale journal device, consider --force.\n");
+ return 1;
+ }
+ pr_err("Journal is missing or stale, starting array read only.\n");
+ c->readonly = 1;
+ }
+
rv = set_array_info(mdfd, st, content);
if (rv && !err_ok) {
pr_err("failed to set array info for %s: %s\n",
@@ -1032,7 +1043,8 @@ static int start_array(int mdfd,
if (content->array.level == LEVEL_CONTAINER) {
if (c->verbose >= 0) {
pr_err("Container %s has been assembled with %d drive%s",
- mddev, okcnt+sparecnt, okcnt+sparecnt==1?"":"s");
+ mddev, okcnt+sparecnt+journalcnt,
+ okcnt+sparecnt+journalcnt==1?"":"s");
if (okcnt < (unsigned)content->array.raid_disks)
fprintf(stderr, " (out of %d)",
content->array.raid_disks);
@@ -1118,6 +1130,8 @@ static int start_array(int mdfd,
fprintf(stderr, "%s %d rebuilding", sparecnt?",":" and", rebuilding_cnt);
if (sparecnt)
fprintf(stderr, " and %d spare%s", sparecnt, sparecnt==1?"":"s");
+ if (journal_clean)
+ fprintf(stderr, " and %d journal", journalcnt);
fprintf(stderr, ".\n");
}
if (content->reshape_active &&
@@ -1289,10 +1303,12 @@ int Assemble(struct supertype *st, char *mddev,
int *best = NULL; /* indexed by raid_disk */
int bestcnt = 0;
int devcnt;
- unsigned int okcnt, sparecnt, rebuilding_cnt, replcnt;
+ unsigned int okcnt, sparecnt, rebuilding_cnt, replcnt, journalcnt;
int i;
int was_forced = 0;
int most_recent = 0;
+ int expect_journal = 0;
+ int journal_clean = 0;
int chosen_drive;
int change = 0;
int inargv = 0;
@@ -1355,6 +1371,14 @@ try_again:
if (!st || !st->sb || !content)
return 2;
+ if (st->ss->require_journal) {
+ expect_journal = st->ss->require_journal(st);
+ if (expect_journal == 2) {
+ pr_err("BUG: Superblock not loaded in Assemble.c:Assemble\n");
+ return 1;
+ }
+ }
+
/* We have a full set of devices - we now need to find the
* array device.
* However there is a risk that we are racing with "mdadm -I"
@@ -1530,6 +1554,7 @@ try_again:
okcnt = 0;
replcnt = 0;
sparecnt=0;
+ journalcnt=0;
rebuilding_cnt=0;
for (i=0; i< bestcnt; i++) {
int j = best[i];
@@ -1540,8 +1565,13 @@ try_again:
/* note: we ignore error flags in multipath arrays
* as they don't make sense
*/
- if (content->array.level != LEVEL_MULTIPATH)
- if (!(devices[j].i.disk.state & (1<<MD_DISK_ACTIVE))) {
+ if (content->array.level != LEVEL_MULTIPATH) {
+ if (devices[j].i.disk.state & (1<<MD_DISK_JOURNAL)) {
+ if (expect_journal)
+ journalcnt++;
+ else /* unexpected journal, mark as faulty */
+ devices[j].i.disk.state |= (1<<MD_DISK_FAULTY);
+ } else if (!(devices[j].i.disk.state & (1<<MD_DISK_ACTIVE))) {
if (!(devices[j].i.disk.state
& (1<<MD_DISK_FAULTY))) {
devices[j].uptodate = 1;
@@ -1549,6 +1579,7 @@ try_again:
}
continue;
}
+ }
/* If this device thinks that 'most_recent' has failed, then
* we must reject this device.
*/
@@ -1572,6 +1603,8 @@ try_again:
devices[most_recent].i.events
) {
devices[j].uptodate = 1;
+ if (devices[j].i.disk.state & (1<<MD_DISK_JOURNAL))
+ journal_clean = 1;
if (i < content->array.raid_disks * 2) {
if (devices[j].i.recovery_start == MaxSector ||
(content->reshape_active &&
@@ -1583,7 +1616,7 @@ try_again:
replcnt++;
} else
rebuilding_cnt++;
- } else
+ } else if (devices[j].i.disk.raid_disk != MD_DISK_ROLE_JOURNAL)
sparecnt++;
}
}
@@ -1647,7 +1680,9 @@ try_again:
int j = best[i];
unsigned int desired_state;
- if (i >= content->array.raid_disks * 2)
+ if (devices[j].i.disk.raid_disk == MD_DISK_ROLE_JOURNAL)
+ desired_state = (1<<MD_DISK_JOURNAL);
+ else if (i >= content->array.raid_disks * 2)
desired_state = 0;
else if (i & 1)
desired_state = (1<<MD_DISK_ACTIVE) | (1<<MD_DISK_REPLACEMENT);
@@ -1794,11 +1829,11 @@ try_again:
rv = start_array(mdfd, mddev, content,
st, ident, best, bestcnt,
chosen_drive, devices, okcnt, sparecnt,
- rebuilding_cnt,
+ rebuilding_cnt, journalcnt,
c,
clean, avail, start_partial_ok,
pre_exist != NULL,
- was_forced);
+ was_forced, expect_journal, journal_clean);
if (rv == 1 && !pre_exist)
ioctl(mdfd, STOP_ARRAY, NULL);
free(devices);
diff --git a/mdadm.h b/mdadm.h
index 0b27b43..b1028be 100644
--- a/mdadm.h
+++ b/mdadm.h
@@ -970,6 +970,9 @@ extern struct superswitch {
/* validate container after assemble */
int (*validate_container)(struct mdinfo *info);
+ /* whether the array require a journal device */
+ int (*require_journal)(struct supertype *st);
+
int swapuuid; /* true if uuid is bigending rather than hostendian */
int external;
const char *name; /* canonical metadata name */
diff --git a/super1.c b/super1.c
index 85e3b28..47acdec 100644
--- a/super1.c
+++ b/super1.c
@@ -140,6 +140,34 @@ struct misc_dev_info {
|MD_FEATURE_BITMAP_VERSIONED \
|MD_FEATURE_JOURNAL \
)
+/* return value:
+ * 0, jouranl not required
+ * 1, journal required
+ * 2, no superblock loated (st->sb == NULL)
+ */
+static int require_journal1(struct supertype *st)
+{
+ struct mdp_superblock_1 *sb = st->sb;
+
+ if (sb->feature_map & MD_FEATURE_JOURNAL)
+ return 1;
+ else if (!sb)
+ return 2; /* no sb loaded */
+ return 0;
+}
+
+static int role_from_sb(struct mdp_superblock_1 *sb)
+{
+ unsigned int d;
+ int role;
+
+ d = __le32_to_cpu(sb->dev_number);
+ if (d < __le32_to_cpu(sb->max_dev))
+ role = __le16_to_cpu(sb->dev_roles[d]);
+ else
+ role = MD_DISK_ROLE_SPARE;
+ return role;
+}
/* return how many bytes are needed for bitmap, for cluster-md each node
* should have it's own bitmap */
@@ -482,11 +510,7 @@ static void examine_super1(struct supertype *st, char *homehost)
printf(")\n");
#endif
printf(" Device Role : ");
- d = __le32_to_cpu(sb->dev_number);
- if (d < __le32_to_cpu(sb->max_dev))
- role = __le16_to_cpu(sb->dev_roles[d]);
- else
- role = MD_DISK_ROLE_SPARE;
+ role = role_from_sb(sb);
if (role >= MD_DISK_ROLE_FAULTY)
printf("spare\n");
else if (role == MD_DISK_ROLE_JOURNAL)
@@ -1126,6 +1150,8 @@ static int update_super1(struct supertype *st, struct mdinfo *info,
int want;
if (info->disk.state & (1<<MD_DISK_ACTIVE))
want = info->disk.raid_disk;
+ else if (info->disk.state & (1<<MD_DISK_JOURNAL))
+ want = MD_DISK_ROLE_JOURNAL;
else
want = MD_DISK_ROLE_SPARE;
if (sb->dev_roles[d] != __cpu_to_le16(want)) {
@@ -2560,6 +2586,7 @@ struct superswitch super1 = {
.locate_bitmap = locate_bitmap1,
.write_bitmap = write_bitmap1,
.free_super = free_super1,
+ .require_journal = require_journal1,
#if __BYTE_ORDER == BIG_ENDIAN
.swapuuid = 0,
#else
--
2.4.6
^ permalink raw reply related
* [PATCH v2 3/6] Enable create array with write journal (--write-journal DEVICE).
From: Song Liu @ 2015-10-09 5:51 UTC (permalink / raw)
To: linux-raid; +Cc: neilb, shli, hch, dan.j.williams, kernel-team, Song Liu
In-Reply-To: <1444369906-1804157-1-git-send-email-songliubraving@fb.com>
Specify the write journal device with --write-journal DEVICE
./mdadm --create -f /dev/md0 --assume-clean -c 32 --raid-devices=4 --level=5 /dev/sd[c-f] --write-journal /dev/sdb1
mdadm: Defaulting to version 1.2 metadata
mdadm: array /dev/md0 started.
Only one journal device is allowed. If multiple --write-journal
are given, mdadm will use the first and ignore others
./mdadm --create -f /dev/md0 --assume-clean -c 32 --raid-devices=4 --level=5 /dev/sd[c-f] --write-journal /dev/sdb1 --write-journal /dev/sdx
mdadm: Please specify only one journal device for the array.
mdadm: Ignoring --write-journal /dev/sdx...
mdadm: Defaulting to version 1.2 metadata
mdadm: array /dev/md0 started.
Signed-off-by: Song Liu <songliubraving@fb.com>
Signed-off-by: Shaohua Li <shli@fb.com>
---
Create.c | 20 +++++++++++++------
ReadMe.c | 1 +
md_p.h | 58 +++++++++++++++++++++++++++++++++++++++++++++++++++++
mdadm.c | 23 +++++++++++++++++++++
mdadm.h | 2 ++
super1.c | 70 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-
6 files changed, 167 insertions(+), 7 deletions(-)
diff --git a/Create.c b/Create.c
index b200d97..21d1374 100644
--- a/Create.c
+++ b/Create.c
@@ -87,7 +87,7 @@ int Create(struct supertype *st, char *mddev,
unsigned long long minsize=0, maxsize=0;
char *mindisc = NULL;
char *maxdisc = NULL;
- int dnum;
+ int dnum, raid_disk_num;
struct mddev_dev *dv;
int fail=0, warn=0;
struct stat stb;
@@ -182,11 +182,11 @@ int Create(struct supertype *st, char *mddev,
pr_err("This metadata type does not support spare disks at create time\n");
return 1;
}
- if (subdevs > s->raiddisks+s->sparedisks) {
+ if (subdevs > s->raiddisks+s->sparedisks+s->journaldisks) {
pr_err("You have listed more devices (%d) than are in the array(%d)!\n", subdevs, s->raiddisks+s->sparedisks);
return 1;
}
- if (!have_container && subdevs < s->raiddisks+s->sparedisks) {
+ if (!have_container && subdevs < s->raiddisks+s->sparedisks+s->journaldisks) {
pr_err("You haven't given enough devices (real or missing) to create this array\n");
return 1;
}
@@ -399,6 +399,9 @@ int Create(struct supertype *st, char *mddev,
}
}
+ if (dv->disposition == 'j')
+ continue; /* skip write journal for size check */
+
freesize /= 2; /* convert to K */
if (s->chunk && s->chunk != UnSet) {
/* round to chunk size */
@@ -839,7 +842,7 @@ int Create(struct supertype *st, char *mddev,
for (pass=1; pass <=2 ; pass++) {
struct mddev_dev *moved_disk = NULL; /* the disk that was moved out of the insert point */
- for (dnum=0, dv = devlist ; dv ;
+ for (dnum=0, raid_disk_num=0, dv = devlist ; dv ;
dv=(dv->next)?(dv->next):moved_disk, dnum++) {
int fd;
struct stat stb;
@@ -864,8 +867,13 @@ int Create(struct supertype *st, char *mddev,
*inf = info;
inf->disk.number = dnum;
- inf->disk.raid_disk = dnum;
- if (inf->disk.raid_disk < s->raiddisks)
+ inf->disk.raid_disk = raid_disk_num++;
+
+ if (dv->disposition == 'j') {
+ inf->disk.raid_disk = MD_DISK_ROLE_JOURNAL;
+ inf->disk.state = (1<<MD_DISK_JOURNAL);
+ raid_disk_num--;
+ } else if (inf->disk.raid_disk < s->raiddisks)
inf->disk.state = (1<<MD_DISK_ACTIVE) |
(1<<MD_DISK_SYNC);
else
diff --git a/ReadMe.c b/ReadMe.c
index c242319..10921e3 100644
--- a/ReadMe.c
+++ b/ReadMe.c
@@ -142,6 +142,7 @@ struct option long_options[] = {
{"data-offset",1, 0, DataOffset},
{"nodes",1, 0, Nodes}, /* also for --assemble */
{"home-cluster",1, 0, ClusterName},
+ {"write-journal",1, 0, WriteJournal},
/* For assemble */
{"uuid", 1, 0, 'u'},
diff --git a/md_p.h b/md_p.h
index fae73ba..0d691fb 100644
--- a/md_p.h
+++ b/md_p.h
@@ -208,4 +208,62 @@ static inline __u64 md_event(mdp_super_t *sb) {
return (ev<<32)| sb->events_lo;
}
+struct r5l_payload_header {
+ __u16 type;
+ __u16 flags;
+} __attribute__ ((__packed__));
+
+enum r5l_payload_type {
+ R5LOG_PAYLOAD_DATA = 0,
+ R5LOG_PAYLOAD_PARITY = 1,
+ R5LOG_PAYLOAD_FLUSH = 2,
+};
+
+struct r5l_payload_data_parity {
+ struct r5l_payload_header header;
+ __u32 size; /* sector. data/parity size. each 4k has a checksum */
+ __u64 location; /* sector. For data, it's raid sector. For
+ parity, it's stripe sector */
+ __u32 checksum[];
+} __attribute__ ((__packed__));
+
+enum r5l_payload_data_parity_flag {
+ R5LOG_PAYLOAD_FLAG_DISCARD = 1, /* payload is discard */
+ /*
+ * RESHAPED/RESHAPING is only set when there is reshape activity. Note,
+ * both data/parity of a stripe should have the same flag set
+ *
+ * RESHAPED: reshape is running, and this stripe finished reshape
+ * RESHAPING: reshape is running, and this stripe isn't reshaped
+ * */
+ R5LOG_PAYLOAD_FLAG_RESHAPED = 2,
+ R5LOG_PAYLOAD_FLAG_RESHAPING = 3,
+};
+
+struct r5l_payload_flush {
+ struct r5l_payload_header header;
+ __u32 size; /* flush_stripes size, bytes */
+ __u64 flush_stripes[];
+} __attribute__ ((__packed__));
+
+enum r5l_payload_flush_flag {
+ R5LOG_PAYLOAD_FLAG_FLUSH_STRIPE = 1, /* data represents whole stripe */
+};
+
+struct r5l_meta_block {
+ __u32 magic;
+ __u32 checksum;
+ __u8 version;
+ __u8 __zero_pading_1;
+ __u16 __zero_pading_2;
+ __u32 meta_size; /* whole size of the block */
+
+ __u64 seq;
+ __u64 position; /* sector, start from rdev->data_offset, current position */
+ struct r5l_payload_header payloads[];
+} __attribute__ ((__packed__));
+
+#define R5LOG_VERSION 0x1
+#define R5LOG_MAGIC 0x6433c509
+
#endif
diff --git a/mdadm.c b/mdadm.c
index 183f6c8..f32a3d4 100644
--- a/mdadm.c
+++ b/mdadm.c
@@ -74,6 +74,7 @@ int main(int argc, char *argv[])
.require_homehost = 1,
};
struct shape s = {
+ .journaldisks = 0,
.level = UnSet,
.layout = UnSet,
.bitmap_chunk = UnSet,
@@ -1170,6 +1171,23 @@ int main(int argc, char *argv[])
case O(INCREMENTAL, IncrementalPath):
remove_path = optarg;
continue;
+ case O(CREATE, WriteJournal):
+ if (s.journaldisks) {
+ pr_err("Please specify only one journal device for the array.\n");
+ pr_err("Ignoring --write-journal %s...\n", optarg);
+ continue;
+ }
+ dv = xmalloc(sizeof(*dv));
+ dv->devname = optarg;
+ dv->disposition = 'j'; /* WriteJournal */
+ dv->used = 0;
+ dv->next = NULL;
+ *devlistend = dv;
+ devlistend = &dv->next;
+ devs_found++;
+
+ s.journaldisks = 1;
+ continue;
}
/* We have now processed all the valid options. Anything else is
* an error
@@ -1197,6 +1215,11 @@ int main(int argc, char *argv[])
exit(0);
}
+ if (s.journaldisks && (s.level < 4 || s.level > 6)) {
+ pr_err("--write-journal is only supported for RAID level 4/5/6.\n");
+ exit(2);
+ }
+
if (!mode && devs_found) {
mode = MISC;
devmode = 'Q';
diff --git a/mdadm.h b/mdadm.h
index 5633663..0b27b43 100644
--- a/mdadm.h
+++ b/mdadm.h
@@ -347,6 +347,7 @@ enum special_options {
Nodes,
ClusterName,
ClusterConfirm,
+ WriteJournal,
};
enum prefix_standard {
@@ -434,6 +435,7 @@ struct context {
struct shape {
int raiddisks;
int sparedisks;
+ int journaldisks;
int level;
int layout;
char *layout_str;
diff --git a/super1.c b/super1.c
index 6905b6d..85e3b28 100644
--- a/super1.c
+++ b/super1.c
@@ -68,7 +68,10 @@ struct mdp_superblock_1 {
__u64 data_offset; /* sector start of data, often 0 */
__u64 data_size; /* sectors in this device that can be used for data */
__u64 super_offset; /* sector start of this superblock */
- __u64 recovery_offset;/* sectors before this offset (from data_offset) have been recovered */
+ union {
+ __u64 recovery_offset;/* sectors before this offset (from data_offset) have been recovered */
+ __u64 journal_tail;/* journal tail of journal device (from data_offset) */
+ };
__u32 dev_number; /* permanent identifier of this device - not role in raid */
__u32 cnt_corrected_read; /* number of read errors that were corrected by re-writing */
__u8 device_uuid[16]; /* user-space setable, ignored by kernel */
@@ -1447,6 +1450,8 @@ static int add_to_super1(struct supertype *st, mdu_disk_info_t *dk,
if ((dk->state & 6) == 6) /* active, sync */
*rp = __cpu_to_le16(dk->raid_disk);
+ else if (dk->state & (1<<MD_DISK_JOURNAL))
+ *rp = MD_DISK_ROLE_JOURNAL;
else if ((dk->state & ~2) == 0) /* active or idle -> spare */
*rp = MD_DISK_ROLE_SPARE;
else
@@ -1566,6 +1571,57 @@ static unsigned long choose_bm_space(unsigned long devsize)
static void free_super1(struct supertype *st);
+#define META_BLOCK_SIZE 4096
+unsigned long crc32(
+ unsigned long crc,
+ const unsigned char *buf,
+ unsigned len);
+
+static int write_empty_r5l_meta_block(struct supertype *st, int fd)
+{
+ struct r5l_meta_block *mb;
+ struct mdp_superblock_1 *sb = st->sb;
+ struct align_fd afd;
+ __u32 crc;
+
+ init_afd(&afd, fd);
+
+ if (posix_memalign((void**)&mb, 4096, META_BLOCK_SIZE) != 0) {
+ pr_err("Could not allocate memory for the meta block.\n");
+ return 1;
+ }
+
+ memset(mb, 0, META_BLOCK_SIZE);
+
+ mb->magic = __cpu_to_le32(R5LOG_MAGIC);
+ mb->version = R5LOG_VERSION;
+ mb->meta_size = __cpu_to_le32(sizeof(struct r5l_meta_block));
+ mb->seq = __cpu_to_le64(random32());
+ mb->position = __cpu_to_le64(0);
+
+ crc = crc32(0xffffffff, sb->set_uuid, sizeof(sb->set_uuid));
+ crc = crc32(crc, (void *)mb, META_BLOCK_SIZE);
+ mb->checksum = __cpu_to_le32(crc);
+
+ if (lseek64(fd, (sb->data_offset) * 512, 0) < 0LL) {
+ pr_err("cannot seek to offset of the meta block\n");
+ goto fail_to_write;
+ }
+
+ if (awrite(&afd, mb, META_BLOCK_SIZE) != META_BLOCK_SIZE) {
+ pr_err("failed to store write the meta block \n");
+ goto fail_to_write;
+ }
+ fsync(fd);
+
+ free(mb);
+ return 0;
+
+fail_to_write:
+ free(mb);
+ return 1;
+}
+
#ifndef MDASSEMBLE
static int write_init_super1(struct supertype *st)
{
@@ -1580,6 +1636,11 @@ static int write_init_super1(struct supertype *st)
unsigned long long data_offset;
for (di = st->info; di; di = di->next) {
+ if (di->disk.state & (1 << MD_DISK_JOURNAL))
+ sb->feature_map |= MD_FEATURE_JOURNAL;
+ }
+
+ for (di = st->info; di; di = di->next) {
if (di->disk.state & (1 << MD_DISK_FAULTY))
continue;
if (di->fd < 0)
@@ -1718,6 +1779,13 @@ static int write_init_super1(struct supertype *st)
sb->sb_csum = calc_sb_1_csum(sb);
rv = store_super1(st, di->fd);
+
+ if (rv == 0 && (di->disk.state & (1 << MD_DISK_JOURNAL))) {
+ rv = write_empty_r5l_meta_block(st, di->fd);
+ if (rv)
+ goto error_out;
+ }
+
if (rv == 0 && (__le32_to_cpu(sb->feature_map) & 1))
rv = st->ss->write_bitmap(st, di->fd, NoUpdate);
close(di->fd);
--
2.4.6
^ permalink raw reply related
* [PATCH v2 2/6] Show device as journal in --detail --examine
From: Song Liu @ 2015-10-09 5:51 UTC (permalink / raw)
To: linux-raid; +Cc: neilb, shli, hch, dan.j.williams, kernel-team, Song Liu
In-Reply-To: <1444369906-1804157-1-git-send-email-songliubraving@fb.com>
Example output:
./mdadm --detail /dev/md127
/dev/md127:
Version : 1.2
Creation Time : Wed May 13 17:01:12 2015
Raid Level : raid5
Array Size : 11720662464 (11177.69 GiB 12001.96 GB)
Used Dev Size : 3906887488 (3725.90 GiB 4000.65 GB)
Raid Devices : 4
Total Devices : 5
Persistence : Superblock is persistent
Intent Bitmap : Internal
Update Time : Wed May 13 17:01:12 2015
State : clean
Active Devices : 4
Working Devices : 5
Failed Devices : 0
Spare Devices : 1
Layout : left-symmetric
Chunk Size : 32K
Name : 0
UUID : 8fb9ee05:3831d52f:e5c23825:28cd6881
Events : 0
Number Major Minor RaidDevice State
0 8 32 0 active sync /dev/sdc
1 8 48 1 active sync /dev/sdd
2 8 64 2 active sync /dev/sde
3 8 80 3 active sync /dev/sdf
4 8 17 - journal /dev/sdb1
./mdadm -E /dev/sdb2
/dev/sdb2:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x201
Array UUID : 562b2334:35b9bcc1:add50892:1f30c4bd
Name : 0
Creation Time : Thu Aug 27 12:55:26 2015
Raid Level : raid5
Raid Devices : 15
Avail Dev Size : 249796608 (119.11 GiB 127.90 GB)
Array Size : 54696423936 (52162.57 GiB 56009.14 GB)
Used Dev Size : 7813774848 (3725.90 GiB 4000.65 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Unused Space : before=262056 sectors, after=0 sectors
State : active
Device UUID : 5015e522:d39ba566:5909cf3c:9c51f2ff
Internal Bitmap : 8 sectors from superblock
Update Time : Thu Aug 27 13:16:55 2015
Bad Block Log : 512 entries available at offset 72 sectors
Checksum : 4e6fd76d - correct
Events : 262
Layout : left-symmetric
Chunk Size : 256K
Device Role : Journal
Array State : AAAAAAAAAAAAAAA ('A' == active, '.' == missing, 'R' == replacing)
Signed-off-by: Song Liu <songliubraving@fb.com>
Signed-off-by: Shaohua Li <shli@fb.com>
---
Detail.c | 3 ++-
md_p.h | 2 ++
super1.c | 9 +++++++++
3 files changed, 13 insertions(+), 1 deletion(-)
diff --git a/Detail.c b/Detail.c
index dd72ede..200f65f 100644
--- a/Detail.c
+++ b/Detail.c
@@ -650,9 +650,10 @@ This is pretty boring
}
if (disk.state & (1<<MD_DISK_REMOVED)) printf(" removed");
if (disk.state & (1<<MD_DISK_WRITEMOSTLY)) printf(" writemostly");
+ if (disk.state & (1<<MD_DISK_JOURNAL)) printf(" journal");
if ((disk.state &
((1<<MD_DISK_ACTIVE)|(1<<MD_DISK_SYNC)
- |(1<<MD_DISK_REMOVED)|(1<<MD_DISK_FAULTY)))
+ |(1<<MD_DISK_REMOVED)|(1<<MD_DISK_FAULTY)|(1<<MD_DISK_JOURNAL)))
== 0) {
printf(" spare");
if (is_26) {
diff --git a/md_p.h b/md_p.h
index 3a3b8af..fae73ba 100644
--- a/md_p.h
+++ b/md_p.h
@@ -91,9 +91,11 @@
*/
#define MD_DISK_REPLACEMENT 17
+#define MD_DISK_JOURNAL 18 /* disk is used as the write journal in RAID-5/6 */
#define MD_DISK_ROLE_SPARE 0xffff
#define MD_DISK_ROLE_FAULTY 0xfffe
+#define MD_DISK_ROLE_JOURNAL 0xfffd
#define MD_DISK_ROLE_MAX 0xff00 /* max value of regular disk role */
typedef struct mdp_device_descriptor_s {
diff --git a/super1.c b/super1.c
index b881eb5..6905b6d 100644
--- a/super1.c
+++ b/super1.c
@@ -126,6 +126,7 @@ struct misc_dev_info {
*/
#define MD_FEATURE_NEW_OFFSET 64 /* new_offset must be honoured */
#define MD_FEATURE_BITMAP_VERSIONED 256 /* bitmap version number checked properly */
+#define MD_FEATURE_JOURNAL 512 /* support write journal */
#define MD_FEATURE_ALL (MD_FEATURE_BITMAP_OFFSET \
|MD_FEATURE_RECOVERY_OFFSET \
|MD_FEATURE_RESHAPE_ACTIVE \
@@ -134,6 +135,7 @@ struct misc_dev_info {
|MD_FEATURE_RESHAPE_BACKWARDS \
|MD_FEATURE_NEW_OFFSET \
|MD_FEATURE_BITMAP_VERSIONED \
+ |MD_FEATURE_JOURNAL \
)
/* return how many bytes are needed for bitmap, for cluster-md each node
@@ -484,6 +486,8 @@ static void examine_super1(struct supertype *st, char *homehost)
role = MD_DISK_ROLE_SPARE;
if (role >= MD_DISK_ROLE_FAULTY)
printf("spare\n");
+ else if (role == MD_DISK_ROLE_JOURNAL)
+ printf("Journal\n");
else if (sb->feature_map & __cpu_to_le32(MD_FEATURE_REPLACEMENT))
printf("Replacement device %d\n", role);
else
@@ -995,6 +999,11 @@ static void getinfo_super1(struct supertype *st, struct mdinfo *info, char *map)
case MD_DISK_ROLE_FAULTY:
info->disk.state = 1; /* faulty */
break;
+ case MD_DISK_ROLE_JOURNAL:
+ info->disk.state = (1 << MD_DISK_JOURNAL);
+ info->disk.raid_disk = role;
+ info->space_after = (misc->device_size - info->data_offset) % 8; /* journal uses all 4kB blocks*/
+ break;
default:
info->disk.state = 6; /* active and in sync */
info->disk.raid_disk = role;
--
2.4.6
^ permalink raw reply related
* [PATCH v2 1/6] add macros for MD_DISK_ROLE_(SPARE/FAULTY)
From: Song Liu @ 2015-10-09 5:51 UTC (permalink / raw)
To: linux-raid; +Cc: neilb, shli, hch, dan.j.williams, kernel-team, Song Liu
In-Reply-To: <1444369906-1804157-1-git-send-email-songliubraving@fb.com>
Replace special disk roles (0xffff, 0xfffe) with macros:
define MD_DISK_ROLE_SPARE 0xffff
define MD_DISK_ROLE_FAULTY 0xfffe
Will add macro for journal device in next patch:
define MD_DISK_ROLE_JOURNAL 0xfffd
Signed-off-by: Song Liu <songliubraving@fb.com>
Signed-off-by: Shaohua Li <shli@fb.com>
---
md_p.h | 4 ++++
super1.c | 30 +++++++++++++++---------------
2 files changed, 19 insertions(+), 15 deletions(-)
diff --git a/md_p.h b/md_p.h
index 9b6b5f8..3a3b8af 100644
--- a/md_p.h
+++ b/md_p.h
@@ -92,6 +92,10 @@
#define MD_DISK_REPLACEMENT 17
+#define MD_DISK_ROLE_SPARE 0xffff
+#define MD_DISK_ROLE_FAULTY 0xfffe
+#define MD_DISK_ROLE_MAX 0xff00 /* max value of regular disk role */
+
typedef struct mdp_device_descriptor_s {
__u32 number; /* 0 Device number in the entire set */
__u32 major; /* 1 Device major number */
diff --git a/super1.c b/super1.c
index 6f42291..b881eb5 100644
--- a/super1.c
+++ b/super1.c
@@ -465,13 +465,13 @@ static void examine_super1(struct supertype *st, char *homehost)
/* This turns out to just be confusing */
printf(" Array Slot : %d (", __le32_to_cpu(sb->dev_number));
for (i= __le32_to_cpu(sb->max_dev); i> 0 ; i--)
- if (__le16_to_cpu(sb->dev_roles[i-1]) != 0xffff)
+ if (__le16_to_cpu(sb->dev_roles[i-1]) != MD_DISK_ROLE_SPARE)
break;
for (d=0; d < i; d++) {
int role = __le16_to_cpu(sb->dev_roles[d]);
if (d) printf(", ");
- if (role == 0xffff) printf("empty");
- else if(role == 0xfffe) printf("failed");
+ if (role == MD_DISK_ROLE_SPARE) printf("empty");
+ else if(role == MD_DISK_ROLE_FAULTY) printf("failed");
else printf("%d", role);
}
printf(")\n");
@@ -481,8 +481,8 @@ static void examine_super1(struct supertype *st, char *homehost)
if (d < __le32_to_cpu(sb->max_dev))
role = __le16_to_cpu(sb->dev_roles[d]);
else
- role = 0xFFFF;
- if (role >= 0xFFFE)
+ role = MD_DISK_ROLE_SPARE;
+ if (role >= MD_DISK_ROLE_FAULTY)
printf("spare\n");
else if (sb->feature_map & __cpu_to_le32(MD_FEATURE_REPLACEMENT))
printf("Replacement device %d\n", role);
@@ -512,7 +512,7 @@ static void examine_super1(struct supertype *st, char *homehost)
faulty = 0;
for (i=0; i< __le32_to_cpu(sb->max_dev); i++) {
int role = __le16_to_cpu(sb->dev_roles[i]);
- if (role == 0xFFFE)
+ if (role == MD_DISK_ROLE_FAULTY)
faulty++;
}
if (faulty) printf(" %d failed", faulty);
@@ -922,7 +922,7 @@ static void getinfo_super1(struct supertype *st, struct mdinfo *info, char *map)
info->disk.number = __le32_to_cpu(sb->dev_number);
if (__le32_to_cpu(sb->dev_number) >= __le32_to_cpu(sb->max_dev) ||
__le32_to_cpu(sb->dev_number) >= MAX_DEVS)
- role = 0xfffe;
+ role = MD_DISK_ROLE_FAULTY;
else
role = __le16_to_cpu(sb->dev_roles[__le32_to_cpu(sb->dev_number)]);
@@ -989,10 +989,10 @@ static void getinfo_super1(struct supertype *st, struct mdinfo *info, char *map)
info->disk.raid_disk = -1;
switch(role) {
- case 0xFFFF:
+ case MD_DISK_ROLE_SPARE:
info->disk.state = 0; /* spare: not active, not sync, not faulty */
break;
- case 0xFFFE:
+ case MD_DISK_ROLE_FAULTY:
info->disk.state = 1; /* faulty */
break;
default:
@@ -1042,7 +1042,7 @@ static void getinfo_super1(struct supertype *st, struct mdinfo *info, char *map)
map[i] = 0;
for (i = 0; i < __le32_to_cpu(sb->max_dev); i++) {
role = __le16_to_cpu(sb->dev_roles[i]);
- if (/*role == 0xFFFF || */role < (unsigned) info->array.raid_disks) {
+ if (/*role == MD_DISK_ROLE_SPARE || */role < (unsigned) info->array.raid_disks) {
working++;
if (map && role < map_disks)
map[role] = 1;
@@ -1115,7 +1115,7 @@ static int update_super1(struct supertype *st, struct mdinfo *info,
if (info->disk.state & (1<<MD_DISK_ACTIVE))
want = info->disk.raid_disk;
else
- want = 0xFFFF;
+ want = MD_DISK_ROLE_SPARE;
if (sb->dev_roles[d] != __cpu_to_le16(want)) {
sb->dev_roles[d] = __cpu_to_le16(want);
rv = 1;
@@ -1140,7 +1140,7 @@ static int update_super1(struct supertype *st, struct mdinfo *info,
unsigned int max = __le32_to_cpu(sb->max_dev);
for (i=0 ; i < max ; i++)
- if (__le16_to_cpu(sb->dev_roles[i]) >= 0xfffe)
+ if (__le16_to_cpu(sb->dev_roles[i]) >= MD_DISK_ROLE_FAULTY)
break;
sb->dev_number = __cpu_to_le32(i);
info->disk.number = i;
@@ -1439,9 +1439,9 @@ static int add_to_super1(struct supertype *st, mdu_disk_info_t *dk,
if ((dk->state & 6) == 6) /* active, sync */
*rp = __cpu_to_le16(dk->raid_disk);
else if ((dk->state & ~2) == 0) /* active or idle -> spare */
- *rp = 0xffff;
+ *rp = MD_DISK_ROLE_SPARE;
else
- *rp = 0xfffe;
+ *rp = MD_DISK_ROLE_FAULTY;
if (dk->number >= (int)__le32_to_cpu(sb->max_dev) &&
__le32_to_cpu(sb->max_dev) < MAX_DEVS)
@@ -2445,7 +2445,7 @@ void *super1_make_v0(struct supertype *st, struct mdinfo *info, mdp_super_t *sb0
for (i = 0; i < MD_SB_DISKS; i++) {
int state = sb0->disks[i].state;
- sb->dev_roles[i] = 0xFFFF;
+ sb->dev_roles[i] = MD_DISK_ROLE_SPARE;
if ((state & (1<<MD_DISK_SYNC)) &&
!(state & (1<<MD_DISK_FAULTY)))
sb->dev_roles[i] = __cpu_to_le16(sb0->disks[i].raid_disk);
--
2.4.6
^ permalink raw reply related
* [PATCH v2 0/6] mdadm support for journal device of RAID-4/5/6
From: Song Liu @ 2015-10-09 5:51 UTC (permalink / raw)
To: linux-raid; +Cc: neilb, shli, hch, dan.j.williams, kernel-team, Song Liu
Hi,
These are v2 of mdadm patches to support journal device in RAID-4/5/6.
Shaohua has sent latest kernel patches earlier today
http://marc.info/?l=linux-raid&m=144436646730381
Most of these patches are very close to v1, except --assemble, where
we improved checks for journal devices.
The following are copied from v1 pathes.
These patches add write journal support for the following commands:
mdadm --detail
mdadm --examine
mdadm --create --write-journal DEVICE
mdadm --assemble
mdadm --incremental
Journal device is assigned with dev_role 0xFFFD (where 0xFFFF is for
spare and 0xFFFE is for failed). Note that there is compatibility
issue that older mdadm will show journal device as spare in --detail:
Number Major Minor RaidDevice State
0 8 32 0 active sync /dev/sdc
1 8 48 1 active sync /dev/sdd
2 8 64 2 active sync /dev/sde
3 8 80 3 active sync /dev/sdf
4 8 17 - spare /dev/sdb1
Also, older mdadm will show journal device as "Active device 65533"
in --examine:
Device Role : Active device 65533
Array State : AAAA ('A' == active, '.' == missing, 'R' == replacing)
Song Liu (6):
add macros for MD_DISK_ROLE_(SPARE/FAULTY)
Show device as journal in --detail --examine
Enable create array with write journal (--write-journal DEVICE).
Assemble array with write journal
Check write journal in incremental
Add help message and man entry for --write-journal
Assemble.c | 57 ++++++++++++++++++-----
Create.c | 20 +++++---
Detail.c | 3 +-
Incremental.c | 31 +++++++++++--
ReadMe.c | 2 +
md_p.h | 64 ++++++++++++++++++++++++++
mdadm.8.in | 6 +++
mdadm.c | 23 ++++++++++
mdadm.h | 5 ++
super1.c | 144 ++++++++++++++++++++++++++++++++++++++++++++++++++--------
10 files changed, 313 insertions(+), 42 deletions(-)
--
2.4.6
^ permalink raw reply
* Re: Change "mv $targetdir/log $logdir/$log" to "cp $targetdir/log $logdir/$log"
From: Neil Brown @ 2015-10-09 5:02 UTC (permalink / raw)
To: Yi Zhang; +Cc: linux-raid
In-Reply-To: <1023141143.22094768.1441596507637.JavaMail.zimbra@redhat.com>
[-- Attachment #1: Type: text/plain, Size: 1853 bytes --]
Yi Zhang <yizhan@redhat.com> writes:
> Hi Neil
> When I execute mdadm test with --save-logs set, found below error, could you help check it?
>
> Error log:
> tests/10ddf-fail-readd... FAILED - see /var/tmp/log-10ddf-fail-readd for details
> cp: cannot stat ‘/var/tmp/log’: No such file or directory
>
Thanks for the report.
I've applied the following which should fix this (and other issues).
Thanks,
NeilBrown
From c54f485288b50c90fa289e79fa66ae01e9162a2d Mon Sep 17 00:00:00 2001
From: NeilBrown <neilb@suse.com>
Date: Fri, 9 Oct 2015 16:00:59 +1100
Subject: [PATCH] test: don't move $target/log
Recent change to rename $target/log to a more stable name on
error broke --save-logs as $targetdir/log no longer existed to be
copied.
So change it to a 'cp', not a 'mv'.
While we are fixing things, use $_basename instead of calling
`basename` again, and only use "$logdir" if logs were requested.
And make use "see .. for details" has correct file name.
Reported-by: Yi Zhang <yizhan@redhat.com>
Signed-off-by: NeilBrown <neilb@suse.com>
diff --git a/test b/test
index d0a6cb858212..5315ab97dde0 100755
--- a/test
+++ b/test
@@ -17,7 +17,6 @@ then
fi
testdir="tests"
-logdir="$testdir/logs"
logsave=0
exitonerror=1
@@ -317,10 +316,11 @@ do_test() {
echo "=======================dmesg=================" >> $targetdir/log
dmesg | tail -n 200 >> $targetdir/log
if [ $exitonerror == 0 ]; then
- log=log-`basename $_script`
- mv $targetdir/log $logdir/$log
+ cp $targetdir/log $targetdir/log-$_basename
+ echo "FAILED - see $targetdir/log-$_basename for details"
+ else
+ echo "FAILED - see $targetdir/log for details"
fi
- echo "FAILED - see $logdir/$log for details"
_fail=1
fi
if [ "$savelogs" == "1" ]; then
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 818 bytes --]
^ permalink raw reply related
* [PATCH 9/9] MD: when RAID journal is missing/faulty, block RESTART_ARRAY_RW
From: Shaohua Li @ 2015-10-09 4:54 UTC (permalink / raw)
To: linux-raid; +Cc: Kernel-team, songliubraving, hch, dan.j.williams, neilb
In-Reply-To: <cover.1444360850.git.shli@fb.com>
From: Song Liu <songliubraving@fb.com>
When RAID-4/5/6 array suffers from missing journal device, we put
the array in read only state. We should not allow trasition to
read-write states (clean and active) before replacing journal device.
Signed-off-by: Song Liu <songliubraving@fb.com>
Signed-off-by: Shaohua Li <shli@fb.com>
---
drivers/md/md.c | 27 +++++++++++++++++++++++++--
1 file changed, 25 insertions(+), 2 deletions(-)
diff --git a/drivers/md/md.c b/drivers/md/md.c
index 59b38ef..c23d21b 100644
--- a/drivers/md/md.c
+++ b/drivers/md/md.c
@@ -3961,7 +3961,9 @@ array_state_store(struct mddev *mddev, const char *buf, size_t len)
break;
case clean:
if (mddev->pers) {
- restart_array(mddev);
+ err = restart_array(mddev);
+ if (err)
+ break;
spin_lock(&mddev->lock);
if (atomic_read(&mddev->writes_pending) == 0) {
if (mddev->in_sync == 0) {
@@ -3979,7 +3981,9 @@ array_state_store(struct mddev *mddev, const char *buf, size_t len)
break;
case active:
if (mddev->pers) {
- restart_array(mddev);
+ err = restart_array(mddev);
+ if (err)
+ break;
clear_bit(MD_CHANGE_PENDING, &mddev->flags);
wake_up(&mddev->sb_wait);
err = 0;
@@ -5336,6 +5340,25 @@ static int restart_array(struct mddev *mddev)
return -EINVAL;
if (!mddev->ro)
return -EBUSY;
+ if (test_bit(MD_HAS_JOURNAL, &mddev->flags)) {
+ struct md_rdev *rdev;
+ bool has_journal = false;
+
+ rcu_read_lock();
+ rdev_for_each_rcu(rdev, mddev) {
+ if (test_bit(Journal, &rdev->flags) &&
+ !test_bit(Faulty, &rdev->flags)) {
+ has_journal = true;
+ break;
+ }
+ }
+ rcu_read_unlock();
+
+ /* Don't restart rw with journal missing/faulty */
+ if (!has_journal)
+ return -EINVAL;
+ }
+
mddev->safemode = 0;
mddev->ro = 0;
set_disk_ro(disk, 0);
--
2.4.6
^ permalink raw reply related
* [PATCH 8/9] MD: set journal disk ->raid_disk
From: Shaohua Li @ 2015-10-09 4:54 UTC (permalink / raw)
To: linux-raid; +Cc: Kernel-team, songliubraving, hch, dan.j.williams, neilb
In-Reply-To: <cover.1444360850.git.shli@fb.com>
Set journal disk ->raid_disk to >=0, I choose raid_disks + 1 instead of
0, because we already have a disk with ->raid_disk 0 and this causes
sysfs entry creation conflict. A lot of places assumes disk with
->raid_disk >=0 is normal raid disk, so we add check for journal disk.
Signed-off-by: Shaohua Li <shli@fb.com>
---
drivers/md/md.c | 27 ++++++++++++++++++++++-----
drivers/md/raid5.c | 6 ++++--
2 files changed, 26 insertions(+), 7 deletions(-)
diff --git a/drivers/md/md.c b/drivers/md/md.c
index a8db2e2..59b38ef 100644
--- a/drivers/md/md.c
+++ b/drivers/md/md.c
@@ -1650,6 +1650,7 @@ static int super_1_validate(struct mddev *mddev, struct md_rdev *rdev)
rdev->journal_tail = le64_to_cpu(sb->journal_tail);
if (mddev->recovery_cp == MaxSector)
set_bit(MD_JOURNAL_CLEAN, &mddev->flags);
+ rdev->raid_disk = mddev->raid_disks;
break;
default:
rdev->saved_raid_disk = role;
@@ -1719,7 +1720,7 @@ static void super_1_sync(struct mddev *mddev, struct md_rdev *rdev)
sb->feature_map = cpu_to_le32(MD_FEATURE_BITMAP_OFFSET);
}
- if (rdev->raid_disk >= 0 &&
+ if (rdev->raid_disk >= 0 && !test_bit(Journal, &rdev->flags) &&
!test_bit(In_sync, &rdev->flags)) {
sb->feature_map |=
cpu_to_le32(MD_FEATURE_RECOVERY_OFFSET);
@@ -2303,6 +2304,7 @@ void md_update_sb(struct mddev *mddev, int force_change)
rdev_for_each(rdev, mddev) {
if (rdev->raid_disk >= 0 &&
mddev->delta_disks >= 0 &&
+ !test_bit(Journal, &rdev->flags) &&
!test_bit(In_sync, &rdev->flags) &&
mddev->curr_resync_completed > rdev->recovery_offset)
rdev->recovery_offset = mddev->curr_resync_completed;
@@ -2539,6 +2541,7 @@ state_show(struct md_rdev *rdev, char *page)
sep = ",";
}
if (!test_bit(Faulty, &flags) &&
+ !test_bit(Journal, &flags) &&
!test_bit(In_sync, &flags)) {
len += sprintf(page+len, "%sspare", sep);
sep = ",";
@@ -2622,7 +2625,8 @@ state_store(struct md_rdev *rdev, const char *buf, size_t len)
} else if (cmd_match(buf, "insync") && rdev->raid_disk == -1) {
set_bit(In_sync, &rdev->flags);
err = 0;
- } else if (cmd_match(buf, "-insync") && rdev->raid_disk >= 0) {
+ } else if (cmd_match(buf, "-insync") && rdev->raid_disk >= 0 &&
+ !test_bit(Journal, &rdev->flags)) {
if (rdev->mddev->pers == NULL) {
clear_bit(In_sync, &rdev->flags);
rdev->saved_raid_disk = rdev->raid_disk;
@@ -2641,6 +2645,7 @@ state_store(struct md_rdev *rdev, const char *buf, size_t len)
* check if recovery is needed.
*/
if (rdev->raid_disk >= 0 &&
+ !test_bit(Journal, &rdev->flags) &&
!test_bit(Replacement, &rdev->flags))
set_bit(WantReplacement, &rdev->flags);
set_bit(MD_RECOVERY_NEEDED, &rdev->mddev->recovery);
@@ -2718,7 +2723,9 @@ __ATTR(errors, S_IRUGO|S_IWUSR, errors_show, errors_store);
static ssize_t
slot_show(struct md_rdev *rdev, char *page)
{
- if (rdev->raid_disk < 0)
+ if (test_bit(Journal, &rdev->flags))
+ return sprintf(page, "journal\n");
+ else if (rdev->raid_disk < 0)
return sprintf(page, "none\n");
else
return sprintf(page, "%d\n", rdev->raid_disk);
@@ -2730,6 +2737,8 @@ slot_store(struct md_rdev *rdev, const char *buf, size_t len)
int slot;
int err;
+ if (test_bit(Journal, &rdev->flags))
+ return -EBUSY;
if (strncmp(buf, "none", 4)==0)
slot = -1;
else {
@@ -2928,6 +2937,8 @@ rdev_size_store(struct md_rdev *rdev, const char *buf, size_t len)
sector_t oldsectors = rdev->sectors;
sector_t sectors;
+ if (test_bit(Journal, &rdev->flags))
+ return -EBUSY;
if (strict_blocks_to_sectors(buf, §ors) < 0)
return -EINVAL;
if (rdev->data_offset != rdev->new_data_offset)
@@ -3290,7 +3301,9 @@ static void analyze_sbs(struct mddev *mddev)
rdev->desc_nr = i++;
rdev->raid_disk = rdev->desc_nr;
set_bit(In_sync, &rdev->flags);
- } else if (rdev->raid_disk >= (mddev->raid_disks - min(0, mddev->delta_disks))) {
+ } else if (rdev->raid_disk >=
+ (mddev->raid_disks - min(0, mddev->delta_disks)) &&
+ !test_bit(Journal, &rdev->flags)) {
rdev->raid_disk = -1;
clear_bit(In_sync, &rdev->flags);
}
@@ -7805,6 +7818,7 @@ void md_do_sync(struct md_thread *thread)
rcu_read_lock();
rdev_for_each_rcu(rdev, mddev)
if (rdev->raid_disk >= 0 &&
+ !test_bit(Journal, &rdev->flags) &&
!test_bit(Faulty, &rdev->flags) &&
!test_bit(In_sync, &rdev->flags) &&
rdev->recovery_offset < j)
@@ -8026,6 +8040,7 @@ void md_do_sync(struct md_thread *thread)
rdev_for_each_rcu(rdev, mddev)
if (rdev->raid_disk >= 0 &&
mddev->delta_disks >= 0 &&
+ !test_bit(Journal, &rdev->flags) &&
!test_bit(Faulty, &rdev->flags) &&
!test_bit(In_sync, &rdev->flags) &&
rdev->recovery_offset < mddev->curr_resync)
@@ -8066,7 +8081,8 @@ static int remove_and_add_spares(struct mddev *mddev,
rdev->raid_disk >= 0 &&
!test_bit(Blocked, &rdev->flags) &&
(test_bit(Faulty, &rdev->flags) ||
- ! test_bit(In_sync, &rdev->flags)) &&
+ (!test_bit(In_sync, &rdev->flags) &&
+ !test_bit(Journal, &rdev->flags))) &&
atomic_read(&rdev->nr_pending)==0) {
if (mddev->pers->hot_remove_disk(
mddev, rdev) == 0) {
@@ -8088,6 +8104,7 @@ static int remove_and_add_spares(struct mddev *mddev,
continue;
if (rdev->raid_disk >= 0 &&
!test_bit(In_sync, &rdev->flags) &&
+ !test_bit(Journal, &rdev->flags) &&
!test_bit(Faulty, &rdev->flags))
spares++;
if (rdev->raid_disk >= 0)
diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c
index 8024cad..efbb5da 100644
--- a/drivers/md/raid5.c
+++ b/drivers/md/raid5.c
@@ -6560,7 +6560,7 @@ static struct r5conf *setup_conf(struct mddev *mddev)
rdev_for_each(rdev, mddev) {
raid_disk = rdev->raid_disk;
if (raid_disk >= max_disks
- || raid_disk < 0)
+ || raid_disk < 0 || test_bit(Journal, &rdev->flags))
continue;
disk = conf->disks + raid_disk;
@@ -6694,8 +6694,10 @@ static int run(struct mddev *mddev)
rdev_for_each(rdev, mddev) {
long long diff;
- if (test_bit(Journal, &rdev->flags))
+ if (test_bit(Journal, &rdev->flags)) {
journal_dev = rdev;
+ continue;
+ }
if (rdev->raid_disk < 0)
continue;
diff = (rdev->new_data_offset - rdev->data_offset);
--
2.4.6
^ permalink raw reply related
* [PATCH 7/9] MD: kick out journal disk if it's not fresh
From: Shaohua Li @ 2015-10-09 4:54 UTC (permalink / raw)
To: linux-raid; +Cc: Kernel-team, songliubraving, hch, dan.j.williams, neilb
In-Reply-To: <cover.1444360850.git.shli@fb.com>
From: Song Liu <songliubraving@fb.com>
When journal disk is faulty and we are reassemabling the raid array, the
journal disk is old. We don't allow the journal disk added to the raid
array. Since journal disk is missing in the array, the raid5 will mark
the array readonly.
Signed-off-by: Song Liu <songliubraving@fb.com>
Signed-off-by: Shaohua Li <shli@fb.com>
---
drivers/md/md.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/drivers/md/md.c b/drivers/md/md.c
index a5b0bbd..a8db2e2 100644
--- a/drivers/md/md.c
+++ b/drivers/md/md.c
@@ -1608,7 +1608,8 @@ static int super_1_validate(struct mddev *mddev, struct md_rdev *rdev)
++ev1;
if (rdev->desc_nr >= 0 &&
rdev->desc_nr < le32_to_cpu(sb->max_dev) &&
- le16_to_cpu(sb->dev_roles[rdev->desc_nr]) < MD_DISK_ROLE_MAX)
+ (le16_to_cpu(sb->dev_roles[rdev->desc_nr]) < MD_DISK_ROLE_MAX ||
+ le16_to_cpu(sb->dev_roles[rdev->desc_nr]) == MD_DISK_ROLE_JOURNAL))
if (ev1 < mddev->events)
return -EINVAL;
} else if (mddev->bitmap) {
--
2.4.6
^ permalink raw reply related
* [PATCH 6/9] raid5-cache: start raid5 readonly if journal is missing
From: Shaohua Li @ 2015-10-09 4:54 UTC (permalink / raw)
To: linux-raid; +Cc: Kernel-team, songliubraving, hch, dan.j.williams, neilb
In-Reply-To: <cover.1444360850.git.shli@fb.com>
If raid array is expected to have journal (eg, journal is set in MD
superblock feature map) and the array is started without journal disk,
start the array readonly.
Signed-off-by: Shaohua Li <shli@fb.com>
---
drivers/md/raid5-cache.c | 3 ++-
drivers/md/raid5.c | 8 ++++++++
2 files changed, 10 insertions(+), 1 deletion(-)
diff --git a/drivers/md/raid5-cache.c b/drivers/md/raid5-cache.c
index 2cf6b72..e64684b 100644
--- a/drivers/md/raid5-cache.c
+++ b/drivers/md/raid5-cache.c
@@ -806,8 +806,9 @@ void r5l_quiesce(struct r5l_log *log, int state)
bool r5l_log_disk_error(struct r5conf *conf)
{
+ /* don't allow write if journal disk is missing */
if (!conf->log)
- return false;
+ return test_bit(MD_HAS_JOURNAL, &conf->mddev->flags);
return test_bit(Faulty, &conf->log->rdev->flags);
}
diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c
index 0d1c6c4..8024cad 100644
--- a/drivers/md/raid5.c
+++ b/drivers/md/raid5.c
@@ -6810,6 +6810,14 @@ static int run(struct mddev *mddev)
if (IS_ERR(conf))
return PTR_ERR(conf);
+ if (test_bit(MD_HAS_JOURNAL, &mddev->flags) && !journal_dev) {
+ printk(KERN_ERR"md/raid:%s: journal disk is missing, "
+ "force array readonly\n",
+ mdname(mddev));
+ mddev->ro = 1;
+ set_disk_ro(mddev->gendisk, 1);
+ }
+
conf->min_offset_diff = min_offset_diff;
mddev->thread = conf->thread;
conf->thread = NULL;
--
2.4.6
^ permalink raw reply related
* [PATCH 5/9] MD: add new bit to indicate raid array with journal
From: Shaohua Li @ 2015-10-09 4:54 UTC (permalink / raw)
To: linux-raid; +Cc: Kernel-team, songliubraving, hch, dan.j.williams, neilb
In-Reply-To: <cover.1444360850.git.shli@fb.com>
From: Song Liu <songliubraving@fb.com>
If a raid array has journal feature bit set, add a new bit to indicate
this. If the array is started without journal disk existing, we know
there is something wrong.
Signed-off-by: Song Liu <songliubraving@fb.com>
Signed-off-by: Shaohua Li <shli@fb.com>
---
drivers/md/md.c | 10 +++++++---
drivers/md/md.h | 1 +
2 files changed, 8 insertions(+), 3 deletions(-)
diff --git a/drivers/md/md.c b/drivers/md/md.c
index daf42bb..a5b0bbd 100644
--- a/drivers/md/md.c
+++ b/drivers/md/md.c
@@ -1667,6 +1667,8 @@ static int super_1_validate(struct mddev *mddev, struct md_rdev *rdev)
set_bit(WriteMostly, &rdev->flags);
if (le32_to_cpu(sb->feature_map) & MD_FEATURE_REPLACEMENT)
set_bit(Replacement, &rdev->flags);
+ if (le32_to_cpu(sb->feature_map) & MD_FEATURE_JOURNAL)
+ set_bit(MD_HAS_JOURNAL, &mddev->flags);
} else /* MULTIPATH are always insync */
set_bit(In_sync, &rdev->flags);
@@ -1807,16 +1809,18 @@ static void super_1_sync(struct mddev *mddev, struct md_rdev *rdev)
for (i=0; i<max_dev;i++)
sb->dev_roles[i] = cpu_to_le16(MD_DISK_ROLE_FAULTY);
+ if (test_bit(MD_HAS_JOURNAL, &mddev->flags))
+ sb->feature_map |= MD_FEATURE_JOURNAL;
+
rdev_for_each(rdev2, mddev) {
i = rdev2->desc_nr;
if (test_bit(Faulty, &rdev2->flags))
sb->dev_roles[i] = cpu_to_le16(MD_DISK_ROLE_FAULTY);
else if (test_bit(In_sync, &rdev2->flags))
sb->dev_roles[i] = cpu_to_le16(rdev2->raid_disk);
- else if (test_bit(Journal, &rdev2->flags)) {
+ else if (test_bit(Journal, &rdev2->flags))
sb->dev_roles[i] = cpu_to_le16(MD_DISK_ROLE_JOURNAL);
- sb->feature_map |= cpu_to_le32(MD_FEATURE_JOURNAL);
- } else if (rdev2->raid_disk >= 0)
+ else if (rdev2->raid_disk >= 0)
sb->dev_roles[i] = cpu_to_le16(rdev2->raid_disk);
else
sb->dev_roles[i] = cpu_to_le16(MD_DISK_ROLE_SPARE);
diff --git a/drivers/md/md.h b/drivers/md/md.h
index e14e667..2bea51e 100644
--- a/drivers/md/md.h
+++ b/drivers/md/md.h
@@ -233,6 +233,7 @@ struct mddev {
* md_ioctl checked on it.
*/
#define MD_JOURNAL_CLEAN 5 /* A raid with journal is already clean */
+#define MD_HAS_JOURNAL 6 /* The raid array has journal feature set */
int suspended;
atomic_t active_io;
--
2.4.6
^ permalink raw reply related
* [PATCH 4/9] raid5-cache: IO error handling
From: Shaohua Li @ 2015-10-09 4:54 UTC (permalink / raw)
To: linux-raid; +Cc: Kernel-team, songliubraving, hch, dan.j.williams, neilb
In-Reply-To: <cover.1444360850.git.shli@fb.com>
There are 3 places the raid5-cache dispatches IO. The discard IO error
doesn't matter, so we ignore it. The superblock write IO error can be
handled in MD core. The remaining are log write and flush. When the IO
error happens, we mark log disk faulty and fail all write IO. Read IO is
still allowed to run. Userspace will get a notification too and
corresponding daemon can choose setting raid array readonly for example.
Signed-off-by: Shaohua Li <shli@fb.com>
---
drivers/md/raid5-cache.c | 15 ++++++++++++++-
drivers/md/raid5.c | 4 +++-
drivers/md/raid5.h | 2 ++
3 files changed, 19 insertions(+), 2 deletions(-)
diff --git a/drivers/md/raid5-cache.c b/drivers/md/raid5-cache.c
index 5dbe084..2cf6b72 100644
--- a/drivers/md/raid5-cache.c
+++ b/drivers/md/raid5-cache.c
@@ -190,7 +190,6 @@ static void r5l_io_run_stripes(struct r5l_io_unit *io)
}
}
-/* XXX: totally ignores I/O errors */
static void r5l_log_run_stripes(struct r5l_log *log)
{
struct r5l_io_unit *io, *next;
@@ -213,6 +212,9 @@ static void r5l_log_endio(struct bio *bio)
struct r5l_log *log = io->log;
unsigned long flags;
+ if (bio->bi_error)
+ md_error(log->rdev->mddev, log->rdev);
+
bio_put(bio);
spin_lock_irqsave(&log->io_list_lock, flags);
@@ -598,6 +600,9 @@ static void r5l_log_flush_endio(struct bio *bio)
unsigned long flags;
struct r5l_io_unit *io;
+ if (bio->bi_error)
+ md_error(log->rdev->mddev, log->rdev);
+
spin_lock_irqsave(&log->io_list_lock, flags);
list_for_each_entry(io, &log->flushing_ios, log_sibling)
r5l_io_run_stripes(io);
@@ -684,6 +689,7 @@ static void r5l_write_super_and_discard_space(struct r5l_log *log,
md_update_sb(mddev, 1);
}
+ /* discard IO error really doesn't matter, ignore it */
if (log->last_checkpoint < end) {
blkdev_issue_discard(bdev,
log->last_checkpoint + log->rdev->data_offset,
@@ -798,6 +804,13 @@ void r5l_quiesce(struct r5l_log *log, int state)
}
}
+bool r5l_log_disk_error(struct r5conf *conf)
+{
+ if (!conf->log)
+ return false;
+ return test_bit(Faulty, &conf->log->rdev->flags);
+}
+
struct r5l_recovery_ctx {
struct page *meta_page; /* current meta */
sector_t meta_total_blocks; /* total size of current meta and data */
diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c
index 6430098..0d1c6c4 100644
--- a/drivers/md/raid5.c
+++ b/drivers/md/raid5.c
@@ -3147,6 +3147,7 @@ handle_failed_stripe(struct r5conf *conf, struct stripe_head *sh,
* the data has not reached the cache yet.
*/
if (!test_bit(R5_Wantfill, &sh->dev[i].flags) &&
+ s->failed > conf->max_degraded &&
(!test_bit(R5_Insync, &sh->dev[i].flags) ||
test_bit(R5_ReadError, &sh->dev[i].flags))) {
spin_lock_irq(&sh->stripe_lock);
@@ -4015,6 +4016,7 @@ static void analyse_stripe(struct stripe_head *sh, struct stripe_head_state *s)
s->expanded = test_bit(STRIPE_EXPAND_READY, &sh->state) && !sh->batch_head;
s->failed_num[0] = -1;
s->failed_num[1] = -1;
+ s->log_failed = r5l_log_disk_error(conf);
/* Now to look around and see what can be done */
rcu_read_lock();
@@ -4358,7 +4360,7 @@ static void handle_stripe(struct stripe_head *sh)
/* check if the array has lost more than max_degraded devices and,
* if so, some requests might need to be failed.
*/
- if (s.failed > conf->max_degraded) {
+ if (s.failed > conf->max_degraded || s.log_failed) {
sh->check_state = 0;
sh->reconstruct_state = 0;
break_stripe_batch_list(sh, 0);
diff --git a/drivers/md/raid5.h b/drivers/md/raid5.h
index 1ab534c..a415e1c 100644
--- a/drivers/md/raid5.h
+++ b/drivers/md/raid5.h
@@ -272,6 +272,7 @@ struct stripe_head_state {
struct bio_list return_bi;
struct md_rdev *blocked_rdev;
int handle_bad_blocks;
+ int log_failed;
};
/* Flags for struct r5dev.flags */
@@ -631,4 +632,5 @@ extern void r5l_flush_stripe_to_raid(struct r5l_log *log);
extern void r5l_stripe_write_finished(struct stripe_head *sh);
extern int r5l_handle_flush_request(struct r5l_log *log, struct bio *bio);
extern void r5l_quiesce(struct r5l_log *log, int state);
+extern bool r5l_log_disk_error(struct r5conf *conf);
#endif
--
2.4.6
^ permalink raw reply related
* [PATCH 3/9] raid5: journal disk can't be removed
From: Shaohua Li @ 2015-10-09 4:54 UTC (permalink / raw)
To: linux-raid; +Cc: Kernel-team, songliubraving, hch, dan.j.williams, neilb
In-Reply-To: <cover.1444360850.git.shli@fb.com>
raid5-cache uses journal disk rdev->bdev, rdev->mddev in several places.
Don't allow journal disk disappear magically. On the other hand, we do
need to update superblock for other disks to bump up ->events, so next
time journal disk will be identified as stale.
Signed-off-by: Shaohua Li <shli@fb.com>
---
drivers/md/raid5.c | 11 +++++++++++
1 file changed, 11 insertions(+)
diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c
index 216fa3c..6430098 100644
--- a/drivers/md/raid5.c
+++ b/drivers/md/raid5.c
@@ -7128,6 +7128,15 @@ static int raid5_remove_disk(struct mddev *mddev, struct md_rdev *rdev)
struct disk_info *p = conf->disks + number;
print_raid5_conf(conf);
+ if (test_bit(Journal, &rdev->flags)) {
+ /*
+ * journal disk is not removable, but we need give a chance to
+ * update superblock of other disks. Otherwise journal disk
+ * will be considered as 'fresh'
+ * */
+ set_bit(MD_CHANGE_DEVS, &mddev->flags);
+ return -EINVAL;
+ }
if (rdev == p->rdev)
rdevp = &p->rdev;
else if (rdev == p->replacement)
@@ -7190,6 +7199,8 @@ static int raid5_add_disk(struct mddev *mddev, struct md_rdev *rdev)
int first = 0;
int last = conf->raid_disks - 1;
+ if (test_bit(Journal, &rdev->flags))
+ return -EINVAL;
if (mddev->recovery_disabled == conf->recovery_disabled)
return -EBUSY;
--
2.4.6
^ permalink raw reply related
* [PATCH 2/9] raid5-cache: add trim support for log
From: Shaohua Li @ 2015-10-09 4:54 UTC (permalink / raw)
To: linux-raid; +Cc: Kernel-team, songliubraving, hch, dan.j.williams, neilb
In-Reply-To: <cover.1444360850.git.shli@fb.com>
Since superblock is updated infrequently, we do a simple trim of log
disk (a synchronous trim)
Signed-off-by: Shaohua Li <shli@fb.com>
---
drivers/md/raid5-cache.c | 63 +++++++++++++++++++++++++++++++++++++++++++++++-
1 file changed, 62 insertions(+), 1 deletion(-)
diff --git a/drivers/md/raid5-cache.c b/drivers/md/raid5-cache.c
index caeec10..5dbe084 100644
--- a/drivers/md/raid5-cache.c
+++ b/drivers/md/raid5-cache.c
@@ -85,6 +85,7 @@ struct r5l_log {
spinlock_t no_space_stripes_lock;
bool need_cache_flush;
+ bool in_teardown;
};
/*
@@ -644,6 +645,60 @@ void r5l_flush_stripe_to_raid(struct r5l_log *log)
}
static void r5l_write_super(struct r5l_log *log, sector_t cp);
+static void r5l_write_super_and_discard_space(struct r5l_log *log,
+ sector_t end)
+{
+ struct block_device *bdev = log->rdev->bdev;
+ struct mddev *mddev;
+
+ r5l_write_super(log, end);
+
+ if (!blk_queue_discard(bdev_get_queue(bdev)))
+ return;
+
+ mddev = log->rdev->mddev;
+ /*
+ * This is to avoid a deadlock. r5l_quiesce holds reconfig_mutex and
+ * wait for this thread to finish. This thread waits for
+ * MD_CHANGE_PENDING clear, which is supposed to be done in
+ * md_check_recovery(). md_check_recovery() tries to get
+ * reconfig_mutex. Since r5l_quiesce already holds the mutex,
+ * md_check_recovery() fails, so the PENDING never get cleared. The
+ * in_teardown check workaround this issue.
+ * */
+ if (!log->in_teardown) {
+ set_bit(MD_CHANGE_DEVS, &mddev->flags);
+ set_bit(MD_CHANGE_PENDING, &mddev->flags);
+ md_wakeup_thread(mddev->thread);
+ wait_event(mddev->sb_wait,
+ !test_bit(MD_CHANGE_PENDING, &mddev->flags) ||
+ log->in_teardown);
+ /*
+ * r5l_quiesce could run after in_teardown check and hold
+ * mutex first. Superblock might get updated twice.
+ * */
+ if (log->in_teardown)
+ md_update_sb(mddev, 1);
+ } else {
+ BUG_ON(!mddev_is_locked(mddev));
+ md_update_sb(mddev, 1);
+ }
+
+ if (log->last_checkpoint < end) {
+ blkdev_issue_discard(bdev,
+ log->last_checkpoint + log->rdev->data_offset,
+ end - log->last_checkpoint, GFP_NOIO, 0);
+ } else {
+ blkdev_issue_discard(bdev,
+ log->last_checkpoint + log->rdev->data_offset,
+ log->device_size - log->last_checkpoint,
+ GFP_NOIO, 0);
+ blkdev_issue_discard(bdev, log->rdev->data_offset, end,
+ GFP_NOIO, 0);
+ }
+}
+
+
static void r5l_do_reclaim(struct r5l_log *log)
{
sector_t reclaim_target = xchg(&log->reclaim_target, 0);
@@ -685,7 +740,7 @@ static void r5l_do_reclaim(struct r5l_log *log)
* here, because the log area might be reused soon and we don't want to
* confuse recovery
*/
- r5l_write_super(log, next_checkpoint);
+ r5l_write_super_and_discard_space(log, next_checkpoint);
mutex_lock(&log->io_mutex);
log->last_checkpoint = next_checkpoint;
@@ -721,9 +776,11 @@ static void r5l_wake_reclaim(struct r5l_log *log, sector_t space)
void r5l_quiesce(struct r5l_log *log, int state)
{
+ struct mddev *mddev;
if (!log || state == 2)
return;
if (state == 0) {
+ log->in_teardown = 0;
log->reclaim_thread = md_register_thread(r5l_reclaim_thread,
log->rdev->mddev, "reclaim");
} else if (state == 1) {
@@ -731,6 +788,10 @@ void r5l_quiesce(struct r5l_log *log, int state)
* at this point all stripes are finished, so io_unit is at
* least in STRIPE_END state
*/
+ log->in_teardown = 1;
+ /* make sure r5l_write_super_and_discard_space exits */
+ mddev = log->rdev->mddev;
+ wake_up(&mddev->sb_wait);
r5l_wake_reclaim(log, -1L);
md_unregister_thread(&log->reclaim_thread);
r5l_do_reclaim(log);
--
2.4.6
^ permalink raw reply related
* [PATCH 1/9] MD: fix info output for journal disk
From: Shaohua Li @ 2015-10-09 4:54 UTC (permalink / raw)
To: linux-raid; +Cc: Kernel-team, songliubraving, hch, dan.j.williams, neilb
In-Reply-To: <cover.1444360850.git.shli@fb.com>
journal disk can be faulty. The Journal and Faulty aren't exclusive with
each other.
Signed-off-by: Shaohua Li <shli@fb.com>
---
drivers/md/md.c | 10 +++++++---
1 file changed, 7 insertions(+), 3 deletions(-)
diff --git a/drivers/md/md.c b/drivers/md/md.c
index 0729cc7..daf42bb 100644
--- a/drivers/md/md.c
+++ b/drivers/md/md.c
@@ -5857,7 +5857,8 @@ static int get_disk_info(struct mddev *mddev, void __user * arg)
else if (test_bit(In_sync, &rdev->flags)) {
info.state |= (1<<MD_DISK_ACTIVE);
info.state |= (1<<MD_DISK_SYNC);
- } else if (test_bit(Journal, &rdev->flags))
+ }
+ if (test_bit(Journal, &rdev->flags))
info.state |= (1<<MD_DISK_JOURNAL);
if (test_bit(WriteMostly, &rdev->flags))
info.state |= (1<<MD_DISK_WRITEMOSTLY);
@@ -7335,18 +7336,21 @@ static int md_seq_show(struct seq_file *seq, void *v)
rcu_read_lock();
rdev_for_each_rcu(rdev, mddev) {
char b[BDEVNAME_SIZE];
+ bool skip = false;
seq_printf(seq, " %s[%d]",
bdevname(rdev->bdev,b), rdev->desc_nr);
if (test_bit(WriteMostly, &rdev->flags))
seq_printf(seq, "(W)");
if (test_bit(Faulty, &rdev->flags)) {
seq_printf(seq, "(F)");
- continue;
+ skip = true;
}
if (test_bit(Journal, &rdev->flags)) {
seq_printf(seq, "(J)");
- continue;
+ skip = true;
}
+ if (skip)
+ continue;
if (rdev->raid_disk < 0)
seq_printf(seq, "(S)"); /* spare */
if (test_bit(Replacement, &rdev->flags))
--
2.4.6
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox