* Neil Brown is on leave
From: NeilBrown @ 2015-01-07 20:27 UTC (permalink / raw)
To: linux-raid
Cc: Artur Paszkiewicz, Manibalan P, 董杨, Anthony Wright,
Matt Callaghan
[-- Attachment #1: Type: text/plain, Size: 500 bytes --]
Hi,
various people have asked my things of late and may be a bit bothered by my
continuing silence.
I'm on leave at the moment and won't be thinking about RAID if I can help it
until I return to work in late January. If you haven't heard from me by the
second week of Feb, please ask again, but I will try to catch up by then.
I'll be at linux.conf.au and would be happy to chat about RAID in person
there, but otherwise I'm leaving it all until after Australia Day.
NeilBrown
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 811 bytes --]
^ permalink raw reply
* Re: Unable to make mdadm from git (fatal error: ansidecl.h)
From: Matt Callaghan @ 2015-01-07 14:47 UTC (permalink / raw)
To: linux-raid
In-Reply-To: <BLU436-SMTP110D88E7A927B5BBD9760981460@phx.gbl>
Turns out this functionality is available in latest mdadm 3.3.2 by using
"--data-offset=varable" and appending the offset to each device like:
<device_name>:<offset>. It also supports specifying in raw sectors.
-------- Original Message --------
From: Matt Callaghan <matt_callaghan@sympatico.ca>
Sent: Wed 07 Jan 2015 09:41:16 AM EST
To: linux-raid@vger.kernel.org
Cc:
Subject: Unable to make mdadm from git (fatal error: ansidecl.h)
> Hi Neil,
As per http://www.spinics.net/lists/raid/msg39353.html, I also am
running into a recreation of RAID6 problem where the devices had
differing offsets due to addition to an array over time (growth) in
between mdadm upgrades... it seems I need a version of mdadm that
supports creating an array with varying data offsets per device.
As per suggested by you:
git clone git://neil.brown.name/mdadm -b data_offset
cd mdadm
make
But it failed:
{{{
In file included from super0.c:27:0:
sha1.h:29:22: fatal error: ansidecl.h: No such file or directory
compilation terminated.
}}}
full output:
{{{
fermulator@fermmy-server:/usr/local/src/mdadm/mdadm_git/mdadm$ make
gcc -Wall -Werror -Wstrict-prototypes -Wextra -Wno-unused-parameter
-ggdb -DSendmail=\""/usr/sbin/sendmail -t"\"
-DCONFFILE=\"/etc/mdadm.conf\" -DCONFFILE2=\"/etc/mdadm/mdadm.conf\"
-DMAP_DIR=\"/run/mdadm\" -DMAP_FILE=\"map\" -DMDMON_DIR=\"/run/mdadm\"
-DFAILED_SLOTS_DIR=\"/run/mdadm/failed-slots\" -DUSE_PTHREADS -c -o
mdadm.o mdadm.c
gcc -Wall -Werror -Wstrict-prototypes -Wextra -Wno-unused-parameter
-ggdb -DSendmail=\""/usr/sbin/sendmail -t"\"
-DCONFFILE=\"/etc/mdadm.conf\" -DCONFFILE2=\"/etc/mdadm/mdadm.conf\"
-DMAP_DIR=\"/run/mdadm\" -DMAP_FILE=\"map\" -DMDMON_DIR=\"/run/mdadm\"
-DFAILED_SLOTS_DIR=\"/run/mdadm/failed-slots\" -DUSE_PTHREADS -c -o
config.o config.c
gcc -Wall -Werror -Wstrict-prototypes -Wextra -Wno-unused-parameter
-ggdb -DSendmail=\""/usr/sbin/sendmail -t"\"
-DCONFFILE=\"/etc/mdadm.conf\" -DCONFFILE2=\"/etc/mdadm/mdadm.conf\"
-DMAP_DIR=\"/run/mdadm\" -DMAP_FILE=\"map\" -DMDMON_DIR=\"/run/mdadm\"
-DFAILED_SLOTS_DIR=\"/run/mdadm/failed-slots\" -DUSE_PTHREADS -c -o
policy.o policy.c
gcc -Wall -Werror -Wstrict-prototypes -Wextra -Wno-unused-parameter
-ggdb -DSendmail=\""/usr/sbin/sendmail -t"\"
-DCONFFILE=\"/etc/mdadm.conf\" -DCONFFILE2=\"/etc/mdadm/mdadm.conf\"
-DMAP_DIR=\"/run/mdadm\" -DMAP_FILE=\"map\" -DMDMON_DIR=\"/run/mdadm\"
-DFAILED_SLOTS_DIR=\"/run/mdadm/failed-slots\" -DUSE_PTHREADS -c -o
mdstat.o mdstat.c
gcc -Wall -Werror -Wstrict-prototypes -Wextra -Wno-unused-parameter
-ggdb -DSendmail=\""/usr/sbin/sendmail -t"\"
-DCONFFILE=\"/etc/mdadm.conf\" -DCONFFILE2=\"/etc/mdadm/mdadm.conf\"
-DMAP_DIR=\"/run/mdadm\" -DMAP_FILE=\"map\" -DMDMON_DIR=\"/run/mdadm\"
-DFAILED_SLOTS_DIR=\"/run/mdadm/failed-slots\" -DUSE_PTHREADS -c -o
ReadMe.o ReadMe.c
gcc -Wall -Werror -Wstrict-prototypes -Wextra -Wno-unused-parameter
-ggdb -DSendmail=\""/usr/sbin/sendmail -t"\"
-DCONFFILE=\"/etc/mdadm.conf\" -DCONFFILE2=\"/etc/mdadm/mdadm.conf\"
-DMAP_DIR=\"/run/mdadm\" -DMAP_FILE=\"map\" -DMDMON_DIR=\"/run/mdadm\"
-DFAILED_SLOTS_DIR=\"/run/mdadm/failed-slots\" -DUSE_PTHREADS -c -o
util.o util.c
gcc -Wall -Werror -Wstrict-prototypes -Wextra -Wno-unused-parameter
-ggdb -DSendmail=\""/usr/sbin/sendmail -t"\"
-DCONFFILE=\"/etc/mdadm.conf\" -DCONFFILE2=\"/etc/mdadm/mdadm.conf\"
-DMAP_DIR=\"/run/mdadm\" -DMAP_FILE=\"map\" -DMDMON_DIR=\"/run/mdadm\"
-DFAILED_SLOTS_DIR=\"/run/mdadm/failed-slots\" -DUSE_PTHREADS -c -o
maps.o maps.c
gcc -Wall -Werror -Wstrict-prototypes -Wextra -Wno-unused-parameter
-ggdb -DSendmail=\""/usr/sbin/sendmail -t"\"
-DCONFFILE=\"/etc/mdadm.conf\" -DCONFFILE2=\"/etc/mdadm/mdadm.conf\"
-DMAP_DIR=\"/run/mdadm\" -DMAP_FILE=\"map\" -DMDMON_DIR=\"/run/mdadm\"
-DFAILED_SLOTS_DIR=\"/run/mdadm/failed-slots\" -DUSE_PTHREADS -c -o
lib.o lib.c
gcc -Wall -Werror -Wstrict-prototypes -Wextra -Wno-unused-parameter
-ggdb -DSendmail=\""/usr/sbin/sendmail -t"\"
-DCONFFILE=\"/etc/mdadm.conf\" -DCONFFILE2=\"/etc/mdadm/mdadm.conf\"
-DMAP_DIR=\"/run/mdadm\" -DMAP_FILE=\"map\" -DMDMON_DIR=\"/run/mdadm\"
-DFAILED_SLOTS_DIR=\"/run/mdadm/failed-slots\" -DUSE_PTHREADS -c -o
Manage.o Manage.c
gcc -Wall -Werror -Wstrict-prototypes -Wextra -Wno-unused-parameter
-ggdb -DSendmail=\""/usr/sbin/sendmail -t"\"
-DCONFFILE=\"/etc/mdadm.conf\" -DCONFFILE2=\"/etc/mdadm/mdadm.conf\"
-DMAP_DIR=\"/run/mdadm\" -DMAP_FILE=\"map\" -DMDMON_DIR=\"/run/mdadm\"
-DFAILED_SLOTS_DIR=\"/run/mdadm/failed-slots\" -DUSE_PTHREADS -c -o
Assemble.o Assemble.c
gcc -Wall -Werror -Wstrict-prototypes -Wextra -Wno-unused-parameter
-ggdb -DSendmail=\""/usr/sbin/sendmail -t"\"
-DCONFFILE=\"/etc/mdadm.conf\" -DCONFFILE2=\"/etc/mdadm/mdadm.conf\"
-DMAP_DIR=\"/run/mdadm\" -DMAP_FILE=\"map\" -DMDMON_DIR=\"/run/mdadm\"
-DFAILED_SLOTS_DIR=\"/run/mdadm/failed-slots\" -DUSE_PTHREADS -c -o
Build.o Build.c
gcc -Wall -Werror -Wstrict-prototypes -Wextra -Wno-unused-parameter
-ggdb -DSendmail=\""/usr/sbin/sendmail -t"\"
-DCONFFILE=\"/etc/mdadm.conf\" -DCONFFILE2=\"/etc/mdadm/mdadm.conf\"
-DMAP_DIR=\"/run/mdadm\" -DMAP_FILE=\"map\" -DMDMON_DIR=\"/run/mdadm\"
-DFAILED_SLOTS_DIR=\"/run/mdadm/failed-slots\" -DUSE_PTHREADS -c -o
Create.o Create.c
gcc -Wall -Werror -Wstrict-prototypes -Wextra -Wno-unused-parameter
-ggdb -DSendmail=\""/usr/sbin/sendmail -t"\"
-DCONFFILE=\"/etc/mdadm.conf\" -DCONFFILE2=\"/etc/mdadm/mdadm.conf\"
-DMAP_DIR=\"/run/mdadm\" -DMAP_FILE=\"map\" -DMDMON_DIR=\"/run/mdadm\"
-DFAILED_SLOTS_DIR=\"/run/mdadm/failed-slots\" -DUSE_PTHREADS -c -o
Detail.o Detail.c
gcc -Wall -Werror -Wstrict-prototypes -Wextra -Wno-unused-parameter
-ggdb -DSendmail=\""/usr/sbin/sendmail -t"\"
-DCONFFILE=\"/etc/mdadm.conf\" -DCONFFILE2=\"/etc/mdadm/mdadm.conf\"
-DMAP_DIR=\"/run/mdadm\" -DMAP_FILE=\"map\" -DMDMON_DIR=\"/run/mdadm\"
-DFAILED_SLOTS_DIR=\"/run/mdadm/failed-slots\" -DUSE_PTHREADS -c -o
Examine.o Examine.c
gcc -Wall -Werror -Wstrict-prototypes -Wextra -Wno-unused-parameter
-ggdb -DSendmail=\""/usr/sbin/sendmail -t"\"
-DCONFFILE=\"/etc/mdadm.conf\" -DCONFFILE2=\"/etc/mdadm/mdadm.conf\"
-DMAP_DIR=\"/run/mdadm\" -DMAP_FILE=\"map\" -DMDMON_DIR=\"/run/mdadm\"
-DFAILED_SLOTS_DIR=\"/run/mdadm/failed-slots\" -DUSE_PTHREADS -c -o
Grow.o Grow.c
gcc -Wall -Werror -Wstrict-prototypes -Wextra -Wno-unused-parameter
-ggdb -DSendmail=\""/usr/sbin/sendmail -t"\"
-DCONFFILE=\"/etc/mdadm.conf\" -DCONFFILE2=\"/etc/mdadm/mdadm.conf\"
-DMAP_DIR=\"/run/mdadm\" -DMAP_FILE=\"map\" -DMDMON_DIR=\"/run/mdadm\"
-DFAILED_SLOTS_DIR=\"/run/mdadm/failed-slots\" -DUSE_PTHREADS -c -o
Monitor.o Monitor.c
gcc -Wall -Werror -Wstrict-prototypes -Wextra -Wno-unused-parameter
-ggdb -DSendmail=\""/usr/sbin/sendmail -t"\"
-DCONFFILE=\"/etc/mdadm.conf\" -DCONFFILE2=\"/etc/mdadm/mdadm.conf\"
-DMAP_DIR=\"/run/mdadm\" -DMAP_FILE=\"map\" -DMDMON_DIR=\"/run/mdadm\"
-DFAILED_SLOTS_DIR=\"/run/mdadm/failed-slots\" -DUSE_PTHREADS -c -o
dlink.o dlink.c
gcc -Wall -Werror -Wstrict-prototypes -Wextra -Wno-unused-parameter
-ggdb -DSendmail=\""/usr/sbin/sendmail -t"\"
-DCONFFILE=\"/etc/mdadm.conf\" -DCONFFILE2=\"/etc/mdadm/mdadm.conf\"
-DMAP_DIR=\"/run/mdadm\" -DMAP_FILE=\"map\" -DMDMON_DIR=\"/run/mdadm\"
-DFAILED_SLOTS_DIR=\"/run/mdadm/failed-slots\" -DUSE_PTHREADS -c -o
Kill.o Kill.c
gcc -Wall -Werror -Wstrict-prototypes -Wextra -Wno-unused-parameter
-ggdb -DSendmail=\""/usr/sbin/sendmail -t"\"
-DCONFFILE=\"/etc/mdadm.conf\" -DCONFFILE2=\"/etc/mdadm/mdadm.conf\"
-DMAP_DIR=\"/run/mdadm\" -DMAP_FILE=\"map\" -DMDMON_DIR=\"/run/mdadm\"
-DFAILED_SLOTS_DIR=\"/run/mdadm/failed-slots\" -DUSE_PTHREADS -c -o
Query.o Query.c
gcc -Wall -Werror -Wstrict-prototypes -Wextra -Wno-unused-parameter
-ggdb -DSendmail=\""/usr/sbin/sendmail -t"\"
-DCONFFILE=\"/etc/mdadm.conf\" -DCONFFILE2=\"/etc/mdadm/mdadm.conf\"
-DMAP_DIR=\"/run/mdadm\" -DMAP_FILE=\"map\" -DMDMON_DIR=\"/run/mdadm\"
-DFAILED_SLOTS_DIR=\"/run/mdadm/failed-slots\" -DUSE_PTHREADS -c -o
Incremental.o Incremental.c
gcc -Wall -Werror -Wstrict-prototypes -Wextra -Wno-unused-parameter
-ggdb -DSendmail=\""/usr/sbin/sendmail -t"\"
-DCONFFILE=\"/etc/mdadm.conf\" -DCONFFILE2=\"/etc/mdadm/mdadm.conf\"
-DMAP_DIR=\"/run/mdadm\" -DMAP_FILE=\"map\" -DMDMON_DIR=\"/run/mdadm\"
-DFAILED_SLOTS_DIR=\"/run/mdadm/failed-slots\" -DUSE_PTHREADS -c -o
mdopen.o mdopen.c
gcc -Wall -Werror -Wstrict-prototypes -Wextra -Wno-unused-parameter
-ggdb -DSendmail=\""/usr/sbin/sendmail -t"\"
-DCONFFILE=\"/etc/mdadm.conf\" -DCONFFILE2=\"/etc/mdadm/mdadm.conf\"
-DMAP_DIR=\"/run/mdadm\" -DMAP_FILE=\"map\" -DMDMON_DIR=\"/run/mdadm\"
-DFAILED_SLOTS_DIR=\"/run/mdadm/failed-slots\" -DUSE_PTHREADS -c -o
super0.o super0.c
In file included from super0.c:27:0:
sha1.h:29:22: fatal error: ansidecl.h: No such file or directory
compilation terminated.
}}}
Anything I need to do different? or is that git branch unstable atm?
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Unable to make mdadm from git (fatal error: ansidecl.h)
From: Matt Callaghan @ 2015-01-07 14:41 UTC (permalink / raw)
To: linux-raid
Hi Neil,
As per http://www.spinics.net/lists/raid/msg39353.html, I also am
running into a recreation of RAID6 problem where the devices had
differing offsets due to addition to an array over time (growth) in
between mdadm upgrades... it seems I need a version of mdadm that
supports creating an array with varying data offsets per device.
As per suggested by you:
git clone git://neil.brown.name/mdadm -b data_offset
cd mdadm
make
But it failed:
{{{
In file included from super0.c:27:0:
sha1.h:29:22: fatal error: ansidecl.h: No such file or directory
compilation terminated.
}}}
full output:
{{{
fermulator@fermmy-server:/usr/local/src/mdadm/mdadm_git/mdadm$ make
gcc -Wall -Werror -Wstrict-prototypes -Wextra -Wno-unused-parameter
-ggdb -DSendmail=\""/usr/sbin/sendmail -t"\"
-DCONFFILE=\"/etc/mdadm.conf\" -DCONFFILE2=\"/etc/mdadm/mdadm.conf\"
-DMAP_DIR=\"/run/mdadm\" -DMAP_FILE=\"map\" -DMDMON_DIR=\"/run/mdadm\"
-DFAILED_SLOTS_DIR=\"/run/mdadm/failed-slots\" -DUSE_PTHREADS -c -o
mdadm.o mdadm.c
gcc -Wall -Werror -Wstrict-prototypes -Wextra -Wno-unused-parameter
-ggdb -DSendmail=\""/usr/sbin/sendmail -t"\"
-DCONFFILE=\"/etc/mdadm.conf\" -DCONFFILE2=\"/etc/mdadm/mdadm.conf\"
-DMAP_DIR=\"/run/mdadm\" -DMAP_FILE=\"map\" -DMDMON_DIR=\"/run/mdadm\"
-DFAILED_SLOTS_DIR=\"/run/mdadm/failed-slots\" -DUSE_PTHREADS -c -o
config.o config.c
gcc -Wall -Werror -Wstrict-prototypes -Wextra -Wno-unused-parameter
-ggdb -DSendmail=\""/usr/sbin/sendmail -t"\"
-DCONFFILE=\"/etc/mdadm.conf\" -DCONFFILE2=\"/etc/mdadm/mdadm.conf\"
-DMAP_DIR=\"/run/mdadm\" -DMAP_FILE=\"map\" -DMDMON_DIR=\"/run/mdadm\"
-DFAILED_SLOTS_DIR=\"/run/mdadm/failed-slots\" -DUSE_PTHREADS -c -o
policy.o policy.c
gcc -Wall -Werror -Wstrict-prototypes -Wextra -Wno-unused-parameter
-ggdb -DSendmail=\""/usr/sbin/sendmail -t"\"
-DCONFFILE=\"/etc/mdadm.conf\" -DCONFFILE2=\"/etc/mdadm/mdadm.conf\"
-DMAP_DIR=\"/run/mdadm\" -DMAP_FILE=\"map\" -DMDMON_DIR=\"/run/mdadm\"
-DFAILED_SLOTS_DIR=\"/run/mdadm/failed-slots\" -DUSE_PTHREADS -c -o
mdstat.o mdstat.c
gcc -Wall -Werror -Wstrict-prototypes -Wextra -Wno-unused-parameter
-ggdb -DSendmail=\""/usr/sbin/sendmail -t"\"
-DCONFFILE=\"/etc/mdadm.conf\" -DCONFFILE2=\"/etc/mdadm/mdadm.conf\"
-DMAP_DIR=\"/run/mdadm\" -DMAP_FILE=\"map\" -DMDMON_DIR=\"/run/mdadm\"
-DFAILED_SLOTS_DIR=\"/run/mdadm/failed-slots\" -DUSE_PTHREADS -c -o
ReadMe.o ReadMe.c
gcc -Wall -Werror -Wstrict-prototypes -Wextra -Wno-unused-parameter
-ggdb -DSendmail=\""/usr/sbin/sendmail -t"\"
-DCONFFILE=\"/etc/mdadm.conf\" -DCONFFILE2=\"/etc/mdadm/mdadm.conf\"
-DMAP_DIR=\"/run/mdadm\" -DMAP_FILE=\"map\" -DMDMON_DIR=\"/run/mdadm\"
-DFAILED_SLOTS_DIR=\"/run/mdadm/failed-slots\" -DUSE_PTHREADS -c -o
util.o util.c
gcc -Wall -Werror -Wstrict-prototypes -Wextra -Wno-unused-parameter
-ggdb -DSendmail=\""/usr/sbin/sendmail -t"\"
-DCONFFILE=\"/etc/mdadm.conf\" -DCONFFILE2=\"/etc/mdadm/mdadm.conf\"
-DMAP_DIR=\"/run/mdadm\" -DMAP_FILE=\"map\" -DMDMON_DIR=\"/run/mdadm\"
-DFAILED_SLOTS_DIR=\"/run/mdadm/failed-slots\" -DUSE_PTHREADS -c -o
maps.o maps.c
gcc -Wall -Werror -Wstrict-prototypes -Wextra -Wno-unused-parameter
-ggdb -DSendmail=\""/usr/sbin/sendmail -t"\"
-DCONFFILE=\"/etc/mdadm.conf\" -DCONFFILE2=\"/etc/mdadm/mdadm.conf\"
-DMAP_DIR=\"/run/mdadm\" -DMAP_FILE=\"map\" -DMDMON_DIR=\"/run/mdadm\"
-DFAILED_SLOTS_DIR=\"/run/mdadm/failed-slots\" -DUSE_PTHREADS -c -o
lib.o lib.c
gcc -Wall -Werror -Wstrict-prototypes -Wextra -Wno-unused-parameter
-ggdb -DSendmail=\""/usr/sbin/sendmail -t"\"
-DCONFFILE=\"/etc/mdadm.conf\" -DCONFFILE2=\"/etc/mdadm/mdadm.conf\"
-DMAP_DIR=\"/run/mdadm\" -DMAP_FILE=\"map\" -DMDMON_DIR=\"/run/mdadm\"
-DFAILED_SLOTS_DIR=\"/run/mdadm/failed-slots\" -DUSE_PTHREADS -c -o
Manage.o Manage.c
gcc -Wall -Werror -Wstrict-prototypes -Wextra -Wno-unused-parameter
-ggdb -DSendmail=\""/usr/sbin/sendmail -t"\"
-DCONFFILE=\"/etc/mdadm.conf\" -DCONFFILE2=\"/etc/mdadm/mdadm.conf\"
-DMAP_DIR=\"/run/mdadm\" -DMAP_FILE=\"map\" -DMDMON_DIR=\"/run/mdadm\"
-DFAILED_SLOTS_DIR=\"/run/mdadm/failed-slots\" -DUSE_PTHREADS -c -o
Assemble.o Assemble.c
gcc -Wall -Werror -Wstrict-prototypes -Wextra -Wno-unused-parameter
-ggdb -DSendmail=\""/usr/sbin/sendmail -t"\"
-DCONFFILE=\"/etc/mdadm.conf\" -DCONFFILE2=\"/etc/mdadm/mdadm.conf\"
-DMAP_DIR=\"/run/mdadm\" -DMAP_FILE=\"map\" -DMDMON_DIR=\"/run/mdadm\"
-DFAILED_SLOTS_DIR=\"/run/mdadm/failed-slots\" -DUSE_PTHREADS -c -o
Build.o Build.c
gcc -Wall -Werror -Wstrict-prototypes -Wextra -Wno-unused-parameter
-ggdb -DSendmail=\""/usr/sbin/sendmail -t"\"
-DCONFFILE=\"/etc/mdadm.conf\" -DCONFFILE2=\"/etc/mdadm/mdadm.conf\"
-DMAP_DIR=\"/run/mdadm\" -DMAP_FILE=\"map\" -DMDMON_DIR=\"/run/mdadm\"
-DFAILED_SLOTS_DIR=\"/run/mdadm/failed-slots\" -DUSE_PTHREADS -c -o
Create.o Create.c
gcc -Wall -Werror -Wstrict-prototypes -Wextra -Wno-unused-parameter
-ggdb -DSendmail=\""/usr/sbin/sendmail -t"\"
-DCONFFILE=\"/etc/mdadm.conf\" -DCONFFILE2=\"/etc/mdadm/mdadm.conf\"
-DMAP_DIR=\"/run/mdadm\" -DMAP_FILE=\"map\" -DMDMON_DIR=\"/run/mdadm\"
-DFAILED_SLOTS_DIR=\"/run/mdadm/failed-slots\" -DUSE_PTHREADS -c -o
Detail.o Detail.c
gcc -Wall -Werror -Wstrict-prototypes -Wextra -Wno-unused-parameter
-ggdb -DSendmail=\""/usr/sbin/sendmail -t"\"
-DCONFFILE=\"/etc/mdadm.conf\" -DCONFFILE2=\"/etc/mdadm/mdadm.conf\"
-DMAP_DIR=\"/run/mdadm\" -DMAP_FILE=\"map\" -DMDMON_DIR=\"/run/mdadm\"
-DFAILED_SLOTS_DIR=\"/run/mdadm/failed-slots\" -DUSE_PTHREADS -c -o
Examine.o Examine.c
gcc -Wall -Werror -Wstrict-prototypes -Wextra -Wno-unused-parameter
-ggdb -DSendmail=\""/usr/sbin/sendmail -t"\"
-DCONFFILE=\"/etc/mdadm.conf\" -DCONFFILE2=\"/etc/mdadm/mdadm.conf\"
-DMAP_DIR=\"/run/mdadm\" -DMAP_FILE=\"map\" -DMDMON_DIR=\"/run/mdadm\"
-DFAILED_SLOTS_DIR=\"/run/mdadm/failed-slots\" -DUSE_PTHREADS -c -o
Grow.o Grow.c
gcc -Wall -Werror -Wstrict-prototypes -Wextra -Wno-unused-parameter
-ggdb -DSendmail=\""/usr/sbin/sendmail -t"\"
-DCONFFILE=\"/etc/mdadm.conf\" -DCONFFILE2=\"/etc/mdadm/mdadm.conf\"
-DMAP_DIR=\"/run/mdadm\" -DMAP_FILE=\"map\" -DMDMON_DIR=\"/run/mdadm\"
-DFAILED_SLOTS_DIR=\"/run/mdadm/failed-slots\" -DUSE_PTHREADS -c -o
Monitor.o Monitor.c
gcc -Wall -Werror -Wstrict-prototypes -Wextra -Wno-unused-parameter
-ggdb -DSendmail=\""/usr/sbin/sendmail -t"\"
-DCONFFILE=\"/etc/mdadm.conf\" -DCONFFILE2=\"/etc/mdadm/mdadm.conf\"
-DMAP_DIR=\"/run/mdadm\" -DMAP_FILE=\"map\" -DMDMON_DIR=\"/run/mdadm\"
-DFAILED_SLOTS_DIR=\"/run/mdadm/failed-slots\" -DUSE_PTHREADS -c -o
dlink.o dlink.c
gcc -Wall -Werror -Wstrict-prototypes -Wextra -Wno-unused-parameter
-ggdb -DSendmail=\""/usr/sbin/sendmail -t"\"
-DCONFFILE=\"/etc/mdadm.conf\" -DCONFFILE2=\"/etc/mdadm/mdadm.conf\"
-DMAP_DIR=\"/run/mdadm\" -DMAP_FILE=\"map\" -DMDMON_DIR=\"/run/mdadm\"
-DFAILED_SLOTS_DIR=\"/run/mdadm/failed-slots\" -DUSE_PTHREADS -c -o
Kill.o Kill.c
gcc -Wall -Werror -Wstrict-prototypes -Wextra -Wno-unused-parameter
-ggdb -DSendmail=\""/usr/sbin/sendmail -t"\"
-DCONFFILE=\"/etc/mdadm.conf\" -DCONFFILE2=\"/etc/mdadm/mdadm.conf\"
-DMAP_DIR=\"/run/mdadm\" -DMAP_FILE=\"map\" -DMDMON_DIR=\"/run/mdadm\"
-DFAILED_SLOTS_DIR=\"/run/mdadm/failed-slots\" -DUSE_PTHREADS -c -o
Query.o Query.c
gcc -Wall -Werror -Wstrict-prototypes -Wextra -Wno-unused-parameter
-ggdb -DSendmail=\""/usr/sbin/sendmail -t"\"
-DCONFFILE=\"/etc/mdadm.conf\" -DCONFFILE2=\"/etc/mdadm/mdadm.conf\"
-DMAP_DIR=\"/run/mdadm\" -DMAP_FILE=\"map\" -DMDMON_DIR=\"/run/mdadm\"
-DFAILED_SLOTS_DIR=\"/run/mdadm/failed-slots\" -DUSE_PTHREADS -c -o
Incremental.o Incremental.c
gcc -Wall -Werror -Wstrict-prototypes -Wextra -Wno-unused-parameter
-ggdb -DSendmail=\""/usr/sbin/sendmail -t"\"
-DCONFFILE=\"/etc/mdadm.conf\" -DCONFFILE2=\"/etc/mdadm/mdadm.conf\"
-DMAP_DIR=\"/run/mdadm\" -DMAP_FILE=\"map\" -DMDMON_DIR=\"/run/mdadm\"
-DFAILED_SLOTS_DIR=\"/run/mdadm/failed-slots\" -DUSE_PTHREADS -c -o
mdopen.o mdopen.c
gcc -Wall -Werror -Wstrict-prototypes -Wextra -Wno-unused-parameter
-ggdb -DSendmail=\""/usr/sbin/sendmail -t"\"
-DCONFFILE=\"/etc/mdadm.conf\" -DCONFFILE2=\"/etc/mdadm/mdadm.conf\"
-DMAP_DIR=\"/run/mdadm\" -DMAP_FILE=\"map\" -DMDMON_DIR=\"/run/mdadm\"
-DFAILED_SLOTS_DIR=\"/run/mdadm/failed-slots\" -DUSE_PTHREADS -c -o
super0.o super0.c
In file included from super0.c:27:0:
sha1.h:29:22: fatal error: ansidecl.h: No such file or directory
compilation terminated.
}}}
Anything I need to do different? or is that git branch unstable atm?
^ permalink raw reply
* Re: mdadm RAID6 "active" with spares and failed disks; need help
From: Matt Callaghan @ 2015-01-07 13:34 UTC (permalink / raw)
To: linux-raid
In-Reply-To: <54ABEE54.6020707@sympatico.ca>
Just to give a small update (I realize many people may still be on holidays)
I've tried to work with a few people on IRC, and in conjunction with
lots of reading from others' experiences attempting to recover the array
but no luck yet.
I /hope/ I haven't ruined anything.
The forum post referenced below has full details, but here's a summary
of "what happened"
notice how some drives are "moving" around :( [either due to a mistake I
made, or the server haulting/lockup during rebuilds, I'm not sure]
{{{
-------------------------------------------------------------------------------------------
| | Device Role #
-------------------------------------------------------------------------------------------
| DEVICE | COMMENTS | Dec GOOD | Jan4 6:28AM | 12:10PM | 12:40PM |
Jan5 12:30AM | 12:50AM | 8:30AM | 6:34PM | Jan6 6:45AM |
-------------------------------------------------------------------------------------------
| /dev/sdi | | 4 | 4 | 4 | 4 |
4 | 4 | 4 | 4 | 4 |
| /dev/sdj | failing | 5 | 5 FAIL | ( ) | 8 |
8 | 8 FAIL | ( ) | ( ) | ( ) |
| /dev/sdk | failing? | 0 | 0 | 0 | 0 |
0 | 0 | 0 | 0 FAIL | 0 FAIL |
| /dev/sdl | | 6 | 6 | 6 | 6 |
6 | 6 | 6 | 6 | 6 |
| /dev/sdm | | 1 | 1 | 1 | 1 |
( ) | ( ) | ( ) | 8 | 8 SPARE |
| /dev/sdn | | 2 | 2 | 2 | 2 |
2 | 2 | 2 | 2 | 2 |
| /dev/sdo | | 3 | 3 | 3 | 3 |
3 | 3 | 3 | 3 | 3 |
| /dev/sdp | | 7 | 7 | 7 | 7 |
7 | 7 | 7 | 7 | 7 |
-------------------------------------------------------------------------------------------
}}}
Full details from my e-mail notifications of /proc/mdstat (although
unfortunately I don't have FULL mdadm --detail/examine information per
state transition)
{{{
Dec GOOD
md2000 : active raid6 sdo1[3] sdj1[5] sdk1[0] sdi1[4] sdn1[2] sdm1[1]
sdp1[7] sdl1[6]
11721080448 blocks super 1.1 level 6, 64k chunk, algorithm 2
[8/8] [UUUUUUUU]
FAIL EVENT on Jan 4th @ 6:28AM
md2000 : active raid6 sdo1[3] sdj1[5](F) sdk1[0] sdi1[4] sdn1[2] sdm1[1]
sdp1[7] sdl1[6]
11721080448 blocks super 1.1 level 6, 64k chunk, algorithm 2
[8/7] [UUUU_UUU]
[==============>......] check = 73.6% (1439539228/1953513408)
finish=536.6min speed=15960K/sec
DEGRADED EVENT on Jan 4th @ 6:39AM
md2000 : active raid6 sdo1[3] sdj1[5](F) sdk1[0] sdi1[4] sdn1[2] sdm1[1]
sdp1[7] sdl1[6]
11721080448 blocks super 1.1 level 6, 64k chunk, algorithm 2
[8/7] [UUUU_UUU]
[==============>......] check = 73.6% (1439539228/1953513408)
finish=5091.8min speed=1682K/sec
DEGRADED EVENT on Jan 4th @ 12:10PM
md2000 : active raid6 sdo1[3] sdn1[2] sdi1[4] sdm1[1] sdk1[0] sdp1[7]
sdl1[6]
11721080448 blocks super 1.1 level 6, 64k chunk, algorithm 2
[8/7] [UUUU_UUU]
DEGRADED EVENT on Jan 4th @ 12:21PM
md2000 : active raid6 sdk1[0] sdo1[3] sdm1[1] sdn1[2] sdi1[4] sdp1[7]
sdl1[6]
11721080448 blocks super 1.1 level 6, 64k chunk, algorithm 2
[8/7] [UUUU_UUU]
DEGRADED EVENT on Jan 4th @ 12:40PM
md2000 : active raid6 sdj1[8] sdm1[1] sdo1[3] sdn1[2] sdk1[0] sdi1[4]
sdp1[7] sdl1[6]
11721080448 blocks super 1.1 level 6, 64k chunk, algorithm 2
[8/7] [UUUU_UUU]
[>....................] recovery = 0.2% (5137892/1953513408)
finish=921.7min speed=35227K/sec
DEGRADED EVENT on Jan 5th @ 12:30AM
md2000 : active raid6 sdk1[0] sdo1[3] sdn1[2] sdj1[8] sdi1[4] sdl1[6]
sdp1[7]
11721080448 blocks super 1.1 level 6, 64k chunk, algorithm 2
[8/6] [U_UU_UUU]
[============>........] recovery = 62.9% (1229102028/1953513408)
finish=259.8min speed=46466K/sec
FAIL SPARE EVENT on Jan 5th @ 12:50AM
md2000 : active raid6 sdk1[0] sdo1[3] sdn1[2] sdj1[8](F) sdi1[4] sdl1[6]
sdp1[7]
11721080448 blocks super 1.1 level 6, 64k chunk, algorithm 2
[8/6] [U_UU_UUU]
[=============>.......] recovery = 68.1% (1332029020/1953513408)
finish=150.3min speed=68897K/sec
DEGRADED EVENT on Jan 5th @ 6:43AM
md2000 : active raid6 sdk1[0] sdo1[3] sdn1[2] sdj1[8](F) sdi1[4] sdl1[6]
sdp1[7]
11721080448 blocks super 1.1 level 6, 64k chunk, algorithm 2
[8/6] [U_UU_UUU]
[=============>.......] recovery = 68.1% (1332029020/1953513408)
finish=76028.6min speed=136K/sec
TEST MESSAGE on Jan 5th @ 8:30AM
md2000 : active raid6 sdo1[3] sdi1[4] sdn1[2] sdk1[0] sdl1[6] sdp1[7]
11721080448 blocks super 1.1 level 6, 64k chunk, algorithm 2
[8/6] [U_UU_UUU]
}}}
I've tried mdadm --create --assume-clean for several combinations of the
"device role # ordering", but so far none have exposed a usable ext4
partition for /dev/md2000.
Was speaking with someone on IRC, and it's been shown that the data
offset for the devices has changed over time in mdadm, so I need to
recompile mdadm 3.3.x and attempt it that way.
I'll update when I get to trying that.
~Fermmy
-------- Original Message --------
From: Matt Callaghan <matt_callaghan@sympatico.ca>
Sent: Tue 06 Jan 2015 09:16:52 AM EST
To: linux-raid@vger.kernel.org
Cc:
Subject: mdadm RAID6 "active" with spares and failed disks; need help
> I think I'm in a really bad state. Could an expert w/ mdadm please
> help?
I have a RAID6 mdadm device, and it got really messed up with spares:
{{{
md2000 : active raid6 sdm1[8](S) sdo1[3] sdi1[4] sdn1[2] sdk1[0](F)
sdl1[6] sdp1[7]
11721080448 blocks super 1.1 level 6, 64k chunk, algorithm 2
[8/5] [__UU_UUU]
}}}
And is now really broken (inactive)
{{{
md2000 : inactive sdn1[2](S) sdm1[8](S) sdl1[6](S) sdp1[7](S) sdi1[4](S)
sdo1[3](S) sdk1[0](S)
13674593976 blocks super 1.1
}}}
I have a forum post going w/ full details
http://www.linuxquestions.org/questions/linux-server-73/mdadm-raid6-active-with-spares-and-failed-disks%3B-need-help-4175530127/
I /think/ I need to force re-assembly here, but I'd like some review
from the experts before proceeding.
Thank you in advance for your time,
~Matt/Fermulator
^ permalink raw reply
* Re: md raid performance with 3-18-rc3
From: Manish Awasthi @ 2015-01-07 10:52 UTC (permalink / raw)
To: NeilBrown; +Cc: linux-raid
In-Reply-To: <54ABAFC5.6060402@caviumnetworks.com>
[-- Attachment #1: Type: text/plain, Size: 4060 bytes --]
Here are the results with 3.12 kernel and except the better cpu
utilization for writes, rest of the numbers are lower than those of
3.6.11. Please let me know if there is any particular area that I can
check. Since 3.12 introduced multithread raid for the first time, does
it make sense to go further down the kernel revision to rule in/out the
multithread support to be the cause of the degradation in throughput?
Manish
On 01/06/2015 03:19 PM, Manish Awasthi wrote:
> Hi Neil,
>
> Any findings on some of the logs I shared earlier?
>
> Thanks in advance for reply. I'm having trouble booting 3.12 kernel,
> should probably sort it out soon and come back with results.
>
> Manish
>
> On 12/10/2014 01:29 PM, Manish Awasthi wrote:
>> Here is the perf report for the tests run on 3.6-11 and 3.18.
>> Compating both the results, it just appears that raid in older
>> version is busier than it is with the latest version. I will also
>> monitor the system activity via `perf top` now. Also, I should be
>> back with results on 3.12 by the weekend
>>
>> Manish
>>
>> On 12/09/2014 01:56 PM, Manish Awasthi wrote:
>>> this time with attachment:
>>>
>>> manish
>>> On 12/09/2014 01:54 PM, Manish Awasthi wrote:
>>>> resending:
>>>>
>>>> dirty_ratio same for both the kernels.
>>>>>
>>>>> vm.dirty_background_bytes = 0
>>>>> vm.dirty_background_ratio = 10
>>>>> vm.dirty_bytes = 0
>>>>> vm.dirty_expire_centisecs = 3000
>>>>> vm.dirty_ratio = 20
>>>>> vm.dirty_writeback_centisecs = 500
>>>>>
>>>>>
>>>>> I re-ran the tests with the same set of kernel without enabling
>>>>> multithread support on 3.18 and measured a few things with perf.
>>>>>
>>>>> perf-stat-<kernel>.txt: test ran for some time and measured
>>>>> various parameters.
>>>>>
>>>>> Meanwhile I'm also running complete test under perf record. I'll
>>>>> share the results soon.
>>>>>
>>>>> Manish
>>>>>
>>>>> On 12/03/2014 11:51 AM, NeilBrown wrote:
>>>>>> On Wed, 26 Nov 2014 13:41:39 +0530 Manish Awasthi
>>>>>> <manish.awasthi@caviumnetworks.com> wrote:
>>>>>>
>>>>>>> Whatever data I have on comparison is attached, I have
>>>>>>> consolidated this
>>>>>>> from log files to excel. See if this helps.
>>>>>> raid_3_18_performance.xls shows read throughput to be
>>>>>> consistently 20% down
>>>>>> on 3.18 compared to 3.6.11.
>>>>>>
>>>>>> Writes are a few percent better for 4G/8G files, 20% better for
>>>>>> 16G/32G files.
>>>>>> unchanged above that.
>>>>>> Given that you have 8G of RAM, that seems like it could be some
>>>>>> change in
>>>>>> caching behaviour, and not necessarily a change in RAID behaviour.
>>>>>>
>>>>>> The CPU utilization roughly follows the throughput: 40% higher
>>>>>> when write
>>>>>> throughput is 20% better.
>>>>>> Could you check if the value of /proc/sys/vm/dirty_ratio is the
>>>>>> same for both
>>>>>> tests. That number has changed occasionally and could affect
>>>>>> these tests.
>>>>>>
>>>>>>
>>>>>> The second file, 3SSDs-perf-2-Cores-3.18-rc1 has the "change"
>>>>>> numbers
>>>>>> negative where I expected positive.. i.e. negative mean an increase.
>>>>>>
>>>>>> Writes consistently have higher CPU utilisation.
>>>>>> Reads consistently have much lower CPU utilization.
>>>>>>
>>>>>> I don't know what that means ... it might not mean anything.
>>>>>>
>>>>>> Could you please run the tests between the two kernels *with*
>>>>>> RAID. i.e.
>>>>>> directly on an SSD. That will give us a baseline for what
>>>>>> changes are caused
>>>>>> by other parts of the kernel (filesystem, block layer, MM, etc).
>>>>>> Then we can
>>>>>> see how much change RAID5 is contributing.
>>>>>>
>>>>>> The third file, 3SSDs-perf-4Core.xls seems to show significantly
>>>>>> reduced
>>>>>> throughput across the board.
>>>>>> CPU utilization is less (better) for writes, but worse for
>>>>>> reads. That is
>>>>>> the reverse of what the second file shows.
>>>>>>
>>>>>> I might try running some tests across a set of kernel versions
>>>>>> and see what I
>>>>>> can come up with.
>>>>>>
>>>>>> NeilBrown
>>>>>
>>>>
>>>
>>
>
[-- Attachment #2: 3.6.11-vs-3.12.xls --]
[-- Type: application/vnd.ms-excel, Size: 29696 bytes --]
^ permalink raw reply
* Re: Reassembling RAID1 after good drive was offline [newbie]
From: Aryeh Leib Taurog @ 2015-01-07 8:30 UTC (permalink / raw)
To: linux-raid
In-Reply-To: <20150106075400.4a208293@notabene.brown>
On Tue, Jan 06, 2015 at 07:54:00AM +1300, NeilBrown wrote:
> The fault is detected by the drive, possibly using a CRC, or by the
> controller (hmm.. the drive isn't responding, must be faulty!) and this fault
> is communicated to md. md then manages the fault by accesses the other
> device.
I imagine that RAID does introduce a risk here. If the drive is fine
and the other hardware isn't, one really could end up with disparate
data. I have had situations (perhaps related to the flaky cable I
recently discovered) where I wrote "A" to the drive and then read back
"B." On a single drive system, it's possible to confirm a write by
reading it, but with RAID that is not the case. So RAID increases my
safety against drive failure at the expense of increased reliance on
the other hardware. (Sorry if this is all obvious, I'm new to RAID
and trying to get this clear for myself.)
My current procedure is to make a backup with dar, test the archive,
and then generate par2 recovery files. The par2 files give me some
protection against data corruption, but that only helps if I can rely
on the initial test. So I guess my options are to use more reliable
hardware and/or to backup first to a single drive, on which I test and
generate recovery files, and then copy to RAID device.
> *No* RAID level has error detection ability - *all* RAID levels (except zero)
> have error correction - providing something else detects the error.
> Parity vs mirroring makes no difference here.
>
> And to answer the original question: just let it resync.
Thank you all for your answers.
I still didn't get clear confirmation about what resync does, though.
I understand that md doesn't have any way of knowing *which* drive is
the "correct" drive, but it *has* decided somehow from which to
assemble the array and which to ignore. I am assuming that the
procedure is to add it back with (the man page implies that -a would
have the same effect in this case)
mdadm /dev/md/backup --re-add /dev/sdc2
and that md will 'resync' i.e. make a byte-for-byte copy of the first
device back onto the second device. I thought this was obvious, but
several objections were raised so I'm not entirely sure any more. Is
it any more or less than that?
I'm also not sure, given the objections about md not having psychic
powers, how exactly md did decide which device to include and which to
ignore, and I'm puzzled by this message:
mdadm: ignoring /dev/sdc2 as it reports /dev/sdd2 as failed
The logic seems reversed to me. Is this just the artifact of a
possibly buggy version of md, or am I missing something here also?
> Had you started that when you asked the question it would be done by
> now :-)
Of course, but the goal was to learn more about what I'm doing, not to
save time. I usually favor a deeper understanding over expedience.
> To avoid similar problems in future:
> - use a newer mdadm (sorry, but there are bugs sometimes)
> - add an internal write-intent bitmap. That makes the resync much faster
> when needed
> - Possibly as '--no-degraded' when assembling arrays.
Thanks. This is all quite helpful.
^ permalink raw reply
* Please let me know if you need to print color box, display box and labels
From: Jinghao Printing - CHINA @ 2015-01-07 3:03 UTC (permalink / raw)
Hi, this is David Wu from Shanghai, China.
Please let me know if you need color box, display box, corrugated box,
label, hang tag etc.
I will send you the website.
Best regards,
David Wu
^ permalink raw reply
* [PATCH] [md] raid5: check faulty flag for array status during recovery.
From: Eric Mei @ 2015-01-06 22:24 UTC (permalink / raw)
To: neilb; +Cc: linux-raid, eric.mei
[-- Attachment #1: Type: text/plain, Size: 1447 bytes --]
Hi Neil,
In a MDRAID derived work we found and fixed a data corruption bug. We think this also affect vanilla MDRAID, but we didn’t directly prove that by constructing a test to show the corruption. Following is the theoretical analysis, please kindly review and see if I missed something.
To rebuild a stripe, MD checks whether array will be optimal after rebuild complete, if that’s true, we’ll mark the WIB bit to be cleared, the purpose is to enable “incremental rebuild”. The code section is like this:
/* Need to check if array will still be degraded after recovery/resync
* We don't need to check the 'failed' flag as when that gets set,
* recovery aborts.
*/
for (i = 0; i < conf->raid_disks; i++)
if (conf->disks[i].rdev == NULL)
still_degraded = 1;
The problem is that only checking rdev == NULL might not be enough. Suppose both 2 drives D0 and D1 failed and marked as Faulty; We immediately removed D0 from array, but because some lingering IO on D1, it remains in array with Faulty flags on. A new drive pulled in, rebuild against D0 starts. Now because no rdev is NULL, MD thinks array will be optimal. If some writes happened before rebuild reaches the region, their dirty bits in WIB will be cleared. When later add D1 back into array, we’ll skip rebuilding those stripes, thus data corruption.
The attached patch (against 3.18.0-rc6) is supposed to fix this issue.
Thanks
Eric
[-- Attachment #2: 0001-md-raid5-check-faulty-flag-for-array-status-during-r.patch --]
[-- Type: application/octet-stream, Size: 1550 bytes --]
From 6e569429d9deac31df3c7023a59c93ac862b10e5 Mon Sep 17 00:00:00 2001
From: Eric Mei <eric.mei@seagate.com>
Date: Tue, 6 Jan 2015 09:35:02 -0800
Subject: [PATCH] [md] raid5: check faulty flag for array status during recovery.
When we have more than 1 drive failure, it's possible we start
rebuild one drive while leaving another faulty drive in array.
To determine whether array will be optimal after building, current
code only check whether a drive is missing, which could potentially
lead to data corruption. This patch is to add checking Faulty flag.
---
drivers/md/raid5.c | 7 ++++---
1 files changed, 4 insertions(+), 3 deletions(-)
diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c
index 9c66e59..a01a25e 100644
--- a/drivers/md/raid5.c
+++ b/drivers/md/raid5.c
@@ -5054,11 +5054,12 @@ static inline sector_t sync_request(struct mddev *mddev, sector_t sector_nr, int
schedule_timeout_uninterruptible(1);
}
/* Need to check if array will still be degraded after recovery/resync
- * We don't need to check the 'failed' flag as when that gets set,
- * recovery aborts.
+ * Note in case of > 1 drive failures it's possible we're rebuilding
+ * one drive while leaving another faulty drive in array.
*/
for (i = 0; i < conf->raid_disks; i++)
- if (conf->disks[i].rdev == NULL)
+ if (conf->disks[i].rdev == NULL ||
+ test_bit(Faulty, &conf->disks[i].rdev->flags))
still_degraded = 1;
bitmap_start_sync(mddev->bitmap, sector_nr, &sync_blocks, still_degraded);
--
1.7.1
^ permalink raw reply related
* mdadm RAID6 "active" with spares and failed disks; need help
From: Matt Callaghan @ 2015-01-06 14:16 UTC (permalink / raw)
To: linux-raid
I think I'm in a really bad state. Could an expert w/ mdadm please help?
I have a RAID6 mdadm device, and it got really messed up with spares:
{{{
md2000 : active raid6 sdm1[8](S) sdo1[3] sdi1[4] sdn1[2] sdk1[0](F)
sdl1[6] sdp1[7]
11721080448 blocks super 1.1 level 6, 64k chunk, algorithm 2
[8/5] [__UU_UUU]
}}}
And is now really broken (inactive)
{{{
md2000 : inactive sdn1[2](S) sdm1[8](S) sdl1[6](S) sdp1[7](S) sdi1[4](S)
sdo1[3](S) sdk1[0](S)
13674593976 blocks super 1.1
}}}
I have a forum post going w/ full details
http://www.linuxquestions.org/questions/linux-server-73/mdadm-raid6-active-with-spares-and-failed-disks%3B-need-help-4175530127/
I /think/ I need to force re-assembly here, but I'd like some review
from the experts before proceeding.
Thank you in advance for your time,
~Matt/Fermulator
^ permalink raw reply
* [PATCH] dm bufio: fix time comparison
From: Asaf Vertz @ 2015-01-06 13:44 UTC (permalink / raw)
To: agk; +Cc: snitzer, dm-devel, neilb, linux-raid, linux-kernel
To be future-proof and for better readability the time comparison
modified to use time_after_eq() instead of plain, error-prone math.
Signed-off-by: Asaf Vertz <asaf.vertz@tandemg.com>
---
drivers/md/dm-bufio.c | 3 ++-
1 files changed, 2 insertions(+), 1 deletions(-)
diff --git a/drivers/md/dm-bufio.c b/drivers/md/dm-bufio.c
index c33b497..86dbbc7 100644
--- a/drivers/md/dm-bufio.c
+++ b/drivers/md/dm-bufio.c
@@ -11,6 +11,7 @@
#include <linux/device-mapper.h>
#include <linux/dm-io.h>
#include <linux/slab.h>
+#include <linux/jiffies.h>
#include <linux/vmalloc.h>
#include <linux/shrinker.h>
#include <linux/module.h>
@@ -1739,7 +1740,7 @@ static unsigned get_max_age_hz(void)
static bool older_than(struct dm_buffer *b, unsigned long age_hz)
{
- return (jiffies - b->last_accessed) >= age_hz;
+ return time_after_eq(jiffies, b->last_accessed + age_hz);
}
static void __evict_old_buffers(struct dm_bufio_client *c, unsigned long age_hz)
--
1.7.0.4
^ permalink raw reply related
* Re: Split RAID: Proposal for archival RAID using incremental batch checksum
From: Anshuman Aggarwal @ 2015-01-06 11:40 UTC (permalink / raw)
To: NeilBrown; +Cc: Mdadm
In-Reply-To: <CAK-d5daVxQN3JifvcA9+PL04p1L3O0KXQkgBwyhs1O6tHnHeMg@mail.gmail.com>
On 17 December 2014 at 12:10, Anshuman Aggarwal
<anshuman.aggarwal@gmail.com> wrote:
> On 17 December 2014 at 03:19, NeilBrown <neilb@suse.de> wrote:
>> On Tue, 16 Dec 2014 21:55:15 +0530 Anshuman Aggarwal
>> <anshuman.aggarwal@gmail.com> wrote:
>>
>>> On 2 December 2014 at 17:26, Anshuman Aggarwal
>>> <anshuman.aggarwal@gmail.com> wrote:
>>> > It works! (Atleast on a sample 5 MB device with 5 x 1MB partitions :-)
>>> > will find more space on my drives and do a larger test but don't see
>>> > why it shouldn't work)
>>> > Here are the following caveats (and questions):
>>> > - Neil, like you pointed out, the power of 2 chunk size will probably
>>> > need a code change (in the kernel or only in the userspace tool?)
>>
>> In the kernel too.
>
> Is this something that you would consider implementing soon? Is there
> a performance/other impact to any other consideration to remove this
> limitation.. could you elaborate on the reason why it was there in the
> first place?
>
> If this is a case of patches are welcome, please guide on where to
> start looking/working even if its just
>
>>
>>> > - Any performance or other reasons why a terabyte size chunk may
>>> > not be feasible?
>>
>> Not that I can think of.
>>
>>> > - Implications of safe_mode_delay
>>> > - Would the metadata be updated on the block device be written to
>>> > and the parity device as well?
>>
>> Probably. Hard to give a specific answer to vague question.
>
> I should clarify.
>
> For example in a 5 device RAID4, lets say block is being written to
> device 1 and parity is on device 5 and devices 2,3,4 are sleeping
> (spun down). If we set safe_mode_delay to 0 and md decides to update
> the parity without involving the blocks on the other 3 devices and
> just updates the parity by doing a read, compute, write to device 5
> will the metadata be updated on both device 1 and 5 even though
> safe_mode_delay is 0?
>
>>
>>> > - If the drive fails which is the same as the drive being written
>>> > to, would that lack of metadata updates to the other devices affect
>>> > reconstruction?
>>
>> Again, to give a precise answer, a detailed question is needed. Obviously
>> any change would have to made in such a way to ensure that things which
>> needed to work, did work.
>
> Continuing from the previous example, lets say device 1 fails after a
> write which only updated metadata on 1 and 5 while 2,3,4 were
> sleeping. In that case to access the data from 1, md will use 2,3,4,5
> but will it then update the metadata from 5 onto 2,3,4? I hope I am
> making this clear.
>
>>
>>
>>> > - Adding new devices (is it possible to move the parity to the disk
>>> > being added? How does device addition work for RAID4 ...is it added as
>>> > a zero-ed out device with parity disk remaining the same)
>>
>> RAID5 or RAID6 with ALGORITHM_PARITY_0 puts the parity on the early devices.
>> Currently if you add a device to such an array ...... I'm not sure what it
>> will do. It should be possible to make it just write zeros out.
>>
>
> Once again, is this something that can make its way to your roadmap?
> If so, great.. otherwise could you steer me towards where in the md
> kernel and mdadm source I should be looking to make these changes.
> Thanks again.
>
>>
>> NeilBrown
>>
>>
>>> >
>>> >
>>>
>>> Neil, sorry to try to bump this thread. Could you please look over the
>>> questions and address the points on the remaining items that can make
>>> it a working solution? Thanks
>>
Hi Neil,
Could you please find a minute to give your input to the above? Your
guidance will go a long way towards making this a reality and it may
be useful to the community at large with the new Seagate 8TB archival
drives which seem to be more geared towards occasional use but would
still benefit from a RAID like redundancy.
Many thanks,
Anshuman
^ permalink raw reply
* ignoring %s as it reports %s as failed
From: Valentijn @ 2015-01-06 11:32 UTC (permalink / raw)
To: linux-raid
Hello list,
I'm trying to resurrect an array from a Synology disk station. As
follows: raid5 array with 4 disks, one disk utterly broken
(click-clack-click), one disk thrown out of the array.
Please note, that I used dmsetup to create an overlay disk, so the
"/dev/mapper/diskXp5" actually points to a /dev/sdN5 which is the
original array; I'm trying to not harm the original data. Here is the
result of mdadm v3.2.5 running assemble+force"
~# mdadm --verbose --assemble --force /dev/md99 /dev/mapper/disk1p5
/dev/mapper/disk2p5 /dev/mapper/disk3p5
mdadm: looking for devices for /dev/md99
mdadm: /dev/mapper/disk1p5 is identified as a member of /dev/md99, slot
32768.
mdadm: /dev/mapper/disk2p5 is identified as a member of /dev/md99, slot 1.
mdadm: /dev/mapper/disk3p5 is identified as a member of /dev/md99, slot 2.
mdadm: ignoring /dev/mapper/disk2p5 as it reports /dev/mapper/disk1p5 as
failed
mdadm: ignoring /dev/mapper/disk3p5 as it reports /dev/mapper/disk1p5 as
failed
mdadm: No suitable drives found for /dev/md99
OK. http://permalink.gmane.org/gmane.linux.raid/45064 says I should use
mdadm 3.3+, and luckily I had v3.3.2 - 21st August lying around, so
there we go:
~# ./mdadm --verbose --assemble --force /dev/md99 /dev/mapper/disk1p5
/dev/mapper/disk2p5 /dev/mapper/disk3p5
mdadm: looking for devices for /dev/md99
mdadm: /dev/mapper/disk1p5 is identified as a member of /dev/md99, slot
32768.
mdadm: /dev/mapper/disk2p5 is identified as a member of /dev/md99, slot 1.
mdadm: /dev/mapper/disk3p5 is identified as a member of /dev/md99, slot 2.
mdadm: no uptodate device for slot 0 of /dev/md99
mdadm: added /dev/mapper/disk3p5 to /dev/md99 as 2
mdadm: no uptodate device for slot 6 of /dev/md99
mdadm: added /dev/mapper/disk2p5 to /dev/md99 as 1
mdadm: /dev/md99 assembled from 2 drives - not enough to start the array.
Hmm, that doesn't help us either. (Slight difference between the two
versions, 3.2.5 deploys a broken array named md99 - which you can stop
afterwards - and 3.3.2 seems to auto-remove the array after trying to
start it - not sure if I describe the inner workings correctly)
Disk1 reports:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 8ad57063:cf592975:9fabcc2b:13922e19
Name : 2
Creation Time : Sun May 13 08:23:21 2012
Raid Level : raid5
Raid Devices : 4
Avail Dev Size : 1944057456 (927.00 GiB 995.36 GB)
Array Size : 2916086016 (2781.00 GiB 2986.07 GB)
Used Dev Size : 1944057344 (927.00 GiB 995.36 GB)
Data Offset : 272 sectors
Super Offset : 8 sectors
State : clean
Device UUID : 4ae571dc:3c458e25:d219b536:c8b9a50e
Update Time : Sun Dec 28 10:41:55 2014
Checksum : 9a35a9ae - correct
Events : 744446
Layout : left-symmetric
Chunk Size : 64K
Device Role : Active device 32768
Array State : .AA. ('A' == active, '.' == missing)
Disk2:
[...]
State : clean
Device UUID : b381cfd4:d2474cc4:e4c0ad21:8565b2a5
Update Time : Sun Dec 28 10:41:55 2014
Checksum : b3569b7e - correct
Events : 744446
Layout : left-symmetric
Chunk Size : 64K
Device Role : Active device 1
Array State : .AA. ('A' == active, '.' == missing)
Disk3:
[...]
State : clean
Device UUID : c00d4418:c8313607:fb3a2018:d267a693
Update Time : Sun Dec 28 10:41:55 2014
Checksum : 1e1b8de5 - correct
Events : 744446
Layout : left-symmetric
Chunk Size : 64K
Device Role : Active device 2
Array State : .AA. ('A' == active, '.' == missing)
So the "Active device 32768" seems to be the culprit here, is that correct?
What would be the correct way to reassemble this array?
I managed to get it *working* again by recreating it with:
./mdadm --create --assume-clean --data-offset=136 -e 1.2 --level=5
--bitmap=none --raid-devices=4 --chunk=64 /dev/md99 /dev/mapper/disk1p5
/dev/mapper/disk2p5 /dev/mapper/disk3p5 missing
... but that feels a bit rude.
So, is there a "proper" way to reassemble this array?
For the record: the original array was most likely created on the
Synology disk station itself, which currently contains mdadm - v2.6.7 (I
don't know if that is the version that created the array). Also, I'm not
sure about the kernel version that's running there, but /lib/modules
contains modules that have a string "vermagic=2.6.15 ARMv5 gcc-3.4"
inside, so that might be the kernel version that was running when things
went wrong.
Best regards,
Valentijn
^ permalink raw reply
* Re: md raid performance with 3-18-rc3
From: Manish Awasthi @ 2015-01-06 9:49 UTC (permalink / raw)
To: NeilBrown; +Cc: linux-raid
In-Reply-To: <5487FD79.7000002@caviumnetworks.com>
Hi Neil,
Any findings on some of the logs I shared earlier?
Thanks in advance for reply. I'm having trouble booting 3.12 kernel,
should probably sort it out soon and come back with results.
Manish
On 12/10/2014 01:29 PM, Manish Awasthi wrote:
> Here is the perf report for the tests run on 3.6-11 and 3.18.
> Compating both the results, it just appears that raid in older version
> is busier than it is with the latest version. I will also monitor the
> system activity via `perf top` now. Also, I should be back with
> results on 3.12 by the weekend
>
> Manish
>
> On 12/09/2014 01:56 PM, Manish Awasthi wrote:
>> this time with attachment:
>>
>> manish
>> On 12/09/2014 01:54 PM, Manish Awasthi wrote:
>>> resending:
>>>
>>> dirty_ratio same for both the kernels.
>>>>
>>>> vm.dirty_background_bytes = 0
>>>> vm.dirty_background_ratio = 10
>>>> vm.dirty_bytes = 0
>>>> vm.dirty_expire_centisecs = 3000
>>>> vm.dirty_ratio = 20
>>>> vm.dirty_writeback_centisecs = 500
>>>>
>>>>
>>>> I re-ran the tests with the same set of kernel without enabling
>>>> multithread support on 3.18 and measured a few things with perf.
>>>>
>>>> perf-stat-<kernel>.txt: test ran for some time and measured various
>>>> parameters.
>>>>
>>>> Meanwhile I'm also running complete test under perf record. I'll
>>>> share the results soon.
>>>>
>>>> Manish
>>>>
>>>> On 12/03/2014 11:51 AM, NeilBrown wrote:
>>>>> On Wed, 26 Nov 2014 13:41:39 +0530 Manish Awasthi
>>>>> <manish.awasthi@caviumnetworks.com> wrote:
>>>>>
>>>>>> Whatever data I have on comparison is attached, I have
>>>>>> consolidated this
>>>>>> from log files to excel. See if this helps.
>>>>> raid_3_18_performance.xls shows read throughput to be consistently
>>>>> 20% down
>>>>> on 3.18 compared to 3.6.11.
>>>>>
>>>>> Writes are a few percent better for 4G/8G files, 20% better for
>>>>> 16G/32G files.
>>>>> unchanged above that.
>>>>> Given that you have 8G of RAM, that seems like it could be some
>>>>> change in
>>>>> caching behaviour, and not necessarily a change in RAID behaviour.
>>>>>
>>>>> The CPU utilization roughly follows the throughput: 40% higher
>>>>> when write
>>>>> throughput is 20% better.
>>>>> Could you check if the value of /proc/sys/vm/dirty_ratio is the
>>>>> same for both
>>>>> tests. That number has changed occasionally and could affect
>>>>> these tests.
>>>>>
>>>>>
>>>>> The second file, 3SSDs-perf-2-Cores-3.18-rc1 has the "change" numbers
>>>>> negative where I expected positive.. i.e. negative mean an increase.
>>>>>
>>>>> Writes consistently have higher CPU utilisation.
>>>>> Reads consistently have much lower CPU utilization.
>>>>>
>>>>> I don't know what that means ... it might not mean anything.
>>>>>
>>>>> Could you please run the tests between the two kernels *with*
>>>>> RAID. i.e.
>>>>> directly on an SSD. That will give us a baseline for what changes
>>>>> are caused
>>>>> by other parts of the kernel (filesystem, block layer, MM, etc).
>>>>> Then we can
>>>>> see how much change RAID5 is contributing.
>>>>>
>>>>> The third file, 3SSDs-perf-4Core.xls seems to show significantly
>>>>> reduced
>>>>> throughput across the board.
>>>>> CPU utilization is less (better) for writes, but worse for reads.
>>>>> That is
>>>>> the reverse of what the second file shows.
>>>>>
>>>>> I might try running some tests across a set of kernel versions and
>>>>> see what I
>>>>> can come up with.
>>>>>
>>>>> NeilBrown
>>>>
>>>
>>
>
^ permalink raw reply
* Re: Reassembling RAID1 after good drive was offline [newbie]
From: NeilBrown @ 2015-01-05 18:54 UTC (permalink / raw)
To: Wols Lists; +Cc: Aryeh Leib Taurog, linux-raid
In-Reply-To: <54A9B472.9090905@youngman.org.uk>
[-- Attachment #1: Type: text/plain, Size: 3031 bytes --]
On Sun, 04 Jan 2015 21:45:22 +0000 Wols Lists <antlists@youngman.org.uk>
wrote:
> On 04/01/15 21:07, Aryeh Leib Taurog wrote:
> > On Sun, 4 Jan 2015 at 11:10 Peter Grandi wrote:
> >> Yet another of an endless (but not too frequent fortunately)
> >> stream of "wildly optimistic" messages to this mailing list...
> >
> > No intent to offend. I specifically put "newbie" in the subject.
> >
> >>> Would the resync just copy all the data from the "good" drive
> >>> back to the "failed" drive?
> >>
> >> This seems to me quite "imaginative" based on the dream that
> >> resync has psychic powers.
> >
> > I am not sure what you mean. Two drives in a RAID1 array. At one
> > point, one drive failed to come on line. Now mdadm refuses to include
> > that drive in the array. So there's the "good" drive, which appears
> > in the now degraded array, and the "failed" drive, which does not. I
> > have never done a resync, and I haven't seen a detailed description of
> > what it does, but given that mdadm seems to have decided which drive
> > is good and which not, and assuming mdadm doesn't know anything about
> > the contents of the data, what is so "imaginative" about the notion
> > that if I add the "failed" drive to the array, it would simply copy
> > all the data on the "good" drive byte-by-byte onto the "failed" drive,
> > overwriting whatever is currently on the "failed" drive? I can't
> > imagine how else a resync would work. What am I missing?
>
> That mirroring isn't fault-tolerant-raid? I know it's been given a raid
> classification, but raids 1 and 0 really just give you a bigger faster
> disk. It's only the other raids that have any error correction ability,
> because they use parity etc to be able to tell which set of data is correct.
Bzzt. You lose :-)
Of course RAID1 is fault tolerant!! (I agree that RAID0 isn't).
It cannot tolerate every conceivable fault (e.g. asteroid impact), but it
tolerates most single hardware faults.
The fault is detected by the drive, possibly using a CRC, or by the
controller (hmm.. the drive isn't responding, must be faulty!) and this fault
is communicated to md. md then manages the fault by accesses the other
device.
*No* RAID level has error detection ability - *all* RAID levels (except zero)
have error correction - providing something else detects the error.
Parity vs mirroring makes no difference here.
(All non-zero RAID levels could try to detect faults by reading all blocks
in a stripe and comparing, but there is no threat-model which makes this a
worthwhile practice)
And to answer the original question: just let it resync. Had you started
that when you asked the question it would be done by now :-).
To avoid similar problems in future:
- use a newer mdadm (sorry, but there are bugs sometimes)
- add an internal write-intent bitmap. That makes the resync much faster
when needed
- Possibly as '--no-degraded' when assembling arrays.
NeilBrown
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 811 bytes --]
^ permalink raw reply
* Re: Reassembling RAID1 after good drive was offline [newbie]
From: Robert L Mathews @ 2015-01-05 17:25 UTC (permalink / raw)
To: linux-raid
In-Reply-To: <54A9B472.9090905@youngman.org.uk>
On 1/4/15 1:45 PM, Wols Lists wrote:
> It's only the other raids that have any error correction ability,
> because they use parity etc to be able to tell which set of data is correct.
In theory, yes, but in practice, they don't -- or do they? I thought
that even if you use RAID 6, md doesn't check all members on each read
request, calculate parities, and mask silent platter-level corruption by
ignoring a member spewing bad bits. It just uses parities to fill in
missing data if a member fails.
Or does it actually do error detection, and I'm confused?
--
Robert L Mathews, Tiger Technologies, http://www.tigertech.net/
^ permalink raw reply
* Re: [dm-devel] Questions about the function, queue_empty in dm-cache-policy-mq.c
From: Joe Thornber @ 2015-01-05 9:17 UTC (permalink / raw)
To: device-mapper development; +Cc: agk, linux-raid, linux-kernel, snitzer
In-Reply-To: <54AA03E0.9020900@gmail.com>
On Sun, Jan 04, 2015 at 10:24:16PM -0500, nick wrote:
> Greetings All,
> I have a few questions about the function I stated in the message for this email.
> 1. What is the issues with the CPU usage for this function?
At the moment it can perform up to NR_QUEUE_LEVELS (16) list_empty()
calls. Which I consider too much work for a function that is called
frequently.
> 2. What hardware is this on ?
Irrelevant.
> 3. How scalable is this function in terms of NR_QUEUE_LEVELS and how many does it need to handle?
Linear. NR_QUEUE_LEVELS is a compile time constant currently set to
16, and unlikely to change.
- Joe
^ permalink raw reply
* Questions about the function, queue_empty in dm-cache-policy-mq.c
From: nick @ 2015-01-05 3:24 UTC (permalink / raw)
To: agk; +Cc: linux-raid, dm-devel, linux-kernel, snitzer
Greetings All,
I have a few questions about the function I stated in the message for this email.
1. What is the issues with the CPU usage for this function?
2. What hardware is this on ?
3. How scalable is this function in terms of NR_QUEUE_LEVELS and how many does it need to handle?
Nick
^ permalink raw reply
* Re: Reassembling RAID1 after good drive was offline [newbie]
From: Wols Lists @ 2015-01-04 21:45 UTC (permalink / raw)
To: Aryeh Leib Taurog, linux-raid
In-Reply-To: <20150104210701.GG4713@deb76.aryehleib.com>
On 04/01/15 21:07, Aryeh Leib Taurog wrote:
> On Sun, 4 Jan 2015 at 11:10 Peter Grandi wrote:
>> Yet another of an endless (but not too frequent fortunately)
>> stream of "wildly optimistic" messages to this mailing list...
>
> No intent to offend. I specifically put "newbie" in the subject.
>
>>> Would the resync just copy all the data from the "good" drive
>>> back to the "failed" drive?
>>
>> This seems to me quite "imaginative" based on the dream that
>> resync has psychic powers.
>
> I am not sure what you mean. Two drives in a RAID1 array. At one
> point, one drive failed to come on line. Now mdadm refuses to include
> that drive in the array. So there's the "good" drive, which appears
> in the now degraded array, and the "failed" drive, which does not. I
> have never done a resync, and I haven't seen a detailed description of
> what it does, but given that mdadm seems to have decided which drive
> is good and which not, and assuming mdadm doesn't know anything about
> the contents of the data, what is so "imaginative" about the notion
> that if I add the "failed" drive to the array, it would simply copy
> all the data on the "good" drive byte-by-byte onto the "failed" drive,
> overwriting whatever is currently on the "failed" drive? I can't
> imagine how else a resync would work. What am I missing?
That mirroring isn't fault-tolerant-raid? I know it's been given a raid
classification, but raids 1 and 0 really just give you a bigger faster
disk. It's only the other raids that have any error correction ability,
because they use parity etc to be able to tell which set of data is correct.
>
>>> For diagnostic purposes, it would actually be a lot more
>>> informative to compare the two drives and see if there really
>>> is data corruption on one of them or not.
>>
>> This seems to me "wildly optimistic" that when two blocks differ
>> it is possible in the general case to determine whether one (and
>> which one) or both are "corrupted".
>
> I was only referring to my specific case. One drive was found to have
> faulty hardware, one not. The one with the faulty hardware is
> suspect, the other not. If the two differ, then I, perhaps naively,
> would assume that the suspect drive experienced data corruption and
> the other not. To my mind, whether they differ or not could indicate
> something about the condition of the hardware and may have potentially
> useful implications regarding data on drives previously used (without
> MD) in the suspect hardware. If I'm wrong, I'd be thrilled to learn.
>
>>> Is there a way to do that?
>>
>> 'man cmp' may show a way to "compare the two drives".
>
> I am quite familiar with the unix toolset. But I don't know enough
> about mdadm. Where, for example, is the superblock, and where do my
> data begin? I gather the superblock is expected to differ, since each
> device has a UUID and distinct event count. I read the mdadm man
> page, but it doesn't seem to discuss implementation details such as
> this. So if there's no mdadm-specific approach, it would help to know
> where on the device I could find the data that I should expect to
> match, or alternatively, why I should not interest myself in such a
> comparison.
>
>>> If I were to demonstrate that the data are in sync, I would
>>> want to reassemble without resync.
>>
>> "Fantastic logic".
>
> Please help me understand. It's a RAID1 array. Doesn't that mean the
> devices are supposed to have identical copies of the data? And if I
> can demonstrate that they in fact are identical, why would a resync be
> necessary? I am sure I am missing something. Please clue me in.
They are supposed to be identical. But since the raid came up with one
drive missing, the other drive may have been modified, so they're not
identical any more. And the raid array has NO WAY of knowing which
version is "correct". Sorry :-( but that's mirrors for you ...
>
>>> Also, in my situation, since for now I'm just using a pair of
>>> external drives, I could easily imagine accidentally trying to
>>> assemble the array when one of the drives is powered down.
>>> Then this situation would arise again without faulty hardware.
>>
>> This may be based on the "amazing insight" that differences in
>> content and event counts are the same as data corruption.
>
> Again, I fail to comprehend. If anything, I assumed the opposite. It
> was my understanding that a difference in event counts could very
> easily arise without data corruption, as could differences in content.
> It also seems obvious that data corruption could occur in ways that
> would affect all drives in an array. But I don't understand how any
> of this is relevant.
If there's differences in event counts and content, then the drives are
no longer identical. Reassemble the array, without a resync, and things
could get complicated (and nasty) VERY quickly :-(
>
>>> Prudence notwithstanding, I do think there are valid cases for
>>> reassembling this array without resync.
>>
>> I believe that you think that but I also believe the manual when
>> it says "Use this only if you really know what you are doing";
>> because many MD users may not have the level of skills and
>> insight about RAID that you think you have, MD is designed to
>> protect users by default from their own "amazing optimism",
>> especially users that haven't read yet:
>>
>> https://raid.wiki.kernel.org/index.php/RAID_Recovery
>> https://raid.wiki.kernel.org/index.php/Recovering_a_failed_software_RAID
>
> I don't think I have any skill or insight about RAID. I would like to
> learn and understand. I have read the linked material, as well as the
> mdadm man page a few times, but it seems to me there are some basic
> concepts which aren't explained clearly. It does say that when in
> doubt one should contact "the friendly and helpful folk on the
> linux-raid mailing list."
>
>>> If there's a way to do that, I'd still like to know.
>>
>> 'man mdadm' may show a way to assemble "without resync".
>
> I didn't find it. As I said, it seems to me one needs much more
> experience and understanding of MD than I have to understand most of
> that document. There is an indication in the RAID_Recovery wiki page
> that I can reissue the original --create command with --assume-clean,
> but there's more discussion about why it might not work than about
> what it actually does. It also encourages contacting the mailing list
> first, which I have done, hoping to gain some insight.
The problem with "--create --assume-clean", is that if the array isn't
clean (and in this case it isn't - we don't know what is different
between the drives but *something* is), then you're asking for trouble.
>
> I'd like to think you didn't mean to be offensive, condescending, or
> hyperbolic, and I'm not trying to flame, but I'm feeling very
> defensive after reading your message, and none the wiser. That may
> have come out in my response, despite my best efforts. I read what I
> could find about MD before posting, but it seems to me there's some
> basic information which isn't documented in the places I've looked. I
> tried searching the mailing list, but finding a needle in a haystack
> is much harder when you don't know what a needle looks like.
>
I know it's hard if you don't know what a needle looks like, but in this
case I think the analogy is very apt - if you go looking you are going
to stab yourself with it.
I think you are going to have to bite the bullet and resync. Maybe keep
the existing drive as a backup, get a new drive, and resync to that.
Make that you backup regime - once a month or whatever swap one drive
over as a backup.
The alternative is get another drive and go raid-5 - but see all the
threads warning about using raid 5 with consumer drives - you need to
tweak your system settings otherwise transient faults can break your system.
And at the end of the day, a resync may take time, but it won't have
much impact on you using your system. I've got mirrored 3TB drives, and
the resync when I brought the array up (I moved a non-mirrored system on
to one drive "--create /dev/disk1 missing" then added the other) just
ticked over in the background.
Cheers,
Wol
^ permalink raw reply
* Re: Reassembling RAID1 after good drive was offline [newbie]
From: Aryeh Leib Taurog @ 2015-01-04 21:07 UTC (permalink / raw)
To: linux-raid
In-Reply-To: <21673.8100.414752.391418@tree.ty.sabi.co.uk>
On Sun, 4 Jan 2015 at 11:10 Peter Grandi wrote:
> Yet another of an endless (but not too frequent fortunately)
> stream of "wildly optimistic" messages to this mailing list...
No intent to offend. I specifically put "newbie" in the subject.
>> Would the resync just copy all the data from the "good" drive
>> back to the "failed" drive?
>
> This seems to me quite "imaginative" based on the dream that
> resync has psychic powers.
I am not sure what you mean. Two drives in a RAID1 array. At one
point, one drive failed to come on line. Now mdadm refuses to include
that drive in the array. So there's the "good" drive, which appears
in the now degraded array, and the "failed" drive, which does not. I
have never done a resync, and I haven't seen a detailed description of
what it does, but given that mdadm seems to have decided which drive
is good and which not, and assuming mdadm doesn't know anything about
the contents of the data, what is so "imaginative" about the notion
that if I add the "failed" drive to the array, it would simply copy
all the data on the "good" drive byte-by-byte onto the "failed" drive,
overwriting whatever is currently on the "failed" drive? I can't
imagine how else a resync would work. What am I missing?
>> For diagnostic purposes, it would actually be a lot more
>> informative to compare the two drives and see if there really
>> is data corruption on one of them or not.
>
> This seems to me "wildly optimistic" that when two blocks differ
> it is possible in the general case to determine whether one (and
> which one) or both are "corrupted".
I was only referring to my specific case. One drive was found to have
faulty hardware, one not. The one with the faulty hardware is
suspect, the other not. If the two differ, then I, perhaps naively,
would assume that the suspect drive experienced data corruption and
the other not. To my mind, whether they differ or not could indicate
something about the condition of the hardware and may have potentially
useful implications regarding data on drives previously used (without
MD) in the suspect hardware. If I'm wrong, I'd be thrilled to learn.
>> Is there a way to do that?
>
> 'man cmp' may show a way to "compare the two drives".
I am quite familiar with the unix toolset. But I don't know enough
about mdadm. Where, for example, is the superblock, and where do my
data begin? I gather the superblock is expected to differ, since each
device has a UUID and distinct event count. I read the mdadm man
page, but it doesn't seem to discuss implementation details such as
this. So if there's no mdadm-specific approach, it would help to know
where on the device I could find the data that I should expect to
match, or alternatively, why I should not interest myself in such a
comparison.
>> If I were to demonstrate that the data are in sync, I would
>> want to reassemble without resync.
>
> "Fantastic logic".
Please help me understand. It's a RAID1 array. Doesn't that mean the
devices are supposed to have identical copies of the data? And if I
can demonstrate that they in fact are identical, why would a resync be
necessary? I am sure I am missing something. Please clue me in.
>> Also, in my situation, since for now I'm just using a pair of
>> external drives, I could easily imagine accidentally trying to
>> assemble the array when one of the drives is powered down.
>> Then this situation would arise again without faulty hardware.
>
> This may be based on the "amazing insight" that differences in
> content and event counts are the same as data corruption.
Again, I fail to comprehend. If anything, I assumed the opposite. It
was my understanding that a difference in event counts could very
easily arise without data corruption, as could differences in content.
It also seems obvious that data corruption could occur in ways that
would affect all drives in an array. But I don't understand how any
of this is relevant.
>> Prudence notwithstanding, I do think there are valid cases for
>> reassembling this array without resync.
>
> I believe that you think that but I also believe the manual when
> it says "Use this only if you really know what you are doing";
> because many MD users may not have the level of skills and
> insight about RAID that you think you have, MD is designed to
> protect users by default from their own "amazing optimism",
> especially users that haven't read yet:
>
> https://raid.wiki.kernel.org/index.php/RAID_Recovery
> https://raid.wiki.kernel.org/index.php/Recovering_a_failed_software_RAID
I don't think I have any skill or insight about RAID. I would like to
learn and understand. I have read the linked material, as well as the
mdadm man page a few times, but it seems to me there are some basic
concepts which aren't explained clearly. It does say that when in
doubt one should contact "the friendly and helpful folk on the
linux-raid mailing list."
>> If there's a way to do that, I'd still like to know.
>
> 'man mdadm' may show a way to assemble "without resync".
I didn't find it. As I said, it seems to me one needs much more
experience and understanding of MD than I have to understand most of
that document. There is an indication in the RAID_Recovery wiki page
that I can reissue the original --create command with --assume-clean,
but there's more discussion about why it might not work than about
what it actually does. It also encourages contacting the mailing list
first, which I have done, hoping to gain some insight.
I'd like to think you didn't mean to be offensive, condescending, or
hyperbolic, and I'm not trying to flame, but I'm feeling very
defensive after reading your message, and none the wiser. That may
have come out in my response, despite my best efforts. I read what I
could find about MD before posting, but it seems to me there's some
basic information which isn't documented in the places I've looked. I
tried searching the mailing list, but finding a needle in a haystack
is much harder when you don't know what a needle looks like.
With appreciation,
Aryeh Leib Taurog
Please cc me on your responses, I'm not subscribed to the list and
this thread keeps disappearing from the archives at marc.info
^ permalink raw reply
* Re: Re : Re: Big trouble during reassemble a Raid5
From: sylvain.depuille @ 2015-01-04 18:19 UTC (permalink / raw)
To: Can Jeuleers; +Cc: John Stoffel, linux-raid
In-Reply-To: <54A64FEB.8090109@gmail.com>
Hello,
thank's for the tricks.
I have modify the #! as #!/usr/bin/python2 because i have python 2.7 and 3.3 as default on my PCs.
I keep this script on my PCs, and the output ;-)
Thank's a lot.
Best Regards
----- Mail original -----
De: "Can Jeuleers" <can.jeuleers@gmail.com>
À: "John Stoffel" <john@stoffel.org>, "sylvain depuille" <sylvain.depuille@laposte.net>
Cc: linux-raid@vger.kernel.org
Envoyé: Vendredi 2 Janvier 2015 08:59:39
Objet: Re: Re : Re: Big trouble during reassemble a Raid5
On 02/01/15 05:07, John Stoffel wrote:
> My real suggestion to you is to now collect all the information you
> can on the system, including drive serial numbers, and how they map to
> each device and member of the RAID array. Sometimes when a system
> reboots, or you add new disks, things get re-arranged and you don't
> notice at first. And that leads you down the wrong path quite
> quickly.
Sylvain, a nice script for documenting device serial numbers,
partitions, RAID memberships etc can be found here:
https://github.com/pturmel/lsdrv/raw/master/lsdrv
Written by Phil Turmel; a frequent contributor to this list.
Jan
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: Re : Re: Big trouble during reassemble a Raid5
From: sylvain.depuille @ 2015-01-04 18:16 UTC (permalink / raw)
To: John Stoffel; +Cc: linux-raid
In-Reply-To: <21670.6532.641717.517799@quad.stoffel.home>
Hello,
the raid is completly operationnal.
I take linux up to date with emerge Gentoo stable each day or each week.
I know the attribute pv_timeout on SAN with HP-UX, but not with Linux.
I haven't find the discussion over this point with Google.
Could you help me to find the thread on that?
PS: could i write what's your explain me on my wiki http://wiki.depuille.net/index.php/Accueil (it's a French Unix tricks/memorendum Wiki)?
Thank's in advance.
Best Regards
Sylvain Depuille
----- Mail original -----
De: "John Stoffel" <john@stoffel.org>
À: "sylvain depuille" <sylvain.depuille@laposte.net>
Cc: "John Stoffel" <john@stoffel.org>, linux-raid@vger.kernel.org
Envoyé: Vendredi 2 Janvier 2015 05:07:32
Objet: Re: Re: Re : Re: Big trouble during reassemble a Raid5
sylvain> Happy New Year and received all my best wishes for 2015. i'm
sylvain> back to home.
Me too, but heading to bed soon, been a busy day here.
sylvain> Now, the failed raid is at 700km (~400 miles) :-( I should
sylvain> work only by phone and ssh to repair the Raid.
It sucks having to work at a distance like this.
sylvain> We have changed the right disk and the raid is rebuilding. I
sylvain> wille write you when it's finished. Many many thank's for
sylvain> your help and your patience.
My real suggestion to you is to now collect all the information you
can on the system, including drive serial numbers, and how they map to
each device and member of the RAID array. Sometimes when a system
reboots, or you add new disks, things get re-arranged and you don't
notice at first. And that leads you down the wrong path quite
quickly.
sylvain> I have a question to migrate Raid 5 to Raid 6 :
sylvain> This site
sylvain> http://www.ewams.net/?date=2013/05/02&view=Converting_RAID5_to_RAID6_in_mdadm
sylvain> describe the right cookbook? I don't want re-do this bad
sylvain> Story.
It looks right to me. But I would also suggest that you look through
the archives of this list, and make sure you have a newer version of
the mdadm tool before you try to do this.
I would also suggest that you add two new disks, one to be used as a
hot spare, and the other used to grow the array into a RAID6.
Then go through and make sure that all your drive timeouts are set
properly on system boot, so that if a disk hangs when trying to
re-read a bad block, it won't cause it to be kicked out of the array
too quickly. Again, check the recent archives (last three months of
2014) of the list for examples and discussions of this issue.
Good luck!
John
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: Reassembling RAID1 after good drive was offline [newbie]
From: Peter Grandi @ 2015-01-04 11:10 UTC (permalink / raw)
To: Linux RAID
In-Reply-To: <20150104102009.GA4511@deb76.aryehleib.com>
Yet another of an endless (but not too frequent fortunately)
stream of "wildly optimistic" messages to this mailing list...
> Would the resync just copy all the data from the "good" drive
> back to the "failed" drive?
This seems to me quite "imaginative" based on the dream that
resync has psychic powers.
> For diagnostic purposes, it would actually be a lot more
> informative to compare the two drives and see if there really
> is data corruption on one of them or not.
This seems to me "wildly optimistic" that when two blocks differ
it is possible in the general case to determine whether one (and
which one) or both are "corrupted".
> Is there a way to do that?
'man cmp' may show a way to "compare the two drives".
> If I were to demonstrate that the data are in sync, I would
> want to reassemble without resync.
"Fantastic logic".
> Also, in my situation, since for now I'm just using a pair of
> external drives, I could easily imagine accidentally trying to
> assemble the array when one of the drives is powered down.
> Then this situation would arise again without faulty hardware.
This may be based on the "amazing insight" that differences in
content and event counts are the same as data corruption.
> Prudence notwithstanding, I do think there are valid cases for
> reassembling this array without resync.
I believe that you think that but I also believe the manual when
it says "Use this only if you really know what you are doing";
because many MD users may not have the level of skills and
insight about RAID that you think you have, MD is designed to
protect users by default from their own "amazing optimism",
especially users that haven't read yet:
https://raid.wiki.kernel.org/index.php/RAID_Recovery
https://raid.wiki.kernel.org/index.php/Recovering_a_failed_software_RAID
> If there's a way to do that, I'd still like to know.
'man mdadm' may show a way to assemble "without resync".
^ permalink raw reply
* Re: Reassembling RAID1 after good drive was offline [newbie]
From: Aryeh Leib Taurog @ 2015-01-04 10:20 UTC (permalink / raw)
To: linux-raid
In-Reply-To: <54A6E5B1.2090602@tigertech.com>
On Fri, Jan 02, 2015 at 6:38 PM, Robert L Mathews wrote:
> Your question is about how to reassemble them without a resync,
> which is understandable, but the reason md isn't doing that
> automatically is that it thinks they might be different.
> I personally would always do a resync in such a case. A flaky cable
> suggests that data on it is suspect, and this is one of the things
> RAID 1 is for: it allows you to copy clean data to suspect
> partitions.
> But I'm known to be data-paranoid. Other people may have different
> opinions/suggestions.
Thanks. I tend to agree. Would the resync just copy all the data
from the "good" drive back to the "failed" drive? For diagnostic
purposes, it would actually be a lot more informative to compare the
two drives and see if there really is data corruption on one of them
or not. Is there a way to do that? If I were to demonstrate that the
data are in sync, I would want to reassemble without resync.
Also, in my situation, since for now I'm just using a pair of external
drives, I could easily imagine accidentally trying to assemble the
array when one of the drives is powered down. Then this situation
would arise again without faulty hardware. I suppose an 'assemble'
script could help protect against the latter case by checking first
that both devices are available.
Prudence notwithstanding, I do think there are valid cases for
reassembling this array without resync. If there's a way to do that,
I'd still like to know.
With appreciation,
Aryeh Leib Taurog
^ permalink raw reply
* Re: Reassembling RAID1 after good drive was offline [newbie]
From: Robert L Mathews @ 2015-01-02 18:38 UTC (permalink / raw)
To: linux-raid
In-Reply-To: <20150102130101.GC6294@deb76.aryehleib.com>
On 1/2/15 5:01 AM, Aryeh Leib Taurog wrote:
> While doing this I discovered one of the usb cables is flaky, which
> explains why the device (sdc below) wasn't always coming on line.
Okay, that's probably the source of the problem, then. One of the times
when you restarted, only the non-flaky one was available. That made md
assemble the array with only that member, increasing the event count on
that partition. So now the partitions are "different" (although perhaps
you didn't change the data on them if you never mounted the array after
the problem started).
Your question is about how to reassemble them without a resync, which is
understandable, but the reason md isn't doing that automatically is that
it thinks they might be different.
I personally would always do a resync in such a case. A flaky cable
suggests that data on it is suspect, and this is one of the things RAID
1 is for: it allows you to copy clean data to suspect partitions.
But I'm known to be data-paranoid. Other people may have different
opinions/suggestions.
--
Robert L Mathews, Tiger Technologies, http://www.tigertech.net/
^ permalink raw reply
* Re: Reassembling RAID1 after good drive was offline [newbie]
From: Aryeh Leib Taurog @ 2015-01-02 14:02 UTC (permalink / raw)
To: linux-raid
In-Reply-To: <54A67A8C.8080803@youngman.org.uk>
On Fri, Jan 02, 2015 at 11:02, Anthonys Lists wrote:
>On 01/01/2015 20:54, Robert L Mathews wrote:
>>> >Personalities : [raid1]
>>> >md126 : active (auto-read-only) raid1 sdc2[0]
>>> > 943587136 blocks super 1.2 [2/1] [U_]
>>> >
>>> >md127 : active (auto-read-only) raid1 sdd2[1]
>>> > 943587136 blocks super 1.2 [2/1] [_U]
>> Still sdc2 and sdd2, although now in two arrays.
>>
>>
> What version is mdadm? iirc there is a known bug in older versions
> which does this. (Splits one array into two arrays.)
$ mdadm --version
mdadm - v3.2.5 - 18th May 2012
It's the mdadm packaged with Debian wheezy
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox