Newbie questions: Max Active disks, RAID migration, Compiling mdadm 2.3

Newbie questions: Max Active disks, RAID migration, Compiling mdadm 2.3

[Martin Ritchie]

Sorry if these are total newbie questions.

Why can't I have more than 4 active drives in my md RAID?

Why can't I easily migrate a RAID 0 to RAID 5. As I see it RAID 0 is  
just RAID 5 with a failed parity check drive?

Perhaps this is a limitation of the old v1.11 that FC4 updates to.

I tried to compile 2.3 but I get this error:

$make
gcc -Wall -Werror -Wstrict-prototypes -DCONFFILE=\"/etc/mdadm.conf\" - 
ggdb -DSendmail=\""/usr/sbin/sendmail -t"\"   -c -o super0.o super0.c
In file included from super0.c:31:
/usr/include/asm/byteorder.h:6:2: error: #warning using private  
kernel header; include <endian.h> instead!
make: *** [super0.o] Error 1

I'm not too familiar with compiling this sort of thing. (I usually  
live further away from the hardware and endian issues). I'm guessing  
there is some sort of option i have to specify to say that this  
should use the private kernel headers. Including endian.h instead  
didn't help:

$make
gcc -Wall -Werror -Wstrict-prototypes -DCONFFILE=\"/etc/mdadm.conf\" - 
ggdb -DSendmail=\""/usr/sbin/sendmail -t"\"   -c -o super0.o super0.c
cc1: warnings being treated as errors
super0.c: In function ‘add_internal_bitmap0’:
super0.c:737: warning: implicit declaration of function ‘__cpu_to_le32’
super0.c:742: warning: implicit declaration of function ‘__cpu_to_le64’
make: *** [super0.o] Error 1

Oh just because I know it is going to be an issue I'm building on a  
Athlon 64... my first 64bit linux box so I'm sure there are going to  
be gotchas that I've not thought about.

Is there somewhere I over looked for finding this information.

TIA
-- 
Martin

"What we do in life, echoes in Eternity."
-Maximus Decimus Meridius



-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Newbie questions: Max Active disks, RAID migration, Compiling mdadm 2.3

[Mike Hardy]

If you remove the '-Werror' it'll compile and work, but you still can't
convert a raid 0 to a raid 5. You're raid level understanding is off as
well, raid 5 is a parity block rotating around all drives, you were
thinking of raid 4 which has a single parity disk. Migrating raid 0 to
raid 4 (and vice versa) should be possible technically, but I don't
think it's implemented anywhere

You should be able to have more than 4 active drives though. I am at
this moment building an array with 6 components, I'm running a few with
more than that, and these are by no means the largest arrays that people
are running - just examples of it working.

-Mike

Martin Ritchie wrote:
> Sorry if these are total newbie questions.
> 
> Why can't I have more than 4 active drives in my md RAID?
> 
> Why can't I easily migrate a RAID 0 to RAID 5. As I see it RAID 0 is 
> just RAID 5 with a failed parity check drive?
> 
> Perhaps this is a limitation of the old v1.11 that FC4 updates to.
> 
> I tried to compile 2.3 but I get this error:
> 
> $make
> gcc -Wall -Werror -Wstrict-prototypes -DCONFFILE=\"/etc/mdadm.conf\" -
> ggdb -DSendmail=\""/usr/sbin/sendmail -t"\"   -c -o super0.o super0.c
> In file included from super0.c:31:
> /usr/include/asm/byteorder.h:6:2: error: #warning using private  kernel
> header; include <endian.h> instead!
> make: *** [super0.o] Error 1
> 
> I'm not too familiar with compiling this sort of thing. (I usually  live
> further away from the hardware and endian issues). I'm guessing  there
> is some sort of option i have to specify to say that this  should use
> the private kernel headers. Including endian.h instead  didn't help:
> 
> $make
> gcc -Wall -Werror -Wstrict-prototypes -DCONFFILE=\"/etc/mdadm.conf\" -
> ggdb -DSendmail=\""/usr/sbin/sendmail -t"\"   -c -o super0.o super0.c
> cc1: warnings being treated as errors
> super0.c: In function ‘add_internal_bitmap0’:
> super0.c:737: warning: implicit declaration of function ‘__cpu_to_le32’
> super0.c:742: warning: implicit declaration of function ‘__cpu_to_le64’
> make: *** [super0.o] Error 1
> 
> Oh just because I know it is going to be an issue I'm building on a 
> Athlon 64... my first 64bit linux box so I'm sure there are going to  be
> gotchas that I've not thought about.
> 
> Is there somewhere I over looked for finding this information.
> 
> TIA

-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

How do I find endian-conversion function on Redhar/Fedora

[Neil Brown]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset=unknown, Size: 1954 bytes --]

On Sunday February 5, ritchiem@mac.com wrote:
> 
> I tried to compile 2.3 but I get this error:
> 
> $make
> gcc -Wall -Werror -Wstrict-prototypes -DCONFFILE=\"/etc/mdadm.conf\" - 
> ggdb -DSendmail=\""/usr/sbin/sendmail -t"\"   -c -o super0.o super0.c
> In file included from super0.c:31:
> /usr/include/asm/byteorder.h:6:2: error: #warning using private  
> kernel header; include <endian.h> instead!
> make: *** [super0.o] Error 1
..
... including endian.h instead of byteorder.h ...
> 
> $make
> gcc -Wall -Werror -Wstrict-prototypes -DCONFFILE=\"/etc/mdadm.conf\" - 
> ggdb -DSendmail=\""/usr/sbin/sendmail -t"\"   -c -o super0.o super0.c
> cc1: warnings being treated as errors
> super0.c: In function ‘add_internal_bitmap0’:
> super0.c:737: warning: implicit declaration of function ‘__cpu_to_le32’
> super0.c:742: warning: implicit declaration of function ‘__cpu_to_le64’
> make: *** [super0.o] Error 1

This is starting to really annoy me, and probably bother other people
to.

mdadm needs functions to convert 16, 32, and 64 bit values between
host-endian and little-endian.  The standard 'C' library doesn't seem
to provide any.
The linux-kernel headers provide __cpu_to_leXX and __leXX_to_cpu, so I
use them.  But Redhat/Fedora seem to take this as an affront to the
rightness of things and complain, saying I should use "endian.h". 
However on SuSE and Debian, 'endian.h' doesn't contain any function to
manipulate byte order.

So my plea for help:  Could someone with a Redhat installation please
see if there is any way to get functions that convert between little
endian and host endian, or if there is some name I can #define to
disable the 'private kernel header' warning..

Thanks.

NeilBrown

-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: How do I find endian-conversion function on Redhar/Fedora

[Gil]

[-- Attachment #1: Type: text/plain, Size: 468 bytes --]

Neil Brown wrote:
> So my plea for help:  Could someone with a Redhat installation
> please see if there is any way to get functions that convert
> between little endian and host endian, or if there is some name I
> can #define to disable the 'private kernel header' warning..

I have an FC4 machine that I tried this out on.  I have no idea if
this is the right way to use these headers in /usr/include/linux,
but it made the compile problem go away at least.

--Gil

[-- Attachment #2: mdadm-2.3-byteorder.patch --]
[-- Type: text/plain, Size: 1966 bytes --]

diff -r -u mdadm-2.3/bitmap.c mdadm-2.3-patched/bitmap.c
--- mdadm-2.3/bitmap.c	2005-12-15 17:41:31.000000000 -0800
+++ mdadm-2.3-patched/bitmap.c	2006-02-05 15:09:32.000000000 -0800
@@ -21,7 +21,16 @@
 #include <sys/types.h>
 #include <sys/stat.h>
 #include "mdadm.h"
-#include <asm/byteorder.h>
+#include <endian.h>
+#if __BYTE_ORDER == __LITTLE_ENDIAN
+#  include <linux/byteorder/little_endian.h>
+#elif __BYTE_ORDER == __BIG_ENDIAN
+#  include <linux/byteorder/big_endian.h>
+#elif __BYTE_ORDER == __PDP_ENDIAN
+#  include <linux/byteorder/pdp_endian.h>
+#else
+#  error "unknown endianness."
+#endif
 
 #define min(a,b) (((a) < (b)) ? (a) : (b))
 
diff -r -u mdadm-2.3/super0.c mdadm-2.3-patched/super0.c
--- mdadm-2.3/super0.c	2006-01-26 23:06:09.000000000 -0800
+++ mdadm-2.3-patched/super0.c	2006-02-05 14:59:37.000000000 -0800
@@ -28,7 +28,18 @@
  */
 
 #include "mdadm.h"
-#include <asm/byteorder.h>
+
+#include <endian.h>
+#if __BYTE_ORDER == __LITTLE_ENDIAN
+#  include <linux/byteorder/little_endian.h>
+#elif __BYTE_ORDER == __BIG_ENDIAN
+#  include <linux/byteorder/big_endian.h>
+#elif __BYTE_ORDER == __PDP_ENDIAN
+#  include <linux/byteorder/pdp_endian.h>
+#else
+#  error "unknown endianness."
+#endif
+
 
 /*
  * All handling for the 0.90.0 version superblock is in
Only in mdadm-2.3-patched: .super0.c.swp
diff -r -u mdadm-2.3/super1.c mdadm-2.3-patched/super1.c
--- mdadm-2.3/super1.c	2006-01-30 17:45:32.000000000 -0800
+++ mdadm-2.3-patched/super1.c	2006-02-05 15:09:05.000000000 -0800
@@ -29,7 +29,15 @@
 
 #include "mdadm.h"
 #include <endian.h>
-#include "asm/byteorder.h"
+#if __BYTE_ORDER == __LITTLE_ENDIAN
+#  include <linux/byteorder/little_endian.h>
+#elif __BYTE_ORDER == __BIG_ENDIAN
+#  include <linux/byteorder/big_endian.h>
+#elif __BYTE_ORDER == __PDP_ENDIAN
+#  include <linux/byteorder/pdp_endian.h>
+#else
+#  error "unknown endianness."
+#endif
 /*
  * The version-1 superblock :
  * All numeric fields are little-endian.

Re: How do I find endian-conversion function on Redhar/Fedora

[Neil Brown]

On Sunday February 5, gil@fooplanet.com wrote:
> Neil Brown wrote:
> > So my plea for help:  Could someone with a Redhat installation
> > please see if there is any way to get functions that convert
> > between little endian and host endian, or if there is some name I
> > can #define to disable the 'private kernel header' warning..
> 
> I have an FC4 machine that I tried this out on.  I have no idea if
> this is the right way to use these headers in /usr/include/linux,
> but it made the compile problem go away at least.

Ok,.... that's interesting. 
I would in one sense just be moving the problem and maybe Redhat would
try to declare linux/byteorder/* off limits, but I guess I could
simply include them in the mdadm distrib if they did that.

Thanks.  If nothing better surfaces I'll probably include this in
mdadm 2.4.

NeilBrown

> 
> --Gil
> diff -r -u mdadm-2.3/bitmap.c mdadm-2.3-patched/bitmap.c
> --- mdadm-2.3/bitmap.c	2005-12-15 17:41:31.000000000 -0800
> +++ mdadm-2.3-patched/bitmap.c	2006-02-05 15:09:32.000000000 -0800
> @@ -21,7 +21,16 @@
>  #include <sys/types.h>
>  #include <sys/stat.h>
>  #include "mdadm.h"
> -#include <asm/byteorder.h>
> +#include <endian.h>
> +#if __BYTE_ORDER == __LITTLE_ENDIAN
> +#  include <linux/byteorder/little_endian.h>
> +#elif __BYTE_ORDER == __BIG_ENDIAN
> +#  include <linux/byteorder/big_endian.h>
> +#elif __BYTE_ORDER == __PDP_ENDIAN
> +#  include <linux/byteorder/pdp_endian.h>
> +#else
> +#  error "unknown endianness."
> +#endif

Re: Newbie questions: Max Active disks, RAID migration, Compiling mdadm 2.3

[Martin Ritchie]

I can see what I did now. Very silly.

I created a RAID 5 with 4 disks then went about hot adding disks.  
Silly me thinking it would increase the active space. Of course with  
the parity being shared through the active disks this doesn't makes  
sense.

I was wanting to define a systematic way that I could start with a  
small number of disks and then expand the RAID as I needed more space  
(and could afford more disks).

Is raidreconf the only way for reconfiguring a live system?

Cheers
-- 
Martin

"What we do in life, echoes in Eternity."
-Maximus Decimus Meridius



On Feb 05, 2006, at 21:40, Mark Hahn wrote:

>> Why can't I have more than 4 active drives in my md RAID?
>
> why do you think you can't?
>
>> Why can't I easily migrate a RAID 0 to RAID 5. As I see it RAID 0 is
>> just RAID 5 with a failed parity check drive?
>
> that would be raid3, actually.  with r5, the parityblock rotates
> among all drives.
>
>> $make
>> gcc -Wall -Werror -Wstrict-prototypes -DCONFFILE=\"/etc/mdadm.conf 
>> \" -
>> ggdb -DSendmail=\""/usr/sbin/sendmail -t"\"   -c -o super0.o super0.c
>> In file included from super0.c:31:
>> /usr/include/asm/byteorder.h:6:2: error: #warning using private
>> kernel header; include <endian.h> instead!
>
> that's a bug in FC's includes, nothing to do with MD.
>
>> $make
>> gcc -Wall -Werror -Wstrict-prototypes -DCONFFILE=\"/etc/mdadm.conf 
>> \" -
>> ggdb -DSendmail=\""/usr/sbin/sendmail -t"\"   -c -o super0.o super0.c
>> cc1: warnings being treated as errors
>> super0.c: In function ‘add_internal_bitmap0’:
>> super0.c:737: warning: implicit declaration of function  
>> ‘__cpu_to_le32’
>> super0.c:742: warning: implicit declaration of function  
>> ‘__cpu_to_le64’
>> make: *** [super0.o] Error 1
>
> I'd guess that MD sources should include -D__KERNEL__ in order to
> get those functions, or else include its own definitions.
>
>> Oh just because I know it is going to be an issue I'm building on a
>> Athlon 64... my first 64bit linux box so I'm sure there are going to
>> be gotchas that I've not thought about.
>
> LOTS of people have k8 boxes by now; it's not possible you're seeing
> anything really new...
>

-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Newbie questions: Max Active disks, RAID migration, Compiling mdadm 2.3

[Neil Brown]

On Sunday February 5, ritchiem@mac.com wrote:
> I can see what I did now. Very silly.
> 
> I created a RAID 5 with 4 disks then went about hot adding disks.  
> Silly me thinking it would increase the active space. Of course with  
> the parity being shared through the active disks this doesn't makes  
> sense.
> 
> I was wanting to define a systematic way that I could start with a  
> small number of disks and then expand the RAID as I needed more space  
> (and could afford more disks).
> 
> Is raidreconf the only way for reconfiguring a live system?
> 

raidreconf doesn't allow you to reconfigure a live system - only an
inactive system.
But for the moment, that is the only way, yes.

In 2.6.17 or 2.6.18 it should be possible to restripe a raid5 to have
a larger number of devices.  This can happen while the array is in
active use.
The code is mostly written, and could be used today, but I don't
recommend it on valuable data yet as there are known bugs.

NeilBrown