mdadm source rpm build error + command-line parsing error + resync reporting problem

mdadm source rpm build error + command-line parsing error + resync reporting problem

[Mike Hardy]

I got this when I attempted to rpmbuild -ba mdadm.spec:
------------------------

In file included from super0.c:31:
/usr/include/asm/byteorder.h:6:2: error: #warning using private kernel
header; include <endian.h> instead!
make: *** [super0.o] Error 1
error: Bad exit status from /var/tmp/rpm-tmp.64960 (%build)


RPM build errors:
    Bad exit status from /var/tmp/rpm-tmp.64960 (%build)
[root@istanbul ~]# uname -a
Linux istanbul.tacitknowledge.com 2.6.14-1.1644_FC4smp #1 SMP Sun Nov 27
03:39:31 EST 2005 i686 i686 i386 GNU/Linux
[root@istanbul ~]# cat /etc/redhat-release
Fedora Core release 4 (Stentz)

--------------------------

This occurs for the files super0.c, super1.c, and bitmap.c.

If you remove the -Werror flag from the Makefile, it works though it
still generates warnings.

I installed the new version of mdadm in the hopes that it would improve
the situation with the grow mode commandline parsing, which was broken
in the 1.11 that FC4 ships. Alas, it did not:

[root@istanbul redhat]# mdadm -G /dev/md1
mdadm: no changes to --grow
[root@istanbul redhat]# mdadm -G /dev/md1 --size
mdadm: option `--size' requires an argument

The man page indicates that if I don't set the size, it will grow to
fill the smallest device in the array. It's definitely not doing that...

I read mdadm.c, and see that 'max' is the argument that mdadm is looking
for, but the man page isn't mentioning.

Finally, and I think was reported once before, but I don't recall a
resolution for it, I got this once I specified that it grow to maximum size:

[root@istanbul mdadm-2.1]# cat /proc/mdstat
Personalities : [raid1]
md2 : active raid1 sdb1[0] sda1[1]
      98176 blocks [2/2] [UU]

md1 : active raid1 sdb3[0] sda3[1]
      70573440 blocks [2/2] [UU]
      [=========>...........]  resync = 49.9% (35220224/70573440)
finish=0.2min speed=2515730K/sec

md0 : active raid1 sdb2[0] sda2[1]
      977856 blocks [2/2] [UU]

unused devices: <none>

This is a dual Xeon (so it looks like a 4-CPU machine). Not sure what's
causing that though...

-Mike

Re: mdadm source rpm build error + command-line parsing error + resync reporting problem

[Mike Hardy]

Mike Hardy wrote:

> Finally, and I think was reported once before, but I don't recall a
> resolution for it, I got this once I specified that it grow to maximum size:
> 
> [root@istanbul mdadm-2.1]# cat /proc/mdstat
> Personalities : [raid1]
> md2 : active raid1 sdb1[0] sda1[1]
>       98176 blocks [2/2] [UU]
> 
> md1 : active raid1 sdb3[0] sda3[1]
>       70573440 blocks [2/2] [UU]
>       [=========>...........]  resync = 49.9% (35220224/70573440)
> finish=0.2min speed=2515730K/sec
> 
> md0 : active raid1 sdb2[0] sda2[1]
>       977856 blocks [2/2] [UU]
> 
> unused devices: <none>
> 
> This is a dual Xeon (so it looks like a 4-CPU machine). Not sure what's
> causing that though...

Sorry to reply to myself, but I just did the same thing on a different
machine (with an older mdadm) and I also got an..."optimistic" resync
speed reading on it after the grow, when it 'resumed from checkpoint'.

Oddly, these machines reported normal speeds during the resyncs that
occurred when I rotated larger drives into the machines.

Perhaps it has something to do with resumption from checkpoints?

Anyway, I hope all the reports help.

-Mike

Re: mdadm source rpm build error + command-line parsing error + resync reporting problem

[Neil Brown]

On Thursday December 8, mhardy@h3c.com wrote:
> 
> I got this when I attempted to rpmbuild -ba mdadm.spec:
> ------------------------
> 
> In file included from super0.c:31:
> /usr/include/asm/byteorder.h:6:2: error: #warning using private kernel
> header; include <endian.h> instead!
> make: *** [super0.o] Error 1
> error: Bad exit status from /var/tmp/rpm-tmp.64960 (%build)
> 
> 
> RPM build errors:
>     Bad exit status from /var/tmp/rpm-tmp.64960 (%build)
> [root@istanbul ~]# uname -a
> Linux istanbul.tacitknowledge.com 2.6.14-1.1644_FC4smp #1 SMP Sun Nov 27
> 03:39:31 EST 2005 i686 i686 i386 GNU/Linux
> [root@istanbul ~]# cat /etc/redhat-release
> Fedora Core release 4 (Stentz)
> 

Yes... redhat have done something silly (I think).
<endian.h> simply doesn't contain the stuff that I need from
byteorder.h.

I wonder if there is some #define I can stick in there to trick
byteorder.h into not complaining....


> --------------------------
> 
> This occurs for the files super0.c, super1.c, and bitmap.c.
> 
> If you remove the -Werror flag from the Makefile, it works though it
> still generates warnings.

That's easier to do now, you can just

   mdadm CWFLAGS=

and it won't bother so much about warnings.

> 
> I installed the new version of mdadm in the hopes that it would improve
> the situation with the grow mode commandline parsing, which was broken
> in the 1.11 that FC4 ships. Alas, it did not:
> 
> [root@istanbul redhat]# mdadm -G /dev/md1
> mdadm: no changes to --grow
> [root@istanbul redhat]# mdadm -G /dev/md1 --size
> mdadm: option `--size' requires an argument
> 
> The man page indicates that if I don't set the size, it will grow to
> fill the smallest device in the array. It's definitely not doing
> that...

Could you please point me to the part of the man page that says that?
I will try to fix it.

> 
> I read mdadm.c, and see that 'max' is the argument that mdadm is looking
> for, but the man page isn't mentioning.

Hmmm.. I can find it in the man page - look at the section on --size
(but then, I know where to look...)
You are the first person with this problem, so maybe I need to make it
more explicit.

> 
> Finally, and I think was reported once before, but I don't recall a
> resolution for it, I got this once I specified that it grow to maximum size:
> 
> [root@istanbul mdadm-2.1]# cat /proc/mdstat
> Personalities : [raid1]
> md2 : active raid1 sdb1[0] sda1[1]
>       98176 blocks [2/2] [UU]
> 
> md1 : active raid1 sdb3[0] sda3[1]
>       70573440 blocks [2/2] [UU]
>       [=========>...........]  resync = 49.9% (35220224/70573440)
> finish=0.2min speed=2515730K/sec
> 
> md0 : active raid1 sdb2[0] sda2[1]
>       977856 blocks [2/2] [UU]
> 
> unused devices: <none>
> 
> This is a dual Xeon (so it looks like a 4-CPU machine). Not sure what's
> causing that though...

You mean the excessive speed? 
The accounting assumes that it is resyncing every block.  When it
doesn't have to do that, it gets it a bit wrong... I'll look into it.

Thanks for the reports,
NeilBrown

Re: mdadm source rpm build error + command-line parsing error + resync reporting problem

[Mike Hardy]

Neil Brown wrote:

>>The man page indicates that if I don't set the size, it will grow to
>>fill the smallest device in the array. It's definitely not doing
>>that...
> 
> 
> Could you please point me to the part of the man page that says that?
> I will try to fix it.

I see what you mean now - I got confused because there are two
paragraphs in the size option description, one indicates that you don't
need to set it, then the second paragraph discusses the "grow" context
and indicates you can use 'max'.

The first paragraph doesn't indicate a specific context, so I took it to
cover all possible contexts where 'size' was applicable, and it seems
like it isn't

I did successfully grow my md arrays though, which was nice, but then
ext2online didn't do what I asked it to do, and now I'm back to just
doing the more typical dump/restore thing anyway.

Not really on-topic, but has anyone else gotten ext2online to work? I
get this "Dec  8 21:27:02 istanbul kernel: JBD: ext2online wants too
many credits (2050 > 2048)" after letting it attempt to extend the FS to
cover the entire device.

-Mike

Re: mdadm source rpm build error + command-line parsing error + resync reporting problem

[Neil Brown]

On Thursday December 8, mhardy@h3c.com wrote:
> 
> Not really on-topic, but has anyone else gotten ext2online to work? I
> get this "Dec  8 21:27:02 istanbul kernel: JBD: ext2online wants too
> many credits (2050 > 2048)" after letting it attempt to extend the FS to
> cover the entire device.

I haven't used ext2online myself, but I believe that there are some
aspects of the filesystem that it is not able to resize, and you have
to have created your filesystem with the understanding that you might
want to grow it later.
 From the mkfs.ext3 man page:

       -E extended-options
 ...
                   resize=max-online-resize
                          Reserve  enough  space  so  that  the  block   group
                          descriptor  table  can  grow to support a filesystem
                          that has max-online-resize blocks.


Not a very desirable situation, but just part of the legacy that
ext2/3 brings with it.

So this time when you mkfs your new arrays, plan for growth :-)

NeilBrown

Re: mdadm source rpm build error + command-line parsing error + resync reporting problem

[Mike Hardy]

Neil Brown wrote:
> On Thursday December 8, mhardy@h3c.com wrote:
> 
>>Not really on-topic, but has anyone else gotten ext2online to work? I
>>get this "Dec  8 21:27:02 istanbul kernel: JBD: ext2online wants too
>>many credits (2050 > 2048)" after letting it attempt to extend the FS to
>>cover the entire device.
> 
> 
> I haven't used ext2online myself, but I believe that there are some
> aspects of the filesystem that it is not able to resize, and you have
> to have created your filesystem with the understanding that you might
> want to grow it later.
>  From the mkfs.ext3 man page:
> 
>        -E extended-options
>  ...
>                    resize=max-online-resize
>                           Reserve  enough  space  so  that  the  block   group
>                           descriptor  table  can  grow to support a filesystem
>                           that has max-online-resize blocks.
> 
> 
> Not a very desirable situation, but just part of the legacy that
> ext2/3 brings with it.
> 
> So this time when you mkfs your new arrays, plan for growth :-)
> 
> NeilBrown

I was able to boot off the redhat install CD, use the 'linux rescue'
mode to get a shell, and then use resize2fs to do what I wanted.

Basically, if you didn't plan ahead with the right max-online-resize
option, you can still grow the filesystem, just not while it's mounted
and running.

So at this point, I've got twice the disk space on both the systems I
was working on. If only I had more to do, now that I know what I'm doing...

-Mike