MDADM 3.3 broken?

MDADM 3.3 broken?

[David F.]

Hi,

we updated our linux disk with mdadm 3.3 from 3.2.6 and customers are
finding their RAID is no longer detected.  It's only been a couple
weeks and based on the number of customers, we know there is an issue.
 We're having those with problems workaround by having them load
dmraid instead for now.

We also did tests locally and finding intermittent problems with
RAID-0 on ISW - sometimes 3.3 doesn't identify both drives as RAID
members.  3.2.6 works 100% of the time.

Also with DDF RAID - cisco server for example not detecting RAID5 -
C220M3_LFF_SpecSheet.pdf. I believe they are using the LSI MegaRaid
since DMRAID reports that.

Are these problems known - we wouldn't mind moving to the latest
version if your pretty sure it fixes it, otherwise we're going to have
to revert to 3.2.6?

TIA!!

Re: MDADM 3.3 broken?

[Martin Wilck]

On 11/18/2013 07:26 PM, David F. wrote:
> Hi,
> 
> we updated our linux disk with mdadm 3.3 from 3.2.6 and customers are
> finding their RAID is no longer detected.  It's only been a couple
> weeks and based on the number of customers, we know there is an issue.
>  We're having those with problems workaround by having them load
> dmraid instead for now.
> 
> We also did tests locally and finding intermittent problems with
> RAID-0 on ISW - sometimes 3.3 doesn't identify both drives as RAID
> members.  3.2.6 works 100% of the time.
> 
> Also with DDF RAID - cisco server for example not detecting RAID5 -
> C220M3_LFF_SpecSheet.pdf. I believe they are using the LSI MegaRaid
> since DMRAID reports that.

Could you please provide mdadm -E and possibly mdadm --dump output of
the disks that aren't detected? How does RAID discovery work on your
systems? Are you using standard udev rules or something special? How
does your mdadm.conf look like?

Regards
Martin


> 
> Are these problems known - we wouldn't mind moving to the latest
> version if your pretty sure it fixes it, otherwise we're going to have
> to revert to 3.2.6?
> 
> TIA!!
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: MDADM 3.3 broken?

[David F.]

Sure - Here is the isw issue we can reproduce below (both versions
output is included):

Comparison of results using mdadm 3.2.6 (OK) vs mdadm 3.3 (not OK)
under otherwise identical conditions
from our boot disk, when trying to assemble an imsm RAID0 array (two
entire 500g drives /dev/sdb /dev/sdc).

The boot disk uses kernel 3.11.7, and udev 175 from the Debian Wheezy
udev package, including
their rules file (64-md-raid.rules).

Note: In this case at least, we get 2 different results when running
mdadm 3.3 under the same
conditions (some kind of race condtion?). The 2 sets of results are
under the headings "output1"
and "output2" below. For output2, the array is succesfully assembled
(uses /dev/sdb and /dev/sdc),
while output1 uses /dev/sdb and /dev/sdc2, and fails to assemble it.


contents of mdadm.conf (when attempting to assemble array):
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# mdadm.conf
#
# Please refer to mdadm.conf(5) for information about this file.
#

# by default (built-in), scan all partitions (/proc/partitions) and all
# containers for MD superblocks. alternatively, specify devices to scan, using
# wildcards if desired.
DEVICE partitions containers

# automatically tag new arrays as belonging to the local system
HOMEHOST <system>

ARRAY metadata=imsm UUID=0ee9661b:d0ee3f07:e2f3b890:f658562d
ARRAY /dev/md/RAID0 container=0ee9661b:d0ee3f07:e2f3b890:f658562d
member=0 UUID=540c3a88:7717daff:ccaf97eb:7961ac32


script that starts mdadm on boot (after udev started):
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
#! /bin/bash

if [ ! -e /proc/mdstat ]; then
    echo "Software RAID drivers not loaded"
    exit 0
fi

if [ ! -e /etc/mdadm/mdadm.conf-default ]; then
    echo "Default config file not found in /etc/mdadm"
    exit 0
else
    cp /etc/mdadm/mdadm.conf-default /etc/mdadm/mdadm.conf
fi

mdadm --examine --scan >> /etc/mdadm/mdadm.conf
mdadm --assemble --scan --no-degraded
echo


output of 'mdadm --examine --scan' (same for both):
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ARRAY metadata=imsm UUID=0ee9661b:d0ee3f07:e2f3b890:f658562d
ARRAY /dev/md/RAID0 container=0ee9661b:d0ee3f07:e2f3b890:f658562d
member=0 UUID=540c3a88:7717daff:ccaf97eb:7961ac32


output of 'mdadm --assemble --scan --no-degraded -v' (mdadm 3.2.6):
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
mdadm: looking for devices for further assembly
mdadm: no RAID superblock on /dev/sdd4
mdadm: no RAID superblock on /dev/sdd3
mdadm: no RAID superblock on /dev/sdd2
mdadm: no RAID superblock on /dev/sdd1
mdadm: no RAID superblock on /dev/sdd
mdadm: cannot open device /dev/sr1: No medium found
mdadm: no RAID superblock on /dev/sdc2
mdadm: no RAID superblock on /dev/sdc1
mdadm: no RAID superblock on /dev/sda11
mdadm: no RAID superblock on /dev/sda10
mdadm: no RAID superblock on /dev/sda9
mdadm: no RAID superblock on /dev/sda8
mdadm: no RAID superblock on /dev/sda7
mdadm: no RAID superblock on /dev/sda6
mdadm: no RAID superblock on /dev/sda5
mdadm: no RAID superblock on /dev/sda4
mdadm: no RAID superblock on /dev/sda3
mdadm: no RAID superblock on /dev/sda2
mdadm: no RAID superblock on /dev/sda1
mdadm: no RAID superblock on /dev/sda
mdadm: cannot open device /dev/sr0: No medium found
mdadm: /dev/sdc is identified as a member of /dev/md/imsm0, slot -1.
mdadm: /dev/sdb is identified as a member of /dev/md/imsm0, slot -1.
mdadm: added /dev/sdb to /dev/md/imsm0 as -1
mdadm: added /dev/sdc to /dev/md/imsm0 as -1
mdadm: Container /dev/md/imsm0 has been assembled with 2 drives
mdadm: looking for devices for /dev/md/RAID0
mdadm: looking in container /dev/md127
mdadm: found match on member /md127/0 in /dev/md127
mdadm: Started /dev/md/RAID0 with 2 devices

output of 'dmesg | grep md:' and 'ls -l /dev/sdc*' - mdadm 3.2.6:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
md: linear personality registered for level -1
md: raid0 personality registered for level 0
md: raid1 personality registered for level 1
md: raid10 personality registered for level 10
md: raid6 personality registered for level 6
md: raid5 personality registered for level 5
md: raid4 personality registered for level 4
md: multipath personality registered for level -4
md: md127 stopped.
md: bind<sdb>
md: bind<sdc>
md: md126 stopped.
md: bind<sdb>
md: bind<sdc>
md: RAID0 configuration for md126 - 1 zone
md: zone0=[sdc/sdb]
brw-rw---T    1 root     disk        8,  32 Nov 18 14:59 /dev/sdc
brw-rw---T    1 root     disk        8,  33 Nov 18 09:59 /dev/sdc1
brw-rw---T    1 root     disk        8,  34 Nov 18 09:59 /dev/sdc2

output1 of 'mdadm --assemble --scan --no-degraded -v' (mdadm 3.3 -
note using /dev/sdc2, not /dev/sdc):
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
mdadm: looking for devices for further assembly
mdadm: no RAID superblock on /dev/sdd4
mdadm: no RAID superblock on /dev/sdd3
mdadm: no RAID superblock on /dev/sdd2
mdadm: no RAID superblock on /dev/sdd1
mdadm: no RAID superblock on /dev/sdd
mdadm: cannot open device /dev/sr1: No medium found
mdadm: cannot open device /dev/sr0: No medium found
mdadm: no RAID superblock on /dev/sda11
mdadm: no RAID superblock on /dev/sda10
mdadm: no RAID superblock on /dev/sda9
mdadm: no RAID superblock on /dev/sda8
mdadm: no RAID superblock on /dev/sda7
mdadm: no RAID superblock on /dev/sda6
mdadm: no RAID superblock on /dev/sda5
mdadm: no RAID superblock on /dev/sda4
mdadm: no RAID superblock on /dev/sda3
mdadm: no RAID superblock on /dev/sda2
mdadm: no RAID superblock on /dev/sda1
mdadm: no RAID superblock on /dev/sda
mdadm: no RAID superblock on /dev/sdc1
mdadm: /dev/sdb is identified as a member of /dev/md/imsm0, slot -1.
mdadm: /dev/sdc2 is identified as a member of /dev/md/imsm0, slot -1.
mdadm: /dev/sdc is identified as a member of /dev/md/imsm0, slot -1.
mdadm: added /dev/sdc2 to /dev/md/imsm0 as -1
mdadm: failed to add /dev/sdc to /dev/md/imsm0: Device or resource busy
mdadm: added /dev/sdb to /dev/md/imsm0 as -1
mdadm: Container /dev/md/imsm0 has been assembled with 2 drives
mdadm: looking for devices for /dev/md/RAID0
mdadm: looking in container /dev/md/imsm0
mdadm: found match on member /md127/0 in /dev/md/imsm0
mdadm: Started /dev/md/RAID0 with 2 devices

output1 of 'dmesg | grep md:' and 'ls -l /dev/sdc*' - mdadm 3.3:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
md: linear personality registered for level -1
md: raid0 personality registered for level 0
md: raid1 personality registered for level 1
md: raid10 personality registered for level 10
md: raid6 personality registered for level 6
md: raid5 personality registered for level 5
md: raid4 personality registered for level 4
md: multipath personality registered for level -4
md: md127 stopped.
md: bind<sdc2>
md: could not open unknown-block(8,32).
md: bind<sdb>
md: md126 stopped.
md: bind<sdb>
md: bind<sdc2>
md: RAID0 configuration for md126 - 1 zone
md: zone0=[sdc2/sdb]
brw-rw---T    1 root     disk        8,  32 Nov 18 15:02 /dev/sdc
brw-rw---T    1 root     disk        8,  33 Nov 18 10:02 /dev/sdc1
brw-rw---T    1 root     disk        8,  34 Nov 18 15:02 /dev/sdc2

output2 of 'mdadm --assemble --scan --no-degraded -v' (mdadm 3.3 -
note using /dev/sdc this time):
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
mdadm: looking for devices for further assembly
mdadm: no RAID superblock on /dev/sdd4
mdadm: no RAID superblock on /dev/sdd3
mdadm: no RAID superblock on /dev/sdd2
mdadm: no RAID superblock on /dev/sdd1
mdadm: no RAID superblock on /dev/sdd
mdadm: cannot open device /dev/sr1: No medium found
mdadm: cannot open device /dev/sr0: No medium found
mdadm: no RAID superblock on /dev/sdc1
mdadm: no RAID superblock on /dev/sda11
mdadm: no RAID superblock on /dev/sda10
mdadm: no RAID superblock on /dev/sda9
mdadm: no RAID superblock on /dev/sda8
mdadm: no RAID superblock on /dev/sda7
mdadm: no RAID superblock on /dev/sda6
mdadm: no RAID superblock on /dev/sda5
mdadm: no RAID superblock on /dev/sda4
mdadm: no RAID superblock on /dev/sda3
mdadm: no RAID superblock on /dev/sda2
mdadm: no RAID superblock on /dev/sda1
mdadm: no RAID superblock on /dev/sda
mdadm: /dev/sdc2 is identified as a member of /dev/md/imsm0, slot -1.
mdadm: /dev/sdc is identified as a member of /dev/md/imsm0, slot -1.
mdadm: /dev/sdb is identified as a member of /dev/md/imsm0, slot -1.
mdadm: added /dev/sdc to /dev/md/imsm0 as -1
mdadm: added /dev/sdb to /dev/md/imsm0 as -1
mdadm: failed to add /dev/sdc2 to /dev/md/imsm0: Device or resource busy
mdadm: Container /dev/md/imsm0 has been assembled with 2 drives
mdadm: looking for devices for /dev/md/RAID0
mdadm: looking in container /dev/md/imsm0
mdadm: found match on member /md127/0 in /dev/md/imsm0
mdadm: Started /dev/md/RAID0 with 2 devices

output2 of 'dmesg | grep md:' and 'ls -l /dev/sdc*' - mdadm 3.3:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
md: linear personality registered for level -1
md: raid0 personality registered for level 0
md: raid1 personality registered for level 1
md: raid10 personality registered for level 10
md: raid6 personality registered for level 6
md: raid5 personality registered for level 5
md: raid4 personality registered for level 4
md: multipath personality registered for level -4
md: md127 stopped.
md: bind<sdc>
md: bind<sdb>
md: could not open unknown-block(8,34).
md: md126 stopped.
md: bind<sdb>
md: bind<sdc>
md: RAID0 configuration for md126 - 1 zone
md: zone0=[sdc/sdb]
brw-rw---T    1 root     disk        8,  32 Nov 18 14:52 /dev/sdc
brw-rw---T    1 root     disk        8,  33 Nov 18 09:52 /dev/sdc1
brw-rw---T    1 root     disk        8,  34 Nov 18 14:52 /dev/sdc2


On Mon, Nov 18, 2013 at 12:22 PM, Martin Wilck <mwilck@arcor.de> wrote:
> On 11/18/2013 07:26 PM, David F. wrote:
>> Hi,
>>
>> we updated our linux disk with mdadm 3.3 from 3.2.6 and customers are
>> finding their RAID is no longer detected.  It's only been a couple
>> weeks and based on the number of customers, we know there is an issue.
>>  We're having those with problems workaround by having them load
>> dmraid instead for now.
>>
>> We also did tests locally and finding intermittent problems with
>> RAID-0 on ISW - sometimes 3.3 doesn't identify both drives as RAID
>> members.  3.2.6 works 100% of the time.
>>
>> Also with DDF RAID - cisco server for example not detecting RAID5 -
>> C220M3_LFF_SpecSheet.pdf. I believe they are using the LSI MegaRaid
>> since DMRAID reports that.
>
> Could you please provide mdadm -E and possibly mdadm --dump output of
> the disks that aren't detected? How does RAID discovery work on your
> systems? Are you using standard udev rules or something special? How
> does your mdadm.conf look like?
>
> Regards
> Martin
>
>
>>
>> Are these problems known - we wouldn't mind moving to the latest
>> version if your pretty sure it fixes it, otherwise we're going to have
>> to revert to 3.2.6?
>>
>> TIA!!
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Re: MDADM 3.3 broken?

[NeilBrown]

[-- Attachment #1: Type: text/plain, Size: 1012 bytes --]

On Mon, 18 Nov 2013 15:13:58 -0800 "David F." <df7729@gmail.com> wrote:


> output of 'mdadm --assemble --scan --no-degraded -v' (mdadm 3.2.6):
...
> mdadm: no RAID superblock on /dev/sdc2



> 
> output1 of 'mdadm --assemble --scan --no-degraded -v' (mdadm 3.3 -
> note using /dev/sdc2, not /dev/sdc):
.....
> mdadm: /dev/sdc2 is identified as a member of /dev/md/imsm0, slot -1.

So there is the problem.  mdadm 3.2.6 sees no RAID superblock on sdc2, while
mdadm 3.3 does (but should not).


However that code hasn't changed!

load_super_imsm() still starts with:


	if (test_partition(fd))
		/* IMSM not allowed on partitions */
		return 1;


and test_partition hasn't changed since it was written in April 2010 for
mdadm 3.1.3.

So I'm quite perplexed.

Is your mdadm-3.3 compiled from source or provided by a distro?

Can you run the "mdadm --assemble" under strace and post the result?

  strace -o /tmp/some-file mdadm --assemble --scan ......

Thanks,
NeilBrown

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 828 bytes --]

Re: MDADM 3.3 broken?

[David F.]

Hello,

The mdadm 3.3 binary we've been using was compiled from source on a Debian
Wheezy 32-bit install, with the source downloaded from kernel.org

But also, this morning, I tried the same test with the 3.3 mdadm binary
taken from the Debian Jessie package here:

packages.debian.org slash jessie slash i386 slash mdadm slash download
(mdadm_3.3-2_i386.deb)

The results were the same, as this output shows:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
mdadm: looking for devices for further assembly
mdadm: no RAID superblock on /dev/sdd4
mdadm: no RAID superblock on /dev/sdd3
mdadm: no RAID superblock on /dev/sdd2
mdadm: no RAID superblock on /dev/sdd1
mdadm: no RAID superblock on /dev/sdd
mdadm: cannot open device /dev/sr1: No medium found
mdadm: cannot open device /dev/sr0: No medium found
mdadm: no RAID superblock on /dev/sda11
mdadm: no RAID superblock on /dev/sda10
mdadm: no RAID superblock on /dev/sda9
mdadm: no RAID superblock on /dev/sda8
mdadm: no RAID superblock on /dev/sda7
mdadm: no RAID superblock on /dev/sda6
mdadm: no RAID superblock on /dev/sda5
mdadm: no RAID superblock on /dev/sda4
mdadm: no RAID superblock on /dev/sda3
mdadm: no RAID superblock on /dev/sda2
mdadm: no RAID superblock on /dev/sda1
mdadm: no RAID superblock on /dev/sda
mdadm: no RAID superblock on /dev/sdc1
mdadm: /dev/sdb is identified as a member of /dev/md/imsm0, slot -1.
mdadm: /dev/sdc2 is identified as a member of /dev/md/imsm0, slot -1.
mdadm: /dev/sdc is identified as a member of /dev/md/imsm0, slot -1.
mdadm: added /dev/sdc2 to /dev/md/imsm0 as -1
mdadm: failed to add /dev/sdc to /dev/md/imsm0: Device or resource busy
mdadm: added /dev/sdb to /dev/md/imsm0 as -1
mdadm: Container /dev/md/imsm0 has been assembled with 2 drives
mdadm: looking for devices for /dev/md/RAID0
mdadm: looking in container /dev/md/imsm0
mdadm: found match on member /md127/0 in /dev/md/imsm0
mdadm: Started /dev/md/RAID0 with 2 devices

I ran strace on this version with this command:
strace -o mdadm33trace.txt mdadm --assemble --scan --no-degraded

Here's a link to the strace output file:
www.dropbox.com slash s slash l5tcgu8zvjn7eb7 slash mdadm33trace.txt

On Mon, Nov 18, 2013 at 4:01 PM, NeilBrown <neilb@suse.de> wrote:
> On Mon, 18 Nov 2013 15:13:58 -0800 "David F." <df7729@gmail.com> wrote:
>
>
>> output of 'mdadm --assemble --scan --no-degraded -v' (mdadm 3.2.6):
> ...
>> mdadm: no RAID superblock on /dev/sdc2
>
>
>
>>
>> output1 of 'mdadm --assemble --scan --no-degraded -v' (mdadm 3.3 -
>> note using /dev/sdc2, not /dev/sdc):
> .....
>> mdadm: /dev/sdc2 is identified as a member of /dev/md/imsm0, slot -1.
>
> So there is the problem.  mdadm 3.2.6 sees no RAID superblock on sdc2, while
> mdadm 3.3 does (but should not).
>
>
> However that code hasn't changed!
>
> load_super_imsm() still starts with:
>
>
>         if (test_partition(fd))
>                 /* IMSM not allowed on partitions */
>                 return 1;
>
>
> and test_partition hasn't changed since it was written in April 2010 for
> mdadm 3.1.3.
>
> So I'm quite perplexed.
>
> Is your mdadm-3.3 compiled from source or provided by a distro?
>
> Can you run the "mdadm --assemble" under strace and post the result?
>
>   strace -o /tmp/some-file mdadm --assemble --scan ......
>
> Thanks,
> NeilBrown

Re: MDADM 3.3 broken?

[Martin Wilck]

The question I have is why is there IMSM meta data on sdc2 at all?
IMSM metadata sit at the end of a block device. So I figure that sdc2 is
the last partition, and by some wird circumstance it's so large that it
includes the last sectors of the physical disk where the metadata
resides. That would be a bad idea, a dd if=/dev/zero of=/dev/sdc2 would
wipe not only the partition but also the RAID meta data.

Martin

Re: MDADM 3.3 broken?

[David F.]

For cisco's server:

> mdadm --examine --scan
ARRAY metadata=ddf UUID=7ab254d0:fae71048:404edde9:750a8a05
ARRAY container=7ab254d0:fae71048:404edde9:750a8a05 member=0
UUID=5337ab03:86ca2abc:d42bfbc8:23626c78

> mdadm --assemble --scan --no-degraded -v >
mdadm: looking for devices for further assembly
mdadm: /dev/md/ddf0 is a container, but we are looking for components
mdadm: no RAID superblock on /dev/sdf
mdadm: no RAID superblock on /dev/md/MegaSR2
mdadm: no RAID superblock on /dev/md/MegaSR1
mdadm: no RAID superblock on /dev/md/MegaSR
mdadm: cannot open device /dev/sr0: No medium found
mdadm: /dev/sdd is busy - skipping
mdadm: /dev/sdc is busy - skipping
mdadm: /dev/sdb is busy - skipping
mdadm: /dev/sda is busy - skipping
mdadm: /dev/sde is busy - skipping
mdadm: looking for devices for further assembly
mdadm: looking in container /dev/md/ddf0
mdadm: member /md127/0 in /dev/md/ddf0 is already assembled
mdadm: Cannot assemble mbr metadata on /dev/sdf
mdadm: Cannot assemble mbr metadata on /dev/md/MegaSR2
mdadm: Cannot assemble mbr metadata on /dev/md/MegaSR1
mdadm: Cannot assemble mbr metadata on /dev/md/MegaSR
mdadm: cannot open device /dev/sr0: No medium found
mdadm: /dev/sdd has wrong uuid.
mdadm: /dev/sdc has wrong uuid.
mdadm: /dev/sdb has wrong uuid.
mdadm: /dev/sda has wrong uuid.
mdadm: /dev/sde has wrong uuid.



On Tue, Nov 19, 2013 at 12:38 PM, Martin Wilck <mwilck@arcor.de> wrote:
> The question I have is why is there IMSM meta data on sdc2 at all?
> IMSM metadata sit at the end of a block device. So I figure that sdc2 is
> the last partition, and by some wird circumstance it's so large that it
> includes the last sectors of the physical disk where the metadata
> resides. That would be a bad idea, a dd if=/dev/zero of=/dev/sdc2 would
> wipe not only the partition but also the RAID meta data.
>
> Martin
>

Re: MDADM 3.3 broken?

[David F.]

I believe because the partitions take up the entire provided md device
- so if you don't skew the size of the device to protect the meta
data, the partition could go to the end of the drive.

On Tue, Nov 19, 2013 at 12:38 PM, Martin Wilck <mwilck@arcor.de> wrote:
> The question I have is why is there IMSM meta data on sdc2 at all?
> IMSM metadata sit at the end of a block device. So I figure that sdc2 is
> the last partition, and by some wird circumstance it's so large that it
> includes the last sectors of the physical disk where the metadata
> resides. That would be a bad idea, a dd if=/dev/zero of=/dev/sdc2 would
> wipe not only the partition but also the RAID meta data.
>
> Martin
>

Re: MDADM 3.3 broken?

[Martin Wilck]

On 11/19/2013 01:01 AM, NeilBrown wrote:
> On Mon, 18 Nov 2013 15:13:58 -0800 "David F." <df7729@gmail.com> wrote:
> 
> 
>> output of 'mdadm --assemble --scan --no-degraded -v' (mdadm 3.2.6):
> ...
>> mdadm: no RAID superblock on /dev/sdc2
> 
> 
> 
>>
>> output1 of 'mdadm --assemble --scan --no-degraded -v' (mdadm 3.3 -
>> note using /dev/sdc2, not /dev/sdc):
> .....
>> mdadm: /dev/sdc2 is identified as a member of /dev/md/imsm0, slot -1.
> 
> So there is the problem.  mdadm 3.2.6 sees no RAID superblock on sdc2, while
> mdadm 3.3 does (but should not).
> 
> 
> However that code hasn't changed!
> 
> load_super_imsm() still starts with:
> 
> 
> 	if (test_partition(fd))
> 		/* IMSM not allowed on partitions */
> 		return 1;

Well not quite - you changed that code in commit b31df436 "intel,ddf:
don't require partitions when ignore_hw_compat is set". Maybe there's
something wrong with that ignore_hw_compat logic?

In the strace I don't see indication of test_partition having been
called, that's another hint in that direction.

Martin

> 
> 
> and test_partition hasn't changed since it was written in April 2010 for
> mdadm 3.1.3.
> 
> So I'm quite perplexed.
> 
> Is your mdadm-3.3 compiled from source or provided by a distro?
> 
> Can you run the "mdadm --assemble" under strace and post the result?
> 
>   strace -o /tmp/some-file mdadm --assemble --scan ......
> 
> Thanks,
> NeilBrown

Re: MDADM 3.3 broken?

[David F.]

and don't forget it's not 100% of the time, it's about 50% of the time
3.3 gives a problem so not sure if it's an uninitialized variable or
something like that?   3.2.6 works 100% of the time.  And again on
RAID0 - RAID1 haven't seen a problem.

On Tue, Nov 19, 2013 at 11:45 AM, Martin Wilck <mwilck@arcor.de> wrote:
> On 11/19/2013 01:01 AM, NeilBrown wrote:
>> On Mon, 18 Nov 2013 15:13:58 -0800 "David F." <df7729@gmail.com> wrote:
>>
>>
>>> output of 'mdadm --assemble --scan --no-degraded -v' (mdadm 3.2.6):
>> ...
>>> mdadm: no RAID superblock on /dev/sdc2
>>
>>
>>
>>>
>>> output1 of 'mdadm --assemble --scan --no-degraded -v' (mdadm 3.3 -
>>> note using /dev/sdc2, not /dev/sdc):
>> .....
>>> mdadm: /dev/sdc2 is identified as a member of /dev/md/imsm0, slot -1.
>>
>> So there is the problem.  mdadm 3.2.6 sees no RAID superblock on sdc2, while
>> mdadm 3.3 does (but should not).
>>
>>
>> However that code hasn't changed!
>>
>> load_super_imsm() still starts with:
>>
>>
>>       if (test_partition(fd))
>>               /* IMSM not allowed on partitions */
>>               return 1;
>
> Well not quite - you changed that code in commit b31df436 "intel,ddf:
> don't require partitions when ignore_hw_compat is set". Maybe there's
> something wrong with that ignore_hw_compat logic?
>
> In the strace I don't see indication of test_partition having been
> called, that's another hint in that direction.
>
> Martin
>
>>
>>
>> and test_partition hasn't changed since it was written in April 2010 for
>> mdadm 3.1.3.
>>
>> So I'm quite perplexed.
>>
>> Is your mdadm-3.3 compiled from source or provided by a distro?
>>
>> Can you run the "mdadm --assemble" under strace and post the result?
>>
>>   strace -o /tmp/some-file mdadm --assemble --scan ......
>>
>> Thanks,
>> NeilBrown
>

Re: MDADM 3.3 broken?

[NeilBrown]

[-- Attachment #1: Type: text/plain, Size: 3118 bytes --]

On Tue, 19 Nov 2013 20:45:47 +0100 Martin Wilck <mwilck@arcor.de> wrote:

> On 11/19/2013 01:01 AM, NeilBrown wrote:
> > On Mon, 18 Nov 2013 15:13:58 -0800 "David F." <df7729@gmail.com> wrote:
> > 
> > 
> >> output of 'mdadm --assemble --scan --no-degraded -v' (mdadm 3.2.6):
> > ...
> >> mdadm: no RAID superblock on /dev/sdc2
> > 
> > 
> > 
> >>
> >> output1 of 'mdadm --assemble --scan --no-degraded -v' (mdadm 3.3 -
> >> note using /dev/sdc2, not /dev/sdc):
> > .....
> >> mdadm: /dev/sdc2 is identified as a member of /dev/md/imsm0, slot -1.
> > 
> > So there is the problem.  mdadm 3.2.6 sees no RAID superblock on sdc2, while
> > mdadm 3.3 does (but should not).
> > 
> > 
> > However that code hasn't changed!
> > 
> > load_super_imsm() still starts with:
> > 
> > 
> > 	if (test_partition(fd))
> > 		/* IMSM not allowed on partitions */
> > 		return 1;
> 
> Well not quite - you changed that code in commit b31df436 "intel,ddf:
> don't require partitions when ignore_hw_compat is set". Maybe there's
> something wrong with that ignore_hw_compat logic?
> 
> In the strace I don't see indication of test_partition having been
> called, that's another hint in that direction.
> 

Yes... I seems I was accidentally looking at an old version of mdadm.

I've just committed the following patch which should fix the problem.

(git clone git://neil.brown.name/mdadm/ ; cd mdadm ; make;make install)

Thanks,
NeilBrown

From 357ac1067835d1cdd5f80acc28501db0ffc64957 Mon Sep 17 00:00:00 2001
From: NeilBrown <neilb@suse.de>
Date: Wed, 20 Nov 2013 10:49:14 +1100
Subject: [PATCH] IMSM metadata really should be ignored when found on
 partitions.

commit b31df43682216d1c65813eae49ebdd8253db8907
changed load_super_imsm to not insist on finding a partition if
ignore_hw_compat was set.
Unfortunately this is set for '--assemble' so arrays could get
assembled badly.

The comment says this was to allow e.g. --examine of image files.
A better fixes for this is to change test_partitions to not report
a regular file as being a partition.
The errors from the BLKPG ioctl are:

 ENOTTY : not a block device.
 EINVAL : not a whole device (probably a partition)
 ENXIO  : partition doesn't exist (so not a partition)

Reported-by: "David F." <df7729@gmail.com>
Signed-off-by: NeilBrown <neilb@suse.de>

diff --git a/super-intel.c b/super-intel.c
index 7b2406866493..c103ffdd2dd8 100644
--- a/super-intel.c
+++ b/super-intel.c
@@ -4423,7 +4423,7 @@ static int load_super_imsm(struct supertype *st, int fd, char *devname)
 	struct intel_super *super;
 	int rv;
 
-	if (!st->ignore_hw_compat && test_partition(fd))
+	if (test_partition(fd))
 		/* IMSM not allowed on partitions */
 		return 1;
 
diff --git a/util.c b/util.c
index 5f95f1f97c02..b29a3ee7ce47 100644
--- a/util.c
+++ b/util.c
@@ -307,7 +307,7 @@ int test_partition(int fd)
 	if (ioctl(fd, BLKPG, &a) == 0)
 		/* Very unlikely, but not a partition */
 		return 0;
-	if (errno == ENXIO)
+	if (errno == ENXIO || errno == ENOTTY)
 		/* not a partition */
 		return 0;
 

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 828 bytes --]

Re: MDADM 3.3 broken?

[David F.]

Okay, I'll have them build it and try.

Another note on the last reason meta data may be in the partition -
that system has the raid changed from 0 to 1 all the time so perhaps
the old meta data at the end of drive 0 from RAID1 ends up in the
middle of the partition when it's RAID0. ?

On Tue, Nov 19, 2013 at 3:51 PM, NeilBrown <neilb@suse.de> wrote:
> On Tue, 19 Nov 2013 20:45:47 +0100 Martin Wilck <mwilck@arcor.de> wrote:
>
>> On 11/19/2013 01:01 AM, NeilBrown wrote:
>> > On Mon, 18 Nov 2013 15:13:58 -0800 "David F." <df7729@gmail.com> wrote:
>> >
>> >
>> >> output of 'mdadm --assemble --scan --no-degraded -v' (mdadm 3.2.6):
>> > ...
>> >> mdadm: no RAID superblock on /dev/sdc2
>> >
>> >
>> >
>> >>
>> >> output1 of 'mdadm --assemble --scan --no-degraded -v' (mdadm 3.3 -
>> >> note using /dev/sdc2, not /dev/sdc):
>> > .....
>> >> mdadm: /dev/sdc2 is identified as a member of /dev/md/imsm0, slot -1.
>> >
>> > So there is the problem.  mdadm 3.2.6 sees no RAID superblock on sdc2, while
>> > mdadm 3.3 does (but should not).
>> >
>> >
>> > However that code hasn't changed!
>> >
>> > load_super_imsm() still starts with:
>> >
>> >
>> >     if (test_partition(fd))
>> >             /* IMSM not allowed on partitions */
>> >             return 1;
>>
>> Well not quite - you changed that code in commit b31df436 "intel,ddf:
>> don't require partitions when ignore_hw_compat is set". Maybe there's
>> something wrong with that ignore_hw_compat logic?
>>
>> In the strace I don't see indication of test_partition having been
>> called, that's another hint in that direction.
>>
>
> Yes... I seems I was accidentally looking at an old version of mdadm.
>
> I've just committed the following patch which should fix the problem.
>
> (git clone git://neil.brown.name/mdadm/ ; cd mdadm ; make;make install)
>
> Thanks,
> NeilBrown
>
> From 357ac1067835d1cdd5f80acc28501db0ffc64957 Mon Sep 17 00:00:00 2001
> From: NeilBrown <neilb@suse.de>
> Date: Wed, 20 Nov 2013 10:49:14 +1100
> Subject: [PATCH] IMSM metadata really should be ignored when found on
>  partitions.
>
> commit b31df43682216d1c65813eae49ebdd8253db8907
> changed load_super_imsm to not insist on finding a partition if
> ignore_hw_compat was set.
> Unfortunately this is set for '--assemble' so arrays could get
> assembled badly.
>
> The comment says this was to allow e.g. --examine of image files.
> A better fixes for this is to change test_partitions to not report
> a regular file as being a partition.
> The errors from the BLKPG ioctl are:
>
>  ENOTTY : not a block device.
>  EINVAL : not a whole device (probably a partition)
>  ENXIO  : partition doesn't exist (so not a partition)
>
> Reported-by: "David F." <df7729@gmail.com>
> Signed-off-by: NeilBrown <neilb@suse.de>
>
> diff --git a/super-intel.c b/super-intel.c
> index 7b2406866493..c103ffdd2dd8 100644
> --- a/super-intel.c
> +++ b/super-intel.c
> @@ -4423,7 +4423,7 @@ static int load_super_imsm(struct supertype *st, int fd, char *devname)
>         struct intel_super *super;
>         int rv;
>
> -       if (!st->ignore_hw_compat && test_partition(fd))
> +       if (test_partition(fd))
>                 /* IMSM not allowed on partitions */
>                 return 1;
>
> diff --git a/util.c b/util.c
> index 5f95f1f97c02..b29a3ee7ce47 100644
> --- a/util.c
> +++ b/util.c
> @@ -307,7 +307,7 @@ int test_partition(int fd)
>         if (ioctl(fd, BLKPG, &a) == 0)
>                 /* Very unlikely, but not a partition */
>                 return 0;
> -       if (errno == ENXIO)
> +       if (errno == ENXIO || errno == ENOTTY)
>                 /* not a partition */
>                 return 0;
>

Re: MDADM 3.3 broken?

[David F.]

FWIW, we confirmed that it appears to be the old RAID1 data it was
finding - when the partition is below half the size it didn't find it
in the partition, when resized just beyond half the size (of the RAID0
md drive), it then started finding it in the partition.

Also, was Cisco's information useful for why their cisco server (using
lsi)  RAID5 won't assembly?  Do you think the patch deals with that as
well?

On Tue, Nov 19, 2013 at 4:22 PM, David F. <df7729@gmail.com> wrote:
> Okay, I'll have them build it and try.
>
> Another note on the last reason meta data may be in the partition -
> that system has the raid changed from 0 to 1 all the time so perhaps
> the old meta data at the end of drive 0 from RAID1 ends up in the
> middle of the partition when it's RAID0. ?
>
> On Tue, Nov 19, 2013 at 3:51 PM, NeilBrown <neilb@suse.de> wrote:
>> On Tue, 19 Nov 2013 20:45:47 +0100 Martin Wilck <mwilck@arcor.de> wrote:
>>
>>> On 11/19/2013 01:01 AM, NeilBrown wrote:
>>> > On Mon, 18 Nov 2013 15:13:58 -0800 "David F." <df7729@gmail.com> wrote:
>>> >
>>> >
>>> >> output of 'mdadm --assemble --scan --no-degraded -v' (mdadm 3.2.6):
>>> > ...
>>> >> mdadm: no RAID superblock on /dev/sdc2
>>> >
>>> >
>>> >
>>> >>
>>> >> output1 of 'mdadm --assemble --scan --no-degraded -v' (mdadm 3.3 -
>>> >> note using /dev/sdc2, not /dev/sdc):
>>> > .....
>>> >> mdadm: /dev/sdc2 is identified as a member of /dev/md/imsm0, slot -1.
>>> >
>>> > So there is the problem.  mdadm 3.2.6 sees no RAID superblock on sdc2, while
>>> > mdadm 3.3 does (but should not).
>>> >
>>> >
>>> > However that code hasn't changed!
>>> >
>>> > load_super_imsm() still starts with:
>>> >
>>> >
>>> >     if (test_partition(fd))
>>> >             /* IMSM not allowed on partitions */
>>> >             return 1;
>>>
>>> Well not quite - you changed that code in commit b31df436 "intel,ddf:
>>> don't require partitions when ignore_hw_compat is set". Maybe there's
>>> something wrong with that ignore_hw_compat logic?
>>>
>>> In the strace I don't see indication of test_partition having been
>>> called, that's another hint in that direction.
>>>
>>
>> Yes... I seems I was accidentally looking at an old version of mdadm.
>>
>> I've just committed the following patch which should fix the problem.
>>
>> (git clone git://neil.brown.name/mdadm/ ; cd mdadm ; make;make install)
>>
>> Thanks,
>> NeilBrown
>>
>> From 357ac1067835d1cdd5f80acc28501db0ffc64957 Mon Sep 17 00:00:00 2001
>> From: NeilBrown <neilb@suse.de>
>> Date: Wed, 20 Nov 2013 10:49:14 +1100
>> Subject: [PATCH] IMSM metadata really should be ignored when found on
>>  partitions.
>>
>> commit b31df43682216d1c65813eae49ebdd8253db8907
>> changed load_super_imsm to not insist on finding a partition if
>> ignore_hw_compat was set.
>> Unfortunately this is set for '--assemble' so arrays could get
>> assembled badly.
>>
>> The comment says this was to allow e.g. --examine of image files.
>> A better fixes for this is to change test_partitions to not report
>> a regular file as being a partition.
>> The errors from the BLKPG ioctl are:
>>
>>  ENOTTY : not a block device.
>>  EINVAL : not a whole device (probably a partition)
>>  ENXIO  : partition doesn't exist (so not a partition)
>>
>> Reported-by: "David F." <df7729@gmail.com>
>> Signed-off-by: NeilBrown <neilb@suse.de>
>>
>> diff --git a/super-intel.c b/super-intel.c
>> index 7b2406866493..c103ffdd2dd8 100644
>> --- a/super-intel.c
>> +++ b/super-intel.c
>> @@ -4423,7 +4423,7 @@ static int load_super_imsm(struct supertype *st, int fd, char *devname)
>>         struct intel_super *super;
>>         int rv;
>>
>> -       if (!st->ignore_hw_compat && test_partition(fd))
>> +       if (test_partition(fd))
>>                 /* IMSM not allowed on partitions */
>>                 return 1;
>>
>> diff --git a/util.c b/util.c
>> index 5f95f1f97c02..b29a3ee7ce47 100644
>> --- a/util.c
>> +++ b/util.c
>> @@ -307,7 +307,7 @@ int test_partition(int fd)
>>         if (ioctl(fd, BLKPG, &a) == 0)
>>                 /* Very unlikely, but not a partition */
>>                 return 0;
>> -       if (errno == ENXIO)
>> +       if (errno == ENXIO || errno == ENOTTY)
>>                 /* not a partition */
>>                 return 0;
>>

Re: MDADM 3.3 broken?

[NeilBrown]

[-- Attachment #1: Type: text/plain, Size: 916 bytes --]

On Tue, 19 Nov 2013 16:35:44 -0800 "David F." <df7729@gmail.com> wrote:

> FWIW, we confirmed that it appears to be the old RAID1 data it was
> finding - when the partition is below half the size it didn't find it
> in the partition, when resized just beyond half the size (of the RAID0
> md drive), it then started finding it in the partition.
> 
> Also, was Cisco's information useful for why their cisco server (using
> lsi)  RAID5 won't assembly?  Do you think the patch deals with that as
> well?
> 
>

You did say something about Cisco in the original post but it wasn't at all
clear what you were asking or what the context was.

Could you please spell it all out.  You said something about dmraid reporting
something.  Showing the output of dmraid in that case wouldn't hurt.
And show the output of "mdadm --examine /dev/device" for any relevant device
is often a good idea.

NeilBrown

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 828 bytes --]

Re: MDADM 3.3 broken?

[David F.]

Hi,

Yes, it's in this thread already - but I'll repost here:

For cisco's server:

> mdadm --examine --scan
ARRAY metadata=ddf UUID=7ab254d0:fae71048:
404edde9:750a8a05
ARRAY container=7ab254d0:fae71048:404edde9:750a8a05 member=0
UUID=5337ab03:86ca2abc:d42bfbc8:23626c78

> mdadm --assemble --scan --no-degraded -v
mdadm: looking for devices for further assembly
mdadm: /dev/md/ddf0 is a container, but we are looking for components
mdadm: no RAID superblock on /dev/sdf
mdadm: no RAID superblock on /dev/md/MegaSR2
mdadm: no RAID superblock on /dev/md/MegaSR1
mdadm: no RAID superblock on /dev/md/MegaSR
mdadm: cannot open device /dev/sr0: No medium found
mdadm: /dev/sdd is busy - skipping
mdadm: /dev/sdc is busy - skipping
mdadm: /dev/sdb is busy - skipping
mdadm: /dev/sda is busy - skipping
mdadm: /dev/sde is busy - skipping
mdadm: looking for devices for further assembly
mdadm: looking in container /dev/md/ddf0
mdadm: member /md127/0 in /dev/md/ddf0 is already assembled
mdadm: Cannot assemble mbr metadata on /dev/sdf
mdadm: Cannot assemble mbr metadata on /dev/md/MegaSR2
mdadm: Cannot assemble mbr metadata on /dev/md/MegaSR1
mdadm: Cannot assemble mbr metadata on /dev/md/MegaSR
mdadm: cannot open device /dev/sr0: No medium found
mdadm: /dev/sdd has wrong uuid.
mdadm: /dev/sdc has wrong uuid.
mdadm: /dev/sdb has wrong uuid.
mdadm: /dev/sda has wrong uuid.
mdadm: /dev/sde has wrong uuid.




On Tue, Nov 19, 2013 at 4:48 PM, NeilBrown <neilb@suse.de> wrote:
> On Tue, 19 Nov 2013 16:35:44 -0800 "David F." <df7729@gmail.com> wrote:
>>
>> Also, was Cisco's information useful for why their cisco server (using
>> lsi)  RAID5 won't assembly?  Do you think the patch deals with that as
>> well?
>>
>>
>
> You did say something about Cisco in the original post but it wasn't at all
> clear what you were asking or what the context was.
>
> Could you please spell it all out.  You said something about dmraid reporting
> something.  Showing the output of dmraid in that case wouldn't hurt.
> And show the output of "mdadm --examine /dev/device" for any relevant device
> is often a good idea.
>
> NeilBrown

Re: MDADM 3.3 broken?

[David F.]

And here's some more data I have for them (again they have RAID5):

Storage devices (non-RAID) detected by Linux based on /sys/block:

Optical drives (srx):
  sr0: hp       DVDRAM GT30L     (fw rev = mP06)
Removable drives (sdx):
  sdb: 7453 MiB          Patriot Memory
Fixed hard drives (sdx):
  sdc: 136 GiB TOSHIBA  MK1401GRRB
  sdd: 136 GiB TOSHIBA  MK1401GRRB
  sde: 136 GiB TOSHIBA  MK1401GRRB
  sdf: 136 GiB TOSHIBA  MK1401GRRB
  sdg: 136 GiB TOSHIBA  MK1401GRRB

Contents of /sys/block:
md127  ram0   ram1   sdb    sdc    sdd    sde    sdf    sdg    sr0

Contents of /dev:
block               port                tty1                tty47
bsg                 ppp                 tty10               tty48
bus                 psaux               tty11               tty49
cdrom               ptmx                tty12               tty5
cdrom1              ptp0                tty13               tty50
cdrw                ptp1                tty14               tty51
cdrw1               pts                 tty15               tty52
char                ram0                tty16               tty53
console             ram1                tty17               tty54
core                random              tty18               tty55
cpu_dma_latency     rfkill              tty19               tty56
disk                rtc                 tty2                tty57
dvd                 rtc0                tty20               tty58
dvd1                sda                 tty21               tty59
dvdrw               sdb                 tty22               tty6
dvdrw1              sdb1                tty23               tty60
fd                  sdc                 tty24               tty61
full                sdc1                tty25               tty62
fuse                sdc2                tty26               tty63
hidraw0             sdd                 tty27               tty7
hidraw1             sde                 tty28               tty8
hidraw2             sdf                 tty29               tty9
hidraw3             sdf1                tty3                ttyS0
hidraw4             sdf2                tty30               ttyS1
hidraw5             sdg                 tty31               ttyS2
hpet                sg0                 tty32               ttyS3
input               sg1                 tty33               urandom
kmsg                sg2                 tty34               vcs
log                 sg3                 tty35               vcs1
loop-control        sg4                 tty36               vcs2
loop0               sg5                 tty37               vcs3
loop1               sg6                 tty38               vcs4
mapper              sg7                 tty39               vcs5
mcelog              shm                 tty4                vcsa
md                  sr0                 tty40               vcsa1
md127               stderr              tty41               vcsa2
mem                 stdin               tty42               vcsa3
net                 stdout              tty43               vcsa4
network_latency     synth               tty44               vcsa5
network_throughput  tty                 tty45               vga_arbiter
null                tty0                tty46               zero

Contents of /proc/partitions:
major minor  #blocks  name

   8       32  143638992 sdc
   8       33     102400 sdc1
   8       34  143535568 sdc2
   8       48  143638992 sdd
   8       64  143638992 sde
   8       80  143638992 sdf
   8       81     102400 sdf1
   8       82  143535568 sdf2
   8       96  143638992 sdg
  11        0      48160 sr0
   8       16    7632892 sdb

Disk /dev/sdc: 147.0 GB, 147086327808 bytes
255 heads, 63 sectors/track, 17882 cylinders, total 287277984 sectors
Units = sectors of 1 * 512 = 512 bytes

   Device Boot      Start         End      Blocks  Id System
/dev/sdc1   *        2048      206847      102400   7 HPFS/NTFS
Partition 1 does not end on cylinder boundary
/dev/sdc2          206848   855463935   427628544   7 HPFS/NTFS

Disk /dev/sdd: 147.0 GB, 147086327808 bytes
255 heads, 63 sectors/track, 17882 cylinders, total 287277984 sectors
Units = sectors of 1 * 512 = 512 bytes

Disk /dev/sdd doesn't contain a valid partition table

Disk /dev/sde: 147.0 GB, 147086327808 bytes
255 heads, 63 sectors/track, 17882 cylinders, total 287277984 sectors
Units = sectors of 1 * 512 = 512 bytes

Disk /dev/sde doesn't contain a valid partition table

Disk /dev/sdf: 147.0 GB, 147086327808 bytes
255 heads, 63 sectors/track, 17882 cylinders, total 287277984 sectors
Units = sectors of 1 * 512 = 512 bytes

   Device Boot      Start         End      Blocks  Id System
/dev/sdf1   *        2048      206847      102400   7 HPFS/NTFS
Partition 1 does not end on cylinder boundary
/dev/sdf2          206848   855463935   427628544   7 HPFS/NTFS

Disk /dev/sdg: 147.0 GB, 147086327808 bytes
255 heads, 63 sectors/track, 17882 cylinders, total 287277984 sectors
Units = sectors of 1 * 512 = 512 bytes

Disk /dev/sdg doesn't contain a valid partition table

Disk /dev/sdb: 7816 MB, 7816081408 bytes
241 heads, 62 sectors/track, 1021 cylinders, total 15265784 sectors
Units = sectors of 1 * 512 = 512 bytes

   Device Boot      Start         End      Blocks  Id System
/dev/sdb1   ?   778135908  1919645538   570754815+ 72 Unknown
Partition 1 has different physical/logical beginnings (non-Linux?):
     phys=(357, 116, 40) logical=(52077, 22, 11)
Partition 1 has different physical/logical endings:
     phys=(357, 32, 45) logical=(128473, 31, 51)
Partition 1 does not end on cylinder boundary
/dev/sdb2   ?   168689522  2104717761   968014120  65 Unknown
Partition 2 has different physical/logical beginnings (non-Linux?):
     phys=(288, 115, 43) logical=(11289, 149, 47)
Partition 2 has different physical/logical endings:
     phys=(367, 114, 50) logical=(140859, 41, 42)
Partition 2 does not end on cylinder boundary
/dev/sdb3   ?  1869881465  3805909656   968014096  79 Unknown
Partition 3 has different physical/logical beginnings (non-Linux?):
     phys=(366, 32, 33) logical=(125142, 156, 30)
Partition 3 has different physical/logical endings:
     phys=(357, 32, 43) logical=(254712, 47, 39)
Partition 3 does not end on cylinder boundary
/dev/sdb4   ?  2885681152  2885736650       27749+  d Unknown
Partition 4 has different physical/logical beginnings (non-Linux?):
     phys=(372, 97, 50) logical=(193125, 119, 25)
Partition 4 has different physical/logical endings:
     phys=(0, 10, 0) logical=(193129, 50, 33)
Partition 4 does not end on cylinder boundary

Partition table entries are not in disk order

Contents of /proc/mdstat (Linux software RAID status):
Personalities : [linear] [raid0] [raid1] [raid10] [raid6] [raid5]
[raid4] [multipath]
md127 : inactive sdg[0](S)
      1061328 blocks super external:ddf

unused devices: <none>

Contents of /run/mdadm/map (Linux software RAID arrays):
md127 ddf 7ab254d0:fae71048:404edde9:750a8a05 /dev/md/ddf0

Contents of /etc/mdadm/mdadm.conf (Linux software RAID config file):
# mdadm.conf
#
# Please refer to mdadm.conf(5) for information about this file.
#

# by default (built-in), scan all partitions (/proc/partitions) and all
# containers for MD superblocks. alternatively, specify devices to scan, using
# wildcards if desired.
DEVICE partitions containers

# automatically tag new arrays as belonging to the local system
HOMEHOST <system>

ARRAY metadata=ddf UUID=7ab254d0:fae71048:404edde9:750a8a05
ARRAY container=7ab254d0:fae71048:404edde9:750a8a05 member=0
UUID=45b3ab73:5c998afc:01bbf815:12660984

Long listing of /dev/md directory and /dev/md* device files (Linux
software RAID devices):
brw-rw---T    1 root     disk        9, 127 Nov 18 17:12 /dev/md127

/dev/md:
lrwxrwxrwx    1 root     root             8 Nov 18 17:12 ddf0 -> ../md127

Contents of /dev/mapper directory:
crw------T    1 root     root       10, 236 Nov 18 17:12 control

startraid mode = null (default)

Contents of dmraid-boot.txt file (from activate command on boot):
NOTICE: checking format identifier asr
NOTICE: checking format identifier hpt37x
NOTICE: checking format identifier hpt45x
NOTICE: checking format identifier jmicron
NOTICE: checking format identifier lsi
NOTICE: checking format identifier nvidia
NOTICE: checking format identifier pdc
NOTICE: checking format identifier sil
NOTICE: checking format identifier via
NOTICE: checking format identifier dos
WARN: locking /var/lock/dmraid/.lock
NOTICE: skipping removable device /dev/sda
NOTICE: /dev/sdg: asr     discovering
NOTICE: /dev/sdg: hpt37x  discovering
NOTICE: /dev/sdg: hpt45x  discovering
NOTICE: /dev/sdg: jmicron discovering
NOTICE: /dev/sdg: lsi     discovering
NOTICE: /dev/sdg: nvidia  discovering
NOTICE: /dev/sdg: pdc     discovering
NOTICE: /dev/sdg: sil     discovering
NOTICE: /dev/sdg: via     discovering
NOTICE: /dev/sdf: asr     discovering
NOTICE: /dev/sdf: hpt37x  discovering
NOTICE: /dev/sdf: hpt45x  discovering
NOTICE: /dev/sdf: jmicron discovering
NOTICE: /dev/sdf: lsi     discovering
NOTICE: /dev/sdf: nvidia  discovering
NOTICE: /dev/sdf: pdc     discovering
NOTICE: /dev/sdf: sil     discovering
NOTICE: /dev/sdf: via     discovering
NOTICE: /dev/sde: asr     discovering
NOTICE: /dev/sde: hpt37x  discovering
NOTICE: /dev/sde: hpt45x  discovering
NOTICE: /dev/sde: jmicron discovering
NOTICE: /dev/sde: lsi     discovering
NOTICE: /dev/sde: nvidia  discovering
NOTICE: /dev/sde: pdc     discovering
NOTICE: /dev/sde: sil     discovering
NOTICE: /dev/sde: via     discovering
NOTICE: /dev/sdd: asr     discovering
NOTICE: /dev/sdd: hpt37x  discovering
NOTICE: /dev/sdd: hpt45x  discovering
NOTICE: /dev/sdd: jmicron discovering
NOTICE: /dev/sdd: lsi     discovering
NOTICE: /dev/sdd: nvidia  discovering
NOTICE: /dev/sdd: pdc     discovering
NOTICE: /dev/sdd: sil     discovering
NOTICE: /dev/sdd: via     discovering
NOTICE: /dev/sdc: asr     discovering
NOTICE: /dev/sdc: hpt37x  discovering
NOTICE: /dev/sdc: hpt45x  discovering
NOTICE: /dev/sdc: jmicron discovering
NOTICE: /dev/sdc: lsi     discovering
NOTICE: /dev/sdc: nvidia  discovering
NOTICE: /dev/sdc: pdc     discovering
NOTICE: /dev/sdc: sil     discovering
NOTICE: /dev/sdc: via     discovering
NOTICE: /dev/sdb: asr     discovering
NOTICE: /dev/sdb: hpt37x  discovering
NOTICE: /dev/sdb: hpt45x  discovering
NOTICE: /dev/sdb: jmicron discovering
NOTICE: /dev/sdb: lsi     discovering
NOTICE: /dev/sdb: nvidia  discovering
NOTICE: /dev/sdb: pdc     discovering
NOTICE: /dev/sdb: sil     discovering
NOTICE: /dev/sdb: via     discovering
no raid disks with format:
"asr,hpt37x,hpt45x,jmicron,lsi,nvidia,pdc,sil,via,dos"
WARN: unlocking /var/lock/dmraid/.lock

Output of dmraid --raid_devices command:
/dev/sdg: ddf1, ".ddf1_disks", GROUP, unknown, 285155328 sectors, data@ 0
/dev/sdf: ddf1, ".ddf1_disks", GROUP, ok, 285155328 sectors, data@ 0
/dev/sde: ddf1, ".ddf1_disks", GROUP, ok, 285155328 sectors, data@ 0
/dev/sdd: ddf1, ".ddf1_disks", GROUP, ok, 285155328 sectors, data@ 0
/dev/sdc: ddf1, ".ddf1_disks", GROUP, ok, 285155328 sectors, data@ 0

Output of dmraid -s -s command:
*** Group superset .ddf1_disks
--> Subset
name   : ddf1_MegaSR   R5 #0
size   : 855465984
stride : 128
type   : raid5_ls
status : ok
subsets: 0
devs   : 4
spares : 0

Output of blkid command:
/dev/sdd: UUID="LSI     M-^@M-^F^]h^Q7" TYPE="ddf_raid_member"
/dev/sde: UUID="LSI     M-^@M-^F^]h^Q7" TYPE="ddf_raid_member"
/dev/sr0: LABEL="iflnet" TYPE="iso9660"
/dev/sdg: UUID="LSI     M-^@M-^F^]h^Q7" TYPE="ddf_raid_member"
/dev/sdb: LABEL="CS SUPPORT" UUID="4CC3-3665" TYPE="vfat"

Output of free command (memory):

             total       used       free     shared    buffers     cached
Mem:       2039280     185776    1853504          0      40996      76512
-/+ buffers/cache:      68268    1971012
Swap:            0          0          0

Output of lsmod command (loaded modules):

Module                  Size  Used by    Not tainted
sr_mod                 10387  0
cdrom                  24577  1 sr_mod
sg                     17182  0
sd_mod                 25922  2
crc_t10dif               996  1 sd_mod
usb_storage            32002  1
hid_generic              665  0
usbhid                 18838  0
hid                    60158  2 hid_generic,usbhid
pcspkr                  1227  0
igb                    98565  0
i2c_algo_bit            3747  1 igb
ptp                     5420  1 igb
pps_core                4544  1 ptp
i2c_i801                7257  0
i2c_core               13114  3 igb,i2c_algo_bit,i2c_i801
ehci_pci                2540  0
isci                   71218  1
libsas                 47529  1 isci
processor              14951  8
thermal_sys            12739  1 processor
scsi_transport_sas     16455  2 isci,libsas
hwmon                    881  2 igb,thermal_sys
button                  3413  0
dm_mod                 50912  0
ehci_hcd               28628  1 ehci_pci
edd                     5144  0

Contents of /sys/module:
8250                i2c_algo_bit        ptp                 speakup_dtlk
acpi                i2c_core            raid1               speakup_dummy
auth_rpcgss         i2c_i801            raid10              speakup_keypc
block               i8042               random              speakup_ltlk
brd                 igb                 rcupdate            speakup_soft
button              input_polldev       rcutree             speakup_spkout
cdrom               isci                rfkill              speakup_txprt
cifs                kernel              scsi_mod            spurious
cpuidle             keyboard            scsi_transport_fc   sr_mod
crc_t10dif          libata              scsi_transport_sas  sunrpc
dm_mod              libsas              sd_mod              tcp_cubic
dns_resolver        lockd               sg                  thermal_sys
edd                 md_mod              speakup             usb_storage
ehci_hcd            mousedev            speakup_acntpc      usbcore
ehci_pci            nfs                 speakup_acntsa      usbhid
firmware_class      pcie_aspm           speakup_apollo      vt
fuse                pcspkr              speakup_audptr      workqueue
hid                 pps_core            speakup_bns         xz_dec
hid_generic         printk              speakup_decext
hwmon               processor           speakup_dectlk

On Tue, Nov 19, 2013 at 5:29 PM, David F. <df7729@gmail.com> wrote:
> Hi,
>
> Yes, it's in this thread already - but I'll repost here:
>
> For cisco's server:
>
>> mdadm --examine --scan
> ARRAY metadata=ddf UUID=7ab254d0:fae71048:
> 404edde9:750a8a05
> ARRAY container=7ab254d0:fae71048:404edde9:750a8a05 member=0
> UUID=5337ab03:86ca2abc:d42bfbc8:23626c78
>
>> mdadm --assemble --scan --no-degraded -v
> mdadm: looking for devices for further assembly
> mdadm: /dev/md/ddf0 is a container, but we are looking for components
> mdadm: no RAID superblock on /dev/sdf
> mdadm: no RAID superblock on /dev/md/MegaSR2
> mdadm: no RAID superblock on /dev/md/MegaSR1
> mdadm: no RAID superblock on /dev/md/MegaSR
> mdadm: cannot open device /dev/sr0: No medium found
> mdadm: /dev/sdd is busy - skipping
> mdadm: /dev/sdc is busy - skipping
> mdadm: /dev/sdb is busy - skipping
> mdadm: /dev/sda is busy - skipping
> mdadm: /dev/sde is busy - skipping
> mdadm: looking for devices for further assembly
> mdadm: looking in container /dev/md/ddf0
> mdadm: member /md127/0 in /dev/md/ddf0 is already assembled
> mdadm: Cannot assemble mbr metadata on /dev/sdf
> mdadm: Cannot assemble mbr metadata on /dev/md/MegaSR2
> mdadm: Cannot assemble mbr metadata on /dev/md/MegaSR1
> mdadm: Cannot assemble mbr metadata on /dev/md/MegaSR
> mdadm: cannot open device /dev/sr0: No medium found
> mdadm: /dev/sdd has wrong uuid.
> mdadm: /dev/sdc has wrong uuid.
> mdadm: /dev/sdb has wrong uuid.
> mdadm: /dev/sda has wrong uuid.
> mdadm: /dev/sde has wrong uuid.
>
>
>
>
> On Tue, Nov 19, 2013 at 4:48 PM, NeilBrown <neilb@suse.de> wrote:
>> On Tue, 19 Nov 2013 16:35:44 -0800 "David F." <df7729@gmail.com> wrote:
>>>
>>> Also, was Cisco's information useful for why their cisco server (using
>>> lsi)  RAID5 won't assembly?  Do you think the patch deals with that as
>>> well?
>>>
>>>
>>
>> You did say something about Cisco in the original post but it wasn't at all
>> clear what you were asking or what the context was.
>>
>> Could you please spell it all out.  You said something about dmraid reporting
>> something.  Showing the output of dmraid in that case wouldn't hurt.
>> And show the output of "mdadm --examine /dev/device" for any relevant device
>> is often a good idea.
>>
>> NeilBrown

Re: MDADM 3.3 broken?

[NeilBrown]

[-- Attachment #1: Type: text/plain, Size: 3094 bytes --]

On Tue, 19 Nov 2013 17:34:29 -0800 "David F." <df7729@gmail.com> wrote:


> Contents of /proc/partitions:
> major minor  #blocks  name
> 
>    8       32  143638992 sdc
>    8       33     102400 sdc1
>    8       34  143535568 sdc2
>    8       48  143638992 sdd
>    8       64  143638992 sde
>    8       80  143638992 sdf
>    8       81     102400 sdf1
>    8       82  143535568 sdf2
>    8       96  143638992 sdg
>   11        0      48160 sr0
>    8       16    7632892 sdb

This seems to suggest that there are no md devices that are active.


> Contents of /proc/mdstat (Linux software RAID status):
> Personalities : [linear] [raid0] [raid1] [raid10] [raid6] [raid5]
> [raid4] [multipath]
> md127 : inactive sdg[0](S)
>       1061328 blocks super external:ddf
> 
> unused devices: <none>

And this confirms it - just md127 which is inactive and is a ddf 'container'.

> Contents of /etc/mdadm/mdadm.conf (Linux software RAID config file):
> # mdadm.conf
> #
> # Please refer to mdadm.conf(5) for information about this file.
> #
> 
> # by default (built-in), scan all partitions (/proc/partitions) and all
> # containers for MD superblocks. alternatively, specify devices to scan, using
> # wildcards if desired.
> DEVICE partitions containers
> 
> # automatically tag new arrays as belonging to the local system
> HOMEHOST <system>
> 
> ARRAY metadata=ddf UUID=7ab254d0:fae71048:404edde9:750a8a05
> ARRAY container=7ab254d0:fae71048:404edde9:750a8a05 member=0
> UUID=45b3ab73:5c998afc:01bbf815:12660984

This shows that mdadm is expecting a container with
      UUID=7ab254d0:fae71048:404edde9:750a8a05
which is presumably found, and a member with
      UUID=45b3ab73:5c998afc:01bbf815:12660984
which it presumably has not found.

> >
> >> mdadm --examine --scan
> > ARRAY metadata=ddf UUID=7ab254d0:fae71048:
> > 404edde9:750a8a05
> > ARRAY container=7ab254d0:fae71048:404edde9:750a8a05 member=0
> > UUID=5337ab03:86ca2abc:d42bfbc8:23626c78

This shows that mdadm found a container with the correct UUID, but the member
array inside the container has the wrong uuid.

Martin: I think one of your recent changes would have changed the member UUID
for some specific arrays because the one that was being created before wasn't
reliably stable.  Could  that apply to David's situation?

David: if you remove the "UUID=" part for the array leaving the
"container=.... member=0" as the identification, does it work?


> >
> >> mdadm --assemble --scan --no-degraded -v
> > mdadm: looking for devices for further assembly
> > mdadm: /dev/md/ddf0 is a container, but we are looking for components
> > mdadm: no RAID superblock on /dev/sdf
> > mdadm: no RAID superblock on /dev/md/MegaSR2
> > mdadm: no RAID superblock on /dev/md/MegaSR1
> > mdadm: no RAID superblock on /dev/md/MegaSR

This seems to suggest that there were 3 md arrays active, where as the
previous data didn't show that.  So it seems the two sets of information are
inconsistent and any conclusions I draw are uncertain.

NeilBrown


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 828 bytes --]

Re: MDADM 3.3 broken?

[David F.]

on the raid0 isw - the patch seems to work.

On Tue, Nov 19, 2013 at 6:30 PM, NeilBrown <neilb@suse.de> wrote:
> On Tue, 19 Nov 2013 17:34:29 -0800 "David F." <df7729@gmail.com> wrote:
>
>
>> Contents of /proc/partitions:
>> major minor  #blocks  name
>>
>>    8       32  143638992 sdc
>>    8       33     102400 sdc1
>>    8       34  143535568 sdc2
>>    8       48  143638992 sdd
>>    8       64  143638992 sde
>>    8       80  143638992 sdf
>>    8       81     102400 sdf1
>>    8       82  143535568 sdf2
>>    8       96  143638992 sdg
>>   11        0      48160 sr0
>>    8       16    7632892 sdb
>
> This seems to suggest that there are no md devices that are active.
>
>
>> Contents of /proc/mdstat (Linux software RAID status):
>> Personalities : [linear] [raid0] [raid1] [raid10] [raid6] [raid5]
>> [raid4] [multipath]
>> md127 : inactive sdg[0](S)
>>       1061328 blocks super external:ddf
>>
>> unused devices: <none>
>
> And this confirms it - just md127 which is inactive and is a ddf 'container'.
>
>> Contents of /etc/mdadm/mdadm.conf (Linux software RAID config file):
>> # mdadm.conf
>> #
>> # Please refer to mdadm.conf(5) for information about this file.
>> #
>>
>> # by default (built-in), scan all partitions (/proc/partitions) and all
>> # containers for MD superblocks. alternatively, specify devices to scan, using
>> # wildcards if desired.
>> DEVICE partitions containers
>>
>> # automatically tag new arrays as belonging to the local system
>> HOMEHOST <system>
>>
>> ARRAY metadata=ddf UUID=7ab254d0:fae71048:404edde9:750a8a05
>> ARRAY container=7ab254d0:fae71048:404edde9:750a8a05 member=0
>> UUID=45b3ab73:5c998afc:01bbf815:12660984
>
> This shows that mdadm is expecting a container with
>       UUID=7ab254d0:fae71048:404edde9:750a8a05
> which is presumably found, and a member with
>       UUID=45b3ab73:5c998afc:01bbf815:12660984
> which it presumably has not found.
>
>> >
>> >> mdadm --examine --scan
>> > ARRAY metadata=ddf UUID=7ab254d0:fae71048:
>> > 404edde9:750a8a05
>> > ARRAY container=7ab254d0:fae71048:404edde9:750a8a05 member=0
>> > UUID=5337ab03:86ca2abc:d42bfbc8:23626c78
>
> This shows that mdadm found a container with the correct UUID, but the member
> array inside the container has the wrong uuid.
>
> Martin: I think one of your recent changes would have changed the member UUID
> for some specific arrays because the one that was being created before wasn't
> reliably stable.  Could  that apply to David's situation?
>
> David: if you remove the "UUID=" part for the array leaving the
> "container=.... member=0" as the identification, does it work?
>
>
>> >
>> >> mdadm --assemble --scan --no-degraded -v
>> > mdadm: looking for devices for further assembly
>> > mdadm: /dev/md/ddf0 is a container, but we are looking for components
>> > mdadm: no RAID superblock on /dev/sdf
>> > mdadm: no RAID superblock on /dev/md/MegaSR2
>> > mdadm: no RAID superblock on /dev/md/MegaSR1
>> > mdadm: no RAID superblock on /dev/md/MegaSR
>
> This seems to suggest that there were 3 md arrays active, where as the
> previous data didn't show that.  So it seems the two sets of information are
> inconsistent and any conclusions I draw are uncertain.
>
> NeilBrown
>

Re: MDADM 3.3 broken?

[David F.]

I have the updated info for that cisco server with the lsi controller
- This was run all at the same time, using the latest patch that
worked on the isw system.

Storage devices (non-RAID) detected by Linux based on /sys/block:

Optical drives (srx):
  sr0: hp       DVDRAM GT30L     (fw rev = mP06)
Removable drives (sdx):
  sdg: 7453 MiB          Patriot Memory
Fixed hard drives (sdx):
  sda: 1863 GiB WD       Ext HDD 1021
  sdb: 136 GiB TOSHIBA  MK1401GRRB
  sdc: 136 GiB TOSHIBA  MK1401GRRB
  sdd: 136 GiB TOSHIBA  MK1401GRRB
  sde: 136 GiB TOSHIBA  MK1401GRRB
  sdf: 136 GiB TOSHIBA  MK1401GRRB

Contents of /sys/block:
md127  ram0   ram1   sda    sdb    sdc    sdd    sde    sdf    sdg    sr0

Contents of /dev:
block               port                tty1                tty47
bsg                 ppp                 tty10               tty48
bus                 psaux               tty11               tty49
cdrom               ptmx                tty12               tty5
cdrom1              ptp0                tty13               tty50
cdrw                ptp1                tty14               tty51
cdrw1               pts                 tty15               tty52
char                ram0                tty16               tty53
console             ram1                tty17               tty54
core                random              tty18               tty55
cpu_dma_latency     rfkill              tty19               tty56
disk                rtc                 tty2                tty57
dvd                 rtc0                tty20               tty58
dvd1                sda                 tty21               tty59
dvdrw               sda1                tty22               tty6
dvdrw1              sdb                 tty23               tty60
fd                  sdb1                tty24               tty61
full                sdb2                tty25               tty62
fuse                sdc                 tty26               tty63
hidraw0             sdd                 tty27               tty7
hidraw1             sde                 tty28               tty8
hidraw2             sde1                tty29               tty9
hidraw3             sde2                tty3                ttyS0
hidraw4             sdf                 tty30               ttyS1
hidraw5             sdg                 tty31               ttyS2
hpet                sg0                 tty32               ttyS3
input               sg1                 tty33               urandom
kmsg                sg2                 tty34               vcs
log                 sg3                 tty35               vcs1
loop-control        sg4                 tty36               vcs2
loop0               sg5                 tty37               vcs3
loop1               sg6                 tty38               vcs4
mapper              sg7                 tty39               vcs5
mcelog              shm                 tty4                vcsa
md                  sr0                 tty40               vcsa1
md127               stderr              tty41               vcsa2
mem                 stdin               tty42               vcsa3
net                 stdout              tty43               vcsa4
network_latency     synth               tty44               vcsa5
network_throughput  tty                 tty45               vga_arbiter
null                tty0                tty46               zero

Contents of /proc/partitions:
major minor  #blocks  name

   8        0 1953512448 sda
   8        1 1953511424 sda1
   8       16  143638992 sdb
   8       17     102400 sdb1
   8       18  143535568 sdb2
   8       32  143638992 sdc
   8       48  143638992 sdd
   8       64  143638992 sde
   8       65     102400 sde1
   8       66  143535568 sde2
   8       80  143638992 sdf
  11        0      48608 sr0
   8       96    7632892 sdg

Disk /dev/sda: 2000.3 GB, 2000396746752 bytes
255 heads, 63 sectors/track, 243201 cylinders, total 3907024896 sectors
Units = sectors of 1 * 512 = 512 bytes

   Device Boot      Start         End      Blocks  Id System
/dev/sda1            2048  3907024895  1953511424   7 HPFS/NTFS

Disk /dev/sdb: 147.0 GB, 147086327808 bytes
255 heads, 63 sectors/track, 17882 cylinders, total 287277984 sectors
Units = sectors of 1 * 512 = 512 bytes

   Device Boot      Start         End      Blocks  Id System
/dev/sdb1   *        2048      206847      102400   7 HPFS/NTFS
Partition 1 does not end on cylinder boundary
/dev/sdb2          206848   855463935   427628544   7 HPFS/NTFS

Disk /dev/sdc: 147.0 GB, 147086327808 bytes
255 heads, 63 sectors/track, 17882 cylinders, total 287277984 sectors
Units = sectors of 1 * 512 = 512 bytes

Disk /dev/sdc doesn't contain a valid partition table

Disk /dev/sdd: 147.0 GB, 147086327808 bytes
255 heads, 63 sectors/track, 17882 cylinders, total 287277984 sectors
Units = sectors of 1 * 512 = 512 bytes

Disk /dev/sdd doesn't contain a valid partition table

Disk /dev/sde: 147.0 GB, 147086327808 bytes
255 heads, 63 sectors/track, 17882 cylinders, total 287277984 sectors
Units = sectors of 1 * 512 = 512 bytes

   Device Boot      Start         End      Blocks  Id System
/dev/sde1   *        2048      206847      102400   7 HPFS/NTFS
Partition 1 does not end on cylinder boundary
/dev/sde2          206848   855463935   427628544   7 HPFS/NTFS

Disk /dev/sdf: 147.0 GB, 147086327808 bytes
255 heads, 63 sectors/track, 17882 cylinders, total 287277984 sectors
Units = sectors of 1 * 512 = 512 bytes

Disk /dev/sdf doesn't contain a valid partition table

Disk /dev/sdg: 7816 MB, 7816081408 bytes
241 heads, 62 sectors/track, 1021 cylinders, total 15265784 sectors
Units = sectors of 1 * 512 = 512 bytes

   Device Boot      Start         End      Blocks  Id System
/dev/sdg1   ?   778135908  1919645538   570754815+ 72 Unknown
Partition 1 has different physical/logical beginnings (non-Linux?):
     phys=(357, 116, 40) logical=(52077, 22, 11)
Partition 1 has different physical/logical endings:
     phys=(357, 32, 45) logical=(128473, 31, 51)
Partition 1 does not end on cylinder boundary
/dev/sdg2   ?   168689522  2104717761   968014120  65 Unknown
Partition 2 has different physical/logical beginnings (non-Linux?):
     phys=(288, 115, 43) logical=(11289, 149, 47)
Partition 2 has different physical/logical endings:
     phys=(367, 114, 50) logical=(140859, 41, 42)
Partition 2 does not end on cylinder boundary
/dev/sdg3   ?  1869881465  3805909656   968014096  79 Unknown
Partition 3 has different physical/logical beginnings (non-Linux?):
     phys=(366, 32, 33) logical=(125142, 156, 30)
Partition 3 has different physical/logical endings:
     phys=(357, 32, 43) logical=(254712, 47, 39)
Partition 3 does not end on cylinder boundary
/dev/sdg4   ?  2885681152  2885736650       27749+  d Unknown
Partition 4 has different physical/logical beginnings (non-Linux?):
     phys=(372, 97, 50) logical=(193125, 119, 25)
Partition 4 has different physical/logical endings:
     phys=(0, 10, 0) logical=(193129, 50, 33)
Partition 4 does not end on cylinder boundary

Partition table entries are not in disk order

Contents of /proc/mdstat (Linux software RAID status):
Personalities : [linear] [raid0] [raid1] [raid10] [raid6] [raid5]
[raid4] [multipath]
md127 : inactive sdf[0](S)
      1061328 blocks super external:ddf

unused devices: <none>

Contents of /run/mdadm/map (Linux software RAID arrays):
md127 ddf 7ab254d0:fae71048:404edde9:750a8a05 /dev/md/ddf0

Contents of /etc/mdadm/mdadm.conf (Linux software RAID config file):
# mdadm.conf
#
# Please refer to mdadm.conf(5) for information about this file.
#

# by default (built-in), scan all partitions (/proc/partitions) and all
# containers for MD superblocks. alternatively, specify devices to scan, using
# wildcards if desired.
DEVICE partitions containers

# automatically tag new arrays as belonging to the local system
HOMEHOST <system>

ARRAY metadata=ddf UUID=7ab254d0:fae71048:404edde9:750a8a05
ARRAY /dev/md/MegaSR container=7ab254d0:fae71048:404edde9:750a8a05
member=0 UUID=9515a157:e69d2ad3:8427131f:92b29c7a

Long listing of /dev/md directory and /dev/md* device files (Linux
software RAID devices):
brw-rw---T    1 root     disk        9, 127 Nov 20 15:06 /dev/md127

/dev/md:
lrwxrwxrwx    1 root     root             8 Nov 20 15:06 ddf0 -> ../md127

Contents of /tbu/utility/mdadm.txt (mdadm troubleshooting data
captured when 'start-md' is executed):
mdadm - v3.3-32-g357ac10 - 20th November 2013
Output of 'mdadm --examine --scan'
ARRAY metadata=ddf UUID=7ab254d0:fae71048:404edde9:750a8a05
ARRAY /dev/md/MegaSR container=7ab254d0:fae71048:404edde9:750a8a05
member=0 UUID=9515a157:e69d2ad3:8427131f:92b29c7a
Output of 'mdadm --assemble --scan --no-degraded -v'
mdadm: looking for devices for further assembly
mdadm: no RAID superblock on /dev/sr0
mdadm: no RAID superblock on /dev/sde2
mdadm: no RAID superblock on /dev/sde1
mdadm: no RAID superblock on /dev/sdb2
mdadm: no RAID superblock on /dev/sdb1
mdadm: no RAID superblock on /dev/sda1
mdadm: no RAID superblock on /dev/sda
mdadm: /dev/sdf is identified as a member of /dev/md/ddf0, slot 4.
mdadm: /dev/sde is identified as a member of /dev/md/ddf0, slot 3.
mdadm: /dev/sdd is identified as a member of /dev/md/ddf0, slot 2.
mdadm: /dev/sdc is identified as a member of /dev/md/ddf0, slot 1.
mdadm: /dev/sdb is identified as a member of /dev/md/ddf0, slot 0.
mdadm: ignoring /dev/sdb as it reports /dev/sdf as failed
mdadm: ignoring /dev/sdc as it reports /dev/sdf as failed
mdadm: ignoring /dev/sdd as it reports /dev/sdf as failed
mdadm: ignoring /dev/sde as it reports /dev/sdf as failed
mdadm: no uptodate device for slot 0 of /dev/md/ddf0
mdadm: no uptodate device for slot 2 of /dev/md/ddf0
mdadm: no uptodate device for slot 4 of /dev/md/ddf0
mdadm: no uptodate device for slot 6 of /dev/md/ddf0
mdadm: added /dev/sdf to /dev/md/ddf0 as 4
mdadm: Container /dev/md/ddf0 has been assembled with 1 drive (out of 5)
mdadm: looking for devices for /dev/md/MegaSR
mdadm: looking in container /dev/md/ddf0
mdadm: no recogniseable superblock on /dev/sr0
mdadm: /dev/sdf has wrong uuid.
mdadm: no recogniseable superblock on /dev/sde2
mdadm: no recogniseable superblock on /dev/sde1
mdadm: /dev/sde has wrong uuid.
mdadm: /dev/sdd has wrong uuid.
mdadm: /dev/sdc has wrong uuid.
mdadm: no recogniseable superblock on /dev/sdb2
mdadm: no recogniseable superblock on /dev/sdb1
mdadm: /dev/sdb has wrong uuid.
mdadm: Cannot assemble mbr metadata on /dev/sda1
mdadm: Cannot assemble mbr metadata on /dev/sda
mdadm: looking for devices for /dev/md/MegaSR
mdadm: looking in container /dev/md/ddf0
mdadm: no recogniseable superblock on /dev/sr0
mdadm: /dev/sdf has wrong uuid.
mdadm: no recogniseable superblock on /dev/sde2
mdadm: no recogniseable superblock on /dev/sde1
mdadm: /dev/sde has wrong uuid.
mdadm: /dev/sdd has wrong uuid.
mdadm: /dev/sdc has wrong uuid.
mdadm: no recogniseable superblock on /dev/sdb2
mdadm: no recogniseable superblock on /dev/sdb1
mdadm: /dev/sdb has wrong uuid.
mdadm: Cannot assemble mbr metadata on /dev/sda1
mdadm: Cannot assemble mbr metadata on /dev/sda
Output of 'dmesg | grep md:'
md: linear personality registered for level -1
md: raid0 personality registered for level 0
md: raid1 personality registered for level 1
md: raid10 personality registered for level 10
md: raid6 personality registered for level 6
md: raid5 personality registered for level 5
md: raid4 personality registered for level 4
md: multipath personality registered for level -4
md: md127 stopped.
md: bind<sdf>

Contents of /dev/mapper directory:
crw------T    1 root     root       10, 236 Nov 20 15:05 control

startraid mode = null (default)

Contents of /tbu/utility/dmraid-boot.txt file (from activate command on boot):
NOTICE: checking format identifier asr
NOTICE: checking format identifier hpt37x
NOTICE: checking format identifier hpt45x
NOTICE: checking format identifier jmicron
NOTICE: checking format identifier lsi
NOTICE: checking format identifier nvidia
NOTICE: checking format identifier pdc
NOTICE: checking format identifier sil
NOTICE: checking format identifier via
NOTICE: checking format identifier dos
WARN: locking /var/lock/dmraid/.lock
NOTICE: /dev/sdf: asr     discovering
NOTICE: /dev/sdf: hpt37x  discovering
NOTICE: /dev/sdf: hpt45x  discovering
NOTICE: /dev/sdf: jmicron discovering
NOTICE: /dev/sdf: lsi     discovering
NOTICE: /dev/sdf: nvidia  discovering
NOTICE: /dev/sdf: pdc     discovering
NOTICE: /dev/sdf: sil     discovering
NOTICE: /dev/sdf: via     discovering
NOTICE: /dev/sde: asr     discovering
NOTICE: /dev/sde: hpt37x  discovering
NOTICE: /dev/sde: hpt45x  discovering
NOTICE: /dev/sde: jmicron discovering
NOTICE: /dev/sde: lsi     discovering
NOTICE: /dev/sde: nvidia  discovering
NOTICE: /dev/sde: pdc     discovering
NOTICE: /dev/sde: sil     discovering
NOTICE: /dev/sde: via     discovering
NOTICE: /dev/sdd: asr     discovering
NOTICE: /dev/sdd: hpt37x  discovering
NOTICE: /dev/sdd: hpt45x  discovering
NOTICE: /dev/sdd: jmicron discovering
NOTICE: /dev/sdd: lsi     discovering
NOTICE: /dev/sdd: nvidia  discovering
NOTICE: /dev/sdd: pdc     discovering
NOTICE: /dev/sdd: sil     discovering
NOTICE: /dev/sdd: via     discovering
NOTICE: /dev/sdc: asr     discovering
NOTICE: /dev/sdc: hpt37x  discovering
NOTICE: /dev/sdc: hpt45x  discovering
NOTICE: /dev/sdc: jmicron discovering
NOTICE: /dev/sdc: lsi     discovering
NOTICE: /dev/sdc: nvidia  discovering
NOTICE: /dev/sdc: pdc     discovering
NOTICE: /dev/sdc: sil     discovering
NOTICE: /dev/sdc: via     discovering
NOTICE: /dev/sdb: asr     discovering
NOTICE: /dev/sdb: hpt37x  discovering
NOTICE: /dev/sdb: hpt45x  discovering
NOTICE: /dev/sdb: jmicron discovering
NOTICE: /dev/sdb: lsi     discovering
NOTICE: /dev/sdb: nvidia  discovering
NOTICE: /dev/sdb: pdc     discovering
NOTICE: /dev/sdb: sil     discovering
NOTICE: /dev/sdb: via     discovering
NOTICE: /dev/sda: asr     discovering
NOTICE: /dev/sda: hpt37x  discovering
NOTICE: /dev/sda: hpt45x  discovering
NOTICE: /dev/sda: jmicron discovering
NOTICE: /dev/sda: lsi     discovering
NOTICE: /dev/sda: nvidia  discovering
NOTICE: /dev/sda: pdc     discovering
NOTICE: /dev/sda: sil     discovering
NOTICE: /dev/sda: via     discovering
no raid disks with format:
"asr,hpt37x,hpt45x,jmicron,lsi,nvidia,pdc,sil,via,dos"
WARN: unlocking /var/lock/dmraid/.lock

Output of dmraid --raid_devices command:
/dev/sdf: ddf1, ".ddf1_disks", GROUP, unknown, 285155328 sectors, data@ 0
/dev/sde: ddf1, ".ddf1_disks", GROUP, ok, 285155328 sectors, data@ 0
/dev/sdd: ddf1, ".ddf1_disks", GROUP, ok, 285155328 sectors, data@ 0
/dev/sdc: ddf1, ".ddf1_disks", GROUP, ok, 285155328 sectors, data@ 0
/dev/sdb: ddf1, ".ddf1_disks", GROUP, ok, 285155328 sectors, data@ 0

Output of dmraid -s -s command:
*** Group superset .ddf1_disks
--> Subset
name   : ddf1_MegaSR   R5 #0
size   : 855465984
stride : 128
type   : raid5_ls
status : ok
subsets: 0
devs   : 4
spares : 0

Output of blkid command:
/dev/sda1: LABEL="Elements" UUID="DA1CE6D71CE6AE27" TYPE="ntfs"
/dev/sdc: UUID="LSI     M-^@M-^F^]h^Q7" TYPE="ddf_raid_member"
/dev/sdd: UUID="LSI     M-^@M-^F^]h^Q7" TYPE="ddf_raid_member"
/dev/sdf: UUID="LSI     M-^@M-^F^]h^Q7" TYPE="ddf_raid_member"
/dev/sdg: LABEL="CS SUPPORT" UUID="4CC3-3665" TYPE="vfat"
/dev/sr0: LABEL="iflnet" TYPE="iso9660"

Output of free command (memory):

             total       used       free     shared    buffers     cached
Mem:       2039280     185052    1854228          0      40996      77220
-/+ buffers/cache:      66836    1972444
Swap:            0          0          0

Output of lsmod command (loaded modules):

Module                  Size  Used by    Not tainted
sr_mod                 10387  0
cdrom                  24577  1 sr_mod
sg                     17182  0
sd_mod                 25922  1
crc_t10dif               996  1 sd_mod
usb_storage            32098  0
hid_generic              665  0
usbhid                 18838  0
hid                    60174  2 hid_generic,usbhid
pcspkr                  1227  0
isci                   71218  1
i2c_i801                7257  0
libsas                 47529  1 isci
scsi_transport_sas     16455  2 isci,libsas
igb                    98565  0
i2c_algo_bit            3747  1 igb
i2c_core               13114  3 i2c_i801,igb,i2c_algo_bit
ehci_pci                2540  0
ptp                     5420  1 igb
pps_core                4544  1 ptp
processor              14951  8
thermal_sys            12739  1 processor
hwmon                    881  2 igb,thermal_sys
button                  3413  0
dm_mod                 50912  0
ehci_hcd               28628  1 ehci_pci
edd                     5144  0

Contents of /sys/module:
8250                i2c_algo_bit        ptp                 speakup_dtlk
acpi                i2c_core            raid1               speakup_dummy
auth_rpcgss         i2c_i801            raid10              speakup_keypc
block               i8042               random              speakup_ltlk
brd                 igb                 rcupdate            speakup_soft
button              input_polldev       rcutree             speakup_spkout
cdrom               isci                rfkill              speakup_txprt
cifs                kernel              scsi_mod            spurious
cpuidle             keyboard            scsi_transport_fc   sr_mod
crc_t10dif          libata              scsi_transport_sas  sunrpc
dm_mod              libsas              sd_mod              tcp_cubic
dns_resolver        lockd               sg                  thermal_sys
edd                 md_mod              speakup             usb_storage
ehci_hcd            mousedev            speakup_acntpc      usbcore
ehci_pci            nfs                 speakup_acntsa      usbhid
firmware_class      pcie_aspm           speakup_apollo      vt
fuse                pcspkr              speakup_audptr      workqueue
hid                 pps_core            speakup_bns         xz_dec
hid_generic         printk              speakup_decext
hwmon               processor           speakup_dectlk

Re: MDADM 3.3 broken?

[Martin Wilck]

On 11/21/2013 12:15 AM, David F. wrote:

> mdadm: /dev/sdf is identified as a member of /dev/md/ddf0, slot 4.
> mdadm: /dev/sde is identified as a member of /dev/md/ddf0, slot 3.
> mdadm: /dev/sdd is identified as a member of /dev/md/ddf0, slot 2.
> mdadm: /dev/sdc is identified as a member of /dev/md/ddf0, slot 1.
> mdadm: /dev/sdb is identified as a member of /dev/md/ddf0, slot 0.
> mdadm: ignoring /dev/sdb as it reports /dev/sdf as failed
> mdadm: ignoring /dev/sdc as it reports /dev/sdf as failed
> mdadm: ignoring /dev/sdd as it reports /dev/sdf as failed
> mdadm: ignoring /dev/sde as it reports /dev/sdf as failed
> mdadm: no uptodate device for slot 0 of /dev/md/ddf0
> mdadm: no uptodate device for slot 2 of /dev/md/ddf0
> mdadm: no uptodate device for slot 4 of /dev/md/ddf0
> mdadm: no uptodate device for slot 6 of /dev/md/ddf0
> mdadm: added /dev/sdf to /dev/md/ddf0 as 4
> mdadm: Container /dev/md/ddf0 has been assembled with 1 drive (out of 5)

That looks really weird. The (healthy?) devices sdb-sde are ignored
because they report sdf as failed, and then sdf is used for assembly?

I have no idea at the moment, I need to read the code.

> Output of dmraid --raid_devices command:
> /dev/sdf: ddf1, ".ddf1_disks", GROUP, unknown, 285155328 sectors, data@ 0
> /dev/sde: ddf1, ".ddf1_disks", GROUP, ok, 285155328 sectors, data@ 0
> /dev/sdd: ddf1, ".ddf1_disks", GROUP, ok, 285155328 sectors, data@ 0
> /dev/sdc: ddf1, ".ddf1_disks", GROUP, ok, 285155328 sectors, data@ 0
> /dev/sdb: ddf1, ".ddf1_disks", GROUP, ok, 285155328 sectors, data@ 0

This seems to support then notion that something's wrong with /dev/sdf.

Martin

Re: MDADM 3.3 broken?

[David F.]

On that DMRAID - they are still yet to try it.  But we do know the
RAID5 works via Int13h interface in real mode and via Windows.  I
think they thought it was a 4 disk array?  I'll ask if they know the
actually number of drives in the RAID configuration.

On Thu, Nov 21, 2013 at 12:50 PM, Martin Wilck <mwilck@arcor.de> wrote:
> On 11/21/2013 12:15 AM, David F. wrote:
>
>> mdadm: /dev/sdf is identified as a member of /dev/md/ddf0, slot 4.
>> mdadm: /dev/sde is identified as a member of /dev/md/ddf0, slot 3.
>> mdadm: /dev/sdd is identified as a member of /dev/md/ddf0, slot 2.
>> mdadm: /dev/sdc is identified as a member of /dev/md/ddf0, slot 1.
>> mdadm: /dev/sdb is identified as a member of /dev/md/ddf0, slot 0.
>> mdadm: ignoring /dev/sdb as it reports /dev/sdf as failed
>> mdadm: ignoring /dev/sdc as it reports /dev/sdf as failed
>> mdadm: ignoring /dev/sdd as it reports /dev/sdf as failed
>> mdadm: ignoring /dev/sde as it reports /dev/sdf as failed
>> mdadm: no uptodate device for slot 0 of /dev/md/ddf0
>> mdadm: no uptodate device for slot 2 of /dev/md/ddf0
>> mdadm: no uptodate device for slot 4 of /dev/md/ddf0
>> mdadm: no uptodate device for slot 6 of /dev/md/ddf0
>> mdadm: added /dev/sdf to /dev/md/ddf0 as 4
>> mdadm: Container /dev/md/ddf0 has been assembled with 1 drive (out of 5)
>
> That looks really weird. The (healthy?) devices sdb-sde are ignored
> because they report sdf as failed, and then sdf is used for assembly?
>
> I have no idea at the moment, I need to read the code.
>
>> Output of dmraid --raid_devices command:
>> /dev/sdf: ddf1, ".ddf1_disks", GROUP, unknown, 285155328 sectors, data@ 0
>> /dev/sde: ddf1, ".ddf1_disks", GROUP, ok, 285155328 sectors, data@ 0
>> /dev/sdd: ddf1, ".ddf1_disks", GROUP, ok, 285155328 sectors, data@ 0
>> /dev/sdc: ddf1, ".ddf1_disks", GROUP, ok, 285155328 sectors, data@ 0
>> /dev/sdb: ddf1, ".ddf1_disks", GROUP, ok, 285155328 sectors, data@ 0
>
> This seems to support then notion that something's wrong with /dev/sdf.
>
> Martin

Re: MDADM 3.3 broken?

[Martin Wilck]

On 11/21/2013 10:10 PM, David F. wrote:
> On that DMRAID - they are still yet to try it.  But we do know the
> RAID5 works via Int13h interface in real mode and via Windows.  I
> think they thought it was a 4 disk array?  I'll ask if they know the
> actually number of drives in the RAID configuration.

What distribution are these people using? I am not aware of any distro
that would activate mdadm for DDF RAID by default.

Martin

Re: MDADM 3.3 broken?

[David F.]

Are you saying that the old obsolete DMRAID should still be used for
DDF RAID?   What about the 2TiB limit?  I'd rather see modern linux
RAID support work as good as it does for Windows.

On Thu, Nov 21, 2013 at 1:30 PM, Martin Wilck <mwilck@arcor.de> wrote:
> On 11/21/2013 10:10 PM, David F. wrote:
>> On that DMRAID - they are still yet to try it.  But we do know the
>> RAID5 works via Int13h interface in real mode and via Windows.  I
>> think they thought it was a 4 disk array?  I'll ask if they know the
>> actually number of drives in the RAID configuration.
>
> What distribution are these people using? I am not aware of any distro
> that would activate mdadm for DDF RAID by default.
>
> Martin

Re: MDADM 3.3 broken?

[Martin Wilck]

On 11/21/2013 11:39 PM, David F. wrote:
> Are you saying that the old obsolete DMRAID should still be used for
> DDF RAID?   What about the 2TiB limit?  I'd rather see modern linux
> RAID support work as good as it does for Windows.

No, I haven't said that. On the contrary, I am actively working on
getting mdadm support for DDF into the main distributions.

I was just wondering about your setup because as far as I know, no
distribution enables mdadm support for DDF. Doing that requires changes
in the distribution's udev rules, intrd/initramfs generation code, and
installer. See my DDF page in the Linux RAID wiki for details.

Regards
Martin


> 
> On Thu, Nov 21, 2013 at 1:30 PM, Martin Wilck <mwilck@arcor.de> wrote:
>> On 11/21/2013 10:10 PM, David F. wrote:
>>> On that DMRAID - they are still yet to try it.  But we do know the
>>> RAID5 works via Int13h interface in real mode and via Windows.  I
>>> think they thought it was a 4 disk array?  I'll ask if they know the
>>> actually number of drives in the RAID configuration.
>>
>> What distribution are these people using? I am not aware of any distro
>> that would activate mdadm for DDF RAID by default.
>>
>> Martin

Re: MDADM 3.3 broken?

[Martin Wilck]

On 11/20/2013 03:30 AM, NeilBrown wrote:

>>>> mdadm --examine --scan
>>> ARRAY metadata=ddf UUID=7ab254d0:fae71048:
>>> 404edde9:750a8a05
>>> ARRAY container=7ab254d0:fae71048:404edde9:750a8a05 member=0
>>> UUID=5337ab03:86ca2abc:d42bfbc8:23626c78
> 
> This shows that mdadm found a container with the correct UUID, but the member
> array inside the container has the wrong uuid.
> 
> Martin: I think one of your recent changes would have changed the member UUID
> for some specific arrays because the one that was being created before wasn't
> reliably stable.  Could  that apply to David's situation?

I am confused. AFAIL, my patch bedbf68a first introduced subarray UUIDs
for DDF. I don't understand how this mdadm.conf could have worked with
mdadm 3.2.x.

But you are right, I had to make 7087f02b later that changed the way
subarray UUIDs were calculated. This would hurt people who created their
mdadm.conf file) with stock 3.3 and updated to latest git later.

> David: if you remove the "UUID=" part for the array leaving the
> "container=.... member=0" as the identification, does it work?

I second that. David, please try it. I'd also appreciate "mdadm -E
/dev/sdX" output for all the RAID disks.

Martin

Re: MDADM 3.3 broken?

[David F.]

>> Martin: I think one of your recent changes would have changed the member UUID
>> for some specific arrays because the one that was being created before wasn't
>> reliably stable.  Could  that apply to David's situation?
>
> I am confused. AFAIL, my patch bedbf68a first introduced subarray UUIDs
> for DDF. I don't understand how this mdadm.conf could have worked with
> mdadm 3.2.x.

I'm not sure on the cisco server with lsi raid that the 3.2.x version
works as that is different than the isw issues that most were having.

>
> But you are right, I had to make 7087f02b later that changed the way
> subarray UUIDs were calculated. This would hurt people who created their
> mdadm.conf file) with stock 3.3 and updated to latest git later.
>
>> David: if you remove the "UUID=" part for the array leaving the
>> "container=.... member=0" as the identification, does it work?

We sent them a version that will try that - hope they don't get too
tired of testing.  The int13h interface to the RAID works fine, as
does Windows interface.

>
> I second that. David, please try it. I'd also appreciate "mdadm -E
> /dev/sdX" output for all the RAID disks.

That version we sent should output this as well.

>
> Martin

Re: MDADM 3.3 broken?

[David F.]

They say it still didn't work (we got rid of the member uuid) - but
here's is more detailed stuff you asked for:

Storage devices (non-RAID) detected by Linux based on /sys/block:

Optical drives (srx):
  sr0: hp       DVDRAM GT30L     (fw rev = mP06)
Removable drives (sdx):
  sdg: 7453 MiB          Patriot Memory
Fixed hard drives (sdx):
  sda: 1863 GiB WD       Ext HDD 1021
  sdb: 136 GiB TOSHIBA  MK1401GRRB
  sdc: 136 GiB TOSHIBA  MK1401GRRB
  sdd: 136 GiB TOSHIBA  MK1401GRRB
  sde: 136 GiB TOSHIBA  MK1401GRRB
  sdf: 136 GiB TOSHIBA  MK1401GRRB

Contents of /sys/block:
md127  ram0   ram1   sda    sdb    sdc    sdd    sde    sdf    sdg    sr0

Contents of /dev:
block               port                tty1                tty47
bsg                 ppp                 tty10               tty48
bus                 psaux               tty11               tty49
cdrom               ptmx                tty12               tty5
cdrom1              ptp0                tty13               tty50
cdrw                ptp1                tty14               tty51
cdrw1               pts                 tty15               tty52
char                ram0                tty16               tty53
console             ram1                tty17               tty54
core                random              tty18               tty55
cpu_dma_latency     rfkill              tty19               tty56
disk                rtc                 tty2                tty57
dvd                 rtc0                tty20               tty58
dvd1                sda                 tty21               tty59
dvdrw               sda1                tty22               tty6
dvdrw1              sdb                 tty23               tty60
fd                  sdb1                tty24               tty61
full                sdb2                tty25               tty62
fuse                sdc                 tty26               tty63
hidraw0             sdd                 tty27               tty7
hidraw1             sde                 tty28               tty8
hidraw2             sde1                tty29               tty9
hidraw3             sde2                tty3                ttyS0
hidraw4             sdf                 tty30               ttyS1
hidraw5             sdg                 tty31               ttyS2
hpet                sg0                 tty32               ttyS3
input               sg1                 tty33               urandom
kmsg                sg2                 tty34               vcs
log                 sg3                 tty35               vcs1
loop-control        sg4                 tty36               vcs2
loop0               sg5                 tty37               vcs3
loop1               sg6                 tty38               vcs4
mapper              sg7                 tty39               vcs5
mcelog              shm                 tty4                vcsa
md                  sr0                 tty40               vcsa1
md127               stderr              tty41               vcsa2
mem                 stdin               tty42               vcsa3
net                 stdout              tty43               vcsa4
network_latency     synth               tty44               vcsa5
network_throughput  tty                 tty45               vga_arbiter
null                tty0                tty46               zero

Contents of /proc/partitions:
major minor  #blocks  name

   8        0 1953512448 sda
   8        1 1953511424 sda1
   8       16  143638992 sdb
   8       17     102400 sdb1
   8       18  143535568 sdb2
   8       32  143638992 sdc
   8       48  143638992 sdd
   8       64  143638992 sde
   8       65     102400 sde1
   8       66  143535568 sde2
   8       80  143638992 sdf
  11        0      48608 sr0
   8       96    7632892 sdg

Disk /dev/sda: 2000.3 GB, 2000396746752 bytes
255 heads, 63 sectors/track, 243201 cylinders, total 3907024896 sectors
Units = sectors of 1 * 512 = 512 bytes

   Device Boot      Start         End      Blocks  Id System
/dev/sda1            2048  3907024895  1953511424   7 HPFS/NTFS

Disk /dev/sdb: 147.0 GB, 147086327808 bytes
255 heads, 63 sectors/track, 17882 cylinders, total 287277984 sectors
Units = sectors of 1 * 512 = 512 bytes

   Device Boot      Start         End      Blocks  Id System
/dev/sdb1   *        2048      206847      102400   7 HPFS/NTFS
Partition 1 does not end on cylinder boundary
/dev/sdb2          206848   855463935   427628544   7 HPFS/NTFS

Disk /dev/sdc: 147.0 GB, 147086327808 bytes
255 heads, 63 sectors/track, 17882 cylinders, total 287277984 sectors
Units = sectors of 1 * 512 = 512 bytes

Disk /dev/sdc doesn't contain a valid partition table

Disk /dev/sdd: 147.0 GB, 147086327808 bytes
255 heads, 63 sectors/track, 17882 cylinders, total 287277984 sectors
Units = sectors of 1 * 512 = 512 bytes

Disk /dev/sdd doesn't contain a valid partition table

Disk /dev/sde: 147.0 GB, 147086327808 bytes
255 heads, 63 sectors/track, 17882 cylinders, total 287277984 sectors
Units = sectors of 1 * 512 = 512 bytes

   Device Boot      Start         End      Blocks  Id System
/dev/sde1   *        2048      206847      102400   7 HPFS/NTFS
Partition 1 does not end on cylinder boundary
/dev/sde2          206848   855463935   427628544   7 HPFS/NTFS

Disk /dev/sdf: 147.0 GB, 147086327808 bytes
255 heads, 63 sectors/track, 17882 cylinders, total 287277984 sectors
Units = sectors of 1 * 512 = 512 bytes

Disk /dev/sdf doesn't contain a valid partition table

Disk /dev/sdg: 7816 MB, 7816081408 bytes
241 heads, 62 sectors/track, 1021 cylinders, total 15265784 sectors
Units = sectors of 1 * 512 = 512 bytes

   Device Boot      Start         End      Blocks  Id System
/dev/sdg1   ?   778135908  1919645538   570754815+ 72 Unknown
Partition 1 has different physical/logical beginnings (non-Linux?):
     phys=(357, 116, 40) logical=(52077, 22, 11)
Partition 1 has different physical/logical endings:
     phys=(357, 32, 45) logical=(128473, 31, 51)
Partition 1 does not end on cylinder boundary
/dev/sdg2   ?   168689522  2104717761   968014120  65 Unknown
Partition 2 has different physical/logical beginnings (non-Linux?):
     phys=(288, 115, 43) logical=(11289, 149, 47)
Partition 2 has different physical/logical endings:
     phys=(367, 114, 50) logical=(140859, 41, 42)
Partition 2 does not end on cylinder boundary
/dev/sdg3   ?  1869881465  3805909656   968014096  79 Unknown
Partition 3 has different physical/logical beginnings (non-Linux?):
     phys=(366, 32, 33) logical=(125142, 156, 30)
Partition 3 has different physical/logical endings:
     phys=(357, 32, 43) logical=(254712, 47, 39)
Partition 3 does not end on cylinder boundary
/dev/sdg4   ?  2885681152  2885736650       27749+  d Unknown
Partition 4 has different physical/logical beginnings (non-Linux?):
     phys=(372, 97, 50) logical=(193125, 119, 25)
Partition 4 has different physical/logical endings:
     phys=(0, 10, 0) logical=(193129, 50, 33)
Partition 4 does not end on cylinder boundary

Partition table entries are not in disk order

Contents of /proc/mdstat (Linux software RAID status):
Personalities : [linear] [raid0] [raid1] [raid10] [raid6] [raid5]
[raid4] [multipath]
md127 : inactive sdf[0](S)
      1061328 blocks super external:ddf

unused devices: <none>

Contents of /run/mdadm/map (Linux software RAID arrays):
md127 ddf 7ab254d0:fae71048:404edde9:750a8a05 /dev/md/ddf0

Contents of /etc/mdadm/mdadm.conf (Linux software RAID config file):
# mdadm.conf
#
# Please refer to mdadm.conf(5) for information about this file.
#

# by default (built-in), scan all partitions (/proc/partitions) and all
# containers for MD superblocks. alternatively, specify devices to scan, using
# wildcards if desired.
DEVICE partitions containers

# automatically tag new arrays as belonging to the local system
HOMEHOST <system>

ARRAY metadata=ddf UUID=7ab254d0:fae71048:404edde9:750a8a05
ARRAY /dev/md/MegaSR container=7ab254d0:fae71048:404edde9:750a8a05 member=0

Long listing of /dev/md directory and /dev/md* device files (Linux
software RAID devices):
brw-rw---T    1 root     disk        9, 127 Nov 21 13:49 /dev/md127

/dev/md:
lrwxrwxrwx    1 root     root             8 Nov 21 13:49 ddf0 -> ../md127

Contents of /tbu/utility/mdadm.txt (mdadm troubleshooting data
captured when 'start-md' is executed):
mdadm - v3.3-32-g357ac10 - 20th November 2013
Output of 'mdadm --examine --scan'
ARRAY metadata=ddf UUID=7ab254d0:fae71048:404edde9:750a8a05
ARRAY /dev/md/MegaSR container=7ab254d0:fae71048:404edde9:750a8a05
member=0 UUID=9515a157:e69d2ad3:8427131f:92b29c7a
Output of 'mdadm --assemble --scan --no-degraded -v'
mdadm: looking for devices for further assembly
mdadm: no RAID superblock on /dev/sr0
mdadm: no RAID superblock on /dev/sde2
mdadm: no RAID superblock on /dev/sde1
mdadm: no RAID superblock on /dev/sdb2
mdadm: no RAID superblock on /dev/sdb1
mdadm: no RAID superblock on /dev/sda1
mdadm: no RAID superblock on /dev/sda
mdadm: /dev/sdf is identified as a member of /dev/md/ddf0, slot 4.
mdadm: /dev/sde is identified as a member of /dev/md/ddf0, slot 3.
mdadm: /dev/sdd is identified as a member of /dev/md/ddf0, slot 2.
mdadm: /dev/sdc is identified as a member of /dev/md/ddf0, slot 1.
mdadm: /dev/sdb is identified as a member of /dev/md/ddf0, slot 0.
mdadm: ignoring /dev/sdb as it reports /dev/sdf as failed
mdadm: ignoring /dev/sdc as it reports /dev/sdf as failed
mdadm: ignoring /dev/sdd as it reports /dev/sdf as failed
mdadm: ignoring /dev/sde as it reports /dev/sdf as failed
mdadm: no uptodate device for slot 0 of /dev/md/ddf0
mdadm: no uptodate device for slot 2 of /dev/md/ddf0
mdadm: no uptodate device for slot 4 of /dev/md/ddf0
mdadm: no uptodate device for slot 6 of /dev/md/ddf0
mdadm: added /dev/sdf to /dev/md/ddf0 as 4
mdadm: Container /dev/md/ddf0 has been assembled with 1 drive (out of 5)
mdadm: looking for devices for /dev/md/MegaSR
mdadm: looking in container /dev/md/ddf0
mdadm: no recogniseable superblock on /dev/sr0
mdadm: /dev/sdf is not a container and one is required.
mdadm: no recogniseable superblock on /dev/sde2
mdadm: no recogniseable superblock on /dev/sde1
mdadm: /dev/sde is not a container and one is required.
mdadm: /dev/sdd is not a container and one is required.
mdadm: /dev/sdc is not a container and one is required.
mdadm: no recogniseable superblock on /dev/sdb2
mdadm: no recogniseable superblock on /dev/sdb1
mdadm: /dev/sdb is not a container and one is required.
mdadm: Cannot assemble mbr metadata on /dev/sda1
mdadm: Cannot assemble mbr metadata on /dev/sda
mdadm: looking for devices for /dev/md/MegaSR
mdadm: looking in container /dev/md/ddf0
mdadm: no recogniseable superblock on /dev/sr0
mdadm: /dev/sdf is not a container and one is required.
mdadm: no recogniseable superblock on /dev/sde2
mdadm: no recogniseable superblock on /dev/sde1
mdadm: /dev/sde is not a container and one is required.
mdadm: /dev/sdd is not a container and one is required.
mdadm: /dev/sdc is not a container and one is required.
mdadm: no recogniseable superblock on /dev/sdb2
mdadm: no recogniseable superblock on /dev/sdb1
mdadm: /dev/sdb is not a container and one is required.
mdadm: Cannot assemble mbr metadata on /dev/sda1
mdadm: Cannot assemble mbr metadata on /dev/sda
Output of 'dmesg | grep md:'
md: linear personality registered for level -1
md: raid0 personality registered for level 0
md: raid1 personality registered for level 1
md: raid10 personality registered for level 10
md: raid6 personality registered for level 6
md: raid5 personality registered for level 5
md: raid4 personality registered for level 4
md: multipath personality registered for level -4
md: md127 stopped.
md: bind<sdf>


Output of 'mdadm -E /dev/sda'
/dev/sda:
   MBR Magic : aa55
Partition[0] :   3907022848 sectors at         2048 (type 07)


Output of 'mdadm -E /dev/sdb'
/dev/sdb:
          Magic : de11de11
        Version : 01.00.00
Controller GUID : 4C534920:20202020:FFFFFFFF:FFFFFFFF:FFFFFFFF:FFFFFFFF
                  (LSI     )
 Container GUID : 4C534920:20202020:80861D68:113700B2:3FB602B2:926C6F1A
                  (LSI      11/14/13 10:40:50)
            Seq : 000000de
  Redundant hdr : yes
  Virtual Disks : 1

      VD GUID[0] : 4C534920:20202020:80861D60:00000000:3FBF690F:00001450
                  (LSI      11/21/13 13:47:59)
         unit[0] : 0
        state[0] : Optimal, Consistent
   init state[0] : Fully Initialised
       access[0] : Read/Write
         Name[0] : MegaSR   R5 #0
 Raid Devices[0] : 4 (0 1 2 3)
   Chunk Size[0] : 128 sectors
   Raid Level[0] : RAID5
  Device Size[0] : 142577664
   Array Size[0] : 427732992

 Physical Disks : 5
      Number    RefNo      Size       Device      Type/State
         0    ffffe3be  142577664K /dev/sdb        active/Online
         1    ffbf7ff4  142577664K                 active/Online
         2    ffffd801  142577664K                 active/Online
         3    bff6febe  142577664K                 active/Online
         4    bfffcf03  142577664K                 activeGlobal-Spare/Offline


Output of 'mdadm -E /dev/sdc'
/dev/sdc:
          Magic : de11de11
        Version : 01.00.00
Controller GUID : 4C534920:20202020:FFFFFFFF:FFFFFFFF:FFFFFFFF:FFFFFFFF
                  (LSI     )
 Container GUID : 4C534920:20202020:80861D68:113700B2:3FB602B2:926C6F1A
                  (LSI      11/14/13 10:40:50)
            Seq : 000000de
  Redundant hdr : yes
  Virtual Disks : 1

      VD GUID[0] : 4C534920:20202020:80861D60:00000000:3FBF690F:00001450
                  (LSI      11/21/13 13:47:59)
         unit[0] : 0
        state[0] : Optimal, Consistent
   init state[0] : Fully Initialised
       access[0] : Read/Write
         Name[0] : MegaSR   R5 #0
 Raid Devices[0] : 4 (0 1 2 3)
   Chunk Size[0] : 128 sectors
   Raid Level[0] : RAID5
  Device Size[0] : 142577664
   Array Size[0] : 427732992

 Physical Disks : 5
      Number    RefNo      Size       Device      Type/State
         0    ffffe3be  142577664K                 active/Online
         1    ffbf7ff4  142577664K /dev/sdc        active/Online
         2    ffffd801  142577664K                 active/Online
         3    bff6febe  142577664K                 active/Online
         4    bfffcf03  142577664K                 activeGlobal-Spare/Offline


Output of 'mdadm -E /dev/sdd'
/dev/sdd:
          Magic : de11de11
        Version : 01.00.00
Controller GUID : 4C534920:20202020:FFFFFFFF:FFFFFFFF:FFFFFFFF:FFFFFFFF
                  (LSI     )
 Container GUID : 4C534920:20202020:80861D68:113700B2:3FB602B2:926C6F1A
                  (LSI      11/14/13 10:40:50)
            Seq : 000000de
  Redundant hdr : yes
  Virtual Disks : 1

      VD GUID[0] : 4C534920:20202020:80861D60:00000000:3FBF690F:00001450
                  (LSI      11/21/13 13:47:59)
         unit[0] : 0
        state[0] : Optimal, Consistent
   init state[0] : Fully Initialised
       access[0] : Read/Write
         Name[0] : MegaSR   R5 #0
 Raid Devices[0] : 4 (0 1 2 3)
   Chunk Size[0] : 128 sectors
   Raid Level[0] : RAID5
  Device Size[0] : 142577664
   Array Size[0] : 427732992

 Physical Disks : 5
      Number    RefNo      Size       Device      Type/State
         0    ffffe3be  142577664K                 active/Online
         1    ffbf7ff4  142577664K                 active/Online
         2    ffffd801  142577664K /dev/sdd        active/Online
         3    bff6febe  142577664K                 active/Online
         4    bfffcf03  142577664K                 activeGlobal-Spare/Offline


Output of 'mdadm -E /dev/sde'
/dev/sde:
          Magic : de11de11
        Version : 01.00.00
Controller GUID : 4C534920:20202020:FFFFFFFF:FFFFFFFF:FFFFFFFF:FFFFFFFF
                  (LSI     )
 Container GUID : 4C534920:20202020:80861D68:113700B2:3FB602B2:926C6F1A
                  (LSI      11/14/13 10:40:50)
            Seq : 000000de
  Redundant hdr : yes
  Virtual Disks : 1

      VD GUID[0] : 4C534920:20202020:80861D60:00000000:3FBF690F:00001450
                  (LSI      11/21/13 13:47:59)
         unit[0] : 0
        state[0] : Optimal, Consistent
   init state[0] : Fully Initialised
       access[0] : Read/Write
         Name[0] : MegaSR   R5 #0
 Raid Devices[0] : 4 (0 1 2 3)
   Chunk Size[0] : 128 sectors
   Raid Level[0] : RAID5
  Device Size[0] : 142577664
   Array Size[0] : 427732992

 Physical Disks : 5
      Number    RefNo      Size       Device      Type/State
         0    ffffe3be  142577664K                 active/Online
         1    ffbf7ff4  142577664K                 active/Online
         2    ffffd801  142577664K                 active/Online
         3    bff6febe  142577664K /dev/sde        active/Online
         4    bfffcf03  142577664K                 activeGlobal-Spare/Offline


Output of 'mdadm -E /dev/sdf'
/dev/sdf:
          Magic : de11de11
        Version : 01.00.00
Controller GUID : 4C534920:20202020:FFFFFFFF:FFFFFFFF:FFFFFFFF:FFFFFFFF
                  (LSI     )
 Container GUID : 4C534920:20202020:80861D68:113700B2:3FB602B2:926C6F1A
                  (LSI      11/14/13 10:40:50)
            Seq : 000000de
  Redundant hdr : yes
  Virtual Disks : 1

      VD GUID[0] : 4C534920:20202020:80861D60:00000000:3FBF690F:00001450
                  (LSI      11/21/13 13:47:59)
         unit[0] : 0
        state[0] : Optimal, Consistent
   init state[0] : Fully Initialised
       access[0] : Read/Write
         Name[0] : MegaSR   R5 #0

 Physical Disks : 5
      Number    RefNo      Size       Device      Type/State
         0    ffffe3be  142577664K                 active/Online
         1    ffbf7ff4  142577664K                 active/Online
         2    ffffd801  142577664K                 active/Online
         3    bff6febe  142577664K                 active/Online
         4    bfffcf03  142577664K /dev/sdf        activeGlobal-Spare/Offline


Output of 'mdadm -E /dev/sdg'
/dev/sdg:
   MBR Magic : aa55
Partition[0] :   1141509631 sectors at    778135908 (type 72)
Partition[1] :   1936028240 sectors at    168689522 (type 65)
Partition[2] :   1936028192 sectors at   1869881465 (type 79)
Partition[3] :        55499 sectors at   2885681152 (type 0d)

Contents of /dev/mapper directory:
crw------T    1 root     root       10, 236 Nov 21 13:48 control

startraid mode = null (default)

Contents of /tbu/utility/dmraid-boot.txt file (from activate command on boot):
NOTICE: checking format identifier asr
NOTICE: checking format identifier hpt37x
NOTICE: checking format identifier hpt45x
NOTICE: checking format identifier jmicron
NOTICE: checking format identifier lsi
NOTICE: checking format identifier nvidia
NOTICE: checking format identifier pdc
NOTICE: checking format identifier sil
NOTICE: checking format identifier via
NOTICE: checking format identifier dos
WARN: locking /var/lock/dmraid/.lock
NOTICE: /dev/sdf: asr     discovering
NOTICE: /dev/sdf: hpt37x  discovering
NOTICE: /dev/sdf: hpt45x  discovering
NOTICE: /dev/sdf: jmicron discovering
NOTICE: /dev/sdf: lsi     discovering
NOTICE: /dev/sdf: nvidia  discovering
NOTICE: /dev/sdf: pdc     discovering
NOTICE: /dev/sdf: sil     discovering
NOTICE: /dev/sdf: via     discovering
NOTICE: /dev/sde: asr     discovering
NOTICE: /dev/sde: hpt37x  discovering
NOTICE: /dev/sde: hpt45x  discovering
NOTICE: /dev/sde: jmicron discovering
NOTICE: /dev/sde: lsi     discovering
NOTICE: /dev/sde: nvidia  discovering
NOTICE: /dev/sde: pdc     discovering
NOTICE: /dev/sde: sil     discovering
NOTICE: /dev/sde: via     discovering
NOTICE: /dev/sdd: asr     discovering
NOTICE: /dev/sdd: hpt37x  discovering
NOTICE: /dev/sdd: hpt45x  discovering
NOTICE: /dev/sdd: jmicron discovering
NOTICE: /dev/sdd: lsi     discovering
NOTICE: /dev/sdd: nvidia  discovering
NOTICE: /dev/sdd: pdc     discovering
NOTICE: /dev/sdd: sil     discovering
NOTICE: /dev/sdd: via     discovering
NOTICE: /dev/sdc: asr     discovering
NOTICE: /dev/sdc: hpt37x  discovering
NOTICE: /dev/sdc: hpt45x  discovering
NOTICE: /dev/sdc: jmicron discovering
NOTICE: /dev/sdc: lsi     discovering
NOTICE: /dev/sdc: nvidia  discovering
NOTICE: /dev/sdc: pdc     discovering
NOTICE: /dev/sdc: sil     discovering
NOTICE: /dev/sdc: via     discovering
NOTICE: /dev/sdb: asr     discovering
NOTICE: /dev/sdb: hpt37x  discovering
NOTICE: /dev/sdb: hpt45x  discovering
NOTICE: /dev/sdb: jmicron discovering
NOTICE: /dev/sdb: lsi     discovering
NOTICE: /dev/sdb: nvidia  discovering
NOTICE: /dev/sdb: pdc     discovering
NOTICE: /dev/sdb: sil     discovering
NOTICE: /dev/sdb: via     discovering
NOTICE: /dev/sda: asr     discovering
NOTICE: /dev/sda: hpt37x  discovering
NOTICE: /dev/sda: hpt45x  discovering
NOTICE: /dev/sda: jmicron discovering
NOTICE: /dev/sda: lsi     discovering
NOTICE: /dev/sda: nvidia  discovering
NOTICE: /dev/sda: pdc     discovering
NOTICE: /dev/sda: sil     discovering
NOTICE: /dev/sda: via     discovering
no raid disks with format:
"asr,hpt37x,hpt45x,jmicron,lsi,nvidia,pdc,sil,via,dos"
WARN: unlocking /var/lock/dmraid/.lock

Output of dmraid --raid_devices command:
/dev/sdf: ddf1, ".ddf1_disks", GROUP, unknown, 285155328 sectors, data@ 0
/dev/sde: ddf1, ".ddf1_disks", GROUP, ok, 285155328 sectors, data@ 0
/dev/sdd: ddf1, ".ddf1_disks", GROUP, ok, 285155328 sectors, data@ 0
/dev/sdc: ddf1, ".ddf1_disks", GROUP, ok, 285155328 sectors, data@ 0
/dev/sdb: ddf1, ".ddf1_disks", GROUP, ok, 285155328 sectors, data@ 0

Output of dmraid -s -s command:
*** Group superset .ddf1_disks
--> Subset
name   : ddf1_MegaSR   R5 #0
size   : 855465984
stride : 128
type   : raid5_ls
status : ok
subsets: 0
devs   : 4
spares : 0

Output of blkid command:
/dev/sda1: LABEL="Elements" UUID="DA1CE6D71CE6AE27" TYPE="ntfs"
/dev/sdc: UUID="LSI     M-^@M-^F^]h^Q7" TYPE="ddf_raid_member"
/dev/sdd: UUID="LSI     M-^@M-^F^]h^Q7" TYPE="ddf_raid_member"
/dev/sr0: LABEL="iflnet" TYPE="iso9660"
/dev/sdf: UUID="LSI     M-^@M-^F^]h^Q7" TYPE="ddf_raid_member"
/dev/sdg: LABEL="CS SUPPORT" UUID="4CC3-3665" TYPE="vfat"

Output of free command (memory):

             total       used       free     shared    buffers     cached
Mem:       2039280     185776    1853504          0      40996      77272
-/+ buffers/cache:      67508    1971772
Swap:            0          0          0

Output of lsmod command (loaded modules):

Module                  Size  Used by    Not tainted
sr_mod                 10387  0
cdrom                  24577  1 sr_mod
sg                     17182  0
sd_mod                 25922  1
crc_t10dif               996  1 sd_mod
usb_storage            32098  0
hid_generic              665  0
usbhid                 18838  0
hid                    60174  2 hid_generic,usbhid
pcspkr                  1227  0
i2c_i801                7257  0
igb                    98565  0
i2c_algo_bit            3747  1 igb
isci                   71218  1
i2c_core               13114  3 i2c_i801,igb,i2c_algo_bit
libsas                 47529  1 isci
ptp                     5420  1 igb
pps_core                4544  1 ptp
ehci_pci                2540  0
scsi_transport_sas     16455  2 isci,libsas
processor              14951  8
thermal_sys            12739  1 processor
hwmon                    881  2 igb,thermal_sys
button                  3413  0
dm_mod                 50912  0
ehci_hcd               28628  1 ehci_pci
edd                     5144  0


Output of lspci -knn command:

00:00.0 Host bridge [0600]: Intel Corporation Xeon E5/Core i7 DMI2
[8086:3c00] (rev 07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
00:01.0 PCI bridge [0604]: Intel Corporation Xeon E5/Core i7 IIO PCI
Express Root Port 1a [8086:3c02] (rev 07)
    Kernel driver in use: pcieport
00:02.0 PCI bridge [0604]: Intel Corporation Xeon E5/Core i7 IIO PCI
Express Root Port 2a [8086:3c04] (rev 07)
    Kernel driver in use: pcieport
00:03.0 PCI bridge [0604]: Intel Corporation Xeon E5/Core i7 IIO PCI
Express Root Port 3a in PCI Express Mode [8086:3c08] (rev 07)
    Kernel driver in use: pcieport
00:05.0 System peripheral [0880]: Intel Corporation Xeon E5/Core i7
Address Map, VTd_Misc, System Management [8086:3c28] (rev 07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
00:05.2 System peripheral [0880]: Intel Corporation Xeon E5/Core i7
Control Status and Global Errors [8086:3c2a] (rev 07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
00:05.4 PIC [0800]: Intel Corporation Xeon E5/Core i7 I/O APIC
[8086:3c2c] (rev 07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
00:1a.0 USB controller [0c03]: Intel Corporation C600/X79 series
chipset USB2 Enhanced Host Controller #2 [8086:1d2d] (rev 06)
    Subsystem: Cisco Systems Inc Device [1137:0101]
    Kernel driver in use: ehci-pci
00:1c.0 PCI bridge [0604]: Intel Corporation C600/X79 series chipset
PCI Express Root Port 1 [8086:1d10] (rev b6)
    Kernel driver in use: pcieport
00:1c.7 PCI bridge [0604]: Intel Corporation C600/X79 series chipset
PCI Express Root Port 8 [8086:1d1e] (rev b6)
    Kernel driver in use: pcieport
00:1d.0 USB controller [0c03]: Intel Corporation C600/X79 series
chipset USB2 Enhanced Host Controller #1 [8086:1d26] (rev 06)
    Subsystem: Cisco Systems Inc Device [1137:0101]
    Kernel driver in use: ehci-pci
00:1e.0 PCI bridge [0604]: Intel Corporation 82801 PCI Bridge
[8086:244e] (rev a6)
00:1f.0 ISA bridge [0601]: Intel Corporation C600/X79 series chipset
LPC Controller [8086:1d41] (rev 06)
    Subsystem: Cisco Systems Inc Device [1137:0101]
00:1f.3 SMBus [0c05]: Intel Corporation C600/X79 series chipset SMBus
Host Controller [8086:1d22] (rev 06)
    Subsystem: Cisco Systems Inc Device [1137:0101]
    Kernel driver in use: i801_smbus
00:1f.6 Signal processing controller [1180]: Intel Corporation
C600/X79 series chipset Thermal Management Controller [8086:1d24] (rev
06)
    Subsystem: Cisco Systems Inc Device [1137:0101]
01:00.0 PCI bridge [0604]: Intel Corporation C608/C606/X79 series
chipset PCI Express Upstream Port [8086:1d74] (rev 06)
    Kernel driver in use: pcieport
02:08.0 PCI bridge [0604]: Intel Corporation C608/C606/X79 series
chipset PCI Express Virtual Switch Port [8086:1d3f] (rev 06)
    Kernel driver in use: pcieport
03:00.0 Serial Attached SCSI controller [0107]: Intel Corporation C606
chipset Dual 4-Port SATA/SAS Storage Control Unit [8086:1d68] (rev 06)
    Subsystem: Cisco Systems Inc Device [1137:00b2]
    Kernel driver in use: isci
04:00.0 Ethernet controller [0200]: Intel Corporation I350 Gigabit
Network Connection [8086:1521] (rev 01)
    Subsystem: Cisco Systems Inc Device [1137:008a]
    Kernel driver in use: igb
04:00.1 Ethernet controller [0200]: Intel Corporation I350 Gigabit
Network Connection [8086:1521] (rev 01)
    Subsystem: Cisco Systems Inc Device [1137:008a]
    Kernel driver in use: igb
08:00.0 VGA compatible controller [0300]: Matrox Electronics Systems
Ltd. MGA G200e [Pilot] ServerEngines (SEP1) [102b:0522] (rev 02)
    Subsystem: Cisco Systems Inc Device [1137:0101]
7f:08.0 System peripheral [0880]: Intel Corporation Xeon E5/Core i7
QPI Link 0 [8086:3c80] (rev 07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
7f:08.3 System peripheral [0880]: Intel Corporation Xeon E5/Core i7
QPI Link Reut 0 [8086:3c83] (rev 07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
7f:08.4 System peripheral [0880]: Intel Corporation Xeon E5/Core i7
QPI Link Reut 0 [8086:3c84] (rev 07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
7f:09.0 System peripheral [0880]: Intel Corporation Xeon E5/Core i7
QPI Link 1 [8086:3c90] (rev 07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
7f:09.3 System peripheral [0880]: Intel Corporation Xeon E5/Core i7
QPI Link Reut 1 [8086:3c93] (rev 07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
7f:09.4 System peripheral [0880]: Intel Corporation Xeon E5/Core i7
QPI Link Reut 1 [8086:3c94] (rev 07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
7f:0a.0 System peripheral [0880]: Intel Corporation Xeon E5/Core i7
Power Control Unit 0 [8086:3cc0] (rev 07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
7f:0a.1 System peripheral [0880]: Intel Corporation Xeon E5/Core i7
Power Control Unit 1 [8086:3cc1] (rev 07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
7f:0a.2 System peripheral [0880]: Intel Corporation Xeon E5/Core i7
Power Control Unit 2 [8086:3cc2] (rev 07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
7f:0a.3 System peripheral [0880]: Intel Corporation Xeon E5/Core i7
Power Control Unit 3 [8086:3cd0] (rev 07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
7f:0b.0 System peripheral [0880]: Intel Corporation Xeon E5/Core i7
Interrupt Control Registers [8086:3ce0] (rev 07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
7f:0b.3 System peripheral [0880]: Intel Corporation Xeon E5/Core i7
Semaphore and Scratchpad Configuration Registers [8086:3ce3] (rev 07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
7f:0c.0 System peripheral [0880]: Intel Corporation Xeon E5/Core i7
Unicast Register 0 [8086:3ce8] (rev 07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
7f:0c.1 System peripheral [0880]: Intel Corporation Xeon E5/Core i7
Unicast Register 0 [8086:3ce8] (rev 07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
7f:0c.6 System peripheral [0880]: Intel Corporation Xeon E5/Core i7
Integrated Memory Controller System Address Decoder 0 [8086:3cf4] (rev
07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
7f:0c.7 System peripheral [0880]: Intel Corporation Xeon E5/Core i7
System Address Decoder [8086:3cf6] (rev 07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
7f:0d.0 System peripheral [0880]: Intel Corporation Xeon E5/Core i7
Unicast Register 0 [8086:3ce8] (rev 07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
7f:0d.1 System peripheral [0880]: Intel Corporation Xeon E5/Core i7
Unicast Register 0 [8086:3ce8] (rev 07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
7f:0d.6 System peripheral [0880]: Intel Corporation Xeon E5/Core i7
Integrated Memory Controller System Address Decoder 1 [8086:3cf5] (rev
07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
7f:0e.0 System peripheral [0880]: Intel Corporation Xeon E5/Core i7
Processor Home Agent [8086:3ca0] (rev 07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
7f:0e.1 Performance counters [1101]: Intel Corporation Xeon E5/Core i7
Processor Home Agent Performance Monitoring [8086:3c46] (rev 07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
    Kernel driver in use: snbep_uncore
7f:0f.0 System peripheral [0880]: Intel Corporation Xeon E5/Core i7
Integrated Memory Controller Registers [8086:3ca8] (rev 07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
7f:0f.1 System peripheral [0880]: Intel Corporation Xeon E5/Core i7
Integrated Memory Controller RAS Registers [8086:3c71] (rev 07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
7f:0f.2 System peripheral [0880]: Intel Corporation Xeon E5/Core i7
Integrated Memory Controller Target Address Decoder 0 [8086:3caa] (rev
07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
7f:0f.3 System peripheral [0880]: Intel Corporation Xeon E5/Core i7
Integrated Memory Controller Target Address Decoder 1 [8086:3cab] (rev
07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
7f:0f.4 System peripheral [0880]: Intel Corporation Xeon E5/Core i7
Integrated Memory Controller Target Address Decoder 2 [8086:3cac] (rev
07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
7f:0f.5 System peripheral [0880]: Intel Corporation Xeon E5/Core i7
Integrated Memory Controller Target Address Decoder 3 [8086:3cad] (rev
07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
7f:0f.6 System peripheral [0880]: Intel Corporation Xeon E5/Core i7
Integrated Memory Controller Target Address Decoder 4 [8086:3cae] (rev
07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
7f:10.0 System peripheral [0880]: Intel Corporation Xeon E5/Core i7
Integrated Memory Controller Channel 0-3 Thermal Control 0 [8086:3cb0]
(rev 07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
    Kernel driver in use: snbep_uncore
7f:10.1 System peripheral [0880]: Intel Corporation Xeon E5/Core i7
Integrated Memory Controller Channel 0-3 Thermal Control 1 [8086:3cb1]
(rev 07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
    Kernel driver in use: snbep_uncore
7f:10.2 System peripheral [0880]: Intel Corporation Xeon E5/Core i7
Integrated Memory Controller ERROR Registers 0 [8086:3cb2] (rev 07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
7f:10.3 System peripheral [0880]: Intel Corporation Xeon E5/Core i7
Integrated Memory Controller ERROR Registers 1 [8086:3cb3] (rev 07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
7f:10.4 System peripheral [0880]: Intel Corporation Xeon E5/Core i7
Integrated Memory Controller Channel 0-3 Thermal Control 2 [8086:3cb4]
(rev 07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
    Kernel driver in use: snbep_uncore
7f:10.5 System peripheral [0880]: Intel Corporation Xeon E5/Core i7
Integrated Memory Controller Channel 0-3 Thermal Control 3 [8086:3cb5]
(rev 07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
    Kernel driver in use: snbep_uncore
7f:10.6 System peripheral [0880]: Intel Corporation Xeon E5/Core i7
Integrated Memory Controller ERROR Registers 2 [8086:3cb6] (rev 07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
7f:10.7 System peripheral [0880]: Intel Corporation Xeon E5/Core i7
Integrated Memory Controller ERROR Registers 3 [8086:3cb7] (rev 07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
7f:11.0 System peripheral [0880]: Intel Corporation Xeon E5/Core i7
DDRIO [8086:3cb8] (rev 07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
7f:13.0 System peripheral [0880]: Intel Corporation Xeon E5/Core i7
R2PCIe [8086:3ce4] (rev 07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
7f:13.1 Performance counters [1101]: Intel Corporation Xeon E5/Core i7
Ring to PCI Express Performance Monitor [8086:3c43] (rev 07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
    Kernel driver in use: snbep_uncore
7f:13.4 Performance counters [1101]: Intel Corporation Xeon E5/Core i7
QuickPath Interconnect Agent Ring Registers [8086:3ce6] (rev 07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
7f:13.5 Performance counters [1101]: Intel Corporation Xeon E5/Core i7
Ring to QuickPath Interconnect Link 0 Performance Monitor [8086:3c44]
(rev 07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
    Kernel driver in use: snbep_uncore
7f:13.6 System peripheral [0880]: Intel Corporation Xeon E5/Core i7
Ring to QuickPath Interconnect Link 1 Performance Monitor [8086:3c45]
(rev 07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
    Kernel driver in use: snbep_uncore
80:01.0 PCI bridge [0604]: Intel Corporation Xeon E5/Core i7 IIO PCI
Express Root Port 1a [8086:3c02] (rev 07)
    Kernel driver in use: pcieport
80:03.0 PCI bridge [0604]: Intel Corporation Xeon E5/Core i7 IIO PCI
Express Root Port 3a in PCI Express Mode [8086:3c08] (rev 07)
    Kernel driver in use: pcieport
80:05.0 System peripheral [0880]: Intel Corporation Xeon E5/Core i7
Address Map, VTd_Misc, System Management [8086:3c28] (rev 07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
80:05.2 System peripheral [0880]: Intel Corporation Xeon E5/Core i7
Control Status and Global Errors [8086:3c2a] (rev 07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
80:05.4 PIC [0800]: Intel Corporation Xeon E5/Core i7 I/O APIC
[8086:3c2c] (rev 07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
ff:08.0 System peripheral [0880]: Intel Corporation Xeon E5/Core i7
QPI Link 0 [8086:3c80] (rev 07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
ff:08.3 System peripheral [0880]: Intel Corporation Xeon E5/Core i7
QPI Link Reut 0 [8086:3c83] (rev 07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
ff:08.4 System peripheral [0880]: Intel Corporation Xeon E5/Core i7
QPI Link Reut 0 [8086:3c84] (rev 07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
ff:09.0 System peripheral [0880]: Intel Corporation Xeon E5/Core i7
QPI Link 1 [8086:3c90] (rev 07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
ff:09.3 System peripheral [0880]: Intel Corporation Xeon E5/Core i7
QPI Link Reut 1 [8086:3c93] (rev 07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
ff:09.4 System peripheral [0880]: Intel Corporation Xeon E5/Core i7
QPI Link Reut 1 [8086:3c94] (rev 07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
ff:0a.0 System peripheral [0880]: Intel Corporation Xeon E5/Core i7
Power Control Unit 0 [8086:3cc0] (rev 07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
ff:0a.1 System peripheral [0880]: Intel Corporation Xeon E5/Core i7
Power Control Unit 1 [8086:3cc1] (rev 07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
ff:0a.2 System peripheral [0880]: Intel Corporation Xeon E5/Core i7
Power Control Unit 2 [8086:3cc2] (rev 07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
ff:0a.3 System peripheral [0880]: Intel Corporation Xeon E5/Core i7
Power Control Unit 3 [8086:3cd0] (rev 07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
ff:0b.0 System peripheral [0880]: Intel Corporation Xeon E5/Core i7
Interrupt Control Registers [8086:3ce0] (rev 07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
ff:0b.3 System peripheral [0880]: Intel Corporation Xeon E5/Core i7
Semaphore and Scratchpad Configuration Registers [8086:3ce3] (rev 07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
ff:0c.0 System peripheral [0880]: Intel Corporation Xeon E5/Core i7
Unicast Register 0 [8086:3ce8] (rev 07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
ff:0c.1 System peripheral [0880]: Intel Corporation Xeon E5/Core i7
Unicast Register 0 [8086:3ce8] (rev 07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
ff:0c.6 System peripheral [0880]: Intel Corporation Xeon E5/Core i7
Integrated Memory Controller System Address Decoder 0 [8086:3cf4] (rev
07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
ff:0c.7 System peripheral [0880]: Intel Corporation Xeon E5/Core i7
System Address Decoder [8086:3cf6] (rev 07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
ff:0d.0 System peripheral [0880]: Intel Corporation Xeon E5/Core i7
Unicast Register 0 [8086:3ce8] (rev 07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
ff:0d.1 System peripheral [0880]: Intel Corporation Xeon E5/Core i7
Unicast Register 0 [8086:3ce8] (rev 07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
ff:0d.6 System peripheral [0880]: Intel Corporation Xeon E5/Core i7
Integrated Memory Controller System Address Decoder 1 [8086:3cf5] (rev
07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
ff:0e.0 System peripheral [0880]: Intel Corporation Xeon E5/Core i7
Processor Home Agent [8086:3ca0] (rev 07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
ff:0e.1 Performance counters [1101]: Intel Corporation Xeon E5/Core i7
Processor Home Agent Performance Monitoring [8086:3c46] (rev 07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
    Kernel driver in use: snbep_uncore
ff:0f.0 System peripheral [0880]: Intel Corporation Xeon E5/Core i7
Integrated Memory Controller Registers [8086:3ca8] (rev 07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
ff:0f.1 System peripheral [0880]: Intel Corporation Xeon E5/Core i7
Integrated Memory Controller RAS Registers [8086:3c71] (rev 07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
ff:0f.2 System peripheral [0880]: Intel Corporation Xeon E5/Core i7
Integrated Memory Controller Target Address Decoder 0 [8086:3caa] (rev
07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
ff:0f.3 System peripheral [0880]: Intel Corporation Xeon E5/Core i7
Integrated Memory Controller Target Address Decoder 1 [8086:3cab] (rev
07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
ff:0f.4 System peripheral [0880]: Intel Corporation Xeon E5/Core i7
Integrated Memory Controller Target Address Decoder 2 [8086:3cac] (rev
07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
ff:0f.5 System peripheral [0880]: Intel Corporation Xeon E5/Core i7
Integrated Memory Controller Target Address Decoder 3 [8086:3cad] (rev
07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
ff:0f.6 System peripheral [0880]: Intel Corporation Xeon E5/Core i7
Integrated Memory Controller Target Address Decoder 4 [8086:3cae] (rev
07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
ff:10.0 System peripheral [0880]: Intel Corporation Xeon E5/Core i7
Integrated Memory Controller Channel 0-3 Thermal Control 0 [8086:3cb0]
(rev 07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
    Kernel driver in use: snbep_uncore
ff:10.1 System peripheral [0880]: Intel Corporation Xeon E5/Core i7
Integrated Memory Controller Channel 0-3 Thermal Control 1 [8086:3cb1]
(rev 07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
    Kernel driver in use: snbep_uncore
ff:10.2 System peripheral [0880]: Intel Corporation Xeon E5/Core i7
Integrated Memory Controller ERROR Registers 0 [8086:3cb2] (rev 07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
ff:10.3 System peripheral [0880]: Intel Corporation Xeon E5/Core i7
Integrated Memory Controller ERROR Registers 1 [8086:3cb3] (rev 07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
ff:10.4 System peripheral [0880]: Intel Corporation Xeon E5/Core i7
Integrated Memory Controller Channel 0-3 Thermal Control 2 [8086:3cb4]
(rev 07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
    Kernel driver in use: snbep_uncore
ff:10.5 System peripheral [0880]: Intel Corporation Xeon E5/Core i7
Integrated Memory Controller Channel 0-3 Thermal Control 3 [8086:3cb5]
(rev 07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
    Kernel driver in use: snbep_uncore
ff:10.6 System peripheral [0880]: Intel Corporation Xeon E5/Core i7
Integrated Memory Controller ERROR Registers 2 [8086:3cb6] (rev 07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
ff:10.7 System peripheral [0880]: Intel Corporation Xeon E5/Core i7
Integrated Memory Controller ERROR Registers 3 [8086:3cb7] (rev 07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
ff:11.0 System peripheral [0880]: Intel Corporation Xeon E5/Core i7
DDRIO [8086:3cb8] (rev 07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
ff:13.0 System peripheral [0880]: Intel Corporation Xeon E5/Core i7
R2PCIe [8086:3ce4] (rev 07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
ff:13.1 Performance counters [1101]: Intel Corporation Xeon E5/Core i7
Ring to PCI Express Performance Monitor [8086:3c43] (rev 07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
    Kernel driver in use: snbep_uncore
ff:13.4 Performance counters [1101]: Intel Corporation Xeon E5/Core i7
QuickPath Interconnect Agent Ring Registers [8086:3ce6] (rev 07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
ff:13.5 Performance counters [1101]: Intel Corporation Xeon E5/Core i7
Ring to QuickPath Interconnect Link 0 Performance Monitor [8086:3c44]
(rev 07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
    Kernel driver in use: snbep_uncore
ff:13.6 System peripheral [0880]: Intel Corporation Xeon E5/Core i7
Ring to QuickPath Interconnect Link 1 Performance Monitor [8086:3c45]
(rev 07)
    Subsystem: Cisco Systems Inc Device [1137:0101]
    Kernel driver in use: snbep_uncore


On Thu, Nov 21, 2013 at 12:46 PM, Martin Wilck <mwilck@arcor.de> wrote:
> On 11/20/2013 03:30 AM, NeilBrown wrote:
>
>> David: if you remove the "UUID=" part for the array leaving the
>> "container=.... member=0" as the identification, does it work?
>
> I second that. David, please try it. I'd also appreciate "mdadm -E
> /dev/sdX" output for all the RAID disks.
>

Re: MDADM 3.3 broken?

[David F.]

They also said: RAID 5 with 4 physical drives.

Re: MDADM 3.3 broken?

[David F.]

So it looks like sdf shouldn't be part of the RAID5 set - maybe it's a
stand alone drive configured on the raid controller?  I can ask if
they know...

On Thu, Nov 21, 2013 at 3:09 PM, David F. <df7729@gmail.com> wrote:
> They also said: RAID 5 with 4 physical drives.

Re: MDADM 3.3 broken?

[David F.]

Okay, got the word on what the sdf drive is:

"5th drive is configured as spare for RAID 5"

On Thu, Nov 21, 2013 at 7:06 PM, David F. <df7729@gmail.com> wrote:
> So it looks like sdf shouldn't be part of the RAID5 set - maybe it's a
> stand alone drive configured on the raid controller?  I can ask if
> they know...
>
> On Thu, Nov 21, 2013 at 3:09 PM, David F. <df7729@gmail.com> wrote:
>> They also said: RAID 5 with 4 physical drives.

Re: MDADM 3.3 broken?

[David F.]

Did that report give you everything needed?

When do you think the next release of mdadm will be ready?  Just
trying to plan the next release?

Thanks!

On Fri, Nov 22, 2013 at 10:36 AM, David F. <df7729@gmail.com> wrote:
> Okay, got the word on what the sdf drive is:
>
> "5th drive is configured as spare for RAID 5"
>
> On Thu, Nov 21, 2013 at 7:06 PM, David F. <df7729@gmail.com> wrote:
>> So it looks like sdf shouldn't be part of the RAID5 set - maybe it's a
>> stand alone drive configured on the raid controller?  I can ask if
>> they know...
>>
>> On Thu, Nov 21, 2013 at 3:09 PM, David F. <df7729@gmail.com> wrote:
>>> They also said: RAID 5 with 4 physical drives.

Re: MDADM 3.3 broken?

[Martin Wilck]

The metadata really looks all ok, but mdadm fails to assemble it
correctly. It does look like a bug to me. The problem seems to be
related to the global spare somehow. I have no idea why.

The best thing to proceed would be to run mdadm --dump on
/dev/sd[bcdef], put the output in a tgz and make it available to us
somewhere. So we could actually debut the assembly.

Regards,
Martin

Re: MDADM 3.3 broken?

[David F.]

Thanks. What would be actual command to create the --dump and where
does the file go?   testing it on the ism raid 1 didn't seem to work
for us?



On Mon, Nov 25, 2013 at 1:56 PM, Martin Wilck <mwilck@arcor.de> wrote:
> The metadata really looks all ok, but mdadm fails to assemble it
> correctly. It does look like a bug to me. The problem seems to be
> related to the global spare somehow. I have no idea why.
>
> The best thing to proceed would be to run mdadm --dump on
> /dev/sd[bcdef], put the output in a tgz and make it available to us
> somewhere. So we could actually debut the assembly.
>
> Regards,
> Martin
>
>
>

Re: MDADM 3.3 broken?

[David F.]

does it need to be a tar type file or would virtual hard drive .vmdk
files be okay?

On Mon, Nov 25, 2013 at 1:56 PM, Martin Wilck <mwilck@arcor.de> wrote:
> The metadata really looks all ok, but mdadm fails to assemble it
> correctly. It does look like a bug to me. The problem seems to be
> related to the global spare somehow. I have no idea why.
>
> The best thing to proceed would be to run mdadm --dump on
> /dev/sd[bcdef], put the output in a tgz and make it available to us
> somewhere. So we could actually debut the assembly.
>
> Regards,
> Martin
>
>
>

Re: MDADM 3.3 broken?

[Martin Wilck]

On 11/26/2013 10:59 PM, David F. wrote:
> does it need to be a tar type file or would virtual hard drive .vmdk
> files be okay?

I guess it would be ok (never tried).

Martin

Re: MDADM 3.3 broken?

[David F.]

Hi,

Got the files from the customer - you can download the zipped tar from
https://www.dropbox.com/s/aq8idkjnxyslho7/raid5data.tgz

Let me know when you get it so we can delete it.

Thanks.


On Wed, Nov 27, 2013 at 2:40 PM, Martin Wilck <mwilck@arcor.de> wrote:
> On 11/26/2013 10:59 PM, David F. wrote:
>> does it need to be a tar type file or would virtual hard drive .vmdk
>> files be okay?
>
> I guess it would be ok (never tried).
>
> Martin

Re: MDADM 3.3 broken?

[David F.]

Just wondering if the link to the files to debug the problem made it ???

On Thu, Dec 5, 2013 at 5:53 PM, David F. <df7729@gmail.com> wrote:
> Hi,
>
> Got the files from the customer - you can download the zipped tar from
>
> Let me know when you get it so we can delete it.
>
> Thanks.
>
>
> On Wed, Nov 27, 2013 at 2:40 PM, Martin Wilck <mwilck@arcor.de> wrote:
>> On 11/26/2013 10:59 PM, David F. wrote:
>>> does it need to be a tar type file or would virtual hard drive .vmdk
>>> files be okay?
>>
>> I guess it would be ok (never tried).
>>
>> Martin

Re: MDADM 3.3 broken?

[NeilBrown]

[-- Attachment #1: Type: text/plain, Size: 854 bytes --]

On Fri, 6 Dec 2013 18:28:16 -0800 "David F." <df7729@gmail.com> wrote:

> Just wondering if the link to the files to debug the problem made it ???
> 
> On Thu, Dec 5, 2013 at 5:53 PM, David F. <df7729@gmail.com> wrote:
> > Hi,
> >
> > Got the files from the customer - you can download the zipped tar from
> >
> > Let me know when you get it so we can delete it.
> >
> > Thanks.
> >
> >
> > On Wed, Nov 27, 2013 at 2:40 PM, Martin Wilck <mwilck@arcor.de> wrote:
> >> On 11/26/2013 10:59 PM, David F. wrote:
> >>> does it need to be a tar type file or would virtual hard drive .vmdk
> >>> files be okay?
> >>
> >> I guess it would be ok (never tried).
> >>
> >> Martin

I grabbed a copy, though at 1967 bytes you could have attached it to the
email safely :-)

I haven't had a chance to look at them in detail yet.

NeilBrown

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 828 bytes --]

Re: MDADM 3.3 broken?

[David F.]

Okay, I wasn't sure it was allowed / would go through.  We used tar's
sparse file support or they would be very large.

I'll have the link say up in case Martin wants a copy...

Thanks!


On Fri, Dec 6, 2013 at 7:16 PM, NeilBrown <neilb@suse.de> wrote:
> On Fri, 6 Dec 2013 18:28:16 -0800 "David F." <df7729@gmail.com> wrote:
>
>> Just wondering if the link to the files to debug the problem made it ???
>>
>> On Thu, Dec 5, 2013 at 5:53 PM, David F. <df7729@gmail.com> wrote:
>> > Hi,
>> >
>> > Got the files from the customer - you can download the zipped tar from
>> >
>> > Let me know when you get it so we can delete it.
>> >
>> > Thanks.
>> >
>> >
>> > On Wed, Nov 27, 2013 at 2:40 PM, Martin Wilck <mwilck@arcor.de> wrote:
>> >> On 11/26/2013 10:59 PM, David F. wrote:
>> >>> does it need to be a tar type file or would virtual hard drive .vmdk
>> >>> files be okay?
>> >>
>> >> I guess it would be ok (never tried).
>> >>
>> >> Martin
>
> I grabbed a copy, though at 1967 bytes you could have attached it to the
> email safely :-)
>
> I haven't had a chance to look at them in detail yet.
>
> NeilBrown

Re: MDADM 3.3 broken?

[David F.]

Hi,

Just wondering if this gave you guys everything you needed to figure
out the issue?

Also, any idea on when 3.4 may be out with the various fixes?

Thanks.


On Fri, Dec 6, 2013 at 7:16 PM, NeilBrown <neilb@suse.de> wrote:
> On Fri, 6 Dec 2013 18:28:16 -0800 "David F." <df7729@gmail.com> wrote:
>
>> Just wondering if the link to the files to debug the problem made it ???
>>
>> On Thu, Dec 5, 2013 at 5:53 PM, David F. <df7729@gmail.com> wrote:
>> > Hi,
>> >
>> > Got the files from the customer - you can download the zipped tar from
>> >
>> > Let me know when you get it so we can delete it.
>> >
>> > Thanks.
>> >
>> >
>> > On Wed, Nov 27, 2013 at 2:40 PM, Martin Wilck <mwilck@arcor.de> wrote:
>> >> On 11/26/2013 10:59 PM, David F. wrote:
>> >>> does it need to be a tar type file or would virtual hard drive .vmdk
>> >>> files be okay?
>> >>
>> >> I guess it would be ok (never tried).
>> >>
>> >> Martin
>
> I grabbed a copy, though at 1967 bytes you could have attached it to the
> email safely :-)
>
> I haven't had a chance to look at them in detail yet.
>
> NeilBrown

Re: MDADM 3.3 broken?

[NeilBrown]

[-- Attachment #1: Type: text/plain, Size: 3908 bytes --]

On Sat, 14 Dec 2013 13:01:50 -0800 "David F." <df7729@gmail.com> wrote:

> Hi,
> 
> Just wondering if this gave you guys everything you needed to figure
> out the issue?

I had everything but time.  I've now made the time and have the fix (I hope).

Please try the current HEAD of git://neil.brown.name/mdadm/
The important patch is below.

> 
> Also, any idea on when 3.4 may be out with the various fixes?

I hope to release 3.3.1 some time in February.  Based on past experience it
should be out before Easter, but no promises.

NeilBrown

From f0e876ce03a63f150bb87b2734c139bc8bb285b2 Mon Sep 17 00:00:00 2001
From: NeilBrown <neilb@suse.de>
Date: Mon, 20 Jan 2014 15:27:29 +1100
Subject: [PATCH] DDF: fix detection of failed devices during assembly.

When we call "getinfo_super", we report the working/failed status
of the particular device, and also (via the 'map') the working/failed
status of every other device that this metadata is aware of.

It is important that the way we calculate "working or failed" is
consistent.
As it is, getinfo_super_ddf() will report a spare as "working", but
every other device will see it as "failed", which leads to failure to
assemble arrays with spares.

For getinfo_super_ddf (i.e. for the container), a device is assumed
"working" unless flagged as DDF_Failed.
For getinfo_super_ddf_bvd (for a member array), a device is assumed
"failed" unless DDF_Online is set, and DDF_Failed is not set.

Reported-by: "David F." <df7729@gmail.com>
Signed-off-by: NeilBrown <neilb@suse.de>

diff --git a/super-ddf.c b/super-ddf.c
index d526d8ad3da9..4242af86fea9 100644
--- a/super-ddf.c
+++ b/super-ddf.c
@@ -1913,6 +1913,7 @@ static void getinfo_super_ddf(struct supertype *st, struct mdinfo *info, char *m
 	info->disk.major = 0;
 	info->disk.minor = 0;
 	if (ddf->dlist) {
+		struct phys_disk_entry *pde = NULL;
 		info->disk.number = be32_to_cpu(ddf->dlist->disk.refnum);
 		info->disk.raid_disk = find_phys(ddf, ddf->dlist->disk.refnum);
 
@@ -1920,12 +1921,19 @@ static void getinfo_super_ddf(struct supertype *st, struct mdinfo *info, char *m
 						  entries[info->disk.raid_disk].
 						  config_size);
 		info->component_size = ddf->dlist->size - info->data_offset;
+		if (info->disk.raid_disk >= 0)
+			pde = ddf->phys->entries + info->disk.raid_disk;
+		if (pde &&
+		    !(be16_to_cpu(pde->state) & DDF_Failed))
+			info->disk.state = (1 << MD_DISK_SYNC) | (1 << MD_DISK_ACTIVE);
+		else
+			info->disk.state = 1 << MD_DISK_FAULTY;
 	} else {
 		info->disk.number = -1;
 		info->disk.raid_disk = -1;
 //		info->disk.raid_disk = find refnum in the table and use index;
+		info->disk.state = (1 << MD_DISK_SYNC) | (1 << MD_DISK_ACTIVE);
 	}
-	info->disk.state = (1 << MD_DISK_SYNC) | (1 << MD_DISK_ACTIVE);
 
 	info->recovery_start = MaxSector;
 	info->reshape_active = 0;
@@ -1943,8 +1951,6 @@ static void getinfo_super_ddf(struct supertype *st, struct mdinfo *info, char *m
 		int i;
 		for (i = 0 ; i < map_disks; i++) {
 			if (i < info->array.raid_disks &&
-			    (be16_to_cpu(ddf->phys->entries[i].state)
-			     & DDF_Online) &&
 			    !(be16_to_cpu(ddf->phys->entries[i].state)
 			      & DDF_Failed))
 				map[i] = 1;
@@ -2017,7 +2023,11 @@ static void getinfo_super_ddf_bvd(struct supertype *st, struct mdinfo *info, cha
 		info->disk.raid_disk = cd + conf->sec_elmnt_seq
 			* be16_to_cpu(conf->prim_elmnt_count);
 		info->disk.number = dl->pdnum;
-		info->disk.state = (1<<MD_DISK_SYNC)|(1<<MD_DISK_ACTIVE);
+		info->disk.state = 0;
+		if (info->disk.number >= 0 &&
+		    (be16_to_cpu(ddf->phys->entries[info->disk.number].state) & DDF_Online) &&
+		    !(be16_to_cpu(ddf->phys->entries[info->disk.number].state) & DDF_Failed))
+			info->disk.state = (1<<MD_DISK_SYNC)|(1<<MD_DISK_ACTIVE);
 	}
 
 	info->container_member = ddf->currentconf->vcnum;

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 828 bytes --]

Re: MDADM 3.3 broken?

[Martin Wilck]

On 01/20/2014 05:34 AM, NeilBrown wrote:
> I had everything but time.  I've now made the time and have the fix (I hope).

Thanks a lot for looking into this.

Martin

Re: MDADM 3.3 broken?

[David F.]

Ok, thanks - we have sent it on for them to check.

On Sun, Jan 19, 2014 at 8:34 PM, NeilBrown <neilb@suse.de> wrote:
> On Sat, 14 Dec 2013 13:01:50 -0800 "David F." <df7729@gmail.com> wrote:
>
>> Hi,
>>
>> Just wondering if this gave you guys everything you needed to figure
>> out the issue?
>
> I had everything but time.  I've now made the time and have the fix (I hope).
>
> Please try the current HEAD of git://neil.brown.name/mdadm/
> The important patch is below.
>
>>
>> Also, any idea on when 3.4 may be out with the various fixes?
>
> I hope to release 3.3.1 some time in February.  Based on past experience it
> should be out before Easter, but no promises.
>
> NeilBrown
>
> From f0e876ce03a63f150bb87b2734c139bc8bb285b2 Mon Sep 17 00:00:00 2001
> From: NeilBrown <neilb@suse.de>
> Date: Mon, 20 Jan 2014 15:27:29 +1100
> Subject: [PATCH] DDF: fix detection of failed devices during assembly.
>
> When we call "getinfo_super", we report the working/failed status
> of the particular device, and also (via the 'map') the working/failed
> status of every other device that this metadata is aware of.
>
> It is important that the way we calculate "working or failed" is
> consistent.
> As it is, getinfo_super_ddf() will report a spare as "working", but
> every other device will see it as "failed", which leads to failure to
> assemble arrays with spares.
>
> For getinfo_super_ddf (i.e. for the container), a device is assumed
> "working" unless flagged as DDF_Failed.
> For getinfo_super_ddf_bvd (for a member array), a device is assumed
> "failed" unless DDF_Online is set, and DDF_Failed is not set.
>
> Reported-by: "David F." <df7729@gmail.com>
> Signed-off-by: NeilBrown <neilb@suse.de>
>
> diff --git a/super-ddf.c b/super-ddf.c
> index d526d8ad3da9..4242af86fea9 100644
> --- a/super-ddf.c
> +++ b/super-ddf.c
> @@ -1913,6 +1913,7 @@ static void getinfo_super_ddf(struct supertype *st, struct mdinfo *info, char *m
>         info->disk.major = 0;
>         info->disk.minor = 0;
>         if (ddf->dlist) {
> +               struct phys_disk_entry *pde = NULL;
>                 info->disk.number = be32_to_cpu(ddf->dlist->disk.refnum);
>                 info->disk.raid_disk = find_phys(ddf, ddf->dlist->disk.refnum);
>
> @@ -1920,12 +1921,19 @@ static void getinfo_super_ddf(struct supertype *st, struct mdinfo *info, char *m
>                                                   entries[info->disk.raid_disk].
>                                                   config_size);
>                 info->component_size = ddf->dlist->size - info->data_offset;
> +               if (info->disk.raid_disk >= 0)
> +                       pde = ddf->phys->entries + info->disk.raid_disk;
> +               if (pde &&
> +                   !(be16_to_cpu(pde->state) & DDF_Failed))
> +                       info->disk.state = (1 << MD_DISK_SYNC) | (1 << MD_DISK_ACTIVE);
> +               else
> +                       info->disk.state = 1 << MD_DISK_FAULTY;
>         } else {
>                 info->disk.number = -1;
>                 info->disk.raid_disk = -1;
>  //             info->disk.raid_disk = find refnum in the table and use index;
> +               info->disk.state = (1 << MD_DISK_SYNC) | (1 << MD_DISK_ACTIVE);
>         }
> -       info->disk.state = (1 << MD_DISK_SYNC) | (1 << MD_DISK_ACTIVE);
>
>         info->recovery_start = MaxSector;
>         info->reshape_active = 0;
> @@ -1943,8 +1951,6 @@ static void getinfo_super_ddf(struct supertype *st, struct mdinfo *info, char *m
>                 int i;
>                 for (i = 0 ; i < map_disks; i++) {
>                         if (i < info->array.raid_disks &&
> -                           (be16_to_cpu(ddf->phys->entries[i].state)
> -                            & DDF_Online) &&
>                             !(be16_to_cpu(ddf->phys->entries[i].state)
>                               & DDF_Failed))
>                                 map[i] = 1;
> @@ -2017,7 +2023,11 @@ static void getinfo_super_ddf_bvd(struct supertype *st, struct mdinfo *info, cha
>                 info->disk.raid_disk = cd + conf->sec_elmnt_seq
>                         * be16_to_cpu(conf->prim_elmnt_count);
>                 info->disk.number = dl->pdnum;
> -               info->disk.state = (1<<MD_DISK_SYNC)|(1<<MD_DISK_ACTIVE);
> +               info->disk.state = 0;
> +               if (info->disk.number >= 0 &&
> +                   (be16_to_cpu(ddf->phys->entries[info->disk.number].state) & DDF_Online) &&
> +                   !(be16_to_cpu(ddf->phys->entries[info->disk.number].state) & DDF_Failed))
> +                       info->disk.state = (1<<MD_DISK_SYNC)|(1<<MD_DISK_ACTIVE);
>         }
>
>         info->container_member = ddf->currentconf->vcnum;

Re: MDADM 3.3 broken?

[David F.]

They said it's now showing up as it should.  Although they are going
to do some more tests later....

Thank You.


On Mon, Jan 20, 2014 at 3:54 PM, David F. <df7729@gmail.com> wrote:
> Ok, thanks - we have sent it on for them to check.
>
> On Sun, Jan 19, 2014 at 8:34 PM, NeilBrown <neilb@suse.de> wrote:
>> On Sat, 14 Dec 2013 13:01:50 -0800 "David F." <df7729@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> Just wondering if this gave you guys everything you needed to figure
>>> out the issue?
>>
>> I had everything but time.  I've now made the time and have the fix (I hope).
>>
>> Please try the current HEAD of git://neil.brown.name/mdadm/
>> The important patch is below.
>>
>>>
>>> Also, any idea on when 3.4 may be out with the various fixes?
>>
>> I hope to release 3.3.1 some time in February.  Based on past experience it
>> should be out before Easter, but no promises.
>>
>> NeilBrown
>>
>> From f0e876ce03a63f150bb87b2734c139bc8bb285b2 Mon Sep 17 00:00:00 2001
>> From: NeilBrown <neilb@suse.de>
>> Date: Mon, 20 Jan 2014 15:27:29 +1100
>> Subject: [PATCH] DDF: fix detection of failed devices during assembly.
>>
>> When we call "getinfo_super", we report the working/failed status
>> of the particular device, and also (via the 'map') the working/failed
>> status of every other device that this metadata is aware of.
>>
>> It is important that the way we calculate "working or failed" is
>> consistent.
>> As it is, getinfo_super_ddf() will report a spare as "working", but
>> every other device will see it as "failed", which leads to failure to
>> assemble arrays with spares.
>>
>> For getinfo_super_ddf (i.e. for the container), a device is assumed
>> "working" unless flagged as DDF_Failed.
>> For getinfo_super_ddf_bvd (for a member array), a device is assumed
>> "failed" unless DDF_Online is set, and DDF_Failed is not set.
>>
>> Reported-by: "David F." <df7729@gmail.com>
>> Signed-off-by: NeilBrown <neilb@suse.de>
>>
>> diff --git a/super-ddf.c b/super-ddf.c
>> index d526d8ad3da9..4242af86fea9 100644
>> --- a/super-ddf.c
>> +++ b/super-ddf.c
>> @@ -1913,6 +1913,7 @@ static void getinfo_super_ddf(struct supertype *st, struct mdinfo *info, char *m
>>         info->disk.major = 0;
>>         info->disk.minor = 0;
>>         if (ddf->dlist) {
>> +               struct phys_disk_entry *pde = NULL;
>>                 info->disk.number = be32_to_cpu(ddf->dlist->disk.refnum);
>>                 info->disk.raid_disk = find_phys(ddf, ddf->dlist->disk.refnum);
>>
>> @@ -1920,12 +1921,19 @@ static void getinfo_super_ddf(struct supertype *st, struct mdinfo *info, char *m
>>                                                   entries[info->disk.raid_disk].
>>                                                   config_size);
>>                 info->component_size = ddf->dlist->size - info->data_offset;
>> +               if (info->disk.raid_disk >= 0)
>> +                       pde = ddf->phys->entries + info->disk.raid_disk;
>> +               if (pde &&
>> +                   !(be16_to_cpu(pde->state) & DDF_Failed))
>> +                       info->disk.state = (1 << MD_DISK_SYNC) | (1 << MD_DISK_ACTIVE);
>> +               else
>> +                       info->disk.state = 1 << MD_DISK_FAULTY;
>>         } else {
>>                 info->disk.number = -1;
>>                 info->disk.raid_disk = -1;
>>  //             info->disk.raid_disk = find refnum in the table and use index;
>> +               info->disk.state = (1 << MD_DISK_SYNC) | (1 << MD_DISK_ACTIVE);
>>         }
>> -       info->disk.state = (1 << MD_DISK_SYNC) | (1 << MD_DISK_ACTIVE);
>>
>>         info->recovery_start = MaxSector;
>>         info->reshape_active = 0;
>> @@ -1943,8 +1951,6 @@ static void getinfo_super_ddf(struct supertype *st, struct mdinfo *info, char *m
>>                 int i;
>>                 for (i = 0 ; i < map_disks; i++) {
>>                         if (i < info->array.raid_disks &&
>> -                           (be16_to_cpu(ddf->phys->entries[i].state)
>> -                            & DDF_Online) &&
>>                             !(be16_to_cpu(ddf->phys->entries[i].state)
>>                               & DDF_Failed))
>>                                 map[i] = 1;
>> @@ -2017,7 +2023,11 @@ static void getinfo_super_ddf_bvd(struct supertype *st, struct mdinfo *info, cha
>>                 info->disk.raid_disk = cd + conf->sec_elmnt_seq
>>                         * be16_to_cpu(conf->prim_elmnt_count);
>>                 info->disk.number = dl->pdnum;
>> -               info->disk.state = (1<<MD_DISK_SYNC)|(1<<MD_DISK_ACTIVE);
>> +               info->disk.state = 0;
>> +               if (info->disk.number >= 0 &&
>> +                   (be16_to_cpu(ddf->phys->entries[info->disk.number].state) & DDF_Online) &&
>> +                   !(be16_to_cpu(ddf->phys->entries[info->disk.number].state) & DDF_Failed))
>> +                       info->disk.state = (1<<MD_DISK_SYNC)|(1<<MD_DISK_ACTIVE);
>>         }
>>
>>         info->container_member = ddf->currentconf->vcnum;