Move md raid5 from intel to sparc?

Move md raid5 from intel to sparc?

[Matthew Mitchell]

Hello everyone,

In the interest of keeping around some otherwise-functional Sparcs, I 
was asked if it would be possible to move a linux md raid set from an 
Intel box to a Sun box.  I said, "Sure," and then immediately wondered 
about byte ordering -- will this work?  Take the SCSI box off of the 
intel, plug it into the sparc, and try to run the array...

Then my next thought was that someone here would know the answer. :)

Thanks!

-- 
Matthew Mitchell
Systems Programmer/Administrator            matthew@geodev.com
Geophysical Development Corporation         phone 713 782 1234
1 Riverway Suite 2100, Houston, TX  77056     fax 713 782 1829

Re: Move md raid5 from intel to sparc?

[Mike Tran]

On Wed, 2003-06-11 at 10:02, Matthew Mitchell wrote:
> Hello everyone,
> 
> In the interest of keeping around some otherwise-functional Sparcs, I 
> was asked if it would be possible to move a linux md raid set from an 
> Intel box to a Sun box.  I said, "Sure," and then immediately wondered 
> about byte ordering -- will this work?  Take the SCSI box off of the 
> intel, plug it into the sparc, and try to run the array...
> 
> Then my next thought was that someone here would know the answer. :)
> 

It will not work. The on-disk MD superblock is written using cpu arch
format.  Furthermore, let's say you are able to re-create the raid5
array after moving it to the Sun box.  What about the filesystem and
data?  At least you need a filesystem which manipulates its metadata in
neutral format for this kind of "move" to work.

Regards,
Mike Tran

Re: Move md raid5 from intel to sparc?

[Matthew Mitchell]

Mike Tran wrote:
> On Wed, 2003-06-11 at 10:02, Matthew Mitchell wrote:
> 
>>Hello everyone,
>>
>>In the interest of keeping around some otherwise-functional Sparcs, I 
>>was asked if it would be possible to move a linux md raid set from an 
>>Intel box to a Sun box.  I said, "Sure," and then immediately wondered 
>>about byte ordering -- will this work?  Take the SCSI box off of the 
>>intel, plug it into the sparc, and try to run the array...
>>
>>Then my next thought was that someone here would know the answer. :)
>>
> 
> 
> It will not work. The on-disk MD superblock is written using cpu arch
> format.  Furthermore, let's say you are able to re-create the raid5
> array after moving it to the Sun box.  What about the filesystem and
> data?  At least you need a filesystem which manipulates its metadata in
> neutral format for this kind of "move" to work.

As I feared.

What _about_ the filesystem and data, though?  Some filesystems are 
certainly written in a known byte-order, like ISO 9660.  Are there any 
of (ext3, XFS, JFS, reiserfs) that this is true for?  Or are they all 
written in cpu native byte-order?

-- 
Matthew Mitchell
Systems Programmer/Administrator            matthew@geodev.com
Geophysical Development Corporation         phone 713 782 1234
1 Riverway Suite 2100, Houston, TX  77056     fax 713 782 1829

Re: Move md raid5 from intel to sparc?

[Matt Stegman]

On 11 Jun, Matthew Mitchell wrote:
> What _about_ the filesystem and data, though?  Some filesystems are 
> certainly written in a known byte-order, like ISO 9660.  Are there any 
> of (ext3, XFS, JFS, reiserfs) that this is true for?  Or are they all 
> written in cpu native byte-order?

I know XFS always uses big-endian on disk; one of SGI's selling points
was that you can move disks back and forth between MIPS and Intel
computers.  I think ext2/3 also uses the same format between big and
little endian archs.

-- 
--
    -Matt Stegman

Re: Move md raid5 from intel to sparc?

[Mike Tran]

On Wed, 2003-06-11 at 13:03, Matthew Mitchell wrote:

> What _about_ the filesystem and data, though?  Some filesystems are 
> certainly written in a known byte-order, like ISO 9660.  Are there any 
> of (ext3, XFS, JFS, reiserfs) that this is true for?  Or are they all 
> written in cpu native byte-order?

Just look at the current releases of kernel, 2.4 and 2.5, all of the
above filesytems seem to write metadata using one known byte-order (ei.
always BE or always LE).  Now all we need is testing.

-Mike Tran

RE: Move md raid5 from intel to sparc?

[Cal Webster]

I'd be interested in how to get RAID5 to "swing both ways", from Sparc to
Intel and back to Sparc with minimal disruption in service.

I've got an Ultra II running Aurora Linux which was recently raised from the
dead after a case of heat stroke. During the time it was down I attempted to
connect it to an Intel machine running RedHat Linux, both running 2.4
kernels. As expected, it came up with errors.

The logs showed that the raid superblock didn't match the device names on
the new system (i.e /dev/sda1 became /dev/sdd1). To fix this I knew I'd have
to edit the RAID table using "mdadm". I was sure I could do this since I've
done it before when I setup the raid. What I wasn't sure about was whether
the Intel kernel could read the Sun disk labels and use them as is. Since I
wasn't able to get a good backup of the RAID prior to the failure, I didn't
want to chance corrupting the disk labels.

Now that our Sparc is back on-line, I'd like to get a procedure in place to
allow connecting the RAID to an Intel Linux machine in case our Sparc is
stricken with another case of SIDS (Sudden Information Denial Syndrome).

--Cal Webster
Network Manager
NavAir TSD ISEO CPNC
cwebster@ec.rr.com

> -----Original Message-----
> From: linux-raid-owner@vger.kernel.org
> [mailto:linux-raid-owner@vger.kernel.org]On Behalf Of Matthew Mitchell
> Sent: Wednesday, June 11, 2003 10:02 AM
> To: linux-raid@vger.kernel.org
> Subject: Move md raid5 from intel to sparc?
>
>
> Hello everyone,
>
> In the interest of keeping around some otherwise-functional Sparcs, I
> was asked if it would be possible to move a linux md raid set from an
> Intel box to a Sun box.  I said, "Sure," and then immediately wondered
> about byte ordering -- will this work?  Take the SCSI box off of the
> intel, plug it into the sparc, and try to run the array...
>
> Then my next thought was that someone here would know the answer. :)
>
> Thanks!
>
> --
> Matthew Mitchell
> Systems Programmer/Administrator            matthew@geodev.com
> Geophysical Development Corporation         phone 713 782 1234
> 1 Riverway Suite 2100, Houston, TX  77056     fax 713 782 1829
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Move md raid5 from intel to sparc?

[3tcdgwg3]

I am doing a sparc thingin. I have to build
the disk images on a x86 system, and move the
disk to my sparc, for debug/test.  Everhting works
fine, except the MD's SB.  I tweaked a bit on MD
driver, then everything is good.  I would think that
MD should take care the SB format in next release.


----- Original Message ----- 
From: "Mike Tran" <mhtran@us.ibm.com>
To: "Matthew Mitchell" <matthew@geodev.com>
Cc: <linux-raid@vger.kernel.org>
Sent: Wednesday, June 11, 2003 12:17 PM
Subject: Re: Move md raid5 from intel to sparc?


> On Wed, 2003-06-11 at 13:03, Matthew Mitchell wrote:
> 
> > What _about_ the filesystem and data, though?  Some filesystems are 
> > certainly written in a known byte-order, like ISO 9660.  Are there any 
> > of (ext3, XFS, JFS, reiserfs) that this is true for?  Or are they all 
> > written in cpu native byte-order?
> 
> Just look at the current releases of kernel, 2.4 and 2.5, all of the
> above filesytems seem to write metadata using one known byte-order (ei.
> always BE or always LE).  Now all we need is testing.
> 
> -Mike Tran
> 
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Move md raid5 from intel to sparc?

[Mike Tran]

On Wed, 2003-06-11 at 14:31, 3tcdgwg3 wrote:
> I am doing a sparc thingin. I have to build
> the disk images on a x86 system, and move the
> disk to my sparc, for debug/test.  Everhting works
> fine, except the MD's SB.  I tweaked a bit on MD
> driver, then everything is good.  I would think that
> MD should take care the SB format in next release.
> 

This work should start from the MD kernel driver.  That is the MD driver
has to write the SB in one known format and converts the SB to the
native cpu after reading it from disk.  Next, the MD tools (mdadm,
mdreconf, etc.) will need to follow.

Neil B, Do you have this work item in mind?

-Mike Tran

Re: Move md raid5 from intel to sparc?

[Paul Clements]

Mike Tran wrote:
> 
> On Wed, 2003-06-11 at 14:31, 3tcdgwg3 wrote:
> > I am doing a sparc thingin. I have to build
> > the disk images on a x86 system, and move the
> > disk to my sparc, for debug/test.  Everhting works
> > fine, except the MD's SB.  I tweaked a bit on MD
> > driver, then everything is good.  I would think that
> > MD should take care the SB format in next release.
> >
> 
> This work should start from the MD kernel driver.  That is the MD driver
> has to write the SB in one known format and converts the SB to the
> native cpu after reading it from disk.  Next, the MD tools (mdadm,
> mdreconf, etc.) will need to follow.
> 
> Neil B, Do you have this work item in mind?

Neil's new version 1 superblock, which is present in the 2.5 kernel,
addresses the byte ordering issues that are a problem in the current
0.90 superblock of the 2.4 md driver.
The new superblock code stores all numeric values in little endian
on-disk and converts them to CPU endianness before use.

--
Paul

Re: Move md raid5 from intel to sparc?

[Matti Aarnio]

On Wed, Jun 11, 2003 at 02:08:28PM -0500, Matt Stegman wrote:
> On 11 Jun, Matthew Mitchell wrote:
> > What _about_ the filesystem and data, though?  Some filesystems are 
> > certainly written in a known byte-order, like ISO 9660.  Are there any 
> > of (ext3, XFS, JFS, reiserfs) that this is true for?  Or are they all 
> > written in cpu native byte-order?
> 
> I know XFS always uses big-endian on disk; one of SGI's selling points
> was that you can move disks back and forth between MIPS and Intel
> computers.  I think ext2/3 also uses the same format between big and
> little endian archs.

  EXT2 has two on-disk byte orders, and kernel code understands both.
  The MD physical format could be handled similarly.  Present code
  (drivers/md/md.c) becomes rather upset when it sees unexpected things.

  This quick review was based on 2.4.20, and doesn't say anything
  about 2.5 series.

  Just for the kicks from linux/fs/ext2/super.c:
     *  Big-endian to little-endian byte-swapping/bitmaps by
     *        David S. Miller (davem@caip.rutgers.edu), 1995

  Yes, it was made to support same disk at SPARC and at i386  ;-)

> -- 
>     -Matt Stegman

/Matti Aarnio

Re: Move md raid5 from intel to sparc?

[Tom Vier]

On Wed, Jun 11, 2003 at 11:44:21PM +0300, Matti Aarnio wrote:
>   EXT2 has two on-disk byte orders, and kernel code understands both.
>   The MD physical format could be handled similarly.  Present code
>   (drivers/md/md.c) becomes rather upset when it sees unexpected things.

many years ago, there was ext2-be on m68k and ppc. that was an exception,
though, used before the proper cpu_to_le macros were added to the ext2 code.
ext2 is specified as le and always has been.

-- 
Tom Vier <tmv@comcast.net>
DSA Key ID 0xE6CB97DA

Re: Move md raid5 from intel to sparc?

[Matthew Mitchell]

Paul Clements wrote:
> Mike Tran wrote:
> 
>>On Wed, 2003-06-11 at 14:31, 3tcdgwg3 wrote:
>>
>>>I am doing a sparc thingin. I have to build
>>>the disk images on a x86 system, and move the
>>>disk to my sparc, for debug/test.  Everhting works
>>>fine, except the MD's SB.  I tweaked a bit on MD
>>>driver, then everything is good.  I would think that
>>>MD should take care the SB format in next release.
>>>
>>
>>This work should start from the MD kernel driver.  That is the MD driver
>>has to write the SB in one known format and converts the SB to the
>>native cpu after reading it from disk.  Next, the MD tools (mdadm,
>>mdreconf, etc.) will need to follow.
>>
>>Neil B, Do you have this work item in mind?
> 
> 
> Neil's new version 1 superblock, which is present in the 2.5 kernel,
> addresses the byte ordering issues that are a problem in the current
> 0.90 superblock of the 2.4 md driver.
> The new superblock code stores all numeric values in little endian
> on-disk and converts them to CPU endianness before use.

Any possibility of a backport?  If it's not too hairy I don't mind 
attempting it myself, but I haven't even looked at md in 2.5.

-- 
Matthew Mitchell
Systems Programmer/Administrator            matthew@geodev.com
Geophysical Development Corporation         phone 713 782 1234
1 Riverway Suite 2100, Houston, TX  77056     fax 713 782 1829

Re: Move md raid5 from intel to sparc?

[Paul Clements]

Matthew Mitchell wrote:

> Paul Clements wrote:

> > Neil's new version 1 superblock, which is present in the 2.5 kernel,
> > addresses the byte ordering issues that are a problem in the current
> > 0.90 superblock of the 2.4 md driver.
> > The new superblock code stores all numeric values in little endian
> > on-disk and converts them to CPU endianness before use.

> Any possibility of a backport?  If it's not too hairy I don't mind
> attempting it myself, but I haven't even looked at md in 2.5.

If you just wanted to rip out the superblock code from the 2.5 md.c and
use that in 2.4, it probably wouldn't be too bad. The superblock
handling code has been improved and modularized quite a bit, and the md
driver can now handle various different superblock formats.

--
Paul

Re: Move md raid5 from intel to sparc?

[Neil Brown]

On Thursday June 12, matthew@geodev.com wrote:
> Paul Clements wrote:
> > Mike Tran wrote:
> > 
> >>On Wed, 2003-06-11 at 14:31, 3tcdgwg3 wrote:
> >>
> >>>I am doing a sparc thingin. I have to build
> >>>the disk images on a x86 system, and move the
> >>>disk to my sparc, for debug/test.  Everhting works
> >>>fine, except the MD's SB.  I tweaked a bit on MD
> >>>driver, then everything is good.  I would think that
> >>>MD should take care the SB format in next release.
> >>>
> >>
> >>This work should start from the MD kernel driver.  That is the MD driver
> >>has to write the SB in one known format and converts the SB to the
> >>native cpu after reading it from disk.  Next, the MD tools (mdadm,
> >>mdreconf, etc.) will need to follow.
> >>
> >>Neil B, Do you have this work item in mind?
> > 
> > 
> > Neil's new version 1 superblock, which is present in the 2.5 kernel,
> > addresses the byte ordering issues that are a problem in the current
> > 0.90 superblock of the 2.4 md driver.
> > The new superblock code stores all numeric values in little endian
> > on-disk and converts them to CPU endianness before use.
> 
> Any possibility of a backport?  If it's not too hairy I don't mind 
> attempting it myself, but I haven't even looked at md in 2.5.
> 

A back-port is really out of the question.
There was massive changes to md in 2.5 in order to prepare the code
for this (and other) extensions.
In 2.4, knowledge of the superblock layout is spread throughout the
code.  In 2.5 it is localised, making multiple superblock formats a
possibility.

If you wanted something like this in 2.4, you would need to get the
code that reads the superblock to look at it, determine what format it
was, and then translate it inplace to the required format (e.g. do
byte swapping).  Then when writing out you would need to translate it
to the original format before writing.
It is certainly do-able, but it wouldn't be a back-port.  It would be
writing brand new code to do a vaguely similar task in a completely
different way.

And to answer the question:
> >>Neil B, Do you have this work item in mind?

No.  I am not really interested in anything other than bug fixes for
2.4 md code.  If someone else wants to write some improvements, I'm
happy to review them and sponsor them into the kernel, but I'm not
interested in pushing anything myself.

NeilBrown