mounting hixfs (Hitachi "tuned" XFS) on 2.6 kernel

mounting hixfs (Hitachi "tuned" XFS) on 2.6 kernel

[big beer]

Hello list,

I seem to find myself in the unlucky situation of having myself some
hixfs filesystems I'm trying to migrate off of.
Some background on hixfs (as I understand it).
At some point in the past, prior to purchasing a NAS company, Hitachi
decided that they could make their own NAS solution using
linux/LVM/XFS. They give you a little integrated 2.4 linux blade in
one of their storage subsystems with a nice (circa yr 2000) web
frontend to manage samba and nfs serving. The disks that are presented
to this little guy are encapsulated in LVM and formatted lv's with a
variant of XFS that is shown as hixfs on the machine. You get a very
limited shell on this guy and you have to run everything through sudo
if you want cli access (which is heavily limited).

I'm in the process of trying to get off said solution and am running
into some issues getting the file system on this black box to be
mounted/recognized on a standard 2.6 linux host.

Here is some output from some xfs tools:

box ~ # xfs_check /dev/vghorclu00/lvARRAY2
xfs_check: unexpected XFS SB magic number 0x48584653
bad superblock magic number 48584653, giving up

box ~ # xfs_repair -v /dev/vghorclu00/lvARRAY2
Phase 1 - find and verify superblock...
bad primary superblock - bad magic number !!!

attempting to find secondary superblock... Sorry, could not find valid
secondary superblock
Exiting now.

box ~ # xfs_db /dev/vghorclu00/lvARRAY2
xfs_db: unexpected XFS SB magic number 0x48584653
xfs_db> sb 0
xfs_db> p
magicnum = 0x48584653
blocksize = 4096
dblocks = 136437760
rblocks = 0
rextents = 0
uuid = bc354b71-ad02-4e47-b27a-428060c18622
logstart = 68157444
rootino = 128
rbmino = 129
rsumino = 130
rextsize = 16
agblocks = 1048576
agcount = 131
rbmblocks = 0
logblocks = 1280
versionnum = 0x20d4
sectsize = 512
inodesize = 256
inopblock = 16
fname = "\000\000\000\000\000\000\000\000\000\000\000\000"
blocklog = 12
sectlog = 9
inodelog = 8
inopblog = 4
agblklog = 20
rextslog = 0
inprogress = 0
imax_pct = 25
icount = 9486016
ifree = 3138839
fdblocks = 14074682
frextents = 0
uquotino = 131
gquotino = 132
qflags = 0
flags = 0
shared_vn = 0
inoalignmt = 2
unit = 0
width = 0
dirblklog = 0
logsectlog = 0
logsectsize = 0
logsunit = 0
features2 = 0
bad_features2 = 0
xfs_db>

I would like to be able to get this FS mounted on a box that supports
vanilla XFS.
I'm hoping that Hitachi has done something like change the magic
number so that the normal user land tools will just bail. I've got a
way to make quick copies of this FS so I am fair game to experiment on
it.

I've made some calls to Hitachi to find out what the deal is, so far
no one there has been very helpful, nor provided me with any insight
to getting these mounted. I'm thinking that since XFS is GPL'd and
they made extensions to it, and sold it, they should at least provide
source for their user land tools/kernel module for the FS. Unless they
licensed it from SGI?

I should note that on their black box solution they've got a different
set of user land tools all prefixed with "hi" (hixfs_db, hixfs_repair,
etc). While I do have ways to grab their userland tools, the kernel
module is for 2.4 so I don't think I'll have much luck just c&p
everything over.

If anyone has any ideas on what to do, and/or where to start, I'd
greatly appreciate it.

Thanks!

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: mounting hixfs (Hitachi "tuned" XFS) on 2.6 kernel

[Stan Hoeppner]

big beer put forth on 5/17/2010 7:08 PM:

> If anyone has any ideas on what to do, and/or where to start, I'd
> greatly appreciate it.

Why are you avoiding the obvious solution in favor of hacking?

-- 
Stan

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: mounting hixfs (Hitachi "tuned" XFS) on 2.6 kernel

[big beer]

>On Mon, May 17, 2010 at 6:24 PM, Stan Hoeppner <stan@hardwarefreak.com> wrote:
> big beer put forth on 5/17/2010 7:08 PM:
>
>> If anyone has any ideas on what to do, and/or where to start, I'd
>> greatly appreciate it.
>
> Why are you avoiding the obvious solution in favor of hacking?
>
> --
> Stan
>
> _______________________________________________
> xfs mailing list
> xfs@oss.sgi.com
> http://oss.sgi.com/mailman/listinfo/xfs
>
Sending back to the list this time instead of Stan directly (Sorry Stan) :)


The obvious solution for me would be a backup or rsync. Unfortunately
both of those have issues with this particular setup.

Using a backup over the network to migrate will be way too slow
(days). There are way too many files to index and this poor little nas
box is already falling over with cpu load from daily activities. I can
quickly make a mirror on the storage, and move it over to another
larger host quickly (minutes). Mounting the FS on another machine will
greatly improve the time and accuracy, as I won't have to worry about
inconsistencies as it's a block level copy.

The black-box solution is also very painful to work with, no gcc, no
automake, no rsync, etc.
I would also think that for some reason I can't think of, it would be
nice to have support for this version of XFS be available for free for
others. Some other poor sap might find some value.

So I went and changed the magic number to 0x58465342 by dumping the
1st 512 bytes off the volume, editing, and writing back, now I'm
getting "Can't verify primary superblock". Using xfs_db to look at the
other superblocks indeed still shows HXFS. Any advise how I can
find/dump/re-write one of the other superblocks? I'd like to see if I
can change another one of them if xfs_repair will run.

Thanks!

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: mounting hixfs (Hitachi "tuned" XFS) on 2.6 kernel

[Dave Chinner]

On Mon, May 17, 2010 at 08:34:14PM -0700, big beer wrote:
> >On Mon, May 17, 2010 at 6:24 PM, Stan Hoeppner <stan@hardwarefreak.com> wrote:
> > big beer put forth on 5/17/2010 7:08 PM:
> >
> >> If anyone has any ideas on what to do, and/or where to start, I'd
> >> greatly appreciate it.
> >
> > Why are you avoiding the obvious solution in favor of hacking?
> 
> The obvious solution for me would be a backup or rsync. Unfortunately
> both of those have issues with this particular setup.
> 
> Using a backup over the network to migrate will be way too slow
> (days). There are way too many files to index and this poor little nas
> box is already falling over with cpu load from daily activities. I can
> quickly make a mirror on the storage, and move it over to another
> larger host quickly (minutes). Mounting the FS on another machine will
> greatly improve the time and accuracy, as I won't have to worry about
> inconsistencies as it's a block level copy.
> 
> The black-box solution is also very painful to work with, no gcc, no
> automake, no rsync, etc.
> I would also think that for some reason I can't think of, it would be
> nice to have support for this version of XFS be available for free for
> others. Some other poor sap might find some value.
> 
> So I went and changed the magic number to 0x58465342 by dumping the
> 1st 512 bytes off the volume, editing, and writing back, now I'm
> getting "Can't verify primary superblock". Using xfs_db to look at the
> other superblocks indeed still shows HXFS. Any advise how I can
> find/dump/re-write one of the other superblocks? I'd like to see if I
> can change another one of them if xfs_repair will run.

There's no guarantee that mainline tools will work on this
filesystem. As a matter of principle, I'd be asking Hitachi for the
source code to their kernel module and userland tools and they are
required by the GPL to do supply it on request. If we have the
source code then any hidden surprises (i.e more than superblock
magic number changes) can be dealt with

Start by running 'xfs_check -F <dev>' on an unmodified image of the
filesystem. This will ignore bad superblock magic numbers. If this
does not find any significant problems or stuff it doesn't
understand, then it _might_ be safe to modify each secondary
superblock and run a modern xfs_repair on.

If you get this far modify the secondary superblocks by finding the
disk address of each superblock with xfs_db:

# for i in `seq 0 1 3`; do sudo xfs_db -x -c "sb $i" -c daddr /dev/vda; done
current daddr is 0
current daddr is 2097152
current daddr is 4194304
current daddr is 6291456
#

Just replace the "seq 0 1 3" with "seq 0 1 <number of AGs - 1>" to
suite your filesystem and "/dev/vda" with your block device.

And they can then be read via:

# dd if=/dev/vda of=sb_1.img bs=512 skip=2097152 count=1

modified, and then written by

# dd of=/dev/vda if=sb_1.img bs=512 seek=2097152 count=1

[ /me still wonders WTF the dd maintainers were thinking when they
removed iseek/oseek in favour of skip/seek - I use it these all the
time and I still have to look up the man page every time to make
sure I'm seeking on the correct stream... ]

Assuming everything is OK at this point, run 'xfs_repair -n <dev>'
first (no modify mode), and if it doesn't report anything
uncorrectable, then run without the '-n'.

Good luck!

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: mounting hixfs (Hitachi "tuned" XFS) on 2.6 kernel

[Eric Sandeen]

big beer wrote:

> So I went and changed the magic number to 0x58465342 by dumping the
> 1st 512 bytes off the volume, editing, and writing back, now I'm
> getting "Can't verify primary superblock". Using xfs_db to look at the
> other superblocks indeed still shows HXFS. Any advise how I can
> find/dump/re-write one of the other superblocks? I'd like to see if I
> can change another one of them if xfs_repair will run.


You can select each superblock and write it.

xfs_db -x /dev/blah
xfs_db> sb 1
xfs_db> write magicnum 0x58465342
xfs_db> sb 2
xfs_db> write magicnum 0x58465342
...

this could be scripted with -c <command1> -c <command2>

I don't know if they changed anything other than the magic though,
so you get to keep all pieces if it breaks!

-Eric	

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: mounting hixfs (Hitachi "tuned" XFS) on 2.6 kernel

[Stan Hoeppner]

big beer put forth on 5/17/2010 10:34 PM:
>> On Mon, May 17, 2010 at 6:24 PM, Stan Hoeppner <stan@hardwarefreak.com> wrote:
>> big beer put forth on 5/17/2010 7:08 PM:
>>
>>> If anyone has any ideas on what to do, and/or where to start, I'd
>>> greatly appreciate it.
>>
>> Why are you avoiding the obvious solution in favor of hacking?
>>
> Sending back to the list this time instead of Stan directly (Sorry Stan) :)

No problem.  I love my Tbird "reply-to-list" option. :)

> The obvious solution for me would be a backup or rsync. Unfortunately
> both of those have issues with this particular setup.
> 
> Using a backup over the network to migrate will be way too slow
> (days). There are way too many files to index and this poor little nas
> box is already falling over with cpu load from daily activities. I can
> quickly make a mirror on the storage, and move it over to another
> larger host quickly (minutes). Mounting the FS on another machine will
> greatly improve the time and accuracy, as I won't have to worry about
> inconsistencies as it's a block level copy.

> The black-box solution is also very painful to work with, no gcc, no
> automake, no rsync, etc.
> I would also think that for some reason I can't think of, it would be
> nice to have support for this version of XFS be available for free for
> others. Some other poor sap might find some value.
> 
> So I went and changed the magic number to 0x58465342 by dumping the
> 1st 512 bytes off the volume, editing, and writing back, now I'm
> getting "Can't verify primary superblock". Using xfs_db to look at the
> other superblocks indeed still shows HXFS. Any advise how I can
> find/dump/re-write one of the other superblocks? I'd like to see if I
> can change another one of them if xfs_repair will run.

Seems to me you're taking some big chances with live data.  One wrong turn
and you could hose the FS and lose all the data, yes?  I'd rather give you
recommendations not related to this current path you're taking.  Would you
please provide the model or part number of this Hitachi NAS so I can get an
idea of what exactly it is you're dealing with, and possibly offer other
solutions?

Maybe someone else here can help you pull this off via XFS.  I can't.  But
I'll gladly spend some time researching other possible solutions, mainly
getting a high capacity drive connected so you can do a cp -a or tar and be
done with this overnight, in a data-safe manner.

-- 
Stan

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: mounting hixfs (Hitachi "tuned" XFS) on 2.6 kernel

[big beer]

On Mon, May 17, 2010 at 9:37 PM, Stan Hoeppner <stan@hardwarefreak.com> wrote:
> big beer put forth on 5/17/2010 10:34 PM:
>>> On Mon, May 17, 2010 at 6:24 PM, Stan Hoeppner <stan@hardwarefreak.com> wrote:
>>> big beer put forth on 5/17/2010 7:08 PM:
>>>
>>>> If anyone has any ideas on what to do, and/or where to start, I'd
>>>> greatly appreciate it.
>>>
>>> Why are you avoiding the obvious solution in favor of hacking?
>>>
>> Sending back to the list this time instead of Stan directly (Sorry Stan) :)
>
> No problem.  I love my Tbird "reply-to-list" option. :)
>
>> The obvious solution for me would be a backup or rsync. Unfortunately
>> both of those have issues with this particular setup.
>>
>> Using a backup over the network to migrate will be way too slow
>> (days). There are way too many files to index and this poor little nas
>> box is already falling over with cpu load from daily activities. I can
>> quickly make a mirror on the storage, and move it over to another
>> larger host quickly (minutes). Mounting the FS on another machine will
>> greatly improve the time and accuracy, as I won't have to worry about
>> inconsistencies as it's a block level copy.
>
>> The black-box solution is also very painful to work with, no gcc, no
>> automake, no rsync, etc.
>> I would also think that for some reason I can't think of, it would be
>> nice to have support for this version of XFS be available for free for
>> others. Some other poor sap might find some value.
>>
>> So I went and changed the magic number to 0x58465342 by dumping the
>> 1st 512 bytes off the volume, editing, and writing back, now I'm
>> getting "Can't verify primary superblock". Using xfs_db to look at the
>> other superblocks indeed still shows HXFS. Any advise how I can
>> find/dump/re-write one of the other superblocks? I'd like to see if I
>> can change another one of them if xfs_repair will run.
>
> Seems to me you're taking some big chances with live data.  One wrong turn
> and you could hose the FS and lose all the data, yes?  I'd rather give you
> recommendations not related to this current path you're taking.  Would you
> please provide the model or part number of this Hitachi NAS so I can get an
> idea of what exactly it is you're dealing with, and possibly offer other
> solutions?
>
> Maybe someone else here can help you pull this off via XFS.  I can't.  But
> I'll gladly spend some time researching other possible solutions, mainly
> getting a high capacity drive connected so you can do a cp -a or tar and be
> done with this overnight, in a data-safe manner.
>
> --
> Stan
>
> _______________________________________________
> xfs mailing list
> xfs@oss.sgi.com
> http://oss.sgi.com/mailman/listinfo/xfs
>


It's called a eNAS, but it's really just 2 linux 2.4 blades (debian
woody), with failsafe (think heartbeat), LVM, custom XFS, samba, nfs,
and a pile of sudo available scripts that are 700 and a web based gui
to manage it.

I though about moving it via another FS on the NAS and then connecting
it to my target migration host also. The only exposed connections on
the hardware are ethernet, and it's integrated in the storage
subsystem. I'd have to call a tech  to even come take it out. So I'm
limited to something that is fiber attached to the sub-system, ok
that's fine, just means no USB disk or the like. The real problem is
that since I don't have root there is no way to create or mount any
other devices that contain another FS. The restrictive GUI/scripts
automatically creates and mounts FS's with their modified XFS version,
and there are no options to do otherwise.

I'm playing it "safe" by taking a block level copy of the luns that
are exposed to this thing, and then presenting the copy over to my
target host. I'm not brave enough to totally trash live data. I'm
going to give some of the suggestions a go with a fresh copy of the
data and see what comes of it.

Of course the 1st thing I did was call Hitachi support and ask them
what the deal is, dropping words like "GPL" and "license bound to
distribute", but to no avail. Of course I'm dealing with low level
support people that more than likely didn't work for the big H when
this thing was built. I've escalated through the channels I have
available to me.

We'll see what comes of it.

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: mounting hixfs (Hitachi "tuned" XFS) on 2.6 kernel

[Benjamin Lau]

I think this is the document you might be looking for:
http://www.hds.com/assets/pdf/open-source-licenses-applicable-to-hds-products.pdf

This outlines all of the licenses for the various bits of Hitachi Data
Systems for the FOSS components.

V. NAS File Sharing
Contact information:
Project Manager
NAS File Sharing
Hitachi Data Systems
750 Central Expressway
Santa Clara, California 95050

Please note that the source code for the NAS File Sharing kernel is
available for
download at: http://www.hitachi.co.jp/products/it/storagesolutions/
techsupport/download/index.html. Other GPL source code in the NAS File
Sharing product is available by request.

Unfortunately that link seems to give back a 404 (I understand a
smidgen of Japanese. So you may have no choice but to contact them in
a more direct manner.  But it looks like they had the code posted
someplace at some point in the past.

Hope that helps,

Ben

On Tue, May 18, 2010 at 8:19 AM, big beer <bigbeerjr@gmail.com> wrote:
> On Mon, May 17, 2010 at 9:37 PM, Stan Hoeppner <stan@hardwarefreak.com> wrote:
>> big beer put forth on 5/17/2010 10:34 PM:
>>>> On Mon, May 17, 2010 at 6:24 PM, Stan Hoeppner <stan@hardwarefreak.com> wrote:
>>>> big beer put forth on 5/17/2010 7:08 PM:
>>>>
>>>>> If anyone has any ideas on what to do, and/or where to start, I'd
>>>>> greatly appreciate it.
>>>>
>>>> Why are you avoiding the obvious solution in favor of hacking?
>>>>
>>> Sending back to the list this time instead of Stan directly (Sorry Stan) :)
>>
>> No problem.  I love my Tbird "reply-to-list" option. :)
>>
>>> The obvious solution for me would be a backup or rsync. Unfortunately
>>> both of those have issues with this particular setup.
>>>
>>> Using a backup over the network to migrate will be way too slow
>>> (days). There are way too many files to index and this poor little nas
>>> box is already falling over with cpu load from daily activities. I can
>>> quickly make a mirror on the storage, and move it over to another
>>> larger host quickly (minutes). Mounting the FS on another machine will
>>> greatly improve the time and accuracy, as I won't have to worry about
>>> inconsistencies as it's a block level copy.
>>
>>> The black-box solution is also very painful to work with, no gcc, no
>>> automake, no rsync, etc.
>>> I would also think that for some reason I can't think of, it would be
>>> nice to have support for this version of XFS be available for free for
>>> others. Some other poor sap might find some value.
>>>
>>> So I went and changed the magic number to 0x58465342 by dumping the
>>> 1st 512 bytes off the volume, editing, and writing back, now I'm
>>> getting "Can't verify primary superblock". Using xfs_db to look at the
>>> other superblocks indeed still shows HXFS. Any advise how I can
>>> find/dump/re-write one of the other superblocks? I'd like to see if I
>>> can change another one of them if xfs_repair will run.
>>
>> Seems to me you're taking some big chances with live data.  One wrong turn
>> and you could hose the FS and lose all the data, yes?  I'd rather give you
>> recommendations not related to this current path you're taking.  Would you
>> please provide the model or part number of this Hitachi NAS so I can get an
>> idea of what exactly it is you're dealing with, and possibly offer other
>> solutions?
>>
>> Maybe someone else here can help you pull this off via XFS.  I can't.  But
>> I'll gladly spend some time researching other possible solutions, mainly
>> getting a high capacity drive connected so you can do a cp -a or tar and be
>> done with this overnight, in a data-safe manner.
>>
>> --
>> Stan
>>
>> _______________________________________________
>> xfs mailing list
>> xfs@oss.sgi.com
>> http://oss.sgi.com/mailman/listinfo/xfs
>>
>
>
> It's called a eNAS, but it's really just 2 linux 2.4 blades (debian
> woody), with failsafe (think heartbeat), LVM, custom XFS, samba, nfs,
> and a pile of sudo available scripts that are 700 and a web based gui
> to manage it.
>
> I though about moving it via another FS on the NAS and then connecting
> it to my target migration host also. The only exposed connections on
> the hardware are ethernet, and it's integrated in the storage
> subsystem. I'd have to call a tech  to even come take it out. So I'm
> limited to something that is fiber attached to the sub-system, ok
> that's fine, just means no USB disk or the like. The real problem is
> that since I don't have root there is no way to create or mount any
> other devices that contain another FS. The restrictive GUI/scripts
> automatically creates and mounts FS's with their modified XFS version,
> and there are no options to do otherwise.
>
> I'm playing it "safe" by taking a block level copy of the luns that
> are exposed to this thing, and then presenting the copy over to my
> target host. I'm not brave enough to totally trash live data. I'm
> going to give some of the suggestions a go with a fresh copy of the
> data and see what comes of it.
>
> Of course the 1st thing I did was call Hitachi support and ask them
> what the deal is, dropping words like "GPL" and "license bound to
> distribute", but to no avail. Of course I'm dealing with low level
> support people that more than likely didn't work for the big H when
> this thing was built. I've escalated through the channels I have
> available to me.
>
> We'll see what comes of it.
>
> _______________________________________________
> xfs mailing list
> xfs@oss.sgi.com
> http://oss.sgi.com/mailman/listinfo/xfs
>

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: mounting hixfs (Hitachi "tuned" XFS) on 2.6 kernel

[Roger Willcocks]

The '-' character in the url in the pdf is a literal:

http://www.hitachi.co.jp/products/it/storage-solutions/techsupport/download/index.html

the page points to:

http://www2.hitachi.co.jp/products/it/rsd/download/kernel-src-05-00-02.tgz



On Tue, 2010-05-18 at 09:44 -0700, Benjamin Lau wrote:
> I think this is the document you might be looking for:
> http://www.hds.com/assets/pdf/open-source-licenses-applicable-to-hds-products.pdf
> 
> This outlines all of the licenses for the various bits of Hitachi Data
> Systems for the FOSS components.
> 
> V. NAS File Sharing
> Contact information:
> Project Manager
> NAS File Sharing
> Hitachi Data Systems
> 750 Central Expressway
> Santa Clara, California 95050
> 
> Please note that the source code for the NAS File Sharing kernel is
> available for
> download at: http://www.hitachi.co.jp/products/it/storagesolutions/
> techsupport/download/index.html. Other GPL source code in the NAS File
> Sharing product is available by request.
> 
> Unfortunately that link seems to give back a 404 (I understand a
> smidgen of Japanese. So you may have no choice but to contact them in
> a more direct manner.  But it looks like they had the code posted
> someplace at some point in the past.
> 
> Hope that helps,
> 
> Ben
> 
> On Tue, May 18, 2010 at 8:19 AM, big beer <bigbeerjr@gmail.com> wrote:
> > On Mon, May 17, 2010 at 9:37 PM, Stan Hoeppner <stan@hardwarefreak.com> wrote:
> >> big beer put forth on 5/17/2010 10:34 PM:
> >>>> On Mon, May 17, 2010 at 6:24 PM, Stan Hoeppner <stan@hardwarefreak.com> wrote:
> >>>> big beer put forth on 5/17/2010 7:08 PM:
> >>>>
> >>>>> If anyone has any ideas on what to do, and/or where to start, I'd
> >>>>> greatly appreciate it.
> >>>>
> >>>> Why are you avoiding the obvious solution in favor of hacking?
> >>>>
> >>> Sending back to the list this time instead of Stan directly (Sorry Stan) :)
> >>
> >> No problem.  I love my Tbird "reply-to-list" option. :)
> >>
> >>> The obvious solution for me would be a backup or rsync. Unfortunately
> >>> both of those have issues with this particular setup.
> >>>
> >>> Using a backup over the network to migrate will be way too slow
> >>> (days). There are way too many files to index and this poor little nas
> >>> box is already falling over with cpu load from daily activities. I can
> >>> quickly make a mirror on the storage, and move it over to another
> >>> larger host quickly (minutes). Mounting the FS on another machine will
> >>> greatly improve the time and accuracy, as I won't have to worry about
> >>> inconsistencies as it's a block level copy.
> >>
> >>> The black-box solution is also very painful to work with, no gcc, no
> >>> automake, no rsync, etc.
> >>> I would also think that for some reason I can't think of, it would be
> >>> nice to have support for this version of XFS be available for free for
> >>> others. Some other poor sap might find some value.
> >>>
> >>> So I went and changed the magic number to 0x58465342 by dumping the
> >>> 1st 512 bytes off the volume, editing, and writing back, now I'm
> >>> getting "Can't verify primary superblock". Using xfs_db to look at the
> >>> other superblocks indeed still shows HXFS. Any advise how I can
> >>> find/dump/re-write one of the other superblocks? I'd like to see if I
> >>> can change another one of them if xfs_repair will run.
> >>
> >> Seems to me you're taking some big chances with live data.  One wrong turn
> >> and you could hose the FS and lose all the data, yes?  I'd rather give you
> >> recommendations not related to this current path you're taking.  Would you
> >> please provide the model or part number of this Hitachi NAS so I can get an
> >> idea of what exactly it is you're dealing with, and possibly offer other
> >> solutions?
> >>
> >> Maybe someone else here can help you pull this off via XFS.  I can't.  But
> >> I'll gladly spend some time researching other possible solutions, mainly
> >> getting a high capacity drive connected so you can do a cp -a or tar and be
> >> done with this overnight, in a data-safe manner.
> >>
> >> --
> >> Stan
> >>
> >> _______________________________________________
> >> xfs mailing list
> >> xfs@oss.sgi.com
> >> http://oss.sgi.com/mailman/listinfo/xfs
> >>
> >
> >
> > It's called a eNAS, but it's really just 2 linux 2.4 blades (debian
> > woody), with failsafe (think heartbeat), LVM, custom XFS, samba, nfs,
> > and a pile of sudo available scripts that are 700 and a web based gui
> > to manage it.
> >
> > I though about moving it via another FS on the NAS and then connecting
> > it to my target migration host also. The only exposed connections on
> > the hardware are ethernet, and it's integrated in the storage
> > subsystem. I'd have to call a tech  to even come take it out. So I'm
> > limited to something that is fiber attached to the sub-system, ok
> > that's fine, just means no USB disk or the like. The real problem is
> > that since I don't have root there is no way to create or mount any
> > other devices that contain another FS. The restrictive GUI/scripts
> > automatically creates and mounts FS's with their modified XFS version,
> > and there are no options to do otherwise.
> >
> > I'm playing it "safe" by taking a block level copy of the luns that
> > are exposed to this thing, and then presenting the copy over to my
> > target host. I'm not brave enough to totally trash live data. I'm
> > going to give some of the suggestions a go with a fresh copy of the
> > data and see what comes of it.
> >
> > Of course the 1st thing I did was call Hitachi support and ask them
> > what the deal is, dropping words like "GPL" and "license bound to
> > distribute", but to no avail. Of course I'm dealing with low level
> > support people that more than likely didn't work for the big H when
> > this thing was built. I've escalated through the channels I have
> > available to me.
> >
> > We'll see what comes of it.
> >
> > _______________________________________________
> > xfs mailing list
> > xfs@oss.sgi.com
> > http://oss.sgi.com/mailman/listinfo/xfs
> >
> 
> _______________________________________________
> xfs mailing list
> xfs@oss.sgi.com
> http://oss.sgi.com/mailman/listinfo/xfs
-- 
Roger Willcocks <roger@filmlight.ltd.uk>

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: mounting hixfs (Hitachi "tuned" XFS) on 2.6 kernel

[big beer]

On Tue, May 18, 2010 at 9:57 AM, Roger Willcocks <roger@filmlight.ltd.uk> wrote:
> The '-' character in the url in the pdf is a literal:
>
> http://www.hitachi.co.jp/products/it/storage-solutions/techsupport/download/index.html
>
> the page points to:
>
> http://www2.hitachi.co.jp/products/it/rsd/download/kernel-src-05-00-02.tgz
>
>
>
> On Tue, 2010-05-18 at 09:44 -0700, Benjamin Lau wrote:
>> I think this is the document you might be looking for:
>> http://www.hds.com/assets/pdf/open-source-licenses-applicable-to-hds-products.pdf
>>
>> This outlines all of the licenses for the various bits of Hitachi Data
>> Systems for the FOSS components.
>>
>> V. NAS File Sharing
>> Contact information:
>> Project Manager
>> NAS File Sharing
>> Hitachi Data Systems
>> 750 Central Expressway
>> Santa Clara, California 95050
>>
>> Please note that the source code for the NAS File Sharing kernel is
>> available for
>> download at: http://www.hitachi.co.jp/products/it/storagesolutions/
>> techsupport/download/index.html. Other GPL source code in the NAS File
>> Sharing product is available by request.
>>
>> Unfortunately that link seems to give back a 404 (I understand a
>> smidgen of Japanese. So you may have no choice but to contact them in
>> a more direct manner.  But it looks like they had the code posted
>> someplace at some point in the past.
>>
>> Hope that helps,
>>
>> Ben
>>
>> On Tue, May 18, 2010 at 8:19 AM, big beer <bigbeerjr@gmail.com> wrote:
>> > On Mon, May 17, 2010 at 9:37 PM, Stan Hoeppner <stan@hardwarefreak.com> wrote:
>> >> big beer put forth on 5/17/2010 10:34 PM:
>> >>>> On Mon, May 17, 2010 at 6:24 PM, Stan Hoeppner <stan@hardwarefreak.com> wrote:
>> >>>> big beer put forth on 5/17/2010 7:08 PM:
>> >>>>
>> >>>>> If anyone has any ideas on what to do, and/or where to start, I'd
>> >>>>> greatly appreciate it.
>> >>>>
>> >>>> Why are you avoiding the obvious solution in favor of hacking?
>> >>>>
>> >>> Sending back to the list this time instead of Stan directly (Sorry Stan) :)
>> >>
>> >> No problem.  I love my Tbird "reply-to-list" option. :)
>> >>
>> >>> The obvious solution for me would be a backup or rsync. Unfortunately
>> >>> both of those have issues with this particular setup.
>> >>>
>> >>> Using a backup over the network to migrate will be way too slow
>> >>> (days). There are way too many files to index and this poor little nas
>> >>> box is already falling over with cpu load from daily activities. I can
>> >>> quickly make a mirror on the storage, and move it over to another
>> >>> larger host quickly (minutes). Mounting the FS on another machine will
>> >>> greatly improve the time and accuracy, as I won't have to worry about
>> >>> inconsistencies as it's a block level copy.
>> >>
>> >>> The black-box solution is also very painful to work with, no gcc, no
>> >>> automake, no rsync, etc.
>> >>> I would also think that for some reason I can't think of, it would be
>> >>> nice to have support for this version of XFS be available for free for
>> >>> others. Some other poor sap might find some value.
>> >>>
>> >>> So I went and changed the magic number to 0x58465342 by dumping the
>> >>> 1st 512 bytes off the volume, editing, and writing back, now I'm
>> >>> getting "Can't verify primary superblock". Using xfs_db to look at the
>> >>> other superblocks indeed still shows HXFS. Any advise how I can
>> >>> find/dump/re-write one of the other superblocks? I'd like to see if I
>> >>> can change another one of them if xfs_repair will run.
>> >>
>> >> Seems to me you're taking some big chances with live data.  One wrong turn
>> >> and you could hose the FS and lose all the data, yes?  I'd rather give you
>> >> recommendations not related to this current path you're taking.  Would you
>> >> please provide the model or part number of this Hitachi NAS so I can get an
>> >> idea of what exactly it is you're dealing with, and possibly offer other
>> >> solutions?
>> >>
>> >> Maybe someone else here can help you pull this off via XFS.  I can't.  But
>> >> I'll gladly spend some time researching other possible solutions, mainly
>> >> getting a high capacity drive connected so you can do a cp -a or tar and be
>> >> done with this overnight, in a data-safe manner.
>> >>
>> >> --
>> >> Stan
>> >>
>> >> _______________________________________________
>> >> xfs mailing list
>> >> xfs@oss.sgi.com
>> >> http://oss.sgi.com/mailman/listinfo/xfs
>> >>
>> >
>> >
>> > It's called a eNAS, but it's really just 2 linux 2.4 blades (debian
>> > woody), with failsafe (think heartbeat), LVM, custom XFS, samba, nfs,
>> > and a pile of sudo available scripts that are 700 and a web based gui
>> > to manage it.
>> >
>> > I though about moving it via another FS on the NAS and then connecting
>> > it to my target migration host also. The only exposed connections on
>> > the hardware are ethernet, and it's integrated in the storage
>> > subsystem. I'd have to call a tech  to even come take it out. So I'm
>> > limited to something that is fiber attached to the sub-system, ok
>> > that's fine, just means no USB disk or the like. The real problem is
>> > that since I don't have root there is no way to create or mount any
>> > other devices that contain another FS. The restrictive GUI/scripts
>> > automatically creates and mounts FS's with their modified XFS version,
>> > and there are no options to do otherwise.
>> >
>> > I'm playing it "safe" by taking a block level copy of the luns that
>> > are exposed to this thing, and then presenting the copy over to my
>> > target host. I'm not brave enough to totally trash live data. I'm
>> > going to give some of the suggestions a go with a fresh copy of the
>> > data and see what comes of it.
>> >
>> > Of course the 1st thing I did was call Hitachi support and ask them
>> > what the deal is, dropping words like "GPL" and "license bound to
>> > distribute", but to no avail. Of course I'm dealing with low level
>> > support people that more than likely didn't work for the big H when
>> > this thing was built. I've escalated through the channels I have
>> > available to me.
>> >
>> > We'll see what comes of it.
>> >
>> > _______________________________________________
>> > xfs mailing list
>> > xfs@oss.sgi.com
>> > http://oss.sgi.com/mailman/listinfo/xfs
>> >
>>
>> _______________________________________________
>> xfs mailing list
>> xfs@oss.sgi.com
>> http://oss.sgi.com/mailman/listinfo/xfs
> --
> Roger Willcocks <roger@filmlight.ltd.uk>
>
> _______________________________________________
> xfs mailing list
> xfs@oss.sgi.com
> http://oss.sgi.com/mailman/listinfo/xfs
>



After changing the magic number on the 1st 6 superblocks and a 6 hour
xfs_repair wait I can mount the FS. There are only a couple of things
in lost+found, more than likely because I made my copy as the FS was
still being written to. Now time to do some md5sum'ing across both
copies to see if everything is A-OK. I'll have to try to take a cold
copy of it and start over again to see if I get some better results.

Strangely enough I was told by Hitachi that they licensed XFS from
SGI, so they don't have any GPL code to give back.

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: mounting hixfs (Hitachi "tuned" XFS) on 2.6 kernel

[Stan Hoeppner]

big beer put forth on 5/18/2010 5:43 PM:

> Strangely enough I was told by Hitachi that they licensed XFS from
> SGI, so they don't have any GPL code to give back.

IIRC (and IANAL) any version of XFS, with the exception of CXFS, running on
Linux is GPL'd code.  There is no such beast as non GPL XFS on Linux.

For that SGI licensing statement to be true, they've have to be running XFS
on IRIX, which means they'd need an IRIX license as well.  Not to mention
that IRIX only runs on MIPS hardware, and only SGI MIPS hardware.  This HDS
storage blade is x86.

It seems whoever told you that is misinformed or is being disingenuous in
order to avoid assisting you.

-- 
Stan

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: mounting hixfs (Hitachi "tuned" XFS) on 2.6 kernel

[Eric Sandeen]

Stan Hoeppner wrote:
> big beer put forth on 5/18/2010 5:43 PM:
> 
>> Strangely enough I was told by Hitachi that they licensed XFS from
>> SGI, so they don't have any GPL code to give back.
> 
> IIRC (and IANAL) any version of XFS, with the exception of CXFS, running on
> Linux is GPL'd code.  There is no such beast as non GPL XFS on Linux.
> 
> For that SGI licensing statement to be true, they've have to be running XFS
> on IRIX, which means they'd need an IRIX license as well.  Not to mention
> that IRIX only runs on MIPS hardware, and only SGI MIPS hardware.  This HDS
> storage blade is x86.
> 
> It seems whoever told you that is misinformed or is being disingenuous in
> order to avoid assisting you.

Anyone with clear copyright can license the code under any license(s) they wish.
So assuming SGI had clear copyright on all linux xfs code, it is certainly
possible to offer it under an alternate license.

http://www.gnu.org/licenses/gpl-faq.html#HeardOtherLicense :

> I heard that someone got a copy of a GPL'ed program under another license.
> Is this possible?
> 
> The GNU GPL does not give users permission to attach other licenses to the 
> program. But the copyright holder for a program can release it under several 
> different licenses in parallel. One of them may be the GNU GPL.
> 
> The license that comes in your copy, assuming it was put in by the copyright 
> holder and that you got the copy legitimately, is the license that applies 
> to your copy.

-Eric

(hoping the thread doesn't dissolve into a giant gpl speculation
thread, having just fueled it further...)

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: mounting hixfs (Hitachi "tuned" XFS) on 2.6 kernel

[Stan Hoeppner]

Eric Sandeen put forth on 5/19/2010 8:53 AM:

>> The GNU GPL does not give users permission to attach other licenses to the 
>> program. But the copyright holder for a program can release it under several 
>> different licenses in parallel. One of them may be the GNU GPL.

I admit my ignorance.  I was under the impression the GPL forbade a
copyright holder from releasing under parallel licenses if the GPL was one
of them.  Thank you for educating me Eric.

It seems posts are made weekly to various lists that the reason ZFS
can't/won't be included in Linux is because the Open Solaris license is
incompatible with the GPL.  According to what you posted, there's nothing
stopping Oracle from releasing the ZFS source under multiple licenses
concurrently, including the GPL.  So I guess the "conventional wisdom"
argument relating to Linux+GPL+ZFS is false.  The reasons are "other".

> (hoping the thread doesn't dissolve into a giant gpl speculation
> thread, having just fueled it further...)

Nah.  This will be my only comment on it.  I rarely swim in the GPL pool as
I'm not a dev.  Hence my ignorance. :(  Again thanks for the education.

-- 
Stan

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: mounting hixfs (Hitachi "tuned" XFS) on 2.6 kernel

[Stan Hoeppner]

big beer put forth on 5/18/2010 10:19 AM:

> It's called a eNAS, but it's really just 2 linux 2.4 blades (debian
> woody), with failsafe (think heartbeat), LVM, custom XFS, samba, nfs,
> and a pile of sudo available scripts that are 700 and a web based gui
> to manage it.
> 
> I though about moving it via another FS on the NAS and then connecting
> it to my target migration host also. The only exposed connections on
> the hardware are ethernet, and it's integrated in the storage
> subsystem. I'd have to call a tech  to even come take it out. So I'm
> limited to something that is fiber attached to the sub-system, ok
> that's fine, just means no USB disk or the like. The real problem is
> that since I don't have root there is no way to create or mount any
> other devices that contain another FS. The restrictive GUI/scripts
> automatically creates and mounts FS's with their modified XFS version,
> and there are no options to do otherwise.
> 
> I'm playing it "safe" by taking a block level copy of the luns that
> are exposed to this thing, and then presenting the copy over to my
> target host. I'm not brave enough to totally trash live data. I'm
> going to give some of the suggestions a go with a fresh copy of the
> data and see what comes of it.

Ahh, ok.  We're not talking about some cheap single box NAS appliance here
with 4-10 local disks, but a blade "appliance" which connects via FC to SAN
LUNs on back end arrays.  And lemme guess, the sales guy pitched
"flexibility" as a main selling point.  A poor rich man's NetApp so to
speak.  Sounds like a decent setup.  Except for the fact that protocols
change, along with Samba features.  It is flexible, except in the way you
currently need it to be.

Agreed, you're not taking chances.  I didn't comprehend previously what
exactly you were doing.  Thanks for the explanation.  Snapshotting the LUNs
is obviously a very safe method.

> Of course the 1st thing I did was call Hitachi support and ask them
> what the deal is, dropping words like "GPL" and "license bound to
> distribute", but to no avail. Of course I'm dealing with low level
> support people that more than likely didn't work for the big H when
> this thing was built. I've escalated through the channels I have
> available to me.

Well, they've done what they can to lock you into their NAS solution...

Is the actual blade hardware that terribly slow?  If not, do they offer a
new "firmware" load for that blade bringing it up to a Hitachi proprietary
2.6.x kernel with a much newer Samba/NFS/etc?  I know you'd rather whip up a
solution yourself with standard FOSS stuff, but maybe this would be a viable
option for now if they offer it.  Given the blade's age they probably don't
offer anything to upgrade it.  Can't hurt to ask.

> We'll see what comes of it.

Any chance you can build a system from an old Woody installation set that
closely matches what's on the NAS?  Then copy over their hacked proprietary
HIXFS modules, mount the HIXFS LUNs and copy everything to an EXT2 formatted
LUN?  If you can get this far you're home free.  I guess the hard part will
be getting access to those modules and libraries since you can't login as
root...

-- 
Stan


_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: mounting hixfs (Hitachi "tuned" XFS) on 2.6 kernel

[Eric Sandeen]

big beer wrote:
> Hello list,
> 
> I seem to find myself in the unlucky situation of having myself some
> hixfs filesystems I'm trying to migrate off of.
> Some background on hixfs (as I understand it).
> At some point in the past, prior to purchasing a NAS company, Hitachi
> decided that they could make their own NAS solution using
> linux/LVM/XFS. They give you a little integrated 2.4 linux blade in
> one of their storage subsystems with a nice (circa yr 2000) web
> frontend to manage samba and nfs serving. The disks that are presented
> to this little guy are encapsulated in LVM and formatted lv's with a
> variant of XFS that is shown as hixfs on the machine. You get a very
> limited shell on this guy and you have to run everything through sudo
> if you want cli access (which is heavily limited).
> 
> I'm in the process of trying to get off said solution and am running
> into some issues getting the file system on this black box to be
> mounted/recognized on a standard 2.6 linux host.
> 
> Here is some output from some xfs tools:
> 
> box ~ # xfs_check /dev/vghorclu00/lvARRAY2
> xfs_check: unexpected XFS SB magic number 0x48584653
> bad superblock magic number 48584653, giving up

'HXFS' - heh.

(58 46 53 42 'XFSB' is the proper magic)

> box ~ # xfs_repair -v /dev/vghorclu00/lvARRAY2
> Phase 1 - find and verify superblock...
> bad primary superblock - bad magic number !!!
> 
> attempting to find secondary superblock... Sorry, could not find valid
> secondary superblock
> Exiting now.
> 
> box ~ # xfs_db /dev/vghorclu00/lvARRAY2
> xfs_db: unexpected XFS SB magic number 0x48584653

...

 
> I would like to be able to get this FS mounted on a box that supports
> vanilla XFS.
> I'm hoping that Hitachi has done something like change the magic
> number so that the normal user land tools will just bail. I've got a
> way to make quick copies of this FS so I am fair game to experiment on
> it.
> 
> I've made some calls to Hitachi to find out what the deal is, so far
> no one there has been very helpful, nor provided me with any insight
> to getting these mounted. I'm thinking that since XFS is GPL'd and
> they made extensions to it, and sold it, they should at least provide
> source for their user land tools/kernel module for the FS. Unless they
> licensed it from SGI?

I can't speak to that, I dunno.
 
> I should note that on their black box solution they've got a different
> set of user land tools all prefixed with "hi" (hixfs_db, hixfs_repair,
> etc). While I do have ways to grab their userland tools, the kernel
> module is for 2.4 so I don't think I'll have much luck just c&p
> everything over.
> 
> If anyone has any ideas on what to do, and/or where to start, I'd
> greatly appreciate it.
> 
> Thanks!

*shrug* could try rewriting the magic with xfs_db and then having your
way with it, see if that works.

Or, just copy off this "solution" onto a new filesystem? :)

-Eric

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs