XFS installer

XFS installer

[Brandon Barker]

My intel workstation uses SGI's XFS Installer to create partitions with XFS 
for Redhat 7.1 before it is installed, along with installing a modified 
kernel and utilitiies.  I was wondering if there is anything like this for 
Linux Mips (on Indys), or if IRIX 6.2 can create a compatible XFS partition 
for use with linux.

Thanks, 
Brandon

Re: XFS installer

[Ralf Baechle]

On Thu, Aug 09, 2001 at 08:53:11PM -0400, Brandon Barker wrote:

> My intel workstation uses SGI's XFS Installer to create partitions with XFS 
> for Redhat 7.1 before it is installed, along with installing a modified 
> kernel and utilitiies.  I was wondering if there is anything like this for 
> Linux Mips (on Indys), or if IRIX 6.2 can create a compatible XFS partition 
> for use with linux.

Don't take my answer as authoritative for XFS stuff - the directory format
of XFS on disk has somewhen been changed; the Linux version only supports
v2 while IRIX 6.2 is rather old so I think only supports v1.  Thus both
flavours are incompatible.

  Ralf

Re: XFS installer

[Seth Mos]

On Fri, 10 Aug 2001, Ralf Baechle wrote:

> On Thu, Aug 09, 2001 at 08:53:11PM -0400, Brandon Barker wrote:
> 
> > My intel workstation uses SGI's XFS Installer to create partitions with XFS 
> > for Redhat 7.1 before it is installed, along with installing a modified 
> > kernel and utilitiies.  I was wondering if there is anything like this for 
> > Linux Mips (on Indys), or if IRIX 6.2 can create a compatible XFS partition 
> > for use with linux.
> 
> Don't take my answer as authoritative for XFS stuff - the directory format
> of XFS on disk has somewhen been changed; the Linux version only supports
> v2 while IRIX 6.2 is rather old so I think only supports v1.  Thus both
> flavours are incompatible.

True, linux suports only v2. I don't know if Irix 6.2 supports the v2
mode. Another thing to watch out for is that the blocksize must equal
pagesize to be able to als mount it under linux.

This has different size on different architectures. Most are 4k but ia64
is variable from 2 - 16k or so.

This exlpained a bit in the FAQ http://oss.sgi.com/projects/xfs/faq.html

I will cc the linux-xfs list for an answer on which Irix versions can
support the v2 format. 

>   Ralf
> 

Cheers
Seth

Re: XFS installer

[Nathan Scott]

hi,

On Fri, Aug 10, 2001 at 07:38:15AM +0200, Seth Mos wrote:
> On Fri, 10 Aug 2001, Ralf Baechle wrote:
> 
> > On Thu, Aug 09, 2001 at 08:53:11PM -0400, Brandon Barker wrote:
> > 
> > > My intel workstation uses SGI's XFS Installer to create partitions with XFS 
> > > for Redhat 7.1 before it is installed, along with installing a modified 
> > > kernel and utilitiies.  I was wondering if there is anything like this for 
> > > Linux Mips (on Indys), or if IRIX 6.2 can create a compatible XFS partition 
> > > for use with linux.
> > 
> > Don't take my answer as authoritative for XFS stuff - the directory format
> > of XFS on disk has somewhen been changed; the Linux version only supports
> > v2 while IRIX 6.2 is rather old so I think only supports v1.  Thus both
> > flavours are incompatible.
> 
> True, linux suports only v2. I don't know if Irix 6.2 supports the v2
> mode. Another thing to watch out for is that the blocksize must equal
> pagesize to be able to als mount it under linux.
> 
> This has different size on different architectures. Most are 4k but ia64
> is variable from 2 - 16k or so.
> 
> This exlpained a bit in the FAQ http://oss.sgi.com/projects/xfs/faq.html
> 
> I will cc the linux-xfs list for an answer on which Irix versions can
> support the v2 format. 

V2 directories were first introduced in 6.5.5 and, AFAICT, there was
no IRIX 6.2 patch.  The default XFS blocksize on an Indy is 4K, you
shouldn't have any trouble there.  You could upgrade your Indy to a
recent IRIX (6.5 has been around for 3+ years now...) or figure out
what's wrong with the v1 directory support on Linux - can't remember
what the issues were there, but the code is all still in the tree,
might be just a "simple matter of programming".

I thought RedHat dropped MIPS as a platform awhile back, so you may
be out of luck on the installer front (I'll toss you back to the mips
list on that one, I really don't know).

cheers.

-- 
Nathan

Re: XFS installer

[Ralf Baechle]

Hi Nathan,

On Fri, Aug 10, 2001 at 04:08:19PM +1000, Nathan Scott wrote:

> I thought RedHat dropped MIPS as a platform awhile back, so you may
> be out of luck on the installer front (I'll toss you back to the mips
> list on that one, I really don't know).

Redhat supports MIPS as an embedded target; the only port of their
standard distro they ever publshed was Hardhat Linux aka Rough Cuts
which was a four CD collection of ports of Redhat to various architectures.

(The name Hardhat has been recycled later on by Montavista for their
embedded product which is totally unrelated.)

  Ralf

Re: XFS installer

[Ralf Baechle]

On Fri, Aug 10, 2001 at 07:38:15AM +0200, Seth Mos wrote:

> > of XFS on disk has somewhen been changed; the Linux version only supports
> > v2 while IRIX 6.2 is rather old so I think only supports v1.  Thus both
> > flavours are incompatible.
> 
> True, linux suports only v2. I don't know if Irix 6.2 supports the v2
> mode. Another thing to watch out for is that the blocksize must equal
> pagesize to be able to als mount it under linux.

Urg.  MIPS also supports various page sizes and so this will limit the
use of XFS for file exchange.  Is this planned to be fixed?

  Ralf

Re: XFS installer

[Nathan Scott]

hi,

On Fri, Aug 10, 2001 at 01:19:54PM +0200, Ralf Baechle wrote:
> On Fri, Aug 10, 2001 at 07:38:15AM +0200, Seth Mos wrote:
> 
> > > of XFS on disk has somewhen been changed; the Linux version only supports
> > > v2 while IRIX 6.2 is rather old so I think only supports v1.  Thus both
> > > flavours are incompatible.
> > 
> > True, linux suports only v2. I don't know if Irix 6.2 supports the v2
> > mode. Another thing to watch out for is that the blocksize must equal
> > pagesize to be able to als mount it under linux.
> 
> Urg.  MIPS also supports various page sizes and so this will limit the
> use of XFS for file exchange.  Is this planned to be fixed?
> 

Yes, for the filesystem blocksize < pagesize case anyway - slated
for Linux/XFS 1.1.  On IRIX mkfs defaults to a 4K blocksize, which
is fortunate I guess.

IRIX/XFS supports 512byte through to 64K blocksizes, so having
variable page sizes on Linux/MIPS actually makes sharing easier
on MIPS than on many other architectures.

cheers.

-- 
Nathan

Re: XFS installer

[Ralf Baechle]

On Fri, Aug 10, 2001 at 09:42:11PM +1000, Nathan Scott wrote:

> > Urg.  MIPS also supports various page sizes and so this will limit the
> > use of XFS for file exchange.  Is this planned to be fixed?
> 
> Yes, for the filesystem blocksize < pagesize case anyway - slated
> for Linux/XFS 1.1.  On IRIX mkfs defaults to a 4K blocksize, which
> is fortunate I guess.
> 
> IRIX/XFS supports 512byte through to 64K blocksizes, so having
> variable page sizes on Linux/MIPS actually makes sharing easier
> on MIPS than on many other architectures.

Some chips now support page sizes of 1kb an higher, so the case of
filesystems of large blocksize than pagesize is also going to be of some
interest.

  Ralf

Re: XFS installer

[Steve Lord]

> On Fri, Aug 10, 2001 at 07:38:15AM +0200, Seth Mos wrote:
> 
> > > of XFS on disk has somewhen been changed; the Linux version only supports
> > > v2 while IRIX 6.2 is rather old so I think only supports v1.  Thus both
> > > flavours are incompatible.
> > 
> > True, linux suports only v2. I don't know if Irix 6.2 supports the v2
> > mode. Another thing to watch out for is that the blocksize must equal
> > pagesize to be able to als mount it under linux.
> 
> Urg.  MIPS also supports various page sizes and so this will limit the
> use of XFS for file exchange.  Is this planned to be fixed?

The page size == block size will get fixed, we need to do that, but it
may take a while. Block size less than pagesize will come first, blocksize
greater than pagesize needs PAGE_CACHE_SIZE to be bumped, which appears to
be on the cards for 2.5.

V1 directories mostly work in Linux, but there are glibc getdents issues
with them. The glibc code which lseeks backwards in a directory is the issue,
if you have control over your glibc it can be fixed by using the 64 bit
version of lseek in this code. This is all because the directory offset in
V1 is a 64 bit hash value, not a 32 bit signed number.

Steve

> 
>   Ralf

Re: XFS installer

[Seth Mos]

At 08:46 10-8-2001 -0500, Steve Lord wrote:
> > On Fri, Aug 10, 2001 at 07:38:15AM +0200, Seth Mos wrote:
>V1 directories mostly work in Linux, but there are glibc getdents issues
>with them. The glibc code which lseeks backwards in a directory is the issue,
>if you have control over your glibc it can be fixed by using the 64 bit
>version of lseek in this code. This is all because the directory offset in
>V1 is a 64 bit hash value, not a 32 bit signed number.

Would this have adverse effects on existing code if this would be changed? 
Is it something that can be done without pulling out everything from 
beneath us? Does userspace need to be recompiled? Would something like this 
be needed for other architectures as well?

If this can become standard in glibc we can tell people that it is 
supported from glibc 2.2.? systems and higher.

Would a workaround in the kernel code even be a possibility.

Cheers

--
Seth
Every program has two purposes one for which
it was written and another for which it wasn't
I use the last kind.

Re: XFS installer

[Steve Lord]

> At 08:46 10-8-2001 -0500, Steve Lord wrote:
> > > On Fri, Aug 10, 2001 at 07:38:15AM +0200, Seth Mos wrote:
> >V1 directories mostly work in Linux, but there are glibc getdents issues
> >with them. The glibc code which lseeks backwards in a directory is the issue
> ,
> >if you have control over your glibc it can be fixed by using the 64 bit
> >version of lseek in this code. This is all because the directory offset in
> >V1 is a 64 bit hash value, not a 32 bit signed number.
> 
> Would this have adverse effects on existing code if this would be changed? 
> Is it something that can be done without pulling out everything from 
> beneath us? Does userspace need to be recompiled? Would something like this 
> be needed for other architectures as well?

This is only needed if reasonable V1 directory support is required on
Linux, without it you can get files missing in directories etc. In fact
even with it this can still happen. This was the whole reason we made
V2 the default on Linux (we finally won the battle to make it the
default on Irix too).

> 
> If this can become standard in glibc we can tell people that it is 
> supported from glibc 2.2.? systems and higher.

All attempts to get glibc to budge on this have failed.

> 
> Would a workaround in the kernel code even be a possibility.

I am not sure, no time to think about it right now...

Steve

> 
> Cheers
> 
> --
> Seth
> Every program has two purposes one for which
> it was written and another for which it wasn't
> I use the last kind.

Re: XFS installer

[Ralf Baechle]

On Fri, Aug 10, 2001 at 08:46:26AM -0500, Steve Lord wrote:

> The page size == block size will get fixed, we need to do that, but it
> may take a while. Block size less than pagesize will come first, blocksize
> greater than pagesize needs PAGE_CACHE_SIZE to be bumped, which appears to
> be on the cards for 2.5.
> 
> V1 directories mostly work in Linux, but there are glibc getdents issues
> with them. The glibc code which lseeks backwards in a directory is the issue,
> if you have control over your glibc it can be fixed by using the 64 bit
> version of lseek in this code. This is all because the directory offset in
> V1 is a 64 bit hash value, not a 32 bit signed number.

So in other words that means using kernel 2.4 / glibc 2.2 and for 32-bit
systems building applications with the large file API.

  Ralf

Re: XFS installer

[Steve Lord]

> On Fri, Aug 10, 2001 at 08:46:26AM -0500, Steve Lord wrote:
> 
> > The page size == block size will get fixed, we need to do that, but it
> > may take a while. Block size less than pagesize will come first, blocksize
> > greater than pagesize needs PAGE_CACHE_SIZE to be bumped, which appears to
> > be on the cards for 2.5.
> > 
> > V1 directories mostly work in Linux, but there are glibc getdents issues
> > with them. The glibc code which lseeks backwards in a directory is the issu
> e,
> > if you have control over your glibc it can be fixed by using the 64 bit
> > version of lseek in this code. This is all because the directory offset in
> > V1 is a 64 bit hash value, not a 32 bit signed number.
> 
> So in other words that means using kernel 2.4 / glibc 2.2 and for 32-bit
> systems building applications with the large file API.

No, it all depends on which version of the glibc getdents code you have,
there have been several, I cannot remember what it looks like now.
here is an old patch:

--- glibc-2.1.3/sysdeps/unix/sysv/linux/getdents.c.orig	Fri May  7 09:41:08 1999
+++ glibc-2.1.3/sysdeps/unix/sysv/linux/getdents.c	Thu Mar 23 11:49:11 2000
@@ -53,6 +53,9 @@
 #ifdef GETDENTS64
 # define __getdents __getdents64
 # define dirent dirent64
+# undef off_t
+# define off_t off64_t
+# define __lseek __lseek64
 #endif
 
 /* The problem here is that we cannot simply read the next NBYTES
@@ -107,6 +110,33 @@
 	}
 
       last_offset = kdp->d_off;
+
+#ifdef GETDENTS64
+	if (sizeof(__kernel_off_t) < sizeof(off64_t)) {
+	    /*
+	     * The kernel returns d_off as a 'cookie' that can be used
+	     * in an lseek() to re-position the directory stream.
+	     * "d_off" is signed by current definition of "__kernel_off_t",
+	     * which is consistent with lseek() "off_t" semantics.
+	     * Some file systems, most commonly NFS, may return "d_off"
+	     * 'cookies' that use all 32 bits, or more specifically the
+	     * sign bit.
+	     * This means we need to be careful how we save "last_offset"
+	     * in the case where "last_offset" is defined to be a larger
+	     * container than "__kernel_off_t", we want to prevent the sign
+	     * extension; it's only the lower 32 bits that we want to be
+	     * able to pass back in via lseek64().
+	     */
+	    switch(sizeof(__kernel_off_t)) {
+	    case 4:
+		{
+		    last_offset = (u_int32_t)kdp->d_off;
+
+		    break;
+		}
+	    }
+	}
+#endif
       dp->d_ino = kdp->d_ino;
       dp->d_off = kdp->d_off;
       dp->d_reclen = new_reclen;


I am pretty sure this patch is useless with a 2.2 glibc.

The issue is not so much the application interface as the fact that glibc has
the nasty habit of lseeking back in the directory because its dirent and
the kernels are not the same size. 

Steve