statfs64 missing

statfs64 missing

[Andi Kleen]

I notice 2.5 has 64bit sector_t but no statfs64() system call for 32bit
architectures. How is df supposed to report it? statfs uses 32bit
block counts.

The currently limit depends on the block size, the bigger your block
size the more bytes it can report.

This problem already exists on 2.4. You can actually access NFS servers
which have more than 2TB of disk space. NFS uses the local write size
as block size. When you are lucky then 0xfffffffff*wsize is bigger 
than what your NFS server reports. If not you get wrong results.
The only workaround currently is to increase wsize, but that has its
limits too.

Fixing it properly probably requires statfs64(). Any reason why this 
was not included in the sector_t patchkit ? 

-Andi

Re: statfs64 missing

[Trond Myklebust]

>>>>> " " == Andi Kleen <ak@muc.de> writes:

     > This problem already exists on 2.4. You can actually access NFS
     > servers which have more than 2TB of disk space. NFS uses the
     > local write size as block size. When you are lucky then
     > 0xfffffffff*wsize is bigger than what your NFS server
     > reports. If not you get wrong results.  The only workaround
     > currently is to increase wsize, but that has its limits too.

     > Fixing it properly probably requires statfs64(). Any reason why
     > this was not included in the sector_t patchkit ?

If fixing NFS is the main concern, and we're adding a new syscall,
could someone also add in an equivalent of the Solaris
etc. 'f_frsize'.

Reporting the underlying local filessystem block size can be of use
for some applications. On NFS this value usually differs from the
'optimal transfer block size' aka. f_bsize.

Cheers,
  Trond

Re: statfs64 missing

[Andi Kleen]

On Thu, Oct 17, 2002 at 01:49:27AM +0200, Peter Chubb wrote:
>    2.  Change the in-kernel definition of struct statfs to use 64-bit
>        types, (if CONFIG_LBD) and convert at syscall time depending on
>        whether statfs or statfs64 is called (or even, allow glibc to
>        do it, but that's an externally-visible interface change)
> 
> Which is cleaner?  Personally I dislike multiplying interfaces for no
> good reason, so I lean to (2) --- keep the in-kernel paths 64-bit
> clean.

I prefer (2) too.

Regarding the new fields Trond proposed: there is the issue of 64bit
architectures. They already use 64bit block counts, so in theory they 
don't need a statfs64(). But if we wanted to add new fields patterned
after Solaris 64bit would need the new system call too.

So it boils down to if the new fields are important enough to justify
the pain they cause on 64bit.

(I ran into a similar issue with my nanosecond stat patchkit - 
alpha stat is 64bit clean, but doesn't have the padding for ns fields
added used in later ports)

-Andi

Re: statfs64 missing

[Benjamin LaHaise]

On Thu, Oct 17, 2002 at 02:01:11AM +0200, Andi Kleen wrote:
...
> So it boils down to if the new fields are important enough to justify
> the pain they cause on 64bit.
> 
> (I ran into a similar issue with my nanosecond stat patchkit - 
> alpha stat is 64bit clean, but doesn't have the padding for ns fields
> added used in later ports)

If any new stat() type syscalls are added, make sure that a length parameter 
of the structure gets passed in from userland, as that way we will be able 
to extend the available information without introducing yet another syscall 
on every arch (this has happened enough times now that we should try to get 
it right).

		-ben
-- 
"Do you seek knowledge in time travel?"

statfs64 missing

[Steven French]

Interesting question for other distributed filesystems as well, not just
NFS.
There are two calls sent in the CIFS case to get this info.  One requesting
"FILE_SYSTEM_ATTRIBUTE_INFO" to get f_namelen (which presumably does not
vary so does not need to be requeried every statfs and the field length is
big enough
as is) and "FILE_SYSTEM_INFO" which returns 64 bit quantities for Total and
Free
"Allocation Units" and  then a  32 bit quantity for Sectors per Allocation
Unit
and a 32 bit quantity for bytes for sector.   With big SAN based arrays of
disks
running under some of the high end CIFS based server appliances from
EMC, NetApp etc. it would not be surprising to me to see an overflow
problem for
the 32 bit statfs fields today (mapping from the values in the
FILE_SYSTEM_INFO
returned by the server to statfs struct on i386 clients) unless the local
fs lies
about the block size.   For the cifs vfs adding a statfs64 func would
certainly
be technically feasible from the protocol's perspective and pretty easy.

> 2.5 has 64bit sector_t but no statfs64() system call for
>32bit architectures. How is df supposed to report it? statfs
> uses 32bit block counts.
>
>The currently limit depends on the block size, the bigger your
>block size the more bytes it can report.
>
> You can actually access NFS servers which have more
> than 2TB of disk space. NFS uses the local write size
> as block size. When you are lucky then 0xfffffffff*wsize
> is bigger than what your NFS server reports. If not
> you get wrong results.
>
> -Andi


Steve French
Senior Software Engineer
Linux Technology Center - IBM Austin
phone: 512-838-2294
email: sfrench@us.ibm.com

Re: statfs64 missing

[Petr Vandrovec]

On 17 Oct 02 at 15:41, Benjamin LaHaise wrote:
> On Thu, Oct 17, 2002 at 02:01:11AM +0200, Andi Kleen wrote:
> ...
> > So it boils down to if the new fields are important enough to justify
> > the pain they cause on 64bit.
> > 
> > (I ran into a similar issue with my nanosecond stat patchkit - 
> > alpha stat is 64bit clean, but doesn't have the padding for ns fields
> > added used in later ports)
> 
> If any new stat() type syscalls are added, make sure that a length parameter 
> of the structure gets passed in from userland, as that way we will be able 
> to extend the available information without introducing yet another syscall 
> on every arch (this has happened enough times now that we should try to get 
> it right).

And if VFS could get access to dentry which was used for statfs,
I (and ncpfs) will be very happy, as then I can report space available
for user after applying directory restrictions just through normal
statfs call, instead of using own special call.

I understand that then it becomes more inode than superblock operation,
and that it is too late for 2.6...
                                        Thanks,
                                            Petr Vandrovec
                                            vandrove@vc.cvut.cz
                                            

Re: statfs64 missing

[Andreas Dilger]

On Oct 17, 2002  15:28 -0500, Steven French wrote:
> With big SAN based arrays of disks running under some of the high end
> CIFS based server appliances from EMC, NetApp etc. it would not be
> surprising to me to see an overflow problem for the 32 bit statfs fields
> today (mapping from the values in the FILE_SYSTEM_INFO returned by the
> server to statfs struct on i386 clients) unless the local fs lies
> about the block size.   For the cifs vfs adding a statfs64 func would
> certainly be technically feasible from the protocol's perspective and
> pretty easy.

Yes, I hit this problem a month or two ago with Lustre - the 90TB
filesystem we are testing on wrapped and I thought it was a problem
in our code until I did some more digging.  However, lying about
the blocksize isn't a big loss, since we prefer 64kB vector page I/O
over the network anyways for performance reasons, even though the
backing stores actually use 4kB blocksize.

Cheers, Andreas
--
Andreas Dilger
http://www-mddsp.enel.ucalgary.ca/People/adilger/
http://sourceforge.net/projects/ext2resize/