Re: [LTP] statvfs -> f_bavail

Re: [LTP] statvfs -> f_bavail

[Michal Simek]

Nate Straz wrote:
> On Apr 17 11:12, Michal Simek wrote:
>   
>>> don't you know what is the description of f_bavail in struct statvfs?
>>> On my system I am getting zero for this entry that's why fsync02 failed.
>>>   
>>>       
>> I track down where the problem comes from.
>> There is problem for all fs which use simple_statfs function from
>> fs/libfs.c.
>> In open.c in vfs_statfs function is whole structure set to zero and then
>> in simple_statfs not set this value.
>> I think we should fix it in ltp code.
>> Here is my proposed change. If is ok - I will generate proper patch.
>>     
>
> If the problem is in the kernel, then it should be fixed in the kernel.
> That's the whole point of LTP, pointing out problems in the kernel which
> need to be fixed.  Patching LTP to work around f_bavail not being set
> correctly is not the right thing to do.
>   
:-) And what about if is the kernel code ok? :-)
Then IMHO is the right time to fix LTP.

The main question is if is or not.

Hi guys from linux-fsdevel: Can you told us what is the right solution
for my problem above?

Thanks,
Michal

> Nate
>   


-- 
Michal Simek, Ing. (M.Eng)
PetaLogix - Linux Solutions for a Reconfigurable World
w: www.petalogix.com p: +61-7-30090663,+42-0-721842854 f: +61-7-30090663

Re: [LTP] statvfs -> f_bavail

[Al Viro]

On Mon, Apr 20, 2009 at 08:16:50AM +0200, Michal Simek wrote:
> Nate Straz wrote:
> > On Apr 17 11:12, Michal Simek wrote:
> >   
> >>> don't you know what is the description of f_bavail in struct statvfs?
> >>> On my system I am getting zero for this entry that's why fsync02 failed.
> >>>   
> >>>       
> >> I track down where the problem comes from.
> >> There is problem for all fs which use simple_statfs function from
> >> fs/libfs.c.
> >> In open.c in vfs_statfs function is whole structure set to zero and then
> >> in simple_statfs not set this value.
> >> I think we should fix it in ltp code.
> >> Here is my proposed change. If is ok - I will generate proper patch.
> >>     
> >
> > If the problem is in the kernel, then it should be fixed in the kernel.
> > That's the whole point of LTP, pointing out problems in the kernel which
> > need to be fixed.  Patching LTP to work around f_bavail not being set
> > correctly is not the right thing to do.
> >   
> :-) And what about if is the kernel code ok? :-)
> Then IMHO is the right time to fix LTP.
> 
> The main question is if is or not.
> 
> Hi guys from linux-fsdevel: Can you told us what is the right solution
> for my problem above?

"Fields that are undefined for a particular file system are set to 0".
So what kind of fs are you running that on and is that sucker really
defined for it?  Note that if it's ramfs or tmpfs with -o nr_blocks=0,
there is no such thing as "amount of free space", reserved for root
or not.

Re: [LTP] statvfs -> f_bavail

[Michal Simek]

Al Viro wrote:
> On Mon, Apr 20, 2009 at 08:16:50AM +0200, Michal Simek wrote:
>   
>> Nate Straz wrote:
>>     
>>> On Apr 17 11:12, Michal Simek wrote:
>>>   
>>>       
>>>>> don't you know what is the description of f_bavail in struct statvfs?
>>>>> On my system I am getting zero for this entry that's why fsync02 failed.
>>>>>   
>>>>>       
>>>>>           
>>>> I track down where the problem comes from.
>>>> There is problem for all fs which use simple_statfs function from
>>>> fs/libfs.c.
>>>> In open.c in vfs_statfs function is whole structure set to zero and then
>>>> in simple_statfs not set this value.
>>>> I think we should fix it in ltp code.
>>>> Here is my proposed change. If is ok - I will generate proper patch.
>>>>     
>>>>         
>>> If the problem is in the kernel, then it should be fixed in the kernel.
>>> That's the whole point of LTP, pointing out problems in the kernel which
>>> need to be fixed.  Patching LTP to work around f_bavail not being set
>>> correctly is not the right thing to do.
>>>   
>>>       
>> :-) And what about if is the kernel code ok? :-)
>> Then IMHO is the right time to fix LTP.
>>
>> The main question is if is or not.
>>
>> Hi guys from linux-fsdevel: Can you told us what is the right solution
>> for my problem above?
>>     
>
> "Fields that are undefined for a particular file system are set to 0".
> So what kind of fs are you running that on and is that sucker really
> defined for it?  Note that if it's ramfs or tmpfs with -o nr_blocks=0,
> there is no such thing as "amount of free space", reserved for root
> or not.
I use ramfs and nfs without any -o nr_block=0 option.
That mean that for all other fs is possible to set nr_blocks=0 (f_bavail=0) and for all this cases
fsync02 test failed. That mean that make sense to test f_bavail value in LTP and if is zero
don't work with it. Am I right?

Thanks,
Michal

# mount
rootfs on / type rootfs (rw)
proc on /proc type proc (rw)
none on /var type ramfs (rw)
none on /sys type sysfs (rw)
192.168.0.102:/tftpboot/nfs on /mnt type nfs
(rw,vers=3,rsize=32768,wsize=32768,namlen=255,hard,nointr,nolock,proto=tcp,timeo=70,retrans=3,sec=sys,addr=192.168.0.102)



-- 
Michal Simek, Ing. (M.Eng)
PetaLogix - Linux Solutions for a Reconfigurable World
w: www.petalogix.com p: +61-7-30090663,+42-0-721842854 f: +61-7-30090663

Re: statvfs -> f_bavail

[Nate Straz]

On Apr 20 08:42, Michal Simek wrote:
> Al Viro wrote:
> > On Mon, Apr 20, 2009 at 08:16:50AM +0200, Michal Simek wrote:
> >> Hi guys from linux-fsdevel: Can you told us what is the right solution
> >> for my problem above?
> >
> > "Fields that are undefined for a particular file system are set to 0".
> > So what kind of fs are you running that on and is that sucker really
> > defined for it?  Note that if it's ramfs or tmpfs with -o nr_blocks=0,
> > there is no such thing as "amount of free space", reserved for root
> > or not.
> I use ramfs and nfs without any -o nr_block=0 option.
> That mean that for all other fs is possible to set nr_blocks=0 (f_bavail=0) and for all this cases
> fsync02 test failed. That mean that make sense to test f_bavail value in LTP and if is zero
> don't work with it. Am I right?

Sounds like the patch is the right thing to do based on Al's quote.  I
would suggest modifying the patch to use fsblkcnt_t as f_bavail is
defined in statvfs(2).  Other than that, the patch looks good.

Nate

------------------------------------------------------------------------------
Stay on top of everything new and different, both inside and 
around Java (TM) technology - register by April 22, and save
$200 on the JavaOne (SM) conference, June 2-5, 2009, San Francisco.
300 plus technical and hands-on sessions. Register today. 
Use priority code J9JMT32. http://p.sf.net/sfu/p

Re: [LTP] statvfs -> f_bavail

[Al Viro]

On Mon, Apr 20, 2009 at 08:42:13AM +0200, Michal Simek wrote:

> That mean that for all other fs is possible to set nr_blocks=0 (f_bavail=0) and for all this cases
> fsync02 test failed. That mean that make sense to test f_bavail value in LTP and if is zero
> don't work with it. Am I right?

Huh?
	a) ramfs has no such thing as "amount of available space", simply
because it has no limit on total size occupied.
	b) tmpfs *does* have a limit and will report f_bavail, unless you
tell it not to limit (that's what nr_blocks=... is; it sets the fs size limit
for tmpfs and 0 means "no limit, act as ramfs").
	c) nfs client has no fscking clue how much space is left on server
for non-root; moreover, in case of nfs root on client might very well be
mapped to something else on server.
	d) something local on-disk (ext2, ext3, etc., etc.) can and will fill
->f_bavail with non-zero data
	e) procfs has nothing to put there, period.  You can't create
files there, it doesn't have anything like fixed-sized something that might
be partially empty.

The bottom line: some filesystems have reasonable answer to "how much space
is left on that fs for non-root user".  Those fill the field in question.
And for some filesystems the question makes no sense whatsoever.

So statfs(2) has every right to leave the damn thing zero.  Whether it
will do that or not depends on the fs type.  Userland code must be able
to cope with that, unless it *knows* which filesystem type will it be
dealing with.

Incidentally, nr_blocks=... will be cheerfully shat upon by just about every
fs out there.  It's tmpfs-specific.

Re: [LTP] statvfs -> f_bavail

[Andreas Dilger]

On Apr 21, 2009  01:25 +0100, Al Viro wrote:
> 	a) ramfs has no such thing as "amount of available space", simply
> because it has no limit on total size occupied.

Well, except the amount of RAM in the system, which would be a reasonable
value to use.

Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.

Re: [LTP] statvfs -> f_bavail

[Al Viro]

On Mon, Apr 20, 2009 at 06:38:47PM -0600, Andreas Dilger wrote:
> On Apr 21, 2009  01:25 +0100, Al Viro wrote:
> > 	a) ramfs has no such thing as "amount of available space", simply
> > because it has no limit on total size occupied.
> 
> Well, except the amount of RAM in the system, which would be a reasonable
> value to use.

You are welcome to provide an analysis of differences between the moments
when OOM hits root and non-root resp. if they keep allocating there...

Re: [LTP] statvfs -> f_bavail

[Michal Simek]

Al Viro wrote:
> On Mon, Apr 20, 2009 at 08:42:13AM +0200, Michal Simek wrote:
>
>   
>> That mean that for all other fs is possible to set nr_blocks=0 (f_bavail=0) and for all this cases
>> fsync02 test failed. That mean that make sense to test f_bavail value in LTP and if is zero
>> don't work with it. Am I right?
>>     
>
> Huh?
> 	a) ramfs has no such thing as "amount of available space", simply
> because it has no limit on total size occupied.
>   
As wrote Andreas - for ramfs case make sense to use size of memory.

> 	b) tmpfs *does* have a limit and will report f_bavail, unless you
> tell it not to limit (that's what nr_blocks=... is; it sets the fs size limit
> for tmpfs and 0 means "no limit, act as ramfs").
>   
that mean that if is not limit make no sense to work with it in fsync02
test and use or default test value
or find out your fs size limit? Where is it written?

> 	c) nfs client has no fscking clue how much space is left on server
> for non-root; moreover, in case of nfs root on client might very well be
> mapped to something else on server.
>   
that mean that no make sense to work with f_bavail and return zero value
right?

> 	d) something local on-disk (ext2, ext3, etc., etc.) can and will fill
> ->f_bavail with non-zero data
>   
this is used.
> 	e) procfs has nothing to put there, period.  You can't create
> files there, it doesn't have anything like fixed-sized something that might
> be partially empty.
>   
snip
> The bottom line: some filesystems have reasonable answer to "how much space
> is left on that fs for non-root user".  Those fill the field in question.
> And for some filesystems the question makes no sense whatsoever.
>
> So statfs(2) has every right to leave the damn thing zero.  Whether it
> will do that or not depends on the fs type.  Userland code must be able
> to cope with that, unless it *knows* which filesystem type will it be
> dealing with.
>   
ok. Current LTP fsync02 failed in forever loop and my LTP patch just
solve problem if the f_bavail
is zero - if yes, just use default value which is in test.
Of course I would like to use sensible value(amount of mem or whatever)
for all fs and if is the way, I'll do it. That's question for you
if is possible to do it -> that's why I cc: fsdevel mailing list.
If is not a way to do it in generic way make sense to me use at least my
patch.

Michal
> Incidentally, nr_blocks=... will be cheerfully shat upon by just about every
> fs out there.  It's tmpfs-specific.
>   


-- 
Michal Simek, Ing. (M.Eng)
PetaLogix - Linux Solutions for a Reconfigurable World
w: www.petalogix.com p: +61-7-30090663,+42-0-721842854 f: +61-7-30090663