Zero size and zero blocks mountpoint.

Zero size and zero blocks mountpoint.

[Alex Perez]

I've just found this thread on google, and I subscribed to the list.

http://www.spinics.net/lists/linux-cifs/msg00951.html

I'm experiencing the same problem. the mountpoint is showing a 4096
size, but when I mount a cifs resource, it shows a 0-size. Other
mountpoints that are not using cifs (nfs, ext2-3 ...) doesnt change
the size.

The problem is that the mount point belongs to a directory structure
that is accesed by an apache service (2.0.54). The mount point is
/opt/www/dir1, while the apache's directoryRoot is /opt/www. So, the
problem is that when a client is trying to access to "dir1" I get a
error in apache logs, like:

[Fri Sep 24 10:57:26 2010] [error] [client 192.0.2.147] (75)Value too
large for defined data type: access to /dir1 failed

And the client gets a 403 - forbidden error. It's very tricky because
if I dismount /opt/www/dir1, the client can access without any
problem.

Googling this error seem that is related to accessing files > 2Gb, but
I think that apache doesn't know how to handle a 0-size directory or
it's understanding that is a huge sized directory ...

how can I fix the directory-size that apache is receiving?

-- 
3rd Law of Computing:
Anything that can go wrSegmentation fault -- core dumped.

Re: Zero size and zero blocks mountpoint.

[Jeff Layton]

On Fri, 24 Sep 2010 13:18:45 +0200
Alex Perez <quimicefa-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:

> I've just found this thread on google, and I subscribed to the list.
> 
> http://www.spinics.net/lists/linux-cifs/msg00951.html
> 
> I'm experiencing the same problem. the mountpoint is showing a 4096
> size, but when I mount a cifs resource, it shows a 0-size. Other
> mountpoints that are not using cifs (nfs, ext2-3 ...) doesnt change
> the size.
> 
> The problem is that the mount point belongs to a directory structure
> that is accesed by an apache service (2.0.54). The mount point is
> /opt/www/dir1, while the apache's directoryRoot is /opt/www. So, the
> problem is that when a client is trying to access to "dir1" I get a
> error in apache logs, like:
> 
> [Fri Sep 24 10:57:26 2010] [error] [client 192.0.2.147] (75)Value too
> large for defined data type: access to /dir1 failed
> 
> And the client gets a 403 - forbidden error. It's very tricky because
> if I dismount /opt/www/dir1, the client can access without any
> problem.
> 
> Googling this error seem that is related to accessing files > 2Gb, but
> I think that apache doesn't know how to handle a 0-size directory or
> it's understanding that is a huge sized directory ...
> 
> how can I fix the directory-size that apache is receiving?
> 

Honestly, that sounds like an apache bug. You may want to report that
to them.

I asked Stef this question question and didn't get an answer however:

Does POSIX offer any guidance about what the size of a directory should
represent?

For a local filesystem this is simple -- it's a representation of the
space that the directory occupies on disk (directories are just inodes
like any other).

NFS returns a non-zero size for a directory inode because the server
returns one. CIFS servers however send 0 for the size of a directory.
We could replace that with a fake value, but it's not at all clear to
me what that value should be.

-- 
Jeff Layton <jlayton-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>

Re: Zero size and zero blocks mountpoint.

[Stef Bon]

2010/9/24 Jeff Layton <jlayton-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>:

>> how can I fix the directory-size that apache is receiving?
>>
>
> Honestly, that sounds like an apache bug. You may want to report that
> to them.
>
> I asked Stef this question question and didn't get an answer however:
>
> Does POSIX offer any guidance about what the size of a directory should
> represent?

I'm happy to see this question is again an issue.

I do not know about POSIX, but my common sense says that a directory,
even it is a
network directory, never has zero size.

As you mention, it's the space reserved for the directoryobject.

With a network fs (and every virtual fs) the link with the object on
the harddrive isn't there,
but a directory takes always some memory! In C there would be an entry
object allocated, I do not have to explain that. The size will
probably will be much less than the 4Kb with Ext2/3/4,
but my point is it is never zero.

Stef

Re: Zero size and zero blocks mountpoint.

[Jeff Layton]

On Fri, 24 Sep 2010 19:44:30 +0200
Stef Bon <stefbon-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:

> 2010/9/24 Jeff Layton <jlayton-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>:
> 
> >> how can I fix the directory-size that apache is receiving?
> >>
> >
> > Honestly, that sounds like an apache bug. You may want to report that
> > to them.
> >
> > I asked Stef this question question and didn't get an answer however:
> >
> > Does POSIX offer any guidance about what the size of a directory should
> > represent?
> 
> I'm happy to see this question is again an issue.
> 
> I do not know about POSIX, but my common sense says that a directory,
> even it is a
> network directory, never has zero size.
> 
> As you mention, it's the space reserved for the directoryobject.
> 

That's not a given. Maybe the directories are generated on the fly and
have no persistent storage (even in memory)?

How about /proc? Most directories there (and many of the pseudofiles)
also have a zero size. Obviously there's no space reserved for them --
their contents are often generated on an as-needed basis.

> With a network fs (and every virtual fs) the link with the object on
> the harddrive isn't there,
> but a directory takes always some memory! In C there would be an entry
> object allocated, I do not have to explain that. The size will
> probably will be much less than the 4Kb with Ext2/3/4,
> but my point is it is never zero.
> 

AFAICT, the fact that it's not zero on those filesystems is an
implementation detail. The size for directories in Linux CIFS *is*
zero. We generate or match dentries on the fly based on
FIND_FIRST/FIND_NEXT responses and then discard the response data.
There are no pagecache pages attached to CIFS directory inodes, so a 0
size for directory inodes is actually correct by your definition.

Again, I'll be happy to entertain patches to change this (and even help
write them), but we need more information about what the correct
behavior is.
-- 
Jeff Layton <jlayton-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>

Re: Zero size and zero blocks mountpoint.

[Steve French]

Looks like an Apache bug if they rely on the value of st_size for
directories.  See
http://www.opengroup.org/onlinepubs/000095399/basedefs/sys/stat.h.html

"For regular files, the file size in bytes. For symbolic links, the
length in bytes of the pathname contained in the symbolic link." ...
"For other file types, the use of this field is unspecified."

I don't really mind setting it to a minimum.  What does Windows return
as allocation size on directories?

On Fri, Sep 24, 2010 at 1:02 PM, Jeff Layton <jlayton-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> wrote:
> On Fri, 24 Sep 2010 19:44:30 +0200
> Stef Bon <stefbon-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>
>> 2010/9/24 Jeff Layton <jlayton-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>:
>>
>> >> how can I fix the directory-size that apache is receiving?
>> >>
>> >
>> > Honestly, that sounds like an apache bug. You may want to report that
>> > to them.
>> >
>> > I asked Stef this question question and didn't get an answer however:
>> >
>> > Does POSIX offer any guidance about what the size of a directory should
>> > represent?
>>
>> I'm happy to see this question is again an issue.
>>
>> I do not know about POSIX, but my common sense says that a directory,
>> even it is a
>> network directory, never has zero size.
>>
>> As you mention, it's the space reserved for the directoryobject.
>>
>
> That's not a given. Maybe the directories are generated on the fly and
> have no persistent storage (even in memory)?
>
> How about /proc? Most directories there (and many of the pseudofiles)
> also have a zero size. Obviously there's no space reserved for them --
> their contents are often generated on an as-needed basis.
>
>> With a network fs (and every virtual fs) the link with the object on
>> the harddrive isn't there,
>> but a directory takes always some memory! In C there would be an entry
>> object allocated, I do not have to explain that. The size will
>> probably will be much less than the 4Kb with Ext2/3/4,
>> but my point is it is never zero.
>>
>
> AFAICT, the fact that it's not zero on those filesystems is an
> implementation detail. The size for directories in Linux CIFS *is*
> zero. We generate or match dentries on the fly based on
> FIND_FIRST/FIND_NEXT responses and then discard the response data.
> There are no pagecache pages attached to CIFS directory inodes, so a 0
> size for directory inodes is actually correct by your definition.
>
> Again, I'll be happy to entertain patches to change this (and even help
> write them), but we need more information about what the correct
> behavior is.
> --
> Jeff Layton <jlayton-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-cifs" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>



-- 
Thanks,

Steve

Re: Zero size and zero blocks mountpoint.

[Suresh Jayaraman]

On 09/24/2010 05:05 PM, Jeff Layton wrote:
> On Fri, 24 Sep 2010 13:18:45 +0200
> Alex Perez <quimicefa-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> 
>> I've just found this thread on google, and I subscribed to the list.
>>
>> http://www.spinics.net/lists/linux-cifs/msg00951.html
>>
>> I'm experiencing the same problem. the mountpoint is showing a 4096
>> size, but when I mount a cifs resource, it shows a 0-size. Other
>> mountpoints that are not using cifs (nfs, ext2-3 ...) doesnt change
>> the size.
>>
>> The problem is that the mount point belongs to a directory structure
>> that is accesed by an apache service (2.0.54). The mount point is
>> /opt/www/dir1, while the apache's directoryRoot is /opt/www. So, the
>> problem is that when a client is trying to access to "dir1" I get a
>> error in apache logs, like:
>>
>> [Fri Sep 24 10:57:26 2010] [error] [client 192.0.2.147] (75)Value too
>> large for defined data type: access to /dir1 failed
>>
>> And the client gets a 403 - forbidden error. It's very tricky because
>> if I dismount /opt/www/dir1, the client can access without any
>> problem.
>>
>> Googling this error seem that is related to accessing files > 2Gb, but
>> I think that apache doesn't know how to handle a 0-size directory or
>> it's understanding that is a huge sized directory ...
>>
>> how can I fix the directory-size that apache is receiving?
>>
> 
> Honestly, that sounds like an apache bug. You may want to report that
> to them.
> 
> I asked Stef this question question and didn't get an answer however:
> 
> Does POSIX offer any guidance about what the size of a directory should
> represent?


The nearest I could find is:

"If the file is a character special or block special file, the size of
the file may be replaced with implementation-defined information
associated with the device in question."

   from http://www.opengroup.org/onlinepubs/9699919799/

In our case it's network file object and the Server only seem to know
the the actual size.

> For a local filesystem this is simple -- it's a representation of the
> space that the directory occupies on disk (directories are just inodes
> like any other).
> 
> NFS returns a non-zero size for a directory inode because the server
> returns one. CIFS servers however send 0 for the size of a directory.

+ 1

I think it makes sense to report what the server returns and not fake it
to cover up application bugs/ unwarranted checks. If there are other
reasons, we could consider setting a reasonable size


-- 
Suresh Jayaraman

Zero size and zero blocks mountpoint.

[Stef Bon]

Hello,

whe mounting a smb filesystem using cifs, the mountpoint gets zero
size and zero blocks:


ls -al
total 12
drwxr-xr-x   2 root root     4096 2010-09-19 16:53 .
drwxr-xr-x   3 root root     4096 2010-09-19 16:53 ..
drwxr-xr-x   6 root root        0 2010-09-03 23:41 bononline
-rw-r--r--   1 root root       40 2010-09-19 16:53 .directory
drwxr-xr-x 673 gast netgroup    0 2010-09-12 23:40 ftp
drwxrwxr-x  10 gast netgroup    0 2010-08-06 23:28 public
drwxr-xr-x  34 sbon netgroup    0 2010-09-19 16:03 sbon
drwxr-xr-x  10 root root        0 2010-08-06 23:26 video

The directories bononline, ftp, public, sbon adn video are all
mountpoints of smb shares.

It's causing some unwanted behaviour.
How can I change that, or is this not changeable?
Stef

Re: Zero size and zero blocks mountpoint.

[Jeff Layton]

On Sun, 19 Sep 2010 21:28:41 +0200
Stef Bon <stefbon-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:

> Hello,
> 
> whe mounting a smb filesystem using cifs, the mountpoint gets zero
> size and zero blocks:
> 
> 
> ls -al
> total 12
> drwxr-xr-x   2 root root     4096 2010-09-19 16:53 .
> drwxr-xr-x   3 root root     4096 2010-09-19 16:53 ..
> drwxr-xr-x   6 root root        0 2010-09-03 23:41 bononline
> -rw-r--r--   1 root root       40 2010-09-19 16:53 .directory
> drwxr-xr-x 673 gast netgroup    0 2010-09-12 23:40 ftp
> drwxrwxr-x  10 gast netgroup    0 2010-08-06 23:28 public
> drwxr-xr-x  34 sbon netgroup    0 2010-09-19 16:03 sbon
> drwxr-xr-x  10 root root        0 2010-08-06 23:26 video
> 
> The directories bononline, ftp, public, sbon adn video are all
> mountpoints of smb shares.
> 
> It's causing some unwanted behaviour.
> How can I change that, or is this not changeable?
> Stef

I think it's just a matter of setting the inode->i_size on the
directory. I *think* that when you do a QPathInfo call to a directory
via cifs the size comes back 0. Setting this to something else probably
means an extra call to the server to get the "real" size (whatever that
means). The question is, what should this be set to?

-- 
Jeff Layton <jlayton-eUNUBHrolfbYtjvyW6yDsg@public.gmane.org>

Re: Zero size and zero blocks mountpoint.

[Stef Bon]

2010/9/20 Jeff Layton <jlayton-eUNUBHrolfbYtjvyW6yDsg@public.gmane.org>:
> On Sun, 19 Sep 2010 21:28:41 +0200
> Stef Bon <stefbon-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>
>> Hello,
>>
>>
>> It's causing some unwanted behaviour.
>> How can I change that, or is this not changeable?
>> Stef
>
> I think it's just a matter of setting the inode->i_size on the
> directory. I *think* that when you do a QPathInfo call to a directory
> via cifs the size comes back 0. Setting this to something else probably
> means an extra call to the server to get the "real" size (whatever that
> means). The question is, what should this be set to?
>
> --
> Jeff Layton <jlayton-eUNUBHrolfbYtjvyW6yDsg@public.gmane.org>
>

Yes that's a good question. But a share mounted with cifs should
behave like it is's just a local harddrive partition right? So these
values are obligatory...

 In my construction - so I'm speaking only for my own here -
it's also set the size (and the blocks) in the FUSE fs. Remember, I'm
using the FUSE fs fuse-workspace
as the one which is used, and the various mounts to resources like smb
shares (mounted with cifs of course, managed with autofs) are the
underlying backend.

My FUSE can correct things, like this.

I wonder, it can correct things, but is it also a must?
My fuse fs takes these values just from the underlying fs, and this
can result in
a size of directories of 4096 when the underlying fs is a mounted
partition (of for example an USB disk) and zero
when it's a cifs mounted smb share. Is it a bad thing that these
values differ in the same fs (in my case thus fuse-workspace)??

Maybe you know this, else I'll try at the fsdevel list.

Stef

Re: Zero size and zero blocks mountpoint.

[Jeff Layton]

On Mon, 20 Sep 2010 09:29:38 +0200
Stef Bon <stefbon-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:

> 2010/9/20 Jeff Layton <jlayton-eUNUBHrolfbYtjvyW6yDsg@public.gmane.org>:
> > On Sun, 19 Sep 2010 21:28:41 +0200
> > Stef Bon <stefbon-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> >
> >> Hello,
> >>
> >>
> >> It's causing some unwanted behaviour.
> >> How can I change that, or is this not changeable?
> >> Stef
> >
> > I think it's just a matter of setting the inode->i_size on the
> > directory. I *think* that when you do a QPathInfo call to a directory
> > via cifs the size comes back 0. Setting this to something else probably
> > means an extra call to the server to get the "real" size (whatever that
> > means). The question is, what should this be set to?
> >
> > --
> > Jeff Layton <jlayton-eUNUBHrolfbYtjvyW6yDsg@public.gmane.org>
> >
> 
> Yes that's a good question. But a share mounted with cifs should
> behave like it is's just a local harddrive partition right? So these
> values are obligatory...
> 

No, not necessarily.

>  In my construction - so I'm speaking only for my own here -
> it's also set the size (and the blocks) in the FUSE fs. Remember, I'm
> using the FUSE fs fuse-workspace
> as the one which is used, and the various mounts to resources like smb
> shares (mounted with cifs of course, managed with autofs) are the
> underlying backend.
> 
> My FUSE can correct things, like this.
> 
> I wonder, it can correct things, but is it also a must?
> My fuse fs takes these values just from the underlying fs, and this
> can result in
> a size of directories of 4096 when the underlying fs is a mounted
> partition (of for example an USB disk) and zero
> when it's a cifs mounted smb share. Is it a bad thing that these
> values differ in the same fs (in my case thus fuse-workspace)??
> 
> Maybe you know this, else I'll try at the fsdevel list.
> 

It really comes down to this question:

    What is the size on a directory supposed to signify?

It's certainly possible for us to fake up a size and stuff it into the
i_size field, but what should go there? Why 4096 and not 4097 or 4095?
What if the directory is bigger or smaller than a page?

Also, what sort of application is having problems with a directory of 0
size?

Asking this on fsdevel would probably be a good idea.

-- 
Jeff Layton <jlayton-eUNUBHrolfbYtjvyW6yDsg@public.gmane.org>

Re: Zero size and zero blocks mountpoint.

[Suresh Jayaraman]

On 09/20/2010 09:21 PM, Jeff Layton wrote:
>>>>
>>>> It's causing some unwanted behaviour.
>>>> How can I change that, or is this not changeable?
>>>> Stef
>>>
>>> I think it's just a matter of setting the inode->i_size on the
>>> directory. I *think* that when you do a QPathInfo call to a directory
>>> via cifs the size comes back 0. Setting this to something else probably
>>> means an extra call to the server to get the "real" size (whatever that
>>> means). The question is, what should this be set to?
>>>
>>
>> Yes that's a good question. But a share mounted with cifs should
>> behave like it is's just a local harddrive partition right? So these
>> values are obligatory...
>>
> 
> No, not necessarily.
> 
>>  In my construction - so I'm speaking only for my own here -
>> it's also set the size (and the blocks) in the FUSE fs. Remember, I'm
>> using the FUSE fs fuse-workspace
>> as the one which is used, and the various mounts to resources like smb
>> shares (mounted with cifs of course, managed with autofs) are the
>> underlying backend.
>>
>> My FUSE can correct things, like this.
>>
>> I wonder, it can correct things, but is it also a must?
>> My fuse fs takes these values just from the underlying fs, and this
>> can result in
>> a size of directories of 4096 when the underlying fs is a mounted
>> partition (of for example an USB disk) and zero
>> when it's a cifs mounted smb share. Is it a bad thing that these
>> values differ in the same fs (in my case thus fuse-workspace)??
>>
>> Maybe you know this, else I'll try at the fsdevel list.
>>
> 
> It really comes down to this question:
> 
>     What is the size on a directory supposed to signify?
> 

(Sorry for digging an old thread..)

I think it is really the space required (filesystem thinks it requires)
for storing the metadata about the files that would be stored and it
depends on the internal datastructures of the filesystem. It never
shrinks but expands when the number of files increases.

For e.g. On a ext3 filesystem, it looks like the default size of
directory on creation is 4096/4k and somewhere between 325 files and 350
files, it changes from 4k to 12k.

To get the same behavior on CIFS, the client needs to be aware of the
Server filesystem and need to have a rough idea of datastructures unless
the Server provides the size. Also, the client needs to increase the
size to replicate the same behavior when the number of files on the
server increases.

I think the nearest information about the Server filesystem the client
could get is 'block size' and the client could try to do some clever
guess, but it may not always get it correct.


-- 
Suresh Jayaraman

Re: Zero size and zero blocks mountpoint.

[Jeff Layton]

On Mon, 27 Sep 2010 18:23:40 +0530
Suresh Jayaraman <sjayaraman-l3A5Bk7waGM@public.gmane.org> wrote:

> On 09/20/2010 09:21 PM, Jeff Layton wrote:
> >>>>
> >>>> It's causing some unwanted behaviour.
> >>>> How can I change that, or is this not changeable?
> >>>> Stef
> >>>
> >>> I think it's just a matter of setting the inode->i_size on the
> >>> directory. I *think* that when you do a QPathInfo call to a directory
> >>> via cifs the size comes back 0. Setting this to something else probably
> >>> means an extra call to the server to get the "real" size (whatever that
> >>> means). The question is, what should this be set to?
> >>>
> >>
> >> Yes that's a good question. But a share mounted with cifs should
> >> behave like it is's just a local harddrive partition right? So these
> >> values are obligatory...
> >>
> > 
> > No, not necessarily.
> > 
> >>  In my construction - so I'm speaking only for my own here -
> >> it's also set the size (and the blocks) in the FUSE fs. Remember, I'm
> >> using the FUSE fs fuse-workspace
> >> as the one which is used, and the various mounts to resources like smb
> >> shares (mounted with cifs of course, managed with autofs) are the
> >> underlying backend.
> >>
> >> My FUSE can correct things, like this.
> >>
> >> I wonder, it can correct things, but is it also a must?
> >> My fuse fs takes these values just from the underlying fs, and this
> >> can result in
> >> a size of directories of 4096 when the underlying fs is a mounted
> >> partition (of for example an USB disk) and zero
> >> when it's a cifs mounted smb share. Is it a bad thing that these
> >> values differ in the same fs (in my case thus fuse-workspace)??
> >>
> >> Maybe you know this, else I'll try at the fsdevel list.
> >>
> > 
> > It really comes down to this question:
> > 
> >     What is the size on a directory supposed to signify?
> > 
> 
> (Sorry for digging an old thread..)
> 
> I think it is really the space required (filesystem thinks it requires)
> for storing the metadata about the files that would be stored and it
> depends on the internal datastructures of the filesystem. It never
> shrinks but expands when the number of files increases.
> 
> For e.g. On a ext3 filesystem, it looks like the default size of
> directory on creation is 4096/4k and somewhere between 325 files and 350
> files, it changes from 4k to 12k.
> 
> To get the same behavior on CIFS, the client needs to be aware of the
> Server filesystem and need to have a rough idea of datastructures unless
> the Server provides the size. Also, the client needs to increase the
> size to replicate the same behavior when the number of files on the
> server increases.
> 
> I think the nearest information about the Server filesystem the client
> could get is 'block size' and the client could try to do some clever
> guess, but it may not always get it correct.
> 
> 

I guess my point with all of this is that the size of a directory is
undefined by POSIX. A zero size directory is no less correct than any
other size. If there are applications that depend on that value meaning
something, then they ought to reconsider doing that.

-- 
Jeff Layton <jlayton-eUNUBHrolfbYtjvyW6yDsg@public.gmane.org>