interface to ask is a file is hidden

interface to ask is a file is hidden

[Nicolò Chieffo]

Recently there have been a discussion between the nautilus & GIO
maintainer, and the ntfs-3g maintainer to implement a way to not show
files in nautilus if they are hidden.

Currently only files starting with a "." are hidden, as you can see in
the GIO _g_local_file_info_get()

if (basename != NULL && basename[0] == '.')
   g_file_info_set_is_hidden (info, TRUE);

Of course, since we often have a multi-OS & multi-filesystem
environment, it is a big limitation to consider hidden files only
those files.
There are some use cases in which we need a wider view of hidden files:
- ntfs shared partitions & external disks
- fat32 shared partitions & external disks
- fat16 usb keys/multimedia devices

Both the maintainers state that it's not a good idea to patch GIO to
check which filesystem is hosting the file, and then ask it using the
specific driver whether the file is hidden or not.
Everything should be transparent for GIO, which should only make one
call to a standard interface, that all filesystem drivers shall
implement.

Is it possible to consider this request?
The discussion is in this moment hosted in the nautilus mailing list
with this subject name: "nautilus & hidden files".

Thank you

Re: interface to ask is a file is hidden

[Josef Bacik]

On Fri, Oct 10, 2008 at 09:22:11PM +0200, Nicolò Chieffo wrote:
> Recently there have been a discussion between the nautilus & GIO
> maintainer, and the ntfs-3g maintainer to implement a way to not show
> files in nautilus if they are hidden.
> 
> Currently only files starting with a "." are hidden, as you can see in
> the GIO _g_local_file_info_get()
> 
> if (basename != NULL && basename[0] == '.')
>    g_file_info_set_is_hidden (info, TRUE);
> 
> Of course, since we often have a multi-OS & multi-filesystem
> environment, it is a big limitation to consider hidden files only
> those files.
> There are some use cases in which we need a wider view of hidden files:
> - ntfs shared partitions & external disks
> - fat32 shared partitions & external disks
> - fat16 usb keys/multimedia devices
> 
> Both the maintainers state that it's not a good idea to patch GIO to
> check which filesystem is hosting the file, and then ask it using the
> specific driver whether the file is hidden or not.
> Everything should be transparent for GIO, which should only make one
> call to a standard interface, that all filesystem drivers shall
> implement.
> 
> Is it possible to consider this request?
> The discussion is in this moment hosted in the nautilus mailing list
> with this subject name: "nautilus & hidden files".
>

What files are determined as "hidden" is completely up to the application, and
not the filesystem.  Every linux filesystem is going to return all entries in a
directory when you do a readdir, and then it is up to the app to cull which
entries it doesn't want.  Having the fs/vfs arbitrarily decide which files are
"hidden" and shouldn't be returned via readdir is not the correct way to tackle
this problem, it should be decided via the application.

Josef 
--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: interface to ask is a file is hidden

[Jeremy Allison]

On Fri, Oct 10, 2008 at 03:21:04PM -0400, Josef Bacik wrote:

> What files are determined as "hidden" is completely up to the application, and
> not the filesystem.  Every linux filesystem is going to return all entries in a
> directory when you do a readdir, and then it is up to the app to cull which
> entries it doesn't want.  Having the fs/vfs arbitrarily decide which files are
> "hidden" and shouldn't be returned via readdir is not the correct way to tackle
> this problem, it should be decided via the application.

For Samba, we store an EA on a file representing its DOS attibutes.
"user.DOSATTRIB". Applications can then read that to determine if
a file is meant to be "hidden". Requires applications to coordinate
with the Samba definitions though.

Jeremy

Re: interface to ask is a file is hidden

[Nicolò Chieffo]

On Fri, Oct 10, 2008 at 03:21:04PM -0400, Josef Bacik wrote:

> What files are determined as "hidden" is completely up to the application, and
> not the filesystem.  Every linux filesystem is going to return all entries in a
> directory when you do a readdir, and then it is up to the app to cull which
> entries it doesn't want.  Having the fs/vfs arbitrarily decide which files are
> "hidden" and shouldn't be returned via readdir is not the correct way to tackle
> this problem, it should be decided via the application.

Ok, maybe I was not clear in my request

As if it's a way to get the size of a file, and this way is common to
all filesystem (tell me if I'm wrong), we request a common way to ask
if the file is hidden.
So that the GIO code won't look like this

if (filesystem_is_ext3(fs))
    hidden=ext3_get_hidden(file);
else if (filesystem_is_fat16()fs)
    hidden=fat16_get_hidden(file);
else if (filesystem_is_fat32(fs))
    hidden=fat32_get_hidden(file);
else if (filesystem_is_ntfs(fs))
    hidden=ntfs_get_hidden(file);
else if (filesystem_is_new_filesystem_just_invented(fs)
    /* oops! no way to get if a file is hidden because the maintainer
didn't implement it */
    hidden=FALSE;


If there is a common interface to do this we will gain 2 things
1) all filesystem must implement a way to get the hidden attribute
3) the application VFS must not know every filesystem type to support
hidden files
2) much simpler code in application VFS, that might look like this:
    hidden=get_filesyste_interface(fs)->get_hidden(file)

Re: interface to ask is a file is hidden

[Josef Bacik]

On Fri, Oct 10, 2008 at 10:00:28PM +0200, Nicolò Chieffo wrote:
> On Fri, Oct 10, 2008 at 03:21:04PM -0400, Josef Bacik wrote:
> 
> > What files are determined as "hidden" is completely up to the application, and
> > not the filesystem.  Every linux filesystem is going to return all entries in a
> > directory when you do a readdir, and then it is up to the app to cull which
> > entries it doesn't want.  Having the fs/vfs arbitrarily decide which files are
> > "hidden" and shouldn't be returned via readdir is not the correct way to tackle
> > this problem, it should be decided via the application.
> 
> Ok, maybe I was not clear in my request
> 
> As if it's a way to get the size of a file, and this way is common to
> all filesystem (tell me if I'm wrong), we request a common way to ask
> if the file is hidden.
> So that the GIO code won't look like this
> 
> if (filesystem_is_ext3(fs))
>     hidden=ext3_get_hidden(file);
> else if (filesystem_is_fat16()fs)
>     hidden=fat16_get_hidden(file);
> else if (filesystem_is_fat32(fs))
>     hidden=fat32_get_hidden(file);
> else if (filesystem_is_ntfs(fs))
>     hidden=ntfs_get_hidden(file);
> else if (filesystem_is_new_filesystem_just_invented(fs)
>     /* oops! no way to get if a file is hidden because the maintainer
> didn't implement it */
>     hidden=FALSE;
>
> 
> If there is a common interface to do this we will gain 2 things
> 1) all filesystem must implement a way to get the hidden attribute
> 3) the application VFS must not know every filesystem type to support
> hidden files
> 2) much simpler code in application VFS, that might look like this:
>     hidden=get_filesyste_interface(fs)->get_hidden(file)

Oh ok I see what you are saying, sort of like chattr for ext3, only globally.
Doesn't sound like a terrible idea to me, but I will defer to the more
experienced people on this list.

Josef
--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: interface to ask is a file is hidden

[Nicolò Chieffo]

> Oh ok I see what you are saying, sort of like chattr for ext3, only globally.
> Doesn't sound like a terrible idea to me, but I will defer to the more
> experienced people on this list.
>
> Josef
>

Thank you anyway. Who are the most experienced people?

Re: interface to ask is a file is hidden

[Theodore Tso]

On Fri, Oct 10, 2008 at 10:00:28PM +0200, Nicolò Chieffo wrote:
> On Fri, Oct 10, 2008 at 03:21:04PM -0400, Josef Bacik wrote:
> 
> > What files are determined as "hidden" is completely up to the application, and
> > not the filesystem.  Every linux filesystem is going to return all entries in a
> > directory when you do a readdir, and then it is up to the app to cull which
> > entries it doesn't want.  Having the fs/vfs arbitrarily decide which files are
> > "hidden" and shouldn't be returned via readdir is not the correct way to tackle
> > this problem, it should be decided via the application.
> 
> Ok, maybe I was not clear in my request
> 
> As if it's a way to get the size of a file, and this way is common to
> all filesystem (tell me if I'm wrong), we request a common way to ask
> if the file is hidden.

You realie that except for filesystems that are legacy compatible with
Microsoft, the concept of "hidden file" simply doesn't exist?  So when
you say:

> So that the GIO code won't look like this
> 
> if (filesystem_is_ext3(fs))
>     hidden=ext3_get_hidden(file);

Ext3 has no concept of "hidden file".  Niether does ext2, ext4, jfs,
xfs, reseirfs, ufs, etc.

The only ones that would have that concept is vfat (which supports
fat16/fat32), ntfs, nfsv4 and cifs/smbfs.  Even a filesystem which
normally has very bad taste, MacOS's HFS, doesn't support the hidden
attribute.

> If there is a common interface to do this we will gain 2 things
> 1) all filesystem must implement a way to get the hidden attribute

"Must?"  Bzzzt!  There will be many filesystems that have no place to
store a hidden attribute, where the concept doesn't exist.  For
example, NFSv3 simply doesn't possess anything vaguely like a hidden
attribute.  And even if we made a non-standard extension to NFSv3, it
wouldn't be supported by the millions and millions of non-Linux NFSv3
implementations.

The best you might be able to get is an interface that allows an
application to get or set the hidden attribute if is supported --- but
the application must be willing to accept a permission denied error
(some filesystems may only permit people with certain access right or
on some ACL to set the attribute), or a "operation not supported"
failure, for those filesystems that simply have not concept of "hidden
file".

It also means that if a desktop toolkit wants to depend on all
filesystems supporting the concept of hidden files, that's probably a
really bad idea, since it simply doesn't match with reality.

       	   	       	  	 	       - Ted
--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: interface to ask is a file is hidden

[Christoph Hellwig]

On Fri, Oct 10, 2008 at 05:57:22PM -0400, Theodore Tso wrote:
> The best you might be able to get is an interface that allows an
> application to get or set the hidden attribute if is supported --- but
> the application must be willing to accept a permission denied error
> (some filesystems may only permit people with certain access right or
> on some ACL to set the attribute), or a "operation not supported"
> failure, for those filesystems that simply have not concept of "hidden
> file".

Yes.  he best thing you could do is to add support for the
FAT_IOCTL_GET_ATTRIBUTES/FAT_IOCTL_SET_ATTRIBUTES ioctls to other
windows-heritage filesystems.  Feel free to submit patches.

Re: interface to ask is a file is hidden

[Alexander Larsson]

On Fri, 2008-10-10 at 18:01 -0400, Christoph Hellwig wrote:
> On Fri, Oct 10, 2008 at 05:57:22PM -0400, Theodore Tso wrote:
> > The best you might be able to get is an interface that allows an
> > application to get or set the hidden attribute if is supported --- but
> > the application must be willing to accept a permission denied error
> > (some filesystems may only permit people with certain access right or
> > on some ACL to set the attribute), or a "operation not supported"
> > failure, for those filesystems that simply have not concept of "hidden
> > file".
> 
> Yes.  he best thing you could do is to add support for the
> FAT_IOCTL_GET_ATTRIBUTES/FAT_IOCTL_SET_ATTRIBUTES ioctls to other
> windows-heritage filesystems.  Feel free to submit patches.

This is a bad interface for most apps. You need to open each file to get
a fd to pass to ioctl. This is problematic if for instance you don't
have read access to the file. So, you really want a path-based operation
for this. 

Its also very expensive since it causes lots of extra I/O. For instance,
a file manager that wants to hide files with the hidden attribute set
(on filesystems which support this) would have to open all files just so
it could try calling this ioctl. To avoid this you can do some
filesystem matching and hard code the lists of filesystems that supports
this call, but having a clean API would be nicer and more efficient.

Now, you could argue (and unsurpisingly you do) that ext3 & co doesn't
have a hidden attribute, but that doesn't mean I can ignore actual users
who have data on other filesystems and want to integrate nicely with
them. This includes not showing weird system files that are normally
hidden on said filesystems.

Re: interface to ask is a file is hidden

[Nicolò Chieffo]

I completely agree with you!

Re: interface to ask is a file is hidden

[Christoph Hellwig]

On Mon, Oct 13, 2008 at 10:22:53AM +0200, Alexander Larsson wrote:
> This is a bad interface for most apps. You need to open each file to get
> a fd to pass to ioctl. This is problematic if for instance you don't
> have read access to the file. So, you really want a path-based operation
> for this. 

Sorry, that just not how the unix file interface work like, and we're
not going to add crappy interface because your file manager wants to do
something stupid.  

> Now, you could argue (and unsurpisingly you do) that ext3 & co doesn't
> have a hidden attribute, but that doesn't mean I can ignore actual users
> who have data on other filesystems and want to integrate nicely with
> them. This includes not showing weird system files that are normally
> hidden on said filesystems.

Yes, you can.  These are foreign filesystems, and all access to the
filesystem (ls, echo, shell tab completion) also show them.  No need to
be special in your fancy "file manager".

Re: interface to ask is a file is hidden

[Nicolò Chieffo]

> Sorry, that just not how the unix file interface work like, and we're
> not going to add crappy interface because your file manager wants to do
> something stupid.

You don't have the right to judge an interface as "crappy" and a
feature we'd like to add "stupid", only because you don't need (like)
it.
These 2 words are quite rude. And we never had the intention to be
unkindly in our request.

You could indeed explain why this so wrong to you.
Please add a real motivation, since "unix does not work in this way"
is not enough:
1) linux is not unix
2) after 30 years it's good that things change and evolve
3) if new interfaces fully compatible with the past ones could be
implemented, nobody not interested in this will notice it

> Yes, you can.  These are foreign filesystems, and all access to the
> filesystem (ls, echo, shell tab completion) also show them.  No need to
> be special in your fancy "file manager".
>

His "fancy file manager" is for people who trust in the idea of
multiplatform and likes wide support, new features and
interoperability.
Nobody wants to modify 'ls' or orher legacy software

Re: interface to ask is a file is hidden

[Alexander Larsson]

On Mon, 2008-10-13 at 05:57 -0400, Christoph Hellwig wrote:
> On Mon, Oct 13, 2008 at 10:22:53AM +0200, Alexander Larsson wrote:
> > Now, you could argue (and unsurpisingly you do) that ext3 & co doesn't
> > have a hidden attribute, but that doesn't mean I can ignore actual users
> > who have data on other filesystems and want to integrate nicely with
> > them. This includes not showing weird system files that are normally
> > hidden on said filesystems.
> 
> Yes, you can.  These are foreign filesystems, and all access to the
> filesystem (ls, echo, shell tab completion) also show them.  No need to
> be special in your fancy "file manager".

The traditional unix file systems don't contain "is-hidden" metadata,
but some commonly used ones do. What possible bad can come out of
letting applications access that information efficiently if they want.
Certainly it won't affect the "classical" unix environment at all.

Re: interface to ask is a file is hidden

[Christoph Hellwig]

On Mon, Oct 13, 2008 at 12:37:54PM +0200, Alexander Larsson wrote:
> The traditional unix file systems don't contain "is-hidden" metadata,
> but some commonly used ones do. What possible bad can come out of
> letting applications access that information efficiently if they want.

The bad thing that happens is that it will bloat the kernel with useless
crap.  And a use case of "make graphical directly listings on foreign
filesystems look a little less cluttered" is not enough to add a lot of
code and new interfaces we'll have to maintain forever.

Re: interface to ask is a file is hidden

[Alexander Larsson]

On Mon, 2008-10-13 at 06:43 -0400, Christoph Hellwig wrote:
> On Mon, Oct 13, 2008 at 12:37:54PM +0200, Alexander Larsson wrote:
> > The traditional unix file systems don't contain "is-hidden" metadata,
> > but some commonly used ones do. What possible bad can come out of
> > letting applications access that information efficiently if they want.
> 
> The bad thing that happens is that it will bloat the kernel with useless
> crap.  And a use case of "make graphical directly listings on foreign
> filesystems look a little less cluttered" is not enough to add a lot of
> code and new interfaces we'll have to maintain forever.

Its only useless and bloat if you think the feature is useless, so we're
once again back to your preferences in applications. I'd like to think
that the opinions of others would be taken into consideration for this
too, and I and others would like this feature.

Unfortunately I don't have time myself atm to implement this, but I hope
someone else is interested in working on this.

Re: interface to ask is a file is hidden

[Jörn Engel]

On Mon, 13 October 2008 13:39:21 +0200, Alexander Larsson wrote:
> 
> Its only useless and bloat if you think the feature is useless, so we're
> once again back to your preferences in applications. I'd like to think
> that the opinions of others would be taken into consideration for this
> too, and I and others would like this feature.

It is a tradeoff.  How urgent is the feature, how much bloat will it
create, what's the mainenance load on people like Christoph.  Since he
is one of the _very_ few people who actually carry the maintenance load,
I would take his advice rather seriously and try to convince him on
technical grounds.

> Unfortunately I don't have time myself atm to implement this, but I hope
> someone else is interested in working on this.

And this is a perfect example of how not to convince someone.  Once you
have the time you might want to look at lsattr/chattr.  Until then, this
discussion is a waste of readers time.

Jörn

-- 
Public Domain  - Free as in Beer
General Public - Free as in Speech
BSD License    - Free as in Enterprise
Shared Source  - Free as in "Work will make you..."
--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: interface to ask is a file is hidden

[Christoph Hellwig]

On Mon, Oct 13, 2008 at 10:44:46PM +0200, J??rn Engel wrote:
> > Unfortunately I don't have time myself atm to implement this, but I hope
> > someone else is interested in working on this.
> 
> And this is a perfect example of how not to convince someone.  Once you
> have the time you might want to look at lsattr/chattr.  Until then, this
> discussion is a waste of readers time.

Well, adding it to XFS_IOC_GETXFLAGS/XFS_IOC_SETXFLAGS wouldn't be too
hard.  Then again this doesn't really buy anything over using the
existing vfat ioctls, and what I'm certainly not looking into is adding
completely new interfaces for it.  For one not for such a feature not
useful for native filesystems, and second not without a good
demonstraction that the path to fh conversion actually is a bottleneck
that matters.

Re: interface to ask is a file is hidden

[Alexander Larsson]

On Mon, 2008-10-13 at 22:44 +0200, Jörn Engel wrote:
> On Mon, 13 October 2008 13:39:21 +0200, Alexander Larsson wrote:
> > 
> > Its only useless and bloat if you think the feature is useless, so we're
> > once again back to your preferences in applications. I'd like to think
> > that the opinions of others would be taken into consideration for this
> > too, and I and others would like this feature.
> 
> It is a tradeoff.  How urgent is the feature, how much bloat will it
> create, what's the mainenance load on people like Christoph.  Since he
> is one of the _very_ few people who actually carry the maintenance load,
> I would take his advice rather seriously and try to convince him on
> technical grounds.
> 
> > Unfortunately I don't have time myself atm to implement this, but I hope
> > someone else is interested in working on this.
> 
> And this is a perfect example of how not to convince someone.  Once you
> have the time you might want to look at lsattr/chattr.  Until then, this
> discussion is a waste of readers time.

I'm not really out to convince anyone to do this work for me, because I
understand that nobody here is interested in file managers. 

However I don't think writing this is completely a waste of time.
Consider it an expression of interest in having stat.st_flags
implemented in Linux. I can use at least UF_HIDDEN and UF_IMMUTABLE in
the file manager. Perhaps someone else is interested in this also and
eventually the joint interest in it might get it implemented (it
happened in FreeBSD after all).


--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: interface to ask is a file is hidden

[Szabolcs Szakacsits]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: TEXT/PLAIN; charset=UTF8, Size: 1691 bytes --]


On Fri, 10 Oct 2008, Theodore Tso wrote:
> On Fri, Oct 10, 2008 at 10:00:28PM +0200, Nicolò Chieffo wrote:
> > On Fri, Oct 10, 2008 at 03:21:04PM -0400, Josef Bacik wrote:
> > 
> > > What files are determined as "hidden" is completely up to the application, and
> > > not the filesystem.  Every linux filesystem is going to return all entries in a
> > > directory when you do a readdir, and then it is up to the app to cull which
> > > entries it doesn't want.  Having the fs/vfs arbitrarily decide which files are
> > > "hidden" and shouldn't be returned via readdir is not the correct way to tackle
> > > this problem, it should be decided via the application.
> > 
> > Ok, maybe I was not clear in my request
> > 
> > As if it's a way to get the size of a file, and this way is common to
> > all filesystem (tell me if I'm wrong), we request a common way to ask
> > if the file is hidden.
> 
> You realie that except for filesystems that are legacy compatible with
> Microsoft, the concept of "hidden file" simply doesn't exist?  So when
> you say:

I think it's irrelevant what the [extended] attribute is. 

The real question is how to handle (and standardise) them efficiently and 
cleanly. Files can have quite a lot of properties (extended attributes), 
there are more and more requests for them every day.

New, efficient, extended getdents based on a provided attribute interest 
list?

Regards, 
	   Szaka

-- 
NTFS-3G:  http://ntfs-3g.org

--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: interface to ask is a file is hidden

[Nicolò Chieffo]

> You realie that except for filesystems that are legacy compatible with
> Microsoft, the concept of "hidden file" simply doesn't exist?  So when
> you say:

Sorry, but the concept of hidden files exists in ext3, in fact files
which name starts with a dot are hidden.

> "Must?"  Bzzzt!  There will be many filesystems that have no place to
> store a hidden attribute, where the concept doesn't exist.  For
> example, NFSv3 simply doesn't possess anything vaguely like a hidden
> attribute.  And even if we made a non-standard extension to NFSv3, it
> wouldn't be supported by the millions and millions of non-Linux NFSv3
> implementations.

if the filesystem does not support hidden files the result of the call
to that interface will be always "false", so no problem here.

> The best you might be able to get is an interface that allows an
> application to get or set the hidden attribute if is supported --- but
> the application must be willing to accept a permission denied error
> (some filesystems may only permit people with certain access right or
> on some ACL to set the attribute), or a "operation not supported"
> failure, for those filesystems that simply have not concept of "hidden
> file".

All applications support permission denied, for instance when you try
to write a file which is read-only for you.
If you want it's still possible to add a function that checks the
ability to support hidden files

> It also means that if a desktop toolkit wants to depend on all
> filesystems supporting the concept of hidden files, that's probably a
> really bad idea, since it simply doesn't match with reality.

Cannot understand the point. Hidden files exist (I think in most
filesystems, either with attributes or filenames), so they are reality

Re: interface to ask is a file is hidden

[Christoph Hellwig]

On Sat, Oct 11, 2008 at 12:21:40AM +0200, Nicol?? Chieffo wrote:
> > You realie that except for filesystems that are legacy compatible with
> > Microsoft, the concept of "hidden file" simply doesn't exist?  So when
> > you say:
> 
> Sorry, but the concept of hidden files exists in ext3, in fact files
> which name starts with a dot are hidden.

No, they aren't.  ls doesn't display them with the default option, but
there is nothing anywher near a filesystem "hidden" concept.  Unix
filenames are just opaque bytestream, with only ".", ".." and "\0"
having special meanings.

Re: interface to ask is a file is hidden

[Nicolò Chieffo]

Mm ok, with "hidden" I mean files that are not normally displayed when
you list the directory, nothing more, sorry if I coulnd't explain
well.

I'm just talking as a normal user, sorry.

Which is the correct definition of what I'm trying to explain?

Re: interface to ask is a file is hidden

[Christoph Hellwig]

On Sat, Oct 11, 2008 at 12:33:09AM +0200, Nicol?? Chieffo wrote:
> Mm ok, with "hidden" I mean files that are not normally displayed when
> you list the directory, nothing more, sorry if I coulnd't explain
> well.
> 
> I'm just talking as a normal user, sorry.
> 
> Which is the correct definition of what I'm trying to explain?

I have no idea what you want to display.  But either follow the de-facto
standard that ls has set 30 years or more ago and skip the files that
start with a ".", or do what you thing is right and ignore what unix
systems have been doing forever and do your own thing.

Re: interface to ask is a file is hidden

[Nicolò Chieffo]

The final target is to create a standard attribute list for files, so
that you don't need to depend on the specific filesystem driver to
extract the attributes.

If you don't like the hidden example here's another (I hope it's
correct because I don't know exactly how this is implemented)

you want to know if a file is sparse. For sure ext3 and ntfs support
sparse files, but (maybe) there is not a common way to ask this.

As Szaka told before, the correct approach might be to define a set of
attributes and a function to get/set them

Re: interface to ask is a file is hidden

[Szabolcs Szakacsits]

On Sat, 11 Oct 2008, [UTF-8] Nicolò Chieffo wrote:

> As Szaka told before, the correct approach might be to define a set of
> attributes and a function to get/set them

There are functions to get/set attributes. Just not efficiently. 

That's the real issue here. 

	Szaka

--
NTFS-3G:  http://ntfs-3g.org

--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: interface to ask is a file is hidden

[Bryan Henderson]

> But either follow the de-facto
> standard that ls has set 30 years or more ago

The standard you're talking about is for Unix systems that have Unix 
filesystems contained within them.  But today, one can use a Unix system 
to look at a directory full of files that were created in a Windows 
culture.  In that case it would be suboptimal to hide the files whose 
names start with "." and show the rest.  The person who set up that 
directory didn't intend that.

This is a pretty difficult requirement -- It might even be infeasible to 
satisfy it.  But I think it's real.  The creator of a file either intended 
for it to be omitted from common views or he didn't, and it would be nice 
if there were a simple and reliable way for a file viewer to discern 
which.

Personally, I'm not even qualified to guess at what a user wants to be 
hidden, because I decided long ago that hidden files were not in my best 
interest and I always set up 'ls' to show everything but . and .. .

--
Bryan Henderson                     IBM Almaden Research Center
San Jose CA                         Storage Systems

Re: interface to ask is a file is hidden

[Theodore Tso]

On Sat, Oct 11, 2008 at 12:33:09AM +0200, Nicolò Chieffo wrote:
> Mm ok, with "hidden" I mean files that are not normally displayed when
> you list the directory, nothing more, sorry if I coulnd't explain
> well.
> 
> I'm just talking as a normal user, sorry.
> 
> Which is the correct definition of what I'm trying to explain?

The convention on all Unix systems is that files that begin with the
'.' character are not displayed by "ls" unless it is given the -a
option.  It is not done using an attribute; it is merely a convention
based on a filename.  So you can't "set an attribute" on all standard
Unix filesystems, and there's no hidden attribute to "get".

Note also that if you create a file named ".bashrc" and store it on a
VFAT or ntfs filesystem, it will not be displayed by the ls program.
It has nothing to do with any kind of attribute.

Part of what you clearly don't understand is that different operating
systems and different filesystems do this very differently.

						- Ted


--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: interface to ask is a file is hidden

[Brad Boyer]

On Fri, Oct 10, 2008 at 05:57:22PM -0400, Theodore Tso wrote:
> The only ones that would have that concept is vfat (which supports
> fat16/fat32), ntfs, nfsv4 and cifs/smbfs.  Even a filesystem which
> normally has very bad taste, MacOS's HFS, doesn't support the hidden
> attribute.

Actually, HFS and HFS+ have an "invisible" bit for each entry. This is
read by Finder to tell if it should show an icon for that particular
file or directory. This is similar to the MSDOS hidden attribtue. I
don't think we currently expose that in any way, although it used to
be exposed in HFS if you were using one of the optional modes to
expose the Mac metadata for sharing over AFP.

I will admit that structurally it's very different from the way FAT
handles such a thing. The invisible bit is part of the FinderInfo
structure that is really just the metadata for Finder to draw the
views correctly and just happens to be stored inside the filesystem
metadata area (similar to the way a unix filesystem does xattrs). It
even includes things like point and rect info for icon placement.

>From hfs.h:
/* bits hfs_finfo.fdFlags */
#define HFS_FLG_INVISIBLE       0x4000

I'll also be clear that this doesn't get used by ls or other UNIX
type utilities on OSX. It's purely metadata for Finder.

	Brad Boyer
	flar@allandria.com

Re: interface to ask is a file is hidden

[Anton Altaparmakov]

Hi,

On 10 Oct 2008, at 22:57, Theodore Tso wrote:
> Ext3 has no concept of "hidden file".  Niether does ext2, ext4, jfs,
> xfs, reseirfs, ufs, etc.
>
> The only ones that would have that concept is vfat (which supports
> fat16/fat32), ntfs, nfsv4 and cifs/smbfs.  Even a filesystem which
> normally has very bad taste, MacOS's HFS, doesn't support the hidden
> attribute.

Sorry but you are somewhat wrong: UFS, HFS/HFS+, ZFS, and AFP all  
support the UF_HIDDEN file flag (in addition to the ones you state  
above).

In Mac OS X this is settable via chflags(2) and fchflags(2) and "ls"  
can display whether this flag is set by calling "ls -lO" for example  
(stat(2) returns the flags in st_flags).  The Finder is the primary  
application which then hides such marked files from the GUI view  
analogous to how Windows Explorer hides hidden files.

PS. For completeness sake: On Mac OS X the hidden flag is also  
mirrored in the Finder Info.  Also, all the flags can be obtained/set  
via getattrlist(2)/setattrlist(2) via ATTR_CMN_FLAGS in addition to  
chflags(2)/stat(2).

Best regards,

	Anton
-- 
Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @)
Unix Support, Computing Service, University of Cambridge, CB2 3QH, UK
Linux NTFS maintainer, http://www.linux-ntfs.org/

Re: interface to ask is a file is hidden

[Christoph Hellwig]

On Mon, Oct 13, 2008 at 08:29:50AM +0100, Anton Altaparmakov wrote:
> Sorry but you are somewhat wrong: UFS, HFS/HFS+, ZFS, and AFP all  
> support the UF_HIDDEN file flag (in addition to the ones you state  
> above).

On MacOS.  On FreeBSD none no filesystem has it.

Re: interface to ask is a file is hidden

[Alexander Larsson]

On Mon, 2008-10-13 at 08:29 +0100, Anton Altaparmakov wrote:
> Hi,
> 
> On 10 Oct 2008, at 22:57, Theodore Tso wrote:
> > Ext3 has no concept of "hidden file".  Niether does ext2, ext4, jfs,
> > xfs, reseirfs, ufs, etc.
> >
> > The only ones that would have that concept is vfat (which supports
> > fat16/fat32), ntfs, nfsv4 and cifs/smbfs.  Even a filesystem which
> > normally has very bad taste, MacOS's HFS, doesn't support the hidden
> > attribute.
> 
> Sorry but you are somewhat wrong: UFS, HFS/HFS+, ZFS, and AFP all  
> support the UF_HIDDEN file flag (in addition to the ones you state  
> above).

FreeBSD seems to have added UF_HIDDEN to struct stat.st_flags back in
2006:
http://updraft3.jp.freebsd.org/cgi/query-pr.cgi?pr=kern/94519

They also have other useful flags like immutable, append only, no dump,
etc that certainly apply to ext3 too (i.e. these are settable with
chattr).

Sounds like a very good idea to me. There is even availible unused space
in the userspace/glibc struct stat to add them, and there seems to be
some in the kernel struct too.

They also have chflags/lchflags/fchflags to change these attributes.