Setting project quotas on special files

Setting project quotas on special files

[Tiziano Müller]

Hi everyone

It started with the following error:

   dev # cp -al dvd dvd2
   cp: cannot create hard link ‘dvd2’ to ‘dvd’: Invalid cross-device link

"dvd" is in this case a symlink (but I also did the test with special
devices).

What I did was: copy that symlink from somewhere else, trying to turn on
project quotas on a parent directoy of that "dev" directory which gives
me:

  xfs_quota: skipping special file /var/foo/dev/dvd

When I examine the inode using xfs_db I get for my "dvd" symlink:

  core.projid_lo = 0
  core.projid_hi = 0

When I create a new symlink "foo" (with the same uid+gid as "dvd") and
examine it's inode using xfs_db:

  core.projid_lo = 2398
  core.projid_hi = 61

What I therefore seem to encounter is the problem mentioned in xfs_quota
for the project subcommand:

  An attempt to create a hard link to a file in the tree will only succeed if the project identi‐
       fier matches the project identifier for the tree.

I could now use xfs_io to fix all special files once (since newly
created ones will carry the project id directly), but why does
"xfs_quota -x -c 'project -s test1' /var" skip special files if it is
clearly possible and required to set the project id on them to have
hardlinks working again within the project dir?

Thanks in advance,
best regards,
Tiziano

-- 
stepping stone GmbH
Neufeldstrasse 9
CH-3012 Bern
Telefon: +41 31 332 53 63
www.stepping-stone.ch
tiziano.mueller@stepping-stone.ch

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: Setting project quotas on special files

[Jeff Liu]

On 11/10 2013 15:39 PM, Tiziano Müller wrote:
> Hi everyone
>
> It started with the following error:
>
>    dev # cp -al dvd dvd2
>    cp: cannot create hard link ‘dvd2’ to ‘dvd’: Invalid cross-device link
>
> "dvd" is in this case a symlink (but I also did the test with special
> devices).
This definitely would fail if you trying to create hard links across
volumes.
>
> What I did was: copy that symlink from somewhere else, trying to turn on
> project quotas on a parent directoy of that "dev" directory which gives
> me:
>
>   xfs_quota: skipping special file /var/foo/dev/dvd
symlink files will be skipped and xfs_quota consider it as special file
just like fifo, sock, etc...

If xfs_quota do check with follow symlinks, then the project id is potentially
applied to files on another filesystem. 

>
> When I examine the inode using xfs_db I get for my "dvd" symlink:
>
>   core.projid_lo = 0
>   core.projid_hi = 0
>
> When I create a new symlink "foo" (with the same uid+gid as "dvd") and
> examine it's inode using xfs_db:
>
>   core.projid_lo = 2398
>   core.projid_hi = 61
>
> What I therefore seem to encounter is the problem mentioned in xfs_quota
> for the project subcommand:
>
>   An attempt to create a hard link to a file in the tree will only succeed if the project identi‐
>        fier matches the project identifier for the tree.
>
> I could now use xfs_io to fix all special files once (since newly
> created ones will carry the project id directly), but why does
> "xfs_quota -x -c 'project -s test1' /var" skip special files if it is
> clearly possible and required to set the project id on them to have
> hardlinks working again within the project dir?
Sorry if I misunderstood your question, but the existing hard link files
would
notbe skipped IMO.  For the project subcommand, it means we could not create
hardlinks cross different project trees.

Thanks,
-Jeff

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: Setting project quotas on special files

[Tiziano Müller]

Hi Jeff

Sorry for top-posting, but it may be better to illustrate this with an
isolated example:

I have a volume named "backup" mounted with option "pquota", then I do
the following inside that mountpoint:

localhost backup # mkdir test
localhost backup # ln -s /invalid/location test/symlinkA
localhost backup # echo 42:/var/backup/test >> /etc/projects
localhost backup # echo test:42 >> /etc/projid
localhost backup # xfs_quota -x -c 'project -s test' /var/backup
Setting up project test (path /var/backup/test)...
xfs_quota: skipping special file /var/backup/test/symlinkA
Processed 1 (/etc/projects and cmdline) paths for project test with recursion depth infinite (-1).
localhost backup # cp -al test/symlinkA test/symlinkB
cp: cannot create hard link "test/symlinkB" to "test/symlinkA": Invalid cross-device link
localhost backup # 

Creating a symlink after setting up project quotas in the same directory
yields a different behaviour:

localhost backup # ln -s /invalid/location test/symlinkC
localhost backup # cp -al test/symlinkC test/symlinkD
localhost backup # 

The following shows that a symlink alone (and not its target) can have a
project id assigned and that the project id must be assigned to be able
to create a hardlink of a symlink (no matter where it points to).

For each of the symlinks test/symlink{A,B} run `stat` to get the inode
then use xfs_db on that inode to get the attributes:

localhost backup #  xfs_db -r -c 'inode 4362' -c 'p' /dev/vdb1  | grep projid
core.projid_lo = 0
core.projid_hi = 0
localhost backup #  xfs_db -r -c 'inode 4363' -c 'p' /dev/vdb1  | grep projid
core.projid_lo = 42
core.projid_hi = 0

Unfortunately xfs_io tries to follow the symlinks so it can not be used
to set the project manually.

What works though is renaming the file:

localhost backup # mv test/symlinkA test/symlinkAtmp
localhost backup # mv test/symlinkAtmp test/symlinkA
localhost backup # stat test/symlinkA
  File: ‘test/symlinkA’ -> ‘/invalid/location’
  Size: 17        	Blocks: 0          IO Block: 4096   symbolic link
Device: fe11h/65041d	Inode: 4364        Links: 1
Access: (0777/lrwxrwxrwx)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2013-11-10 15:49:44.870068178 +0100
Modify: 2013-11-10 15:49:44.870068178 +0100
Change: 2013-11-10 16:00:54.957571504 +0100
 Birth: -
localhost backup # xfs_db -r -c 'inode 4364' -c 'p' /dev/vdb1  | grep
projid
core.projid_lo = 42
core.projid_hi = 0
localhost backup # cp -al test/symlinkA test/symlinkB
localhost backup # 

Best regards,
Tiziano

Am Sonntag, den 10.11.2013, 02:46 -0800 schrieb Jeff Liu:
> On 11/10 2013 15:39 PM, Tiziano Müller wrote:
> > Hi everyone
> >
> > It started with the following error:
> >
> >    dev # cp -al dvd dvd2
> >    cp: cannot create hard link ‘dvd2’ to ‘dvd’: Invalid cross-device link
> >
> > "dvd" is in this case a symlink (but I also did the test with special
> > devices).
> This definitely would fail if you trying to create hard links across
> volumes.
> >
> > What I did was: copy that symlink from somewhere else, trying to turn on
> > project quotas on a parent directoy of that "dev" directory which gives
> > me:
> >
> >   xfs_quota: skipping special file /var/foo/dev/dvd
> symlink files will be skipped and xfs_quota consider it as special file
> just like fifo, sock, etc...
> 
> If xfs_quota do check with follow symlinks, then the project id is potentially
> applied to files on another filesystem. 
> 
> >
> > When I examine the inode using xfs_db I get for my "dvd" symlink:
> >
> >   core.projid_lo = 0
> >   core.projid_hi = 0
> >
> > When I create a new symlink "foo" (with the same uid+gid as "dvd") and
> > examine it's inode using xfs_db:
> >
> >   core.projid_lo = 2398
> >   core.projid_hi = 61
> >
> > What I therefore seem to encounter is the problem mentioned in xfs_quota
> > for the project subcommand:
> >
> >   An attempt to create a hard link to a file in the tree will only succeed if the project identi‐
> >        fier matches the project identifier for the tree.
> >
> > I could now use xfs_io to fix all special files once (since newly
> > created ones will carry the project id directly), but why does
> > "xfs_quota -x -c 'project -s test1' /var" skip special files if it is
> > clearly possible and required to set the project id on them to have
> > hardlinks working again within the project dir?
> Sorry if I misunderstood your question, but the existing hard link files
> would
> notbe skipped IMO.  For the project subcommand, it means we could not create
> hardlinks cross different project trees.
> 
> Thanks,
> -Jeff
> 
> _______________________________________________
> xfs mailing list
> xfs@oss.sgi.com
> http://oss.sgi.com/mailman/listinfo/xfs

-- 
stepping stone GmbH
Neufeldstrasse 9
CH-3012 Bern
Telefon: +41 31 332 53 63
www.stepping-stone.ch
tiziano.mueller@stepping-stone.ch

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: Setting project quotas on special files

[Jeff Liu]

Hi Tiziano,

On 11/10 2013 23:06 PM, Tiziano Müller wrote:
> 
> Hi Jeff
> 
> Sorry for top-posting, but it may be better to illustrate this with an
> isolated example:
> 
> I have a volume named "backup" mounted with option "pquota", then I do
> the following inside that mountpoint:
> 
> localhost backup # mkdir test
> localhost backup # ln -s /invalid/location test/symlinkA
> localhost backup # echo 42:/var/backup/test >> /etc/projects
> localhost backup # echo test:42 >> /etc/projid
> localhost backup # xfs_quota -x -c 'project -s test' /var/backup
> Setting up project test (path /var/backup/test)...
> xfs_quota: skipping special file /var/backup/test/symlinkA
So the symlink file is skipped.

> Processed 1 (/etc/projects and cmdline) paths for project test with recursion depth infinite (-1).
> localhost backup # cp -al test/symlinkA test/symlinkB
> cp: cannot create hard link "test/symlinkB" to "test/symlinkA": Invalid cross-device link
This is expected as you attempting to create a hardlink file upon a symlink
which is located at a different volume and the symlink was not inherited
from the project id(42 in this case) at this stage(i.e, it's projid is 0 as
a default projid, which is different to 42).

This restriction can be got from xfs_vn_link():

/*
 * If we are using project inheritance, we only allow hard link
 * creation in our tree when the project IDs are the same; else
 * the tree quota mechanism could be circumvented.
 */
 if (unlikely((tdp->i_d.di_flags & XFS_DIFLAG_PROJINHERIT) &&
     (xfs_get_projid(tdp) != xfs_get_projid(sip)))) {
 	error = XFS_ERROR(EXDEV);
 	goto error_return;
 }

> localhost backup # 
> 
> Creating a symlink after setting up project quotas in the same directory
> yields a different behaviour:
> 
> localhost backup # ln -s /invalid/location test/symlinkC
So a symlink file is created and it is inherited the projid in this case, because
symlink file quota space is accounted according to it's path length. IOWs, it would
be reflect on xfs_quota -xc 'report -[b|h]' if you create it with a longer path name
exceeds the inline inode space.

> localhost backup # cp -al test/symlinkC test/symlinkD
Therefore create a hardlink from this symlink file with an inherited projid is
desired to succeed.

> localhost backup # 
> 
> The following shows that a symlink alone (and not its target) can have a
> project id assigned and that the project id must be assigned to be able
> to create a hardlink of a symlink (no matter where it points to).
> 
> For each of the symlinks test/symlink{A,B} run `stat` to get the inode
> then use xfs_db on that inode to get the attributes:
> 
> localhost backup #  xfs_db -r -c 'inode 4362' -c 'p' /dev/vdb1  | grep projid
> core.projid_lo = 0
> core.projid_hi = 0
> localhost backup #  xfs_db -r -c 'inode 4363' -c 'p' /dev/vdb1  | grep projid
> core.projid_lo = 42
> core.projid_hi = 0
Just as above mentioned...

> Unfortunately xfs_io tries to follow the symlinks so it can not be used
> to set the project manually.
Well, why you want to set project quota via xfs_io manually? :-P.
 
> 
> What works though is renaming the file:
> 
> localhost backup # mv test/symlinkA test/symlinkAtmp
> localhost backup # mv test/symlinkAtmp test/symlinkA
rename(2) will create a new dentry as well as an inode, hence it would inherit
the projid.

> localhost backup # stat test/symlinkA
>   File: ‘test/symlinkA’ -> ‘/invalid/location’
>   Size: 17        	Blocks: 0          IO Block: 4096   symbolic link
> Device: fe11h/65041d	Inode: 4364        Links: 1
> Access: (0777/lrwxrwxrwx)  Uid: (    0/    root)   Gid: (    0/    root)
> Access: 2013-11-10 15:49:44.870068178 +0100
> Modify: 2013-11-10 15:49:44.870068178 +0100
> Change: 2013-11-10 16:00:54.957571504 +0100
>  Birth: -
> localhost backup # xfs_db -r -c 'inode 4364' -c 'p' /dev/vdb1  | grep
> projid
> core.projid_lo = 42
> core.projid_hi = 0
> localhost backup # cp -al test/symlinkA test/symlinkB
> localhost backup # 
In terms of project quota anyway, hardlink file is only accounted once for the owner
of the inode.  It's welcome to report back if you observed different behaviour.

Thanks,
-Jeff

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: Setting project quotas on special files

[Tiziano Müller]

Hi Jeff

Am Montag, den 11.11.2013, 14:43 +0800 schrieb Jeff Liu:
> Hi Tiziano,
> 
> On 11/10 2013 23:06 PM, Tiziano Müller wrote:
> > 
> > Hi Jeff
> > 
> > Sorry for top-posting, but it may be better to illustrate this with an
> > isolated example:
> > 
> > I have a volume named "backup" mounted with option "pquota", then I do
> > the following inside that mountpoint:
> > 
> > localhost backup # mkdir test
> > localhost backup # ln -s /invalid/location test/symlinkA
> > localhost backup # echo 42:/var/backup/test >> /etc/projects
> > localhost backup # echo test:42 >> /etc/projid
> > localhost backup # xfs_quota -x -c 'project -s test' /var/backup
> > Setting up project test (path /var/backup/test)...
> > xfs_quota: skipping special file /var/backup/test/symlinkA
> So the symlink file is skipped.
> 
> > Processed 1 (/etc/projects and cmdline) paths for project test with recursion depth infinite (-1).
> > localhost backup # cp -al test/symlinkA test/symlinkB
> > cp: cannot create hard link "test/symlinkB" to "test/symlinkA": Invalid cross-device link
> This is expected as you attempting to create a hardlink file upon a symlink
> which is located at a different volume

No, 'cp -al' does not follow the symlink (which is invalid in this
example btw) but tries to create a hardlink for that symlink.

>  and the symlink was not inherited
> from the project id(42 in this case) at this stage(i.e, it's projid is 0 as
> a default projid, which is different to 42).

Yes.

> 
> This restriction can be got from xfs_vn_link():
> 
> /*
>  * If we are using project inheritance, we only allow hard link
>  * creation in our tree when the project IDs are the same; else
>  * the tree quota mechanism could be circumvented.
>  */
>  if (unlikely((tdp->i_d.di_flags & XFS_DIFLAG_PROJINHERIT) &&
>      (xfs_get_projid(tdp) != xfs_get_projid(sip)))) {
>  	error = XFS_ERROR(EXDEV);
>  	goto error_return;
>  }
> 
> > localhost backup # 
> > 
> > Creating a symlink after setting up project quotas in the same directory
> > yields a different behaviour:
> > 
> > localhost backup # ln -s /invalid/location test/symlinkC
> So a symlink file is created and it is inherited the projid in this case, because
> symlink file quota space is accounted according to it's path length. IOWs, it would
> be reflect on xfs_quota -xc 'report -[b|h]' if you create it with a longer path name
> exceeds the inline inode space.
> 
> > localhost backup # cp -al test/symlinkC test/symlinkD
> Therefore create a hardlink from this symlink file with an inherited projid is
> desired to succeed.
> 
> > localhost backup # 
> > 
> > The following shows that a symlink alone (and not its target) can have a
> > project id assigned and that the project id must be assigned to be able
> > to create a hardlink of a symlink (no matter where it points to).
> > 
> > For each of the symlinks test/symlink{A,B} run `stat` to get the inode
> > then use xfs_db on that inode to get the attributes:
> > 
> > localhost backup #  xfs_db -r -c 'inode 4362' -c 'p' /dev/vdb1  | grep projid
> > core.projid_lo = 0
> > core.projid_hi = 0
> > localhost backup #  xfs_db -r -c 'inode 4363' -c 'p' /dev/vdb1  | grep projid
> > core.projid_lo = 42
> > core.projid_hi = 0
> Just as above mentioned...
> 
> > Unfortunately xfs_io tries to follow the symlinks so it can not be used
> > to set the project manually.
> Well, why you want to set project quota via xfs_io manually? :-P.

To set the project id for special files which did not get the project id
set during "xfs_quota -x -c 'project -s test'", which was leaving my
tree in a somewhat inconsistent state wrt the project id.

As Dave wrote me on IRC, the following works (I only tested it, credit
goes to him):

1. find the inode of all special files where the project id was not set
2. determine the project id to be set by recording their parent
directorie's project id. You will end up with a tuple (for 16 bit
project ids): (inode, projid_lo), resp. a triple (for 32 bit project
ids): (inode, projid_lo, projid_hi)
3. umount the volume
4. use 'xfs_db -x -c "inode <inode>" -c "write core.projid_lo
<projid_lo>" -c "write core.projid_hi <projid_hi>" <dev>' to set the
project id on those inodes manually
5. mount the volume but without "pquota" and umount again (this will
trigger a quotacheck on the next mount with pquota)
6. mount the volume with the "pquota" option again

As for the "why can xfs_quota not set the project id on special files":
it does it via ioctl(2), which needs a file handle, which is retrieved
via open(2), which "follows" the symlinks/device files.

>  
> > 
> > What works though is renaming the file:
> > 
> > localhost backup # mv test/symlinkA test/symlinkAtmp
> > localhost backup # mv test/symlinkAtmp test/symlinkA
> rename(2) will create a new dentry as well as an inode, hence it would inherit
> the projid.
> 
> > localhost backup # stat test/symlinkA
> >   File: ‘test/symlinkA’ -> ‘/invalid/location’
> >   Size: 17        	Blocks: 0          IO Block: 4096   symbolic link
> > Device: fe11h/65041d	Inode: 4364        Links: 1
> > Access: (0777/lrwxrwxrwx)  Uid: (    0/    root)   Gid: (    0/    root)
> > Access: 2013-11-10 15:49:44.870068178 +0100
> > Modify: 2013-11-10 15:49:44.870068178 +0100
> > Change: 2013-11-10 16:00:54.957571504 +0100
> >  Birth: -
> > localhost backup # xfs_db -r -c 'inode 4364' -c 'p' /dev/vdb1  | grep
> > projid
> > core.projid_lo = 42
> > core.projid_hi = 0
> > localhost backup # cp -al test/symlinkA test/symlinkB
> > localhost backup # 
> In terms of project quota anyway, hardlink file is only accounted once for the owner
> of the inode.  It's welcome to report back if you observed different behaviour.
> 
> Thanks,
> -Jeff
> 

best regards,
Tiziano

-- 
stepping stone GmbH
Neufeldstrasse 9
CH-3012 Bern
Telefon: +41 31 332 53 63
www.stepping-stone.ch
tiziano.mueller@stepping-stone.ch

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: Setting project quotas on special files

[Jeff Liu]

On 11/11 2013 15:38 PM, Tiziano Müller wrote:
> Hi Jeff
> 
> Am Montag, den 11.11.2013, 14:43 +0800 schrieb Jeff Liu:
>> Hi Tiziano,
>>
>> On 11/10 2013 23:06 PM, Tiziano Müller wrote:
>>>
>>> Hi Jeff
>>>
>>> Sorry for top-posting, but it may be better to illustrate this with an
>>> isolated example:
>>>
>>> I have a volume named "backup" mounted with option "pquota", then I do
>>> the following inside that mountpoint:
>>>
>>> localhost backup # mkdir test
>>> localhost backup # ln -s /invalid/location test/symlinkA
>>> localhost backup # echo 42:/var/backup/test >> /etc/projects
>>> localhost backup # echo test:42 >> /etc/projid
>>> localhost backup # xfs_quota -x -c 'project -s test' /var/backup
>>> Setting up project test (path /var/backup/test)...
>>> xfs_quota: skipping special file /var/backup/test/symlinkA
>> So the symlink file is skipped.
>>
>>> Processed 1 (/etc/projects and cmdline) paths for project test with recursion depth infinite (-1).
>>> localhost backup # cp -al test/symlinkA test/symlinkB
>>> cp: cannot create hard link "test/symlinkB" to "test/symlinkA": Invalid cross-device link
>> This is expected as you attempting to create a hardlink file upon a symlink
>> which is located at a different volume
> 
> No, 'cp -al' does not follow the symlink (which is invalid in this
> example btw) but tries to create a hardlink for that symlink.
> 
>>  and the symlink was not inherited
>> from the project id(42 in this case) at this stage(i.e, it's projid is 0 as
>> a default projid, which is different to 42).
> 
> Yes.
> 
>>
>> This restriction can be got from xfs_vn_link():
>>
>> /*
>>  * If we are using project inheritance, we only allow hard link
>>  * creation in our tree when the project IDs are the same; else
>>  * the tree quota mechanism could be circumvented.
>>  */
>>  if (unlikely((tdp->i_d.di_flags & XFS_DIFLAG_PROJINHERIT) &&
>>      (xfs_get_projid(tdp) != xfs_get_projid(sip)))) {
>>  	error = XFS_ERROR(EXDEV);
>>  	goto error_return;
>>  }
>>
>>> localhost backup # 
>>>
>>> Creating a symlink after setting up project quotas in the same directory
>>> yields a different behaviour:
>>>
>>> localhost backup # ln -s /invalid/location test/symlinkC
>> So a symlink file is created and it is inherited the projid in this case, because
>> symlink file quota space is accounted according to it's path length. IOWs, it would
>> be reflect on xfs_quota -xc 'report -[b|h]' if you create it with a longer path name
>> exceeds the inline inode space.
>>
>>> localhost backup # cp -al test/symlinkC test/symlinkD
>> Therefore create a hardlink from this symlink file with an inherited projid is
>> desired to succeed.
>>
>>> localhost backup # 
>>>
>>> The following shows that a symlink alone (and not its target) can have a
>>> project id assigned and that the project id must be assigned to be able
>>> to create a hardlink of a symlink (no matter where it points to).
>>>
>>> For each of the symlinks test/symlink{A,B} run `stat` to get the inode
>>> then use xfs_db on that inode to get the attributes:
>>>
>>> localhost backup #  xfs_db -r -c 'inode 4362' -c 'p' /dev/vdb1  | grep projid
>>> core.projid_lo = 0
>>> core.projid_hi = 0
>>> localhost backup #  xfs_db -r -c 'inode 4363' -c 'p' /dev/vdb1  | grep projid
>>> core.projid_lo = 42
>>> core.projid_hi = 0
>> Just as above mentioned...
>>
>>> Unfortunately xfs_io tries to follow the symlinks so it can not be used
>>> to set the project manually.
>> Well, why you want to set project quota via xfs_io manually? :-P.
> 
> To set the project id for special files which did not get the project id
> set during "xfs_quota -x -c 'project -s test'", which was leaving my
> tree in a somewhat inconsistent state wrt the project id.
Oh, now I can understand that you want to set projid to special files.
> 
> As Dave wrote me on IRC, the following works (I only tested it, credit
> goes to him):
> 
> 1. find the inode of all special files where the project id was not set
> 2. determine the project id to be set by recording their parent
> directorie's project id. You will end up with a tuple (for 16 bit
> project ids): (inode, projid_lo), resp. a triple (for 32 bit project
> ids): (inode, projid_lo, projid_hi)
> 3. umount the volume
> 4. use 'xfs_db -x -c "inode <inode>" -c "write core.projid_lo
> <projid_lo>" -c "write core.projid_hi <projid_hi>" <dev>' to set the
> project id on those inodes manually
> 5. mount the volume but without "pquota" and umount again (this will
> trigger a quotacheck on the next mount with pquota)
> 6. mount the volume with the "pquota" option again
> 
> As for the "why can xfs_quota not set the project id on special files":
> it does it via ioctl(2), which needs a file handle, which is retrieved
> via open(2), which "follows" the symlinks/device files.
Thank for letting me know this as well, I have been blocked for logging into IRC
these days due to network issue.

Thanks,
-Jeff

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs