Problem: LTP linkat01 test fails on nfs directory (NFS v3)

Problem: LTP linkat01 test fails on nfs directory (NFS v3)

[Vitaliy Gusev]

[-- Attachment #1: Type: text/plain, Size: 858 bytes --]

Hello.

Tested kernels: 2.6.18, 2.6.22, 2.6.23-rc2

Steps to reproduce: Suppose that we have mounted some directory from nfs v3 
server with default options. Also we  have the two directories in this 
mountpoint and each directory has hard linked file. Try to open those files 
and write to one and read from another. Data will not be equal. (Testcase: 
attached hardlink_test.c)

Analysis: Although these hard linked files have same nfs_fattr::fileid but 
have different nfs_fh fhandle. In this case nfs_find_actor() function 
(fs/nfs/inode.c) returns false during opening each file:

nfs_find_actor()
{
...
	if (nfs_compare_fh(NFS_FH(inode), fh))
		return 0;
...
}

Therefore for each of hard links new struct inode is allocated. It leads to 
cache aliasing.

Please explain why nfs_find_actor() function compares file handles?

-- 
Thanks,
Vitaliy Gusev


[-- Attachment #2: hardlink_test.c --]
[-- Type: text/x-csrc, Size: 801 bytes --]

#include <stdio.h>
#include <assert.h>
#include <string.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <err.h>

#define SLEEP 15

int main(int argc, char *argv[])
{
	int fd1,fd2;
	char *f1 = argv[1];
	char *f2 = argv[2];
	char buf[1000]= {0,};
	char wrbuf[200];
	int ret;

	assert(argc == 3);

	printf("write to: %s\nread from: %s\n\n",
	       f1, f2);

	fd1 = open(f1, O_RDWR);
	fd2 = open(f2, O_RDONLY);
	if (fd1 < 0 || fd2 < 0)
		err(1, "error open");
		
	sprintf(wrbuf, "test message-%d", getpid());
	printf("Write: [%s]\n", wrbuf);
	ret = write(fd1, wrbuf, strlen(wrbuf));
	if (ret < 0)
		err(1, "error write");

	sleep(1);
	ret = read(fd2, buf, sizeof(buf) - 1);
	if (ret < 0)
		err(1, "error read");
	printf("Read:  [%s]\n", buf);
	return 0;
}

Problem: LTP linkat01 test fails on nfs directory (NFS v3)

[Vitaliy Gusev]

[-- Attachment #1: Type: text/plain, Size: 858 bytes --]

Hello.

Tested kernels: 2.6.18, 2.6.22, 2.6.23-rc2

Steps to reproduce: Suppose that we have mounted some directory from nfs v3 
server with default options. Also we  have the two directories in this 
mountpoint and each directory has hard linked file. Try to open those files 
and write to one and read from another. Data will not be equal. (Testcase: 
attached hardlink_test.c)

Analysis: Although these hard linked files have same nfs_fattr::fileid but 
have different nfs_fh fhandle. In this case nfs_find_actor() function 
(fs/nfs/inode.c) returns false during opening each file:

nfs_find_actor()
{
...
	if (nfs_compare_fh(NFS_FH(inode), fh))
		return 0;
...
}

Therefore for each of hard links new struct inode is allocated. It leads to 
cache aliasing.

Please explain why nfs_find_actor() function compares file handles?

-- 
Thanks,
Vitaliy Gusev


[-- Attachment #2: hardlink_test.c --]
[-- Type: text/x-csrc, Size: 801 bytes --]

#include <stdio.h>
#include <assert.h>
#include <string.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <err.h>

#define SLEEP 15

int main(int argc, char *argv[])
{
	int fd1,fd2;
	char *f1 = argv[1];
	char *f2 = argv[2];
	char buf[1000]= {0,};
	char wrbuf[200];
	int ret;

	assert(argc == 3);

	printf("write to: %s\nread from: %s\n\n",
	       f1, f2);

	fd1 = open(f1, O_RDWR);
	fd2 = open(f2, O_RDONLY);
	if (fd1 < 0 || fd2 < 0)
		err(1, "error open");
		
	sprintf(wrbuf, "test message-%d", getpid());
	printf("Write: [%s]\n", wrbuf);
	ret = write(fd1, wrbuf, strlen(wrbuf));
	if (ret < 0)
		err(1, "error write");

	sleep(1);
	ret = read(fd2, buf, sizeof(buf) - 1);
	if (ret < 0)
		err(1, "error read");
	printf("Read:  [%s]\n", buf);
	return 0;
}

Re: Problem: LTP linkat01 test fails on nfs directory (NFS v3)

[Trond Myklebust]

On Fri, 2007-09-21 at 13:13 +0400, Vitaliy Gusev wrote:
> Hello.
> 
> Tested kernels: 2.6.18, 2.6.22, 2.6.23-rc2
> 
> Steps to reproduce: Suppose that we have mounted some directory from nfs v3 
> server with default options. Also we  have the two directories in this 
> mountpoint and each directory has hard linked file. Try to open those files 
> and write to one and read from another. Data will not be equal. (Testcase: 
> attached hardlink_test.c)

Please retry after re-exporting the filesystem using the highly
recommended "no_subtree_check" option. The default subtree_check option
is broken: it changes the filehandle of the file upon a cross-directory
rename() of that file.

> Analysis: Although these hard linked files have same nfs_fattr::fileid but 
> have different nfs_fh fhandle. In this case nfs_find_actor() function 
> (fs/nfs/inode.c) returns false during opening each file:
> 
> nfs_find_actor()
> {
> ...
> 	if (nfs_compare_fh(NFS_FH(inode), fh))
> 		return 0;
> ...
> }
> 
> Therefore for each of hard links new struct inode is allocated. It leads to 
> cache aliasing.
> 
> Please explain why nfs_find_actor() function compares file handles?

'cos this is the only way to know that two files are the same. fileid is
not always supported by servers: it is an optional NFSv4 attribute, and
on NFSv3, most non-posix filesystems will fake it using something like
iunique().

Comparing filehandles allows you to be certain that two files are the
same if the filehandles are equal. If they are not equal, then that does
not guarantee that the files are different, but then how else are you
going to determine it?

Trond

Re: Problem: LTP linkat01 test fails on nfs directory (NFS v3)

[Vitaliy Gusev]

On the Friday 21 September 2007 16:37 Trond Myklebust, wrote:
> On Fri, 2007-09-21 at 13:13 +0400, Vitaliy Gusev wrote:
> > Hello.
> >
> > Tested kernels: 2.6.18, 2.6.22, 2.6.23-rc2
> >
> > Steps to reproduce: Suppose that we have mounted some directory from nfs
> > v3 server with default options. Also we  have the two directories in this
> > mountpoint and each directory has hard linked file. Try to open those
> > files and write to one and read from another. Data will not be equal.
> > (Testcase: attached hardlink_test.c)
>
> Please retry after re-exporting the filesystem using the highly
> recommended "no_subtree_check" option. The default subtree_check option
> is broken: it changes the filehandle of the file upon a cross-directory
> rename() of that file.

Ok, problem goes out. Default option depends on nfs-utils release.

> >
> > Please explain why nfs_find_actor() function compares file handles?
>
> 'cos this is the only way to know that two files are the same. fileid is
> not always supported by servers: it is an optional NFSv4 attribute, and

>From RFC3010 ( NFSv4 ) section 2.4.1 :
   "For example, if paths /a/b/c and /a/d/c refer to the same file, the       
server SHOULD return the same filehandle for both path names traversals".

For NFSv4  problem should not be shown.

> on NFSv3, most non-posix filesystems will fake it using something like
> iunique().
>
> Comparing filehandles allows you to be certain that two files are the
> same if the filehandles are equal. If they are not equal, then that does
> not guarantee that the files are different, but then how else are you
> going to determine it?
>

-- 
Thanks,
Vitaliy Gusev