[LTP] [PATCH v2 1/2] lib: add tst_fs_link_count.c

[LTP] [PATCH v2 1/2] lib: add tst_fs_link_count.c

[Xiaoguang Wang]

Add two test interfaces:
int tst_fs_fill_hardlinks(void (*cleanup) (void), const char *dir);
We can use tst_fs_fill_hardlinks() to get maximum number of hard links to
regular file in a mounted file system, which @dir is in.

int tst_fs_fill_subdirs(void (*cleanup) (void), const char *dir);
We can use tst_fs_fill_subdirs() to get maximum number of subdirectories
under directory @dir in a mounted file system.

Signed-off-by: Xiaoguang Wang <wangxg.fnst@cn.fujitsu.com>
---
 include/test.h          |  34 ++++++++++++
 lib/tst_fs_link_count.c | 136 ++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 170 insertions(+)
 create mode 100644 lib/tst_fs_link_count.c

diff --git a/include/test.h b/include/test.h
index 43fdc8b..348e5f4 100644
--- a/include/test.h
+++ b/include/test.h
@@ -321,6 +321,40 @@ const char *tst_strerrno(int err);
 int tst_path_has_mnt_flags(void (cleanup_fn)(void),
 		const char *path, const char *flags[]);
 
+/*
+ * lib/tst_fs_link_count.c
+ * Try to get maximum number of hard links to regular file in a mounted file
+ * system, which @dir is in.
+ * In its implementation, it uses link(2) to create hard links to a same file,
+ * try to reach the limit and will try at most 65535 times. If EMLINK is hit,
+ * we can think that the maximum number has been reached, return this number.
+ * Meanwhile under directory @dir: there will be numbers of files, which will
+ * point to the same inode on disk. Like below:
+ * testfile1, testfile2, testfile3, ..., testfile${returned number}.
+ * If some errors(ENOSPC, EDQUOT) occur or no error at all, we can think that
+ * the maximum number of hard links is infinite or not reachable because of
+ * filesystem's free space, for this case, return 0.
+ * If some other errors occur, we call tst_brkm(TBROK, ...).
+ */
+int tst_fs_fill_hardlinks(void (*cleanup) (void), const char *dir);
+
+/*
+ * lib/tst_fs_link_count.c
+ * Try to get maximum number of subdirectories under directory @dir in a
+ * mounted file system.
+ * In its implementation, it uses mkdir(2) to create numbers of subdirectories
+ * in @dir, try to reach the limit and will try at most 65535 times. If EMLINK
+ * is hit, we can think the maximum number has been reached, return this number.
+ * Meanwhile under directory @dir: there will be numbers of subdirectories,
+ * Like below:
+ * testdir1, testdir2, testdir3, ..., testdir${returned number}.
+ * If some errors(ENOSPC, EDQUOT) occur or no errors at all, we can think that
+ * the maximum number of subdirectories is infinite or not reachable because
+ * of filesystem's free space, for this case, return 0.
+ * If some other errors occur, we call tst_brkm(TBROK, ...).
+ */
+int tst_fs_fill_subdirs(void (*cleanup) (void), const char *dir);
+
 #ifdef TST_USE_COMPAT16_SYSCALL
 #define TCID_BIT_SUFFIX "_16"
 #elif  TST_USE_NEWER64_SYSCALL
diff --git a/lib/tst_fs_link_count.c b/lib/tst_fs_link_count.c
new file mode 100644
index 0000000..720d9b7
--- /dev/null
+++ b/lib/tst_fs_link_count.c
@@ -0,0 +1,136 @@
+/*
+ * Copyright (c) 2014 Fujitsu Ltd.
+ * Author: Xiaoguang Wang <wangxg.fnst@cn.fujitsu.com>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of version 2 of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it would be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
+ *
+ * You should have received a copy of the GNU General Public License along
+ * with this program; if not, write the Free Software Foundation, Inc.,
+ * 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+ */
+
+#include <string.h>
+#include <errno.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <unistd.h>
+
+#include "test.h"
+#include "usctest.h"
+#include "safe_macros.h"
+
+#define MAX_SANE_HARD_LINKS	65535
+
+int tst_fs_fill_hardlinks(void (*cleanup) (void), const char *dir)
+{
+	unsigned int i, j;
+	char base_filename[PATH_MAX], link_filename[PATH_MAX];
+	struct stat s;
+
+	SAFE_STAT(cleanup, dir, &s);
+	if ((s.st_mode & S_IFMT) != S_IFDIR)
+		tst_brkm(TBROK, cleanup, "%s is not directory", dir);
+
+	sprintf(base_filename, "%s/testfile1", dir);
+	SAFE_TOUCH(cleanup, base_filename, 0644, NULL);
+
+	for (i = 2; i <= MAX_SANE_HARD_LINKS; i++) {
+		sprintf(link_filename, "%s/testfile%d", dir, i);
+
+		if (link(base_filename, link_filename) == 0)
+			continue;
+
+		switch (errno) {
+		case EMLINK:
+			SAFE_STAT(cleanup, base_filename, &s);
+			if (s.st_nlink != (i - 1)) {
+				tst_brkm(TBROK, cleanup, "wrong number of "
+					 "hard links for %s have %i, should be"
+					 " %d", base_filename,
+					 (int)s.st_nlink, i - 1);
+			} else {
+				tst_resm(TINFO, "the maximum number of hard "
+					 "links to %s is hit: %d",
+					 base_filename, (int)s.st_nlink);
+				return s.st_nlink;
+			}
+		case ENOSPC:
+		case EDQUOT:
+			fprintf(stderr, "link(%s, %s) failed:%s\n",
+				base_filename, link_filename,
+				strerror(errno));
+			goto max_hardlinks_cleanup;
+		default:
+			tst_brkm(TBROK, cleanup, "link(%s, %s) failed"
+				 "unexpectedly: %s", base_filename,
+				 link_filename, strerror(errno));
+		}
+	}
+
+max_hardlinks_cleanup:
+	for (j = 1; j < i; j++) {
+		sprintf(link_filename, "%s/testfile%d", dir, j);
+		SAFE_UNLINK(cleanup, link_filename);
+	}
+
+	return 0;
+}
+
+int tst_fs_fill_subdirs(void (*cleanup) (void), const char *dir)
+{
+	unsigned int i, j;
+	char dirname[PATH_MAX];
+	struct stat s;
+
+	SAFE_STAT(cleanup, dir, &s);
+	if ((s.st_mode & S_IFMT) != S_IFDIR)
+		tst_brkm(TBROK, cleanup, "%s is not directory", dir);
+
+	for (i = 1; i <= MAX_SANE_HARD_LINKS; i++) {
+		sprintf(dirname, "%s/testdir%d", dir, i);
+
+		if (mkdir(dirname, 0755) == 0)
+			continue;
+
+		switch (errno) {
+		case EMLINK:
+			SAFE_STAT(cleanup, dir, &s);
+			/*
+			 * here i+1 is calculated by i-1+2(contain '.' and dir)
+			 */
+			if (s.st_nlink != (i + 1)) {
+				tst_brkm(TBROK, cleanup, "%s link counts have"
+					 "%d, should be %d", dir,
+					 (int)s.st_nlink, i + 1);
+			} else {
+				tst_resm(TINFO, "the maximum subdirectories in "
+				 "%s is hit: %d", dir, (int)s.st_nlink);
+				return s.st_nlink;
+			}
+		case ENOSPC:
+		case EDQUOT:
+			fprintf(stderr, "mkdir(%s, 0755) failed: %s\n",
+				dirname, strerror(errno));
+			goto max_subdirs_cleanup;
+		default:
+			tst_brkm(TBROK, cleanup, "mkdir(%s, 0755) failed"
+				 "unexpectedly: %s", dirname,
+				 strerror(errno));
+		}
+
+	}
+
+max_subdirs_cleanup:
+	for (j = 1; j < i; j++) {
+		sprintf(dirname, "%s/testdir%d", dir, j);
+		SAFE_RMDIR(cleanup, dirname);
+	}
+
+	return 0;
+}
-- 
1.8.2.1


------------------------------------------------------------------------------
HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions
Find What Matters Most in Your Big Data with HPCC Systems
Open Source. Fast. Scalable. Simple. Ideal for Dirty Data.
Leverages Graph Analysis for Fast Processing & Easy Data Exploration
http://www.hpccsystems.com
_______________________________________________
Ltp-list mailing list
Ltp-list@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/ltp-list

[LTP] [PATCH v2 2/2] linkat/linkat02.c: cleanup

[Xiaoguang Wang]

Remove the check against XFS when testing EMLINK error value.
Using the new interface provided by lib/tst_fs_link_count.c.

Signed-off-by: Xiaoguang Wang <wangxg.fnst@cn.fujitsu.com>
---
 testcases/kernel/syscalls/linkat/linkat02.c | 58 ++++++-----------------------
 1 file changed, 11 insertions(+), 47 deletions(-)

diff --git a/testcases/kernel/syscalls/linkat/linkat02.c b/testcases/kernel/syscalls/linkat/linkat02.c
index b899fa6..9fac935 100644
--- a/testcases/kernel/syscalls/linkat/linkat02.c
+++ b/testcases/kernel/syscalls/linkat/linkat02.c
@@ -36,7 +36,6 @@
 #include "linux_syscall_numbers.h"
 #include "safe_macros.h"
 #include "lapi/fcntl.h"
-#include "tst_fs_type.h"
 
 #define DIR_MODE	(S_IRUSR|S_IWUSR|S_IXUSR|S_IRGRP| \
 			 S_IXGRP|S_IROTH|S_IXOTH)
@@ -47,7 +46,7 @@
 #define TEST_EACCES2	"./tmp/testeeacces2"
 #define TEST_EROFS	"mntpoint"
 #define TEST_EROFS2	"mntpoint/testerofs2"
-#define TEST_EMLINK	"mntpoint/testemlink"
+#define TEST_EMLINK	"emlink_dir/testfile1"
 #define BASENAME	"mntpoint/basename"
 
 static char lname[PATH_MAX];
@@ -55,13 +54,13 @@ static char nametoolong[PATH_MAX+2];
 static char *fstype = "ext2";
 static char *device;
 static int mount_flag;
+static int max_hardlinks;
 
 static void setup(void);
 static void cleanup(void);
 static void setup_eacces(void);
 static void cleanup_eacces(void);
 static void setup_erofs(void);
-static void setup_emlink(void);
 static void help(void);
 
 static option_t options[] = {
@@ -84,7 +83,7 @@ static struct test_struct {
 	{TEST_ELOOP, TEST_FILE, AT_SYMLINK_FOLLOW, ELOOP, NULL, NULL},
 	{TEST_EACCES, TEST_EACCES2, 0, EACCES, setup_eacces, cleanup_eacces},
 	{TEST_EROFS, TEST_EROFS2, 0, EROFS, setup_erofs, NULL},
-	{TEST_EMLINK, lname, 0, EMLINK, setup_emlink, NULL},
+	{TEST_EMLINK, lname, 0, EMLINK, NULL, NULL},
 };
 
 char *TCID = "linkat02";
@@ -96,8 +95,6 @@ static void linkat_verify(const struct test_struct *);
 static int exp_enos[] = { ENAMETOOLONG, EEXIST, ELOOP,
 			  EACCES, EROFS, EMLINK, 0 };
 
-static long fs_type;
-
 int main(int ac, char **av)
 {
 	int lc;
@@ -130,14 +127,13 @@ int main(int ac, char **av)
 
 static void linkat_verify(const struct test_struct *desc)
 {
+	if (desc->expected_errno == EMLINK && max_hardlinks == 0) {
+		tst_resm(TCONF, "EMLINK test is not appropriate");
+		return;
+	}
+
 	if (desc->setupfunc != NULL) {
-		if (desc->setupfunc == setup_emlink &&
-		    fs_type == TST_XFS_MAGIC) {
-			tst_resm(TCONF, "Test skipped XFS filesystem.");
-			return;
-		} else {
-			desc->setupfunc();
-		}
+		desc->setupfunc();
 	}
 
 	TEST(ltp_syscall(__NR_linkat, AT_FDCWD, desc->oldfname,
@@ -166,8 +162,6 @@ static void linkat_verify(const struct test_struct *desc)
 
 static void setup(void)
 {
-	long link_max = 0;
-
 	if ((tst_kvercmp(2, 6, 16)) < 0)
 		tst_brkm(TCONF, NULL, "This test needs kernel 2.6.16 or newer");
 
@@ -202,38 +196,8 @@ static void setup(void)
 	}
 	mount_flag = 1;
 
-	SAFE_TOUCH(cleanup, TEST_EMLINK, 0666, NULL);
-
-	fs_type = tst_fs_type(cleanup, "mntpoint");
-	if (fs_type == TST_XFS_MAGIC)
-		return;
-
-	while (1) {
-		sprintf(lname, "%s%ld", BASENAME, ++link_max);
-		TEST(link(TEST_EMLINK, lname));
-		if (TEST_RETURN == -1) {
-			switch (TEST_ERRNO) {
-			case EMLINK:
-				tst_resm(TINFO, "for %s the max links is %ld",
-					 fstype, link_max);
-				break;
-			default:
-				tst_brkm(TBROK | TTERRNO, cleanup,
-					 "Unexpected error: ");
-				break;
-			}
-			break;
-		}
-	}
-}
-
-static void setup_emlink(void)
-{
-	if (mount(device, "mntpoint", fstype, MS_REMOUNT, NULL) < 0) {
-		tst_brkm(TBROK | TERRNO, cleanup,
-			 "remount device:%s failed", device);
-	}
-	mount_flag = 1;
+	SAFE_MKDIR(cleanup, "emlink_dir", DIR_MODE);
+	max_hardlinks = tst_fs_fill_hardlinks(cleanup, "emlink_dir");
 }
 
 static void setup_eacces(void)
-- 
1.8.2.1


------------------------------------------------------------------------------
HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions
Find What Matters Most in Your Big Data with HPCC Systems
Open Source. Fast. Scalable. Simple. Ideal for Dirty Data.
Leverages Graph Analysis for Fast Processing & Easy Data Exploration
http://www.hpccsystems.com
_______________________________________________
Ltp-list mailing list
Ltp-list@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/ltp-list

Re: [LTP] [PATCH v2 1/2] lib: add tst_fs_link_count.c

[chrubis]

Hi!
> +/*
> + * lib/tst_fs_link_count.c
> + * Try to get maximum number of subdirectories under directory @dir in a
> + * mounted file system.
> + * In its implementation, it uses mkdir(2) to create numbers of subdirectories
> + * in @dir, try to reach the limit and will try at most 65535 times. If EMLINK
> + * is hit, we can think the maximum number has been reached, return this number.
> + * Meanwhile under directory @dir: there will be numbers of subdirectories,
> + * Like below:
> + * testdir1, testdir2, testdir3, ..., testdir${returned number}.
> + * If some errors(ENOSPC, EDQUOT) occur or no errors at all, we can think that
> + * the maximum number of subdirectories is infinite or not reachable because
> + * of filesystem's free space, for this case, return 0.
> + * If some other errors occur, we call tst_brkm(TBROK, ...).
> + */
> +int tst_fs_fill_subdirs(void (*cleanup) (void), const char *dir);

This function actually tries to figure out maximal number of elements a
directory can hold (it does not matter if these are files or
directories). So it should be renamed to tst_fs_fill_dir().

Moreover I've been unable to get EMLINK for any filesystem in this case,
it looks like ENOSPC is what you get when maximal number of elemenents
in directory is reached (the free space on filesystem is fine but you
are out of inodes/clusters etc).

I have, slightly modified version of this patch I will commit once this
function is figured out but before I do so I wanted to ask if you plan
to use this function for any testcase? If not I would remove it because
as it is it's not implemented correctly and fixing it would require more
research and effort.

-- 
Cyril Hrubis
chrubis@suse.cz

------------------------------------------------------------------------------
HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions
Find What Matters Most in Your Big Data with HPCC Systems
Open Source. Fast. Scalable. Simple. Ideal for Dirty Data.
Leverages Graph Analysis for Fast Processing & Easy Data Exploration
http://p.sf.net/sfu/hpccsystems
_______________________________________________
Ltp-list mailing list
Ltp-list@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/ltp-list

Re: [LTP] [PATCH v2 1/2] lib: add tst_fs_link_count.c

[chrubis]

Hi!
> > This function actually tries to figure out maximal number of elements a
> > directory can hold (it does not matter if these are files or
> > directories). So it should be renamed to tst_fs_fill_dir().
> Indeed there is difference between creating files and directories. When creating a regular file under a
> directory, the directory's inode count won't be increased, but if creating a sub-directory in a directory,
> the parent directory's inode count will be increased(add 1). This is because the ".." in the created
> sub-directory will point to the parent directory's inode. For regular files, there is no need to increase
> parent's inode count.
> This url has an explanation: http://www.vidarholen.net/contents/blog/?p=18
> 
> Steps to verify this:
> # mkfs.ext3 -N 80000 /dev/loop0
> # mount -t ext3 /dev/loop0 mntpoint/; cd mntpoint
> # mkdir lege; cd lege
> 
> # for ((i=1; i<= 40000; i++)); do touch $i; done; # this will succeed and you will create 40000 regular files.
> 
> # for ((i=1; i<= 40000; i++)); do mkdir $i; done;
> this will fail. in ext3, you are only allowed to create 31998 sub-directories(there is already "." and "..")
> This limit is imposed by " #define EXT3_LINK_MAX 32000" defined in ext3 kernel source code.
> For ext2, the limit is also 32000. for ext4, this is 65000 or infinite, which depends whether dir_index
> feature is enabled.

You are right. I've been confused because the fs I've created was too
small, so the default number of inodes was less than the limit and it
failed with ENOSPC in this case.

And because of that the limit for maximal number of directories and
files in a directory seemed to be the same.

Both creating a bigger loop device and explicitly setting the number of
inodes for mkfs fixes the problem.

> > I have, slightly modified version of this patch I will commit once this
> > function is figured out but before I do so I wanted to ask if you plan
> > to use this function for any testcase? If not I would remove it because
> 
> Yeah, this function will be used.
> For rename(2) or renameat(2)'s EMLINK test, the manpage sasys that:
> EMLINK The file named by old is a directory, and the link count of  the
>               parent directory of new would exceed {LINK_MAX}.
> We need to use tst_fs_fill_subdirs() to construct the condition.

Ok, I will push (slightly modified version).

I did a few small changes:

* it creates the directory automatically if it does not exist

* uses tst_resm() instead of printf()

* changed the file/directory numbering to start from 0
  which simplifies the code a bit

* added two simple tests under lib/tests/

-- 
Cyril Hrubis
chrubis@suse.cz

------------------------------------------------------------------------------
HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions
Find What Matters Most in Your Big Data with HPCC Systems
Open Source. Fast. Scalable. Simple. Ideal for Dirty Data.
Leverages Graph Analysis for Fast Processing & Easy Data Exploration
http://p.sf.net/sfu/hpccsystems
_______________________________________________
Ltp-list mailing list
Ltp-list@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/ltp-list

Re: [LTP] [PATCH v2 2/2] linkat/linkat02.c: cleanup

[chrubis]

Hi!
> -	while (1) {
> -		sprintf(lname, "%s%ld", BASENAME, ++link_max);
> -		TEST(link(TEST_EMLINK, lname));
> -		if (TEST_RETURN == -1) {
> -			switch (TEST_ERRNO) {
> -			case EMLINK:
> -				tst_resm(TINFO, "for %s the max links is %ld",
> -					 fstype, link_max);
> -				break;
> -			default:
> -				tst_brkm(TBROK | TTERRNO, cleanup,
> -					 "Unexpected error: ");
> -				break;
> -			}
> -			break;
> -		}
> -	}
> -}

You have removed the code that initializes lname which broke the test
because empty string was passed to the link() call. I've fixed it and
pushed, thanks.

-- 
Cyril Hrubis
chrubis@suse.cz

------------------------------------------------------------------------------
HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions
Find What Matters Most in Your Big Data with HPCC Systems
Open Source. Fast. Scalable. Simple. Ideal for Dirty Data.
Leverages Graph Analysis for Fast Processing & Easy Data Exploration
http://p.sf.net/sfu/hpccsystems
_______________________________________________
Ltp-list mailing list
Ltp-list@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/ltp-list

Re: [LTP] [PATCH v2 1/2] lib: add tst_fs_link_count.c

[chrubis]

Hi!
> > Ok, I will push (slightly modified version).
> > 
> > I did a few small changes:
> > 
> > * it creates the directory automatically if it does not exist
> > 
> > * uses tst_resm() instead of printf()
> > 
> > * changed the file/directory numbering to start from 0
> >   which simplifies the code a bit
> > 
> > * added two simple tests under lib/tests/
> 
> Thanks very much for your modification.
> It really looks better than my v2 version, especially for the new comments
> in include/test.h, it is more neater, thanks.

We should also add documentation into the Test Writing Guidelines. Will
you send a patch or should I do it?

-- 
Cyril Hrubis
chrubis@suse.cz

------------------------------------------------------------------------------
HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions
Find What Matters Most in Your Big Data with HPCC Systems
Open Source. Fast. Scalable. Simple. Ideal for Dirty Data.
Leverages Graph Analysis for Fast Processing & Easy Data Exploration
http://p.sf.net/sfu/hpccsystems
_______________________________________________
Ltp-list mailing list
Ltp-list@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/ltp-list