linux /dev and normal files

linux /dev and normal files

[Elliott, Robert (Server Storage)]

> -----Original Message-----
> From: fio-owner@vger.kernel.org [mailto:fio-owner@vger.kernel.org] On Behalf
> Of Sitsofe Wheeler
...
> filename=\\.\physicaldrive1
> ioengine=windowsaio
> direct=1
...
> size=86%
> bs=4k
...

I noticed that, in linux, if you select 
	filename=/dev/sd<not there>
	size=<something> 

(e.g., if you run fio after a device has failed), it  creates a 
normal file of the specified size (which could be quite large 
if using a script such as above).

If you use direct=1, the file is created, followed by this error:
fio: pid=8914, err=22/file:filesetup.c:611, func=open(/dev/sdad), error=Invalid argument
fio: looks like your file system does not support direct=1/buffered=0
fio: looks like your file system does not support direct=1/buffered=0
fio: destination does not support O_DIRECT

If you don't use size=, no file is created and this error
occurs:
fio: pid=0, err=22/file:filesetup.c:820, func=total_file_size, error=Invalid argument
drive_ag: you need to specify size=

If you recreate the device, udevd blows away any such file with
the block device node file, so it's not a good place to put
normal files, even if that is intentional.

In Windows, all "\\.\" paths are assumed to mean block devices,
so fio doesn't inadvertently try to create a normal file in
such a location.

Should fio treat "/dev" paths in linux the same way?

---
Rob Elliott    HP Server Storage

Re: linux /dev and normal files

[Jens Axboe]

On 2014-09-30 17:49, Elliott, Robert (Server Storage) wrote:
>
>> -----Original Message-----
>> From: fio-owner@vger.kernel.org [mailto:fio-owner@vger.kernel.org] On Behalf
>> Of Sitsofe Wheeler
> ...
>> filename=\\.\physicaldrive1
>> ioengine=windowsaio
>> direct=1
> ...
>> size=86%
>> bs=4k
> ...
>
> I noticed that, in linux, if you select
> 	filename=/dev/sd<not there>
> 	size=<something>
>
> (e.g., if you run fio after a device has failed), it  creates a
> normal file of the specified size (which could be quite large
> if using a script such as above).
>
> If you use direct=1, the file is created, followed by this error:
> fio: pid=8914, err=22/file:filesetup.c:611, func=open(/dev/sdad), error=Invalid argument
> fio: looks like your file system does not support direct=1/buffered=0
> fio: looks like your file system does not support direct=1/buffered=0
> fio: destination does not support O_DIRECT

This is because /dev is usually devtmpfs these days, and that fs does 
not support O_DIRECT.

> If you don't use size=, no file is created and this error
> occurs:
> fio: pid=0, err=22/file:filesetup.c:820, func=total_file_size, error=Invalid argument
> drive_ag: you need to specify size=
>
> If you recreate the device, udevd blows away any such file with
> the block device node file, so it's not a good place to put
> normal files, even if that is intentional.
>
> In Windows, all "\\.\" paths are assumed to mean block devices,
> so fio doesn't inadvertently try to create a normal file in
> such a location.
>
> Should fio treat "/dev" paths in linux the same way?

No, I don't think so. It's a directory like anything else. Who knows 
what will happen in the future, with all the core OS changes in Linux. 
/dev could be a symlink, or special files could be in a different 
location completely.  What about FreeBSD, should we do the same there? 
Or Solaris?

I'd much rather keep fio ignorant of any "special" directories, even if 
it means that sometimes you do potentially run into issues like the 
above, where you specify a block device that does not exist.

-- 
Jens Axboe

RE: linux /dev and normal files

[Elliott, Robert (Server Storage)]

> -----Original Message-----
> From: Jens Axboe [mailto:axboe@kernel.dk]
...
> I'd much rather keep fio ignorant of any "special" directories, even if
> it means that sometimes you do potentially run into issues like the
> above, where you specify a block device that does not exist.

How about an option in the script:
direct=2   file must exist and be a block device, otherwise skip it

If the size is too big, this is the error:

drive_r: Laying out IO file(s) (1 file(s) / 1572864000MB)
fio: posix_fallocate fails: No space left on device

fio: ENOSPC on laying out file, stopping
fio: looks like your file system does not support direct=1/buffered=0
fio: destination does not support O_DIRECT
fio: looks like your file system does not support direct=1/buffered=0
fio: destination does not support O_DIRECT
fio: pid=8991, err=22/file:filesetup.c:611, func=open(/dev/sdnothere), error=Invalid argument
fio: pid=8990, err=22/file:filesetup.c:611, func=open(/dev/sdnothere), error=Invalid argument

and it leaves a "file" that looks like:

$ ls -alt /dev/sd*
-rw-r--r-- 1 root root 1649267441664000 Sep 30 20:10 /dev/sdnothere

Re: linux /dev and normal files

[Jens Axboe]

On 2014-09-30 19:12, Elliott, Robert (Server Storage) wrote:
>
>
>> -----Original Message-----
>> From: Jens Axboe [mailto:axboe@kernel.dk]
> ...
>> I'd much rather keep fio ignorant of any "special" directories, even if
>> it means that sometimes you do potentially run into issues like the
>> above, where you specify a block device that does not exist.
>
> How about an option in the script:
> direct=2   file must exist and be a block device, otherwise skip it

Might be a bit nasty to overload direct= like that. But I'd be open to 
adding an option that says must_be_bdev or something, which can be a 
bool as well.

> If the size is too big, this is the error:
>
> drive_r: Laying out IO file(s) (1 file(s) / 1572864000MB)
> fio: posix_fallocate fails: No space left on device
>
> fio: ENOSPC on laying out file, stopping
> fio: looks like your file system does not support direct=1/buffered=0
> fio: destination does not support O_DIRECT
> fio: looks like your file system does not support direct=1/buffered=0
> fio: destination does not support O_DIRECT
> fio: pid=8991, err=22/file:filesetup.c:611, func=open(/dev/sdnothere), error=Invalid argument
> fio: pid=8990, err=22/file:filesetup.c:611, func=open(/dev/sdnothere), error=Invalid argument
>
> and it leaves a "file" that looks like:
>
> $ ls -alt /dev/sd*
> -rw-r--r-- 1 root root 1649267441664000 Sep 30 20:10 /dev/sdnothere

Yeah I know, I've had that happen myself 1-2 times.

-- 
Jens Axboe

Re: linux /dev and normal files

[Jens Axboe]

[-- Attachment #1: Type: text/plain, Size: 1238 bytes --]

On 2014-09-30 19:17, Jens Axboe wrote:
> On 2014-09-30 19:12, Elliott, Robert (Server Storage) wrote:
>>
>>
>>> -----Original Message-----
>>> From: Jens Axboe [mailto:axboe@kernel.dk]
>> ...
>>> I'd much rather keep fio ignorant of any "special" directories, even if
>>> it means that sometimes you do potentially run into issues like the
>>> above, where you specify a block device that does not exist.
>>
>> How about an option in the script:
>> direct=2   file must exist and be a block device, otherwise skip it
>
> Might be a bit nasty to overload direct= like that. But I'd be open to
> adding an option that says must_be_bdev or something, which can be a
> bool as well.

Totally untested patch attached (it compiles). There's a new option, 
filetype. If not set, if will allow any file type. Can be set to:

any        default, allow any file type
file       regular file
bdev       block device
chardev    character device
pipe       pipe

So for your case, you'd set

filetype=bdev

and you'd avoid the issue. Note that it only works for files specified 
with filename=. Any other way of adding a file (blktrace replay, 
opendir=, or fio added files when no filename= given) will ignore the 
filetype setting.

-- 
Jens Axboe


[-- Attachment #2: filetype.patch --]
[-- Type: text/x-patch, Size: 7871 bytes --]

diff --git a/cconv.c b/cconv.c
index 4a40ed0d647b..61867625282a 100644
--- a/cconv.c
+++ b/cconv.c
@@ -69,6 +69,7 @@ void convert_thread_options_to_cpu(struct thread_options *o,
 	string_to_cpu(&o->profile, top->profile);
 	string_to_cpu(&o->cgroup, top->cgroup);
 
+	o->filetype = le32_to_cpu(top->filetype);
 	o->td_ddir = le32_to_cpu(top->td_ddir);
 	o->rw_seq = le32_to_cpu(top->rw_seq);
 	o->kb_base = le32_to_cpu(top->kb_base);
@@ -278,6 +279,7 @@ void convert_thread_options_to_net(struct thread_options_pack *top,
 	string_to_net(top->profile, o->profile);
 	string_to_net(top->cgroup, o->cgroup);
 
+	top->filetype = cpu_to_le32(o->filetype);
 	top->td_ddir = cpu_to_le32(o->td_ddir);
 	top->rw_seq = cpu_to_le32(o->rw_seq);
 	top->kb_base = cpu_to_le32(o->kb_base);
diff --git a/engines/net.c b/engines/net.c
index 8087207e505b..9ed4660d77b7 100644
--- a/engines/net.c
+++ b/engines/net.c
@@ -1194,7 +1194,7 @@ static int fio_netio_setup(struct thread_data *td)
 	struct netio_data *nd;
 
 	if (!td->files_index) {
-		add_file(td, td->o.filename ?: "net", 0, 0);
+		add_file(td, td->o.filename ?: "net", 0, 0, 0);
 		td->o.nr_files = td->o.nr_files ?: 1;
 		td->o.open_files++;
 	}
diff --git a/engines/rbd.c b/engines/rbd.c
index 6fe87b8d010c..9b28c9f099ad 100644
--- a/engines/rbd.c
+++ b/engines/rbd.c
@@ -407,7 +407,7 @@ static int fio_rbd_setup(struct thread_data *td)
 	 * The size of the RBD is set instead of a artificial file.
 	 */
 	if (!td->files_index) {
-		add_file(td, td->o.filename ? : "rbd", 0, 0);
+		add_file(td, td->o.filename ? : "rbd", 0, 0, 0);
 		td->o.nr_files = td->o.nr_files ? : 1;
 		td->o.open_files++;
 	}
diff --git a/file.h b/file.h
index add77730fdb4..fd35ef7ef07e 100644
--- a/file.h
+++ b/file.h
@@ -13,12 +13,24 @@
  * The type of object we are working on
  */
 enum fio_filetype {
-	FIO_TYPE_FILE = 1,		/* plain file */
+	FIO_TYPE_ANY = 0,
+	FIO_TYPE_FILE,			/* plain file */
 	FIO_TYPE_BD,			/* block device */
 	FIO_TYPE_CHAR,			/* character device */
 	FIO_TYPE_PIPE,			/* pipe */
+	FIO_TYPE_END,
 };
 
+static inline const char *file_type_name(int type)
+{
+	static const char *name[] = { "any", "file", "bdev", "chardev", "pipe", };
+
+	if (type < FIO_TYPE_END)
+		return name[type];
+
+	return "invalid";
+}
+
 enum fio_file_flags {
 	FIO_FILE_open		= 1 << 0,	/* file is open */
 	FIO_FILE_closing	= 1 << 1,	/* file being closed */
@@ -167,7 +179,7 @@ extern int __must_check generic_close_file(struct thread_data *, struct fio_file
 extern int __must_check generic_get_file_size(struct thread_data *, struct fio_file *);
 extern int __must_check file_lookup_open(struct fio_file *f, int flags);
 extern int __must_check pre_read_files(struct thread_data *);
-extern int add_file(struct thread_data *, const char *, int, int);
+extern int add_file(struct thread_data *, const char *, int, int, int);
 extern int add_file_exclusive(struct thread_data *, const char *);
 extern void get_file(struct fio_file *);
 extern int __must_check put_file(struct thread_data *, struct fio_file *);
diff --git a/filesetup.c b/filesetup.c
index 43146ba7671f..f621813bb138 100644
--- a/filesetup.c
+++ b/filesetup.c
@@ -1241,7 +1241,8 @@ static struct fio_file *alloc_new_file(struct thread_data *td)
 	return f;
 }
 
-int add_file(struct thread_data *td, const char *fname, int numjob, int inc)
+int add_file(struct thread_data *td, const char *fname, int numjob, int inc,
+	     int check_type)
 {
 	int cur_files = td->files_index;
 	char file_name[PATH_MAX];
@@ -1298,6 +1299,18 @@ int add_file(struct thread_data *td, const char *fname, int numjob, int inc)
 
 	get_file_type(f);
 
+	if (td->o.filetype != FIO_TYPE_ANY && f->filetype != td->o.filetype &&
+	    check_type) {
+		sfree(f->file_name);
+		td->files[cur_files] = NULL;
+		sfree(f);
+		log_err("fio: dropping file '%s' of wrong type\n", file_name);
+		log_err("fio: wanted %s, got %s\n",
+					file_type_name(td->o.filetype),
+					file_type_name(f->filetype));
+		return -1;
+	}
+
 	switch (td->o.file_lock_mode) {
 	case FILE_LOCK_NONE:
 		break;
@@ -1337,7 +1350,7 @@ int add_file_exclusive(struct thread_data *td, const char *fname)
 			return i;
 	}
 
-	return add_file(td, fname, 0, 1);
+	return add_file(td, fname, 0, 1, 0);
 }
 
 void get_file(struct fio_file *f)
@@ -1450,7 +1463,7 @@ static int recurse_dir(struct thread_data *td, const char *dirname)
 		}
 
 		if (S_ISREG(sb.st_mode)) {
-			add_file(td, full_path, 0, 1);
+			add_file(td, full_path, 0, 1, 0);
 			continue;
 		}
 		if (!S_ISDIR(sb.st_mode))
diff --git a/init.c b/init.c
index e20845163279..d96154c1ceaf 100644
--- a/init.c
+++ b/init.c
@@ -1110,10 +1110,16 @@ static int add_job(struct thread_data *td, const char *jobname, int job_add_num,
 		file_alloced = 1;
 
 		if (o->nr_files == 1 && exists_and_not_file(jobname))
-			add_file(td, jobname, job_add_num, 0);
+			add_file(td, jobname, job_add_num, 0, 0);
 		else {
-			for (i = 0; i < o->nr_files; i++)
-				add_file(td, make_filename(fname, sizeof(fname), o, jobname, job_add_num, i), job_add_num, 0);
+			for (i = 0; i < o->nr_files; i++) {
+				char *name;
+
+				name = make_filename(fname, sizeof(fname),
+							o, jobname,
+							job_add_num, i);
+				add_file(td, name, job_add_num, 0, 0);
+			}
 		}
 	}
 
diff --git a/iolog.c b/iolog.c
index 4a7d939af251..9c9ff3788c70 100644
--- a/iolog.c
+++ b/iolog.c
@@ -356,7 +356,7 @@ static int read_iolog2(struct thread_data *td, FILE *f)
 		} else if (r == 2) {
 			rw = DDIR_INVAL;
 			if (!strcmp(act, "add")) {
-				fileno = add_file(td, fname, 0, 1);
+				fileno = add_file(td, fname, 0, 1, 0);
 				file_action = FIO_LOG_ADD_FILE;
 				continue;
 			} else if (!strcmp(act, "open")) {
diff --git a/options.c b/options.c
index 918de8e8e34b..7ddbf2793a1d 100644
--- a/options.c
+++ b/options.c
@@ -850,6 +850,7 @@ static int str_filename_cb(void *data, const char *input)
 {
 	struct thread_data *td = data;
 	char *fname, *str, *p;
+	int bad, good;
 
 	p = str = strdup(input);
 
@@ -859,13 +860,22 @@ static int str_filename_cb(void *data, const char *input)
 	if (!td->files_index)
 		td->o.nr_files = 0;
 
+	good = bad = 0;
 	while ((fname = get_next_name(&str)) != NULL) {
 		if (!strlen(fname))
 			break;
-		add_file(td, fname, 0, 1);
+		if (add_file(td, fname, 0, 1, 1) != -1) {
+			good++;
+			continue;
+		}
+		bad++;
 	}
 
 	free(p);
+
+	if (bad && !good)
+		return 1;
+
 	return 0;
 }
 
@@ -1359,6 +1369,39 @@ struct fio_option fio_options[FIO_MAX_OPTS] = {
 		.group	= FIO_OPT_G_FILENAME,
 	},
 	{
+		.name	= "filetype",
+		.type	= FIO_OPT_STR,
+		.off1	= td_var_offset(filetype),
+		.help	= "File must match this filetype",
+		.def	= "any",
+		.category = FIO_OPT_C_FILE,
+		.group	= FIO_OPT_G_FILENAME,
+		.posval = {
+			  { .ival = "any",
+			    .oval = FIO_TYPE_ANY,
+			    .help = "Any filetype",
+			  },
+			  { .ival = "file",
+			    .oval = FIO_TYPE_FILE,
+			    .help = "Regular file",
+			  },
+			  { .ival = "bdev",
+			    .oval = FIO_TYPE_BD,
+			    .help = "Block device",
+			  },
+			  {
+			    .ival = "chardev",
+			    .oval = FIO_TYPE_CHAR,
+			    .help = "Character device",
+			  },
+			  {
+			    .ival = "pipe",
+			    .oval = FIO_TYPE_PIPE,
+			    .help = "Pipe",
+			  },
+		},
+	},
+	{
 		.name	= "lockfile",
 		.lname	= "Lockfile",
 		.type	= FIO_OPT_STR,
diff --git a/thread_options.h b/thread_options.h
index a45d7b79b7ef..d90d3d7de683 100644
--- a/thread_options.h
+++ b/thread_options.h
@@ -40,6 +40,7 @@ struct thread_options {
 	char *opendir;
 	char *ioengine;
 	char *mmapfile;
+	unsigned int filetype;
 	enum td_ddir td_ddir;
 	unsigned int rw_seq;
 	unsigned int kb_base;
@@ -271,6 +272,7 @@ struct thread_options_pack {
 	uint8_t opendir[FIO_TOP_STR_MAX];
 	uint8_t ioengine[FIO_TOP_STR_MAX];
 	uint8_t mmapfile[FIO_TOP_STR_MAX];
+	uint32_t filetype;
 	uint32_t td_ddir;
 	uint32_t rw_seq;
 	uint32_t kb_base;

Re: linux /dev and normal files

[Sitsofe Wheeler]

On 1 October 2014 04:43, Jens Axboe <axboe@kernel.dk> wrote:
> On 2014-09-30 19:17, Jens Axboe wrote:
>>
>> On 2014-09-30 19:12, Elliott, Robert (Server Storage) wrote:
>>>
>>>> -----Original Message-----
>>>> From: Jens Axboe [mailto:axboe@kernel.dk]
>>>>
>>>> I'd much rather keep fio ignorant of any "special" directories, even if
>>>> it means that sometimes you do potentially run into issues like the
>>>> above, where you specify a block device that does not exist.
>>>
>>> How about an option in the script:
>>> direct=2   file must exist and be a block device, otherwise skip it
>>
>> Might be a bit nasty to overload direct= like that. But I'd be open to
>> adding an option that says must_be_bdev or something, which can be a
>> bool as well.
>
> So for your case, you'd set
>
> filetype=bdev

How about a create parameter which defaults to create=1 giving the
current behaviour and create=0 where the job fails if the file doesn't
already exist? It won't catch as much but should be portable.

-- 
Sitsofe | http://sucs.org/~sits/

Re: linux /dev and normal files

[Jens Axboe]

On 2014-10-01 01:36, Sitsofe Wheeler wrote:
> On 1 October 2014 04:43, Jens Axboe <axboe@kernel.dk> wrote:
>> On 2014-09-30 19:17, Jens Axboe wrote:
>>>
>>> On 2014-09-30 19:12, Elliott, Robert (Server Storage) wrote:
>>>>
>>>>> -----Original Message-----
>>>>> From: Jens Axboe [mailto:axboe@kernel.dk]
>>>>>
>>>>> I'd much rather keep fio ignorant of any "special" directories, even if
>>>>> it means that sometimes you do potentially run into issues like the
>>>>> above, where you specify a block device that does not exist.
>>>>
>>>> How about an option in the script:
>>>> direct=2   file must exist and be a block device, otherwise skip it
>>>
>>> Might be a bit nasty to overload direct= like that. But I'd be open to
>>> adding an option that says must_be_bdev or something, which can be a
>>> bool as well.
>>
>> So for your case, you'd set
>>
>> filetype=bdev
>
> How about a create parameter which defaults to create=1 giving the
> current behaviour and create=0 where the job fails if the file doesn't
> already exist? It won't catch as much but should be portable.

Agree, that will be both easier to maintain and probably just as useful 
as the more expanded options. Lets call it 'allow_file_create'.

-- 
Jens Axboe

RE: linux /dev and normal files

[Elliott, Robert (Server Storage)]

...
> >>>>> I'd much rather keep fio ignorant of any "special" directories, even if
> >>>>> it means that sometimes you do potentially run into issues like the
> >>>>> above, where you specify a block device that does not exist.
> >>>>
> >>>> How about an option in the script:
> >>>> direct=2   file must exist and be a block device, otherwise skip it
> >>>
> >>> Might be a bit nasty to overload direct= like that. But I'd be open to
> >>> adding an option that says must_be_bdev or something, which can be a
> >>> bool as well.
> >>
> >> So for your case, you'd set
> >> filetype=bdev
> >
> > How about a create parameter which defaults to create=1 giving the
> > current behaviour and create=0 where the job fails if the file doesn't
> > already exist? It won't catch as much but should be portable.
> 
> Agree, that will be both easier to maintain and probably just as useful
> as the more expanded options. Lets call it 'allow_file_create'.
> 

Thanks, I'll create a patch to add that.

RE: linux /dev and normal files

[Elliott, Robert (Server Storage)]

> -----Original Message-----
> From: Jens Axboe [mailto:axboe@kernel.dk]
> Sent: Tuesday, 30 September, 2014 10:43 PM
> To: Elliott, Robert (Server Storage); fio@vger.kernel.org
> Cc: Sitsofe Wheeler; Jon Tango
> Subject: Re: linux /dev and normal files
> 
> On 2014-09-30 19:17, Jens Axboe wrote:
> > On 2014-09-30 19:12, Elliott, Robert (Server Storage) wrote:
> >>
> >>
> >>> -----Original Message-----
> >>> From: Jens Axboe [mailto:axboe@kernel.dk]
> >> ...
> >>> I'd much rather keep fio ignorant of any "special" directories, even if
> >>> it means that sometimes you do potentially run into issues like the
> >>> above, where you specify a block device that does not exist.
> >>
> >> How about an option in the script:
> >> direct=2   file must exist and be a block device, otherwise skip it
> >
> > Might be a bit nasty to overload direct= like that. But I'd be open to
> > adding an option that says must_be_bdev or something, which can be a
> > bool as well.
> 
> Totally untested patch attached (it compiles). There's a new option,
> filetype. If not set, if will allow any file type. Can be set to:
> 
> any        default, allow any file type
> file       regular file
> bdev       block device
> chardev    character device
> pipe       pipe
> 
> So for your case, you'd set
> 
> filetype=bdev
> 
> and you'd avoid the issue. Note that it only works for files specified
> with filename=. Any other way of adding a file (blktrace replay,
> opendir=, or fio added files when no filename= given) will ignore the
> filetype setting.

Tested with 16 devices sdr to sdag, where sdaf and sdag are
offline.

It prints a message about sdaf and exits, and does not
create a normal file in that location:
parse    11480 [drive_af]
parse    11480 Parsing section [drive_af]
file     11480 dup files: 0
parse    11480 filename=/dev/sdaf
parse    11480
parse    11480 [drive_ag]
parse    11480 handle_option=filename, ptr=/dev/sdaf
parse    11480 __handle_option=filename, type=5, ptr=/dev/sdaf
file     11480 add file /dev/sdaf
file     11480 resize file array to 1 files
fio: dropping file '/dev/sdaf' of wrong type
fio: wanted bdev, got file
fio: failed parsing filename=/dev/sdaf
fio: job drive_af dropped
parse    11480 free options
parse    11480 free options
io       11480 ioengine cpuio unregistered
io       11480 ioengine mmap unregistered
io       11480 ioengine sync unregistered
...

compared to a bdev that exists:
parse    11480 handle_option=filename, ptr=/dev/sdae
parse    11480 __handle_option=filename, type=5, ptr=/dev/sdae
file     11480 add file /dev/sdae
file     11480 resize file array to 1 files
file     11480 file 0x7f7903703a90 "/dev/sdae" added at 0
io       11480 load ioengine libaio
parse    11480 init options
drive_ae: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=96
file     11480 dup files: 1
io       11480 load ioengine libaio

That is because the do {} while (!ret) loop in
__parse_jobs_ini gives up on the first add_job failure.
add_job calls parse_options which calls handle_option
which fails.

It might be better to continue to attempt to run the
other jobs.

Re: linux /dev and normal files

[Jens Axboe]

On 2014-10-01 17:52, Elliott, Robert (Server Storage) wrote:
>
>
>> -----Original Message-----
>> From: Jens Axboe [mailto:axboe@kernel.dk]
>> Sent: Tuesday, 30 September, 2014 10:43 PM
>> To: Elliott, Robert (Server Storage); fio@vger.kernel.org
>> Cc: Sitsofe Wheeler; Jon Tango
>> Subject: Re: linux /dev and normal files
>>
>> On 2014-09-30 19:17, Jens Axboe wrote:
>>> On 2014-09-30 19:12, Elliott, Robert (Server Storage) wrote:
>>>>
>>>>
>>>>> -----Original Message-----
>>>>> From: Jens Axboe [mailto:axboe@kernel.dk]
>>>> ...
>>>>> I'd much rather keep fio ignorant of any "special" directories, even if
>>>>> it means that sometimes you do potentially run into issues like the
>>>>> above, where you specify a block device that does not exist.
>>>>
>>>> How about an option in the script:
>>>> direct=2   file must exist and be a block device, otherwise skip it
>>>
>>> Might be a bit nasty to overload direct= like that. But I'd be open to
>>> adding an option that says must_be_bdev or something, which can be a
>>> bool as well.
>>
>> Totally untested patch attached (it compiles). There's a new option,
>> filetype. If not set, if will allow any file type. Can be set to:
>>
>> any        default, allow any file type
>> file       regular file
>> bdev       block device
>> chardev    character device
>> pipe       pipe
>>
>> So for your case, you'd set
>>
>> filetype=bdev
>>
>> and you'd avoid the issue. Note that it only works for files specified
>> with filename=. Any other way of adding a file (blktrace replay,
>> opendir=, or fio added files when no filename= given) will ignore the
>> filetype setting.
>
> Tested with 16 devices sdr to sdag, where sdaf and sdag are
> offline.
>
> It prints a message about sdaf and exits, and does not
> create a normal file in that location:
> parse    11480 [drive_af]
> parse    11480 Parsing section [drive_af]
> file     11480 dup files: 0
> parse    11480 filename=/dev/sdaf
> parse    11480
> parse    11480 [drive_ag]
> parse    11480 handle_option=filename, ptr=/dev/sdaf
> parse    11480 __handle_option=filename, type=5, ptr=/dev/sdaf
> file     11480 add file /dev/sdaf
> file     11480 resize file array to 1 files
> fio: dropping file '/dev/sdaf' of wrong type
> fio: wanted bdev, got file
> fio: failed parsing filename=/dev/sdaf
> fio: job drive_af dropped
> parse    11480 free options
> parse    11480 free options
> io       11480 ioengine cpuio unregistered
> io       11480 ioengine mmap unregistered
> io       11480 ioengine sync unregistered
> ...
>
> compared to a bdev that exists:
> parse    11480 handle_option=filename, ptr=/dev/sdae
> parse    11480 __handle_option=filename, type=5, ptr=/dev/sdae
> file     11480 add file /dev/sdae
> file     11480 resize file array to 1 files
> file     11480 file 0x7f7903703a90 "/dev/sdae" added at 0
> io       11480 load ioengine libaio
> parse    11480 init options
> drive_ae: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=96
> file     11480 dup files: 1
> io       11480 load ioengine libaio
>
> That is because the do {} while (!ret) loop in
> __parse_jobs_ini gives up on the first add_job failure.
> add_job calls parse_options which calls handle_option
> which fails.
>
> It might be better to continue to attempt to run the
> other jobs.

We should just go for the simpler create/nocreate solution, I prefer 
that. This is too involved, and it's hard to get all the cases right 
since there are a bunch of different ways that files are added. Which 
was why this patch only focused on filename=, since that is the one that 
is the most interesting. But it still isn't pretty. If we add the 
allow_file_create option, it's basically a two-liner change in a few 
places to not use O_CREAT.

-- 
Jens Axboe

Re: linux /dev and normal files

[Jens Axboe]

[-- Attachment #1: Type: text/plain, Size: 4022 bytes --]

On 2014-10-01 18:58, Jens Axboe wrote:
> On 2014-10-01 17:52, Elliott, Robert (Server Storage) wrote:
>>
>>
>>> -----Original Message-----
>>> From: Jens Axboe [mailto:axboe@kernel.dk]
>>> Sent: Tuesday, 30 September, 2014 10:43 PM
>>> To: Elliott, Robert (Server Storage); fio@vger.kernel.org
>>> Cc: Sitsofe Wheeler; Jon Tango
>>> Subject: Re: linux /dev and normal files
>>>
>>> On 2014-09-30 19:17, Jens Axboe wrote:
>>>> On 2014-09-30 19:12, Elliott, Robert (Server Storage) wrote:
>>>>>
>>>>>
>>>>>> -----Original Message-----
>>>>>> From: Jens Axboe [mailto:axboe@kernel.dk]
>>>>> ...
>>>>>> I'd much rather keep fio ignorant of any "special" directories,
>>>>>> even if
>>>>>> it means that sometimes you do potentially run into issues like the
>>>>>> above, where you specify a block device that does not exist.
>>>>>
>>>>> How about an option in the script:
>>>>> direct=2   file must exist and be a block device, otherwise skip it
>>>>
>>>> Might be a bit nasty to overload direct= like that. But I'd be open to
>>>> adding an option that says must_be_bdev or something, which can be a
>>>> bool as well.
>>>
>>> Totally untested patch attached (it compiles). There's a new option,
>>> filetype. If not set, if will allow any file type. Can be set to:
>>>
>>> any        default, allow any file type
>>> file       regular file
>>> bdev       block device
>>> chardev    character device
>>> pipe       pipe
>>>
>>> So for your case, you'd set
>>>
>>> filetype=bdev
>>>
>>> and you'd avoid the issue. Note that it only works for files specified
>>> with filename=. Any other way of adding a file (blktrace replay,
>>> opendir=, or fio added files when no filename= given) will ignore the
>>> filetype setting.
>>
>> Tested with 16 devices sdr to sdag, where sdaf and sdag are
>> offline.
>>
>> It prints a message about sdaf and exits, and does not
>> create a normal file in that location:
>> parse    11480 [drive_af]
>> parse    11480 Parsing section [drive_af]
>> file     11480 dup files: 0
>> parse    11480 filename=/dev/sdaf
>> parse    11480
>> parse    11480 [drive_ag]
>> parse    11480 handle_option=filename, ptr=/dev/sdaf
>> parse    11480 __handle_option=filename, type=5, ptr=/dev/sdaf
>> file     11480 add file /dev/sdaf
>> file     11480 resize file array to 1 files
>> fio: dropping file '/dev/sdaf' of wrong type
>> fio: wanted bdev, got file
>> fio: failed parsing filename=/dev/sdaf
>> fio: job drive_af dropped
>> parse    11480 free options
>> parse    11480 free options
>> io       11480 ioengine cpuio unregistered
>> io       11480 ioengine mmap unregistered
>> io       11480 ioengine sync unregistered
>> ...
>>
>> compared to a bdev that exists:
>> parse    11480 handle_option=filename, ptr=/dev/sdae
>> parse    11480 __handle_option=filename, type=5, ptr=/dev/sdae
>> file     11480 add file /dev/sdae
>> file     11480 resize file array to 1 files
>> file     11480 file 0x7f7903703a90 "/dev/sdae" added at 0
>> io       11480 load ioengine libaio
>> parse    11480 init options
>> drive_ae: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio,
>> iodepth=96
>> file     11480 dup files: 1
>> io       11480 load ioengine libaio
>>
>> That is because the do {} while (!ret) loop in
>> __parse_jobs_ini gives up on the first add_job failure.
>> add_job calls parse_options which calls handle_option
>> which fails.
>>
>> It might be better to continue to attempt to run the
>> other jobs.
>
> We should just go for the simpler create/nocreate solution, I prefer
> that. This is too involved, and it's hard to get all the cases right
> since there are a bunch of different ways that files are added. Which
> was why this patch only focused on filename=, since that is the one that
> is the most interesting. But it still isn't pretty. If we add the
> allow_file_create option, it's basically a two-liner change in a few
> places to not use O_CREAT.

That took roughly 4 minutes... Does this work? Set allow_file_create=0 
to disallow creating new files.

-- 
Jens Axboe


[-- Attachment #2: allow-create.patch --]
[-- Type: text/x-patch, Size: 2747 bytes --]

diff --git a/cconv.c b/cconv.c
index 4a40ed0d647b..913313b68d47 100644
--- a/cconv.c
+++ b/cconv.c
@@ -69,6 +69,7 @@ void convert_thread_options_to_cpu(struct thread_options *o,
 	string_to_cpu(&o->profile, top->profile);
 	string_to_cpu(&o->cgroup, top->cgroup);
 
+	o->allow_create = le32_to_cpu(top->allow_create);
 	o->td_ddir = le32_to_cpu(top->td_ddir);
 	o->rw_seq = le32_to_cpu(top->rw_seq);
 	o->kb_base = le32_to_cpu(top->kb_base);
@@ -278,6 +279,7 @@ void convert_thread_options_to_net(struct thread_options_pack *top,
 	string_to_net(top->profile, o->profile);
 	string_to_net(top->cgroup, o->cgroup);
 
+	top->allow_create = cpu_to_le32(o->allow_create);
 	top->td_ddir = cpu_to_le32(o->td_ddir);
 	top->rw_seq = cpu_to_le32(o->rw_seq);
 	top->kb_base = cpu_to_le32(o->kb_base);
diff --git a/filesetup.c b/filesetup.c
index 43146ba7671f..5b0c82208199 100644
--- a/filesetup.c
+++ b/filesetup.c
@@ -65,7 +65,9 @@ static int extend_file(struct thread_data *td, struct fio_file *f)
 		}
 	}
 
-	flags = O_WRONLY | O_CREAT;
+	flags = O_WRONLY;
+	if (td->o.allow_create)
+		flags |= O_CREAT;
 	if (new_layout)
 		flags |= O_TRUNC;
 
@@ -557,7 +559,7 @@ int generic_open_file(struct thread_data *td, struct fio_file *f)
 	}
 	if (td->o.sync_io)
 		flags |= O_SYNC;
-	if (td->o.create_on_open)
+	if (td->o.create_on_open && td->o.allow_create)
 		flags |= O_CREAT;
 skip_flags:
 	if (f->filetype != FIO_TYPE_FILE)
@@ -568,7 +570,7 @@ open_again:
 		if (!read_only)
 			flags |= O_RDWR;
 
-		if (f->filetype == FIO_TYPE_FILE)
+		if (f->filetype == FIO_TYPE_FILE && td->o.allow_create)
 			flags |= O_CREAT;
 
 		if (is_std)
diff --git a/options.c b/options.c
index 918de8e8e34b..b6bf47264880 100644
--- a/options.c
+++ b/options.c
@@ -1359,6 +1359,15 @@ struct fio_option fio_options[FIO_MAX_OPTS] = {
 		.group	= FIO_OPT_G_FILENAME,
 	},
 	{
+		.name	= "allow_file_create",
+		.type	= FIO_OPT_BOOL,
+		.off1	= td_var_offset(allow_create),
+		.help	= "Permit fio to create files, if they don't exist",
+		.def	= "1",
+		.category = FIO_OPT_C_FILE,
+		.group	= FIO_OPT_G_FILENAME,
+	},
+	{
 		.name	= "lockfile",
 		.lname	= "Lockfile",
 		.type	= FIO_OPT_STR,
diff --git a/thread_options.h b/thread_options.h
index a45d7b79b7ef..dd576643b31c 100644
--- a/thread_options.h
+++ b/thread_options.h
@@ -41,6 +41,7 @@ struct thread_options {
 	char *ioengine;
 	char *mmapfile;
 	enum td_ddir td_ddir;
+	unsigned int allow_create;
 	unsigned int rw_seq;
 	unsigned int kb_base;
 	unsigned int unit_base;
@@ -271,6 +272,7 @@ struct thread_options_pack {
 	uint8_t opendir[FIO_TOP_STR_MAX];
 	uint8_t ioengine[FIO_TOP_STR_MAX];
 	uint8_t mmapfile[FIO_TOP_STR_MAX];
+	uint32_t allow_create;
 	uint32_t td_ddir;
 	uint32_t rw_seq;
 	uint32_t kb_base;

RE: linux /dev and normal files

[Elliott, Robert (Server Storage)]

> >> It might be better to continue to attempt to run the
> >> other jobs.
> >
> > We should just go for the simpler create/nocreate solution, I prefer
> > that. This is too involved, and it's hard to get all the cases right
> > since there are a bunch of different ways that files are added. Which
> > was why this patch only focused on filename=, since that is the one that
> > is the most interesting. But it still isn't pretty. If we add the
> > allow_file_create option, it's basically a two-liner change in a few
> > places to not use O_CREAT.
> 
> That took roughly 4 minutes... Does this work? Set allow_file_create=0
> to disallow creating new files.
> 

Mostly.  

With no size= setting or size=xx%, the error messages it prints are 
about the lack of size, not the fact that file creation is disallowed.
It does proceed to run the jobs that have valid filenames.

drive_af: you need to specify size=
fio: pid=0, err=22/file:filesetup.c:822, func=total_file_size, error=Invalid argument
drive_af: you need to specify size=
fio: pid=0, err=22/file:filesetup.c:822, func=total_file_size, error=Invalid argument
...
drive_ag: you need to specify size=
fio: pid=0, err=22/file:filesetup.c:822, func=total_file_size, error=Invalid argument
Jobs: 168 (f=168): [r(168),X(24)] [0.0% done] [2449MB/0KB/0KB /s] [627K/0/0 iops] [eta 02d:11h:58m:33s]


With size=xxm (a fixed size), it prints complaints about O_DIRECT.
It does proceed to run the jobs that have valid filenames.

fio: looks like your file system does not support direct=1/buffered=0
fio: destination does not support O_DIRECT
fio: pid=21834, err=22/file:filesetup.c:613, func=open(/dev/sdaf), error=Invalid argument
fio: looks like your file system does not support direct=1/buffered=0
fio: destination does not support O_DIRECT
fio: pid=21846, err=22/file:filesetup.c:613, func=open(/dev/sdag), error=Invalid argument
fio: looks like your file system does not support direct=1/buffered=0
fio: destination does not support O_DIRECT
...
fio: pid=21856, err=22/file:filesetup.c:613, func=open(/dev/sdag), error=Invalid argument
Jobs: 168 (f=168): [r(168),X(24)] [0.0% done] [2449MB/0KB/0KB /s] [627K/0/0 iops] [eta 02d:11h:58m:50s]




---
Rob Elliott    HP Server Storage

Re: linux /dev and normal files

[Jens Axboe]

On 10/02/2014 12:21 PM, Elliott, Robert (Server Storage) wrote:
>>>> It might be better to continue to attempt to run the
>>>> other jobs.
>>>
>>> We should just go for the simpler create/nocreate solution, I prefer
>>> that. This is too involved, and it's hard to get all the cases right
>>> since there are a bunch of different ways that files are added. Which
>>> was why this patch only focused on filename=, since that is the one that
>>> is the most interesting. But it still isn't pretty. If we add the
>>> allow_file_create option, it's basically a two-liner change in a few
>>> places to not use O_CREAT.
>>
>> That took roughly 4 minutes... Does this work? Set allow_file_create=0
>> to disallow creating new files.
>>
> 
> Mostly.  
> 
> With no size= setting or size=xx%, the error messages it prints are 
> about the lack of size, not the fact that file creation is disallowed.
> It does proceed to run the jobs that have valid filenames.
> 
> drive_af: you need to specify size=
> fio: pid=0, err=22/file:filesetup.c:822, func=total_file_size, error=Invalid argument
> drive_af: you need to specify size=
> fio: pid=0, err=22/file:filesetup.c:822, func=total_file_size, error=Invalid argument
> ...
> drive_ag: you need to specify size=
> fio: pid=0, err=22/file:filesetup.c:822, func=total_file_size, error=Invalid argument
> Jobs: 168 (f=168): [r(168),X(24)] [0.0% done] [2449MB/0KB/0KB /s] [627K/0/0 iops] [eta 02d:11h:58m:33s]

Fio always does this, if the file does not exist and no size is given.
The reason is that it doesn't know what size to create. But we can
augment that a little bit if allow_file_create == 0, since the size
setting would not change that.

> With size=xxm (a fixed size), it prints complaints about O_DIRECT.
> It does proceed to run the jobs that have valid filenames.
> 
> fio: looks like your file system does not support direct=1/buffered=0
> fio: destination does not support O_DIRECT
> fio: pid=21834, err=22/file:filesetup.c:613, func=open(/dev/sdaf), error=Invalid argument
> fio: looks like your file system does not support direct=1/buffered=0
> fio: destination does not support O_DIRECT
> fio: pid=21846, err=22/file:filesetup.c:613, func=open(/dev/sdag), error=Invalid argument
> fio: looks like your file system does not support direct=1/buffered=0
> fio: destination does not support O_DIRECT
> ...
> fio: pid=21856, err=22/file:filesetup.c:613, func=open(/dev/sdag), error=Invalid argument
> Jobs: 168 (f=168): [r(168),X(24)] [0.0% done] [2449MB/0KB/0KB /s] [627K/0/0 iops] [eta 02d:11h:58m:50s]

OK, let me test that, will spin a new one.

-- 
Jens Axboe

Re: linux /dev and normal files

[Jens Axboe]

[-- Attachment #1: Type: text/plain, Size: 2823 bytes --]

On 10/02/2014 12:32 PM, Jens Axboe wrote:
> On 10/02/2014 12:21 PM, Elliott, Robert (Server Storage) wrote:
>>>>> It might be better to continue to attempt to run the
>>>>> other jobs.
>>>>
>>>> We should just go for the simpler create/nocreate solution, I prefer
>>>> that. This is too involved, and it's hard to get all the cases right
>>>> since there are a bunch of different ways that files are added. Which
>>>> was why this patch only focused on filename=, since that is the one that
>>>> is the most interesting. But it still isn't pretty. If we add the
>>>> allow_file_create option, it's basically a two-liner change in a few
>>>> places to not use O_CREAT.
>>>
>>> That took roughly 4 minutes... Does this work? Set allow_file_create=0
>>> to disallow creating new files.
>>>
>>
>> Mostly.  
>>
>> With no size= setting or size=xx%, the error messages it prints are 
>> about the lack of size, not the fact that file creation is disallowed.
>> It does proceed to run the jobs that have valid filenames.
>>
>> drive_af: you need to specify size=
>> fio: pid=0, err=22/file:filesetup.c:822, func=total_file_size, error=Invalid argument
>> drive_af: you need to specify size=
>> fio: pid=0, err=22/file:filesetup.c:822, func=total_file_size, error=Invalid argument
>> ...
>> drive_ag: you need to specify size=
>> fio: pid=0, err=22/file:filesetup.c:822, func=total_file_size, error=Invalid argument
>> Jobs: 168 (f=168): [r(168),X(24)] [0.0% done] [2449MB/0KB/0KB /s] [627K/0/0 iops] [eta 02d:11h:58m:33s]
> 
> Fio always does this, if the file does not exist and no size is given.
> The reason is that it doesn't know what size to create. But we can
> augment that a little bit if allow_file_create == 0, since the size
> setting would not change that.
> 
>> With size=xxm (a fixed size), it prints complaints about O_DIRECT.
>> It does proceed to run the jobs that have valid filenames.
>>
>> fio: looks like your file system does not support direct=1/buffered=0
>> fio: destination does not support O_DIRECT
>> fio: pid=21834, err=22/file:filesetup.c:613, func=open(/dev/sdaf), error=Invalid argument
>> fio: looks like your file system does not support direct=1/buffered=0
>> fio: destination does not support O_DIRECT
>> fio: pid=21846, err=22/file:filesetup.c:613, func=open(/dev/sdag), error=Invalid argument
>> fio: looks like your file system does not support direct=1/buffered=0
>> fio: destination does not support O_DIRECT
>> ...
>> fio: pid=21856, err=22/file:filesetup.c:613, func=open(/dev/sdag), error=Invalid argument
>> Jobs: 168 (f=168): [r(168),X(24)] [0.0% done] [2449MB/0KB/0KB /s] [627K/0/0 iops] [eta 02d:11h:58m:50s]
> 
> OK, let me test that, will spin a new one.

This works for me - though I cannot reproduce your O_DIRECT warning
above, what was run to generate that?

-- 
Jens Axboe


[-- Attachment #2: create-v2.patch --]
[-- Type: text/x-patch, Size: 4863 bytes --]

diff --git a/HOWTO b/HOWTO
index 693eeb11b12a..854843df5ea9 100644
--- a/HOWTO
+++ b/HOWTO
@@ -1160,6 +1160,11 @@ create_only=bool	If true, fio will only run the setup phase of the job.
 			that will be done. The actual job contents are not
 			executed.
 
+allow_file_create=bool	If true, fio is permitted to create files as part
+		of its workload. This is the default behavior. If this
+		option is false, then fio will error out if the files it
+		needs to use don't already exist.
+
 pre_read=bool	If this is given, files will be pre-read into memory before
 		starting the given IO operation. This will also clear
 		the 'invalidate' flag, since it is pointless to pre-read
diff --git a/cconv.c b/cconv.c
index 4a40ed0d647b..913313b68d47 100644
--- a/cconv.c
+++ b/cconv.c
@@ -69,6 +69,7 @@ void convert_thread_options_to_cpu(struct thread_options *o,
 	string_to_cpu(&o->profile, top->profile);
 	string_to_cpu(&o->cgroup, top->cgroup);
 
+	o->allow_create = le32_to_cpu(top->allow_create);
 	o->td_ddir = le32_to_cpu(top->td_ddir);
 	o->rw_seq = le32_to_cpu(top->rw_seq);
 	o->kb_base = le32_to_cpu(top->kb_base);
@@ -278,6 +279,7 @@ void convert_thread_options_to_net(struct thread_options_pack *top,
 	string_to_net(top->profile, o->profile);
 	string_to_net(top->cgroup, o->cgroup);
 
+	top->allow_create = cpu_to_le32(o->allow_create);
 	top->td_ddir = cpu_to_le32(o->td_ddir);
 	top->rw_seq = cpu_to_le32(o->rw_seq);
 	top->kb_base = cpu_to_le32(o->kb_base);
diff --git a/filesetup.c b/filesetup.c
index 43146ba7671f..cca877634e95 100644
--- a/filesetup.c
+++ b/filesetup.c
@@ -65,7 +65,9 @@ static int extend_file(struct thread_data *td, struct fio_file *f)
 		}
 	}
 
-	flags = O_WRONLY | O_CREAT;
+	flags = O_WRONLY;
+	if (td->o.allow_create)
+		flags |= O_CREAT;
 	if (new_layout)
 		flags |= O_TRUNC;
 
@@ -76,7 +78,13 @@ static int extend_file(struct thread_data *td, struct fio_file *f)
 	dprint(FD_FILE, "open file %s, flags %x\n", f->file_name, flags);
 	f->fd = open(f->file_name, flags, 0644);
 	if (f->fd < 0) {
-		td_verror(td, errno, "open");
+		int err = errno;
+
+		if (err == ENOENT && !td->o.allow_create)
+			log_err("fio: file creation disallowed by "
+					"allow_file_create=0\n");
+		else
+			td_verror(td, err, "open");
 		return 1;
 	}
 
@@ -557,7 +565,7 @@ int generic_open_file(struct thread_data *td, struct fio_file *f)
 	}
 	if (td->o.sync_io)
 		flags |= O_SYNC;
-	if (td->o.create_on_open)
+	if (td->o.create_on_open && td->o.allow_create)
 		flags |= O_CREAT;
 skip_flags:
 	if (f->filetype != FIO_TYPE_FILE)
@@ -568,7 +576,7 @@ open_again:
 		if (!read_only)
 			flags |= O_RDWR;
 
-		if (f->filetype == FIO_TYPE_FILE)
+		if (f->filetype == FIO_TYPE_FILE && td->o.allow_create)
 			flags |= O_CREAT;
 
 		if (is_std)
diff --git a/fio.1 b/fio.1
index d64fbb7ab5e0..ef53bd6302a7 100644
--- a/fio.1
+++ b/fio.1
@@ -1029,6 +1029,11 @@ If true, fio will only run the setup phase of the job. If files need to be
 laid out or updated on disk, only that will be done. The actual job contents
 are not executed.
 .TP
+.BI allow_file_create \fR=\fPbool
+If true, fio is permitted to create files as part of its workload. This is
+the default behavior. If this option is false, then fio will error out if the
+files it needs to use don't already exist.
+.TP
 .BI pre_read \fR=\fPbool
 If this is given, files will be pre-read into memory before starting the given
 IO operation. This will also clear the \fR \fBinvalidate\fR flag, since it is
diff --git a/options.c b/options.c
index 918de8e8e34b..a7dd58adb418 100644
--- a/options.c
+++ b/options.c
@@ -2962,6 +2962,15 @@ struct fio_option fio_options[FIO_MAX_OPTS] = {
 		.def	= "0",
 	},
 	{
+		.name	= "allow_file_create",
+		.type	= FIO_OPT_BOOL,
+		.off1	= td_var_offset(allow_create),
+		.help	= "Permit fio to create files, if they don't exist",
+		.def	= "1",
+		.category = FIO_OPT_C_FILE,
+		.group	= FIO_OPT_G_FILENAME,
+	},
+	{
 		.name	= "pre_read",
 		.lname	= "Pre-read files",
 		.type	= FIO_OPT_BOOL,
diff --git a/server.h b/server.h
index 1b131b92f08a..5213e6ac9e08 100644
--- a/server.h
+++ b/server.h
@@ -38,7 +38,7 @@ struct fio_net_cmd_reply {
 };
 
 enum {
-	FIO_SERVER_VER			= 36,
+	FIO_SERVER_VER			= 37,
 
 	FIO_SERVER_MAX_FRAGMENT_PDU	= 1024,
 	FIO_SERVER_MAX_CMD_MB		= 2048,
diff --git a/thread_options.h b/thread_options.h
index a45d7b79b7ef..5cf697d2bb9c 100644
--- a/thread_options.h
+++ b/thread_options.h
@@ -40,6 +40,7 @@ struct thread_options {
 	char *opendir;
 	char *ioengine;
 	char *mmapfile;
+	unsigned int allow_create;
 	enum td_ddir td_ddir;
 	unsigned int rw_seq;
 	unsigned int kb_base;
@@ -271,6 +272,7 @@ struct thread_options_pack {
 	uint8_t opendir[FIO_TOP_STR_MAX];
 	uint8_t ioengine[FIO_TOP_STR_MAX];
 	uint8_t mmapfile[FIO_TOP_STR_MAX];
+	uint32_t allow_create;
 	uint32_t td_ddir;
 	uint32_t rw_seq;
 	uint32_t kb_base;

RE: linux /dev and normal files

[Elliott, Robert (Server Storage)]

> -----Original Message-----
> From: fio-owner@vger.kernel.org [mailto:fio-owner@vger.kernel.org] On
> Behalf Of Jens Axboe
> Sent: Thursday, October 2, 2014 1:32 PM
> To: Elliott, Robert (Server Storage); fio@vger.kernel.org
> Cc: Sitsofe Wheeler; Jon Tango
> Subject: Re: linux /dev and normal files
> 
> On 10/02/2014 12:21 PM, Elliott, Robert (Server Storage) wrote:
> >>>> It might be better to continue to attempt to run the
> >>>> other jobs.
> >>>
> >>> We should just go for the simpler create/nocreate solution, I prefer
> >>> that. This is too involved, and it's hard to get all the cases right
> >>> since there are a bunch of different ways that files are added. Which
> >>> was why this patch only focused on filename=, since that is the one
> that
> >>> is the most interesting. But it still isn't pretty. If we add the
> >>> allow_file_create option, it's basically a two-liner change in a few
> >>> places to not use O_CREAT.
> >>
> >> That took roughly 4 minutes... Does this work? Set allow_file_create=0
> >> to disallow creating new files.
...
> 
> OK, let me test that, will spin a new one.

I don't think this ever got added.

Re: linux /dev and normal files

[Jens Axboe]

On 04/03/2015 12:13 PM, Elliott, Robert (Server Storage) wrote:
> 
> 
>> -----Original Message-----
>> From: fio-owner@vger.kernel.org [mailto:fio-owner@vger.kernel.org] On
>> Behalf Of Jens Axboe
>> Sent: Thursday, October 2, 2014 1:32 PM
>> To: Elliott, Robert (Server Storage); fio@vger.kernel.org
>> Cc: Sitsofe Wheeler; Jon Tango
>> Subject: Re: linux /dev and normal files
>>
>> On 10/02/2014 12:21 PM, Elliott, Robert (Server Storage) wrote:
>>>>>> It might be better to continue to attempt to run the
>>>>>> other jobs.
>>>>>
>>>>> We should just go for the simpler create/nocreate solution, I prefer
>>>>> that. This is too involved, and it's hard to get all the cases right
>>>>> since there are a bunch of different ways that files are added. Which
>>>>> was why this patch only focused on filename=, since that is the one
>> that
>>>>> is the most interesting. But it still isn't pretty. If we add the
>>>>> allow_file_create option, it's basically a two-liner change in a few
>>>>> places to not use O_CREAT.
>>>>
>>>> That took roughly 4 minutes... Does this work? Set allow_file_create=0
>>>> to disallow creating new files.
> ...
>>
>> OK, let me test that, will spin a new one.
> 
> I don't think this ever got added.

It didn't... Did you re-test? No reason we can't get it added, I'll look
into it start next week.


-- 
Jens Axboe

Re: linux /dev and normal files

[Jens Axboe]

On 04/03/2015 02:24 PM, Jens Axboe wrote:
> On 04/03/2015 12:13 PM, Elliott, Robert (Server Storage) wrote:
>>
>>
>>> -----Original Message-----
>>> From: fio-owner@vger.kernel.org [mailto:fio-owner@vger.kernel.org] On
>>> Behalf Of Jens Axboe
>>> Sent: Thursday, October 2, 2014 1:32 PM
>>> To: Elliott, Robert (Server Storage); fio@vger.kernel.org
>>> Cc: Sitsofe Wheeler; Jon Tango
>>> Subject: Re: linux /dev and normal files
>>>
>>> On 10/02/2014 12:21 PM, Elliott, Robert (Server Storage) wrote:
>>>>>>> It might be better to continue to attempt to run the
>>>>>>> other jobs.
>>>>>>
>>>>>> We should just go for the simpler create/nocreate solution, I prefer
>>>>>> that. This is too involved, and it's hard to get all the cases right
>>>>>> since there are a bunch of different ways that files are added. Which
>>>>>> was why this patch only focused on filename=, since that is the one
>>> that
>>>>>> is the most interesting. But it still isn't pretty. If we add the
>>>>>> allow_file_create option, it's basically a two-liner change in a few
>>>>>> places to not use O_CREAT.
>>>>>
>>>>> That took roughly 4 minutes... Does this work? Set allow_file_create=0
>>>>> to disallow creating new files.
>> ...
>>>
>>> OK, let me test that, will spin a new one.
>>
>> I don't think this ever got added.
>
> It didn't... Did you re-test? No reason we can't get it added, I'll look
> into it start next week.

Robert, I added this. Can you check current -git and see if it behaves 
as expected for you?

-- 
Jens Axboe

RE: linux /dev and normal files

[Elliott, Robert (Server Storage)]

> -----Original Message-----
> From: Jens Axboe [mailto:axboe@kernel.dk]
> Sent: Tuesday, May 12, 2015 10:33 AM
> To: Elliott, Robert (Server Storage); fio@vger.kernel.org
> Cc: Sitsofe Wheeler; Jon Tango
> Subject: Re: linux /dev and normal files
> 
> On 04/03/2015 02:24 PM, Jens Axboe wrote:
> > On 04/03/2015 12:13 PM, Elliott, Robert (Server Storage) wrote:
> >>
> >>
> >>> -----Original Message-----
> >>> From: fio-owner@vger.kernel.org [mailto:fio-owner@vger.kernel.org] On
> >>> Behalf Of Jens Axboe
> >>> Sent: Thursday, October 2, 2014 1:32 PM
> >>> To: Elliott, Robert (Server Storage); fio@vger.kernel.org
> >>> Cc: Sitsofe Wheeler; Jon Tango
> >>> Subject: Re: linux /dev and normal files
> >>>
> >>> On 10/02/2014 12:21 PM, Elliott, Robert (Server Storage) wrote:
> >>>>>>> It might be better to continue to attempt to run the
> >>>>>>> other jobs.
> >>>>>>
> >>>>>> We should just go for the simpler create/nocreate solution, I
> prefer
> >>>>>> that. This is too involved, and it's hard to get all the cases
> right
> >>>>>> since there are a bunch of different ways that files are added.
> Which
> >>>>>> was why this patch only focused on filename=, since that is the one
> >>> that
> >>>>>> is the most interesting. But it still isn't pretty. If we add the
> >>>>>> allow_file_create option, it's basically a two-liner change in a
> few
> >>>>>> places to not use O_CREAT.
> >>>>>
> >>>>> That took roughly 4 minutes... Does this work? Set
> allow_file_create=0
> >>>>> to disallow creating new files.
> >> ...
> >>>
> >>> OK, let me test that, will spin a new one.
> >>
> >> I don't think this ever got added.
> >
> > It didn't... Did you re-test? No reason we can't get it added, I'll look
> > into it start next week.
> 
> Robert, I added this. Can you check current -git and see if it behaves
> as expected for you?
> 

Looks good.  Adding
allow_file_create=0

prevents it from creating files in /dev when the specified path
does not exist and size= is specified.

Starting 80 threads
drive_0: Laying out IO file(s) (1 file(s) / 10MB)
fio: file creation disallowed by allow_file_create=0
drive_0: Laying out IO file(s) (1 file(s) / 10MB)
fio: file creation disallowed by allow_file_create=0
drive_0: Laying out IO file(s) (1 file(s) / 10MB)
fio: file creation disallowed by allow_file_create=0
...

---
Robert Elliott, HP Server Storage

Re: linux /dev and normal files

[Jens Axboe]

On 05/12/2015 12:34 PM, Elliott, Robert (Server Storage) wrote:
>
>> -----Original Message-----
>> From: Jens Axboe [mailto:axboe@kernel.dk]
>> Sent: Tuesday, May 12, 2015 10:33 AM
>> To: Elliott, Robert (Server Storage); fio@vger.kernel.org
>> Cc: Sitsofe Wheeler; Jon Tango
>> Subject: Re: linux /dev and normal files
>>
>> On 04/03/2015 02:24 PM, Jens Axboe wrote:
>>> On 04/03/2015 12:13 PM, Elliott, Robert (Server Storage) wrote:
>>>>
>>>>
>>>>> -----Original Message-----
>>>>> From: fio-owner@vger.kernel.org [mailto:fio-owner@vger.kernel.org] On
>>>>> Behalf Of Jens Axboe
>>>>> Sent: Thursday, October 2, 2014 1:32 PM
>>>>> To: Elliott, Robert (Server Storage); fio@vger.kernel.org
>>>>> Cc: Sitsofe Wheeler; Jon Tango
>>>>> Subject: Re: linux /dev and normal files
>>>>>
>>>>> On 10/02/2014 12:21 PM, Elliott, Robert (Server Storage) wrote:
>>>>>>>>> It might be better to continue to attempt to run the
>>>>>>>>> other jobs.
>>>>>>>>
>>>>>>>> We should just go for the simpler create/nocreate solution, I
>> prefer
>>>>>>>> that. This is too involved, and it's hard to get all the cases
>> right
>>>>>>>> since there are a bunch of different ways that files are added.
>> Which
>>>>>>>> was why this patch only focused on filename=, since that is the one
>>>>> that
>>>>>>>> is the most interesting. But it still isn't pretty. If we add the
>>>>>>>> allow_file_create option, it's basically a two-liner change in a
>> few
>>>>>>>> places to not use O_CREAT.
>>>>>>>
>>>>>>> That took roughly 4 minutes... Does this work? Set
>> allow_file_create=0
>>>>>>> to disallow creating new files.
>>>> ...
>>>>>
>>>>> OK, let me test that, will spin a new one.
>>>>
>>>> I don't think this ever got added.
>>>
>>> It didn't... Did you re-test? No reason we can't get it added, I'll look
>>> into it start next week.
>>
>> Robert, I added this. Can you check current -git and see if it behaves
>> as expected for you?
>>
>
> Looks good.  Adding
> allow_file_create=0
>
> prevents it from creating files in /dev when the specified path
> does not exist and size= is specified.
>
> Starting 80 threads
> drive_0: Laying out IO file(s) (1 file(s) / 10MB)
> fio: file creation disallowed by allow_file_create=0
> drive_0: Laying out IO file(s) (1 file(s) / 10MB)
> fio: file creation disallowed by allow_file_create=0
> drive_0: Laying out IO file(s) (1 file(s) / 10MB)
> fio: file creation disallowed by allow_file_create=0

Perfect, thanks for (re) testing.

-- 
Jens Axboe