fio headers as build dependency for external-ioengines

fio headers as build dependency for external-ioengines

[Daniel Gollub]

Hi,

I am working right now in preparing for the Ceph project an external
fio ioengine, which depends on Ceph-internal libraries. So I thought
about building this ioengine as an external ioengine. (Thats why I
send couple of weeks/month ago this C++-compiler-issues-fixes for
fio.h)

Since fio is not (yet) distributed with a -dev/-devel package in
linux distribution and was very likely not yet consider doing so
I wanted to start the discussion here if this in interested of the
project "exporting"/"installing" the required fio headers to build
external ioengines. So fio can be later a "packaged" build-dependency.

For time-being I thought about solving this like this:

One can introduce in the external project's build-env a parameter
like: --with-fio=~/projects/fio/ and the resulting build-env will
pass this path as include directory as compiler flags for the
external ioengine build.

Unfortunately there are some issues with double-declaration and
one ambiguous header file name (e.g. time.h - see provided patch)

Double-declaration issues of defines/macros/... I hit while building
the Ceph filestore external ioengine:

 * ARRAY_SIZE
 * CONFIG_CPU_COUNT
 * le16_to_cpu
 * le32_to_cpu
 * le64_to_cpu


If you want to reproduce the described issue with ceph,
checkout following revision:
https://github.com/gollub/ceph/commit/079c08c67f811c0173e70201dc91168b60d2e86b
( Working branch: https://github.com/gollub/ceph/tree/fio_filestore_v2 )

And run:
./configure --with-fio-dir=/path/to/fio/
cd src
make libfio_ceph_filestore.la


This could be solved/workaround on the external ioengine side, as well
as in the fio header. One possibility on the fio side could be to relax
fio.h header a bit, so only mandatory declarations and further
includes are part of it to successfully build an external-ioengine.
I am happy to help on this if fio decides to go this route.

Thoughts?

Best Regards
Daniel




Daniel Gollub (1):
  Rename time.h for third-party include of fio.h

 fio.h      |    2 +-
 fio_time.h |   19 +++++++++++++++++++
 mutex.c    |    2 +-
 time.h     |   19 -------------------
 4 files changed, 21 insertions(+), 21 deletions(-)
 create mode 100644 fio_time.h
 delete mode 100644 time.h

-- 
1.7.10.4

[PATCH] Rename time.h for third-party include of fio.h

[Daniel Gollub]

External project might build external ioengines and need to
include there for fio.h. If a project set the include path
to the fio root source directory and the third-party source
holds an "time.h" as well things get complicated.

Signed-off-by: Daniel Gollub <d.gollub@telekom.de>
Cc: Daniel Gollub <daniel.gollub@t-online.de>
---
 fio.h      |    2 +-
 fio_time.h |   19 +++++++++++++++++++
 mutex.c    |    2 +-
 time.h     |   19 -------------------
 4 files changed, 21 insertions(+), 21 deletions(-)
 create mode 100644 fio_time.h
 delete mode 100644 time.h

diff --git a/fio.h b/fio.h
index 9eecba3..ed034cb 100644
--- a/fio.h
+++ b/fio.h
@@ -30,7 +30,7 @@
 #include "helpers.h"
 #include "options.h"
 #include "profile.h"
-#include "time.h"
+#include "fio_time.h"
 #include "gettime.h"
 #include "lib/getopt.h"
 #include "lib/rand.h"
diff --git a/fio_time.h b/fio_time.h
new file mode 100644
index 0000000..c550a55
--- /dev/null
+++ b/fio_time.h
@@ -0,0 +1,19 @@
+#ifndef FIO_TIME_H
+#define FIO_TIME_H
+
+extern uint64_t utime_since(struct timeval *, struct timeval *);
+extern uint64_t utime_since_now(struct timeval *);
+extern uint64_t mtime_since(struct timeval *, struct timeval *);
+extern uint64_t mtime_since_now(struct timeval *);
+extern uint64_t time_since_now(struct timeval *);
+extern uint64_t mtime_since_genesis(void);
+extern uint64_t utime_since_genesis(void);
+extern void usec_spin(unsigned int);
+extern void usec_sleep(struct thread_data *, unsigned long);
+extern void fill_start_time(struct timeval *);
+extern void set_genesis_time(void);
+extern int ramp_time_over(struct thread_data *);
+extern int in_ramp_time(struct thread_data *);
+extern void fio_time_init(void);
+
+#endif
diff --git a/mutex.c b/mutex.c
index 466e20e..9d10c2c 100644
--- a/mutex.c
+++ b/mutex.c
@@ -15,7 +15,7 @@
 #include "arch/arch.h"
 #include "os/os.h"
 #include "helpers.h"
-#include "time.h"
+#include "fio_time.h"
 #include "gettime.h"
 
 void fio_mutex_remove(struct fio_mutex *mutex)
diff --git a/time.h b/time.h
deleted file mode 100644
index c550a55..0000000
--- a/time.h
+++ /dev/null
@@ -1,19 +0,0 @@
-#ifndef FIO_TIME_H
-#define FIO_TIME_H
-
-extern uint64_t utime_since(struct timeval *, struct timeval *);
-extern uint64_t utime_since_now(struct timeval *);
-extern uint64_t mtime_since(struct timeval *, struct timeval *);
-extern uint64_t mtime_since_now(struct timeval *);
-extern uint64_t time_since_now(struct timeval *);
-extern uint64_t mtime_since_genesis(void);
-extern uint64_t utime_since_genesis(void);
-extern void usec_spin(unsigned int);
-extern void usec_sleep(struct thread_data *, unsigned long);
-extern void fill_start_time(struct timeval *);
-extern void set_genesis_time(void);
-extern int ramp_time_over(struct thread_data *);
-extern int in_ramp_time(struct thread_data *);
-extern void fio_time_init(void);
-
-#endif
-- 
1.7.10.4

Re: fio headers as build dependency for external-ioengines

[Jens Axboe]

On 04/30/2014 03:25 AM, Daniel Gollub wrote:
> Hi,
> 
> I am working right now in preparing for the Ceph project an external
> fio ioengine, which depends on Ceph-internal libraries. So I thought
> about building this ioengine as an external ioengine. (Thats why I
> send couple of weeks/month ago this C++-compiler-issues-fixes for
> fio.h)
> 
> Since fio is not (yet) distributed with a -dev/-devel package in
> linux distribution and was very likely not yet consider doing so
> I wanted to start the discussion here if this in interested of the
> project "exporting"/"installing" the required fio headers to build
> external ioengines. So fio can be later a "packaged" build-dependency.
> 
> For time-being I thought about solving this like this:
> 
> One can introduce in the external project's build-env a parameter
> like: --with-fio=~/projects/fio/ and the resulting build-env will
> pass this path as include directory as compiler flags for the
> external ioengine build.
> 
> Unfortunately there are some issues with double-declaration and
> one ambiguous header file name (e.g. time.h - see provided patch)
> 
> Double-declaration issues of defines/macros/... I hit while building
> the Ceph filestore external ioengine:
> 
>  * ARRAY_SIZE
>  * CONFIG_CPU_COUNT
>  * le16_to_cpu
>  * le32_to_cpu
>  * le64_to_cpu
> 
> 
> If you want to reproduce the described issue with ceph,
> checkout following revision:
> https://github.com/gollub/ceph/commit/079c08c67f811c0173e70201dc91168b60d2e86b
> ( Working branch: https://github.com/gollub/ceph/tree/fio_filestore_v2 )
> 
> And run:
> ./configure --with-fio-dir=/path/to/fio/
> cd src
> make libfio_ceph_filestore.la
> 
> 
> This could be solved/workaround on the external ioengine side, as well
> as in the fio header. One possibility on the fio side could be to relax
> fio.h header a bit, so only mandatory declarations and further
> includes are part of it to successfully build an external-ioengine.
> I am happy to help on this if fio decides to go this route.
> 
> Thoughts?

It'd be a useful thing to be able to do. I applied your time patch,
looks fine to me. Are you going to be able to work through the other
issues? I think we just need to privatize some of the names a bit more.

-- 
Jens Axboe