* Re loading libaio engine error and pvsync2 operation not permitted error
@ 2018-11-25 19:26 Mohanraj B
2018-11-25 19:28 ` Mohanraj B
0 siblings, 1 reply; 8+ messages in thread
From: Mohanraj B @ 2018-11-25 19:26 UTC (permalink / raw)
To: fio
Hello,
I am trying to use libaio engine for a job file in my docker
container, but it throws fio: engine libaio not loadable error.
fio --name=hello1 --size=1m --ioengine=libaio
fio: engine libaio not loadable
fio: engine libaio not loadable
fio: failed to load engine
fio: file:ioengines.c:89, func=dlopen, error=libaio: cannot open
shared object file: No such file or directory
Available IO engines:
[root@210df130f2c7 fio-data]# fio --enghelp
Available IO engines:
cpuio
mmap
sync
psync
vsync
pvsync
pvsync2
null
net
netsplice
ftruncate
filecreate
posixaio
falloc
e4defrag
splice
mtd
sg
libaio is not listed above, but its installed in container.
yum install libaio
Loaded plugins: fastestmirror, ovl
Loading mirror speeds from cached hostfile
* base: ftp.iitm.ac.in
* extras: ftp.iitm.ac.in
* updates: ftp.iitm.ac.in
Package libaio-0.3.109-13.el7.x86_64 already installed and latest version
Nothing to do
Centos docker container : centos-release-7-5.1804.5.el7.centos.x86_64
Re pvsync2 operation not permitted error
fio --name=hello --size=10m --ioengine=pvsync2
hello: (g=0): rw=read, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T)
4096B-4096B, ioengine=pvsync2, iodepth=1
fio-3.12-18-g92a1
Starting 1 process
hello: Laying out IO file (1 file / 10MiB)
fio: io_u error on file hello.0.0: Operation not permitted: read
offset=0, buflen=4096
fio: pid=59, err=1/file:io_u.c:1789, func=io_u error, error=Operation
not permitted
hello: (groupid=0, jobs=1): err= 1 (file:io_u.c:1789, func=io_u error,
error=Operation not permitted): pid=59: Sun Nov 25 19:23:14 2018
cpu : usr=0.00%, sys=0.00%, ctx=2, majf=0, minf=19
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=1,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
Disk stats (read/write):
sda: ios=0/0, merge=0/0, ticks=0/0, in_queue=0, util=0.00%
I dont have access to machine running centos, so running in centos
docker container. Are these issues occuring because of running in fio?
host machine OS is ubuntu 18.04.
Thanks and Regards,
Mohan
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Re loading libaio engine error and pvsync2 operation not permitted error
2018-11-25 19:26 Re loading libaio engine error and pvsync2 operation not permitted error Mohanraj B
@ 2018-11-25 19:28 ` Mohanraj B
2018-11-25 19:41 ` Beierl, Mark
0 siblings, 1 reply; 8+ messages in thread
From: Mohanraj B @ 2018-11-25 19:28 UTC (permalink / raw)
To: fio
typo
running in docker container.
On 11/26/18, Mohanraj B <bmohanraj91@gmail.com> wrote:
> Hello,
>
> I am trying to use libaio engine for a job file in my docker
> container, but it throws fio: engine libaio not loadable error.
>
> fio --name=hello1 --size=1m --ioengine=libaio
>
> fio: engine libaio not loadable
> fio: engine libaio not loadable
> fio: failed to load engine
> fio: file:ioengines.c:89, func=dlopen, error=libaio: cannot open
> shared object file: No such file or directory
>
> Available IO engines:
> [root@210df130f2c7 fio-data]# fio --enghelp
> Available IO engines:
> cpuio
> mmap
> sync
> psync
> vsync
> pvsync
> pvsync2
> null
> net
> netsplice
> ftruncate
> filecreate
> posixaio
> falloc
> e4defrag
> splice
> mtd
> sg
>
>
> libaio is not listed above, but its installed in container.
>
> yum install libaio
> Loaded plugins: fastestmirror, ovl
> Loading mirror speeds from cached hostfile
> * base: ftp.iitm.ac.in
> * extras: ftp.iitm.ac.in
> * updates: ftp.iitm.ac.in
> Package libaio-0.3.109-13.el7.x86_64 already installed and latest version
> Nothing to do
>
> Centos docker container : centos-release-7-5.1804.5.el7.centos.x86_64
>
>
> Re pvsync2 operation not permitted error
>
> fio --name=hello --size=10m --ioengine=pvsync2
>
> hello: (g=0): rw=read, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T)
> 4096B-4096B, ioengine=pvsync2, iodepth=1
> fio-3.12-18-g92a1
> Starting 1 process
> hello: Laying out IO file (1 file / 10MiB)
> fio: io_u error on file hello.0.0: Operation not permitted: read
> offset=0, buflen=4096
> fio: pid=59, err=1/file:io_u.c:1789, func=io_u error, error=Operation
> not permitted
>
> hello: (groupid=0, jobs=1): err= 1 (file:io_u.c:1789, func=io_u error,
> error=Operation not permitted): pid=59: Sun Nov 25 19:23:14 2018
> cpu : usr=0.00%, sys=0.00%, ctx=2, majf=0, minf=19
> IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%,
>>=64=0.0%
> submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%,
>>=64=0.0%
> complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%,
>>=64=0.0%
> issued rwts: total=1,0,0,0 short=0,0,0,0 dropped=0,0,0,0
> latency : target=0, window=0, percentile=100.00%, depth=1
>
> Run status group 0 (all jobs):
>
> Disk stats (read/write):
> sda: ios=0/0, merge=0/0, ticks=0/0, in_queue=0, util=0.00%
>
> I dont have access to machine running centos, so running in centos
> docker container. Are these issues occuring because of running in fio?
>
> host machine OS is ubuntu 18.04.
>
> Thanks and Regards,
> Mohan
>
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Re loading libaio engine error and pvsync2 operation not permitted error
2018-11-25 19:28 ` Mohanraj B
@ 2018-11-25 19:41 ` Beierl, Mark
2018-11-25 20:01 ` Mohanraj B
0 siblings, 1 reply; 8+ messages in thread
From: Beierl, Mark @ 2018-11-25 19:41 UTC (permalink / raw)
To: Mohanraj B, fio
Hello!
I have a couple of questions, as I also build FIO in an Alpine container, but there I build it from source so that I have all required libraries. The source for the container is here [1] and it is part of the OPNFV StorPerf project, which is for wrapping FIO in a web front end and making it easier to test against various distributions or VMs [2].
My questions are:
1) How are you installing FIO – from the apt repos, or building from source?
2) What storage are you using in the container – a mounted volume or are you trying to use the host OS/raw device?
3) Is the container running as privileged if you are trying to access the host?
[1] https://git.opnfv.org/storperf/tree/docker/storperf-master/Dockerfile
[2] https://wiki.opnfv.org/display/storperf
Regards,
Mark
Mark Beierl
SW System Sr Principal Developer
Dell EMC | Cloud & Communication Service Provider Solution
Mark.Beierl@Dell.com
On 2018-11-25, 15:28, "fio-owner@vger.kernel.org on behalf of Mohanraj B" <fio-owner@vger.kernel.org on behalf of bmohanraj91@gmail.com> wrote:
[EXTERNAL EMAIL]
typo
running in docker container.
On 11/26/18, Mohanraj B <bmohanraj91@gmail.com> wrote:
> Hello,
>
> I am trying to use libaio engine for a job file in my docker
> container, but it throws fio: engine libaio not loadable error.
>
> fio --name=hello1 --size=1m --ioengine=libaio
>
> fio: engine libaio not loadable
> fio: engine libaio not loadable
> fio: failed to load engine
> fio: file:ioengines.c:89, func=dlopen, error=libaio: cannot open
> shared object file: No such file or directory
>
> Available IO engines:
> [root@210df130f2c7 fio-data]# fio --enghelp
> Available IO engines:
> cpuio
> mmap
> sync
> psync
> vsync
> pvsync
> pvsync2
> null
> net
> netsplice
> ftruncate
> filecreate
> posixaio
> falloc
> e4defrag
> splice
> mtd
> sg
>
>
> libaio is not listed above, but its installed in container.
>
> yum install libaio
> Loaded plugins: fastestmirror, ovl
> Loading mirror speeds from cached hostfile
> * base: ftp.iitm.ac.in
> * extras: ftp.iitm.ac.in
> * updates: ftp.iitm.ac.in
> Package libaio-0.3.109-13.el7.x86_64 already installed and latest version
> Nothing to do
>
> Centos docker container : centos-release-7-5.1804.5.el7.centos.x86_64
>
>
> Re pvsync2 operation not permitted error
>
> fio --name=hello --size=10m --ioengine=pvsync2
>
> hello: (g=0): rw=read, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T)
> 4096B-4096B, ioengine=pvsync2, iodepth=1
> fio-3.12-18-g92a1
> Starting 1 process
> hello: Laying out IO file (1 file / 10MiB)
> fio: io_u error on file hello.0.0: Operation not permitted: read
> offset=0, buflen=4096
> fio: pid=59, err=1/file:io_u.c:1789, func=io_u error, error=Operation
> not permitted
>
> hello: (groupid=0, jobs=1): err= 1 (file:io_u.c:1789, func=io_u error,
> error=Operation not permitted): pid=59: Sun Nov 25 19:23:14 2018
> cpu : usr=0.00%, sys=0.00%, ctx=2, majf=0, minf=19
> IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%,
>>=64=0.0%
> submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%,
>>=64=0.0%
> complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%,
>>=64=0.0%
> issued rwts: total=1,0,0,0 short=0,0,0,0 dropped=0,0,0,0
> latency : target=0, window=0, percentile=100.00%, depth=1
>
> Run status group 0 (all jobs):
>
> Disk stats (read/write):
> sda: ios=0/0, merge=0/0, ticks=0/0, in_queue=0, util=0.00%
>
> I dont have access to machine running centos, so running in centos
> docker container. Are these issues occuring because of running in fio?
>
> host machine OS is ubuntu 18.04.
>
> Thanks and Regards,
> Mohan
>
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Re loading libaio engine error and pvsync2 operation not permitted error
2018-11-25 19:41 ` Beierl, Mark
@ 2018-11-25 20:01 ` Mohanraj B
2018-11-25 20:12 ` Beierl, Mark
2018-11-25 23:51 ` Damien Le Moal
0 siblings, 2 replies; 8+ messages in thread
From: Mohanraj B @ 2018-11-25 20:01 UTC (permalink / raw)
To: Beierl, Mark; +Cc: fio
Hi,
1) How are you installing FIO – from the apt repos, or building from source?
I build fio from source, below is my dockerfile.
FROM centos:latest
RUN yum update -y
RUN yum install git gcc gcc-c++ make -y
RUN git clone https://github.com/axboe/fio
RUN yum install zlib-devel libaio -y
WORKDIR /fio
RUN ./configure
RUN make
RUN make install
VOLUME /tmp/fio-data
WORKDIR /tmp/fio-data
ENTRYPOINT ["fio"]
2) What storage are you using in the container – a mounted volume or
are you trying to use the host OS/raw device?
After reading this question, I getting more doubts, with the above
docker file and few examples given below.
docker run fio:6.0 --name:job1 --size=50m --rw=read
docker run fio:6.0 --name:job1 --size=100m --rw=readwrite
Am I really testing the host HDD or testing docker volume only. If I
am testing only the docker volume then its not my intended use case, I
need to test host machine HDD.
What is the best way to test host machine HDD ?
3) Is the container running as privileged if you are trying to access the host?
Only when I am trying to test a RAW device, I use --privileged=true
like below otherwise I dont supply --privileged=true .
docker run --privileged=true fio:6.0 --name=file --size=100m
--filename=/dev/sdc
Thanks and Regards,
Mohan
On 11/26/18, Beierl, Mark <Mark.Beierl@dell.com> wrote:
> Hello!
>
> I have a couple of questions, as I also build FIO in an Alpine container,
> but there I build it from source so that I have all required libraries. The
> source for the container is here [1] and it is part of the OPNFV StorPerf
> project, which is for wrapping FIO in a web front end and making it easier
> to test against various distributions or VMs [2].
>
> My questions are:
>
> 1) How are you installing FIO – from the apt repos, or building from
> source?
> 2) What storage are you using in the container – a mounted volume or are you
> trying to use the host OS/raw device?
> 3) Is the container running as privileged if you are trying to access the
> host?
>
> [1] https://git.opnfv.org/storperf/tree/docker/storperf-master/Dockerfile
> [2] https://wiki.opnfv.org/display/storperf
>
> Regards,
> Mark
>
> Mark Beierl
> SW System Sr Principal Developer
> Dell EMC | Cloud & Communication Service Provider Solution
> Mark.Beierl@Dell.com
>
> On 2018-11-25, 15:28, "fio-owner@vger.kernel.org on behalf of Mohanraj B"
> <fio-owner@vger.kernel.org on behalf of bmohanraj91@gmail.com> wrote:
>
>
> [EXTERNAL EMAIL]
>
> typo
> running in docker container.
>
> On 11/26/18, Mohanraj B <bmohanraj91@gmail.com> wrote:
> > Hello,
> >
> > I am trying to use libaio engine for a job file in my docker
> > container, but it throws fio: engine libaio not loadable error.
> >
> > fio --name=hello1 --size=1m --ioengine=libaio
> >
> > fio: engine libaio not loadable
> > fio: engine libaio not loadable
> > fio: failed to load engine
> > fio: file:ioengines.c:89, func=dlopen, error=libaio: cannot open
> > shared object file: No such file or directory
> >
> > Available IO engines:
> > [root@210df130f2c7 fio-data]# fio --enghelp
> > Available IO engines:
> > cpuio
> > mmap
> > sync
> > psync
> > vsync
> > pvsync
> > pvsync2
> > null
> > net
> > netsplice
> > ftruncate
> > filecreate
> > posixaio
> > falloc
> > e4defrag
> > splice
> > mtd
> > sg
> >
> >
> > libaio is not listed above, but its installed in container.
> >
> > yum install libaio
> > Loaded plugins: fastestmirror, ovl
> > Loading mirror speeds from cached hostfile
> > * base: ftp.iitm.ac.in
> > * extras: ftp.iitm.ac.in
> > * updates: ftp.iitm.ac.in
> > Package libaio-0.3.109-13.el7.x86_64 already installed and latest
> version
> > Nothing to do
> >
> > Centos docker container : centos-release-7-5.1804.5.el7.centos.x86_64
> >
> >
> > Re pvsync2 operation not permitted error
> >
> > fio --name=hello --size=10m --ioengine=pvsync2
> >
> > hello: (g=0): rw=read, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T)
> > 4096B-4096B, ioengine=pvsync2, iodepth=1
> > fio-3.12-18-g92a1
> > Starting 1 process
> > hello: Laying out IO file (1 file / 10MiB)
> > fio: io_u error on file hello.0.0: Operation not permitted: read
> > offset=0, buflen=4096
> > fio: pid=59, err=1/file:io_u.c:1789, func=io_u error, error=Operation
> > not permitted
> >
> > hello: (groupid=0, jobs=1): err= 1 (file:io_u.c:1789, func=io_u
> error,
> > error=Operation not permitted): pid=59: Sun Nov 25 19:23:14 2018
> > cpu : usr=0.00%, sys=0.00%, ctx=2, majf=0, minf=19
> > IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%,
> >>=64=0.0%
> > submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%,
> >>=64=0.0%
> > complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%,
> >>=64=0.0%
> > issued rwts: total=1,0,0,0 short=0,0,0,0 dropped=0,0,0,0
> > latency : target=0, window=0, percentile=100.00%, depth=1
> >
> > Run status group 0 (all jobs):
> >
> > Disk stats (read/write):
> > sda: ios=0/0, merge=0/0, ticks=0/0, in_queue=0, util=0.00%
> >
> > I dont have access to machine running centos, so running in centos
> > docker container. Are these issues occuring because of running in
> fio?
> >
> > host machine OS is ubuntu 18.04.
> >
> > Thanks and Regards,
> > Mohan
> >
>
>
>
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Re loading libaio engine error and pvsync2 operation not permitted error
2018-11-25 20:01 ` Mohanraj B
@ 2018-11-25 20:12 ` Beierl, Mark
2018-11-25 23:51 ` Damien Le Moal
1 sibling, 0 replies; 8+ messages in thread
From: Beierl, Mark @ 2018-11-25 20:12 UTC (permalink / raw)
To: Mohanraj B; +Cc: fio
Ok, so maybe just launch the container with /bin/bash or /bin/sh instead of having it run FIO. Then once you are inside the running container, cd to /fio and manually run the ./configure to see if libaiao is being found. It is possible that it got installed in a different directory than what ./configure is expecting.
As far as the host filesystem goes, when running a simple container as you showed below, it will be using AUFS as an overlay filesystem and not performing I/O against the host filesystem. I would recommend mounting a volume in a directory that is part of the mounted filesystem you wish to profile, and tell FIO to use that path instead. That way you are closer to profiling the host vs. profiling Docker AUFS.
If you really want to ensure you are going directly against the host filesystem, I would think that you would need to run privileged and talk directly to the desired path.
Regards,
Mark
Mark Beierl
SW System Sr Principal Developer
Dell EMC | Cloud & Communication Service Provider Solution
Mark.Beierl@Dell.com
On 2018-11-25, 16:01, "Mohanraj B" <bmohanraj91@gmail.com> wrote:
[EXTERNAL EMAIL]
Hi,
1) How are you installing FIO – from the apt repos, or building from source?
I build fio from source, below is my dockerfile.
FROM centos:latest
RUN yum update -y
RUN yum install git gcc gcc-c++ make -y
RUN git clone https://github.com/axboe/fio
RUN yum install zlib-devel libaio -y
WORKDIR /fio
RUN ./configure
RUN make
RUN make install
VOLUME /tmp/fio-data
WORKDIR /tmp/fio-data
ENTRYPOINT ["fio"]
2) What storage are you using in the container – a mounted volume or
are you trying to use the host OS/raw device?
After reading this question, I getting more doubts, with the above
docker file and few examples given below.
docker run fio:6.0 --name:job1 --size=50m --rw=read
docker run fio:6.0 --name:job1 --size=100m --rw=readwrite
Am I really testing the host HDD or testing docker volume only. If I
am testing only the docker volume then its not my intended use case, I
need to test host machine HDD.
What is the best way to test host machine HDD ?
3) Is the container running as privileged if you are trying to access the host?
Only when I am trying to test a RAW device, I use --privileged=true
like below otherwise I dont supply --privileged=true .
docker run --privileged=true fio:6.0 --name=file --size=100m
--filename=/dev/sdc
Thanks and Regards,
Mohan
On 11/26/18, Beierl, Mark <Mark.Beierl@dell.com> wrote:
> Hello!
>
> I have a couple of questions, as I also build FIO in an Alpine container,
> but there I build it from source so that I have all required libraries. The
> source for the container is here [1] and it is part of the OPNFV StorPerf
> project, which is for wrapping FIO in a web front end and making it easier
> to test against various distributions or VMs [2].
>
> My questions are:
>
> 1) How are you installing FIO – from the apt repos, or building from
> source?
> 2) What storage are you using in the container – a mounted volume or are you
> trying to use the host OS/raw device?
> 3) Is the container running as privileged if you are trying to access the
> host?
>
> [1] https://git.opnfv.org/storperf/tree/docker/storperf-master/Dockerfile
> [2] https://wiki.opnfv.org/display/storperf
>
> Regards,
> Mark
>
> Mark Beierl
> SW System Sr Principal Developer
> Dell EMC | Cloud & Communication Service Provider Solution
> Mark.Beierl@Dell.com
>
> On 2018-11-25, 15:28, "fio-owner@vger.kernel.org on behalf of Mohanraj B"
> <fio-owner@vger.kernel.org on behalf of bmohanraj91@gmail.com> wrote:
>
>
> [EXTERNAL EMAIL]
>
> typo
> running in docker container.
>
> On 11/26/18, Mohanraj B <bmohanraj91@gmail.com> wrote:
> > Hello,
> >
> > I am trying to use libaio engine for a job file in my docker
> > container, but it throws fio: engine libaio not loadable error.
> >
> > fio --name=hello1 --size=1m --ioengine=libaio
> >
> > fio: engine libaio not loadable
> > fio: engine libaio not loadable
> > fio: failed to load engine
> > fio: file:ioengines.c:89, func=dlopen, error=libaio: cannot open
> > shared object file: No such file or directory
> >
> > Available IO engines:
> > [root@210df130f2c7 fio-data]# fio --enghelp
> > Available IO engines:
> > cpuio
> > mmap
> > sync
> > psync
> > vsync
> > pvsync
> > pvsync2
> > null
> > net
> > netsplice
> > ftruncate
> > filecreate
> > posixaio
> > falloc
> > e4defrag
> > splice
> > mtd
> > sg
> >
> >
> > libaio is not listed above, but its installed in container.
> >
> > yum install libaio
> > Loaded plugins: fastestmirror, ovl
> > Loading mirror speeds from cached hostfile
> > * base: ftp.iitm.ac.in
> > * extras: ftp.iitm.ac.in
> > * updates: ftp.iitm.ac.in
> > Package libaio-0.3.109-13.el7.x86_64 already installed and latest
> version
> > Nothing to do
> >
> > Centos docker container : centos-release-7-5.1804.5.el7.centos.x86_64
> >
> >
> > Re pvsync2 operation not permitted error
> >
> > fio --name=hello --size=10m --ioengine=pvsync2
> >
> > hello: (g=0): rw=read, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T)
> > 4096B-4096B, ioengine=pvsync2, iodepth=1
> > fio-3.12-18-g92a1
> > Starting 1 process
> > hello: Laying out IO file (1 file / 10MiB)
> > fio: io_u error on file hello.0.0: Operation not permitted: read
> > offset=0, buflen=4096
> > fio: pid=59, err=1/file:io_u.c:1789, func=io_u error, error=Operation
> > not permitted
> >
> > hello: (groupid=0, jobs=1): err= 1 (file:io_u.c:1789, func=io_u
> error,
> > error=Operation not permitted): pid=59: Sun Nov 25 19:23:14 2018
> > cpu : usr=0.00%, sys=0.00%, ctx=2, majf=0, minf=19
> > IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%,
> >>=64=0.0%
> > submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%,
> >>=64=0.0%
> > complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%,
> >>=64=0.0%
> > issued rwts: total=1,0,0,0 short=0,0,0,0 dropped=0,0,0,0
> > latency : target=0, window=0, percentile=100.00%, depth=1
> >
> > Run status group 0 (all jobs):
> >
> > Disk stats (read/write):
> > sda: ios=0/0, merge=0/0, ticks=0/0, in_queue=0, util=0.00%
> >
> > I dont have access to machine running centos, so running in centos
> > docker container. Are these issues occuring because of running in
> fio?
> >
> > host machine OS is ubuntu 18.04.
> >
> > Thanks and Regards,
> > Mohan
> >
>
>
>
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Re loading libaio engine error and pvsync2 operation not permitted error
2018-11-25 20:01 ` Mohanraj B
2018-11-25 20:12 ` Beierl, Mark
@ 2018-11-25 23:51 ` Damien Le Moal
2018-11-28 15:28 ` Jens Axboe
1 sibling, 1 reply; 8+ messages in thread
From: Damien Le Moal @ 2018-11-25 23:51 UTC (permalink / raw)
To: Mohanraj B, Beierl, Mark; +Cc: fio
On 2018/11/26 5:01, Mohanraj B wrote:
> Hi,
>
> 1) How are you installing FIO – from the apt repos, or building from source?
> I build fio from source, below is my dockerfile.
>
> FROM centos:latest
> RUN yum update -y
> RUN yum install git gcc gcc-c++ make -y
> RUN git clone https://github.com/axboe/fio
> RUN yum install zlib-devel libaio -y
yum install libaio-devel ?
Otherwise, libaio will not be compiled...
> WORKDIR /fio
> RUN ./configure
> RUN make
> RUN make install
> VOLUME /tmp/fio-data
> WORKDIR /tmp/fio-data
> ENTRYPOINT ["fio"]
>
>
> 2) What storage are you using in the container – a mounted volume or
> are you trying to use the host OS/raw device?
> After reading this question, I getting more doubts, with the above
> docker file and few examples given below.
> docker run fio:6.0 --name:job1 --size=50m --rw=read
> docker run fio:6.0 --name:job1 --size=100m --rw=readwrite
>
> Am I really testing the host HDD or testing docker volume only. If I
> am testing only the docker volume then its not my intended use case, I
> need to test host machine HDD.
> What is the best way to test host machine HDD ?
>
> 3) Is the container running as privileged if you are trying to access the host?
> Only when I am trying to test a RAW device, I use --privileged=true
> like below otherwise I dont supply --privileged=true .
> docker run --privileged=true fio:6.0 --name=file --size=100m
> --filename=/dev/sdc
>
> Thanks and Regards,
> Mohan
>
> On 11/26/18, Beierl, Mark <Mark.Beierl@dell.com> wrote:
>> Hello!
>>
>> I have a couple of questions, as I also build FIO in an Alpine container,
>> but there I build it from source so that I have all required libraries. The
>> source for the container is here [1] and it is part of the OPNFV StorPerf
>> project, which is for wrapping FIO in a web front end and making it easier
>> to test against various distributions or VMs [2].
>>
>> My questions are:
>>
>> 1) How are you installing FIO – from the apt repos, or building from
>> source?
>> 2) What storage are you using in the container – a mounted volume or are you
>> trying to use the host OS/raw device?
>> 3) Is the container running as privileged if you are trying to access the
>> host?
>>
>> [1] https://git.opnfv.org/storperf/tree/docker/storperf-master/Dockerfile
>> [2] https://wiki.opnfv.org/display/storperf
>>
>> Regards,
>> Mark
>>
>> Mark Beierl
>> SW System Sr Principal Developer
>> Dell EMC | Cloud & Communication Service Provider Solution
>> Mark.Beierl@Dell.com
>>
>> On 2018-11-25, 15:28, "fio-owner@vger.kernel.org on behalf of Mohanraj B"
>> <fio-owner@vger.kernel.org on behalf of bmohanraj91@gmail.com> wrote:
>>
>>
>> [EXTERNAL EMAIL]
>>
>> typo
>> running in docker container.
>>
>> On 11/26/18, Mohanraj B <bmohanraj91@gmail.com> wrote:
>> > Hello,
>> >
>> > I am trying to use libaio engine for a job file in my docker
>> > container, but it throws fio: engine libaio not loadable error.
>> >
>> > fio --name=hello1 --size=1m --ioengine=libaio
>> >
>> > fio: engine libaio not loadable
>> > fio: engine libaio not loadable
>> > fio: failed to load engine
>> > fio: file:ioengines.c:89, func=dlopen, error=libaio: cannot open
>> > shared object file: No such file or directory
>> >
>> > Available IO engines:
>> > [root@210df130f2c7 fio-data]# fio --enghelp
>> > Available IO engines:
>> > cpuio
>> > mmap
>> > sync
>> > psync
>> > vsync
>> > pvsync
>> > pvsync2
>> > null
>> > net
>> > netsplice
>> > ftruncate
>> > filecreate
>> > posixaio
>> > falloc
>> > e4defrag
>> > splice
>> > mtd
>> > sg
>> >
>> >
>> > libaio is not listed above, but its installed in container.
>> >
>> > yum install libaio
>> > Loaded plugins: fastestmirror, ovl
>> > Loading mirror speeds from cached hostfile
>> > * base: ftp.iitm.ac.in
>> > * extras: ftp.iitm.ac.in
>> > * updates: ftp.iitm.ac.in
>> > Package libaio-0.3.109-13.el7.x86_64 already installed and latest
>> version
>> > Nothing to do
>> >
>> > Centos docker container : centos-release-7-5.1804.5.el7.centos.x86_64
>> >
>> >
>> > Re pvsync2 operation not permitted error
>> >
>> > fio --name=hello --size=10m --ioengine=pvsync2
>> >
>> > hello: (g=0): rw=read, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T)
>> > 4096B-4096B, ioengine=pvsync2, iodepth=1
>> > fio-3.12-18-g92a1
>> > Starting 1 process
>> > hello: Laying out IO file (1 file / 10MiB)
>> > fio: io_u error on file hello.0.0: Operation not permitted: read
>> > offset=0, buflen=4096
>> > fio: pid=59, err=1/file:io_u.c:1789, func=io_u error, error=Operation
>> > not permitted
>> >
>> > hello: (groupid=0, jobs=1): err= 1 (file:io_u.c:1789, func=io_u
>> error,
>> > error=Operation not permitted): pid=59: Sun Nov 25 19:23:14 2018
>> > cpu : usr=0.00%, sys=0.00%, ctx=2, majf=0, minf=19
>> > IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%,
>> >>=64=0.0%
>> > submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%,
>> >>=64=0.0%
>> > complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%,
>> >>=64=0.0%
>> > issued rwts: total=1,0,0,0 short=0,0,0,0 dropped=0,0,0,0
>> > latency : target=0, window=0, percentile=100.00%, depth=1
>> >
>> > Run status group 0 (all jobs):
>> >
>> > Disk stats (read/write):
>> > sda: ios=0/0, merge=0/0, ticks=0/0, in_queue=0, util=0.00%
>> >
>> > I dont have access to machine running centos, so running in centos
>> > docker container. Are these issues occuring because of running in
>> fio?
>> >
>> > host machine OS is ubuntu 18.04.
>> >
>> > Thanks and Regards,
>> > Mohan
>> >
>>
>>
>>
>
--
Damien Le Moal
Western Digital Research
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Re loading libaio engine error and pvsync2 operation not permitted error
2018-11-25 23:51 ` Damien Le Moal
@ 2018-11-28 15:28 ` Jens Axboe
2018-11-29 4:07 ` Mohanraj B
0 siblings, 1 reply; 8+ messages in thread
From: Jens Axboe @ 2018-11-28 15:28 UTC (permalink / raw)
To: Damien Le Moal, Mohanraj B, Beierl, Mark; +Cc: fio
On 11/25/18 4:51 PM, Damien Le Moal wrote:
> On 2018/11/26 5:01, Mohanraj B wrote:
>> Hi,
>>
>> 1) How are you installing FIO � from the apt repos, or building from source?
>> I build fio from source, below is my dockerfile.
>>
>> FROM centos:latest
>> RUN yum update -y
>> RUN yum install git gcc gcc-c++ make -y
>> RUN git clone https://github.com/axboe/fio
>> RUN yum install zlib-devel libaio -y
>
> yum install libaio-devel ?
> Otherwise, libaio will not be compiled...
That's the real answer, I believe that's also mentioned in the
instructions for building fio.
--
Jens Axboe
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Re loading libaio engine error and pvsync2 operation not permitted error
2018-11-28 15:28 ` Jens Axboe
@ 2018-11-29 4:07 ` Mohanraj B
0 siblings, 0 replies; 8+ messages in thread
From: Mohanraj B @ 2018-11-29 4:07 UTC (permalink / raw)
To: Jens Axboe; +Cc: Damien Le Moal, Beierl, Mark, fio
Thanks, its working fine.
On 11/28/18, Jens Axboe <axboe@kernel.dk> wrote:
> On 11/25/18 4:51 PM, Damien Le Moal wrote:
>> On 2018/11/26 5:01, Mohanraj B wrote:
>>> Hi,
>>>
>>> 1) How are you installing FIO – from the apt repos, or building from
>>> source?
>>> I build fio from source, below is my dockerfile.
>>>
>>> FROM centos:latest
>>> RUN yum update -y
>>> RUN yum install git gcc gcc-c++ make -y
>>> RUN git clone https://github.com/axboe/fio
>>> RUN yum install zlib-devel libaio -y
>>
>> yum install libaio-devel ?
>> Otherwise, libaio will not be compiled...
>
> That's the real answer, I believe that's also mentioned in the
> instructions for building fio.
>
> --
> Jens Axboe
>
>
^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2018-11-29 4:07 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2018-11-25 19:26 Re loading libaio engine error and pvsync2 operation not permitted error Mohanraj B
2018-11-25 19:28 ` Mohanraj B
2018-11-25 19:41 ` Beierl, Mark
2018-11-25 20:01 ` Mohanraj B
2018-11-25 20:12 ` Beierl, Mark
2018-11-25 23:51 ` Damien Le Moal
2018-11-28 15:28 ` Jens Axboe
2018-11-29 4:07 ` Mohanraj B
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox