From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: MIME-Version: 1.0 Date: Thu, 05 Sep 2013 12:56:11 -0700 (PDT) Message-Id: <1378410970250.8ab05bfc@Nodemailer> In-Reply-To: <5228DDC8.9070201@kernel.dk> References: <5228DDC8.9070201@kernel.dk> From: "Brian Lam" Subject: Re: Invalid argument error (EINVAL) on Lustre filesystem with ioengine=libaio enabled Content-Type: multipart/alternative; boundary="----Nodemailer-0.5.0-?=_1-1378410971352" To: Jens Axboe Cc: fio@vger.kernel.org, Juergen Salk List-ID: ------Nodemailer-0.5.0-?=_1-1378410971352 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable I experienced an error when trying to use libaio on local disks. I will = check when I get back to my desk to see if it is similiar to your errors. = =C2=A0I vaguely remember getting a segfault so I am not sure if they are = related.=C2=A0 =E2=80=94 Sent from Mailbox for iPhone On Thu, Sep 5, 2013 at 12:39 PM, Jens Axboe wrote: > On 09/05/2013 09:15 AM, Juergen Salk wrote: >> Hi, >>=20 >> I tried to run fio on a Lustre filesystem with ioengine=3Dlibaio = enabled: >>=20 >> $ ../tools/fio/2.1.2/bin/fio seqwrite.fio=20 >> sequential=5Fwrite: (g=3D0): rw=3Dwrite, bs=3D1M-1M/1M-1M/1M-1M, = ioengine=3Dlibaio, iodepth=3D32 >> fio-2.1.2 >> Starting 1 process >> fio: pid=3D31956, err=3D22/file:ioengines.c:381, func=3Dio commit, = error=3DInvalid argument >>=20 >> sequential=5Fwrite: (groupid=3D0, jobs=3D1): err=3D22 (file:ioengines.= c:381, func=3Dio commit, error=3DInvalid argument): pid=3D31956: Thu Sep 5= 16:43:56 2013 >> cpu : usr=3D0.00%, sys=3D100.00%, ctx=3D23, majf=3D0, = minf=3D49 >> IO depths : 1=3D100.0%, 2=3D0.0%, 4=3D0.0%, 8=3D0.0%, 16=3D0.0%, = 32=3D0.0%, >=3D64=3D0.0% >> submit : 0=3D0.0%, 4=3D0.0%, 8=3D0.0%, 16=3D0.0%, 32=3D0.0%, = 64=3D0.0%, >=3D64=3D0.0% >> complete : 0=3D100.0%, 4=3D0.0%, 8=3D0.0%, 16=3D0.0%, 32=3D0.0%, = 64=3D0.0%, >=3D64=3D0.0% >> issued : total=3Dr=3D0/w=3D1/d=3D0, short=3Dr=3D0/w=3D0/d=3D0 >>=20 >> Run status group 0 (all jobs): >>=20 >> This is the content of the job file: >>=20 >> --- snip --- >>=20 >> [global] >> ioengine=3Dlibaio >> # ioengine=3Dsync >> iodepth=3D32 >> direct=3D1 >> invalidate=3D1 >> filename=5Fformat=3Dfile.$jobnum >> group=5Freporting >> bs=3D1024k >> size=3D1g >> numjobs=3D1 >> directory=3D/scratch1/ws/jsalk-bench-0 >>=20 >> [sequential=5Fwrite] >> rw=3Dwrite >>=20 >> --- snip --- >>=20 >> It works with ioengine=3Dsync, though. It also works on the same = machine >> with ioengine=3Dlibaio enabled but with the work directory located on a >> locally attached disk (with ext3 file system). >>=20 >> An strace log is attached. Are there any ideas of what might be >> causing this problem on Lustre=3F > I'm guessing that lustre doesn't support async IO, though I have never > tried it myself. From the error out of fio and the strace, io=5Fsubmit() > is returning EINVAL. The context id is valid, fio is submitting 1 iocb, > and the opcode is a read operation on fd 3 - all of which are good. > --=20 > Jens Axboe > -- > To unsubscribe from this list: send the line =22unsubscribe fio=22 in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.= html ------Nodemailer-0.5.0-?=_1-1378410971352 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable
I experienced an error when trying to use libaio on local disks. I= will check when I get back to my desk to see if it is similiar to your = errors. =C2=A0I vaguely remember getting a segfault so I am not sure if = they are related.=C2=A0
=E2=80=94
Sent from Mailbox for iPhone


On Thu, Sep 5, 2013 at 12:39 = PM, Jens Axboe <axboe@kernel.dk> = wrote:

On 09/05/2013 = 09:15 AM, Juergen Salk wrote:
> Hi,
>=20
> I tried to run fio on a Lustre filesystem with ioengine=3Dlibaio = enabled:
>=20
> $ ../tools/fio/2.1.2/bin/fio seqwrite.fio=20
> sequential=5Fwrite: (g=3D0): rw=3Dwrite, bs=3D1M-1M/1M-1M/1M-1M, = ioengine=3Dlibaio, iodepth=3D32
> fio-2.1.2
> Starting 1 process
> fio: pid=3D31956, err=3D22/file:ioengines.c:381, func=3Dio commit,= error=3DInvalid argument
>=20
> sequential=5Fwrite: (groupid=3D0, jobs=3D1): err=3D22 = (file:ioengines.c:381, func=3Dio commit, error=3DInvalid argument): = pid=3D31956: Thu Sep 5 16:43:56 2013
> cpu : usr=3D0.00%, sys=3D100.00%, ctx=3D23, majf=3D0, = minf=3D49
> IO depths : 1=3D100.0%, 2=3D0.0%, 4=3D0.0%, 8=3D0.0%, 16=3D0.= 0%, 32=3D0.0%, >=3D64=3D0.0%
> submit : 0=3D0.0%, 4=3D0.0%, 8=3D0.0%, 16=3D0.0%, 32=3D0.= 0%, 64=3D0.0%, >=3D64=3D0.0%
> complete : 0=3D100.0%, 4=3D0.0%, 8=3D0.0%, 16=3D0.0%, 32=3D0= .0%, 64=3D0.0%, >=3D64=3D0.0%
> issued : total=3Dr=3D0/w=3D1/d=3D0, = short=3Dr=3D0/w=3D0/d=3D0
>=20
> Run status group 0 (all jobs):
>=20
> This is the content of the job file:
>=20
> --- snip ---
>=20
> [global]
> ioengine=3Dlibaio
> # ioengine=3Dsync
> iodepth=3D32
> direct=3D1
> invalidate=3D1
> filename=5Fformat=3Dfile.$jobnum
> group=5Freporting
> bs=3D1024k
> size=3D1g
> numjobs=3D1
> directory=3D/scratch1/ws/jsalk-bench-0
>=20
> [sequential=5Fwrite]
> rw=3Dwrite
>=20
> --- snip ---
>=20
> It works with ioengine=3Dsync, though. It also works on the same = machine
> with ioengine=3Dlibaio enabled but with the work directory located= on a
> locally attached disk (with ext3 file system).
>=20
> An strace log is attached. Are there any ideas of what might be
> causing this problem on Lustre=3F

I'm guessing that lustre doesn't support async IO, though I have = never
tried it myself. From the error out of fio and the strace, = io=5Fsubmit()
is returning EINVAL. The context id is valid, fio is submitting 1 iocb,=
and the opcode is a read operation on fd 3 - all of which are good.

--=20
Jens Axboe

--
To unsubscribe from this list: send the line =22unsubscribe fio=22 in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html


------Nodemailer-0.5.0-?=_1-1378410971352--