Re-2: verify seems not to take place on iSCSI-Target writing

Re-2: verify seems not to take place on iSCSI-Target writing

[Stefan Hübner]

Dear Jens,

this is the output.  I added some supplementary details in the hope it's useful ...

UltraBitch ~ # ISCSI_DEVICE=/dev/sdb SECONDS=$((20*3600)) fio iscsi-stress.fio 
iSCSI: (g=0): rw=randwrite, bs=4K-4M/4K-4M, ioengine=libaio, iodepth=16
fio 2.0.3
Starting 1 process
Jobs: 1 (f=1): [w] [17.3% done] [0K/113.2M /s] [0 /71  iops] [eta 03d:23h:28m:21s]
iSCSI: (groupid=0, jobs=1): err= 0: pid=18184
  write: io=7740.2GB, bw=112723KB/s, iops=63 , runt=72000222msec
    bw (KB/s)  : min= 3669, max=593424, per=100.00%, avg=113561.25, stdev=26868.27
  cpu          : usr=4.18%, sys=1.28%, ctx=4760480, majf=0, minf=106912
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=4560578/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
  WRITE: io=7740.2GB, aggrb=112723KB/s, minb=115428KB/s, maxb=115428KB/s, mint=72000222msec, maxt=72000222msec

Disk stats (read/write):
  sdb: ios=80/18153218, merge=0/0, ticks=60/156412264, in_queue=156415080, util=100.00%
UltraBitch ~ # lsscsi 
[0:0:0:0]    disk    ATA      GB0160EAPRR      HPG2  /dev/sda 
[3:0:0:0]    cd/dvd  TSSTcorp CDDVDW SN-S083F  SB01  /dev/sr0 
[19:0:0:0]   disk    SCST_BIO WgAdboGFHRVydqoQ  200  /dev/sdb
UltraBitch ~ # iscsiadm -m session -P 3
iSCSI Transport Class version 2.0-870
version 2.0-871
Target: iqn.2012-02:dss-hh-pri.target0
        Current Portal: 192.168.4.231:3260,1
        Persistent Portal: 192.168.4.231:3260,1
                **********
                Interface:
                **********
                Iface Name: default
                Iface Transport: tcp
                Iface Initiatorname: iqn.2011-12.de.wwwgroup.intern.UltraBitch:openiscsi-6f9714c1547a126c14b1721b310daf11
                Iface IPaddress: 192.168.4.1
                Iface HWaddress: (null)
                Iface Netdev: (null)
                SID: 13
                iSCSI Connection State: LOGGED IN
                iSCSI Session State: LOGGED_IN
                Internal iscsid Session State: NO CHANGE
                ************************
                Negotiated iSCSI params:
                ************************
                HeaderDigest: None
                DataDigest: None
                MaxRecvDataSegmentLength: 262144
                MaxXmitDataSegmentLength: 8192
                FirstBurstLength: 65536
                MaxBurstLength: 262144
                ImmediateData: No
                InitialR2T: Yes
                MaxOutstandingR2T: 1
                ************************
                Attached SCSI devices:
                ************************
                Host Number: 19 State: running
                scsi19 Channel 00 Id 0 Lun: 0
                        Attached scsi disk sdb          State: running

Cheers,
Stefan

-------- Original Message --------
Subject: Re: verify seems not to take place on iSCSI-Target writing (01-Mrz-2012 14:02)
From:    Jens Axboe <axboe@kernel.dk>
To:      shu@wwwgroup.de

> On 03/01/2012 11:09 AM, Stefan Hübner wrote:
> > Hi List,
> > 
> > I'm using this FIO-job to stress-test iSCSI-Targets:
> > 
> > [global]
> > disable_lat=1
> > disable_clat=1
> > disable_slat=1
> > clat_percentiles=0
> > time_based
> > runtime=${SECONDS}
> > nice=10
> > prio=5
> > ioengine=libaio
> > iodepth=16
> > direct=1
> > iomem_align=4M
> > verify=crc32c-intel
> > verify_async=2
> > do_verify=1
> > verify_dump=1
> > [iSCSI]
> > rw=randwrite
> > bsrange=4k-4M
> > filename=${ISCSI_DEVICE}
> > 
> > ISCSI_DEVICE=/dev/sdX
> > SECONDS=some value how long it should take
> > 
> > When running this job I cannot see the verification taking place.  I.e. 
> > network thruput to the target (writing) is happening at around 115MB/s, but 
> > thruput from the target (reading/verifying) doesn't exceed 2,7MB/s.
> > 
> > Am I doing something wrong, here?
> 
> Does the fio job transition to the verify phase? You can see that in the
> [] listing, a V would be the verify. When the job exits, it also shows
> you both how many bytes were read and written. The read could should be
> the verified part.
> 
> So please post the output of the job so we can take a closer look.
> 
> -- 
> Jens Axboe
> 


To: axboe@kernel.dk
Cc: fio@vger.kernel.org

Re: Re-2: verify seems not to take place on iSCSI-Target writing

[Jens Axboe]

On 2012-03-02 18:06, Stefan Hübner wrote:
> Dear Jens,
> 
> this is the output.  I added some supplementary details in the hope it's useful ...
> 
> UltraBitch ~ # ISCSI_DEVICE=/dev/sdb SECONDS=$((20*3600)) fio iscsi-stress.fio 
> iSCSI: (g=0): rw=randwrite, bs=4K-4M/4K-4M, ioengine=libaio, iodepth=16
> fio 2.0.3
> Starting 1 process
> Jobs: 1 (f=1): [w] [17.3% done] [0K/113.2M /s] [0 /71  iops] [eta 03d:23h:28m:21s]
> iSCSI: (groupid=0, jobs=1): err= 0: pid=18184
>   write: io=7740.2GB, bw=112723KB/s, iops=63 , runt=72000222msec
>     bw (KB/s)  : min= 3669, max=593424, per=100.00%, avg=113561.25, stdev=26868.27
>   cpu          : usr=4.18%, sys=1.28%, ctx=4760480, majf=0, minf=106912
>   IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0%
>      submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
>      complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
>      issued    : total=r=0/w=4560578/d=0, short=r=0/w=0/d=0
> 
> Run status group 0 (all jobs):
>   WRITE: io=7740.2GB, aggrb=112723KB/s, minb=115428KB/s, maxb=115428KB/s, mint=72000222msec, maxt=72000222msec

No reads happened - quick guess, you had the timeout hit before the
write phase has finished. Hence it'll exit, and don't do verifies.

The reads you saw in the stats must be from other processes. Fio didn't
do any.

-- 
Jens Axboe