* Re: Abort Task ?
[not found] ` <1192721909.3596.15.camel@dhcp-117.ibrix.com>
@ 2007-10-18 16:33 ` BERTRAND Joël
2007-10-18 17:14 ` BERTRAND Joël
2007-10-18 17:30 ` Ming Zhang
0 siblings, 2 replies; 13+ messages in thread
From: BERTRAND Joël @ 2007-10-18 16:33 UTC (permalink / raw)
To: blackmagic02881; +Cc: iscsitarget-devel, linux-raid
Ming Zhang wrote:
> On Thu, 2007-10-18 at 11:33 -0400, Ross S. W. Walker wrote:
>> BERTRAND Joël wrote:
>>> BERTRAND Joël wrote:
>>>> BERTRAND Joël wrote:
>>>>> Hello,
>>>>>
>>>>> When I try to create a raid1 volume over iscsi, process
>>> aborts with :
>>>>> - on target side:
>>>>>
>>>>> iscsi_trgt: cmnd_abort(1156) 29 1 0 42 57344 0 0
>>>>> iscsi_trgt: Abort Task (01) issued on tid:1 lun:0 by
>>> sid:630024457682948
>>>>> (Unknown Task)
>>>> Next run:
>>>> iscsi_trgt: cmnd_abort(1156) 13 1 0 42 57344 0 0
>>>> iscsi_trgt: Abort Task (01) issued on tid:1 lun:0 by
>>> sid:630058817421315
>>>> (Unknown Task)
>>>>
>>>> You can see that both lines are very similar. I shall
>>> try to use
>>>> blockio instead fileio.
>>> With blockio, I got the following message...
>>>
>>> iscsi_trgt: cmnd_abort(1156) c 1 0 42 8192 0 0
>>> iscsi_trgt: Abort Task (01) issued on tid:1 lun:0 by
>>> sid:630024457682946
>>> (Unknown Task)
>>>
>>> Command is the same. What is the signification of 1156 ?
>> Both outputs are from the same Abort Task management function
>> the 1156 refers to the line in iscsi.c where the debug printf
>> was issued.
>>
>> The other is the more verbose informative message that says
>> an Abort Task command was issued, but the task was not found.
>
> pure guess, this might because the sparc64 you are using.
>
> could you export a NULLIO target and do some intensive io tests? sort
> out these platform issues first...
I can format serveral times (mkfs.ext3) a 1.5 TB volume over iSCSI
without any trouble. I can read and write on this virtual disk without
any trouble.
Now, I have configured ietd with :
Lun 0 Sectors=1464725758,Type=nullio
and I run on initiator side :
Root gershwin:[/dev] > dd if=/dev/zero of=/dev/sdj bs=8192
479482+0 records in
479482+0 records out
3927916544 bytes (3.9 GB) copied, 153.222 seconds, 25.6 MB/s
Root gershwin:[/dev] > dd if=/dev/zero of=/dev/sdj bs=8192
I'm waitinfor a crash. No one when I write these lines. I suspect an
interaction between raid and iscsi.
Regards,
JKB
-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems? Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
Iscsitarget-devel mailing list
Iscsitarget-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/iscsitarget-devel
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Abort Task ?
2007-10-18 16:33 ` Abort Task ? BERTRAND Joël
@ 2007-10-18 17:14 ` BERTRAND Joël
2007-10-18 21:03 ` Ross S. W. Walker
2007-10-18 17:30 ` Ming Zhang
1 sibling, 1 reply; 13+ messages in thread
From: BERTRAND Joël @ 2007-10-18 17:14 UTC (permalink / raw)
To: blackmagic02881; +Cc: iscsitarget-devel, linux-raid
BERTRAND Joël wrote:
> I can format serveral times (mkfs.ext3) a 1.5 TB volume over iSCSI
> without any trouble. I can read and write on this virtual disk without
> any trouble.
>
> Now, I have configured ietd with :
>
> Lun 0 Sectors=1464725758,Type=nullio
>
> and I run on initiator side :
>
> Root gershwin:[/dev] > dd if=/dev/zero of=/dev/sdj bs=8192
> 479482+0 records in
> 479482+0 records out
> 3927916544 bytes (3.9 GB) copied, 153.222 seconds, 25.6 MB/s
>
> Root gershwin:[/dev] > dd if=/dev/zero of=/dev/sdj bs=8192
>
> I'm waitinfor a crash. No one when I write these lines. I suspect
> an interaction between raid and iscsi.
I simultanely run :
Root gershwin:[/dev] > dd if=/dev/zero of=/dev/sdj bs=8192
8397210+0 records in
8397210+0 records out
68789944320 bytes (69 GB) copied, 2732.55 seconds, 25.2 MB/s
and
Root gershwin:[~] > dd if=/dev/sdj of=/dev/null bs=8192
739200+0 records in
739199+0 records out
6055518208 bytes (6.1 GB) copied, 447.178 seconds, 13.5 MB/s
without any trouble.
Regards,
JKB
-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems? Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
Iscsitarget-devel mailing list
Iscsitarget-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/iscsitarget-devel
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [Iscsitarget-devel] Abort Task ?
2007-10-18 16:33 ` Abort Task ? BERTRAND Joël
2007-10-18 17:14 ` BERTRAND Joël
@ 2007-10-18 17:30 ` Ming Zhang
1 sibling, 0 replies; 13+ messages in thread
From: Ming Zhang @ 2007-10-18 17:30 UTC (permalink / raw)
To: BERTRAND Joël; +Cc: Ross S. W. Walker, iscsitarget-devel, linux-raid
On Thu, 2007-10-18 at 18:33 +0200, BERTRAND Joël wrote:
> Ming Zhang wrote:
> > On Thu, 2007-10-18 at 11:33 -0400, Ross S. W. Walker wrote:
> >> BERTRAND Joël wrote:
> >>> BERTRAND Joël wrote:
> >>>> BERTRAND Joël wrote:
> >>>>> Hello,
> >>>>>
> >>>>> When I try to create a raid1 volume over iscsi, process
> >>> aborts with :
> >>>>> - on target side:
> >>>>>
> >>>>> iscsi_trgt: cmnd_abort(1156) 29 1 0 42 57344 0 0
> >>>>> iscsi_trgt: Abort Task (01) issued on tid:1 lun:0 by
> >>> sid:630024457682948
> >>>>> (Unknown Task)
> >>>> Next run:
> >>>> iscsi_trgt: cmnd_abort(1156) 13 1 0 42 57344 0 0
> >>>> iscsi_trgt: Abort Task (01) issued on tid:1 lun:0 by
> >>> sid:630058817421315
> >>>> (Unknown Task)
> >>>>
> >>>> You can see that both lines are very similar. I shall
> >>> try to use
> >>>> blockio instead fileio.
> >>> With blockio, I got the following message...
> >>>
> >>> iscsi_trgt: cmnd_abort(1156) c 1 0 42 8192 0 0
> >>> iscsi_trgt: Abort Task (01) issued on tid:1 lun:0 by
> >>> sid:630024457682946
> >>> (Unknown Task)
> >>>
> >>> Command is the same. What is the signification of 1156 ?
> >> Both outputs are from the same Abort Task management function
> >> the 1156 refers to the line in iscsi.c where the debug printf
> >> was issued.
> >>
> >> The other is the more verbose informative message that says
> >> an Abort Task command was issued, but the task was not found.
> >
> > pure guess, this might because the sparc64 you are using.
> >
> > could you export a NULLIO target and do some intensive io tests? sort
> > out these platform issues first...
>
> I can format serveral times (mkfs.ext3) a 1.5 TB volume over iSCSI
> without any trouble. I can read and write on this virtual disk without
> any trouble.
>
> Now, I have configured ietd with :
>
> Lun 0 Sectors=1464725758,Type=nullio
>
> and I run on initiator side :
>
> Root gershwin:[/dev] > dd if=/dev/zero of=/dev/sdj bs=8192
> 479482+0 records in
> 479482+0 records out
> 3927916544 bytes (3.9 GB) copied, 153.222 seconds, 25.6 MB/s
25MB/s over a NULLIO is a very slow result...
>
> Root gershwin:[/dev] > dd if=/dev/zero of=/dev/sdj bs=8192
>
> I'm waitinfor a crash. No one when I write these lines. I suspect an
> interaction between raid and iscsi.
>
> Regards,
>
> JKB
--
Ming Zhang
@#$%^ purging memory... (*!%
http://blackmagic02881.wordpress.com/
http://www.linkedin.com/in/blackmagic02881
--------------------------------------------
-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Abort Task ?
2007-10-18 17:14 ` BERTRAND Joël
@ 2007-10-18 21:03 ` Ross S. W. Walker
2007-10-18 21:35 ` [Iscsitarget-devel] " Ming Zhang
2007-10-19 7:48 ` BERTRAND Joël
0 siblings, 2 replies; 13+ messages in thread
From: Ross S. W. Walker @ 2007-10-18 21:03 UTC (permalink / raw)
To: BERTRAND Joël, blackmagic02881; +Cc: iscsitarget-devel, linux-raid
BERTRAND Joël wrote:
>
> BERTRAND Joël wrote:
> > I can format serveral times (mkfs.ext3) a 1.5 TB volume
> over iSCSI
> > without any trouble. I can read and write on this virtual
> disk without
> > any trouble.
> >
> > Now, I have configured ietd with :
> >
> > Lun 0 Sectors=1464725758,Type=nullio
> >
> > and I run on initiator side :
> >
> > Root gershwin:[/dev] > dd if=/dev/zero of=/dev/sdj bs=8192
> > 479482+0 records in
> > 479482+0 records out
> > 3927916544 bytes (3.9 GB) copied, 153.222 seconds, 25.6 MB/s
> >
> > Root gershwin:[/dev] > dd if=/dev/zero of=/dev/sdj bs=8192
> >
> > I'm waitinfor a crash. No one when I write these lines.
> I suspect
> > an interaction between raid and iscsi.
>
> I simultanely run :
>
> Root gershwin:[/dev] > dd if=/dev/zero of=/dev/sdj bs=8192
> 8397210+0 records in
> 8397210+0 records out
> 68789944320 bytes (69 GB) copied, 2732.55 seconds, 25.2 MB/s
>
> and
>
> Root gershwin:[~] > dd if=/dev/sdj of=/dev/null bs=8192
> 739200+0 records in
> 739199+0 records out
> 6055518208 bytes (6.1 GB) copied, 447.178 seconds, 13.5 MB/s
>
> without any trouble.
The speed can definitely be improved. Look at your network setup
and use ping to try and get the network latency to a minimum.
# ping -A -s 8192 172.16.24.140
....
--- 172.16.24.140 ping statistics ---
14058 packets transmitted, 14057 received, 0% packet loss, time 9988ms
rtt min/avg/max/mdev = 0.234/0.268/2.084/0.041 ms, ipg/ewma 0.710/0.260 ms
You want your avg ping time for 8192 byte payloads to be 300us or less.
1000/.268 = 3731 IOPS @ 8k = 30 MB/s
If you use apps that do overlapping asynchronous IO you can see better
numbers.
-Ross
______________________________________________________________________
This e-mail, and any attachments thereto, is intended only for use by
the addressee(s) named herein and may contain legally privileged
and/or confidential information. If you are not the intended recipient
of this e-mail, you are hereby notified that any dissemination,
distribution or copying of this e-mail, and any attachments thereto,
is strictly prohibited. If you have received this e-mail in error,
please immediately notify the sender and permanently delete the
original and any copy or printout thereof.
-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems? Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
^ permalink raw reply [flat|nested] 13+ messages in thread
* RE: [Iscsitarget-devel] Abort Task ?
2007-10-18 21:03 ` Ross S. W. Walker
@ 2007-10-18 21:35 ` Ming Zhang
2007-10-19 7:48 ` BERTRAND Joël
1 sibling, 0 replies; 13+ messages in thread
From: Ming Zhang @ 2007-10-18 21:35 UTC (permalink / raw)
To: Ross S. W. Walker; +Cc: BERTRAND Joël, iscsitarget-devel, linux-raid
On Thu, 2007-10-18 at 17:03 -0400, Ross S. W. Walker wrote:
> BERTRAND Joël wrote:
> >
> > BERTRAND Joël wrote:
> > > I can format serveral times (mkfs.ext3) a 1.5 TB volume
> > over iSCSI
> > > without any trouble. I can read and write on this virtual
> > disk without
> > > any trouble.
> > >
> > > Now, I have configured ietd with :
> > >
> > > Lun 0 Sectors=1464725758,Type=nullio
> > >
> > > and I run on initiator side :
> > >
> > > Root gershwin:[/dev] > dd if=/dev/zero of=/dev/sdj bs=8192
> > > 479482+0 records in
> > > 479482+0 records out
> > > 3927916544 bytes (3.9 GB) copied, 153.222 seconds, 25.6 MB/s
> > >
> > > Root gershwin:[/dev] > dd if=/dev/zero of=/dev/sdj bs=8192
> > >
> > > I'm waitinfor a crash. No one when I write these lines.
> > I suspect
> > > an interaction between raid and iscsi.
> >
> > I simultanely run :
> >
> > Root gershwin:[/dev] > dd if=/dev/zero of=/dev/sdj bs=8192
> > 8397210+0 records in
> > 8397210+0 records out
> > 68789944320 bytes (69 GB) copied, 2732.55 seconds, 25.2 MB/s
> >
> > and
> >
> > Root gershwin:[~] > dd if=/dev/sdj of=/dev/null bs=8192
> > 739200+0 records in
> > 739199+0 records out
> > 6055518208 bytes (6.1 GB) copied, 447.178 seconds, 13.5 MB/s
> >
> > without any trouble.
>
> The speed can definitely be improved. Look at your network setup
> and use ping to try and get the network latency to a minimum.
>
> # ping -A -s 8192 172.16.24.140
> ....
> --- 172.16.24.140 ping statistics ---
> 14058 packets transmitted, 14057 received, 0% packet loss, time 9988ms
> rtt min/avg/max/mdev = 0.234/0.268/2.084/0.041 ms, ipg/ewma 0.710/0.260 ms
>
> You want your avg ping time for 8192 byte payloads to be 300us or less.
>
> 1000/.268 = 3731 IOPS @ 8k = 30 MB/s
he use normal dd, which in turn use page cache and read ahead, so actual
request size at scsi layer can be much larger than 8k....
>
> If you use apps that do overlapping asynchronous IO you can see better
> numbers.
>
> -Ross
>
> ______________________________________________________________________
> This e-mail, and any attachments thereto, is intended only for use by
> the addressee(s) named herein and may contain legally privileged
> and/or confidential information. If you are not the intended recipient
> of this e-mail, you are hereby notified that any dissemination,
> distribution or copying of this e-mail, and any attachments thereto,
> is strictly prohibited. If you have received this e-mail in error,
> please immediately notify the sender and permanently delete the
> original and any copy or printout thereof.
>
--
Ming Zhang
@#$%^ purging memory... (*!%
http://blackmagic02881.wordpress.com/
http://www.linkedin.com/in/blackmagic02881
--------------------------------------------
-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Abort Task ?
2007-10-18 21:03 ` Ross S. W. Walker
2007-10-18 21:35 ` [Iscsitarget-devel] " Ming Zhang
@ 2007-10-19 7:48 ` BERTRAND Joël
2007-10-19 13:55 ` Ming Zhang
1 sibling, 1 reply; 13+ messages in thread
From: BERTRAND Joël @ 2007-10-19 7:48 UTC (permalink / raw)
To: Ross S. W. Walker; +Cc: iscsitarget-devel, linux-raid
Ross S. W. Walker wrote:
> BERTRAND Joël wrote:
>> BERTRAND Joël wrote:
>>> I can format serveral times (mkfs.ext3) a 1.5 TB volume
>> over iSCSI
>>> without any trouble. I can read and write on this virtual
>> disk without
>>> any trouble.
>>>
>>> Now, I have configured ietd with :
>>>
>>> Lun 0 Sectors=1464725758,Type=nullio
>>>
>>> and I run on initiator side :
>>>
>>> Root gershwin:[/dev] > dd if=/dev/zero of=/dev/sdj bs=8192
>>> 479482+0 records in
>>> 479482+0 records out
>>> 3927916544 bytes (3.9 GB) copied, 153.222 seconds, 25.6 MB/s
>>>
>>> Root gershwin:[/dev] > dd if=/dev/zero of=/dev/sdj bs=8192
>>>
>>> I'm waitinfor a crash. No one when I write these lines.
>> I suspect
>>> an interaction between raid and iscsi.
>> I simultanely run :
>>
>> Root gershwin:[/dev] > dd if=/dev/zero of=/dev/sdj bs=8192
>> 8397210+0 records in
>> 8397210+0 records out
>> 68789944320 bytes (69 GB) copied, 2732.55 seconds, 25.2 MB/s
>>
>> and
>>
>> Root gershwin:[~] > dd if=/dev/sdj of=/dev/null bs=8192
>> 739200+0 records in
>> 739199+0 records out
>> 6055518208 bytes (6.1 GB) copied, 447.178 seconds, 13.5 MB/s
>>
>> without any trouble.
>
> The speed can definitely be improved. Look at your network setup
> and use ping to try and get the network latency to a minimum.
>
> # ping -A -s 8192 172.16.24.140
> ....
> --- 172.16.24.140 ping statistics ---
> 14058 packets transmitted, 14057 received, 0% packet loss, time 9988ms
> rtt min/avg/max/mdev = 0.234/0.268/2.084/0.041 ms, ipg/ewma 0.710/0.260 ms
gershwin:[~] > ping -A -s 8192 192.168.0.2
PING 192.168.0.2 (192.168.0.2) 8192(8220) bytes of data.
8200 bytes from 192.168.0.2: icmp_seq=1 ttl=64 time=0.693 ms
8200 bytes from 192.168.0.2: icmp_seq=2 ttl=64 time=0.595 ms
8200 bytes from 192.168.0.2: icmp_seq=3 ttl=64 time=0.583 ms
8200 bytes from 192.168.0.2: icmp_seq=4 ttl=64 time=0.589 ms
8200 bytes from 192.168.0.2: icmp_seq=5 ttl=64 time=0.580 ms
8200 bytes from 192.168.0.2: icmp_seq=6 ttl=64 time=0.594 ms
8200 bytes from 192.168.0.2: icmp_seq=7 ttl=64 time=0.580 ms
8200 bytes from 192.168.0.2: icmp_seq=8 ttl=64 time=0.592 ms
8200 bytes from 192.168.0.2: icmp_seq=9 ttl=64 time=0.589 ms
8200 bytes from 192.168.0.2: icmp_seq=10 ttl=64 time=0.571 ms
8200 bytes from 192.168.0.2: icmp_seq=11 ttl=64 time=0.588 ms
8200 bytes from 192.168.0.2: icmp_seq=12 ttl=64 time=0.580 ms
8200 bytes from 192.168.0.2: icmp_seq=13 ttl=64 time=0.587 ms
--- 192.168.0.2 ping statistics ---
13 packets transmitted, 13 received, 0% packet loss, time 2400ms
rtt min/avg/max/mdev = 0.571/0.593/0.693/0.044 ms, ipg/ewma 200.022/0.607 ms
gershwin:[~] >
Both initiator and target are alone on a gigabit NIC (Tigon3). On
target server, istd1 takes 100% of a CPU (and only one CPU, even my
T1000 can simultaneous run 32 threads). I think the limitation comes
from istd1.
> You want your avg ping time for 8192 byte payloads to be 300us or less.
>
> 1000/.268 = 3731 IOPS @ 8k = 30 MB/s
>
> If you use apps that do overlapping asynchronous IO you can see better
> numbers.
Regards,
JKB
-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems? Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Abort Task ?
@ 2007-10-19 12:25 Ross S. W. Walker
0 siblings, 0 replies; 13+ messages in thread
From: Ross S. W. Walker @ 2007-10-19 12:25 UTC (permalink / raw)
To: joel.bertrand; +Cc: iscsitarget-devel, linux-raid
[-- Attachment #1.1: Type: text/plain, Size: 4576 bytes --]
Bertrand,
If you are using dd then there will only be 1 outstanding io at a time so only istd1 will have work since the initiator will not send the next io till istd1 returns the results from the first io and then the next io will be issued to istd1 again.
600us is a little slow, try disabling or limiting any interrupt coalesence on the network adapters. ISCSI is more sensitive to network latency then NFS or CIFS.
-Ross
-----Original Message-----
From: BERTRAND Joël <joel.bertrand@systella.fr>
To: Ross S. W. Walker
CC: blackmagic02881@gmail.com <blackmagic02881@gmail.com>; iscsitarget-devel@lists.sourceforge.net <iscsitarget-devel@lists.sourceforge.net>; linux-raid@vger.kernel.org <linux-raid@vger.kernel.org>
Sent: Fri Oct 19 03:48:07 2007
Subject: Re: [Iscsitarget-devel] Abort Task ?
Ross S. W. Walker wrote:
> BERTRAND Joël wrote:
>> BERTRAND Joël wrote:
>>> I can format serveral times (mkfs.ext3) a 1.5 TB volume
>> over iSCSI
>>> without any trouble. I can read and write on this virtual
>> disk without
>>> any trouble.
>>>
>>> Now, I have configured ietd with :
>>>
>>> Lun 0 Sectors=1464725758,Type=nullio
>>>
>>> and I run on initiator side :
>>>
>>> Root gershwin:[/dev] > dd if=/dev/zero of=/dev/sdj bs=8192
>>> 479482+0 records in
>>> 479482+0 records out
>>> 3927916544 bytes (3.9 GB) copied, 153.222 seconds, 25.6 MB/s
>>>
>>> Root gershwin:[/dev] > dd if=/dev/zero of=/dev/sdj bs=8192
>>>
>>> I'm waitinfor a crash. No one when I write these lines.
>> I suspect
>>> an interaction between raid and iscsi.
>> I simultanely run :
>>
>> Root gershwin:[/dev] > dd if=/dev/zero of=/dev/sdj bs=8192
>> 8397210+0 records in
>> 8397210+0 records out
>> 68789944320 bytes (69 GB) copied, 2732.55 seconds, 25.2 MB/s
>>
>> and
>>
>> Root gershwin:[~] > dd if=/dev/sdj of=/dev/null bs=8192
>> 739200+0 records in
>> 739199+0 records out
>> 6055518208 bytes (6.1 GB) copied, 447.178 seconds, 13.5 MB/s
>>
>> without any trouble.
>
> The speed can definitely be improved. Look at your network setup
> and use ping to try and get the network latency to a minimum.
>
> # ping -A -s 8192 172.16.24.140
> ....
> --- 172.16.24.140 ping statistics ---
> 14058 packets transmitted, 14057 received, 0% packet loss, time 9988ms
> rtt min/avg/max/mdev = 0.234/0.268/2.084/0.041 ms, ipg/ewma 0.710/0.260 ms
gershwin:[~] > ping -A -s 8192 192.168.0.2
PING 192.168.0.2 (192.168.0.2) 8192(8220) bytes of data.
8200 bytes from 192.168.0.2: icmp_seq=1 ttl=64 time=0.693 ms
8200 bytes from 192.168.0.2: icmp_seq=2 ttl=64 time=0.595 ms
8200 bytes from 192.168.0.2: icmp_seq=3 ttl=64 time=0.583 ms
8200 bytes from 192.168.0.2: icmp_seq=4 ttl=64 time=0.589 ms
8200 bytes from 192.168.0.2: icmp_seq=5 ttl=64 time=0.580 ms
8200 bytes from 192.168.0.2: icmp_seq=6 ttl=64 time=0.594 ms
8200 bytes from 192.168.0.2: icmp_seq=7 ttl=64 time=0.580 ms
8200 bytes from 192.168.0.2: icmp_seq=8 ttl=64 time=0.592 ms
8200 bytes from 192.168.0.2: icmp_seq=9 ttl=64 time=0.589 ms
8200 bytes from 192.168.0.2: icmp_seq=10 ttl=64 time=0.571 ms
8200 bytes from 192.168.0.2: icmp_seq=11 ttl=64 time=0.588 ms
8200 bytes from 192.168.0.2: icmp_seq=12 ttl=64 time=0.580 ms
8200 bytes from 192.168.0.2: icmp_seq=13 ttl=64 time=0.587 ms
--- 192.168.0.2 ping statistics ---
13 packets transmitted, 13 received, 0% packet loss, time 2400ms
rtt min/avg/max/mdev = 0.571/0.593/0.693/0.044 ms, ipg/ewma 200.022/0.607 ms
gershwin:[~] >
Both initiator and target are alone on a gigabit NIC (Tigon3). On
target server, istd1 takes 100% of a CPU (and only one CPU, even my
T1000 can simultaneous run 32 threads). I think the limitation comes
from istd1.
> You want your avg ping time for 8192 byte payloads to be 300us or less.
>
> 1000/.268 = 3731 IOPS @ 8k = 30 MB/s
>
> If you use apps that do overlapping asynchronous IO you can see better
> numbers.
Regards,
JKB
______________________________________________________________________
This e-mail, and any attachments thereto, is intended only for use by
the addressee(s) named herein and may contain legally privileged
and/or confidential information. If you are not the intended recipient
of this e-mail, you are hereby notified that any dissemination,
distribution or copying of this e-mail, and any attachments thereto,
is strictly prohibited. If you have received this e-mail in error,
please immediately notify the sender and permanently delete the
original and any copy or printout thereof.
[-- Attachment #1.2: Type: text/html, Size: 5929 bytes --]
[-- Attachment #2: Type: text/plain, Size: 314 bytes --]
-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems? Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
[-- Attachment #3: Type: text/plain, Size: 182 bytes --]
_______________________________________________
Iscsitarget-devel mailing list
Iscsitarget-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/iscsitarget-devel
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Abort Task ?
2007-10-19 7:48 ` BERTRAND Joël
@ 2007-10-19 13:55 ` Ming Zhang
2007-10-19 14:30 ` BERTRAND Joël
0 siblings, 1 reply; 13+ messages in thread
From: Ming Zhang @ 2007-10-19 13:55 UTC (permalink / raw)
To: BERTRAND Joël; +Cc: iscsitarget-devel, linux-raid
On Fri, 2007-10-19 at 09:48 +0200, BERTRAND Joël wrote:
> Ross S. W. Walker wrote:
> > BERTRAND Joël wrote:
> >> BERTRAND Joël wrote:
> >>> I can format serveral times (mkfs.ext3) a 1.5 TB volume
> >> over iSCSI
> >>> without any trouble. I can read and write on this virtual
> >> disk without
> >>> any trouble.
> >>>
> >>> Now, I have configured ietd with :
> >>>
> >>> Lun 0 Sectors=1464725758,Type=nullio
> >>>
> >>> and I run on initiator side :
> >>>
> >>> Root gershwin:[/dev] > dd if=/dev/zero of=/dev/sdj bs=8192
> >>> 479482+0 records in
> >>> 479482+0 records out
> >>> 3927916544 bytes (3.9 GB) copied, 153.222 seconds, 25.6 MB/s
> >>>
> >>> Root gershwin:[/dev] > dd if=/dev/zero of=/dev/sdj bs=8192
> >>>
> >>> I'm waitinfor a crash. No one when I write these lines.
> >> I suspect
> >>> an interaction between raid and iscsi.
> >> I simultanely run :
> >>
> >> Root gershwin:[/dev] > dd if=/dev/zero of=/dev/sdj bs=8192
> >> 8397210+0 records in
> >> 8397210+0 records out
> >> 68789944320 bytes (69 GB) copied, 2732.55 seconds, 25.2 MB/s
> >>
> >> and
> >>
> >> Root gershwin:[~] > dd if=/dev/sdj of=/dev/null bs=8192
> >> 739200+0 records in
> >> 739199+0 records out
> >> 6055518208 bytes (6.1 GB) copied, 447.178 seconds, 13.5 MB/s
> >>
> >> without any trouble.
> >
> > The speed can definitely be improved. Look at your network setup
> > and use ping to try and get the network latency to a minimum.
> >
> > # ping -A -s 8192 172.16.24.140
> > ....
> > --- 172.16.24.140 ping statistics ---
> > 14058 packets transmitted, 14057 received, 0% packet loss, time 9988ms
> > rtt min/avg/max/mdev = 0.234/0.268/2.084/0.041 ms, ipg/ewma 0.710/0.260 ms
>
> gershwin:[~] > ping -A -s 8192 192.168.0.2
> PING 192.168.0.2 (192.168.0.2) 8192(8220) bytes of data.
> 8200 bytes from 192.168.0.2: icmp_seq=1 ttl=64 time=0.693 ms
> 8200 bytes from 192.168.0.2: icmp_seq=2 ttl=64 time=0.595 ms
> 8200 bytes from 192.168.0.2: icmp_seq=3 ttl=64 time=0.583 ms
> 8200 bytes from 192.168.0.2: icmp_seq=4 ttl=64 time=0.589 ms
> 8200 bytes from 192.168.0.2: icmp_seq=5 ttl=64 time=0.580 ms
> 8200 bytes from 192.168.0.2: icmp_seq=6 ttl=64 time=0.594 ms
> 8200 bytes from 192.168.0.2: icmp_seq=7 ttl=64 time=0.580 ms
> 8200 bytes from 192.168.0.2: icmp_seq=8 ttl=64 time=0.592 ms
> 8200 bytes from 192.168.0.2: icmp_seq=9 ttl=64 time=0.589 ms
> 8200 bytes from 192.168.0.2: icmp_seq=10 ttl=64 time=0.571 ms
> 8200 bytes from 192.168.0.2: icmp_seq=11 ttl=64 time=0.588 ms
> 8200 bytes from 192.168.0.2: icmp_seq=12 ttl=64 time=0.580 ms
> 8200 bytes from 192.168.0.2: icmp_seq=13 ttl=64 time=0.587 ms
>
> --- 192.168.0.2 ping statistics ---
> 13 packets transmitted, 13 received, 0% packet loss, time 2400ms
> rtt min/avg/max/mdev = 0.571/0.593/0.693/0.044 ms, ipg/ewma 200.022/0.607 ms
> gershwin:[~] >
>
> Both initiator and target are alone on a gigabit NIC (Tigon3). On
> target server, istd1 takes 100% of a CPU (and only one CPU, even my
> T1000 can simultaneous run 32 threads). I think the limitation comes
> from istd1.
usually istdx will not take 100% cpu with 1G network, especially when
using disk as back storage, some kind of profiling work might be helpful
to tell what happened...
forgot to ask, your sparc64 platform cpu spec.
>
> > You want your avg ping time for 8192 byte payloads to be 300us or less.
> >
> > 1000/.268 = 3731 IOPS @ 8k = 30 MB/s
> >
> > If you use apps that do overlapping asynchronous IO you can see better
> > numbers.
>
> Regards,
>
> JKB
--
Ming Zhang
@#$%^ purging memory... (*!%
http://blackmagic02881.wordpress.com/
http://www.linkedin.com/in/blackmagic02881
--------------------------------------------
-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems? Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
Iscsitarget-devel mailing list
Iscsitarget-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/iscsitarget-devel
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Abort Task ?
2007-10-19 13:55 ` Ming Zhang
@ 2007-10-19 14:30 ` BERTRAND Joël
2007-10-19 14:37 ` Ming Zhang
0 siblings, 1 reply; 13+ messages in thread
From: BERTRAND Joël @ 2007-10-19 14:30 UTC (permalink / raw)
To: blackmagic02881; +Cc: iscsitarget-devel, linux-raid
Ming Zhang wrote:
> On Fri, 2007-10-19 at 09:48 +0200, BERTRAND Joël wrote:
>> Ross S. W. Walker wrote:
>>> BERTRAND Joël wrote:
>>>> BERTRAND Joël wrote:
>>>>> I can format serveral times (mkfs.ext3) a 1.5 TB volume
>>>> over iSCSI
>>>>> without any trouble. I can read and write on this virtual
>>>> disk without
>>>>> any trouble.
>>>>>
>>>>> Now, I have configured ietd with :
>>>>>
>>>>> Lun 0 Sectors=1464725758,Type=nullio
>>>>>
>>>>> and I run on initiator side :
>>>>>
>>>>> Root gershwin:[/dev] > dd if=/dev/zero of=/dev/sdj bs=8192
>>>>> 479482+0 records in
>>>>> 479482+0 records out
>>>>> 3927916544 bytes (3.9 GB) copied, 153.222 seconds, 25.6 MB/s
>>>>>
>>>>> Root gershwin:[/dev] > dd if=/dev/zero of=/dev/sdj bs=8192
>>>>>
>>>>> I'm waitinfor a crash. No one when I write these lines.
>>>> I suspect
>>>>> an interaction between raid and iscsi.
>>>> I simultanely run :
>>>>
>>>> Root gershwin:[/dev] > dd if=/dev/zero of=/dev/sdj bs=8192
>>>> 8397210+0 records in
>>>> 8397210+0 records out
>>>> 68789944320 bytes (69 GB) copied, 2732.55 seconds, 25.2 MB/s
>>>>
>>>> and
>>>>
>>>> Root gershwin:[~] > dd if=/dev/sdj of=/dev/null bs=8192
>>>> 739200+0 records in
>>>> 739199+0 records out
>>>> 6055518208 bytes (6.1 GB) copied, 447.178 seconds, 13.5 MB/s
>>>>
>>>> without any trouble.
>>> The speed can definitely be improved. Look at your network setup
>>> and use ping to try and get the network latency to a minimum.
>>>
>>> # ping -A -s 8192 172.16.24.140
>>> ....
>>> --- 172.16.24.140 ping statistics ---
>>> 14058 packets transmitted, 14057 received, 0% packet loss, time 9988ms
>>> rtt min/avg/max/mdev = 0.234/0.268/2.084/0.041 ms, ipg/ewma 0.710/0.260 ms
>> gershwin:[~] > ping -A -s 8192 192.168.0.2
>> PING 192.168.0.2 (192.168.0.2) 8192(8220) bytes of data.
>> 8200 bytes from 192.168.0.2: icmp_seq=1 ttl=64 time=0.693 ms
>> 8200 bytes from 192.168.0.2: icmp_seq=2 ttl=64 time=0.595 ms
>> 8200 bytes from 192.168.0.2: icmp_seq=3 ttl=64 time=0.583 ms
>> 8200 bytes from 192.168.0.2: icmp_seq=4 ttl=64 time=0.589 ms
>> 8200 bytes from 192.168.0.2: icmp_seq=5 ttl=64 time=0.580 ms
>> 8200 bytes from 192.168.0.2: icmp_seq=6 ttl=64 time=0.594 ms
>> 8200 bytes from 192.168.0.2: icmp_seq=7 ttl=64 time=0.580 ms
>> 8200 bytes from 192.168.0.2: icmp_seq=8 ttl=64 time=0.592 ms
>> 8200 bytes from 192.168.0.2: icmp_seq=9 ttl=64 time=0.589 ms
>> 8200 bytes from 192.168.0.2: icmp_seq=10 ttl=64 time=0.571 ms
>> 8200 bytes from 192.168.0.2: icmp_seq=11 ttl=64 time=0.588 ms
>> 8200 bytes from 192.168.0.2: icmp_seq=12 ttl=64 time=0.580 ms
>> 8200 bytes from 192.168.0.2: icmp_seq=13 ttl=64 time=0.587 ms
>>
>> --- 192.168.0.2 ping statistics ---
>> 13 packets transmitted, 13 received, 0% packet loss, time 2400ms
>> rtt min/avg/max/mdev = 0.571/0.593/0.693/0.044 ms, ipg/ewma 200.022/0.607 ms
>> gershwin:[~] >
>>
>> Both initiator and target are alone on a gigabit NIC (Tigon3). On
>> target server, istd1 takes 100% of a CPU (and only one CPU, even my
>> T1000 can simultaneous run 32 threads). I think the limitation comes
>> from istd1.
>
> usually istdx will not take 100% cpu with 1G network, especially when
> using disk as back storage, some kind of profiling work might be helpful
> to tell what happened...
>
> forgot to ask, your sparc64 platform cpu spec.
Root gershwin:[/mnt/solaris] > cat /proc/cpuinfo
cpu : UltraSparc T1 (Niagara)
fpu : UltraSparc T1 integrated FPU
prom : OBP 4.23.4 2006/08/04 20:45
type : sun4v
ncpus probed : 24
ncpus active : 24
D$ parity tl1 : 0
I$ parity tl1 : 0
Both servers are built with 1 GHz T1 processors (6 cores, 24 threads).
Regards,
JKB
-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems? Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
Iscsitarget-devel mailing list
Iscsitarget-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/iscsitarget-devel
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Abort Task ?
2007-10-19 14:30 ` BERTRAND Joël
@ 2007-10-19 14:37 ` Ming Zhang
2007-10-19 14:47 ` BERTRAND Joël
2007-10-19 14:51 ` [Iscsitarget-devel] " Ross S. W. Walker
0 siblings, 2 replies; 13+ messages in thread
From: Ming Zhang @ 2007-10-19 14:37 UTC (permalink / raw)
To: BERTRAND Joël; +Cc: iscsitarget-devel, linux-raid
On Fri, 2007-10-19 at 16:30 +0200, BERTRAND Joël wrote:
> Ming Zhang wrote:
> > On Fri, 2007-10-19 at 09:48 +0200, BERTRAND Joël wrote:
> >> Ross S. W. Walker wrote:
> >>> BERTRAND Joël wrote:
> >>>> BERTRAND Joël wrote:
> >>>>> I can format serveral times (mkfs.ext3) a 1.5 TB volume
> >>>> over iSCSI
> >>>>> without any trouble. I can read and write on this virtual
> >>>> disk without
> >>>>> any trouble.
> >>>>>
> >>>>> Now, I have configured ietd with :
> >>>>>
> >>>>> Lun 0 Sectors=1464725758,Type=nullio
> >>>>>
> >>>>> and I run on initiator side :
> >>>>>
> >>>>> Root gershwin:[/dev] > dd if=/dev/zero of=/dev/sdj bs=8192
> >>>>> 479482+0 records in
> >>>>> 479482+0 records out
> >>>>> 3927916544 bytes (3.9 GB) copied, 153.222 seconds, 25.6 MB/s
> >>>>>
> >>>>> Root gershwin:[/dev] > dd if=/dev/zero of=/dev/sdj bs=8192
> >>>>>
> >>>>> I'm waitinfor a crash. No one when I write these lines.
> >>>> I suspect
> >>>>> an interaction between raid and iscsi.
> >>>> I simultanely run :
> >>>>
> >>>> Root gershwin:[/dev] > dd if=/dev/zero of=/dev/sdj bs=8192
> >>>> 8397210+0 records in
> >>>> 8397210+0 records out
> >>>> 68789944320 bytes (69 GB) copied, 2732.55 seconds, 25.2 MB/s
> >>>>
> >>>> and
> >>>>
> >>>> Root gershwin:[~] > dd if=/dev/sdj of=/dev/null bs=8192
> >>>> 739200+0 records in
> >>>> 739199+0 records out
> >>>> 6055518208 bytes (6.1 GB) copied, 447.178 seconds, 13.5 MB/s
> >>>>
> >>>> without any trouble.
> >>> The speed can definitely be improved. Look at your network setup
> >>> and use ping to try and get the network latency to a minimum.
> >>>
> >>> # ping -A -s 8192 172.16.24.140
> >>> ....
> >>> --- 172.16.24.140 ping statistics ---
> >>> 14058 packets transmitted, 14057 received, 0% packet loss, time 9988ms
> >>> rtt min/avg/max/mdev = 0.234/0.268/2.084/0.041 ms, ipg/ewma 0.710/0.260 ms
> >> gershwin:[~] > ping -A -s 8192 192.168.0.2
> >> PING 192.168.0.2 (192.168.0.2) 8192(8220) bytes of data.
> >> 8200 bytes from 192.168.0.2: icmp_seq=1 ttl=64 time=0.693 ms
> >> 8200 bytes from 192.168.0.2: icmp_seq=2 ttl=64 time=0.595 ms
> >> 8200 bytes from 192.168.0.2: icmp_seq=3 ttl=64 time=0.583 ms
> >> 8200 bytes from 192.168.0.2: icmp_seq=4 ttl=64 time=0.589 ms
> >> 8200 bytes from 192.168.0.2: icmp_seq=5 ttl=64 time=0.580 ms
> >> 8200 bytes from 192.168.0.2: icmp_seq=6 ttl=64 time=0.594 ms
> >> 8200 bytes from 192.168.0.2: icmp_seq=7 ttl=64 time=0.580 ms
> >> 8200 bytes from 192.168.0.2: icmp_seq=8 ttl=64 time=0.592 ms
> >> 8200 bytes from 192.168.0.2: icmp_seq=9 ttl=64 time=0.589 ms
> >> 8200 bytes from 192.168.0.2: icmp_seq=10 ttl=64 time=0.571 ms
> >> 8200 bytes from 192.168.0.2: icmp_seq=11 ttl=64 time=0.588 ms
> >> 8200 bytes from 192.168.0.2: icmp_seq=12 ttl=64 time=0.580 ms
> >> 8200 bytes from 192.168.0.2: icmp_seq=13 ttl=64 time=0.587 ms
> >>
> >> --- 192.168.0.2 ping statistics ---
> >> 13 packets transmitted, 13 received, 0% packet loss, time 2400ms
> >> rtt min/avg/max/mdev = 0.571/0.593/0.693/0.044 ms, ipg/ewma 200.022/0.607 ms
> >> gershwin:[~] >
> >>
> >> Both initiator and target are alone on a gigabit NIC (Tigon3). On
> >> target server, istd1 takes 100% of a CPU (and only one CPU, even my
> >> T1000 can simultaneous run 32 threads). I think the limitation comes
> >> from istd1.
> >
> > usually istdx will not take 100% cpu with 1G network, especially when
> > using disk as back storage, some kind of profiling work might be helpful
> > to tell what happened...
> >
> > forgot to ask, your sparc64 platform cpu spec.
>
> Root gershwin:[/mnt/solaris] > cat /proc/cpuinfo
> cpu : UltraSparc T1 (Niagara)
> fpu : UltraSparc T1 integrated FPU
> prom : OBP 4.23.4 2006/08/04 20:45
> type : sun4v
> ncpus probed : 24
> ncpus active : 24
> D$ parity tl1 : 0
> I$ parity tl1 : 0
>
> Both servers are built with 1 GHz T1 processors (6 cores, 24 threads).
>
as Ross pointed out, many io pattern only have 1 outstanding io at any
time, so there is only one work thread actively to serve it. so it can
not exploit the multiple core here.
you see 100% at nullio or fileio? with disk, most time should spend on
iowait and cpu utilization should not high at all.
> Regards,
>
> JKB
--
Ming Zhang
@#$%^ purging memory... (*!%
http://blackmagic02881.wordpress.com/
http://www.linkedin.com/in/blackmagic02881
--------------------------------------------
-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems? Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
Iscsitarget-devel mailing list
Iscsitarget-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/iscsitarget-devel
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Abort Task ?
2007-10-19 14:37 ` Ming Zhang
@ 2007-10-19 14:47 ` BERTRAND Joël
2007-10-19 14:53 ` Ming Zhang
2007-10-19 14:51 ` [Iscsitarget-devel] " Ross S. W. Walker
1 sibling, 1 reply; 13+ messages in thread
From: BERTRAND Joël @ 2007-10-19 14:47 UTC (permalink / raw)
To: blackmagic02881; +Cc: iscsitarget-devel, linux-raid
Ming Zhang wrote:
>
> as Ross pointed out, many io pattern only have 1 outstanding io at any
> time, so there is only one work thread actively to serve it. so it can
> not exploit the multiple core here.
>
>
> you see 100% at nullio or fileio? with disk, most time should spend on
> iowait and cpu utilization should not high at all.
With both nullio and fileio...
-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems? Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
^ permalink raw reply [flat|nested] 13+ messages in thread
* RE: [Iscsitarget-devel] Abort Task ?
2007-10-19 14:37 ` Ming Zhang
2007-10-19 14:47 ` BERTRAND Joël
@ 2007-10-19 14:51 ` Ross S. W. Walker
1 sibling, 0 replies; 13+ messages in thread
From: Ross S. W. Walker @ 2007-10-19 14:51 UTC (permalink / raw)
To: blackmagic02881, BERTRAND Joël; +Cc: iscsitarget-devel, linux-raid
Ming Zhang wrote:
>
> On Fri, 2007-10-19 at 16:30 +0200, BERTRAND Joël wrote:
> > Ming Zhang wrote:
> > > On Fri, 2007-10-19 at 09:48 +0200, BERTRAND Joël wrote:
> > >> Ross S. W. Walker wrote:
> > >>> BERTRAND Joël wrote:
> > >>>> BERTRAND Joël wrote:
> > >>>>> I can format serveral times (mkfs.ext3) a 1.5 TB volume
> > >>>> over iSCSI
> > >>>>> without any trouble. I can read and write on this virtual
> > >>>> disk without
> > >>>>> any trouble.
> > >>>>>
> > >>>>> Now, I have configured ietd with :
> > >>>>>
> > >>>>> Lun 0 Sectors=1464725758,Type=nullio
> > >>>>>
> > >>>>> and I run on initiator side :
> > >>>>>
> > >>>>> Root gershwin:[/dev] > dd if=/dev/zero of=/dev/sdj bs=8192
> > >>>>> 479482+0 records in
> > >>>>> 479482+0 records out
> > >>>>> 3927916544 bytes (3.9 GB) copied, 153.222 seconds, 25.6 MB/s
> > >>>>>
> > >>>>> Root gershwin:[/dev] > dd if=/dev/zero of=/dev/sdj bs=8192
> > >>>>>
> > >>>>> I'm waitinfor a crash. No one when I write these lines.
> > >>>> I suspect
> > >>>>> an interaction between raid and iscsi.
> > >>>> I simultanely run :
> > >>>>
> > >>>> Root gershwin:[/dev] > dd if=/dev/zero of=/dev/sdj bs=8192
> > >>>> 8397210+0 records in
> > >>>> 8397210+0 records out
> > >>>> 68789944320 bytes (69 GB) copied, 2732.55 seconds, 25.2 MB/s
> > >>>>
> > >>>> and
> > >>>>
> > >>>> Root gershwin:[~] > dd if=/dev/sdj of=/dev/null bs=8192
> > >>>> 739200+0 records in
> > >>>> 739199+0 records out
> > >>>> 6055518208 bytes (6.1 GB) copied, 447.178 seconds, 13.5 MB/s
> > >>>>
> > >>>> without any trouble.
> > >>> The speed can definitely be improved. Look at your network setup
> > >>> and use ping to try and get the network latency to a minimum.
> > >>>
> > >>> # ping -A -s 8192 172.16.24.140
> > >>> ....
> > >>> --- 172.16.24.140 ping statistics ---
> > >>> 14058 packets transmitted, 14057 received, 0% packet
> loss, time 9988ms
> > >>> rtt min/avg/max/mdev = 0.234/0.268/2.084/0.041 ms,
> ipg/ewma 0.710/0.260 ms
> > >> gershwin:[~] > ping -A -s 8192 192.168.0.2
> > >> PING 192.168.0.2 (192.168.0.2) 8192(8220) bytes of data.
> > >> 8200 bytes from 192.168.0.2: icmp_seq=1 ttl=64 time=0.693 ms
> > >> 8200 bytes from 192.168.0.2: icmp_seq=2 ttl=64 time=0.595 ms
> > >> 8200 bytes from 192.168.0.2: icmp_seq=3 ttl=64 time=0.583 ms
> > >> 8200 bytes from 192.168.0.2: icmp_seq=4 ttl=64 time=0.589 ms
> > >> 8200 bytes from 192.168.0.2: icmp_seq=5 ttl=64 time=0.580 ms
> > >> 8200 bytes from 192.168.0.2: icmp_seq=6 ttl=64 time=0.594 ms
> > >> 8200 bytes from 192.168.0.2: icmp_seq=7 ttl=64 time=0.580 ms
> > >> 8200 bytes from 192.168.0.2: icmp_seq=8 ttl=64 time=0.592 ms
> > >> 8200 bytes from 192.168.0.2: icmp_seq=9 ttl=64 time=0.589 ms
> > >> 8200 bytes from 192.168.0.2: icmp_seq=10 ttl=64 time=0.571 ms
> > >> 8200 bytes from 192.168.0.2: icmp_seq=11 ttl=64 time=0.588 ms
> > >> 8200 bytes from 192.168.0.2: icmp_seq=12 ttl=64 time=0.580 ms
> > >> 8200 bytes from 192.168.0.2: icmp_seq=13 ttl=64 time=0.587 ms
> > >>
> > >> --- 192.168.0.2 ping statistics ---
> > >> 13 packets transmitted, 13 received, 0% packet loss, time 2400ms
> > >> rtt min/avg/max/mdev = 0.571/0.593/0.693/0.044 ms,
> ipg/ewma 200.022/0.607 ms
> > >> gershwin:[~] >
> > >>
> > >> Both initiator and target are alone on a gigabit NIC
> (Tigon3). On
> > >> target server, istd1 takes 100% of a CPU (and only one
> CPU, even my
> > >> T1000 can simultaneous run 32 threads). I think the
> limitation comes
> > >> from istd1.
> > >
> > > usually istdx will not take 100% cpu with 1G network,
> especially when
> > > using disk as back storage, some kind of profiling work
> might be helpful
> > > to tell what happened...
> > >
> > > forgot to ask, your sparc64 platform cpu spec.
> >
> > Root gershwin:[/mnt/solaris] > cat /proc/cpuinfo
> > cpu : UltraSparc T1 (Niagara)
> > fpu : UltraSparc T1 integrated FPU
> > prom : OBP 4.23.4 2006/08/04 20:45
> > type : sun4v
> > ncpus probed : 24
> > ncpus active : 24
> > D$ parity tl1 : 0
> > I$ parity tl1 : 0
> >
> > Both servers are built with 1 GHz T1 processors (6
> cores, 24 threads).
> >
>
> as Ross pointed out, many io pattern only have 1 outstanding io at any
> time, so there is only one work thread actively to serve it. so it can
> not exploit the multiple core here.
>
>
> you see 100% at nullio or fileio? with disk, most time should spend on
> iowait and cpu utilization should not high at all.
Maybe it has to do with the endian-ness fix?
Look at where the fix was implemented and if there was a simpler way
of implementing it? (If that is the cause)
The network is still slower then expected, I don't know what chipset
the Sparcs use for their interfaces, if it is e1000 then you can set
low-latency interrupt throttling with InterruptThrottleRate=1 which
works well. You can explore other interface module options around
Interrupt throttling or coalesence.
-Ross
______________________________________________________________________
This e-mail, and any attachments thereto, is intended only for use by
the addressee(s) named herein and may contain legally privileged
and/or confidential information. If you are not the intended recipient
of this e-mail, you are hereby notified that any dissemination,
distribution or copying of this e-mail, and any attachments thereto,
is strictly prohibited. If you have received this e-mail in error,
please immediately notify the sender and permanently delete the
original and any copy or printout thereof.
-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Abort Task ?
2007-10-19 14:47 ` BERTRAND Joël
@ 2007-10-19 14:53 ` Ming Zhang
0 siblings, 0 replies; 13+ messages in thread
From: Ming Zhang @ 2007-10-19 14:53 UTC (permalink / raw)
To: BERTRAND Joël; +Cc: iscsitarget-devel, linux-raid
On Fri, 2007-10-19 at 16:47 +0200, BERTRAND Joël wrote:
> Ming Zhang wrote:
> >
> > as Ross pointed out, many io pattern only have 1 outstanding io at any
> > time, so there is only one work thread actively to serve it. so it can
> > not exploit the multiple core here.
> >
> >
> > you see 100% at nullio or fileio? with disk, most time should spend on
> > iowait and cpu utilization should not high at all.
>
> With both nullio and fileio...
it is weired. with file io, run some io load, then run "vmstat 1" and
post here. supposed to see some iowait instead of high sys cpu usage...
--
Ming Zhang
@#$%^ purging memory... (*!%
http://blackmagic02881.wordpress.com/
http://www.linkedin.com/in/blackmagic02881
--------------------------------------------
-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems? Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
Iscsitarget-devel mailing list
Iscsitarget-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/iscsitarget-devel
^ permalink raw reply [flat|nested] 13+ messages in thread
end of thread, other threads:[~2007-10-19 14:53 UTC | newest]
Thread overview: 13+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
[not found] <471720F1.40409@systella.fr>
[not found] ` <471727F3.2080500@systella.fr>
[not found] ` <471734C5.8090405@systella.fr>
[not found] ` <E2BB8074E5500C42984D980D4BD78EF9017A2958@MFG-NYC-EXCH2.mfg.prv>
[not found] ` <1192721909.3596.15.camel@dhcp-117.ibrix.com>
2007-10-18 16:33 ` Abort Task ? BERTRAND Joël
2007-10-18 17:14 ` BERTRAND Joël
2007-10-18 21:03 ` Ross S. W. Walker
2007-10-18 21:35 ` [Iscsitarget-devel] " Ming Zhang
2007-10-19 7:48 ` BERTRAND Joël
2007-10-19 13:55 ` Ming Zhang
2007-10-19 14:30 ` BERTRAND Joël
2007-10-19 14:37 ` Ming Zhang
2007-10-19 14:47 ` BERTRAND Joël
2007-10-19 14:53 ` Ming Zhang
2007-10-19 14:51 ` [Iscsitarget-devel] " Ross S. W. Walker
2007-10-18 17:30 ` Ming Zhang
2007-10-19 12:25 Ross S. W. Walker
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).