High read Latency test (Anticipatory I/O scheduler)

High read Latency test (Anticipatory I/O scheduler)

[John Chatelle]

   I haven't seen much duplicated results regarding the Robert Love article 
in the February 2004 Linux Journal article, also reachable in the hyperlink:
          http://www.linuxjournal.com/article.php?sid=6931

 Although the 1st simple test: "Write starved reads" gets results comparable
to the results reported in the Article, Our results for the 2nd test: "High 
Read latency" delivers results opposite our expectations...
 
Kernel 2.6.2 Results: (Anticipatory I/O scheduler).

real    43m41.138s       Nearly 44 minutes! 2nd run similar.
user    0m4.715s
sys     0m11.179s

Kernel 2.4.20-28.7p21gsmp Results:

real    0m41.535s        Only 42 seconds!   2nd run similar.
user    0m4.720s
sys     0m15.470s

Our dmesg shows wer're running the Anticipatory scheduler when testing under
the 2.6.2 kernel. 

The 2 shell scripts, StreamingRead.sh and WHR.sh (bash, actually), implement 
the test:

#StreamingRead.sh       --simple 4 line shell script:
   while true
   do
     cat ../data/oneGBfile >/dev/null
   done

#WHR.sh                -- simple 2 (or 3) line shell script.
   StreamingRead.sh &
   time find /usr/src/linux-2.4.20-18.7  -type f -exec cat \
     '{}' ';' > /dev/null

   I'm reading a 1G binary garbage file repeatedly while timing 
the transversal and reading of the 2.4 Kernel source tree, just as the test 
2 example shows in the article.  I would think I/O anticipating the 1G 
garbage file would be likely, where the I/O anticipation of the reading of 
the source tree under the 'find' command would be far choppier and more 
difficult.The 'time' command, however measures the less anticipatory and 
choppier reads of the 'find' command!  I therefore see the results of table 
1 for test 2 to be very counter intuitive!  

  Has anyone else seen such divergent results compared to those reported in 
the article?  Does anyone else see the same results with the anticipatory 
I/O scheduler?  


    

John Chatelle 
johnch@medent.com 
Community Computer Service 
15 Hulbert Street - P.O. Box 980 
Auburn, New York  13021 
Phone:  (315)-255-1751 
Fax:    (315)-255-3539


--
This message and any attachments may contain information that is protected
by law as privileged and confidential, and is transmitted for the sole use
of the intended recipient(s). If you are not the intended recipient, you
are hereby notified that any use, dissemination, copying or retention of
this e-mail or the information contained herein is strictly prohibited. If
you have received this e-mail in error, please immediately notify the
sender by e-mail, and permanently delete this e-mail.



-- 
This message has been scanned for viruses and
dangerous content by MailScanner.

Re: High read Latency test (Anticipatory I/O scheduler)

[Dave Olien]

I was able to repeat Robert Love's results on a machine with
a simple IDE controller, two-processer system, with 500 meg memory,
ext2 file system, mounted with the noatime option.  I think
Robert's article also indicated he used an IDE controller for disk.

The results can be influenced a lot by your disk/controller
configuration.  RAID controllers in particular, or disks with large
TCQ may not perform as well with anticipatory scheduler.

Also, there were some changes to the read-ahead policy that
came  between 2.6.2 and 2.6.3, I think.  That has been shown
to make a measureable difference on systems here.

Try again with 2.6.3.

Also, try setting the antic_expire scheduler parameter for
the queue for your disk, and see what results you get.
That should hopefully be close to 2.4's behavior.  If it isn't,
then there's probably something more going on than just the
IO scheduler.

On Fri, Feb 20, 2004 at 03:20:22PM -0500, John Chatelle wrote:
>    I haven't seen much duplicated results regarding the Robert Love article 
> in the February 2004 Linux Journal article, also reachable in the hyperlink:
>           http://www.linuxjournal.com/article.php?sid=6931
> 
>  Although the 1st simple test: "Write starved reads" gets results comparable
> to the results reported in the Article, Our results for the 2nd test: "High 
> Read latency" delivers results opposite our expectations...
>  
> Kernel 2.6.2 Results: (Anticipatory I/O scheduler).
> 
> real    43m41.138s       Nearly 44 minutes! 2nd run similar.
> user    0m4.715s
> sys     0m11.179s
> 
> Kernel 2.4.20-28.7p21gsmp Results:
> 
> real    0m41.535s        Only 42 seconds!   2nd run similar.
> user    0m4.720s
> sys     0m15.470s
> 
> Our dmesg shows wer're running the Anticipatory scheduler when testing under
> the 2.6.2 kernel. 
> 
> The 2 shell scripts, StreamingRead.sh and WHR.sh (bash, actually), implement 
> the test:
> 
> #StreamingRead.sh       --simple 4 line shell script:
>    while true
>    do
>      cat ../data/oneGBfile >/dev/null
>    done
> 
> #WHR.sh                -- simple 2 (or 3) line shell script.
>    StreamingRead.sh &
>    time find /usr/src/linux-2.4.20-18.7  -type f -exec cat \
>      '{}' ';' > /dev/null
> 
>    I'm reading a 1G binary garbage file repeatedly while timing 
> the transversal and reading of the 2.4 Kernel source tree, just as the test 
> 2 example shows in the article.  I would think I/O anticipating the 1G 
> garbage file would be likely, where the I/O anticipation of the reading of 
> the source tree under the 'find' command would be far choppier and more 
> difficult.The 'time' command, however measures the less anticipatory and 
> choppier reads of the 'find' command!  I therefore see the results of table 
> 1 for test 2 to be very counter intuitive!  
> 
>   Has anyone else seen such divergent results compared to those reported in 
> the article?  Does anyone else see the same results with the anticipatory 
> I/O scheduler?  
> 
> 
>     
> 
> John Chatelle 
> johnch@medent.com 
> Community Computer Service 
> 15 Hulbert Street - P.O. Box 980 
> Auburn, New York  13021 
> Phone:  (315)-255-1751 
> Fax:    (315)-255-3539
> 
> 
> --
> This message and any attachments may contain information that is protected
> by law as privileged and confidential, and is transmitted for the sole use
> of the intended recipient(s). If you are not the intended recipient, you
> are hereby notified that any use, dissemination, copying or retention of
> this e-mail or the information contained herein is strictly prohibited. If
> you have received this e-mail in error, please immediately notify the
> sender by e-mail, and permanently delete this e-mail.
> 
> 
> 
> -- 
> This message has been scanned for viruses and
> dangerous content by MailScanner.
> 
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

Re: High read Latency test (Anticipatory I/O scheduler)

[Nick Piggin]

[-- Attachment #1: Type: text/plain, Size: 671 bytes --]



John Chatelle wrote:

>   I haven't seen much duplicated results regarding the Robert Love article 
>in the February 2004 Linux Journal article, also reachable in the hyperlink:
>          http://www.linuxjournal.com/article.php?sid=6931
>
> Although the 1st simple test: "Write starved reads" gets results comparable
>to the results reported in the Article, Our results for the 2nd test: "High 
>Read latency" delivers results opposite our expectations...
>
>

Hi John,
Can you try the following patch please? If that doesn't help, can you
show me what /sys/block/hda/queue/iosched/est_time says after your
test has been running for a couple of minutes.

Thanks
Nick


[-- Attachment #2: as-exit-prob.patch --]
[-- Type: text/plain, Size: 676 bytes --]

 linux-2.6-npiggin/drivers/block/as-iosched.c |    2 ++
 1 files changed, 2 insertions(+)

diff -puN drivers/block/as-iosched.c~as-exit-prob drivers/block/as-iosched.c
--- linux-2.6/drivers/block/as-iosched.c~as-exit-prob	2004-02-21 09:38:54.000000000 +1100
+++ linux-2.6-npiggin/drivers/block/as-iosched.c	2004-02-21 09:39:22.000000000 +1100
@@ -734,8 +734,10 @@ static int as_can_break_anticipation(str
 	if (aic->ttime_samples == 0) {
 		if (ad->new_ttime_mean > ad->antic_expire)
 			return 1;
+#if 0
 		if (ad->exit_prob > 128)
 			return 1;
+#endif
 	} else if (aic->ttime_mean > ad->antic_expire) {
 		/* the process thinks too much between requests */
 		return 1;

_

Re: High read Latency test (Anticipatory I/O scheduler)

[Andrew Morton]

"John Chatelle" <johnch@medent.com> wrote:
>
> #StreamingRead.sh       --simple 4 line shell script:
>    while true
>    do
>      cat ../data/oneGBfile >/dev/null
>    done
> 
> #WHR.sh                -- simple 2 (or 3) line shell script.
>    StreamingRead.sh &
>    time find /usr/src/linux-2.4.20-18.7  -type f -exec cat \
>      '{}' ';' > /dev/null

Note that the latter test runs `cat' once per file: over ten thousand
times.  You should also test:

	time  (find /usr/src/linux-2.4.20-18.7 -type f | xargs cat > /dev/null )

which will run `cat' only some tens of times.

For the anticipatory scheduler, this makes a significant difference - the
run-cat-once-per-file problem has specific fixes and perhaps they're not
working right at present.

And yes, please describe your disk system, tell us the tag depth (if it is
scsi) and try Nick's patch, thanks.