From: "Antonio Larrosa Jiménez" <antlarr@tedial.com>
To: linux-kernel@vger.kernel.org
Subject: 2.6.5 io scheduler problem? (cfq/deadline/noop)
Date: Wed, 13 Jul 2005 16:55:16 +0200 [thread overview]
Message-ID: <200507131655.17185.antlarr@tedial.com> (raw)
Hello,
I have a problem when reading and writing at the same time to an array of scsi
hard disks (an HP Smart Array 30) connected to an ADAPTEC ASR-2230S RAID5
controller using the aacraid module.
I'm using a SLES 9 system (kernel 2.6.5) which enables the cfq scheduler by
default (I checked that it's really using cfq).
I really don't understand why, but when running a bash script that contains:
#!/bin/sh
sync
time dd if=test.read of=/dev/null bs=32k count=32k &
time dd if=/dev/zero of=test.write bs=32k count=32k
(test.read is a file generated just like test.write is generated)
I got this when running vmstat 1 :
0 0 12 9512 5820 916920 0 0 0 0 1021 169 0 0 100 0
0 1 12 9448 5820 917180 0 0 108 0 1043 176 0 0 100 0
1 3 12 9508 5800 912780 0 0 8828 122504 1587 1002 0 25 48 27
0 4 12 8464 5808 913032 0 0 128 51940 1212 323 0 4 48 49
0 4 12 10276 5820 910940 0 0 0 30892 1278 297 0 2 50 49
0 4 12 9828 5844 910656 0 0 0 40600 1232 317 0 2 50 48
0 4 12 9764 5812 910168 0 0 0 45244 1192 247 0 3 35 62
0 4 12 10788 5840 908580 0 0 0 45468 1213 267 0 2 45 52
0 4 12 12196 5440 907160 0 0 0 37160 1216 281 0 2 50 48
procs -----------memory---------- ---swap-- -----io---- --system-- ----cpu----
r b swpd free buff cache si so bi bo in cs us sy id wa
2 3 12 9204 5472 909728 0 0 0 38580 1217 275 0 2 48 50
0 4 12 10356 5500 907880 0 0 0 46748 1197 306 0 2 50 48
0 4 12 15580 5504 902156 0 0 0 42028 1286 330 0 3 50 48
0 4 12 8692 5544 908356 0 0 0 43048 1203 234 0 3 49 49
0 4 12 12348 5552 904448 0 0 32 38692 1216 305 0 2 50 48
0 4 12 9852 5592 907008 0 0 0 49200 1230 292 0 3 50 48
0 2 12 14052 5700 902480 0 0 0 31060 1288 340 0 1 63 37
0 2 12 14052 5700 902480 0 0 0 42408 1220 280 0 0 60 40
0 2 12 13912 5700 902740 0 0 128 43116 1242 262 0 0 63 36
0 2 12 13764 5700 902740 0 0 128 41832 1311 362 0 0 56 44
0 2 12 13764 5700 902740 0 0 0 41832 1266 268 0 0 61 39
0 2 12 13764 5720 902720 0 0 0 42700 1260 287 0 0 60 40
0 1 12 13764 5720 902720 0 0 0 40164 1267 311 0 1 56 44
0 1 12 9412 5720 907140 0 0 4352 0 1156 200 0 0 75 25
0 1 12 8460 5708 908712 0 0 42904 0 1374 889 0 1 75 24
0 1 12 9612 5680 907960 0 0 38676 0 1360 794 0 2 74 24
0 1 12 10508 5680 907440 0 0 43032 20 1503 945 0 2 75 24
0 1 12 10376 5680 907960 0 0 42020 0 1348 870 0 1 75 24
0 1 12 10328 5664 908496 0 0 37912 0 1334 763 0 1 75 24
0 1 12 8908 5652 910328 0 0 38936 0 1328 832 0 1 75 24
0 1 12 8972 5636 910864 0 0 41884 0 1364 836 0 2 75 24
0 1 12 9740 5636 910344 0 0 41116 172 1385 823 0 1 75 24
0 1 12 8844 5632 911908 0 0 40736 0 1356 854 0 1 75 24
0 1 12 8716 5624 912436 0 0 36892 2048 1444 821 0 1 75 24
0 1 12 10252 5596 911424 0 0 42912 2180 1433 896 0 1 75 24
1 1 12 10764 5572 911448 0 0 40216 2160 1359 785 0 1 75 24
0 1 12 8588 5564 914056 0 0 41752 20 1367 850 0 1 75 24
0 1 12 9548 5528 913572 0 0 42908 128 1390 1136 0 1 75 24
0 1 12 8908 5480 914660 0 0 34192 256 1305 725 0 1 75 24
0 1 12 8780 5444 915476 0 0 41640 128 1348 879 0 1 75 24
0 1 12 9548 5464 915196 0 0 43044 256 1466 919 0 1 75 24
0 1 12 10468 5512 914888 0 0 38812 276 1331 817 0 2 75 24
0 1 12 8460 5524 917476 0 0 41620 256 1368 832 0 1 75 24
0 1 12 9800 5548 916412 0 0 38296 196 1355 808 0 1 75 24
0 1 12 166392 5488 759952 0 0 37020 41820 1476 863 0 3 73 24
0 1 12 159796 5488 766712 0 0 6912 0 1286 286 0 0 75 25
0 1 12 118092 5524 808536 0 0 41744 20 1368 838 0 1 75 24
0 1 12 75084 5552 851668 0 0 43036 0 1360 791 0 1 75 24
0 1 12 33868 5560 892740 0 0 41224 0 1365 786 0 1 75 24
0 1 12 9484 5592 916888 0 0 38816 0 1449 875 0 1 75 24
0 0 12 10216 5604 916616 0 0 18124 0 1182 447 0 0 87 13
0 0 12 10216 5632 916588 0 0 0 28 1050 185 0 0 100 0
0 0 12 10240 5632 916588 0 0 0 0 1066 155 0 0 100 0
Then I changed my /boot/grub/menu.lst to include elevator=deadline, but
curiously, after restarting, I got:
> dmesg | grep sch
Using cfq io scheduler
Which is quite odd, since I specified deadline in the kernel parameters.
Anyway, I did a test and I got:
0 0 0 11100 19460 944360 0 0 0 0 1023 46 0 0 100 0
0 0 0 11100 19460 944360 0 0 0 0 1018 38 0 0 100 0
0 0 0 11100 19460 944360 0 0 0 0 1019 42 0 0 100 0
0 2 0 10108 19584 944756 0 0 2532 172 1098 195 0 0 90 10
0 2 0 9020 19792 945588 0 0 27708 36 1277 584 0 1 61 38
0 2 0 10940 19968 943592 0 0 28464 0 1389 629 0 1 64 35
0 2 0 9596 20084 945036 0 0 26160 0 1352 602 0 1 62 37
0 2 8 18108 20224 936048 0 0 26016 0 1287 550 0 1 56 43
0 2 8 9148 20376 944996 0 0 27560 0 1269 557 0 1 59 40
0 2 8 9852 20556 944036 0 0 26404 16 1261 558 0 1 61 38
0 2 8 9276 20732 944380 0 0 27184 0 1400 608 0 1 58 42
1 3 8 10684 20612 938520 0 0 11932 108320 1501 973 0 27 42 30
0 4 8 11132 20552 937800 0 0 256 55604 1324 230 0 3 62 35
1 3 8 12924 20492 936040 0 0 128 41436 1189 145 0 2 57 40
0 4 8 9852 20460 939452 0 0 0 38772 1189 161 0 2 50 48
0 4 8 15108 20332 934380 0 0 256 47072 1190 169 0 3 50 48
0 4 8 9084 20044 940388 0 0 128 38516 1230 186 0 2 50 48
1 3 8 8956 19616 940816 0 0 128 36900 1228 190 0 2 48 50
0 4 8 13436 19328 936684 0 0 0 47580 1223 196 0 2 50 48
0 4 8 10276 19300 939832 0 0 256 46008 1200 197 0 3 50 48
0 4 8 11452 19276 938296 0 0 128 38812 1191 153 0 2 39 59
0 4 8 12668 19108 937164 0 0 128 41080 1185 162 0 2 30 67
0 4 8 14320 18612 936360 0 0 128 42200 1190 178 0 3 48 49
0 4 8 8828 18396 942556 0 0 0 38696 1192 157 0 2 50 48
0 4 8 14012 18048 938224 0 0 0 38692 1218 187 0 3 40 58
0 2 8 13428 18068 938464 0 0 128 36556 1223 204 0 1 61 38
0 2 8 13364 18068 938724 0 0 128 42988 1222 178 0 0 63 36
0 2 8 13108 18068 938984 0 0 256 41832 1221 165 0 0 64 36
0 2 8 12980 18068 938984 0 0 128 42344 1250 187 0 0 57 43
0 2 8 12852 18068 939244 0 0 128 37476 1234 179 0 0 62 38
0 2 8 12724 18088 939224 0 0 116 42628 1226 167 0 0 74 25
0 1 8 12484 18088 939484 0 0 272 43176 1243 174 0 0 63 37
0 1 8 11332 18088 940784 0 0 1152 0 1126 66 0 0 75 25
0 1 8 8836 17744 944248 0 0 40616 0 1337 734 0 1 75 24
0 1 8 10244 17476 943996 0 0 42920 0 1355 762 0 1 75 24
0 1 8 9156 17204 946088 0 0 43052 16 1477 837 0 1 74 24
0 1 8 8452 16988 947864 0 0 40872 0 1432 788 0 1 75 24
0 1 8 8708 16620 948752 0 0 37924 0 1347 693 0 1 75 24
0 1 8 10052 16404 948708 0 0 43048 0 1356 778 0 1 75 24
0 1 8 9156 16164 950248 0 0 41000 0 1341 730 0 1 74 24
procs -----------memory---------- ---swap-- -----io---- --system-- ----cpu----
r b swpd free buff cache si so bi bo in cs us sy id wa
0 1 8 10180 15968 950192 0 8 42924 32 1490 826 0 2 75 24
0 1 8 9284 15596 952124 0 0 43048 0 1357 750 0 1 75 24
0 1 8 9284 15384 952856 0 0 35876 0 1389 676 0 1 75 24
0 1 8 8516 15156 954904 0 0 42924 0 1354 760 0 2 74 24
0 1 8 9732 14920 954360 0 0 43048 0 1354 762 0 1 75 24
0 1 8 9540 14708 955612 0 0 39976 148 1365 714 0 1 75 24
0 1 8 10948 14356 955184 0 0 42920 0 1387 781 0 2 74 24
0 1 8 8580 14136 958524 0 0 38952 1580 1378 707 0 1 75 24
0 1 8 9668 13912 958488 0 0 39976 2468 1392 730 0 1 74 24
0 1 8 8836 13696 960264 0 0 42920 2580 1381 740 0 1 75 24
0 1 8 9476 13484 960216 0 0 43052 1060 1368 766 0 1 75 24
0 1 8 9156 13260 961480 0 0 42024 128 1349 744 0 2 75 24
0 1 8 10116 13032 961188 0 0 36772 256 1309 658 0 1 75 24
0 0 8 9668 12920 962080 0 0 19280 128 1176 382 0 0 89 11
0 0 8 9732 12920 962080 0 0 0 0 1049 60 0 0 100 0
0 0 8 9756 12936 962064 0 0 0 16 1024 46 0 0 100 0
Then, I set elevator=noop in the kernel parameters in grub, and still got:
> dmesg | grep sche
Using cfq io scheduler
An when running vmstat, the results were similar to the first ones.
0 0 0 9672 5364 966256 0 0 0 0 1023 50 0 0 100 0
0 0 0 9696 5364 966256 0 0 0 0 1020 40 0 0 100 0
0 0 0 9696 5364 966256 0 0 0 0 1019 38 0 0 100 0
0 1 0 8976 5472 966668 0 0 292 99208 1424 218 0 1 88 11
1 3 0 124388 5588 853452 0 0 9308 35904 1242 667 0 12 61 27
0 5 0 4656 4912 970608 0 0 128 43452 1248 446 0 14 25 61
0 5 0 4004 4952 971348 0 0 128 44052 1202 214 0 4 18 78
0 5 0 4340 4984 971056 0 0 128 32956 1188 166 0 3 25 72
0 5 0 5824 5028 969452 0 0 256 38300 1204 182 0 4 28 68
0 5 0 4664 5044 970476 0 0 128 16052 1111 97 0 1 28 71
0 5 0 7756 5084 967316 0 0 128 57792 1297 222 0 4 31 65
0 5 0 8788 4112 967248 0 0 128 31208 1174 142 0 2 20 77
0 4 0 14440 4120 961780 0 0 256 34572 1207 160 0 2 35 63
0 4 0 18752 4160 957320 0 0 128 37216 1190 166 0 3 37 60
0 4 0 13244 4192 962488 0 0 0 34440 1158 137 0 2 44 55
0 5 0 12444 4216 963244 0 0 256 29076 1240 179 0 2 40 58
0 4 8 17652 4244 958008 0 0 128 39808 1231 198 0 2 40 57
0 4 8 10124 4280 964992 0 0 128 38912 1212 174 0 3 34 63
0 4 8 11020 4308 964444 0 0 256 31516 1146 144 0 2 27 71
0 5 8 13124 4344 961808 0 0 256 41112 1208 185 0 3 24 73
0 5 8 11524 4368 963084 0 0 128 23316 1141 148 0 2 51 48
0 5 8 8508 4416 966156 0 0 128 47152 1219 186 0 3 47 50
0 2 8 9084 4444 965608 0 0 220 36104 1221 197 0 0 69 31
0 2 8 9084 4516 965536 0 0 0 41464 1237 220 0 0 59 40
0 2 8 8956 4516 965536 0 0 128 42600 1217 173 0 0 67 33
0 2 8 8828 4516 965796 0 0 128 42664 1223 170 0 0 61 39
0 2 8 8828 4516 965796 0 0 0 41580 1217 154 0 0 67 33
0 2 8 8836 4516 965796 0 0 0 38244 1203 156 0 0 61 39
0 2 8 8836 4516 965796 0 0 0 41832 1221 154 0 0 55 45
0 1 8 9604 4520 965012 0 0 1284 5764 1155 89 0 0 75 26
0 1 8 10256 4556 964456 0 0 38820 0 1337 708 0 1 75 24
0 1 8 9484 4584 965468 0 0 40232 0 1334 714 0 1 75 24
0 1 8 9932 4624 965688 0 0 43052 0 1354 770 0 1 75 24
1 1 8 9676 4664 966168 0 0 43176 0 1373 791 0 1 75 24
0 1 8 9100 4660 966952 0 0 38436 0 1320 683 0 1 75 24
0 1 8 8652 4680 967972 0 0 40232 8 1400 767 0 1 75 24
0 1 8 9228 4724 967928 0 0 43052 0 1356 759 0 1 75 24
0 1 8 9740 4764 967628 0 0 43176 0 1372 775 0 1 75 24
procs -----------memory---------- ---swap-- -----io---- --system-- ----cpu----
r b swpd free buff cache si so bi bo in cs us sy id wa
0 1 8 8780 4820 968872 0 0 40232 16 1337 742 0 1 75 24
0 1 8 10124 4856 967796 0 0 38180 0 1320 694 0 1 75 24
0 1 8 8460 4896 970104 0 8 43052 8 1407 782 0 2 75 24
0 1 8 9996 4936 968764 0 0 40104 0 1334 720 0 1 75 24
1 0 8 8652 4976 970544 0 0 43176 0 1356 775 0 1 75 24
0 1 8 8396 5036 971264 0 0 43308 1980 1387 772 0 1 75 24
0 1 8 12364 5100 968080 0 0 35776 2000 1327 646 0 2 75 24
0 1 8 9484 5140 971160 0 0 43048 3560 1469 804 0 1 75 24
0 1 8 10380 5172 970088 0 0 31904 25300 1411 601 0 2 74 24
0 1 8 8972 5196 971884 0 0 24088 11748 1276 450 0 1 75 24
1 0 8 17164 5264 963756 0 0 33696 5992 1319 603 0 1 75 24
0 1 8 9036 5304 972036 0 0 42920 0 1387 778 0 1 74 24
0 1 8 9164 5288 971792 0 0 43180 0 1357 776 0 1 75 24
0 1 8 10292 5316 970984 0 0 21412 0 1211 454 0 1 75 24
0 1 8 10320 5352 970688 0 0 6072 0 1096 198 0 1 75 24
0 1 8 8588 5352 972508 0 0 5912 280 1160 188 0 1 75 24
0 0 8 10124 5364 970936 0 0 524 0 1030 66 0 1 97 3
0 0 8 10140 5364 970936 0 0 0 0 1019 42 0 0 100 0
0 0 8 10140 5364 970936 0 0 0 0 1021 38 0 0 100 0
0 0 8 10148 5368 970932 0 0 4 0 1026 60 0 0 100 0
In the three cases, the real time spent in the operation was approximately the
same.
Am I setting the scheduler wrong? Anybody knows why writes/reads are
"serialized" even when using cfq ?
Thanks,
--
Antonio Larrosa
antlarr@tedial.com
TEDIAL - Tecnologías Digitales Audiovisuales, S.L.
Parque Tecnológico de Andalucía
C/ Severo Ochoa, nº 10, 1ª planta
29590 Campanillas, Málaga
Tfno. +34 951 012 624
Fax +34 951 012 610
next reply other threads:[~2005-07-13 14:52 UTC|newest]
Thread overview: 2+ messages / expand[flat|nested] mbox.gz Atom feed top
2005-07-13 14:55 Antonio Larrosa Jiménez [this message]
2005-07-13 15:28 ` 2.6.5 io scheduler problem? (cfq/deadline/noop) Christoph Hellwig
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=200507131655.17185.antlarr@tedial.com \
--to=antlarr@tedial.com \
--cc=linux-kernel@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.