From: "Jianpeng Ma" <majianpeng@gmail.com>
To: shli <shli@kernel.org>
Cc: axboe <axboe@kernel.dk>, linux-kernel <linux-kernel@vger.kernel.org>
Subject: Re: Re: [PATCH] block: Add blk_rq_pos(rq) to sort rq when plushing plug-list.
Date: Tue, 16 Oct 2012 10:40:14 +0800 [thread overview]
Message-ID: <201210161040097340482@gmail.com> (raw)
In-Reply-To: CANejiEWWnrY_dKKuKLvjRms85crO+A7hi8P6nTkGMX9O2TL4kQ@mail.gmail.com
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="gb2312", Size: 6492 bytes --]
On 2012-10-15 21:18 Shaohua Li <shli@kernel.org> Wrote:
>2012/10/15 Shaohua Li <shli@fusionio.com>:
>> 2012/10/15 Jianpeng Ma <majianpeng@gmail.com>:
>>> My workload is a raid5 which had 16 disks. And used our filesystem to
>>> write using direct-io mode.
>>> I used the blktrace to find those message:
>>>
>>> 8,16 0 3570 1.083923979 2519 I W 144323176 + 24 [md127_raid5]
>>> 8,16 0 0 1.083926214 0 m N cfq2519 insert_request
>>> 8,16 0 3571 1.083926586 2519 I W 144323072 + 104 [md127_raid5]
>>> 8,16 0 0 1.083926952 0 m N cfq2519 insert_request
>>> 8,16 0 3572 1.083927180 2519 U N [md127_raid5] 2
>>> 8,16 0 0 1.083927870 0 m N cfq2519 Not idling.st->count:1
>>> 8,16 0 0 1.083928320 0 m N cfq2519 dispatch_insert
>>> 8,16 0 0 1.083928951 0 m N cfq2519 dispatched a request
>>> 8,16 0 0 1.083929443 0 m N cfq2519 activate rq,drv=1
>>> 8,16 0 3573 1.083929530 2519 D W 144323176 + 24 [md127_raid5]
>>> 8,16 0 0 1.083933883 0 m N cfq2519 Not idling.st->count:1
>>> 8,16 0 0 1.083934189 0 m N cfq2519 dispatch_insert
>>> 8,16 0 0 1.083934654 0 m N cfq2519 dispatched a request
>>> 8,16 0 0 1.083935014 0 m N cfq2519 activate rq,drv=2
>>> 8,16 0 3574 1.083935101 2519 D W 144323072 + 104 [md127_raid5]
>>> 8,16 0 3575 1.084196179 0 C W 144323176 + 24 [0]
>>> 8,16 0 0 1.084197979 0 m N cfq2519 complete rqnoidle 0
>>> 8,16 0 3576 1.084769073 0 C W 144323072 + 104 [0]
>>> ......
>>> 8,16 1 3596 1.091394357 2519 I W 144322544 + 16 [md127_raid5]
>>> 8,16 1 0 1.091396181 0 m N cfq2519 insert_request
>>> 8,16 1 3597 1.091396571 2519 I W 144322520 + 24 [md127_raid5]
>>> 8,16 1 0 1.091396934 0 m N cfq2519 insert_request
>>> 8,16 1 3598 1.091397165 2519 I W 144322488 + 32 [md127_raid5]
>>> 8,16 1 0 1.091397477 0 m N cfq2519 insert_request
>>> 8,16 1 3599 1.091397708 2519 I W 144322432 + 56 [md127_raid5]
>>> 8,16 1 0 1.091398023 0 m N cfq2519 insert_request
>>> 8,16 1 3600 1.091398284 2519 U N [md127_raid5] 4
>>> 8,16 1 0 1.091398986 0 m N cfq2519 Not idling. st->count:1
>>> 8,16 1 0 1.091399511 0 m N cfq2519 dispatch_insert
>>> 8,16 1 0 1.091400217 0 m N cfq2519 dispatched a request
>>> 8,16 1 0 1.091400688 0 m N cfq2519 activate rq,drv=1
>>> 8,16 1 3601 1.091400766 2519 D W 144322544 + 16 [md127_raid5]
>>> 8,16 1 0 1.091406151 0 m N cfq2519 Not idling.st->count:1
>>> 8,16 1 0 1.091406460 0 m N cfq2519 dispatch_insert
>>> 8,16 1 0 1.091406931 0 m N cfq2519 dispatched a request
>>> 8,16 1 0 1.091407291 0 m N cfq2519 activate rq,drv=2
>>> 8,16 1 3602 1.091407378 2519 D W 144322520 + 24 [md127_raid5]
>>> 8,16 1 0 1.091414006 0 m N cfq2519 Not idling.st->count:1
>>> 8,16 1 0 1.091414297 0 m N cfq2519 dispatch_insert
>>> 8,16 1 0 1.091414702 0 m N cfq2519 dispatched a request
>>> 8,16 1 0 1.091415047 0 m N cfq2519 activate rq, drv=3
>>> 8,16 1 3603 1.091415125 2519 D W 144322488 + 32 [md127_raid5]
>>> 8,16 1 0 1.091416469 0 m N cfq2519 Not idling.st->count:1
>>> 8,16 1 0 1.091416754 0 m N cfq2519 dispatch_insert
>>> 8,16 1 0 1.091417186 0 m N cfq2519 dispatched a request
>>> 8,16 1 0 1.091417535 0 m N cfq2519 activate rq,drv=4
>>> 8,16 1 3604 1.091417628 2519 D W 144322432 + 56 [md127_raid5]
>>> 8,16 1 3605 1.091857225 4393 C W 144322544 + 16 [0]
>>> 8,16 1 0 1.091858753 0 m N cfq2519 complete rqnoidle 0
>>> 8,16 1 3606 1.092068456 4393 C W 144322520 + 24 [0]
>>> 8,16 1 0 1.092069851 0 m N cfq2519 complete rqnoidle 0
>>> 8,16 1 3607 1.092350440 4393 C W 144322488 + 32 [0]
>>> 8,16 1 0 1.092351688 0 m N cfq2519 complete rqnoidle 0
>>> 8,16 1 3608 1.093629323 0 C W 144322432 + 56 [0]
>>> 8,16 1 0 1.093631151 0 m N cfq2519 complete rqnoidle 0
>>> 8,16 1 0 1.093631574 0 m N cfq2519 will busy wait
>>> 8,16 1 0 1.093631829 0 m N cfq schedule dispatch
>>>
>>> Because in func "elv_attempt_insert_merge", it only to try to
>>> backmerge.So the four request can't merge in theory.
>>> I trace ten minutes and count those situation, it can count 25%.
>>>
>>> With the patch,i tested and not found situation like above.
>>>
>>> Signed-off-by: Jianpeng Ma <majianpeng@gmail.com>
>>> ---
>>> block/blk-core.c | 3 ++-
>>> 1 file changed, 2 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/block/blk-core.c b/block/blk-core.c
>>> index a33870b..3c95c4d 100644
>>> --- a/block/blk-core.c
>>> +++ b/block/blk-core.c
>>> @@ -2868,7 +2868,8 @@ static int plug_rq_cmp(void *priv, struct list_head *a, struct list_head *b)
>>> struct request *rqa = container_of(a, struct request, queuelist);
>>> struct request *rqb = container_of(b, struct request, queuelist);
>>>
>>> - return !(rqa->q <= rqb->q);
>>> + return !(rqa->q < rqb->q ||
>>> + (rqa->q == rqb->q && blk_rq_pos(rqa) < blk_rq_pos(rqb)));
>>> }
>>
>> Does this one help too?
>> http://marc.info/?l=linux-kernel&m=132399972114668&w=2
>>
>> I thought the real problem is we don't do recursive request
>> merge. I had no objection to the patch itself, but just hope we
>> can make recursive merge work, which is more generic.
>
>Oh, wait, the 4 requests aren't merged completely in your case.
>And the requests are from one thread and plug context.
>Not the issue I mentioned. I'm wondering how this could happen.
>they should be merged in attempt_plug_merge already.
Yes, i ignore this. The real problem is why not merge in attempt_plug_mege?
I'll do further study.
Thanks!
Jianpengÿôèº{.nÇ+‰·Ÿ®‰†+%ŠËÿ±éݶ\x17¥Šwÿº{.nÇ+‰·¥Š{±þG«éÿŠ{ayº\x1dʇڙë,j\a¢f£¢·hšïêÿ‘êçz_è®\x03(階ŠÝ¢j"ú\x1a¶^[m§ÿÿ¾\a«þG«éÿ¢¸?™¨èÚ&£ø§~á¶iO•æ¬z·švØ^\x14\x04\x1a¶^[m§ÿÿÃ\fÿ¶ìÿ¢¸?–I¥
next prev parent reply other threads:[~2012-10-16 2:39 UTC|newest]
Thread overview: 7+ messages / expand[flat|nested] mbox.gz Atom feed top
2012-10-15 12:35 [PATCH] block: Add blk_rq_pos(rq) to sort rq when plushing plug-list Jianpeng Ma
2012-10-15 12:55 ` Shaohua Li
2012-10-15 13:18 ` Shaohua Li
2012-10-16 2:40 ` Jianpeng Ma [this message]
2012-10-16 6:38 ` Jianpeng Ma
2012-10-16 7:48 ` Shaohua Li
2012-10-17 1:00 ` Jianpeng Ma
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=201210161040097340482@gmail.com \
--to=majianpeng@gmail.com \
--cc=axboe@kernel.dk \
--cc=linux-kernel@vger.kernel.org \
--cc=shli@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox