From: Vivek Goyal <vgoyal@redhat.com>
To: Gui Jianfeng <guijianfeng@cn.fujitsu.com>
Cc: Shaohua Li <shli@kernel.org>, Jens Axboe <jaxboe@fusionio.com>,
linux-kernel@vger.kernel.org
Subject: Re: fio posixaio performance problem
Date: Thu, 4 Aug 2011 10:12:10 -0400 [thread overview]
Message-ID: <20110804141210.GA429@redhat.com> (raw)
In-Reply-To: <4E3A4DF7.3020605@cn.fujitsu.com>
On Thu, Aug 04, 2011 at 03:44:55PM +0800, Gui Jianfeng wrote:
[..]
> > oh, not related per your blktrace. so we have two problems here:
> > 1. fio doesn't dispatch request in 8ms.
> > 2. no close request preempt.
>
> Yes, these're actual factors why performance is so bad.
>
> > both looks quite wield. can you post a longer blktrace output, like
> > for one second? the piece is too short.
>
> Attached.
Gui, few observations from you log file.
- preemption happened 1631 times and did not happen 527 times and idle
timer fired.
- In some cases where preemption did not happen, next request seems to
be too far away (more than CFQQ_CLOSE_THR=8K sectors).
- I noticed couple of cases where next request was with-in 8K distanace
still preemption did not happen. This makes me curious. Can you please
put some trace messages in should_preempt() and rq_close() call and see
what's going on?
For example, following trace shows that next request is 5176 sector behind
the previous one completed. I am wondering why did preemption not take
place.
8,0 0 606 2.751892651 16420 D W 512146800 + 8 [fio]
8,0 2 579 2.752127950 0 C W 512146800 + 8 [0]
8,0 0 609 2.752235995 16421 Q WS 512141624 + 8 [fio]
8,0 0 610 2.752238859 16421 G WS 512141624 + 8 [fio]
8,0 0 612 2.752243818 16421 I W 512141624 + 8 [fio]
8,0 0 0 2.752246262 0 m N cfq16421S / insert_request
8,0 0 0 2.752247729 0 m N cfq16421S / add_to_rr
8,0 2 0 2.759710295 0 m N cfq idle timer fired
Putting some extra trace messages in CFQ might help here. BTW, which
kernel version are you using? 3.0?
Thanks
Vivek
next prev parent reply other threads:[~2011-08-04 14:12 UTC|newest]
Thread overview: 20+ messages / expand[flat|nested] mbox.gz Atom feed top
2011-08-03 3:40 fio posixaio performance problem Gui Jianfeng
2011-08-03 4:06 ` Dave Chinner
2011-08-03 4:47 ` Gui Jianfeng
2011-08-03 5:12 ` Gui Jianfeng
2011-08-03 7:38 ` Shaohua Li
2011-08-03 8:11 ` Gui Jianfeng
2011-08-03 8:22 ` Shaohua Li
2011-08-03 9:48 ` Gui Jianfeng
2011-08-03 15:45 ` Vivek Goyal
2011-08-03 17:51 ` Vivek Goyal
2011-08-04 0:53 ` Shaohua Li
2011-08-04 2:00 ` Gui Jianfeng
2011-08-04 3:14 ` Shaohua Li
[not found] ` <4E3A4DF7.3020605@cn.fujitsu.com>
2011-08-04 8:25 ` Shaohua Li
2011-08-04 8:35 ` Gui Jianfeng
2011-08-04 9:01 ` Jens Axboe
2011-08-04 14:12 ` Vivek Goyal [this message]
2011-08-05 0:56 ` Gui Jianfeng
2011-08-05 1:31 ` Gui Jianfeng
2011-08-04 1:55 ` Gui Jianfeng
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20110804141210.GA429@redhat.com \
--to=vgoyal@redhat.com \
--cc=guijianfeng@cn.fujitsu.com \
--cc=jaxboe@fusionio.com \
--cc=linux-kernel@vger.kernel.org \
--cc=shli@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox