RE: Processes stuck on D state on Dual Opteron

Chen, Kenneth W
2005-04-12T08:04:56Z

On Tue, Apr 12 2005, Nick Piggin wrote:
> Actually the patches I have sent you do fix real bugs, but they also
> make the block layer less likely to recurse into page reclaim, so it
> may be eg. hiding the problem that Neil's patch fixes.

Jens Axboe wrote on Tuesday, April 12, 2005 12:08 AM
> Can you push those to Andrew? I'm quite happy with the way they turned
> out. It would be nice if Ken would bench 2.6.12-rc2 with and without
> those patches.

I like the patch a lot and already did bench it on our db setup. However,
I'm seeing a negative regression compare to a very very crappy patch (see
attached, you can laugh at me for doing things like that :-).

My first reaction is that the overhead is in wait queue setup and tear down
in get_request_wait function. Throwing the following patch on top does improve
things a bit, but we are still in the negative territory. I can't explain why.
Everything suppose to be faster. So I'm staring at the execution profile at
the moment.

diff -Nru a/drivers/block/ll_rw_blk.c b/drivers/block/ll_rw_blk.c
--- a/drivers/block/ll_rw_blk.c 2005-04-12 00:48:12 -07:00
+++ b/drivers/block/ll_rw_blk.c 2005-04-12 00:48:12 -07:00
@@ -1740,10 +1740,35 @@
*/
static struct request *get_request_wait(request_queue_t *q, int rw)
{
- DEFINE_WAIT(wait);
struct request *rq;
+ struct request_list *rl = &q->rq;
+ int gfp_flag = GFP_ATOMIC;
+
+ if (rl->count[rw] < queue_congestion_off_threshold(q)) {
+ rq = kmem_cache_alloc(request_cachep, gfp_flag);
+ if (rq) {
+ if (!elv_set_request(q, rq, gfp_flag)) {
+
+ rl->count[rw]++;
+ INIT_LIST_HEAD(&rq->queuelist);
+ rq->flags = rw;
+ rq->rq_status = RQ_ACTIVE;
+ rq->ref_count = 1;
+ rq->q = q;
+ rq->rl = rl;
+ rq->special = NULL;
+ rq->data_len = 0;
+ rq->data = NULL;
+ rq->sense = NULL;
+
+ return rq;
+ }
+ kmem_cache_free(request_cachep, rq);
+ }
+ }

do {
+ DEFINE_WAIT(wait);
struct request_list *rl = &q->rq;

prepare_to_wait_exclusive(&rl->wait[rw], &wait,

begin 666 old_freereq.patch
M9&EF9B M3G)U(&$O9')I=F5R<R]B;&]C:R]L;%]R=U]B;&LN8R!B+V1R:79E
M<G,O8FQO8VLO;&Q?<G=?8FQK+F,*+2TM(&$O9')I=F5R<R]B;&]C:R]L;%]R
M=U]B;&LN8PDR,# U+3 T+3 T(# P.C4X.C4U("TP-SHP, HK*RL@8B]D<FEV
M97)S+V)L;V-K+VQL7W)W7V)L:RYC"3(P,#4M,#0M,#0@,# Z-3@Z-34@+3 W
M.C P"D! ("TQ.3DV+#$U("LQ.3DV+#$T($! "B *( ER82 ](&)I;RT^8FE?
M<G<@)B H,2 \/"!"24]?4E=?04A%040I.PH@"BL)+RH@1W)A8B!A(&9R964@
M<F5Q=65S="!F<F]M('1H92!F<F5E;&ES=" J+PHK"69R965R97$@/2!G971?
M<F5Q=65S="AQ+"!R=RP@1T907T%43TU)0RD["BL*(&%G86EN.@H@"7-P:6Y?
M;&]C:U]I<G$H<2T^<75E=65?;&]C:RD["B *+0EI9B H96QV7W%U975E7V5M
M<'1Y*'$I*2!["BL):68@*&5L=E]Q=65U95]E;7!T>2AQ*2D*( D)8FQK7W!L
M=6=?9&5V:6-E*'$I.PHM"0EG;W1O(&=E=%]R<3L*+0E]"BT):68@*&)A<G)I
M97(I"BT)"6=O=&\@9V5T7W)Q.PH@"B )96Q?<F5T(#T@96QV7VUE<F=E*'$L
A("9R97$L(&)I;RD["B )<W=I=&-H("AE;%]R970I('L*
`
end