Re: [RFC 2/2] x86_64: expand kernel stack to 16K

Dave Chinner
2014-06-02T22:59:16Z

On Fri, May 30, 2014 at 08:06:53PM -0600, Jens Axboe wrote:
> On 2014-05-28 20:42, Linus Torvalds wrote:
> >Well, we've definitely have had some issues with deeper callchains
> >with md, but I suspect virtio might be worse, and the new blk-mq code
> >is lilkely worse in this respect too.
>
> I don't think blk-mq is worse than the older stack, in fact it
> should be better. The call chains are shorter, and a lot less cruft
> on the stack. Historically the stack issues have been nested
> devices, however. And for sync IO, we do run it inline, so if the
> driver chews up a lot of stack, well...

Hi Jens - as we found out with the mm code, there's a significant
disconnect between what the code looks like (i.e. it may use very
little stack directly) and what the compiler is generating.

Before blk-mq:

9) 3952 112 scsi_request_fn+0x4b/0x490
10) 3840 32 __blk_run_queue+0x37/0x50
11) 3808 64 queue_unplugged+0x39/0xb0
12) 3744 112 blk_flush_plug_list+0x20b/0x240

Now with blk-mq:

3) 4672 96 virtio_queue_rq+0xd2/0x1e0
4) 4576 128 __blk_mq_run_hw_queue+0x1f0/0x3e0
5) 4448 16 blk_mq_run_hw_queue+0x35/0x40
6) 4432 80 blk_mq_insert_requests+0xc7/0x130
7) 4352 96 blk_mq_flush_plug_list+0x129/0x140
8) 4256 112 blk_flush_plug_list+0xe7/0x230

So previously flushing a plug used rough 200 bytes of stack. With
blk-mq, it's over 400 bytes. IOWs, blk-mq has more than doubled the
block layer stack usage...

Cheers,

Dave.
--
Dave Chinner
david@fromorbit.com