From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752597AbaFBW7Q (ORCPT ); Mon, 2 Jun 2014 18:59:16 -0400 Received: from ipmail05.adl6.internode.on.net ([150.101.137.143]:2575 "EHLO ipmail05.adl6.internode.on.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751036AbaFBW7P (ORCPT ); Mon, 2 Jun 2014 18:59:15 -0400 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Ash1ANIAjVN5LL1sPGdsb2JhbABZgweDRagXAQEBAQEBBpgbAYEOFwMBAQEBODWCJQEBBTocIxAIAxgJJQ8FJQMHGhOIQdAeFxaFP4h9B4RABJl/ixyLXCs Date: Tue, 3 Jun 2014 08:59:11 +1000 From: Dave Chinner To: Jens Axboe Cc: Linus Torvalds , Minchan Kim , Linux Kernel Mailing List , Andrew Morton , linux-mm , "H. Peter Anvin" , Ingo Molnar , Peter Zijlstra , Mel Gorman , Rik van Riel , Johannes Weiner , Hugh Dickins , Rusty Russell , "Michael S. Tsirkin" , Dave Hansen , Steven Rostedt Subject: Re: [RFC 2/2] x86_64: expand kernel stack to 16K Message-ID: <20140602225911.GU14410@dastard> References: <1401260039-18189-1-git-send-email-minchan@kernel.org> <1401260039-18189-2-git-send-email-minchan@kernel.org> <20140528223142.GO8554@dastard> <20140529013007.GF6677@dastard> <5389393D.2030305@kernel.dk> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <5389393D.2030305@kernel.dk> User-Agent: Mutt/1.5.21 (2010-09-15) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Fri, May 30, 2014 at 08:06:53PM -0600, Jens Axboe wrote: > On 2014-05-28 20:42, Linus Torvalds wrote: > >Well, we've definitely have had some issues with deeper callchains > >with md, but I suspect virtio might be worse, and the new blk-mq code > >is lilkely worse in this respect too. > > I don't think blk-mq is worse than the older stack, in fact it > should be better. The call chains are shorter, and a lot less cruft > on the stack. Historically the stack issues have been nested > devices, however. And for sync IO, we do run it inline, so if the > driver chews up a lot of stack, well... Hi Jens - as we found out with the mm code, there's a significant disconnect between what the code looks like (i.e. it may use very little stack directly) and what the compiler is generating. Before blk-mq: 9) 3952 112 scsi_request_fn+0x4b/0x490 10) 3840 32 __blk_run_queue+0x37/0x50 11) 3808 64 queue_unplugged+0x39/0xb0 12) 3744 112 blk_flush_plug_list+0x20b/0x240 Now with blk-mq: 3) 4672 96 virtio_queue_rq+0xd2/0x1e0 4) 4576 128 __blk_mq_run_hw_queue+0x1f0/0x3e0 5) 4448 16 blk_mq_run_hw_queue+0x35/0x40 6) 4432 80 blk_mq_insert_requests+0xc7/0x130 7) 4352 96 blk_mq_flush_plug_list+0x129/0x140 8) 4256 112 blk_flush_plug_list+0xe7/0x230 So previously flushing a plug used rough 200 bytes of stack. With blk-mq, it's over 400 bytes. IOWs, blk-mq has more than doubled the block layer stack usage... Cheers, Dave. -- Dave Chinner david@fromorbit.com