From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jens Axboe Subject: Re: [PATCH v1 5/9] block: loop: convert to blk-mq Date: Wed, 20 Aug 2014 22:16:22 -0500 Message-ID: <53F56486.6060509@kernel.dk> References: <1408031441-31156-1-git-send-email-ming.lei@canonical.com> <1408031441-31156-6-git-send-email-ming.lei@canonical.com> <20140815163111.GA16652@infradead.org> <53EE370D.1060106@kernel.dk> <53EE3966.60609@kernel.dk> <53F0EAEC.9040505@kernel.dk> <53F3B89D.6070703@kernel.dk> <53F4C835.7030407@kernel.dk> <53F5605C.2010304@kernel.dk> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Cc: Christoph Hellwig , Linux Kernel Mailing List , Andrew Morton , Dave Kleikamp , Zach Brown , Benjamin LaHaise , Kent Overstreet , open@kvack.org, list@kvack.org:AIO , Linux FS Devel , Dave Chinner , Tejun Heo To: Ming Lei Return-path: In-Reply-To: Sender: owner-linux-aio@kvack.org List-Id: linux-fsdevel.vger.kernel.org On 2014-08-20 22:13, Ming Lei wrote: > On Thu, Aug 21, 2014 at 10:58 AM, Jens Axboe wrote: >> On 2014-08-20 21:54, Ming Lei wrote: >>>>> >>>>> From my investigation, context switch increases almost 50% with >>>>> workqueue compared with kthread in loop in a quad-core VM. With >>>>> kthread, requests may be handled as batch in cases which won't be >>>>> blocked in read()/write()(like null_blk, tmpfs, ...), but it is >>>>> impossible >>>>> with >>>>> workqueue any more. Also block plug&unplug should have been used >>>>> with kthread to optimize the case, especially when kernel AIO is >>>>> applied, >>>>> still impossible with work queue too. >>>> >>>> >>>> >>>> OK, that one is actually a good point, since one need not do per-item >>>> queueing. We could handle different units, though. And we should have >>>> proper >>>> marking of the last item in a chain of stuff, so we might even be able to >>>> offload based on that instead of doing single items. It wont help the >>>> sync >>>> case, but for that, workqueue and kthread would be identical. >>> >>> >>> We may do that by introducing callback of queue_rq_list in blk_mq_ops, >>> and I will figure out one patch today to see if it can help the case. >> >> >> I don't think we should add to the interface, I prefer keeping it clean like >> it is right now. At least not if we can get around it. My point is that the >> driver already knows when the chain is complete, when REQ_LAST is set. So >> before that event triggers, it need not kick off IO, or at least i could do >> it in batches before that. That may not be fully reliable in case of >> queueing errors, but if REQ_LAST or 'error return' is used as the way to >> kick off pending IO, then that should be good enough. Haven't audited this >> in a while, but at least that is the intent of REQ_LAST. > > Yes, I thought of too, but driver need another context for handling that, > either workqueue or kthread, which may cause the introduced per-device > workqueue useless. Yeah, if we are going to go this route instead, then it may make more sense to have a ->commit() hook to compliment ->queue_rq(). ->queue_rq() would then continue to run like it is now, and ->commit() would offload the pieces to a workqueue. Or we'd just do this on REQ_LAST. Same sort of thing, just handled a bit differently. -- Jens Axboe -- To unsubscribe, send a message with 'unsubscribe linux-aio' in the body to majordomo@kvack.org. For more info on Linux AIO, see: http://www.kvack.org/aio/ Don't email: aart@kvack.org