From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from verein.lst.de ([213.95.11.211]:60606 "EHLO newverein.lst.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726555AbeHGNx0 (ORCPT ); Tue, 7 Aug 2018 09:53:26 -0400 Date: Tue, 7 Aug 2018 13:44:51 +0200 From: Christoph Hellwig To: viro@zeniv.linux.org.uk Cc: Avi Kivity , Linus Torvalds , linux-aio@kvack.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 4/4 v2] aio: allow direct aio poll comletions for keyed wakeups Message-ID: <20180807114451.GA8853@lst.de> References: <20180806083058.14724-1-hch@lst.de> <20180806083058.14724-5-hch@lst.de> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20180806083058.14724-5-hch@lst.de> Sender: linux-fsdevel-owner@vger.kernel.org List-ID: If we get a keyed wakeup for a aio poll waitqueue and wake can acquire the ctx_lock without spinning we can just complete the iocb straight from the wakeup callback to avoid a context switch. Signed-off-by: Christoph Hellwig Tested-by: Avi Kivity --- Fix a subject line typo and improve a comment. fs/aio.c | 20 ++++++++++++++++++-- 1 file changed, 18 insertions(+), 2 deletions(-) diff --git a/fs/aio.c b/fs/aio.c index 2fd19521d8a8..5943098a87c6 100644 --- a/fs/aio.c +++ b/fs/aio.c @@ -1672,13 +1672,29 @@ static int aio_poll_wake(struct wait_queue_entry *wait, unsigned mode, int sync, void *key) { struct poll_iocb *req = container_of(wait, struct poll_iocb, wait); + struct aio_kiocb *iocb = container_of(req, struct aio_kiocb, poll); __poll_t mask = key_to_poll(key); req->woken = true; /* for instances that support it check for an event match first: */ - if (mask && !(mask & req->events)) - return 0; + if (mask) { + if (!(mask & req->events)) + return 0; + + /* + * Try to complete the iocb inline if we can to avoid a costly + * context switch. + */ + if (spin_trylock(&iocb->ki_ctx->ctx_lock)) { + list_del(&iocb->ki_list); + spin_unlock(&iocb->ki_ctx->ctx_lock); + + list_del_init(&req->wait.entry); + aio_poll_complete(iocb, mask); + return 1; + } + } list_del_init(&req->wait.entry); schedule_work(&req->work); -- 2.18.0