From: Suparna Bhattacharya <suparna@in.ibm.com>
To: linux-aio@kvack.org, linux-kernel@vger.kernel.org
Cc: linux-osdl@osdl.org
Subject: Re: [PATCH 5/22] AIO wait on page support
Date: Fri, 2 Jul 2004 18:46:55 +0530 [thread overview]
Message-ID: <20040702131655.GE4374@in.ibm.com> (raw)
In-Reply-To: <20040702130030.GA4256@in.ibm.com>
On Fri, Jul 02, 2004 at 06:30:30PM +0530, Suparna Bhattacharya wrote:
> The patchset contains modifications and fixes to the AIO core
> to support the full retry model, an implementation of AIO
> support for buffered filesystem AIO reads and O_SYNC writes
> (the latter courtesy O_SYNC speedup changes from Andrew Morton),
> an implementation of AIO reads and writes to pipes (from
> Chris Mason) and AIO poll (again from Chris Mason).
>
> Full retry infrastructure and fixes
> [1] aio-retry.patch
> [2] 4g4g-aio-hang-fix.patch
> [3] aio-retry-elevated-refcount.patch
> [4] aio-splice-runlist.patch
>
> FS AIO read
> [5] aio-wait-page.patch
--
Suparna Bhattacharya (suparna@in.ibm.com)
Linux Technology Center
IBM Software Lab, India
-----------------------------------
From: Suparna Bhattacharya <suparna@in.ibm.com>
Async wait on page support. Implements async versions of lock_page,
wait_on_page_locked, and wait_on_page_writeback which accept a
wait queue entry as a parameter, and where blocking waits
converted into retry exits if the wait queue entry specifies
an async callback for AIO.
include/linux/pagemap.h | 38 ++++++++++++++----
mm/filemap.c | 100 +++++++++++++++++++++++++++++++++++-------------
2 files changed, 105 insertions(+), 33 deletions(-)
--- aio/include/linux/pagemap.h 2004-06-17 14:12:37.170571824 -0700
+++ aio-wait-page/include/linux/pagemap.h 2004-06-17 14:10:41.974084352 -0700
@@ -150,17 +150,27 @@ static inline pgoff_t linear_page_index(
extern void FASTCALL(__lock_page(struct page *page));
extern void FASTCALL(unlock_page(struct page *page));
-static inline void lock_page(struct page *page)
+
+extern int FASTCALL(__lock_page_wq(struct page *page, wait_queue_t *wait));
+static inline int lock_page_wq(struct page *page, wait_queue_t *wait)
{
if (TestSetPageLocked(page))
- __lock_page(page);
+ return __lock_page_wq(page, wait);
+ else
+ return 0;
+}
+
+static inline void lock_page(struct page *page)
+{
+ lock_page_wq(page, NULL);
}
/*
* This is exported only for wait_on_page_locked/wait_on_page_writeback.
* Never use this directly!
*/
-extern void FASTCALL(wait_on_page_bit(struct page *page, int bit_nr));
+extern int FASTCALL(wait_on_page_bit_wq(struct page *page, int bit_nr,
+ wait_queue_t *wait));
/*
* Wait for a page to be unlocked.
@@ -169,19 +179,33 @@ extern void FASTCALL(wait_on_page_bit(st
* ie with increased "page->count" so that the page won't
* go away during the wait..
*/
-static inline void wait_on_page_locked(struct page *page)
+static inline int wait_on_page_locked_wq(struct page *page, wait_queue_t *wait)
{
if (PageLocked(page))
- wait_on_page_bit(page, PG_locked);
+ return wait_on_page_bit_wq(page, PG_locked, wait);
+ return 0;
+}
+
+static inline int wait_on_page_writeback_wq(struct page *page,
+ wait_queue_t *wait)
+{
+ if (PageWriteback(page))
+ return wait_on_page_bit_wq(page, PG_writeback, wait);
+ return 0;
+}
+
+static inline void wait_on_page_locked(struct page *page)
+{
+ wait_on_page_locked_wq(page, NULL);
}
/*
* Wait for a page to complete writeback
*/
+
static inline void wait_on_page_writeback(struct page *page)
{
- if (PageWriteback(page))
- wait_on_page_bit(page, PG_writeback);
+ wait_on_page_writeback_wq(page, NULL);
}
extern void end_page_writeback(struct page *page);
--- aio/mm/filemap.c 2004-06-17 14:11:57.420614736 -0700
+++ aio-wait-page/mm/filemap.c 2004-06-17 14:02:17.137831128 -0700
@@ -340,22 +340,43 @@ static void wake_up_page(struct page *pa
__wake_up(waitqueue, mode, 1, page);
}
-void fastcall wait_on_page_bit(struct page *page, int bit_nr)
+/*
+ * wait for the specified page bit to be cleared
+ * this could be a synchronous wait or could just queue an async
+ * notification callback depending on the wait queue entry parameter
+ *
+ * A NULL wait queue parameter defaults to sync behaviour
+ */
+int fastcall wait_on_page_bit_wq(struct page *page, int bit_nr, wait_queue_t *wait)
{
wait_queue_head_t *waitqueue = page_waitqueue(page);
- DEFINE_PAGE_WAIT(wait, page, bit_nr);
+ DEFINE_PAGE_WAIT(local_wait, page, bit_nr);
- do {
- prepare_to_wait(waitqueue, &wait.wait, TASK_UNINTERRUPTIBLE);
- if (test_bit(bit_nr, &page->flags)) {
- sync_page(page);
- io_schedule();
- }
- } while (test_bit(bit_nr, &page->flags));
- finish_wait(waitqueue, &wait.wait);
+ if (!wait)
+ wait = &local_wait.wait; /* default to a sync wait entry */
+
+ do {
+ prepare_to_wait(waitqueue, wait, TASK_UNINTERRUPTIBLE);
+ if (test_bit(bit_nr, &page->flags)) {
+ sync_page(page);
+ if (!is_sync_wait(wait)) {
+ /*
+ * if we've queued an async wait queue
+ * callback do not block; just tell the
+ * caller to return and retry later when
+ * the callback is notified
+ */
+ return -EIOCBRETRY;
+ }
+ io_schedule();
+ }
+ } while (test_bit(bit_nr, &page->flags));
+ finish_wait(waitqueue, wait);
+
+ return 0;
}
-
-EXPORT_SYMBOL(wait_on_page_bit);
+EXPORT_SYMBOL(wait_on_page_bit_wq);
+
/**
* unlock_page() - unlock a locked page
@@ -365,8 +386,9 @@ EXPORT_SYMBOL(wait_on_page_bit);
* Unlocks the page and wakes up sleepers in ___wait_on_page_locked().
* Also wakes sleepers in wait_on_page_writeback() because the wakeup
* mechananism between PageLocked pages and PageWriteback pages is shared.
- * But that's OK - sleepers in wait_on_page_writeback() just go back to sleep.
- *
+ * But that's OK - sleepers in wait_on_page_writeback() just go back to sleep,
+ * or in case the wakeup notifies async wait queue entries, as in the case
+ * of aio, retries would be triggered and may re-queue their callbacks.
* The first mb is necessary to safely close the critical section opened by the
* TestSetPageLocked(), the second mb is necessary to enforce ordering between
* the clear_bit and the read of the waitqueue (to avoid SMP races with a
@@ -399,29 +421,55 @@ void end_page_writeback(struct page *pag
EXPORT_SYMBOL(end_page_writeback);
+
/*
- * Get a lock on the page, assuming we need to sleep to get it.
+ * Get a lock on the page, assuming we need to either sleep to get it
+ * or to queue an async notification callback to try again when its
+ * available.
+ *
+ * A NULL wait queue parameter defaults to sync behaviour. Otherwise
+ * it specifies the wait queue entry to be used for async notification
+ * or waiting.
*
* Ugly: running sync_page() in state TASK_UNINTERRUPTIBLE is scary. If some
* random driver's requestfn sets TASK_RUNNING, we could busywait. However
* chances are that on the second loop, the block layer's plug list is empty,
* so sync_page() will then return in state TASK_UNINTERRUPTIBLE.
*/
-void fastcall __lock_page(struct page *page)
+int fastcall __lock_page_wq(struct page *page, wait_queue_t *wait)
{
- wait_queue_head_t *wqh = page_waitqueue(page);
- DEFINE_PAGE_WAIT_EXCLUSIVE(wait, page, PG_locked);
+ wait_queue_head_t *wqh = page_waitqueue(page);
+ DEFINE_PAGE_WAIT_EXCLUSIVE(local_wait, page, PG_locked);
- while (TestSetPageLocked(page)) {
- prepare_to_wait_exclusive(wqh, &wait.wait, TASK_UNINTERRUPTIBLE);
- if (PageLocked(page)) {
- sync_page(page);
- io_schedule();
- }
- }
- finish_wait(wqh, &wait.wait);
+ if (!wait)
+ wait = &local_wait.wait;
+
+ while (TestSetPageLocked(page)) {
+ prepare_to_wait_exclusive(wqh, wait, TASK_UNINTERRUPTIBLE);
+ if (PageLocked(page)) {
+ sync_page(page);
+ if (!is_sync_wait(wait)) {
+ /*
+ * if we've queued an async wait queue
+ * callback do not block; just tell the
+ * caller to return and retry later when
+ * the callback is notified
+ */
+ return -EIOCBRETRY;
+ }
+ io_schedule();
+ }
+ }
+ finish_wait(wqh, wait);
+ return 0;
}
+EXPORT_SYMBOL(__lock_page_wq);
+void fastcall __lock_page(struct page *page)
+{
+ __lock_page_wq(page, NULL);
+}
+
EXPORT_SYMBOL(__lock_page);
/*
next prev parent reply other threads:[~2004-07-02 13:08 UTC|newest]
Thread overview: 34+ messages / expand[flat|nested] mbox.gz Atom feed top
2004-07-02 13:00 [PATCH 0/22] fsaio, pipe aio and aio poll upgraded to 2.6.7 Suparna Bhattacharya
2004-07-02 13:07 ` [PATCH 1/22] High-level AIO retry infrastructure and fixes Suparna Bhattacharya
2004-07-02 13:11 ` [PATCH 2/22] use_mm fix (helps AIO hangs on 4:4 split) Suparna Bhattacharya
2004-07-02 13:14 ` [PATCH 3/22] Refcounting fixes Suparna Bhattacharya
2004-07-02 13:15 ` [PATCH 4/22] Splice ioctx runlist for fairness Suparna Bhattacharya
2004-07-02 13:16 ` Suparna Bhattacharya [this message]
2004-07-02 13:18 ` [PATCH 6/22] FS AIO read Suparna Bhattacharya
2004-07-02 13:19 ` [PATCH 7/22] Upfront readahead to help streaming AIO reads Suparna Bhattacharya
2004-07-02 13:20 ` [PATCH 8/22] AIO cancellation fix Suparna Bhattacharya
2004-07-02 13:23 ` [PATCH 9/22] AIO immediate read (needed for AIO pipes & sockets) Suparna Bhattacharya
2004-07-02 13:23 ` [PATCH 10/22] AIO pipe support Suparna Bhattacharya
2004-07-02 13:26 ` [PATCH 11/22] Reduce AIO worker context switches Suparna Bhattacharya
2004-07-02 16:05 ` [PATCH 12/22] Writeback page range hint Suparna Bhattacharya
2004-07-02 16:18 ` [PATCH 13/22] Fix writeback page range to use exact limits Suparna Bhattacharya
2004-07-02 16:22 ` [PATCH 14/22] mpage writepages range limit fix Suparna Bhattacharya
2004-07-02 16:25 ` [PATCH 15/22] filemap_fdatawrite range interface Suparna Bhattacharya
2004-07-02 16:27 ` [PATCH 16/22] Concurrent O_SYNC write support Suparna Bhattacharya
2004-07-02 16:31 ` [PATCH 17/22] AIO wait on writeback Suparna Bhattacharya
2004-07-02 16:33 ` [PATCH 18/22] AIO O_SYNC write Suparna Bhattacharya
2004-07-02 16:34 ` [PATCH 19/22] Fix math error in AIO wait on writeback Suparna Bhattacharya
2004-07-02 16:39 ` [PATCH 20/22] AIO poll Suparna Bhattacharya
2004-07-29 15:19 ` Jeff Moyer
2004-07-29 16:02 ` Avi Kivity
2004-07-29 16:16 ` Arjan van de Ven
2004-07-29 16:37 ` Benjamin LaHaise
2004-07-29 17:23 ` William Lee Irwin III
2004-07-29 17:10 ` William Lee Irwin III
2004-07-29 17:24 ` Avi Kivity
2004-07-29 17:26 ` William Lee Irwin III
2004-07-29 17:30 ` Avi Kivity
2004-07-29 17:32 ` William Lee Irwin III
2004-07-02 16:42 ` [PATCH 21/22] fix: flush workqueue on put_ioctx Suparna Bhattacharya
2004-07-02 16:44 ` [PATCH 22/22] Fix stalls with the AIO context switch patch Suparna Bhattacharya
2004-07-05 9:24 ` [PATCH 0/22] fsaio, pipe aio and aio poll upgraded to 2.6.7 Christoph Hellwig
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20040702131655.GE4374@in.ibm.com \
--to=suparna@in.ibm.com \
--cc=linux-aio@kvack.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-osdl@osdl.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.