From: Suparna Bhattacharya <suparna@in.ibm.com>
To: linux-aio@kvack.org, linux-kernel@vger.kernel.org
Cc: linux-osdl@osdl.org
Subject: Re: [PATCH 5/22] AIO wait on page support
Date: Fri, 2 Jul 2004 18:46:55 +0530 [thread overview]
Message-ID: <20040702131655.GE4374@in.ibm.com> (raw)
In-Reply-To: <20040702130030.GA4256@in.ibm.com>
On Fri, Jul 02, 2004 at 06:30:30PM +0530, Suparna Bhattacharya wrote:
> The patchset contains modifications and fixes to the AIO core
> to support the full retry model, an implementation of AIO
> support for buffered filesystem AIO reads and O_SYNC writes
> (the latter courtesy O_SYNC speedup changes from Andrew Morton),
> an implementation of AIO reads and writes to pipes (from
> Chris Mason) and AIO poll (again from Chris Mason).
>
> Full retry infrastructure and fixes
> [1] aio-retry.patch
> [2] 4g4g-aio-hang-fix.patch
> [3] aio-retry-elevated-refcount.patch
> [4] aio-splice-runlist.patch
>
> FS AIO read
> [5] aio-wait-page.patch
--
Suparna Bhattacharya (suparna@in.ibm.com)
Linux Technology Center
IBM Software Lab, India
-----------------------------------
From: Suparna Bhattacharya <suparna@in.ibm.com>
Async wait on page support. Implements async versions of lock_page,
wait_on_page_locked, and wait_on_page_writeback which accept a
wait queue entry as a parameter, and where blocking waits
converted into retry exits if the wait queue entry specifies
an async callback for AIO.
include/linux/pagemap.h | 38 ++++++++++++++----
mm/filemap.c | 100 +++++++++++++++++++++++++++++++++++-------------
2 files changed, 105 insertions(+), 33 deletions(-)
--- aio/include/linux/pagemap.h 2004-06-17 14:12:37.170571824 -0700
+++ aio-wait-page/include/linux/pagemap.h 2004-06-17 14:10:41.974084352 -0700
@@ -150,17 +150,27 @@ static inline pgoff_t linear_page_index(
extern void FASTCALL(__lock_page(struct page *page));
extern void FASTCALL(unlock_page(struct page *page));
-static inline void lock_page(struct page *page)
+
+extern int FASTCALL(__lock_page_wq(struct page *page, wait_queue_t *wait));
+static inline int lock_page_wq(struct page *page, wait_queue_t *wait)
{
if (TestSetPageLocked(page))
- __lock_page(page);
+ return __lock_page_wq(page, wait);
+ else
+ return 0;
+}
+
+static inline void lock_page(struct page *page)
+{
+ lock_page_wq(page, NULL);
}
/*
* This is exported only for wait_on_page_locked/wait_on_page_writeback.
* Never use this directly!
*/
-extern void FASTCALL(wait_on_page_bit(struct page *page, int bit_nr));
+extern int FASTCALL(wait_on_page_bit_wq(struct page *page, int bit_nr,
+ wait_queue_t *wait));
/*
* Wait for a page to be unlocked.
@@ -169,19 +179,33 @@ extern void FASTCALL(wait_on_page_bit(st
* ie with increased "page->count" so that the page won't
* go away during the wait..
*/
-static inline void wait_on_page_locked(struct page *page)
+static inline int wait_on_page_locked_wq(struct page *page, wait_queue_t *wait)
{
if (PageLocked(page))
- wait_on_page_bit(page, PG_locked);
+ return wait_on_page_bit_wq(page, PG_locked, wait);
+ return 0;
+}
+
+static inline int wait_on_page_writeback_wq(struct page *page,
+ wait_queue_t *wait)
+{
+ if (PageWriteback(page))
+ return wait_on_page_bit_wq(page, PG_writeback, wait);
+ return 0;
+}
+
+static inline void wait_on_page_locked(struct page *page)
+{
+ wait_on_page_locked_wq(page, NULL);
}
/*
* Wait for a page to complete writeback
*/
+
static inline void wait_on_page_writeback(struct page *page)
{
- if (PageWriteback(page))
- wait_on_page_bit(page, PG_writeback);
+ wait_on_page_writeback_wq(page, NULL);
}
extern void end_page_writeback(struct page *page);
--- aio/mm/filemap.c 2004-06-17 14:11:57.420614736 -0700
+++ aio-wait-page/mm/filemap.c 2004-06-17 14:02:17.137831128 -0700
@@ -340,22 +340,43 @@ static void wake_up_page(struct page *pa
__wake_up(waitqueue, mode, 1, page);
}
-void fastcall wait_on_page_bit(struct page *page, int bit_nr)
+/*
+ * wait for the specified page bit to be cleared
+ * this could be a synchronous wait or could just queue an async
+ * notification callback depending on the wait queue entry parameter
+ *
+ * A NULL wait queue parameter defaults to sync behaviour
+ */
+int fastcall wait_on_page_bit_wq(struct page *page, int bit_nr, wait_queue_t *wait)
{
wait_queue_head_t *waitqueue = page_waitqueue(page);
- DEFINE_PAGE_WAIT(wait, page, bit_nr);
+ DEFINE_PAGE_WAIT(local_wait, page, bit_nr);
- do {
- prepare_to_wait(waitqueue, &wait.wait, TASK_UNINTERRUPTIBLE);
- if (test_bit(bit_nr, &page->flags)) {
- sync_page(page);
- io_schedule();
- }
- } while (test_bit(bit_nr, &page->flags));
- finish_wait(waitqueue, &wait.wait);
+ if (!wait)
+ wait = &local_wait.wait; /* default to a sync wait entry */
+
+ do {
+ prepare_to_wait(waitqueue, wait, TASK_UNINTERRUPTIBLE);
+ if (test_bit(bit_nr, &page->flags)) {
+ sync_page(page);
+ if (!is_sync_wait(wait)) {
+ /*
+ * if we've queued an async wait queue
+ * callback do not block; just tell the
+ * caller to return and retry later when
+ * the callback is notified
+ */
+ return -EIOCBRETRY;
+ }
+ io_schedule();
+ }
+ } while (test_bit(bit_nr, &page->flags));
+ finish_wait(waitqueue, wait);
+
+ return 0;
}
-
-EXPORT_SYMBOL(wait_on_page_bit);
+EXPORT_SYMBOL(wait_on_page_bit_wq);
+
/**
* unlock_page() - unlock a locked page
@@ -365,8 +386,9 @@ EXPORT_SYMBOL(wait_on_page_bit);
* Unlocks the page and wakes up sleepers in ___wait_on_page_locked().
* Also wakes sleepers in wait_on_page_writeback() because the wakeup
* mechananism between PageLocked pages and PageWriteback pages is shared.
- * But that's OK - sleepers in wait_on_page_writeback() just go back to sleep.
- *
+ * But that's OK - sleepers in wait_on_page_writeback() just go back to sleep,
+ * or in case the wakeup notifies async wait queue entries, as in the case
+ * of aio, retries would be triggered and may re-queue their callbacks.
* The first mb is necessary to safely close the critical section opened by the
* TestSetPageLocked(), the second mb is necessary to enforce ordering between
* the clear_bit and the read of the waitqueue (to avoid SMP races with a
@@ -399,29 +421,55 @@ void end_page_writeback(struct page *pag
EXPORT_SYMBOL(end_page_writeback);
+
/*
- * Get a lock on the page, assuming we need to sleep to get it.
+ * Get a lock on the page, assuming we need to either sleep to get it
+ * or to queue an async notification callback to try again when its
+ * available.
+ *
+ * A NULL wait queue parameter defaults to sync behaviour. Otherwise
+ * it specifies the wait queue entry to be used for async notification
+ * or waiting.
*
* Ugly: running sync_page() in state TASK_UNINTERRUPTIBLE is scary. If some
* random driver's requestfn sets TASK_RUNNING, we could busywait. However
* chances are that on the second loop, the block layer's plug list is empty,
* so sync_page() will then return in state TASK_UNINTERRUPTIBLE.
*/
-void fastcall __lock_page(struct page *page)
+int fastcall __lock_page_wq(struct page *page, wait_queue_t *wait)
{
- wait_queue_head_t *wqh = page_waitqueue(page);
- DEFINE_PAGE_WAIT_EXCLUSIVE(wait, page, PG_locked);
+ wait_queue_head_t *wqh = page_waitqueue(page);
+ DEFINE_PAGE_WAIT_EXCLUSIVE(local_wait, page, PG_locked);
- while (TestSetPageLocked(page)) {
- prepare_to_wait_exclusive(wqh, &wait.wait, TASK_UNINTERRUPTIBLE);
- if (PageLocked(page)) {
- sync_page(page);
- io_schedule();
- }
- }
- finish_wait(wqh, &wait.wait);
+ if (!wait)
+ wait = &local_wait.wait;
+
+ while (TestSetPageLocked(page)) {
+ prepare_to_wait_exclusive(wqh, wait, TASK_UNINTERRUPTIBLE);
+ if (PageLocked(page)) {
+ sync_page(page);
+ if (!is_sync_wait(wait)) {
+ /*
+ * if we've queued an async wait queue
+ * callback do not block; just tell the
+ * caller to return and retry later when
+ * the callback is notified
+ */
+ return -EIOCBRETRY;
+ }
+ io_schedule();
+ }
+ }
+ finish_wait(wqh, wait);
+ return 0;
}
+EXPORT_SYMBOL(__lock_page_wq);
+void fastcall __lock_page(struct page *page)
+{
+ __lock_page_wq(page, NULL);
+}
+
EXPORT_SYMBOL(__lock_page);
/*
next prev parent reply other threads:[~2004-07-02 13:08 UTC|newest]
Thread overview: 34+ messages / expand[flat|nested] mbox.gz Atom feed top
2004-07-02 13:00 [PATCH 0/22] fsaio, pipe aio and aio poll upgraded to 2.6.7 Suparna Bhattacharya
2004-07-02 13:07 ` [PATCH 1/22] High-level AIO retry infrastructure and fixes Suparna Bhattacharya
2004-07-02 13:11 ` [PATCH 2/22] use_mm fix (helps AIO hangs on 4:4 split) Suparna Bhattacharya
2004-07-02 13:14 ` [PATCH 3/22] Refcounting fixes Suparna Bhattacharya
2004-07-02 13:15 ` [PATCH 4/22] Splice ioctx runlist for fairness Suparna Bhattacharya
2004-07-02 13:16 ` Suparna Bhattacharya [this message]
2004-07-02 13:18 ` [PATCH 6/22] FS AIO read Suparna Bhattacharya
2004-07-02 13:19 ` [PATCH 7/22] Upfront readahead to help streaming AIO reads Suparna Bhattacharya
2004-07-02 13:20 ` [PATCH 8/22] AIO cancellation fix Suparna Bhattacharya
2004-07-02 13:23 ` [PATCH 9/22] AIO immediate read (needed for AIO pipes & sockets) Suparna Bhattacharya
2004-07-02 13:23 ` [PATCH 10/22] AIO pipe support Suparna Bhattacharya
2004-07-02 13:26 ` [PATCH 11/22] Reduce AIO worker context switches Suparna Bhattacharya
2004-07-02 16:05 ` [PATCH 12/22] Writeback page range hint Suparna Bhattacharya
2004-07-02 16:18 ` [PATCH 13/22] Fix writeback page range to use exact limits Suparna Bhattacharya
2004-07-02 16:22 ` [PATCH 14/22] mpage writepages range limit fix Suparna Bhattacharya
2004-07-02 16:25 ` [PATCH 15/22] filemap_fdatawrite range interface Suparna Bhattacharya
2004-07-02 16:27 ` [PATCH 16/22] Concurrent O_SYNC write support Suparna Bhattacharya
2004-07-02 16:31 ` [PATCH 17/22] AIO wait on writeback Suparna Bhattacharya
2004-07-02 16:33 ` [PATCH 18/22] AIO O_SYNC write Suparna Bhattacharya
2004-07-02 16:34 ` [PATCH 19/22] Fix math error in AIO wait on writeback Suparna Bhattacharya
2004-07-02 16:39 ` [PATCH 20/22] AIO poll Suparna Bhattacharya
2004-07-29 15:19 ` Jeff Moyer
2004-07-29 16:02 ` Avi Kivity
2004-07-29 16:16 ` Arjan van de Ven
2004-07-29 16:37 ` Benjamin LaHaise
2004-07-29 17:23 ` William Lee Irwin III
2004-07-29 17:10 ` William Lee Irwin III
2004-07-29 17:24 ` Avi Kivity
2004-07-29 17:26 ` William Lee Irwin III
2004-07-29 17:30 ` Avi Kivity
2004-07-29 17:32 ` William Lee Irwin III
2004-07-02 16:42 ` [PATCH 21/22] fix: flush workqueue on put_ioctx Suparna Bhattacharya
2004-07-02 16:44 ` [PATCH 22/22] Fix stalls with the AIO context switch patch Suparna Bhattacharya
2004-07-05 9:24 ` [PATCH 0/22] fsaio, pipe aio and aio poll upgraded to 2.6.7 Christoph Hellwig
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20040702131655.GE4374@in.ibm.com \
--to=suparna@in.ibm.com \
--cc=linux-aio@kvack.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-osdl@osdl.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox