* Re: [PATCH] improve read ahead in kernel
2012-12-09 15:57 [PATCH] improve read ahead in kernel xtu4
@ 2012-12-09 10:18 ` Randy Dunlap
2012-12-11 3:40 ` resend--[PATCH] " xtu4
1 sibling, 0 replies; 6+ messages in thread
From: Randy Dunlap @ 2012-12-09 10:18 UTC (permalink / raw)
To: xtu4; +Cc: linux-kernel, linux-tip-commits, linux-mm, di.zhang
On 12/09/12 07:57, xtu4 wrote:
>
> Subject: [PATCH] when system in low memory scenario, imaging there is a mp3
> play, or video play, we need to read mp3 or video file
> from memory to page cache,but when system lack of memory,
> page cache of mp3 or video file will be reclaimed.once read
> in memory, then reclaimed, it will cause audio or video
> glitch,and it will increase the io operation at the same
> time.
>
> Signed-off-by: xiaobing tu <xiaobing.tu@intel.com>
What tree does this patch apply to?
It looks like most (or all) of the tabs in the source files have
been changed to spaces, which is Not Good.
Maybe some info in Documentation/email-clients.txt could help you.
> ---
> include/linux/mm_types.h | 4 ++++
> mm/Kconfig | 6 ++++++
> mm/filemap.c | 4 ++++
> mm/readahead.c | 20 +++++++++++++++++---
> mm/vmscan.c | 10 ++++++++--
> 5 files changed, 39 insertions(+), 5 deletions(-)
>
> diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h
> index 5b42f1b..541864d 100644
> --- a/include/linux/mm_types.h
> +++ b/include/linux/mm_types.h
> @@ -149,6 +149,10 @@ struct page {
> */
> void *shadow;
> #endif
> +#ifdef CONFIG_LOWMEMORY_READAHEAD
> + unsigned int ioprio;
> +#endif
> +
> }
> /*
> * If another subsystem starts using the double word pairing for atomic
> diff --git a/mm/Kconfig b/mm/Kconfig
> index e338407..dade8d3 100644
> --- a/mm/Kconfig
> +++ b/mm/Kconfig
> @@ -140,6 +140,12 @@ config ARCH_DISCARD_MEMBLOCK
> config NO_BOOTMEM
> boolean
>
> +# improve readahead in low memory scenario
> +config LOWMEMORY_READAHEAD
> + bool "improve readahead in low memory scenario"
> + depends on (IA64 || X86)
Why this depends on?
I don't see anything that is arch-specific in the patch.
> +
> +
> # eventually, we can have this option just 'select SPARSEMEM'
> config MEMORY_HOTPLUG
> bool "Allow for memory hot-add"
> diff --git a/mm/filemap.c b/mm/filemap.c
> index a0701e6..e32efed8 100644
> --- a/mm/filemap.c
> +++ b/mm/filemap.c
> @@ -129,6 +129,10 @@ void __delete_from_page_cache(struct page *page)
> page->mapping = NULL;
> /* Leave page->index set: truncation lookup relies upon it */
> mapping->nrpages--;
> + #ifdef CONFIG_LOWMEMORY_READAHEAD
> + page->ioprio = 0;
> + #endif
> +
> __dec_zone_page_state(page, NR_FILE_PAGES);
> if (PageSwapBacked(page))
> __dec_zone_page_state(page, NR_SHMEM);
> diff --git a/mm/readahead.c b/mm/readahead.c
> index cbcbb02..dd07cfe 100644
> --- a/mm/readahead.c
> +++ b/mm/readahead.c
> @@ -159,6 +159,11 @@ __do_page_cache_readahead(struct address_space *mapping, struct file *filp,
> int page_idx;
> int ret = 0;
> loff_t isize = i_size_read(inode);
> +#ifdef CONFIG_LOWMEMORY_READAHEAD
> + int class = 0;
> + if (p->io_context)
> + class = IOPRIO_PRIO_CLASS(p->io_context->ioprio);
> +#endif
>
> if (isize == 0)
> goto out;
> @@ -177,12 +182,21 @@ __do_page_cache_readahead(struct address_space *mapping, struct file *filp,
> rcu_read_lock();
> page = radix_tree_lookup(&mapping->page_tree, page_offset);
> rcu_read_unlock();
> - if (page)
> - continue;
> -
> + if (page){
> +#ifdef CONFIG_LOWMEMORY_READAHEAD
> + if (class == IOPRIO_CLASS_RT) {
> + page->ioprio = 1;
> +#endif
> + continue;
> + }
> page = page_cache_alloc_readahead(mapping);
> if (!page)
> break;
> +#ifdef CONFIG_LOWMEMORY_READAHEAD
> + if (class == IOPRIO_CLASS_RT) {
> + page->ioprio = 1;
> +#endif
> +
> page->index = page_offset;
> list_add(&page->lru, &page_pool);
> if (page_idx == nr_to_read - lookahead_size)
> diff --git a/mm/vmscan.c b/mm/vmscan.c
> index 753a2dc..86e03aa 100644
> --- a/mm/vmscan.c
> +++ b/mm/vmscan.c
> @@ -728,8 +728,14 @@ static enum page_references page_check_references(struct page *page,
> }
>
> /* Reclaim if clean, defer dirty pages to writeback */
> - if (referenced_page && !PageSwapBacked(page))
> - return PAGEREF_RECLAIM_CLEAN;
> + if (referenced_page && !PageSwapBacked(page)) {
> +#ifdef CONFIG_LOWMEMORY_READAHEAD
> + if (page->ioprio == 1) {
> + return PAGEREF_ACTIVATE;
> + } else
> +#endif
> + return PAGEREF_RECLAIM_CLEAN;
> + }
>
> return PAGEREF_RECLAIM;
> }
--
~Randy
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH] improve read ahead in kernel
@ 2012-12-09 15:57 xtu4
2012-12-09 10:18 ` Randy Dunlap
2012-12-11 3:40 ` resend--[PATCH] " xtu4
0 siblings, 2 replies; 6+ messages in thread
From: xtu4 @ 2012-12-09 15:57 UTC (permalink / raw)
To: linux-kernel, linux-tip-commits, linux-mm, di.zhang, xtu4
Subject: [PATCH] when system in low memory scenario, imaging there is a mp3
play, or video play, we need to read mp3 or video file
from memory to page cache,but when system lack of memory,
page cache of mp3 or video file will be reclaimed.once read
in memory, then reclaimed, it will cause audio or video
glitch,and it will increase the io operation at the same
time.
Signed-off-by: xiaobing tu <xiaobing.tu@intel.com>
---
include/linux/mm_types.h | 4 ++++
mm/Kconfig | 6 ++++++
mm/filemap.c | 4 ++++
mm/readahead.c | 20 +++++++++++++++++---
mm/vmscan.c | 10 ++++++++--
5 files changed, 39 insertions(+), 5 deletions(-)
diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h
index 5b42f1b..541864d 100644
--- a/include/linux/mm_types.h
+++ b/include/linux/mm_types.h
@@ -149,6 +149,10 @@ struct page {
*/
void *shadow;
#endif
+#ifdef CONFIG_LOWMEMORY_READAHEAD
+ unsigned int ioprio;
+#endif
+
}
/*
* If another subsystem starts using the double word pairing for atomic
diff --git a/mm/Kconfig b/mm/Kconfig
index e338407..dade8d3 100644
--- a/mm/Kconfig
+++ b/mm/Kconfig
@@ -140,6 +140,12 @@ config ARCH_DISCARD_MEMBLOCK
config NO_BOOTMEM
boolean
+# improve readahead in low memory scenario
+config LOWMEMORY_READAHEAD
+ bool "improve readahead in low memory scenario"
+ depends on (IA64 || X86)
+
+
# eventually, we can have this option just 'select SPARSEMEM'
config MEMORY_HOTPLUG
bool "Allow for memory hot-add"
diff --git a/mm/filemap.c b/mm/filemap.c
index a0701e6..e32efed8 100644
--- a/mm/filemap.c
+++ b/mm/filemap.c
@@ -129,6 +129,10 @@ void __delete_from_page_cache(struct page *page)
page->mapping = NULL;
/* Leave page->index set: truncation lookup relies upon it */
mapping->nrpages--;
+ #ifdef CONFIG_LOWMEMORY_READAHEAD
+ page->ioprio = 0;
+ #endif
+
__dec_zone_page_state(page, NR_FILE_PAGES);
if (PageSwapBacked(page))
__dec_zone_page_state(page, NR_SHMEM);
diff --git a/mm/readahead.c b/mm/readahead.c
index cbcbb02..dd07cfe 100644
--- a/mm/readahead.c
+++ b/mm/readahead.c
@@ -159,6 +159,11 @@ __do_page_cache_readahead(struct address_space
*mapping, struct file *filp,
int page_idx;
int ret = 0;
loff_t isize = i_size_read(inode);
+#ifdef CONFIG_LOWMEMORY_READAHEAD
+ int class = 0;
+ if (p->io_context)
+ class = IOPRIO_PRIO_CLASS(p->io_context->ioprio);
+#endif
if (isize == 0)
goto out;
@@ -177,12 +182,21 @@ __do_page_cache_readahead(struct address_space
*mapping, struct file *filp,
rcu_read_lock();
page = radix_tree_lookup(&mapping->page_tree, page_offset);
rcu_read_unlock();
- if (page)
- continue;
-
+ if (page){
+#ifdef CONFIG_LOWMEMORY_READAHEAD
+ if (class == IOPRIO_CLASS_RT) {
+ page->ioprio = 1;
+#endif
+ continue;
+ }
page = page_cache_alloc_readahead(mapping);
if (!page)
break;
+#ifdef CONFIG_LOWMEMORY_READAHEAD
+ if (class == IOPRIO_CLASS_RT) {
+ page->ioprio = 1;
+#endif
+
page->index = page_offset;
list_add(&page->lru, &page_pool);
if (page_idx == nr_to_read - lookahead_size)
diff --git a/mm/vmscan.c b/mm/vmscan.c
index 753a2dc..86e03aa 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -728,8 +728,14 @@ static enum page_references
page_check_references(struct page *page,
}
/* Reclaim if clean, defer dirty pages to writeback */
- if (referenced_page && !PageSwapBacked(page))
- return PAGEREF_RECLAIM_CLEAN;
+ if (referenced_page && !PageSwapBacked(page)) {
+#ifdef CONFIG_LOWMEMORY_READAHEAD
+ if (page->ioprio == 1) {
+ return PAGEREF_ACTIVATE;
+ } else
+#endif
+ return PAGEREF_RECLAIM_CLEAN;
+ }
return PAGEREF_RECLAIM;
}
--
1.7.6
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related [flat|nested] 6+ messages in thread
* resend--[PATCH] improve read ahead in kernel
2012-12-09 15:57 [PATCH] improve read ahead in kernel xtu4
2012-12-09 10:18 ` Randy Dunlap
@ 2012-12-11 3:40 ` xtu4
2012-12-16 2:15 ` Eric Wong
1 sibling, 1 reply; 6+ messages in thread
From: xtu4 @ 2012-12-11 3:40 UTC (permalink / raw)
To: xtu4; +Cc: linux-kernel, linux-tip-commits, linux-mm, di.zhang
resend it, due to format error
Subject: [PATCH] when system in low memory scenario, imaging there is a mp3
play, ora video play, we need to read mp3 or video file
from memory to page cache,but when system lack of memory,
page cache of mp3 or video file will be reclaimed.once read
in memory, then reclaimed, it will cause audio or video
glitch,and it will increase the io operation at the same
time.
Signed-off-by: xiaobing tu <xiaobing.tu@intel.com>
---
include/linux/mm_types.h | 4 ++++
mm/filemap.c | 4 ++++
mm/readahead.c | 20 +++++++++++++++++---
mm/vmscan.c | 10 ++++++++--
4 files changed, 33 insertions(+), 5 deletions(-)
diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h
index 5b42f1b..2739995 100644
--- a/include/linux/mm_types.h
+++ b/include/linux/mm_types.h
@@ -149,6 +149,10 @@ struct page {
*/
void *shadow;
#endif
+#ifdef CONFIG_LOWMEMORY_READAHEAD
+ unsigned int ioprio;
+#endif
+
}
/*
* If another subsystem starts using the double word pairing for atomic
diff --git a/mm/filemap.c b/mm/filemap.c
index a0701e6..ca3a3e8 100644
--- a/mm/filemap.c
+++ b/mm/filemap.c
@@ -129,6 +129,10 @@ void __delete_from_page_cache(struct page *page)
page->mapping = NULL;
/* Leave page->index set: truncation lookup relies upon it */
mapping->nrpages--;
+ #ifdef CONFIG_LOWMEMORY_READAHEAD
+ page->ioprio = 0;
+ #endif
+
__dec_zone_page_state(page, NR_FILE_PAGES);
if (PageSwapBacked(page))
__dec_zone_page_state(page, NR_SHMEM);
diff --git a/mm/readahead.c b/mm/readahead.c
index cbcbb02..5c2d2ff 100644
--- a/mm/readahead.c
+++ b/mm/readahead.c
@@ -159,6 +159,11 @@ __do_page_cache_readahead(struct address_space
*mapping, struct file *filp,
int page_idx;
int ret = 0;
loff_t isize = i_size_read(inode);
+#ifdef CONFIG_LOWMEMORY_READAHEAD
+ int class = 0;
+ if (p->io_context)
+ class = IOPRIO_PRIO_CLASS(p->io_context->ioprio);
+#endif
if (isize == 0)
goto out;
@@ -177,12 +182,21 @@ __do_page_cache_readahead(struct address_space
*mapping, struct file *filp,
rcu_read_lock();
page = radix_tree_lookup(&mapping->page_tree, page_offset);
rcu_read_unlock();
- if (page)
- continue;
-
+ if (page){
+#ifdef CONFIG_LOWMEMORY_READAHEAD
+ if (class == IOPRIO_CLASS_RT) {
+ page->ioprio = 1;
+#endif
+ continue;
+ }
page = page_cache_alloc_readahead(mapping);
if (!page)
break;
+#ifdef CONFIG_LOWMEMORY_READAHEAD
+ if (class == IOPRIO_CLASS_RT) {
+ page->ioprio = 1;
+#endif
+
page->index = page_offset;
list_add(&page->lru, &page_pool);
if (page_idx == nr_to_read - lookahead_size)
diff --git a/mm/vmscan.c b/mm/vmscan.c
index 753a2dc..0a1cae8 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -728,8 +728,14 @@ static enum page_references
page_check_references(struct page *page,
}
/* Reclaim if clean, defer dirty pages to writeback */
- if (referenced_page && !PageSwapBacked(page))
- return PAGEREF_RECLAIM_CLEAN;
+ if (referenced_page && !PageSwapBacked(page)) {
+#ifdef CONFIG_LOWMEMORY_READAHEAD
+ if (page->ioprio == 1) {
+ return PAGEREF_ACTIVATE;
+ } else
+#endif
+ return PAGEREF_RECLAIM_CLEAN;
+ }
return PAGEREF_RECLAIM;
}
--
1.7.6
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: resend--[PATCH] improve read ahead in kernel
2012-12-11 3:40 ` resend--[PATCH] " xtu4
@ 2012-12-16 2:15 ` Eric Wong
2012-12-20 6:20 ` Simon Jeons
0 siblings, 1 reply; 6+ messages in thread
From: Eric Wong @ 2012-12-16 2:15 UTC (permalink / raw)
To: xtu4; +Cc: linux-kernel, linux-tip-commits, linux-mm, di.zhang
xtu4 <xiaobing.tu@intel.com> wrote:
> resend it, due to format error
>
> Subject: [PATCH] when system in low memory scenario, imaging there is a mp3
> play, ora video play, we need to read mp3 or video file
> from memory to page cache,but when system lack of memory,
> page cache of mp3 or video file will be reclaimed.once read
> in memory, then reclaimed, it will cause audio or video
> glitch,and it will increase the io operation at the same
> time.
To me, this basically describes how POSIX_FADV_NOREUSE should work.
I would like to have this ability via fadvise (and not CONFIG_).
Also, I think your patch has too many #ifdefs to be accepted.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: resend--[PATCH] improve read ahead in kernel
2012-12-16 2:15 ` Eric Wong
@ 2012-12-20 6:20 ` Simon Jeons
2012-12-20 10:55 ` Roman Gushchin
0 siblings, 1 reply; 6+ messages in thread
From: Simon Jeons @ 2012-12-20 6:20 UTC (permalink / raw)
To: Eric Wong; +Cc: xtu4, linux-kernel, linux-tip-commits, linux-mm, di.zhang
On Sun, 2012-12-16 at 02:15 +0000, Eric Wong wrote:
> xtu4 <xiaobing.tu@intel.com> wrote:
> > resend it, due to format error
> >
> > Subject: [PATCH] when system in low memory scenario, imaging there is a mp3
> > play, ora video play, we need to read mp3 or video file
> > from memory to page cache,but when system lack of memory,
> > page cache of mp3 or video file will be reclaimed.once read
> > in memory, then reclaimed, it will cause audio or video
> > glitch,and it will increase the io operation at the same
> > time.
>
> To me, this basically describes how POSIX_FADV_NOREUSE should work.
Hi Eric,
But why fadvise POSIX_FADV_NOREUSE almost do nothing? Why not set some
flag or other things for these use once data?
> I would like to have this ability via fadvise (and not CONFIG_).
>
> Also, I think your patch has too many #ifdefs to be accepted.
>
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org. For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: resend--[PATCH] improve read ahead in kernel
2012-12-20 6:20 ` Simon Jeons
@ 2012-12-20 10:55 ` Roman Gushchin
0 siblings, 0 replies; 6+ messages in thread
From: Roman Gushchin @ 2012-12-20 10:55 UTC (permalink / raw)
To: Simon Jeons, Eric Wong
Cc: xtu4, linux-kernel@vger.kernel.org,
linux-tip-commits@vger.kernel.org, linux-mm@kvack.org,
di.zhang@intel.com
Hi Simon,
20.12.2012, 10:21, "Simon Jeons" <simon.jeons@gmail.com>:
> On Sun, 2012-12-16 at 02:15 +0000, Eric Wong wrote:
>
>> ?xtu4 <xiaobing.tu@intel.com> wrote:
>>> ?resend it, due to format error
>>>
>>> ?Subject: [PATCH] when system in low memory scenario, imaging there is a mp3
>>> ??play, ora video play, we need to read mp3 or video file
>>> ??from memory to page cache,but when system lack of memory,
>>> ??page cache of mp3 or video file will be reclaimed.once read
>>> ??in memory, then reclaimed, it will cause audio or video
>>> ??glitch,and it will increase the io operation at the same
>>> ??time.
>> ?To me, this basically describes how POSIX_FADV_NOREUSE should work.
>
> Hi Eric,
>
> But why fadvise POSIX_FADV_NOREUSE almost do nothing? Why not set some
> flag or other things for these use once data?
Because, it's not clear how should it work in some cases.
Do we expect one access? Should we track a page as accessed after any access (even if only one byte was read)?
What should we do with already-cached pages? etc.
IMHO, it will be better to introduce something like POSIX_FADV_DONTCACHE.
Corresponding pages can be added (ratated) to the tail of the inactive LRU after copying data in read() internals,
or after minor/major pagefault.
>
>> ?I would like to have this ability via fadvise (and not CONFIG_).
>>
>> ?Also, I think your patch has too many #ifdefs to be accepted.
>>
>> ?--
>> ?To unsubscribe, send a message with 'unsubscribe linux-mm' in
>> ?the body to majordomo@kvack.org. ?For more info on Linux MM,
>> ?see: http://www.linux-mm.org/ .
>> ?Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
>
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org. ?For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2012-12-20 10:55 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2012-12-09 15:57 [PATCH] improve read ahead in kernel xtu4
2012-12-09 10:18 ` Randy Dunlap
2012-12-11 3:40 ` resend--[PATCH] " xtu4
2012-12-16 2:15 ` Eric Wong
2012-12-20 6:20 ` Simon Jeons
2012-12-20 10:55 ` Roman Gushchin
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).