* Re: [PATCH -mm] fs: shrinker: always scan at least one object of each type
@ 2015-01-13 23:56 ` Andrew Morton
0 siblings, 0 replies; 9+ messages in thread
From: Andrew Morton @ 2015-01-13 23:56 UTC (permalink / raw)
To: Vladimir Davydov
Cc: Johannes Weiner, Michal Hocko, Alexander Viro, linux-mm,
linux-fsdevel, linux-kernel
On Mon, 12 Jan 2015 13:20:46 +0300 Vladimir Davydov <vdavydov@parallels.com> wrote:
> In super_cache_scan() we divide the number of objects of particular type
> by the total number of objects in order to distribute pressure among
> different types of fs objects (inodes, dentries, fs-private objects).
> As a result, in some corner cases we can get nr_to_scan=0 even if there
> are some objects to reclaim, e.g. dentries=1, inodes=1, fs_objects=1,
> nr_to_scan=1/3=0.
>
> This is unacceptable for per memcg kmem accounting, because this means
> that some objects may never get reclaimed after memcg death, preventing
> it from being freed.
>
> This patch therefore assures that super_cache_scan() will scan at least
> one object of each type if any.
>
> --- a/fs/super.c
> +++ b/fs/super.c
> @@ -92,13 +92,13 @@ static unsigned long super_cache_scan(struct shrinker *shrink,
> * prune the dcache first as the icache is pinned by it, then
> * prune the icache, followed by the filesystem specific caches
> */
> - sc->nr_to_scan = dentries;
> + sc->nr_to_scan = dentries + 1;
> freed = prune_dcache_sb(sb, sc);
> - sc->nr_to_scan = inodes;
> + sc->nr_to_scan = inodes + 1;
> freed += prune_icache_sb(sb, sc);
>
> if (fs_objects) {
> - sc->nr_to_scan = fs_objects;
> + sc->nr_to_scan = fs_objects + 1;
> freed += sb->s_op->free_cached_objects(sb, sc);
> }
A reader of this code will wonder "why is it adding 1 everywhere".
Let's tell them?
--- a/fs/super.c~fs-shrinker-always-scan-at-least-one-object-of-each-type-fix
+++ a/fs/super.c
@@ -91,6 +91,9 @@ static unsigned long super_cache_scan(st
/*
* prune the dcache first as the icache is pinned by it, then
* prune the icache, followed by the filesystem specific caches
+ *
+ * Ensure that we always scan at least one object - memcg kmem
+ * accounting uses this to fully empty the caches.
*/
sc->nr_to_scan = dentries + 1;
freed = prune_dcache_sb(sb, sc);
_
^ permalink raw reply [flat|nested] 9+ messages in thread* Re: [PATCH -mm] fs: shrinker: always scan at least one object of each type
@ 2015-01-13 23:56 ` Andrew Morton
0 siblings, 0 replies; 9+ messages in thread
From: Andrew Morton @ 2015-01-13 23:56 UTC (permalink / raw)
To: Vladimir Davydov
Cc: Johannes Weiner, Michal Hocko, Alexander Viro, linux-mm,
linux-fsdevel, linux-kernel
On Mon, 12 Jan 2015 13:20:46 +0300 Vladimir Davydov <vdavydov@parallels.com> wrote:
> In super_cache_scan() we divide the number of objects of particular type
> by the total number of objects in order to distribute pressure among
> different types of fs objects (inodes, dentries, fs-private objects).
> As a result, in some corner cases we can get nr_to_scan=0 even if there
> are some objects to reclaim, e.g. dentries=1, inodes=1, fs_objects=1,
> nr_to_scan=1/3=0.
>
> This is unacceptable for per memcg kmem accounting, because this means
> that some objects may never get reclaimed after memcg death, preventing
> it from being freed.
>
> This patch therefore assures that super_cache_scan() will scan at least
> one object of each type if any.
>
> --- a/fs/super.c
> +++ b/fs/super.c
> @@ -92,13 +92,13 @@ static unsigned long super_cache_scan(struct shrinker *shrink,
> * prune the dcache first as the icache is pinned by it, then
> * prune the icache, followed by the filesystem specific caches
> */
> - sc->nr_to_scan = dentries;
> + sc->nr_to_scan = dentries + 1;
> freed = prune_dcache_sb(sb, sc);
> - sc->nr_to_scan = inodes;
> + sc->nr_to_scan = inodes + 1;
> freed += prune_icache_sb(sb, sc);
>
> if (fs_objects) {
> - sc->nr_to_scan = fs_objects;
> + sc->nr_to_scan = fs_objects + 1;
> freed += sb->s_op->free_cached_objects(sb, sc);
> }
A reader of this code will wonder "why is it adding 1 everywhere".
Let's tell them?
--- a/fs/super.c~fs-shrinker-always-scan-at-least-one-object-of-each-type-fix
+++ a/fs/super.c
@@ -91,6 +91,9 @@ static unsigned long super_cache_scan(st
/*
* prune the dcache first as the icache is pinned by it, then
* prune the icache, followed by the filesystem specific caches
+ *
+ * Ensure that we always scan at least one object - memcg kmem
+ * accounting uses this to fully empty the caches.
*/
sc->nr_to_scan = dentries + 1;
freed = prune_dcache_sb(sb, sc);
_
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 9+ messages in thread* Re: [PATCH -mm] fs: shrinker: always scan at least one object of each type
2015-01-13 23:56 ` Andrew Morton
(?)
@ 2015-01-14 7:36 ` Vladimir Davydov
-1 siblings, 0 replies; 9+ messages in thread
From: Vladimir Davydov @ 2015-01-14 7:36 UTC (permalink / raw)
To: Andrew Morton
Cc: Johannes Weiner, Michal Hocko, Alexander Viro, linux-mm,
linux-fsdevel, linux-kernel
On Tue, Jan 13, 2015 at 03:56:39PM -0800, Andrew Morton wrote:
> On Mon, 12 Jan 2015 13:20:46 +0300 Vladimir Davydov <vdavydov@parallels.com> wrote:
>
> > In super_cache_scan() we divide the number of objects of particular type
> > by the total number of objects in order to distribute pressure among
> > different types of fs objects (inodes, dentries, fs-private objects).
> > As a result, in some corner cases we can get nr_to_scan=0 even if there
> > are some objects to reclaim, e.g. dentries=1, inodes=1, fs_objects=1,
> > nr_to_scan=1/3=0.
> >
> > This is unacceptable for per memcg kmem accounting, because this means
> > that some objects may never get reclaimed after memcg death, preventing
> > it from being freed.
> >
> > This patch therefore assures that super_cache_scan() will scan at least
> > one object of each type if any.
> >
> > --- a/fs/super.c
> > +++ b/fs/super.c
> > @@ -92,13 +92,13 @@ static unsigned long super_cache_scan(struct shrinker *shrink,
> > * prune the dcache first as the icache is pinned by it, then
> > * prune the icache, followed by the filesystem specific caches
> > */
> > - sc->nr_to_scan = dentries;
> > + sc->nr_to_scan = dentries + 1;
> > freed = prune_dcache_sb(sb, sc);
> > - sc->nr_to_scan = inodes;
> > + sc->nr_to_scan = inodes + 1;
> > freed += prune_icache_sb(sb, sc);
> >
> > if (fs_objects) {
> > - sc->nr_to_scan = fs_objects;
> > + sc->nr_to_scan = fs_objects + 1;
> > freed += sb->s_op->free_cached_objects(sb, sc);
> > }
>
> A reader of this code will wonder "why is it adding 1 everywhere".
> Let's tell them?
Yeah, sounds reasonable. Thank you!
>
> --- a/fs/super.c~fs-shrinker-always-scan-at-least-one-object-of-each-type-fix
> +++ a/fs/super.c
> @@ -91,6 +91,9 @@ static unsigned long super_cache_scan(st
> /*
> * prune the dcache first as the icache is pinned by it, then
> * prune the icache, followed by the filesystem specific caches
> + *
> + * Ensure that we always scan at least one object - memcg kmem
> + * accounting uses this to fully empty the caches.
> */
> sc->nr_to_scan = dentries + 1;
> freed = prune_dcache_sb(sb, sc);
> _
>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 9+ messages in thread* Re: [PATCH -mm] fs: shrinker: always scan at least one object of each type
@ 2015-01-14 7:36 ` Vladimir Davydov
0 siblings, 0 replies; 9+ messages in thread
From: Vladimir Davydov @ 2015-01-14 7:36 UTC (permalink / raw)
To: Andrew Morton
Cc: Johannes Weiner, Michal Hocko, Alexander Viro, linux-mm,
linux-fsdevel, linux-kernel
On Tue, Jan 13, 2015 at 03:56:39PM -0800, Andrew Morton wrote:
> On Mon, 12 Jan 2015 13:20:46 +0300 Vladimir Davydov <vdavydov@parallels.com> wrote:
>
> > In super_cache_scan() we divide the number of objects of particular type
> > by the total number of objects in order to distribute pressure among
> > different types of fs objects (inodes, dentries, fs-private objects).
> > As a result, in some corner cases we can get nr_to_scan=0 even if there
> > are some objects to reclaim, e.g. dentries=1, inodes=1, fs_objects=1,
> > nr_to_scan=1/3=0.
> >
> > This is unacceptable for per memcg kmem accounting, because this means
> > that some objects may never get reclaimed after memcg death, preventing
> > it from being freed.
> >
> > This patch therefore assures that super_cache_scan() will scan at least
> > one object of each type if any.
> >
> > --- a/fs/super.c
> > +++ b/fs/super.c
> > @@ -92,13 +92,13 @@ static unsigned long super_cache_scan(struct shrinker *shrink,
> > * prune the dcache first as the icache is pinned by it, then
> > * prune the icache, followed by the filesystem specific caches
> > */
> > - sc->nr_to_scan = dentries;
> > + sc->nr_to_scan = dentries + 1;
> > freed = prune_dcache_sb(sb, sc);
> > - sc->nr_to_scan = inodes;
> > + sc->nr_to_scan = inodes + 1;
> > freed += prune_icache_sb(sb, sc);
> >
> > if (fs_objects) {
> > - sc->nr_to_scan = fs_objects;
> > + sc->nr_to_scan = fs_objects + 1;
> > freed += sb->s_op->free_cached_objects(sb, sc);
> > }
>
> A reader of this code will wonder "why is it adding 1 everywhere".
> Let's tell them?
Yeah, sounds reasonable. Thank you!
>
> --- a/fs/super.c~fs-shrinker-always-scan-at-least-one-object-of-each-type-fix
> +++ a/fs/super.c
> @@ -91,6 +91,9 @@ static unsigned long super_cache_scan(st
> /*
> * prune the dcache first as the icache is pinned by it, then
> * prune the icache, followed by the filesystem specific caches
> + *
> + * Ensure that we always scan at least one object - memcg kmem
> + * accounting uses this to fully empty the caches.
> */
> sc->nr_to_scan = dentries + 1;
> freed = prune_dcache_sb(sb, sc);
> _
>
^ permalink raw reply [flat|nested] 9+ messages in thread* Re: [PATCH -mm] fs: shrinker: always scan at least one object of each type
@ 2015-01-14 7:36 ` Vladimir Davydov
0 siblings, 0 replies; 9+ messages in thread
From: Vladimir Davydov @ 2015-01-14 7:36 UTC (permalink / raw)
To: Andrew Morton
Cc: Johannes Weiner, Michal Hocko, Alexander Viro, linux-mm,
linux-fsdevel, linux-kernel
On Tue, Jan 13, 2015 at 03:56:39PM -0800, Andrew Morton wrote:
> On Mon, 12 Jan 2015 13:20:46 +0300 Vladimir Davydov <vdavydov@parallels.com> wrote:
>
> > In super_cache_scan() we divide the number of objects of particular type
> > by the total number of objects in order to distribute pressure among
> > different types of fs objects (inodes, dentries, fs-private objects).
> > As a result, in some corner cases we can get nr_to_scan=0 even if there
> > are some objects to reclaim, e.g. dentries=1, inodes=1, fs_objects=1,
> > nr_to_scan=1/3=0.
> >
> > This is unacceptable for per memcg kmem accounting, because this means
> > that some objects may never get reclaimed after memcg death, preventing
> > it from being freed.
> >
> > This patch therefore assures that super_cache_scan() will scan at least
> > one object of each type if any.
> >
> > --- a/fs/super.c
> > +++ b/fs/super.c
> > @@ -92,13 +92,13 @@ static unsigned long super_cache_scan(struct shrinker *shrink,
> > * prune the dcache first as the icache is pinned by it, then
> > * prune the icache, followed by the filesystem specific caches
> > */
> > - sc->nr_to_scan = dentries;
> > + sc->nr_to_scan = dentries + 1;
> > freed = prune_dcache_sb(sb, sc);
> > - sc->nr_to_scan = inodes;
> > + sc->nr_to_scan = inodes + 1;
> > freed += prune_icache_sb(sb, sc);
> >
> > if (fs_objects) {
> > - sc->nr_to_scan = fs_objects;
> > + sc->nr_to_scan = fs_objects + 1;
> > freed += sb->s_op->free_cached_objects(sb, sc);
> > }
>
> A reader of this code will wonder "why is it adding 1 everywhere".
> Let's tell them?
Yeah, sounds reasonable. Thank you!
>
> --- a/fs/super.c~fs-shrinker-always-scan-at-least-one-object-of-each-type-fix
> +++ a/fs/super.c
> @@ -91,6 +91,9 @@ static unsigned long super_cache_scan(st
> /*
> * prune the dcache first as the icache is pinned by it, then
> * prune the icache, followed by the filesystem specific caches
> + *
> + * Ensure that we always scan at least one object - memcg kmem
> + * accounting uses this to fully empty the caches.
> */
> sc->nr_to_scan = dentries + 1;
> freed = prune_dcache_sb(sb, sc);
> _
>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 9+ messages in thread