[PATCH -mm] fs: shrinker: always scan at least one object of each type

[PATCH -mm] fs: shrinker: always scan at least one object of each type

[Vladimir Davydov]

In super_cache_scan() we divide the number of objects of particular type
by the total number of objects in order to distribute pressure among
different types of fs objects (inodes, dentries, fs-private objects).
As a result, in some corner cases we can get nr_to_scan=0 even if there
are some objects to reclaim, e.g. dentries=1, inodes=1, fs_objects=1,
nr_to_scan=1/3=0.

This is unacceptable for per memcg kmem accounting, because this means
that some objects may never get reclaimed after memcg death, preventing
it from being freed.

This patch therefore assures that super_cache_scan() will scan at least
one object of each type if any.

Signed-off-by: Vladimir Davydov <vdavydov@parallels.com>
---
 fs/super.c |    6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/fs/super.c b/fs/super.c
index 482b4071f4de..63136156867e 100644
--- a/fs/super.c
+++ b/fs/super.c
@@ -92,13 +92,13 @@ static unsigned long super_cache_scan(struct shrinker *shrink,
 	 * prune the dcache first as the icache is pinned by it, then
 	 * prune the icache, followed by the filesystem specific caches
 	 */
-	sc->nr_to_scan = dentries;
+	sc->nr_to_scan = dentries + 1;
 	freed = prune_dcache_sb(sb, sc);
-	sc->nr_to_scan = inodes;
+	sc->nr_to_scan = inodes + 1;
 	freed += prune_icache_sb(sb, sc);
 
 	if (fs_objects) {
-		sc->nr_to_scan = fs_objects;
+		sc->nr_to_scan = fs_objects + 1;
 		freed += sb->s_op->free_cached_objects(sb, sc);
 	}
 
-- 
1.7.10.4

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH -mm] fs: shrinker: always scan at least one object of each type

[Andrew Morton]

On Mon, 12 Jan 2015 13:20:46 +0300 Vladimir Davydov <vdavydov@parallels.com> wrote:

> In super_cache_scan() we divide the number of objects of particular type
> by the total number of objects in order to distribute pressure among
> different types of fs objects (inodes, dentries, fs-private objects).
> As a result, in some corner cases we can get nr_to_scan=0 even if there
> are some objects to reclaim, e.g. dentries=1, inodes=1, fs_objects=1,
> nr_to_scan=1/3=0.
> 
> This is unacceptable for per memcg kmem accounting, because this means
> that some objects may never get reclaimed after memcg death, preventing
> it from being freed.
> 
> This patch therefore assures that super_cache_scan() will scan at least
> one object of each type if any.
> 
> --- a/fs/super.c
> +++ b/fs/super.c
> @@ -92,13 +92,13 @@ static unsigned long super_cache_scan(struct shrinker *shrink,
>  	 * prune the dcache first as the icache is pinned by it, then
>  	 * prune the icache, followed by the filesystem specific caches
>  	 */
> -	sc->nr_to_scan = dentries;
> +	sc->nr_to_scan = dentries + 1;
>  	freed = prune_dcache_sb(sb, sc);
> -	sc->nr_to_scan = inodes;
> +	sc->nr_to_scan = inodes + 1;
>  	freed += prune_icache_sb(sb, sc);
>  
>  	if (fs_objects) {
> -		sc->nr_to_scan = fs_objects;
> +		sc->nr_to_scan = fs_objects + 1;
>  		freed += sb->s_op->free_cached_objects(sb, sc);
>  	}

A reader of this code will wonder "why is it adding 1 everywhere". 
Let's tell them?

--- a/fs/super.c~fs-shrinker-always-scan-at-least-one-object-of-each-type-fix
+++ a/fs/super.c
@@ -91,6 +91,9 @@ static unsigned long super_cache_scan(st
 	/*
 	 * prune the dcache first as the icache is pinned by it, then
 	 * prune the icache, followed by the filesystem specific caches
+	 *
+	 * Ensure that we always scan at least one object - memcg kmem
+	 * accounting uses this to fully empty the caches.
 	 */
 	sc->nr_to_scan = dentries + 1;
 	freed = prune_dcache_sb(sb, sc);
_

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH -mm] fs: shrinker: always scan at least one object of each type

[Vladimir Davydov]

On Tue, Jan 13, 2015 at 03:56:39PM -0800, Andrew Morton wrote:
> On Mon, 12 Jan 2015 13:20:46 +0300 Vladimir Davydov <vdavydov@parallels.com> wrote:
> 
> > In super_cache_scan() we divide the number of objects of particular type
> > by the total number of objects in order to distribute pressure among
> > different types of fs objects (inodes, dentries, fs-private objects).
> > As a result, in some corner cases we can get nr_to_scan=0 even if there
> > are some objects to reclaim, e.g. dentries=1, inodes=1, fs_objects=1,
> > nr_to_scan=1/3=0.
> > 
> > This is unacceptable for per memcg kmem accounting, because this means
> > that some objects may never get reclaimed after memcg death, preventing
> > it from being freed.
> > 
> > This patch therefore assures that super_cache_scan() will scan at least
> > one object of each type if any.
> > 
> > --- a/fs/super.c
> > +++ b/fs/super.c
> > @@ -92,13 +92,13 @@ static unsigned long super_cache_scan(struct shrinker *shrink,
> >  	 * prune the dcache first as the icache is pinned by it, then
> >  	 * prune the icache, followed by the filesystem specific caches
> >  	 */
> > -	sc->nr_to_scan = dentries;
> > +	sc->nr_to_scan = dentries + 1;
> >  	freed = prune_dcache_sb(sb, sc);
> > -	sc->nr_to_scan = inodes;
> > +	sc->nr_to_scan = inodes + 1;
> >  	freed += prune_icache_sb(sb, sc);
> >  
> >  	if (fs_objects) {
> > -		sc->nr_to_scan = fs_objects;
> > +		sc->nr_to_scan = fs_objects + 1;
> >  		freed += sb->s_op->free_cached_objects(sb, sc);
> >  	}
> 
> A reader of this code will wonder "why is it adding 1 everywhere". 
> Let's tell them?

Yeah, sounds reasonable. Thank you!

> 
> --- a/fs/super.c~fs-shrinker-always-scan-at-least-one-object-of-each-type-fix
> +++ a/fs/super.c
> @@ -91,6 +91,9 @@ static unsigned long super_cache_scan(st
>  	/*
>  	 * prune the dcache first as the icache is pinned by it, then
>  	 * prune the icache, followed by the filesystem specific caches
> +	 *
> +	 * Ensure that we always scan at least one object - memcg kmem
> +	 * accounting uses this to fully empty the caches.
>  	 */
>  	sc->nr_to_scan = dentries + 1;
>  	freed = prune_dcache_sb(sb, sc);
> _
> 

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>