All of lore.kernel.org
 help / color / mirror / Atom feed
From: Andrew Morton <akpm@linux-foundation.org>
To: Dave Chinner <david@fromorbit.com>
Cc: linux-fsdevel@vger.kernel.org, linux-mm@kvack.org,
	linux-kernel@vger.kernel.org, xfs@oss.sgi.com
Subject: Re: [PATCH 4/5] superblock: add filesystem shrinker operations
Date: Thu, 27 May 2010 13:32:39 -0700	[thread overview]
Message-ID: <20100527133239.5d038d9c.akpm@linux-foundation.org> (raw)
In-Reply-To: <1274777588-21494-5-git-send-email-david@fromorbit.com>

On Tue, 25 May 2010 18:53:07 +1000
Dave Chinner <david@fromorbit.com> wrote:

> From: Dave Chinner <dchinner@redhat.com>
> 
> Now we have a per-superblock shrinker implementation, we can add a
> filesystem specific callout to it to allow filesystem internal
> caches to be shrunk by the superblock shrinker.
> 
> Rather than perpetuate the multipurpose shrinker callback API (i.e.
> nr_to_scan == 0 meaning "tell me how many objects freeable in the
> cache), two operations will be added. The first will return the
> number of objects that are freeable, the second is the actual
> shrinker call.
> 
>
> ...
>
>  static int prune_super(struct shrinker *shrink, int nr_to_scan, gfp_t gfp_mask)
>  {
>  	struct super_block *sb;
> -	int count;
> +	int	fs_objects = 0;
> +	int	total_objects;
>  
>  	sb = container_of(shrink, struct super_block, s_shrink);
>  
> @@ -63,22 +64,40 @@ static int prune_super(struct shrinker *shrink, int nr_to_scan, gfp_t gfp_mask)
>  		return -1;
>  	}
>  
> -	if (nr_to_scan) {
> -		/* proportion the scan between the two cache__ */
> -		int total;
> -
> -		total = sb->s_nr_dentry_unused + sb->s_nr_inodes_unused + 1;
> -		count = (nr_to_scan * sb->s_nr_dentry_unused) / total;
> +	if (sb->s_op && sb->s_op->nr_cached_objects)
> +		fs_objects = sb->s_op->nr_cached_objects(sb);
>  
> -		/* prune dcache first as icache is pinned by it */
> -		prune_dcache_sb(sb, count);
> -		prune_icache_sb(sb, nr_to_scan - count);
> +	total_objects = sb->s_nr_dentry_unused +
> +			sb->s_nr_inodes_unused + fs_objects + 1;
> +	if (nr_to_scan) {
> +		int	dentries;
> +		int	inodes;
> +
> +		/* proportion the scan between the cache__ */
> +		dentries = (nr_to_scan * sb->s_nr_dentry_unused) /
> +							total_objects;
> +		inodes = (nr_to_scan * sb->s_nr_inodes_unused) /
> +							total_objects;
> +		if (fs_objects)
> +			fs_objects = (nr_to_scan * fs_objects) /
> +							total_objects;
> +		/*
> +		 * prune the dcache first as the icache is pinned by it, then
> +		 * prune the icache, followed by the filesystem specific caches
> +		 */
> +		prune_dcache_sb(sb, dentries);
> +		prune_icache_sb(sb, inodes);
> +		if (sb->s_op && sb->s_op->free_cached_objects) {

Under which circumstances is a NULL ->free_cached_objects valid?

> +			sb->s_op->free_cached_objects(sb, fs_objects);
> +			fs_objects = sb->s_op->nr_cached_objects(sb);
> +		}
> +		total_objects = sb->s_nr_dentry_unused +
> +				sb->s_nr_inodes_unused + fs_objects;
>  	}

The return value from ->free_cached_objects() doesn't actually get
used.  Instead the code calls ->nr_cached_objects() twice.


> -	count = ((sb->s_nr_dentry_unused + sb->s_nr_inodes_unused) / 100)
> -						* sysctl_vfs_cache_pressure;
> +	total_objects = (total_objects / 100) * sysctl_vfs_cache_pressure;
>  	up_read(&sb->s_umount);
> -	return count;
> +	return total_objects;
>  }
>  
>  /**
> diff --git a/include/linux/fs.h b/include/linux/fs.h
> index 5bff2dc..efcdcc6 100644
> --- a/include/linux/fs.h
> +++ b/include/linux/fs.h
> @@ -1590,6 +1590,17 @@ struct super_operations {
>  	ssize_t (*quota_write)(struct super_block *, int, const char *, size_t, loff_t);
>  #endif
>  	int (*bdev_try_to_free_page)(struct super_block*, struct page*, gfp_t);
> +
> +	/*
> +	 * memory shrinker operations.
> +	 * ->nr_cached_objects() should return the number of freeable cached
> +	 * objects the filesystem holds.
> +	 * ->free_cache_objects() should attempt to free the number of cached
> +	 * objects indicated. It should return how many objects it attempted to
> +	 * free.
> +	 */

I'd have thought that ->free_cache_objects() would always return the
number which it was passed.  Unless someone asked it to scan more
objects than exist, perhaps.

> +	int (*nr_cached_objects)(struct super_block *);
> +	int (*free_cached_objects)(struct super_block *, int);
>  };

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

WARNING: multiple messages have this Message-ID (diff)
From: Andrew Morton <akpm@linux-foundation.org>
To: Dave Chinner <david@fromorbit.com>
Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org,
	linux-mm@kvack.org, xfs@oss.sgi.com
Subject: Re: [PATCH 4/5] superblock: add filesystem shrinker operations
Date: Thu, 27 May 2010 13:32:39 -0700	[thread overview]
Message-ID: <20100527133239.5d038d9c.akpm@linux-foundation.org> (raw)
In-Reply-To: <1274777588-21494-5-git-send-email-david@fromorbit.com>

On Tue, 25 May 2010 18:53:07 +1000
Dave Chinner <david@fromorbit.com> wrote:

> From: Dave Chinner <dchinner@redhat.com>
> 
> Now we have a per-superblock shrinker implementation, we can add a
> filesystem specific callout to it to allow filesystem internal
> caches to be shrunk by the superblock shrinker.
> 
> Rather than perpetuate the multipurpose shrinker callback API (i.e.
> nr_to_scan == 0 meaning "tell me how many objects freeable in the
> cache), two operations will be added. The first will return the
> number of objects that are freeable, the second is the actual
> shrinker call.
> 
>
> ...
>
>  static int prune_super(struct shrinker *shrink, int nr_to_scan, gfp_t gfp_mask)
>  {
>  	struct super_block *sb;
> -	int count;
> +	int	fs_objects = 0;
> +	int	total_objects;
>  
>  	sb = container_of(shrink, struct super_block, s_shrink);
>  
> @@ -63,22 +64,40 @@ static int prune_super(struct shrinker *shrink, int nr_to_scan, gfp_t gfp_mask)
>  		return -1;
>  	}
>  
> -	if (nr_to_scan) {
> -		/* proportion the scan between the two cache__ */
> -		int total;
> -
> -		total = sb->s_nr_dentry_unused + sb->s_nr_inodes_unused + 1;
> -		count = (nr_to_scan * sb->s_nr_dentry_unused) / total;
> +	if (sb->s_op && sb->s_op->nr_cached_objects)
> +		fs_objects = sb->s_op->nr_cached_objects(sb);
>  
> -		/* prune dcache first as icache is pinned by it */
> -		prune_dcache_sb(sb, count);
> -		prune_icache_sb(sb, nr_to_scan - count);
> +	total_objects = sb->s_nr_dentry_unused +
> +			sb->s_nr_inodes_unused + fs_objects + 1;
> +	if (nr_to_scan) {
> +		int	dentries;
> +		int	inodes;
> +
> +		/* proportion the scan between the cache__ */
> +		dentries = (nr_to_scan * sb->s_nr_dentry_unused) /
> +							total_objects;
> +		inodes = (nr_to_scan * sb->s_nr_inodes_unused) /
> +							total_objects;
> +		if (fs_objects)
> +			fs_objects = (nr_to_scan * fs_objects) /
> +							total_objects;
> +		/*
> +		 * prune the dcache first as the icache is pinned by it, then
> +		 * prune the icache, followed by the filesystem specific caches
> +		 */
> +		prune_dcache_sb(sb, dentries);
> +		prune_icache_sb(sb, inodes);
> +		if (sb->s_op && sb->s_op->free_cached_objects) {

Under which circumstances is a NULL ->free_cached_objects valid?

> +			sb->s_op->free_cached_objects(sb, fs_objects);
> +			fs_objects = sb->s_op->nr_cached_objects(sb);
> +		}
> +		total_objects = sb->s_nr_dentry_unused +
> +				sb->s_nr_inodes_unused + fs_objects;
>  	}

The return value from ->free_cached_objects() doesn't actually get
used.  Instead the code calls ->nr_cached_objects() twice.


> -	count = ((sb->s_nr_dentry_unused + sb->s_nr_inodes_unused) / 100)
> -						* sysctl_vfs_cache_pressure;
> +	total_objects = (total_objects / 100) * sysctl_vfs_cache_pressure;
>  	up_read(&sb->s_umount);
> -	return count;
> +	return total_objects;
>  }
>  
>  /**
> diff --git a/include/linux/fs.h b/include/linux/fs.h
> index 5bff2dc..efcdcc6 100644
> --- a/include/linux/fs.h
> +++ b/include/linux/fs.h
> @@ -1590,6 +1590,17 @@ struct super_operations {
>  	ssize_t (*quota_write)(struct super_block *, int, const char *, size_t, loff_t);
>  #endif
>  	int (*bdev_try_to_free_page)(struct super_block*, struct page*, gfp_t);
> +
> +	/*
> +	 * memory shrinker operations.
> +	 * ->nr_cached_objects() should return the number of freeable cached
> +	 * objects the filesystem holds.
> +	 * ->free_cache_objects() should attempt to free the number of cached
> +	 * objects indicated. It should return how many objects it attempted to
> +	 * free.
> +	 */

I'd have thought that ->free_cache_objects() would always return the
number which it was passed.  Unless someone asked it to scan more
objects than exist, perhaps.

> +	int (*nr_cached_objects)(struct super_block *);
> +	int (*free_cached_objects)(struct super_block *, int);
>  };


WARNING: multiple messages have this Message-ID (diff)
From: Andrew Morton <akpm@linux-foundation.org>
To: Dave Chinner <david@fromorbit.com>
Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org,
	linux-mm@kvack.org, xfs@oss.sgi.com
Subject: Re: [PATCH 4/5] superblock: add filesystem shrinker operations
Date: Thu, 27 May 2010 13:32:39 -0700	[thread overview]
Message-ID: <20100527133239.5d038d9c.akpm@linux-foundation.org> (raw)
In-Reply-To: <1274777588-21494-5-git-send-email-david@fromorbit.com>

On Tue, 25 May 2010 18:53:07 +1000
Dave Chinner <david@fromorbit.com> wrote:

> From: Dave Chinner <dchinner@redhat.com>
> 
> Now we have a per-superblock shrinker implementation, we can add a
> filesystem specific callout to it to allow filesystem internal
> caches to be shrunk by the superblock shrinker.
> 
> Rather than perpetuate the multipurpose shrinker callback API (i.e.
> nr_to_scan == 0 meaning "tell me how many objects freeable in the
> cache), two operations will be added. The first will return the
> number of objects that are freeable, the second is the actual
> shrinker call.
> 
>
> ...
>
>  static int prune_super(struct shrinker *shrink, int nr_to_scan, gfp_t gfp_mask)
>  {
>  	struct super_block *sb;
> -	int count;
> +	int	fs_objects = 0;
> +	int	total_objects;
>  
>  	sb = container_of(shrink, struct super_block, s_shrink);
>  
> @@ -63,22 +64,40 @@ static int prune_super(struct shrinker *shrink, int nr_to_scan, gfp_t gfp_mask)
>  		return -1;
>  	}
>  
> -	if (nr_to_scan) {
> -		/* proportion the scan between the two cache__ */
> -		int total;
> -
> -		total = sb->s_nr_dentry_unused + sb->s_nr_inodes_unused + 1;
> -		count = (nr_to_scan * sb->s_nr_dentry_unused) / total;
> +	if (sb->s_op && sb->s_op->nr_cached_objects)
> +		fs_objects = sb->s_op->nr_cached_objects(sb);
>  
> -		/* prune dcache first as icache is pinned by it */
> -		prune_dcache_sb(sb, count);
> -		prune_icache_sb(sb, nr_to_scan - count);
> +	total_objects = sb->s_nr_dentry_unused +
> +			sb->s_nr_inodes_unused + fs_objects + 1;
> +	if (nr_to_scan) {
> +		int	dentries;
> +		int	inodes;
> +
> +		/* proportion the scan between the cache__ */
> +		dentries = (nr_to_scan * sb->s_nr_dentry_unused) /
> +							total_objects;
> +		inodes = (nr_to_scan * sb->s_nr_inodes_unused) /
> +							total_objects;
> +		if (fs_objects)
> +			fs_objects = (nr_to_scan * fs_objects) /
> +							total_objects;
> +		/*
> +		 * prune the dcache first as the icache is pinned by it, then
> +		 * prune the icache, followed by the filesystem specific caches
> +		 */
> +		prune_dcache_sb(sb, dentries);
> +		prune_icache_sb(sb, inodes);
> +		if (sb->s_op && sb->s_op->free_cached_objects) {

Under which circumstances is a NULL ->free_cached_objects valid?

> +			sb->s_op->free_cached_objects(sb, fs_objects);
> +			fs_objects = sb->s_op->nr_cached_objects(sb);
> +		}
> +		total_objects = sb->s_nr_dentry_unused +
> +				sb->s_nr_inodes_unused + fs_objects;
>  	}

The return value from ->free_cached_objects() doesn't actually get
used.  Instead the code calls ->nr_cached_objects() twice.


> -	count = ((sb->s_nr_dentry_unused + sb->s_nr_inodes_unused) / 100)
> -						* sysctl_vfs_cache_pressure;
> +	total_objects = (total_objects / 100) * sysctl_vfs_cache_pressure;
>  	up_read(&sb->s_umount);
> -	return count;
> +	return total_objects;
>  }
>  
>  /**
> diff --git a/include/linux/fs.h b/include/linux/fs.h
> index 5bff2dc..efcdcc6 100644
> --- a/include/linux/fs.h
> +++ b/include/linux/fs.h
> @@ -1590,6 +1590,17 @@ struct super_operations {
>  	ssize_t (*quota_write)(struct super_block *, int, const char *, size_t, loff_t);
>  #endif
>  	int (*bdev_try_to_free_page)(struct super_block*, struct page*, gfp_t);
> +
> +	/*
> +	 * memory shrinker operations.
> +	 * ->nr_cached_objects() should return the number of freeable cached
> +	 * objects the filesystem holds.
> +	 * ->free_cache_objects() should attempt to free the number of cached
> +	 * objects indicated. It should return how many objects it attempted to
> +	 * free.
> +	 */

I'd have thought that ->free_cache_objects() would always return the
number which it was passed.  Unless someone asked it to scan more
objects than exist, perhaps.

> +	int (*nr_cached_objects)(struct super_block *);
> +	int (*free_cached_objects)(struct super_block *, int);
>  };

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

  reply	other threads:[~2010-05-27 20:30 UTC|newest]

Thread overview: 133+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2010-05-25  8:53 [PATCH 0/5] Per superblock shrinkers V2 Dave Chinner
2010-05-25  8:53 ` Dave Chinner
2010-05-25  8:53 ` Dave Chinner
2010-05-25  8:53 ` [PATCH 1/5] inode: Make unused inode LRU per superblock Dave Chinner
2010-05-25  8:53   ` Dave Chinner
2010-05-25  8:53   ` Dave Chinner
2010-05-26 16:17   ` Nick Piggin
2010-05-26 16:17     ` Nick Piggin
2010-05-26 16:17     ` Nick Piggin
2010-05-26 23:01     ` Dave Chinner
2010-05-26 23:01       ` Dave Chinner
2010-05-26 23:01       ` Dave Chinner
2010-05-26 23:01       ` Dave Chinner
2010-05-27  2:04       ` Nick Piggin
2010-05-27  2:04         ` Nick Piggin
2010-05-27  2:04         ` Nick Piggin
2010-05-27  2:04         ` Nick Piggin
2010-05-27  4:02         ` Dave Chinner
2010-05-27  4:02           ` Dave Chinner
2010-05-27  4:02           ` Dave Chinner
2010-05-27  4:02           ` Dave Chinner
2010-05-27  4:23           ` Nick Piggin
2010-05-27  4:23             ` Nick Piggin
2010-05-27  4:23             ` Nick Piggin
2010-05-27  4:23             ` Nick Piggin
2010-05-27 20:32   ` Andrew Morton
2010-05-27 20:32     ` Andrew Morton
2010-05-27 20:32     ` Andrew Morton
2010-05-27 22:54     ` Dave Chinner
2010-05-27 22:54       ` Dave Chinner
2010-05-27 22:54       ` Dave Chinner
2010-05-28 10:07       ` Nick Piggin
2010-05-28 10:07         ` Nick Piggin
2010-05-28 10:07         ` Nick Piggin
2010-05-25  8:53 ` [PATCH 2/5] mm: add context argument to shrinker callback Dave Chinner
2010-05-25  8:53   ` Dave Chinner
2010-05-25  8:53   ` Dave Chinner
2010-05-25  8:53 ` [PATCH 3/5] superblock: introduce per-sb cache shrinker infrastructure Dave Chinner
2010-05-25  8:53   ` Dave Chinner
2010-05-25  8:53   ` Dave Chinner
2010-05-25  8:53   ` Dave Chinner
2010-05-26 16:41   ` Nick Piggin
2010-05-26 16:41     ` Nick Piggin
2010-05-26 16:41     ` Nick Piggin
2010-05-26 16:41     ` Nick Piggin
2010-05-26 23:12     ` Dave Chinner
2010-05-26 23:12       ` Dave Chinner
2010-05-26 23:12       ` Dave Chinner
2010-05-26 23:12       ` Dave Chinner
2010-05-27  1:53       ` [PATCH 3/5 v2] " Dave Chinner
2010-05-27  1:53         ` Dave Chinner
2010-05-27  1:53         ` Dave Chinner
2010-05-27  4:01         ` Al Viro
2010-05-27  4:01           ` Al Viro
2010-05-27  4:01           ` Al Viro
2010-05-27  6:17           ` Dave Chinner
2010-05-27  6:17             ` Dave Chinner
2010-05-27  6:17             ` Dave Chinner
2010-05-27  6:46             ` Nick Piggin
2010-05-27  6:46               ` Nick Piggin
2010-05-27  6:46               ` Nick Piggin
2010-05-27  2:19       ` [PATCH 3/5] " Nick Piggin
2010-05-27  2:19         ` Nick Piggin
2010-05-27  2:19         ` Nick Piggin
2010-05-27  2:19         ` Nick Piggin
2010-05-27  4:07         ` Dave Chinner
2010-05-27  4:07           ` Dave Chinner
2010-05-27  4:07           ` Dave Chinner
2010-05-27  4:24           ` Nick Piggin
2010-05-27  4:24             ` Nick Piggin
2010-05-27  4:24             ` Nick Piggin
2010-05-27  6:35   ` Nick Piggin
2010-05-27  6:35     ` Nick Piggin
2010-05-27  6:35     ` Nick Piggin
2010-05-27  6:35     ` Nick Piggin
2010-05-27 22:40     ` Dave Chinner
2010-05-27 22:40       ` Dave Chinner
2010-05-27 22:40       ` Dave Chinner
2010-05-27 22:40       ` Dave Chinner
2010-05-28  5:19       ` Nick Piggin
2010-05-28  5:19         ` Nick Piggin
2010-05-28  5:19         ` Nick Piggin
2010-05-28  5:19         ` Nick Piggin
2010-05-31  6:39         ` Dave Chinner
2010-05-31  6:39           ` Dave Chinner
2010-05-31  6:39           ` Dave Chinner
2010-05-31  6:39           ` Dave Chinner
2010-05-31  7:28           ` Nick Piggin
2010-05-31  7:28             ` Nick Piggin
2010-05-31  7:28             ` Nick Piggin
2010-05-31  7:28             ` Nick Piggin
2010-05-27 20:32   ` Andrew Morton
2010-05-27 20:32     ` Andrew Morton
2010-05-27 20:32     ` Andrew Morton
2010-05-27 23:01     ` Dave Chinner
2010-05-27 23:01       ` Dave Chinner
2010-05-27 23:01       ` Dave Chinner
2010-05-25  8:53 ` [PATCH 4/5] superblock: add filesystem shrinker operations Dave Chinner
2010-05-25  8:53   ` Dave Chinner
2010-05-25  8:53   ` Dave Chinner
2010-05-25  8:53   ` Dave Chinner
2010-05-27 20:32   ` Andrew Morton [this message]
2010-05-27 20:32     ` Andrew Morton
2010-05-27 20:32     ` Andrew Morton
2010-05-25  8:53 ` [PATCH 5/5] xfs: make use of new shrinker callout Dave Chinner
2010-05-25  8:53   ` Dave Chinner
2010-05-25  8:53   ` Dave Chinner
2010-05-26 16:44 ` [PATCH 0/5] Per superblock shrinkers V2 Nick Piggin
2010-05-26 16:44   ` Nick Piggin
2010-05-26 16:44   ` Nick Piggin
2010-05-27 20:32 ` Andrew Morton
2010-05-27 20:32   ` Andrew Morton
2010-05-27 20:32   ` Andrew Morton
2010-05-28  0:30   ` Dave Chinner
2010-05-28  0:30     ` Dave Chinner
2010-05-28  0:30     ` Dave Chinner
2010-05-28  7:42   ` Artem Bityutskiy
2010-05-28  7:42     ` Artem Bityutskiy
2010-05-28  7:42     ` Artem Bityutskiy
2010-05-28  7:42     ` Artem Bityutskiy
2010-07-02 12:13 ` Christoph Hellwig
2010-07-02 12:13   ` Christoph Hellwig
2010-07-02 12:13   ` Christoph Hellwig
2010-07-12  2:41   ` Dave Chinner
2010-07-12  2:41     ` Dave Chinner
2010-07-12  2:41     ` Dave Chinner
2010-07-12  2:52     ` Christoph Hellwig
2010-07-12  2:52       ` Christoph Hellwig
2010-07-12  2:52       ` Christoph Hellwig
  -- strict thread matches above, loose matches on Subject: below --
2010-05-14  7:24 [PATCH 0/5] Per-superblock shrinkers Dave Chinner
2010-05-14  7:24 ` [PATCH 4/5] superblock: add filesystem shrinker operations Dave Chinner
2010-05-14  7:24   ` Dave Chinner
2010-05-14  7:24   ` Dave Chinner
2010-05-14  7:24   ` Dave Chinner

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20100527133239.5d038d9c.akpm@linux-foundation.org \
    --to=akpm@linux-foundation.org \
    --cc=david@fromorbit.com \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=xfs@oss.sgi.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.