new dm-bufio

new dm-bufio

[Mikulas Patocka]

I uploaded new dm-bufio patch at 
http://people.redhat.com/mpatocka/patches/kernel/dm-thinp-bufio/dm-bufio.patch

It reports the number of allocated bytes in 
"/sys/module/dm_bufio/parameters/". There is a file "total_allocated" that 
contains the number of bytes allocated for buffer cache. There are files 
"allocated_get_free_pages", "allocated_kmalloc", "allocated_vmalloc" that 
contain a number of allocated bytes with each method.

BTW. if you store "1" in "cache_size", it switches to a mode where it 
won't allocate any memory during operation, relying only on preallocated 
reserves --- useful for testing reservation deadlocks.

Mikulas

Re: new dm-bufio

[Joe Thornber]

On Mon, Aug 22, 2011 at 04:05:28PM -0400, Mikulas Patocka wrote:
> I uploaded new dm-bufio patch at 
> http://people.redhat.com/mpatocka/patches/kernel/dm-thinp-bufio/dm-bufio.patch
> 
> It reports the number of allocated bytes in 
> "/sys/module/dm_bufio/parameters/". There is a file "total_allocated" that 
> contains the number of bytes allocated for buffer cache. There are files 
> "allocated_get_free_pages", "allocated_kmalloc", "allocated_vmalloc" that 
> contain a number of allocated bytes with each method.
> 
> BTW. if you store "1" in "cache_size", it switches to a mode where it 
> won't allocate any memory during operation, relying only on preallocated 
> reserves --- useful for testing reservation deadlocks.

Great stuff, will merge straight away.

Are these stats broken down by bufio client?  ie. can we tell which pool/thinps are consuming all the space?

Would it be possible to record a high watermark as well?  This would
be very useful for benchmarking.  eg, we'd run a heavy load, then be
able to get the peak memory usage.  (Resetting this hwm would also be useful).

- Joe

Re: new dm-bufio

[Alasdair G Kergon]

On Tue, Aug 23, 2011 at 12:30:48PM +0100, Joe Thornber wrote:
> Are these stats broken down by bufio client?

That can indeed be done later, but it's a bigger job to do it properly I think,
with some core dm changes to support callbacks for target-based sysfs
attributes on mapped devices.

My only esssential upstream requirement for the initial release is that the
total memory usage of this module is easily monitored by people who aren't dm
developers.

Alasdair

Re: new dm-bufio

[Mikulas Patocka]

On Tue, 23 Aug 2011, Joe Thornber wrote:

> On Mon, Aug 22, 2011 at 04:05:28PM -0400, Mikulas Patocka wrote:
> > I uploaded new dm-bufio patch at 
> > http://people.redhat.com/mpatocka/patches/kernel/dm-thinp-bufio/dm-bufio.patch
> > 
> > It reports the number of allocated bytes in 
> > "/sys/module/dm_bufio/parameters/". There is a file "total_allocated" that 
> > contains the number of bytes allocated for buffer cache. There are files 
> > "allocated_get_free_pages", "allocated_kmalloc", "allocated_vmalloc" that 
> > contain a number of allocated bytes with each method.
> > 
> > BTW. if you store "1" in "cache_size", it switches to a mode where it 
> > won't allocate any memory during operation, relying only on preallocated 
> > reserves --- useful for testing reservation deadlocks.
> 
> Great stuff, will merge straight away.
> 
> Are these stats broken down by bufio client?  ie. can we tell which 
> pool/thinps are consuming all the space?

No, it would me much more complicated to export such numbers.

> Would it be possible to record a high watermark as well?  This would
> be very useful for benchmarking.  eg, we'd run a heavy load, then be
> able to get the peak memory usage.  (Resetting this hwm would also be useful).

I uploaded new version to
http://people.redhat.com/mpatocka/patches/kernel/dm-thinp-bufio/dm-bufio.patch

There is file "/sys/module/dm_bufio/parameters/peak_allocated" which 
contains peak memory consumption. You can reset the value by writing 0 to 
the file.

Mikulas


> - Joe
> 

Re: new dm-bufio

[Christoph Hellwig]

FYI, the "shrinker" interface in include/linux/shrinker.h is how the
VM tells subsystems to shrink their memory usage.  It is used for
various slab caches, and the XFS buffercache in
fs/xfs/linux-2.6/xfs_buf.c

Re: new dm-bufio

[Mikulas Patocka]

On Tue, 23 Aug 2011, Christoph Hellwig wrote:

> FYI, the "shrinker" interface in include/linux/shrinker.h is how the
> VM tells subsystems to shrink their memory usage.  It is used for
> various slab caches, and the XFS buffercache in
> fs/xfs/linux-2.6/xfs_buf.c

BTW. shrinker functions from xfs_qm.c - xfs_qm_shake and 
xfs_qm_shake_freelist - look completely bogus. They don't read 
sc->nr_to_scan to determine how many values they should scan (and compare 
it with zero to test if they should scan at all) and they return the 
number of entries freed - the interface specifies that they should return 
the number of entries remaining the in cache.

Mikulas

Re: [dm-devel] new dm-bufio

[Mikulas Patocka]

On Tue, 23 Aug 2011, Christoph Hellwig wrote:

> FYI, the "shrinker" interface in include/linux/shrinker.h is how the
> VM tells subsystems to shrink their memory usage.  It is used for
> various slab caches, and the XFS buffercache in
> fs/xfs/linux-2.6/xfs_buf.c

BTW. shrinker functions from xfs_qm.c - xfs_qm_shake and 
xfs_qm_shake_freelist - look completely bogus. They don't read 
sc->nr_to_scan to determine how many values they should scan (and compare 
it with zero to test if they should scan at all) and they return the 
number of entries freed - the interface specifies that they should return 
the number of entries remaining the in cache.

Mikulas

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [xfs-masters] new dm-bufio

[Dave Chinner]

On Mon, Aug 29, 2011 at 02:53:51PM -0400, Mikulas Patocka wrote:
> On Tue, 23 Aug 2011, Christoph Hellwig wrote:
> 
> > FYI, the "shrinker" interface in include/linux/shrinker.h is how the
> > VM tells subsystems to shrink their memory usage.  It is used for
> > various slab caches, and the XFS buffercache in
> > fs/xfs/linux-2.6/xfs_buf.c
> 
> BTW. shrinker functions from xfs_qm.c - xfs_qm_shake and 
> xfs_qm_shake_freelist - look completely bogus. They don't read 
> sc->nr_to_scan to determine how many values they should scan (and compare 
> it with zero to test if they should scan at all) and they return the 
> number of entries freed - the interface specifies that they should return 
> the number of entries remaining the in cache.

It's a relic from the days of Irix that we've kept around and never
really touched because it's never shown up as a problem. Such code
is cleaned up as we go along or as problems arise.

As it is, this patchset:

http://thread.gmane.org/gmane.linux.kernel.mm/67326

that I'm currently working on changes the shrinker API to be much
more sane - the shrinker now has a separate "count objects"
operation and the "scan objects" operation now returns the number of
items freed rather than the number of items left in the cache.

As the API is being changed, the xfs_qm shrinker is also fixed to
return the right values according to the new API. It still doesn't
pay attention to nr_to_scan, but that's not a big deal and will get
addressed in future when other upcoming changes are made to the
code....

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

Re: [xfs-masters] [dm-devel] new dm-bufio

[Dave Chinner]

On Mon, Aug 29, 2011 at 02:53:51PM -0400, Mikulas Patocka wrote:
> On Tue, 23 Aug 2011, Christoph Hellwig wrote:
> 
> > FYI, the "shrinker" interface in include/linux/shrinker.h is how the
> > VM tells subsystems to shrink their memory usage.  It is used for
> > various slab caches, and the XFS buffercache in
> > fs/xfs/linux-2.6/xfs_buf.c
> 
> BTW. shrinker functions from xfs_qm.c - xfs_qm_shake and 
> xfs_qm_shake_freelist - look completely bogus. They don't read 
> sc->nr_to_scan to determine how many values they should scan (and compare 
> it with zero to test if they should scan at all) and they return the 
> number of entries freed - the interface specifies that they should return 
> the number of entries remaining the in cache.

It's a relic from the days of Irix that we've kept around and never
really touched because it's never shown up as a problem. Such code
is cleaned up as we go along or as problems arise.

As it is, this patchset:

http://thread.gmane.org/gmane.linux.kernel.mm/67326

that I'm currently working on changes the shrinker API to be much
more sane - the shrinker now has a separate "count objects"
operation and the "scan objects" operation now returns the number of
items freed rather than the number of items left in the cache.

As the API is being changed, the xfs_qm shrinker is also fixed to
return the right values according to the new API. It still doesn't
pay attention to nr_to_scan, but that's not a big deal and will get
addressed in future when other upcoming changes are made to the
code....

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [xfs-masters] new dm-bufio

[Christoph Hellwig]

On Tue, Aug 30, 2011 at 11:49:06AM +1000, Dave Chinner wrote:
> 
> It's a relic from the days of Irix that we've kept around and never
> really touched because it's never shown up as a problem. Such code
> is cleaned up as we go along or as problems arise.
> 
> As it is, this patchset:
> 
> http://thread.gmane.org/gmane.linux.kernel.mm/67326
> 
> that I'm currently working on changes the shrinker API to be much
> more sane - the shrinker now has a separate "count objects"
> operation and the "scan objects" operation now returns the number of
> items freed rather than the number of items left in the cache.

I think if you find bugs like that they should be in patches separate
from the main conversion.  Especially given that the 3.2 merge window
is getting closer a first batch of patches with these fixes and some
of the preparatory icache/dcache changes that can go into linux-next
ASAP would be nice.

Re: [xfs-masters] [dm-devel] new dm-bufio

[Christoph Hellwig]

On Tue, Aug 30, 2011 at 11:49:06AM +1000, Dave Chinner wrote:
> 
> It's a relic from the days of Irix that we've kept around and never
> really touched because it's never shown up as a problem. Such code
> is cleaned up as we go along or as problems arise.
> 
> As it is, this patchset:
> 
> http://thread.gmane.org/gmane.linux.kernel.mm/67326
> 
> that I'm currently working on changes the shrinker API to be much
> more sane - the shrinker now has a separate "count objects"
> operation and the "scan objects" operation now returns the number of
> items freed rather than the number of items left in the cache.

I think if you find bugs like that they should be in patches separate
from the main conversion.  Especially given that the 3.2 merge window
is getting closer a first batch of patches with these fixes and some
of the preparatory icache/dcache changes that can go into linux-next
ASAP would be nice.

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: new dm-bufio

[Joe Thornber]

On Tue, Aug 23, 2011 at 10:16:03AM -0400, Mikulas Patocka wrote:
> I uploaded new version to
> http://people.redhat.com/mpatocka/patches/kernel/dm-thinp-bufio/dm-bufio.patch
> 
> There is file "/sys/module/dm_bufio/parameters/peak_allocated" which 
> contains peak memory consumption. You can reset the value by writing 0 to 
> the file.

Thanks, Mikulas :)