All of lore.kernel.org
 help / color / mirror / Atom feed
From: Lee Schermerhorn <Lee.Schermerhorn@hp.com>
To: Christoph Lameter <clameter@sgi.com>
Cc: Mel Gorman <mel@csn.ul.ie>,
	akpm@linux-foundatin.org, linux-kernel@vger.kernel.org,
	linux-mm@kvack.org
Subject: Re: [patch 00/23] Slab defragmentation V6
Date: Thu, 08 Nov 2007 15:58:15 -0500	[thread overview]
Message-ID: <1194555495.5295.27.camel@localhost> (raw)
In-Reply-To: <Pine.LNX.4.64.0711081103360.8954@schroedinger.engr.sgi.com>

On Thu, 2007-11-08 at 11:12 -0800, Christoph Lameter wrote:
> On Thu, 8 Nov 2007, Mel Gorman wrote:
> 
> > On Tue, 2007-11-06 at 17:11 -0800, Christoph Lameter wrote:
> > > Slab defragmentation is mainly an issue if Linux is used as a fileserver
> > 
> > Was hoping this would get renamed to SLUB Targetted Reclaim from
> > discussions at VM Summit. As no copying is taking place, it's confusing
> > to call it defragmentation to me anyway. Not a major deal but it made
> > reading the patches a little confusing.
> 
> The problem is that people are focusing on one feature here and forget 
> about the rest. Targetted reclaim is one feature that was added later when 
> lumpy reclaim was added to the kernel. The primary intend of this patchset 
> was always to reduce the fragmentation. The name is appropriate and the 
> patchset will support copying of objects as soon as support for that is 
> added to the kick(). In that case the copying you are looking for will be 
> there. The simple implementation for the kick() methods is to simply copy
> pieces of the reclaim code. That is what is included here.
> 
> > > With lumpy reclaim slab defragmentation can be used to enhance the
> > > ability to recover larger contiguous areas of memory. Lumpy reclaim currently
> > > cannot do anything if a slab page is encountered. With slab defragmentation
> > > that slab page can be removed and a large contiguous page freed. It may
> > > be possible to have slab pages also part of ZONE_MOVABLE (Mel's defrag
> > > scheme in 2.6.23)
> > 
> > More terminology nit-pick - ZONE_MOVABLE is not defragmenting anything.
> > It's just partitioning memory. The slab pages need to be 100%
> > reclaimable or movable for that to happen but even with targetted
> > reclaim, some dentries such as the root directory one cannot be
> > reclaimed, right?
> 
> 100%? I am so fond of these categorical statements ....
> 
> ZONE_MOVABLE also contains mlocked pages that are also not reclaimable. 
> The question is at what level would it be possible to make them MOVABLE? 
> It may take some improvements to the kick() methods to make eviction more 
> reliable. Allowing the moving of objects in the kick() methods will 
> likely get usthere.

Christoph:  Although mlocked pages are not reclaimable, they ARE
migratable.  You fixed that a long time ago.  [And I just verified with
memtoy.]  Doesn't this make them "movable"?

Lee


WARNING: multiple messages have this Message-ID (diff)
From: Lee Schermerhorn <Lee.Schermerhorn@hp.com>
To: Christoph Lameter <clameter@sgi.com>
Cc: Mel Gorman <mel@csn.ul.ie>,
	akpm@linux-foundatin.org, linux-kernel@vger.kernel.org,
	linux-mm@kvack.org
Subject: Re: [patch 00/23] Slab defragmentation V6
Date: Thu, 08 Nov 2007 15:58:15 -0500	[thread overview]
Message-ID: <1194555495.5295.27.camel@localhost> (raw)
In-Reply-To: <Pine.LNX.4.64.0711081103360.8954@schroedinger.engr.sgi.com>

On Thu, 2007-11-08 at 11:12 -0800, Christoph Lameter wrote:
> On Thu, 8 Nov 2007, Mel Gorman wrote:
> 
> > On Tue, 2007-11-06 at 17:11 -0800, Christoph Lameter wrote:
> > > Slab defragmentation is mainly an issue if Linux is used as a fileserver
> > 
> > Was hoping this would get renamed to SLUB Targetted Reclaim from
> > discussions at VM Summit. As no copying is taking place, it's confusing
> > to call it defragmentation to me anyway. Not a major deal but it made
> > reading the patches a little confusing.
> 
> The problem is that people are focusing on one feature here and forget 
> about the rest. Targetted reclaim is one feature that was added later when 
> lumpy reclaim was added to the kernel. The primary intend of this patchset 
> was always to reduce the fragmentation. The name is appropriate and the 
> patchset will support copying of objects as soon as support for that is 
> added to the kick(). In that case the copying you are looking for will be 
> there. The simple implementation for the kick() methods is to simply copy
> pieces of the reclaim code. That is what is included here.
> 
> > > With lumpy reclaim slab defragmentation can be used to enhance the
> > > ability to recover larger contiguous areas of memory. Lumpy reclaim currently
> > > cannot do anything if a slab page is encountered. With slab defragmentation
> > > that slab page can be removed and a large contiguous page freed. It may
> > > be possible to have slab pages also part of ZONE_MOVABLE (Mel's defrag
> > > scheme in 2.6.23)
> > 
> > More terminology nit-pick - ZONE_MOVABLE is not defragmenting anything.
> > It's just partitioning memory. The slab pages need to be 100%
> > reclaimable or movable for that to happen but even with targetted
> > reclaim, some dentries such as the root directory one cannot be
> > reclaimed, right?
> 
> 100%? I am so fond of these categorical statements ....
> 
> ZONE_MOVABLE also contains mlocked pages that are also not reclaimable. 
> The question is at what level would it be possible to make them MOVABLE? 
> It may take some improvements to the kick() methods to make eviction more 
> reliable. Allowing the moving of objects in the kick() methods will 
> likely get usthere.

Christoph:  Although mlocked pages are not reclaimable, they ARE
migratable.  You fixed that a long time ago.  [And I just verified with
memtoy.]  Doesn't this make them "movable"?

Lee

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

  parent reply	other threads:[~2007-11-08 21:20 UTC|newest]

Thread overview: 152+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2007-11-07  1:11 [patch 00/23] Slab defragmentation V6 Christoph Lameter
2007-11-07  1:11 ` Christoph Lameter
2007-11-07  1:11 ` [patch 01/23] SLUB: Move count_partial() Christoph Lameter
2007-11-07  1:11   ` Christoph Lameter
2007-11-07  1:11 ` [patch 02/23] SLUB: Rename NUMA defrag_ratio to remote_node_defrag_ratio Christoph Lameter
2007-11-07  1:11   ` Christoph Lameter
2007-11-08 14:50   ` Mel Gorman
2007-11-08 14:50     ` Mel Gorman
2007-11-08 17:25     ` Matt Mackall
2007-11-08 17:25       ` Matt Mackall
2007-11-08 19:16       ` Christoph Lameter
2007-11-08 19:16         ` Christoph Lameter
2007-11-08 19:47         ` Matt Mackall
2007-11-08 19:47           ` Matt Mackall
2007-11-08 20:01           ` Christoph Lameter
2007-11-08 20:01             ` Christoph Lameter
2007-11-08 21:03             ` Matt Mackall
2007-11-08 21:03               ` Matt Mackall
2007-11-08 21:28               ` Christoph Lameter
2007-11-08 21:28                 ` Christoph Lameter
2007-11-08 23:08                 ` Matt Mackall
2007-11-08 23:08                   ` Matt Mackall
2007-11-08 18:56     ` Christoph Lameter
2007-11-08 18:56       ` Christoph Lameter
2007-11-08 20:10       ` Mel Gorman
2007-11-08 20:10         ` Mel Gorman
2007-11-07  1:11 ` [patch 03/23] bufferhead: Revert constructor removal Christoph Lameter
2007-11-07  1:11   ` Christoph Lameter
2007-11-07  1:11 ` [patch 04/23] dentries: Extract common code to remove dentry from lru Christoph Lameter
2007-11-07  1:11   ` Christoph Lameter
2007-11-07  8:50   ` Johannes Weiner
2007-11-07  8:50     ` Johannes Weiner
2007-11-07  9:43     ` Jörn Engel
2007-11-07  9:43       ` Jörn Engel
2007-11-07 18:55       ` Christoph Lameter
2007-11-07 18:54         ` Jörn Engel
2007-11-07 18:54           ` Jörn Engel
2007-11-07 19:00           ` Christoph Lameter
2007-11-07 18:28     ` Christoph Lameter
2007-11-07 18:28       ` Christoph Lameter
2007-11-07  1:11 ` [patch 05/23] VM: Allow get_page_unless_zero on compound pages Christoph Lameter
2007-11-07  1:11   ` Christoph Lameter
2007-11-07  1:11 ` [patch 06/23] SLUB: Extend slabinfo to support -D and -C options Christoph Lameter
2007-11-07  1:11   ` Christoph Lameter
2007-11-08 15:00   ` Mel Gorman
2007-11-08 15:00     ` Mel Gorman
2007-11-07  1:11 ` [patch 07/23] SLUB: Add defrag_ratio field and sysfs support Christoph Lameter
2007-11-07  1:11   ` Christoph Lameter
2007-11-07  8:55   ` Johannes Weiner
2007-11-07  8:55     ` Johannes Weiner
2007-11-07 18:30     ` Christoph Lameter
2007-11-07 18:30       ` Christoph Lameter
2007-11-08 15:07   ` Mel Gorman
2007-11-08 15:07     ` Mel Gorman
2007-11-08 18:59     ` Christoph Lameter
2007-11-08 18:59       ` Christoph Lameter
2007-11-07  1:11 ` [patch 08/23] SLUB: Replace ctor field with ops field in /sys/slab/:0000008 /sys/slab/:0000016 /sys/slab/:0000024 /sys/slab/:0000032 /sys/slab/:0000040 /sys/slab/:0000048 /sys/slab/:0000056 /sys/slab/:0000064 /sys/slab/:0000072 /sys/slab/:0000080 /sys/slab/:0000088 /sys/slab/:0000096 /sys/slab/:0000104 /sys/slab/:0000128 /sys/slab/:0000144 /sys/slab/:0000184 /sys/slab/:0000192 /sys/slab/:0000216 /sys/slab/:0000256 /sys/slab/:0000344 /sys/slab/:0000384 /sys/slab/:0000448 /sys/slab/:0000512 /sys/slab/:0000768 /sys/slab/:0000968 /sys/slab/:0001024 /sys/slab/:0001152 /sys/slab/:0001312 /sys/slab/:0001536 /sys/slab/:0002048 /sys/slab/:0003072 /sys/slab/:0004096 /sys/slab/:a-0000016 /sys/slab/:a-0000024 /sys/slab/:a-0000056 /sys/slab/:a-0000080 /sys/slab/:a-0000128 /sys/slab/Acpi-Namespace /sys/slab/Acpi-Operand /sys/slab/Acpi-Parse /sys/slab/Acpi-ParseExt /sys/slab/Acpi-State /sys/slab/RAW /sys/slab/TCP /sys/slab/UDP /sys/slab/UDP-Lite /sys/slab/UNIX /sys/slab/anon_vma /s ys/slab/arp_cache /sys/slab/bdev_cache /sys/slab/bio /sys/slab/biovec-1 /sys/slab/biovec-128 /sys/slab/biovec-16 /sys/slab/biovec-256 /sys/slab/biovec-4 /sys/slab/biovec-64 /sys/slab/blkdev_ioc /sys/slab/blkdev_queue /sys/slab/blkdev_requests /sys/slab/buffer_head /sys/slab/cfq_io_context /sys/slab/cfq_queue /sys/slab/dentry /sys/slab/eventpoll_epi /sys/slab/eventpoll_pwq /sys/slab/ext2_inode_cache /sys/slab/ext3_inode_cache /sys/slab/fasync_cache /sys/slab/file_lock_cache /sys/slab/files_cache /sys/slab/filp /sys/slab/flow_cache /sys/slab/fs_cache /sys/slab/idr_layer_cache /sys/slab/inet_peer_cache /sys/slab/inode_cache /sys/slab/inotify_event_cache /sys/slab/inotify_watch_cache /sys/slab/ip_dst_cache /sys/slab/ip_fib_alias /sys/slab/ip_fib_hash /sys/slab/jbd_1k /sys/slab/jbd_4k /sys/slab/journal_handle /sys/slab/journal_head /sys/slab/kiocb /sys/slab/kioctx /sys/slab/kmalloc-1024 /sys/slab/kmalloc-128 /sys/slab/kmalloc-16 /sys/slab/kmalloc-192 /sys/slab/kmalloc-2048 /sys/s lab/kmalloc-256 /sys/slab/kmalloc-32 /sys/slab/kmalloc-512 /sys/slab/kmalloc-64 /sys/slab/kmalloc-8 /sys/slab/kmalloc-96 /sys/slab/mm_struct /sys/slab/mnt_cache /sys/slab/mqueue_inode_cache /sys/slab/names_cache /sys/slab/net_namespace /sys/slab/nfs_direct_cache /sys/slab/nfs_inode_cache /sys/slab/nfs_page /sys/slab/nfs_read_data /sys/slab/nfs_write_data /sys/slab/nfsd4_delegations /sys/slab/nfsd4_files /sys/slab/nfsd4_stateids /sys/slab/nfsd4_stateowners /sys/slab/nsproxy /sys/slab/pid /sys/slab/posix_timers_cache /sys/slab/proc_inode_cache /sys/slab/radix_tree_node /sys/slab/request_sock_TCP /sys/slab/revoke_record /sys/slab/revoke_table /sys/slab/rpc_buffers /sys/slab/rpc_inode_cache /sys/slab/rpc_tasks /sys/slab/scsi_cmd_cache /sys/slab/scsi_io_context /sys/slab/secpath_cache /sys/slab/sgpool-128 /sys/slab/sgpool-16 /sys/slab/sgpool-32 /sys/slab/sgpool-64 /sys/slab/sgpool-8 /sys/slab/shmem_inode_cache /sys/slab/sighand_cache /sys/slab/signal_cache /sys/slab/sigqueue /sys /slab/skbuff_fclone_cache /sys/slab/skbuff_head_cache /sys/slab/sock_inode_cache /sys/slab/sysfs_dir_cache /sys/slab/task_struct /sys/slab/tcp_bind_bucket /sys/slab/tw_sock_TCP /sys/slab/uhci_urb_priv /sys/slab/uid_cache /sys/slab/vm_area_struct /sys/slab/xfrm_dst_cache Christoph Lameter
2007-11-07  1:11   ` [patch 08/23] SLUB: Replace ctor field with ops field in /sys/slab/:0000008 /sys/slab/:0000016 /sys/slab/:0000024 /sys/slab/:0000032 /sys/slab/:0000040 /sys/slab/:0000048 /sys/slab/:0000056 /sys/slab/:0000064 /sys/slab/:0000072 /sys/slab/:0000080 /sys/slab/:0000088 /sys/slab/:0000096 /sys/slab/:0000104 /sys/slab/:0000128 /sys/slab/:0000144 /sys/slab/:0000184 /sys/slab/:0000192 /sys/slab/:0000216 /sys/slab/:0000256 /sys/slab/:0000344 /sys/slab/:0000384 /sys/slab/:0000448 /sys/slab/:0000512 /sys/slab/:0000768 /sys/slab/:0000968 /sys/slab/:0001024 /sys/slab/:0001152 /sys/slab/:0001312 /sys/slab/:0001536 /sys/slab/:0002048 /sys/slab/:0003072 /sys/slab/:0004096 /sys/slab/:a-0000016 /sys/slab/:a-0000024 /sys/slab/:a-0000056 /sys/slab/:a-0000080 /sys/slab/:a-0000128 /sys/slab/Acpi-Namesp ace /sys/slab/Acpi-Operand /sys/slab/Acpi-Parse /sys/slab/Acpi-ParseExt /sys/slab/Acpi-State /sys/ Christoph Lameter
2007-11-07  1:11 ` [patch 09/23] SLUB: Add get() and kick() methods Christoph Lameter
2007-11-07  1:11   ` Christoph Lameter
2007-11-07  2:37   ` Adrian Bunk
2007-11-07  2:37     ` Adrian Bunk
2007-11-07  3:07     ` Christoph Lameter
2007-11-07  3:07       ` Christoph Lameter
2007-11-07  3:26       ` Adrian Bunk
2007-11-07  3:26         ` Adrian Bunk
2007-11-07  1:11 ` [patch 10/23] SLUB: Sort slab cache list and establish maximum objects for defrag slabs Christoph Lameter
2007-11-07  1:11   ` Christoph Lameter
2007-11-07  1:11 ` [patch 11/23] SLUB: Slab defrag core Christoph Lameter
2007-11-07  1:11   ` Christoph Lameter
2007-11-07 22:13   ` Christoph Lameter
2007-11-07 22:13     ` Christoph Lameter
2007-11-07  1:11 ` [patch 12/23] SLUB: Trigger defragmentation from memory reclaim Christoph Lameter
2007-11-07  1:11   ` Christoph Lameter
2007-11-07  9:28   ` Johannes Weiner
2007-11-07  9:28     ` Johannes Weiner
2007-11-07 18:34     ` Christoph Lameter
2007-11-07 18:34       ` Christoph Lameter
2007-11-08 15:12   ` Mel Gorman
2007-11-08 15:12     ` Mel Gorman
2007-11-08 19:00     ` Christoph Lameter
2007-11-08 19:00       ` Christoph Lameter
2007-11-07  1:11 ` [patch 13/23] Buffer heads: Support slab defrag Christoph Lameter
2007-11-07  1:11   ` Christoph Lameter
2007-11-07  1:11 ` [patch 14/23] inodes: Support generic defragmentation Christoph Lameter
2007-11-07  1:11   ` Christoph Lameter
2007-11-07 10:17   ` Jörn Engel
2007-11-07 10:17     ` Jörn Engel
2007-11-07 10:31     ` Jörn Engel
2007-11-07 10:31       ` Jörn Engel
2007-11-07 10:35     ` Andreas Schwab
2007-11-07 10:35       ` Andreas Schwab
2007-11-07 10:35       ` Jörn Engel
2007-11-07 10:35         ` Jörn Engel
2007-11-07 18:40     ` Christoph Lameter
2007-11-07 18:51       ` Jörn Engel
2007-11-07 18:51         ` Jörn Engel
2007-11-07 19:00         ` Christoph Lameter
2007-11-07  1:11 ` [patch 15/23] FS: ExtX filesystem defrag Christoph Lameter
2007-11-07  1:11   ` Christoph Lameter
2007-11-07  1:11 ` [patch 16/23] FS: XFS slab defragmentation Christoph Lameter
2007-11-07  1:11   ` Christoph Lameter
2007-11-07  1:11 ` [patch 17/23] FS: Proc filesystem support for slab defrag Christoph Lameter
2007-11-07  1:11   ` Christoph Lameter
2007-11-07  1:11 ` [patch 18/23] FS: Slab defrag: Reiserfs support Christoph Lameter
2007-11-07  1:11   ` Christoph Lameter
2007-11-07  1:11 ` [patch 19/23] FS: Socket inode defragmentation Christoph Lameter
2007-11-07  1:11   ` Christoph Lameter
2007-11-07  1:11 ` [patch 20/23] dentries: Add constructor Christoph Lameter
2007-11-07  1:11   ` Christoph Lameter
2007-11-08 15:23   ` Mel Gorman
2007-11-08 15:23     ` Mel Gorman
2007-11-08 19:03     ` Christoph Lameter
2007-11-08 19:03       ` Christoph Lameter
2007-11-07  1:11 ` [patch 21/23] dentries: dentry defragmentation Christoph Lameter
2007-11-07  1:11   ` Christoph Lameter
2007-11-07  1:11 ` [patch 22/23] SLUB: Slab reclaim through Lumpy reclaim Christoph Lameter
2007-11-07  1:11   ` Christoph Lameter
2007-11-07  1:11 ` [patch 23/23] SLUB: Add SlabReclaimable() to avoid repeated reclaim attempts Christoph Lameter
2007-11-07  1:11   ` Christoph Lameter
2007-11-07  8:37 ` [patch 00/23] Slab defragmentation V6 Peter Zijlstra
2007-11-07 18:04   ` Christoph Lameter
2007-11-08 15:26 ` Mel Gorman
2007-11-08 15:26   ` Mel Gorman
2007-11-08 16:01   ` Plans for Onezonelist patch series ??? Lee Schermerhorn
2007-11-08 16:01     ` Lee Schermerhorn
2007-11-08 18:34     ` Christoph Lameter
2007-11-08 18:34       ` Christoph Lameter
2007-11-08 18:40       ` Mel Gorman
2007-11-08 18:40         ` Mel Gorman
2007-11-08 18:43         ` Christoph Lameter
2007-11-08 18:43           ` Christoph Lameter
2007-11-08 20:06           ` Mel Gorman
2007-11-08 20:06             ` Mel Gorman
2007-11-08 20:20             ` Christoph Lameter
2007-11-08 20:20               ` Christoph Lameter
2007-11-08 20:29               ` Mel Gorman
2007-11-08 20:29                 ` Mel Gorman
2007-11-08 18:39     ` Mel Gorman
2007-11-08 18:39       ` Mel Gorman
2007-11-08 19:39       ` Christoph Lameter
2007-11-08 19:39         ` Christoph Lameter
2007-11-08 19:12   ` [patch 00/23] Slab defragmentation V6 Christoph Lameter
2007-11-08 19:12     ` Christoph Lameter
2007-11-08 20:24     ` Mel Gorman
2007-11-08 20:24       ` Mel Gorman
2007-11-08 20:28       ` Christoph Lameter
2007-11-08 20:28         ` Christoph Lameter
2007-11-08 20:58     ` Lee Schermerhorn [this message]
2007-11-08 20:58       ` Lee Schermerhorn
2007-11-08 21:27       ` Christoph Lameter
2007-11-08 21:27         ` Christoph Lameter

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1194555495.5295.27.camel@localhost \
    --to=lee.schermerhorn@hp.com \
    --cc=akpm@linux-foundatin.org \
    --cc=clameter@sgi.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=mel@csn.ul.ie \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.