From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from psmtp.com (na3sys010amx106.postini.com [74.125.245.106]) by kanga.kvack.org (Postfix) with SMTP id 9E3FE6B0044 for ; Fri, 23 Mar 2012 04:05:27 -0400 (EDT) Received: by eaal1 with SMTP id l1so1081715eaa.14 for ; Fri, 23 Mar 2012 01:05:25 -0700 (PDT) Message-ID: <4F6C2E9B.9010200@gmail.com> Date: Fri, 23 Mar 2012 16:04:43 +0800 From: bill4carson MIME-Version: 1.0 Subject: Why memory.usage_in_bytes is always increasing after every mmap/dirty/unmap sequence Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Sender: owner-linux-mm@kvack.org List-ID: To: "linux-mm@kvack.org" Hi, all I'm playing with memory cgroup, I'm a bit confused why memory.usage in bytes is steadily increasing at 4K page pace after every mmap/dirty/unmap sequence. On linux-3.6.34.10/linux-3.3.0-rc5 A simple test case does following: a) mmap 128k memory in private anonymous way b) dirty all 128k to demand physical page c) print memory.usage_in_bytes <-- increased at 4K after every loop d) unmap previous 128 memory e) goto a) to repeat And when the test case exit, memory.usage_in_bytes is not *ZERO*, but the previous increased value. I'm puzzled about what I saw, can anyone please give me some tips to understand this? Thanks in advance. -- Love each day! --bill -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/ Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from psmtp.com (na3sys010amx182.postini.com [74.125.245.182]) by kanga.kvack.org (Postfix) with SMTP id 2245C6B0044 for ; Fri, 23 Mar 2012 04:20:48 -0400 (EDT) Received: from m4.gw.fujitsu.co.jp (unknown [10.0.50.74]) by fgwmail5.fujitsu.co.jp (Postfix) with ESMTP id 2C6313EE0C1 for ; Fri, 23 Mar 2012 17:20:46 +0900 (JST) Received: from smail (m4 [127.0.0.1]) by outgoing.m4.gw.fujitsu.co.jp (Postfix) with ESMTP id 0865E45DE4F for ; Fri, 23 Mar 2012 17:20:46 +0900 (JST) Received: from s4.gw.fujitsu.co.jp (s4.gw.fujitsu.co.jp [10.0.50.94]) by m4.gw.fujitsu.co.jp (Postfix) with ESMTP id E40DF45DE4E for ; Fri, 23 Mar 2012 17:20:45 +0900 (JST) Received: from s4.gw.fujitsu.co.jp (localhost.localdomain [127.0.0.1]) by s4.gw.fujitsu.co.jp (Postfix) with ESMTP id C764C1DB8041 for ; Fri, 23 Mar 2012 17:20:45 +0900 (JST) Received: from m107.s.css.fujitsu.com (m107.s.css.fujitsu.com [10.240.81.147]) by s4.gw.fujitsu.co.jp (Postfix) with ESMTP id 6FEDC1DB8047 for ; Fri, 23 Mar 2012 17:20:45 +0900 (JST) Message-ID: <4F6C31F7.2010804@jp.fujitsu.com> Date: Fri, 23 Mar 2012 17:19:03 +0900 From: KAMEZAWA Hiroyuki MIME-Version: 1.0 Subject: Re: Why memory.usage_in_bytes is always increasing after every mmap/dirty/unmap sequence References: <4F6C2E9B.9010200@gmail.com> In-Reply-To: <4F6C2E9B.9010200@gmail.com> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Sender: owner-linux-mm@kvack.org List-ID: To: bill4carson Cc: "linux-mm@kvack.org" (2012/03/23 17:04), bill4carson wrote: > Hi, all > > I'm playing with memory cgroup, I'm a bit confused why > memory.usage in bytes is steadily increasing at 4K page pace > after every mmap/dirty/unmap sequence. > > On linux-3.6.34.10/linux-3.3.0-rc5 > A simple test case does following: > > a) mmap 128k memory in private anonymous way > b) dirty all 128k to demand physical page > c) print memory.usage_in_bytes <-- increased at 4K after every loop > d) unmap previous 128 memory > e) goto a) to repeat In Documentation/cgroup/memory.txt == 5.5 usage_in_bytes For efficiency, as other kernel components, memory cgroup uses some optimization to avoid unnecessary cacheline false sharing. usage_in_bytes is affected by the method and doesn't show 'exact' value of memory(and swap) usage, it's an fuzz value for efficient access. (Of course, when necessary, it's synchronized.) If you want to know more exact memory usage, you should use RSS+CACHE(+SWAP) value in memory.stat(see 5.2). == In current implementation, memcg tries to charge resource in size of 32 pages. So, if you get 32 pages and free 32pages, usage_in_bytes may not change. This is affected by caches in other cpus and other flushing operations caused by some workload in other cgroups. memcg's usage_in_bytes is not precise in 128k degree. - How memory.stat changes ? - What happens when you do test with 4M alloc/free ? Thanks, -Kame -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/ Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from psmtp.com (na3sys010amx101.postini.com [74.125.245.101]) by kanga.kvack.org (Postfix) with SMTP id 0DB516B0044 for ; Fri, 23 Mar 2012 04:44:04 -0400 (EDT) Received: by ghrr18 with SMTP id r18so3206609ghr.14 for ; Fri, 23 Mar 2012 01:44:04 -0700 (PDT) Message-ID: <4F6C37AC.4080101@gmail.com> Date: Fri, 23 Mar 2012 16:43:24 +0800 From: bill4carson MIME-Version: 1.0 Subject: Re: Why memory.usage_in_bytes is always increasing after every mmap/dirty/unmap sequence References: <4F6C2E9B.9010200@gmail.com> <4F6C31F7.2010804@jp.fujitsu.com> In-Reply-To: <4F6C31F7.2010804@jp.fujitsu.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Sender: owner-linux-mm@kvack.org List-ID: To: KAMEZAWA Hiroyuki Cc: "linux-mm@kvack.org" On 2012a1'03ae??23ae?JPY 16:19, KAMEZAWA Hiroyuki wrote: > (2012/03/23 17:04), bill4carson wrote: > >> Hi, all >> >> I'm playing with memory cgroup, I'm a bit confused why >> memory.usage in bytes is steadily increasing at 4K page pace >> after every mmap/dirty/unmap sequence. >> >> On linux-3.6.34.10/linux-3.3.0-rc5 >> A simple test case does following: >> >> a) mmap 128k memory in private anonymous way >> b) dirty all 128k to demand physical page >> c) print memory.usage_in_bytes<-- increased at 4K after every loop >> d) unmap previous 128 memory >> e) goto a) to repeat > > In Documentation/cgroup/memory.txt > == > 5.5 usage_in_bytes > > For efficiency, as other kernel components, memory cgroup uses some optimization > to avoid unnecessary cacheline false sharing. usage_in_bytes is affected by the > method and doesn't show 'exact' value of memory(and swap) usage, it's an fuzz > value for efficient access. (Of course, when necessary, it's synchronized.) > If you want to know more exact memory usage, you should use RSS+CACHE(+SWAP) > value in memory.stat(see 5.2). > == > > In current implementation, memcg tries to charge resource in size of 32 pages. > So, if you get 32 pages and free 32pages, usage_in_bytes may not change. > This is affected by caches in other cpus and other flushing operations caused > by some workload in other cgroups. memcg's usage_in_bytes is not precise in > 128k degree. Thanks for the internal design details. I noticed on 2.6.34, it's checked on every 512 Kbytes See http://lxr.linux.no/#linux+v2.6.34/mm/memcontrol.c#L571 And I haven't see the 3.3.0 changes. > > - How memory.stat changes ? root@localhost:/sys/fs/cgroup/memory/a> cat memory.stat;cat memory.usage_in_bytes cache 0 rss 131072 <------ when mmap/dirty/ mapped_file 0 pgpgin 1278 pgpgout 1246 inactive_anon 0 active_anon 131072 inactive_file 0 active_file 0 unevictable 0 hierarchical_memory_limit 9223372036854775807 total_cache 0 total_rss 131072 total_mapped_file 0 total_pgpgin 1278 total_pgpgout 1246 total_inactive_anon 0 total_active_anon 131072 total_inactive_file 0 total_active_file 0 total_unevictable 0 root@localhost:/sys/fs/cgroup/memory/a> cat memory.stat;cat memory.usage_in_bytes cache 0 rss 4096 <------ when mmap/dirty/unmap mapped_file 0 pgpgin 1278 pgpgout 1277 inactive_anon 0 active_anon 4096 inactive_file 0 active_file 0 unevictable 0 hierarchical_memory_limit 9223372036854775807 total_cache 0 total_rss 4096 total_mapped_file 0 total_pgpgin 1278 total_pgpgout 1277 total_inactive_anon 0 total_active_anon 4096 total_inactive_file 0 total_active_file 0 total_unevictable 0 > - What happens when you do test with 4M alloc/free ? > I tried on 2.6.34, it's the same behavior. > Thanks, > -Kame > > > > > > > > > -- Love each day! --bill -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/ Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from psmtp.com (na3sys010amx207.postini.com [74.125.245.207]) by kanga.kvack.org (Postfix) with SMTP id 415806B0044 for ; Fri, 23 Mar 2012 05:00:23 -0400 (EDT) Received: by obbta14 with SMTP id ta14so2898718obb.14 for ; Fri, 23 Mar 2012 02:00:22 -0700 (PDT) Message-ID: <4F6C3B7F.1070705@gmail.com> Date: Fri, 23 Mar 2012 16:59:43 +0800 From: bill4carson MIME-Version: 1.0 Subject: Re: Why memory.usage_in_bytes is always increasing after every mmap/dirty/unmap sequence References: <4F6C2E9B.9010200@gmail.com> <4F6C31F7.2010804@jp.fujitsu.com> In-Reply-To: <4F6C31F7.2010804@jp.fujitsu.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Sender: owner-linux-mm@kvack.org List-ID: To: KAMEZAWA Hiroyuki Cc: "linux-mm@kvack.org" On 2012a1'03ae??23ae?JPY 16:19, KAMEZAWA Hiroyuki wrote: > (2012/03/23 17:04), bill4carson wrote: > >> Hi, all >> >> I'm playing with memory cgroup, I'm a bit confused why >> memory.usage in bytes is steadily increasing at 4K page pace >> after every mmap/dirty/unmap sequence. >> >> On linux-3.6.34.10/linux-3.3.0-rc5 >> A simple test case does following: >> >> a) mmap 128k memory in private anonymous way >> b) dirty all 128k to demand physical page >> c) print memory.usage_in_bytes<-- increased at 4K after every loop >> d) unmap previous 128 memory >> e) goto a) to repeat > > In Documentation/cgroup/memory.txt > == > 5.5 usage_in_bytes > > For efficiency, as other kernel components, memory cgroup uses some optimization > to avoid unnecessary cacheline false sharing. usage_in_bytes is affected by the > method and doesn't show 'exact' value of memory(and swap) usage, it's an fuzz > value for efficient access. (Of course, when necessary, it's synchronized.) > If you want to know more exact memory usage, you should use RSS+CACHE(+SWAP) > value in memory.stat(see 5.2). > == > > In current implementation, memcg tries to charge resource in size of 32 pages. > So, if you get 32 pages and free 32pages, usage_in_bytes may not change. > This is affected by caches in other cpus and other flushing operations caused > by some workload in other cgroups. memcg's usage_in_bytes is not precise in > 128k degree. > Yes, I tried to mmap/dirty/unmap in 32 times, when the usage_in_bytes reached 128k, it rolls back to 4k again. So it doesn't hurt any more. I haven't found the code regarding to this behavior. > - How memory.stat changes ? > - What happens when you do test with 4M alloc/free ? > > Thanks, > -Kame > > > > > > > > > -- Love each day! --bill -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/ Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from psmtp.com (na3sys010amx199.postini.com [74.125.245.199]) by kanga.kvack.org (Postfix) with SMTP id 31FDA6B0044 for ; Fri, 23 Mar 2012 05:05:46 -0400 (EDT) Received: from m4.gw.fujitsu.co.jp (unknown [10.0.50.74]) by fgwmail6.fujitsu.co.jp (Postfix) with ESMTP id 3E3F53EE0AE for ; Fri, 23 Mar 2012 18:05:44 +0900 (JST) Received: from smail (m4 [127.0.0.1]) by outgoing.m4.gw.fujitsu.co.jp (Postfix) with ESMTP id 2148445DE52 for ; Fri, 23 Mar 2012 18:05:44 +0900 (JST) Received: from s4.gw.fujitsu.co.jp (s4.gw.fujitsu.co.jp [10.0.50.94]) by m4.gw.fujitsu.co.jp (Postfix) with ESMTP id 0672D45DE4E for ; Fri, 23 Mar 2012 18:05:44 +0900 (JST) Received: from s4.gw.fujitsu.co.jp (localhost.localdomain [127.0.0.1]) by s4.gw.fujitsu.co.jp (Postfix) with ESMTP id DFE8E1DB8041 for ; Fri, 23 Mar 2012 18:05:43 +0900 (JST) Received: from m107.s.css.fujitsu.com (m107.s.css.fujitsu.com [10.240.81.147]) by s4.gw.fujitsu.co.jp (Postfix) with ESMTP id 87FA91DB802F for ; Fri, 23 Mar 2012 18:05:43 +0900 (JST) Message-ID: <4F6C3C88.5090800@jp.fujitsu.com> Date: Fri, 23 Mar 2012 18:04:08 +0900 From: KAMEZAWA Hiroyuki MIME-Version: 1.0 Subject: Re: Why memory.usage_in_bytes is always increasing after every mmap/dirty/unmap sequence References: <4F6C2E9B.9010200@gmail.com> <4F6C31F7.2010804@jp.fujitsu.com> <4F6C3B7F.1070705@gmail.com> In-Reply-To: <4F6C3B7F.1070705@gmail.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Sender: owner-linux-mm@kvack.org List-ID: To: bill4carson Cc: "linux-mm@kvack.org" (2012/03/23 17:59), bill4carson wrote: > > > On 2012a1'03ae??23ae?JPY 16:19, KAMEZAWA Hiroyuki wrote: >> (2012/03/23 17:04), bill4carson wrote: >> >>> Hi, all >>> >>> I'm playing with memory cgroup, I'm a bit confused why >>> memory.usage in bytes is steadily increasing at 4K page pace >>> after every mmap/dirty/unmap sequence. >>> >>> On linux-3.6.34.10/linux-3.3.0-rc5 >>> A simple test case does following: >>> >>> a) mmap 128k memory in private anonymous way >>> b) dirty all 128k to demand physical page >>> c) print memory.usage_in_bytes<-- increased at 4K after every loop >>> d) unmap previous 128 memory >>> e) goto a) to repeat >> >> In Documentation/cgroup/memory.txt >> == >> 5.5 usage_in_bytes >> >> For efficiency, as other kernel components, memory cgroup uses some optimization >> to avoid unnecessary cacheline false sharing. usage_in_bytes is affected by the >> method and doesn't show 'exact' value of memory(and swap) usage, it's an fuzz >> value for efficient access. (Of course, when necessary, it's synchronized.) >> If you want to know more exact memory usage, you should use RSS+CACHE(+SWAP) >> value in memory.stat(see 5.2). >> == >> >> In current implementation, memcg tries to charge resource in size of 32 pages. >> So, if you get 32 pages and free 32pages, usage_in_bytes may not change. >> This is affected by caches in other cpus and other flushing operations caused >> by some workload in other cgroups. memcg's usage_in_bytes is not precise in >> 128k degree. >> > Yes, I tried to mmap/dirty/unmap in 32 times, when the usage_in_bytes > reached 128k, it rolls back to 4k again. So it doesn't hurt any more. rolls back before unmap() ? > > I haven't found the code regarding to this behavior. Could you post your test program ? Thanks, -Kame -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/ Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from psmtp.com (na3sys010amx119.postini.com [74.125.245.119]) by kanga.kvack.org (Postfix) with SMTP id 1E5766B0044 for ; Fri, 23 Mar 2012 05:16:03 -0400 (EDT) Received: by obbta14 with SMTP id ta14so2911800obb.14 for ; Fri, 23 Mar 2012 02:16:02 -0700 (PDT) Message-ID: <4F6C3F29.8090402@gmail.com> Date: Fri, 23 Mar 2012 17:15:21 +0800 From: bill4carson MIME-Version: 1.0 Subject: Re: Why memory.usage_in_bytes is always increasing after every mmap/dirty/unmap sequence References: <4F6C2E9B.9010200@gmail.com> <4F6C31F7.2010804@jp.fujitsu.com> <4F6C3B7F.1070705@gmail.com> <4F6C3C88.5090800@jp.fujitsu.com> In-Reply-To: <4F6C3C88.5090800@jp.fujitsu.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Sender: owner-linux-mm@kvack.org List-ID: To: KAMEZAWA Hiroyuki Cc: "linux-mm@kvack.org" On 2012a1'03ae??23ae?JPY 17:04, KAMEZAWA Hiroyuki wrote: > (2012/03/23 17:59), bill4carson wrote: > >> >> >> On 2012a1'03ae??23ae?JPY 16:19, KAMEZAWA Hiroyuki wrote: >>> (2012/03/23 17:04), bill4carson wrote: >>> >>>> Hi, all >>>> >>>> I'm playing with memory cgroup, I'm a bit confused why >>>> memory.usage in bytes is steadily increasing at 4K page pace >>>> after every mmap/dirty/unmap sequence. >>>> >>>> On linux-3.6.34.10/linux-3.3.0-rc5 >>>> A simple test case does following: >>>> >>>> a) mmap 128k memory in private anonymous way >>>> b) dirty all 128k to demand physical page >>>> c) print memory.usage_in_bytes<-- increased at 4K after every loop >>>> d) unmap previous 128 memory >>>> e) goto a) to repeat >>> >>> In Documentation/cgroup/memory.txt >>> == >>> 5.5 usage_in_bytes >>> >>> For efficiency, as other kernel components, memory cgroup uses some optimization >>> to avoid unnecessary cacheline false sharing. usage_in_bytes is affected by the >>> method and doesn't show 'exact' value of memory(and swap) usage, it's an fuzz >>> value for efficient access. (Of course, when necessary, it's synchronized.) >>> If you want to know more exact memory usage, you should use RSS+CACHE(+SWAP) >>> value in memory.stat(see 5.2). >>> == >>> >>> In current implementation, memcg tries to charge resource in size of 32 pages. >>> So, if you get 32 pages and free 32pages, usage_in_bytes may not change. >>> This is affected by caches in other cpus and other flushing operations caused >>> by some workload in other cgroups. memcg's usage_in_bytes is not precise in >>> 128k degree. >>> >> Yes, I tried to mmap/dirty/unmap in 32 times, when the usage_in_bytes >> reached 128k, it rolls back to 4k again. So it doesn't hurt any more. > > > rolls back before unmap() ? > After unmap >> >> I haven't found the code regarding to this behavior. > > > Could you post your test program ? > Yes, it's a bit of messy, you can mock at me:) ------------------------------------------------------------- #include #include #include #include #include #include #include #include #include #include #define MMAP_SIZE (128*1024) int * ptr_array[1024]; int depth=0; char cmd; int pagesize; int mapsize; void get_key(void) { cmd = getchar(); putchar('\n'); getchar(); } int is_this_key(char key) { return (cmd == key) ? 1:0; } void getpage(int *ptr) { int i; for (i = 0; i < mapsize/4098; i++){ *ptr = 4;/* alloc a physical page */ ptr += 1024; /*move to next page*/ } } void show_stat(void) { int i; for (i = depth; i--; i < 0){ printf("[%2d]:%8p\n", i, ptr_array[i]); } } int main(int argc, char ** argv) { int *ptr; int i; char usage_in_bytes_path[PATH_MAX]; char tasks_path[PATH_MAX]; char tmp[PATH_MAX]; int usage_in_bytes = -1; int fd_tasks = -1; char *root_path; int ret; if (argc > 2) printf("Usage: oomtst [map size]\n"); mapsize = MMAP_SIZE; strcpy(tmp, argv[1]); root_path = dirname(tmp); printf("root_path:%s\n", root_path); ret = snprintf(usage_in_bytes_path, PATH_MAX, "%s/memory.usage_in_bytes", root_path); if (ret >= PATH_MAX) { fputs("Path to memory.usage_in_bytes is too long\n", stderr); goto out; } puts(usage_in_bytes_path); ret = snprintf(tasks_path, PATH_MAX, "%s/tasks", root_path); if (ret >= PATH_MAX) { fputs("Path to memory.usage_in_bytes is too long\n", stderr); goto out; } puts(tasks_path); fd_tasks = open(tasks_path, O_WRONLY); if (fd_tasks == -1) { fprintf(stderr, "Cannot open %s: %s\n", tasks_path, strerror(errno)); goto out; } printf("Using PID:%u\n", getpid()); { char tasks_str[32]; ret = sprintf(tasks_str, "%d", getpid()); ret = write(fd_tasks, tasks_str, strlen(tasks_str)); } while(1){ char used_bytes[64]; uint64_t tmp; usage_in_bytes = open(usage_in_bytes_path, O_RDWR); if (usage_in_bytes == -1) { fprintf(stderr, "Cannot open %s: %s\n", usage_in_bytes_path, strerror(errno)); goto out; } printf("Enter a command (m: malloc f: free ?: exit):"); get_key(); if (is_this_key('m')) { ptr_array[depth] = mmap(NULL, mapsize, PROT_READ | PROT_WRITE, MAP_PRIVATE|MAP_ANON, 0, 0); if (ptr_array[depth] == NULL){ perror("msg: cannot malloc\n"); exit(2); } printf("malloc %d Kbytes at %p\n", mapsize/1024, ptr_array[depth]); getpage(ptr_array[depth]); depth++; }else if (is_this_key('f')) { if (depth == 0) break; printf("free %d Kbytes at %p\n", mapsize/1024, ptr_array[depth -1]); munmap(ptr_array[depth -1], mapsize); --depth; }else break; show_stat(); memset(&used_bytes, 0, 33/*sizeof(used_bytes)*/); ret = read(usage_in_bytes, &used_bytes, 32); if (ret == -1) { perror("Cannot read from usage_in_bytes"); break; } tmp = atoll(used_bytes); printf("used_bytes:%llu Kbytes\n", tmp/1024); if (usage_in_bytes >= 0) close(usage_in_bytes); } out: for (i = 0; i < depth; i++){ printf("free %d Kbytes at %p\n", mapsize/1024, ptr_array[i]); munmap(ptr_array[i], mapsize); } if (usage_in_bytes >= 0) close(usage_in_bytes); return 0; } > Thanks, > -Kame > > -- Love each day! --bill -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/ Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from psmtp.com (na3sys010amx193.postini.com [74.125.245.193]) by kanga.kvack.org (Postfix) with SMTP id 233AA6B0044 for ; Fri, 23 Mar 2012 05:59:07 -0400 (EDT) Received: from m2.gw.fujitsu.co.jp (unknown [10.0.50.72]) by fgwmail5.fujitsu.co.jp (Postfix) with ESMTP id 796893EE081 for ; Fri, 23 Mar 2012 18:59:05 +0900 (JST) Received: from smail (m2 [127.0.0.1]) by outgoing.m2.gw.fujitsu.co.jp (Postfix) with ESMTP id 604E445DE53 for ; Fri, 23 Mar 2012 18:59:05 +0900 (JST) Received: from s2.gw.fujitsu.co.jp (s2.gw.fujitsu.co.jp [10.0.50.92]) by m2.gw.fujitsu.co.jp (Postfix) with ESMTP id 3C2CB45DE4E for ; Fri, 23 Mar 2012 18:59:05 +0900 (JST) Received: from s2.gw.fujitsu.co.jp (localhost.localdomain [127.0.0.1]) by s2.gw.fujitsu.co.jp (Postfix) with ESMTP id 2BC1E1DB802C for ; Fri, 23 Mar 2012 18:59:05 +0900 (JST) Received: from m106.s.css.fujitsu.com (m106.s.css.fujitsu.com [10.240.81.146]) by s2.gw.fujitsu.co.jp (Postfix) with ESMTP id DB0BE1DB803A for ; Fri, 23 Mar 2012 18:59:04 +0900 (JST) Message-ID: <4F6C490A.4030608@jp.fujitsu.com> Date: Fri, 23 Mar 2012 18:57:30 +0900 From: KAMEZAWA Hiroyuki MIME-Version: 1.0 Subject: Re: Why memory.usage_in_bytes is always increasing after every mmap/dirty/unmap sequence References: <4F6C2E9B.9010200@gmail.com> <4F6C31F7.2010804@jp.fujitsu.com> <4F6C3B7F.1070705@gmail.com> <4F6C3C88.5090800@jp.fujitsu.com> <4F6C3F29.8090402@gmail.com> In-Reply-To: <4F6C3F29.8090402@gmail.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Sender: owner-linux-mm@kvack.org List-ID: To: bill4carson Cc: "linux-mm@kvack.org" (2012/03/23 18:15), bill4carson wrote: > > > On 2012a1'03ae??23ae?JPY 17:04, KAMEZAWA Hiroyuki wrote: >> (2012/03/23 17:59), bill4carson wrote: >> >>> >>> >>> On 2012a1'03ae??23ae?JPY 16:19, KAMEZAWA Hiroyuki wrote: >>>> (2012/03/23 17:04), bill4carson wrote: >>>> >>>>> Hi, all >>>>> >>>>> I'm playing with memory cgroup, I'm a bit confused why >>>>> memory.usage in bytes is steadily increasing at 4K page pace >>>>> after every mmap/dirty/unmap sequence. >>>>> >>>>> On linux-3.6.34.10/linux-3.3.0-rc5 >>>>> A simple test case does following: >>>>> >>>>> a) mmap 128k memory in private anonymous way >>>>> b) dirty all 128k to demand physical page >>>>> c) print memory.usage_in_bytes<-- increased at 4K after every loop >>>>> d) unmap previous 128 memory >>>>> e) goto a) to repeat >>>> >>>> In Documentation/cgroup/memory.txt >>>> == >>>> 5.5 usage_in_bytes >>>> >>>> For efficiency, as other kernel components, memory cgroup uses some optimization >>>> to avoid unnecessary cacheline false sharing. usage_in_bytes is affected by the >>>> method and doesn't show 'exact' value of memory(and swap) usage, it's an fuzz >>>> value for efficient access. (Of course, when necessary, it's synchronized.) >>>> If you want to know more exact memory usage, you should use RSS+CACHE(+SWAP) >>>> value in memory.stat(see 5.2). >>>> == >>>> >>>> In current implementation, memcg tries to charge resource in size of 32 pages. >>>> So, if you get 32 pages and free 32pages, usage_in_bytes may not change. >>>> This is affected by caches in other cpus and other flushing operations caused >>>> by some workload in other cgroups. memcg's usage_in_bytes is not precise in >>>> 128k degree. >>>> >>> Yes, I tried to mmap/dirty/unmap in 32 times, when the usage_in_bytes >>> reached 128k, it rolls back to 4k again. So it doesn't hurt any more. >> >> >> rolls back before unmap() ? >> > After unmap > >>> >>> I haven't found the code regarding to this behavior. >> >> >> Could you post your test program ? >> > Yes, it's a bit of messy, you can mock at me:) Thank you. I tried. I checked /proc//smaps and compares it before I press 'm' and after I press 'm'. So, anonymous should be increased 128k. Then... == [root@bluextal kamezawa]# diff -Nru before.txt after.txt | grep Anonymous Anonymous: 0 kB -Anonymous: 12 kB +Anonymous: 140 kB <============= 128kbytes you allocated -Anonymous: 8 kB +Anonymous: 12 kB <============= maybe some library's heap. == Then, 4k remains even if you free all allocated area by your program, I think. Thanks, -Kame -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/ Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from psmtp.com (na3sys010amx161.postini.com [74.125.245.161]) by kanga.kvack.org (Postfix) with SMTP id D6FFF6B007E for ; Tue, 27 Mar 2012 04:26:34 -0400 (EDT) Message-ID: <4F7179B4.7080405@parallels.com> Date: Tue, 27 Mar 2012 10:26:28 +0200 From: Glauber Costa MIME-Version: 1.0 Subject: Re: Why memory.usage_in_bytes is always increasing after every mmap/dirty/unmap sequence References: <4F6C2E9B.9010200@gmail.com> <4F6C31F7.2010804@jp.fujitsu.com> <4F6C3B7F.1070705@gmail.com> In-Reply-To: <4F6C3B7F.1070705@gmail.com> Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 7bit Sender: owner-linux-mm@kvack.org List-ID: To: bill4carson Cc: KAMEZAWA Hiroyuki , "linux-mm@kvack.org" On 03/23/2012 09:59 AM, bill4carson wrote: >> > Yes, I tried to mmap/dirty/unmap in 32 times, when the usage_in_bytes > reached 128k, it rolls back to 4k again. So it doesn't hurt any more. > > I haven't found the code regarding to this behavior. That's actually quite annoying, IMHO. I personally think that everytime one tries to read from usage, we should flush the caches and show the correct figures, or at least as correct as we can. That's specially bad because under load, this is wrong by O(#cpus)... For just reading the file, this might be okay because there is an alternative for it (although not that intuitive), but for the threshold code, we are probably hitting them a lot more than we should in big machines. -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/ Don't email: email@kvack.org