[RFCv2][PATCH 1/4] break units out of string_get_size()

[RFCv2][PATCH 1/4] break units out of string_get_size()

[Dave Hansen]

I would like to use these (well one of them) arrays in
another function.  Might as well break both versions
out for consistency.

Signed-off-by: Dave Hansen <dave@linux.vnet.ibm.com>
---

 linux-2.6.git-dave/lib/string_helpers.c |   25 +++++++++++++------------
 1 file changed, 13 insertions(+), 12 deletions(-)

diff -puN lib/string_helpers.c~string_get_size-pow2 lib/string_helpers.c
--- linux-2.6.git/lib/string_helpers.c~string_get_size-pow2	2011-09-30 12:58:43.856800824 -0700
+++ linux-2.6.git-dave/lib/string_helpers.c	2011-09-30 12:58:43.864800812 -0700
@@ -8,6 +8,19 @@
 #include <linux/module.h>
 #include <linux/string_helpers.h>
 
+const char *units_10[] = { "B", "kB", "MB", "GB", "TB", "PB",
+			   "EB", "ZB", "YB", NULL};
+const char *units_2[] = {"B", "KiB", "MiB", "GiB", "TiB", "PiB",
+			 "EiB", "ZiB", "YiB", NULL };
+static const char **units_str[] = {
+	[STRING_UNITS_10] =  units_10,
+	[STRING_UNITS_2] = units_2,
+};
+static const unsigned int divisor[] = {
+	[STRING_UNITS_10] = 1000,
+	[STRING_UNITS_2] = 1024,
+};
+
 /**
  * string_get_size - get the size in the specified units
  * @size:	The size to be converted
@@ -23,18 +36,6 @@
 int string_get_size(u64 size, const enum string_size_units units,
 		    char *buf, int len)
 {
-	const char *units_10[] = { "B", "kB", "MB", "GB", "TB", "PB",
-				   "EB", "ZB", "YB", NULL};
-	const char *units_2[] = {"B", "KiB", "MiB", "GiB", "TiB", "PiB",
-				 "EiB", "ZiB", "YiB", NULL };
-	const char **units_str[] = {
-		[STRING_UNITS_10] =  units_10,
-		[STRING_UNITS_2] = units_2,
-	};
-	const unsigned int divisor[] = {
-		[STRING_UNITS_10] = 1000,
-		[STRING_UNITS_2] = 1024,
-	};
 	int i, j;
 	u64 remainder = 0, sf_cap;
 	char tmp[8];
_

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

[RFCv2][PATCH 2/4] add string_get_size_pow2()

[Dave Hansen]

This is a specialized version of string_get_size().

It only works on powers-of-two, and only outputs in
KiB/MiB/etc...  Doing it this way means that we do
not have to do any division like string_get_size()
does.

Signed-off-by: Dave Hansen <dave@linux.vnet.ibm.com>
---

 linux-2.6.git-dave/include/linux/string_helpers.h |    1 
 linux-2.6.git-dave/lib/string_helpers.c           |   23 ++++++++++++++++++++++
 2 files changed, 24 insertions(+)

diff -puN lib/string_helpers.c~string_get_size-pow2-1 lib/string_helpers.c
--- linux-2.6.git/lib/string_helpers.c~string_get_size-pow2-1	2011-09-30 12:10:31.653729703 -0700
+++ linux-2.6.git-dave/lib/string_helpers.c	2011-09-30 12:40:13.090605408 -0700
@@ -21,6 +21,29 @@ static const unsigned int divisor[] = {
 	[STRING_UNITS_2] = 1024,
 };
 
+u64 string_get_size_pow2(u64 size, const char **unit_ret)
+{
+	int log2;
+	int unit_index;
+
+	if (!size)
+		log2 = 0;
+	else
+		log2 = ilog2(size);
+
+	/* KiB is log2=0->9, MiB is 10->19, etc... */
+	unit_index = log2 / 10;
+	/* Can not overflow since YiB=2^80 does
+	 * not fit in a u64. */
+	*unit_ret = units_2[unit_index];
+
+	/* 512 aka 2^9 is the largest integer without
+	 * overflowing to the next power-of-two, so
+	 * use %10 to make it max out there */
+	return (1 << (log2 % 10));
+}
+EXPORT_SYMBOL(string_get_size_pow2);
+
 /**
  * string_get_size - get the size in the specified units
  * @size:	The size to be converted
diff -puN include/linux/string_helpers.h~string_get_size-pow2-1 include/linux/string_helpers.h
--- linux-2.6.git/include/linux/string_helpers.h~string_get_size-pow2-1	2011-09-30 12:40:21.110592191 -0700
+++ linux-2.6.git-dave/include/linux/string_helpers.h	2011-09-30 12:40:31.186575591 -0700
@@ -10,6 +10,7 @@ enum string_size_units {
 	STRING_UNITS_2,		/* use binary powers of 2^10 */
 };
 
+u64 string_get_size_pow2(u64 size, const char **unit_ret);
 int string_get_size(u64 size, enum string_size_units units,
 		    char *buf, int len);
 
_

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

[RFCv2][PATCH 3/4] add seq_print_pow2() function

[Dave Hansen]

In order to get nice, human-readable output, we are going to
use MiB/KiB, etc... in numa_maps.  Introduce a helper to do
the conversion from a raw integer over to a string.

I thought about doing this as a new printk() format specifier.
That would be interesting, but it's hard to argue with this
since it's so short and sweet.

Signed-off-by: Dave Hansen <dave@linux.vnet.ibm.com>
---

 linux-2.6.git-dave/fs/seq_file.c            |   11 +++++++++++
 linux-2.6.git-dave/include/linux/seq_file.h |    2 ++
 2 files changed, 13 insertions(+)

diff -puN fs/seq_file.c~add-seq_print_size fs/seq_file.c
--- linux-2.6.git/fs/seq_file.c~add-seq_print_size	2011-09-30 13:00:29.160632197 -0700
+++ linux-2.6.git-dave/fs/seq_file.c	2011-09-30 13:22:57.229978028 -0700
@@ -386,6 +386,17 @@ int seq_printf(struct seq_file *m, const
 }
 EXPORT_SYMBOL(seq_printf);
 
+/*
+ * Prints output with KiB/MiB/etc... suffixes
+ */
+int seq_print_pow2(struct seq_file *seq, u64 size)
+{
+	u64 shifted_size;
+	char *unit_str;
+	shifted_size = string_get_size_pow2(size, &unit_str);
+	return seq_printf(seq, "%llu%s", shifted_size, unit_str);
+}
+
 /**
  *	mangle_path -	mangle and copy path to buffer beginning
  *	@s: buffer start
diff -puN include/linux/seq_file.h~add-seq_print_size include/linux/seq_file.h
--- linux-2.6.git/include/linux/seq_file.h~add-seq_print_size	2011-09-30 13:00:29.164632191 -0700
+++ linux-2.6.git-dave/include/linux/seq_file.h	2011-09-30 13:22:43.910000779 -0700
@@ -3,6 +3,7 @@
 
 #include <linux/types.h>
 #include <linux/string.h>
+#include <linux/string_helpers.h>
 #include <linux/mutex.h>
 #include <linux/cpumask.h>
 #include <linux/nodemask.h>
@@ -83,6 +84,7 @@ int seq_escape(struct seq_file *, const 
 int seq_putc(struct seq_file *m, char c);
 int seq_puts(struct seq_file *m, const char *s);
 int seq_write(struct seq_file *seq, const void *data, size_t len);
+int seq_print_pow2(struct seq_file *seq, u64 size);
 
 int seq_printf(struct seq_file *, const char *, ...)
 	__attribute__ ((format (printf,2,3)));
_

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

[RFCv2][PATCH 4/4] show page size in /proc/$pid/numa_maps

[Dave Hansen]

The output of /proc/$pid/numa_maps is in terms of number of pages
like anon=22 or dirty=54.  Here's some output:

7f4680000000 default file=/hugetlb/bigfile anon=50 dirty=50 N0=50
7f7659600000 default file=/anon_hugepage\040(deleted) anon=50 dirty=50 N0=50
7fff8d425000 default stack anon=50 dirty=50 N0=50

Looks like we have a stack and a couple of anonymous hugetlbfs
areas page which both use the same amount of memory.  They don't.

The 'bigfile' uses 1GB pages and takes up ~50GB of space.  The
anon_hugepage uses 2MB pages and takes up ~100MB of space while
the stack uses normal 4k pages.  You can go over to smaps to
figure out what the page size _really_ is with KernelPageSize
or MMUPageSize.  But, I think this is a pretty nasty and
counterintuitive interface as it stands.

The following patch adds a pagesize= field.  Note that this only
shows the kernel's notion of page size.  For transparent
hugepages, it still shows the base page size.  Here's some real
output.  Note the anon_hugepage in there.

# cat /proc/`pidof memknobs`/numa_maps
00400000 default file=/root/memknobs pagesize=4KiB dirty=3 active=2 N0=3
00602000 default file=/root/memknobs pagesize=4KiB anon=1 dirty=1 N0=1
00603000 default file=/root/memknobs pagesize=4KiB anon=1 dirty=1 N0=1
00604000 default heap pagesize=4KiB anon=6 dirty=6 N0=6
7f6766216000 default file=/lib/libc-2.9.so pagesize=4KiB mapped=98 mapmax=25 active=97 N0=98
7f676637e000 default file=/lib/libc-2.9.so
7f676657e000 default file=/lib/libc-2.9.so pagesize=4KiB anon=4 dirty=4 N0=4
7f6766582000 default file=/lib/libc-2.9.so pagesize=4KiB anon=1 dirty=1 N0=1
7f6766583000 default pagesize=4KiB anon=3 dirty=3 N0=3
7f6766588000 default file=/lib/ld-2.9.so pagesize=4KiB mapped=25 mapmax=24 N0=25
7f676679d000 default pagesize=4KiB anon=2 dirty=2 N0=2
7f67667a3000 default pagesize=4KiB anon=4 dirty=4 N0=4
7f67667a7000 default file=/lib/ld-2.9.so pagesize=4KiB anon=1 dirty=1 N0=1
7f67667a8000 default file=/lib/ld-2.9.so pagesize=4KiB anon=1 dirty=1 N0=1
7f6766800000 default file=/anon_hugepage\040(deleted) pagesize=2MiB anon=10 dirty=10 N0=10
7fff5b948000 default stack pagesize=4KiB anon=2 dirty=2 N0=2
7fff5b96d000 default

Signed-off-by: Dave Haneen <dave@linux.vnet.ibm.com>
---

 linux-2.6.git-dave/fs/proc/task_mmu.c |    5 +++++
 1 file changed, 5 insertions(+)

diff -puN fs/proc/task_mmu.c~show-page-size fs/proc/task_mmu.c
--- linux-2.6.git/fs/proc/task_mmu.c~show-page-size	2011-09-30 13:27:55.973467993 -0700
+++ linux-2.6.git-dave/fs/proc/task_mmu.c	2011-09-30 13:27:55.981467979 -0700
@@ -1044,6 +1044,11 @@ static int show_numa_map(struct seq_file
 	if (!md->pages)
 		goto out;
 
+	/* Only interesting for hugetlbfs pages.
+	 * Transparent hugepages are still pagesize=4k */
+	seq_puts(m, " pagesize=");
+	seq_print_pow2(m, vma_kernel_pagesize(vma));
+
 	if (md->anon)
 		seq_printf(m, " anon=%lu", md->anon);
 
_

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFCv2][PATCH 1/4] break units out of string_get_size()

[H. Peter Anvin]

On 09/30/2011 01:32 PM, Dave Hansen wrote:
> I would like to use these (well one of them) arrays in
> another function.  Might as well break both versions
> out for consistency.
> 
> Signed-off-by: Dave Hansen <dave@linux.vnet.ibm.com>
> ---
> 
>  linux-2.6.git-dave/lib/string_helpers.c |   25 +++++++++++++------------
>  1 file changed, 13 insertions(+), 12 deletions(-)
> 
> diff -puN lib/string_helpers.c~string_get_size-pow2 lib/string_helpers.c
>  
> +const char *units_10[] = { "B", "kB", "MB", "GB", "TB", "PB",
> +			   "EB", "ZB", "YB", NULL};
> +const char *units_2[] = {"B", "KiB", "MiB", "GiB", "TiB", "PiB",
> +			 "EiB", "ZiB", "YiB", NULL };

These names are way too generic to be public symbols.

Another thing worth thinking about is whether or not the -B suffix
should be part of these arrays.

	-hpa

-- 
H. Peter Anvin, Intel Open Source Technology Center
I work for Intel.  I don't speak on their behalf.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFCv2][PATCH 1/4] break units out of string_get_size()

[Dave Hansen]

On Fri, 2011-09-30 at 14:29 -0700, H. Peter Anvin wrote:
> On 09/30/2011 01:32 PM, Dave Hansen wrote:
> > diff -puN lib/string_helpers.c~string_get_size-pow2 lib/string_helpers.c
> >  
> > +const char *units_10[] = { "B", "kB", "MB", "GB", "TB", "PB",
> > +			   "EB", "ZB", "YB", NULL};
> > +const char *units_2[] = {"B", "KiB", "MiB", "GiB", "TiB", "PiB",
> > +			 "EiB", "ZiB", "YiB", NULL };
> 
> These names are way too generic to be public symbols.

Ack, I managed to drop the static when I broke this out for the third
time. :)

> Another thing worth thinking about is whether or not the -B suffix
> should be part of these arrays.

... or the 'i' for that matter.

I'll give it a go.

-- Dave

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFCv2][PATCH 1/4] break units out of string_get_size()

[H. Peter Anvin]

On 09/30/2011 03:46 PM, Dave Hansen wrote:
> 
> ... or the 'i' for that matter.
> 

The "i" you need in the array unless you want to say "500 iB" and the
like, or have special logic for it.

	-hpa


-- 
H. Peter Anvin, Intel Open Source Technology Center
I work for Intel.  I don't speak on their behalf.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>