[PATCH 0/4] lib: rework bitmap_parse

[PATCH 0/4] lib: rework bitmap_parse

[Yury Norov]

On top of next-20190418.

Similarly to recently revisited bitmap_parselist() [1],
bitmap_parse() is ineffective and overcomplicated.  This
series reworks it, aligns its interface with bitmap_parselist()
and makes usage simpler.

The series also adds a test for the function and fixes usage of it
in cpumask_parse() according to new design - drops the calculating
of length of an input string.

The following users would also consider to drop the length argument,
if possible:
drivers/block/drbd/drbd_main.c:2608
kernel/padata.c:924
net/core/net-sysfs.c:726
net/core/net-sysfs.c:1309
net/core/net-sysfs.c:1391

bitmap_parse() takes the array of numbers to be put into the map in
the BE order which is reversed to the natural LE order for bitmaps.
For example, to construct bitmap containing a bit on the position 42,
we have to put a line '400,0'. Current implementation reads chunk
one by one from the beginning ('400' before '0') and makes bitmap
shift after each successful parse. It makes the complexity of the
whole process as O(n^2). We can do it in reverse direction ('0'
before '400') and avoid shifting, but it requires reverse parsing
helpers.

Tested on arm64 and BE mips.

[1] https://lkml.org/lkml/2019/4/16/66

Yury Norov (6):
  lib/string: add strnchrnul()
  bitops: more BITS_TO_* macros
  lib/bitmap: make bitmap_parse_user a wrapper on bitmap_parse
  lib: rework bitmap_parse()
  lib: add test for bitmap_parse()
  cpumask: don't calculate length of the input string

 include/linux/bitmap.h       |   8 +-
 include/linux/bitops.h       |   3 +
 include/linux/cpumask.h      |   4 +-
 include/linux/string.h       |   3 +
 lib/bitmap.c                 | 197 +++++++++++++++++------------------
 lib/string.c                 |  20 ++++
 lib/test_bitmap.c            | 102 +++++++++++++++++-
 tools/include/linux/bitops.h |   9 +-
 8 files changed, 230 insertions(+), 116 deletions(-)

-- 
2.17.1

[PATCH 1/6] lib/string: add strnchrnul()

[Yury Norov]

New function works like strchrnul() with a length limited strings.

Signed-off-by: Yury Norov <ynorov@marvell.com>
---
 include/linux/string.h |  3 +++
 lib/string.c           | 20 ++++++++++++++++++++
 2 files changed, 23 insertions(+)

diff --git a/include/linux/string.h b/include/linux/string.h
index 4deb11f7976b..69e8df51b630 100644
--- a/include/linux/string.h
+++ b/include/linux/string.h
@@ -62,6 +62,9 @@ extern char * strchr(const char *,int);
 #ifndef __HAVE_ARCH_STRCHRNUL
 extern char * strchrnul(const char *,int);
 #endif
+#ifndef __HAVE_ARCH_STRNCHRNUL
+extern char * strnchrnul(const char *, int, int);
+#endif
 #ifndef __HAVE_ARCH_STRNCHR
 extern char * strnchr(const char *, size_t, int);
 #endif
diff --git a/lib/string.c b/lib/string.c
index 6016eb3ac73d..dd99e6ac517a 100644
--- a/lib/string.c
+++ b/lib/string.c
@@ -429,6 +429,26 @@ char *strchrnul(const char *s, int c)
 EXPORT_SYMBOL(strchrnul);
 #endif
 
+#ifndef __HAVE_ARCH_STRNCHRNUL
+/**
+ * strnchrnul - Find and return a character in a length limited string,
+ * or end of string
+ * @s: The string to be searched
+ * @count: The number of characters to be searched
+ * @c: The character to search for
+ *
+ * Returns pointer to first occurrence of 'c' in s. If c is not found,
+ * then return a pointer to the null byte at the end of s.
+ */
+char *strnchrnul(const char *s, int count, int c)
+{
+	while (count-- && *s && *s != (char)c)
+		s++;
+	return (char *)s;
+}
+EXPORT_SYMBOL(strnchrnul);
+#endif
+
 #ifndef __HAVE_ARCH_STRRCHR
 /**
  * strrchr - Find the last occurrence of a character in a string
-- 
2.17.1

Re: [PATCH 1/6] lib/string: add strnchrnul()

[Andy Shevchenko]

On Sat, Apr 27, 2019 at 08:29:31PM -0700, Yury Norov wrote:
> New function works like strchrnul() with a length limited strings.

The prototype better to be consistent with strchrnul() and memchr(), i.e.
the size parameter to go last.

-- 
With Best Regards,
Andy Shevchenko

Re: [PATCH 1/6] lib/string: add strnchrnul()

[Yury Norov]

On Sun, Apr 28, 2019 at 07:04:00PM +0300, Andy Shevchenko wrote:
> On Sat, Apr 27, 2019 at 08:29:31PM -0700, Yury Norov wrote:
> > New function works like strchrnul() with a length limited strings.
> 
> The prototype better to be consistent with strchrnul() and memchr(), i.e.
> the size parameter to go last.
 
But if so it would be inconsistent with strnchr(). I'm OK to do the change,
but I'm not sure if this version better than that.

Parameter 'count' should be size_t, not int.

Re: [PATCH 1/6] lib/string: add strnchrnul()

[Andy Shevchenko]

On Sun, Apr 28, 2019 at 11:26:12AM -0700, Yury Norov wrote:
> On Sun, Apr 28, 2019 at 07:04:00PM +0300, Andy Shevchenko wrote:
> > On Sat, Apr 27, 2019 at 08:29:31PM -0700, Yury Norov wrote:
> > > New function works like strchrnul() with a length limited strings.
> > 
> > The prototype better to be consistent with strchrnul() and memchr(), i.e.
> > the size parameter to go last.
>  
> But if so it would be inconsistent with strnchr(). I'm OK to do the change,
> but I'm not sure if this version better than that.

Forgot to check it. In this case, yes, strnchr() + size.

-- 
With Best Regards,
Andy Shevchenko

Re: [PATCH 1/6] lib/string: add strnchrnul()

[Rasmus Villemoes]

On 28/04/2019 05.29, Yury Norov wrote:
> New function works like strchrnul() with a length limited strings.
> 
> Signed-off-by: Yury Norov <ynorov@marvell.com>
> ---
>  include/linux/string.h |  3 +++
>  lib/string.c           | 20 ++++++++++++++++++++
>  2 files changed, 23 insertions(+)
> 
> diff --git a/include/linux/string.h b/include/linux/string.h
> index 4deb11f7976b..69e8df51b630 100644
> --- a/include/linux/string.h
> +++ b/include/linux/string.h
> @@ -62,6 +62,9 @@ extern char * strchr(const char *,int);
>  #ifndef __HAVE_ARCH_STRCHRNUL
>  extern char * strchrnul(const char *,int);
>  #endif
> +#ifndef __HAVE_ARCH_STRNCHRNUL

I find it unlikely this would ever be performance-critical, so an arch
would want to implement its own. Please let's avoid adding redundant
ifdeffery (and yes, there's probably quite a few of the existing ones
that should go away).

>  
> +#ifndef __HAVE_ARCH_STRNCHRNUL
> +/**
> + * strnchrnul - Find and return a character in a length limited string,
> + * or end of string
> + * @s: The string to be searched
> + * @count: The number of characters to be searched
> + * @c: The character to search for
> + *
> + * Returns pointer to first occurrence of 'c' in s. If c is not found,
> + * then return a pointer to the null byte at the end of s.

Pet peeve: nul, not null. Especially important when this is in contrast
to returning a NULL pointer. Also, the description is inaccurate; the
whole point of the n-variant is that there is not necessarily any nul
terminator.

> + */
> +char *strnchrnul(const char *s, int count, int c)
> +{

Yes, make count size_t so it's easier to distinguish from the c, and
keep this parameter ordering (that matches the str n chr name).

> +	while (count-- && *s && *s != (char)c)
> +		s++;
> +	return (char *)s;
> +}
> +EXPORT_SYMBOL(strnchrnul);

This can be added if a modular user shows up. No reason to add that
bloat to the kernel image immediately.

Rasmus

[PATCH 2/6] bitops: more BITS_TO_* macros

[Yury Norov]

Introduce BITS_TO_U64, BITS_TO_U32 and BITS_TO_BYTES as they are handy
in the following changes (BITS_TO_U32 specifically). Sync tools/
version of the macros with the kernel implementation.

Signed-off-by: Yury Norov <ynorov@marvell.com>
---
 include/linux/bitops.h       | 3 +++
 tools/include/linux/bitops.h | 9 +++++----
 2 files changed, 8 insertions(+), 4 deletions(-)

diff --git a/include/linux/bitops.h b/include/linux/bitops.h
index cf074bce3eb3..3477e179dca7 100644
--- a/include/linux/bitops.h
+++ b/include/linux/bitops.h
@@ -6,6 +6,9 @@
 
 #define BITS_PER_TYPE(type) (sizeof(type) * BITS_PER_BYTE)
 #define BITS_TO_LONGS(nr)	DIV_ROUND_UP(nr, BITS_PER_TYPE(long))
+#define BITS_TO_U64(nr)		DIV_ROUND_UP(nr, BITS_PER_TYPE(u64))
+#define BITS_TO_U32(nr)		DIV_ROUND_UP(nr, BITS_PER_TYPE(u32))
+#define BITS_TO_BYTES(nr)	DIV_ROUND_UP(nr, BITS_PER_TYPE(char))
 
 extern unsigned int __sw_hweight8(unsigned int w);
 extern unsigned int __sw_hweight16(unsigned int w);
diff --git a/tools/include/linux/bitops.h b/tools/include/linux/bitops.h
index 0b0ef3abc966..959dcb8214ba 100644
--- a/tools/include/linux/bitops.h
+++ b/tools/include/linux/bitops.h
@@ -13,10 +13,11 @@
 #include <linux/bits.h>
 #include <linux/compiler.h>
 
-#define BITS_TO_LONGS(nr)	DIV_ROUND_UP(nr, BITS_PER_BYTE * sizeof(long))
-#define BITS_TO_U64(nr)		DIV_ROUND_UP(nr, BITS_PER_BYTE * sizeof(u64))
-#define BITS_TO_U32(nr)		DIV_ROUND_UP(nr, BITS_PER_BYTE * sizeof(u32))
-#define BITS_TO_BYTES(nr)	DIV_ROUND_UP(nr, BITS_PER_BYTE)
+#define BITS_PER_TYPE(type)	(sizeof(type) * BITS_PER_BYTE)
+#define BITS_TO_LONGS(nr)	DIV_ROUND_UP(nr, BITS_PER_TYPE(long))
+#define BITS_TO_U64(nr)		DIV_ROUND_UP(nr, BITS_PER_TYPE(u64))
+#define BITS_TO_U32(nr)		DIV_ROUND_UP(nr, BITS_PER_TYPE(u32))
+#define BITS_TO_BYTES(nr)	DIV_ROUND_UP(nr, BITS_PER_TYPE(char))
 
 extern unsigned int __sw_hweight8(unsigned int w);
 extern unsigned int __sw_hweight16(unsigned int w);
-- 
2.17.1

Re: [PATCH 2/6] bitops: more BITS_TO_* macros

[Andy Shevchenko]

On Sat, Apr 27, 2019 at 08:29:32PM -0700, Yury Norov wrote:
> Introduce BITS_TO_U64, BITS_TO_U32 and BITS_TO_BYTES as they are handy
> in the following changes (BITS_TO_U32 specifically). Sync tools/
> version of the macros with the kernel implementation.

AFAICS you basically reimplement them for kernel use from tools/.

>  #define BITS_PER_TYPE(type) (sizeof(type) * BITS_PER_BYTE)

Hmm... no TAB here? Perhaps another small fix can be done.

>  #define BITS_TO_LONGS(nr)	DIV_ROUND_UP(nr, BITS_PER_TYPE(long))
> +#define BITS_TO_U64(nr)		DIV_ROUND_UP(nr, BITS_PER_TYPE(u64))
> +#define BITS_TO_U32(nr)		DIV_ROUND_UP(nr, BITS_PER_TYPE(u32))
> +#define BITS_TO_BYTES(nr)	DIV_ROUND_UP(nr, BITS_PER_TYPE(char))
>  
>  extern unsigned int __sw_hweight8(unsigned int w);
>  extern unsigned int __sw_hweight16(unsigned int w);
> diff --git a/tools/include/linux/bitops.h b/tools/include/linux/bitops.h
> index 0b0ef3abc966..959dcb8214ba 100644
> --- a/tools/include/linux/bitops.h
> +++ b/tools/include/linux/bitops.h
> @@ -13,10 +13,11 @@
>  #include <linux/bits.h>
>  #include <linux/compiler.h>
>  
> -#define BITS_TO_LONGS(nr)	DIV_ROUND_UP(nr, BITS_PER_BYTE * sizeof(long))
> -#define BITS_TO_U64(nr)		DIV_ROUND_UP(nr, BITS_PER_BYTE * sizeof(u64))
> -#define BITS_TO_U32(nr)		DIV_ROUND_UP(nr, BITS_PER_BYTE * sizeof(u32))
> -#define BITS_TO_BYTES(nr)	DIV_ROUND_UP(nr, BITS_PER_BYTE)
> +#define BITS_PER_TYPE(type)	(sizeof(type) * BITS_PER_BYTE)
> +#define BITS_TO_LONGS(nr)	DIV_ROUND_UP(nr, BITS_PER_TYPE(long))
> +#define BITS_TO_U64(nr)		DIV_ROUND_UP(nr, BITS_PER_TYPE(u64))
> +#define BITS_TO_U32(nr)		DIV_ROUND_UP(nr, BITS_PER_TYPE(u32))
> +#define BITS_TO_BYTES(nr)	DIV_ROUND_UP(nr, BITS_PER_TYPE(char))

-- 
With Best Regards,
Andy Shevchenko

[PATCH 3/6] lib/bitmap: make bitmap_parse_user a wrapper on bitmap_parse

[Yury Norov]

Currently we parse user data byte after byte which leads to
overcomplicating of parsing algorithm. There are no performance
critical users of bitmap_parse_user(), and so we can duplicate
user data to kernel buffer and simply call bitmap_parselist().
This rework lets us unify and simplify bitmap_parse() and
bitmap_parse_user(), which is done in the following patch.

Signed-off-by: Yury Norov <ynorov@marvell.com>
---
 lib/bitmap.c | 20 ++++++++++----------
 1 file changed, 10 insertions(+), 10 deletions(-)

diff --git a/lib/bitmap.c b/lib/bitmap.c
index f235434df87b..300732031fad 100644
--- a/lib/bitmap.c
+++ b/lib/bitmap.c
@@ -434,22 +434,22 @@ EXPORT_SYMBOL(__bitmap_parse);
  *    then it must be terminated with a \0.
  * @maskp: pointer to bitmap array that will contain result.
  * @nmaskbits: size of bitmap, in bits.
- *
- * Wrapper for __bitmap_parse(), providing it with user buffer.
- *
- * We cannot have this as an inline function in bitmap.h because it needs
- * linux/uaccess.h to get the access_ok() declaration and this causes
- * cyclic dependencies.
  */
 int bitmap_parse_user(const char __user *ubuf,
 			unsigned int ulen, unsigned long *maskp,
 			int nmaskbits)
 {
-	if (!access_ok(ubuf, ulen))
-		return -EFAULT;
-	return __bitmap_parse((const char __force *)ubuf,
-				ulen, 1, maskp, nmaskbits);
+	char *buf;
+	int ret;
 
+	buf = memdup_user_nul(ubuf, ulen);
+	if (IS_ERR(buf))
+		return PTR_ERR(buf);
+
+	ret = bitmap_parse(buf, ulen, maskp, nmaskbits);
+
+	kfree(buf);
+	return ret;
 }
 EXPORT_SYMBOL(bitmap_parse_user);
 
-- 
2.17.1

[PATCH 4/6] lib: rework bitmap_parse()

[Yury Norov]

bitmap_parse() is ineffective and full of opaque variables and opencoded
parts. It leads to hard understanding of it. This rework includes:
 - remove bitmap_shift_left() call from the cycle. Now it makes the
   complexity of the algorithm as O(nbits^2). In the suggested approach
   the input string is parsed in reverse direction, so no shifts needed;
 - relax requirement on a single comma and no white spaces between chunks.
   It is considered useful in scripting, and it aligns with
   bitmap_parselist();
 - split bitmap_parse() to small readable helpers;
 - make an explicit calculation of the end of input line at the
   beginning, so users of the bitmap_parse() won't bother doing this.

Signed-off-by: Yury Norov <ynorov@marvell.com>
---
 include/linux/bitmap.h |   8 +-
 lib/bitmap.c           | 179 ++++++++++++++++++++---------------------
 2 files changed, 88 insertions(+), 99 deletions(-)

diff --git a/include/linux/bitmap.h b/include/linux/bitmap.h
index f58e97446abc..c3e84864cc33 100644
--- a/include/linux/bitmap.h
+++ b/include/linux/bitmap.h
@@ -172,7 +172,7 @@ bitmap_find_next_zero_area(unsigned long *map,
 					      align_mask, 0);
 }
 
-extern int __bitmap_parse(const char *buf, unsigned int buflen, int is_user,
+extern int bitmap_parse(const char *buf, unsigned int buflen,
 			unsigned long *dst, int nbits);
 extern int bitmap_parse_user(const char __user *ubuf, unsigned int ulen,
 			unsigned long *dst, int nbits);
@@ -408,12 +408,6 @@ static inline void bitmap_shift_left(unsigned long *dst, const unsigned long *sr
 		__bitmap_shift_left(dst, src, shift, nbits);
 }
 
-static inline int bitmap_parse(const char *buf, unsigned int buflen,
-			unsigned long *maskp, int nmaskbits)
-{
-	return __bitmap_parse(buf, buflen, 0, maskp, nmaskbits);
-}
-
 /**
  * BITMAP_FROM_U64() - Represent u64 value in the format suitable for bitmap.
  * @n: u64 value
diff --git a/lib/bitmap.c b/lib/bitmap.c
index 300732031fad..c276c7e0bb53 100644
--- a/lib/bitmap.c
+++ b/lib/bitmap.c
@@ -335,97 +335,6 @@ EXPORT_SYMBOL(bitmap_find_next_zero_area_off);
  * second version by Paul Jackson, third by Joe Korty.
  */
 
-#define CHUNKSZ				32
-#define nbits_to_hold_value(val)	fls(val)
-#define BASEDEC 10		/* fancier cpuset lists input in decimal */
-
-/**
- * __bitmap_parse - convert an ASCII hex string into a bitmap.
- * @buf: pointer to buffer containing string.
- * @buflen: buffer size in bytes.  If string is smaller than this
- *    then it must be terminated with a \0.
- * @is_user: location of buffer, 0 indicates kernel space
- * @maskp: pointer to bitmap array that will contain result.
- * @nmaskbits: size of bitmap, in bits.
- *
- * Commas group hex digits into chunks.  Each chunk defines exactly 32
- * bits of the resultant bitmask.  No chunk may specify a value larger
- * than 32 bits (%-EOVERFLOW), and if a chunk specifies a smaller value
- * then leading 0-bits are prepended.  %-EINVAL is returned for illegal
- * characters and for grouping errors such as "1,,5", ",44", "," and "".
- * Leading and trailing whitespace accepted, but not embedded whitespace.
- */
-int __bitmap_parse(const char *buf, unsigned int buflen,
-		int is_user, unsigned long *maskp,
-		int nmaskbits)
-{
-	int c, old_c, totaldigits, ndigits, nchunks, nbits;
-	u32 chunk;
-	const char __user __force *ubuf = (const char __user __force *)buf;
-
-	bitmap_zero(maskp, nmaskbits);
-
-	nchunks = nbits = totaldigits = c = 0;
-	do {
-		chunk = 0;
-		ndigits = totaldigits;
-
-		/* Get the next chunk of the bitmap */
-		while (buflen) {
-			old_c = c;
-			if (is_user) {
-				if (__get_user(c, ubuf++))
-					return -EFAULT;
-			}
-			else
-				c = *buf++;
-			buflen--;
-			if (isspace(c))
-				continue;
-
-			/*
-			 * If the last character was a space and the current
-			 * character isn't '\0', we've got embedded whitespace.
-			 * This is a no-no, so throw an error.
-			 */
-			if (totaldigits && c && isspace(old_c))
-				return -EINVAL;
-
-			/* A '\0' or a ',' signal the end of the chunk */
-			if (c == '\0' || c == ',')
-				break;
-
-			if (!isxdigit(c))
-				return -EINVAL;
-
-			/*
-			 * Make sure there are at least 4 free bits in 'chunk'.
-			 * If not, this hexdigit will overflow 'chunk', so
-			 * throw an error.
-			 */
-			if (chunk & ~((1UL << (CHUNKSZ - 4)) - 1))
-				return -EOVERFLOW;
-
-			chunk = (chunk << 4) | hex_to_bin(c);
-			totaldigits++;
-		}
-		if (ndigits == totaldigits)
-			return -EINVAL;
-		if (nchunks == 0 && chunk == 0)
-			continue;
-
-		__bitmap_shift_left(maskp, maskp, CHUNKSZ, nmaskbits);
-		*maskp |= chunk;
-		nchunks++;
-		nbits += (nchunks == 1) ? nbits_to_hold_value(chunk) : CHUNKSZ;
-		if (nbits > nmaskbits)
-			return -EOVERFLOW;
-	} while (buflen && c == ',');
-
-	return 0;
-}
-EXPORT_SYMBOL(__bitmap_parse);
-
 /**
  * bitmap_parse_user - convert an ASCII hex string in a user buffer into a bitmap
  *
@@ -446,7 +355,7 @@ int bitmap_parse_user(const char __user *ubuf,
 	if (IS_ERR(buf))
 		return PTR_ERR(buf);
 
-	ret = bitmap_parse(buf, ulen, maskp, nmaskbits);
+	ret = bitmap_parse(buf, UINT_MAX, maskp, nmaskbits);
 
 	kfree(buf);
 	return ret;
@@ -545,6 +454,11 @@ static inline bool end_of_region(char c)
 	return __end_of_region(c) || end_of_str(c);
 }
 
+static inline bool in_str(const char *start, const char *ptr)
+{
+	return start <= ptr;
+}
+
 /*
  * The format allows commas and whitespases at the beginning
  * of the region.
@@ -557,6 +471,14 @@ static const char *bitmap_find_region(const char *str)
 	return end_of_str(*str) ? NULL : str;
 }
 
+static const char *bitmap_find_region_rev(const char *start, const char *end)
+{
+	while (in_str(start, end) && __end_of_region(*end))
+		end--;
+
+	return end;
+}
+
 static const char *bitmap_parse_region(const char *str, struct region *r)
 {
 	str = bitmap_getnum(str, &r->start);
@@ -680,6 +602,79 @@ int bitmap_parselist_user(const char __user *ubuf,
 }
 EXPORT_SYMBOL(bitmap_parselist_user);
 
+static const char *bitmap_get_hex32_rev(const char *start,
+					const char *end, u32 *num)
+{
+	u32 ret = 0;
+	int c, i;
+
+	if (hex_to_bin(*end) < 0)
+		return ERR_PTR(-EINVAL);
+
+	for (i = 0; i < 32; i += 4) {
+		c = hex_to_bin(*end--);
+		if (c < 0)
+			return ERR_PTR(-EINVAL);
+
+		ret |= c << i;
+
+		if (!in_str(start, end) || __end_of_region(*end))
+			goto out;
+	}
+
+	if (hex_to_bin(*end) >= 0)
+		return ERR_PTR(-EOVERFLOW);
+out:
+	*num = ret;
+	return end;
+}
+
+/**
+ * bitmap_parse - convert an ASCII hex string into a bitmap.
+ * @start: pointer to buffer containing string.
+ * @buflen: buffer size in bytes.  If string is smaller than this
+ *    then it must be terminated with a \0 or \n. In that case,
+ *    UINT_MAX may be provided instead of string length.
+ * @maskp: pointer to bitmap array that will contain result.
+ * @nmaskbits: size of bitmap, in bits.
+ *
+ * Commas group hex digits into chunks.  Each chunk defines exactly 32
+ * bits of the resultant bitmask.  No chunk may specify a value larger
+ * than 32 bits (%-EOVERFLOW), and if a chunk specifies a smaller value
+ * then leading 0-bits are prepended.  %-EINVAL is returned for illegal
+ * characters. Grouping such as "1,,5", ",44", "," or "" is allowed.
+ * Leading, embedded and trailing whitespace accepted.
+ */
+int bitmap_parse(const char *start, unsigned int buflen,
+		unsigned long *maskp, int nmaskbits)
+{
+	const char *end = strnchrnul(start, buflen, '\n') - 1;
+	int chunks = BITS_TO_U32(nmaskbits);
+	u32 *bitmap = (u32 *)maskp;
+	int unset_bit;
+
+	while (in_str(start, (end = bitmap_find_region_rev(start, end)))) {
+		if (!chunks--)
+			return -EOVERFLOW;
+
+		end = bitmap_get_hex32_rev(start, end, bitmap++);
+		if (IS_ERR(end))
+			return PTR_ERR(end);
+	}
+
+	unset_bit = (BITS_TO_U32(nmaskbits) - chunks) * 32;
+	if (unset_bit < nmaskbits) {
+		bitmap_clear(maskp, unset_bit, nmaskbits);
+		return 0;
+	}
+
+	if (find_next_bit(maskp, unset_bit, nmaskbits) != unset_bit)
+		return -EOVERFLOW;
+
+	return 0;
+}
+EXPORT_SYMBOL(bitmap_parse);
+
 
 #ifdef CONFIG_NUMA
 /**
-- 
2.17.1

Re: [PATCH 4/6] lib: rework bitmap_parse()

[Andy Shevchenko]

On Sat, Apr 27, 2019 at 08:29:34PM -0700, Yury Norov wrote:
> bitmap_parse() is ineffective and full of opaque variables and opencoded
> parts. It leads to hard understanding of it. This rework includes:
>  - remove bitmap_shift_left() call from the cycle. Now it makes the
>    complexity of the algorithm as O(nbits^2). In the suggested approach
>    the input string is parsed in reverse direction, so no shifts needed;
>  - relax requirement on a single comma and no white spaces between chunks.
>    It is considered useful in scripting, and it aligns with
>    bitmap_parselist();
>  - split bitmap_parse() to small readable helpers;
>  - make an explicit calculation of the end of input line at the
>    beginning, so users of the bitmap_parse() won't bother doing this.

> +static inline bool in_str(const char *start, const char *ptr)
> +{
> +	return start <= ptr;
> +}
> +

I don't see how it's better than explicit use. Moreover, explicit use shows the
exact condition in-line. Even by used characters it's longer.

> +static const char *bitmap_get_hex32_rev(const char *start,
> +					const char *end, u32 *num)

In kernel few functions to work with hex u32 named foo_x32(). I would rather
use that. Besides, we spell "reverse" in full.

> +{
> +	u32 ret = 0;
> +	int c, i;
> +
> +	if (hex_to_bin(*end) < 0)
> +		return ERR_PTR(-EINVAL);
> +
> +	for (i = 0; i < 32; i += 4) {
> +		c = hex_to_bin(*end--);
> +		if (c < 0)

Perhaps we may need similar patch for hex_to_bin() as in the commit
9888a588ea96 ("lib/hexdump.c: return -EINVAL in case of error in hex2bin()")

> +			return ERR_PTR(-EINVAL);

> +
> +		ret |= c << i;
> +
> +		if (!in_str(start, end) || __end_of_region(*end))
> +			goto out;
> +	}
> +
> +	if (hex_to_bin(*end) >= 0)
> +		return ERR_PTR(-EOVERFLOW);
> +out:
> +	*num = ret;
> +	return end;
> +}

-- 
With Best Regards,
Andy Shevchenko

Re: [PATCH 4/6] lib: rework bitmap_parse()

[Yury Norov]

On Sun, Apr 28, 2019 at 07:57:45PM +0300, Andy Shevchenko wrote:
> On Sat, Apr 27, 2019 at 08:29:34PM -0700, Yury Norov wrote:
> > bitmap_parse() is ineffective and full of opaque variables and opencoded
> > parts. It leads to hard understanding of it. This rework includes:
> >  - remove bitmap_shift_left() call from the cycle. Now it makes the
> >    complexity of the algorithm as O(nbits^2). In the suggested approach
> >    the input string is parsed in reverse direction, so no shifts needed;
> >  - relax requirement on a single comma and no white spaces between chunks.
> >    It is considered useful in scripting, and it aligns with
> >    bitmap_parselist();
> >  - split bitmap_parse() to small readable helpers;
> >  - make an explicit calculation of the end of input line at the
> >    beginning, so users of the bitmap_parse() won't bother doing this.
> 
> > +static inline bool in_str(const char *start, const char *ptr)
> > +{
> > +	return start <= ptr;
> > +}
> > +
> 
> I don't see how it's better than explicit use. Moreover, explicit use shows the
> exact condition in-line. Even by used characters it's longer.

I did 2 mistakes with a condition ('<' instead of '<=') during the
development, after that I decided to introduce it. I would prefer
keep it.
 
> > +static const char *bitmap_get_hex32_rev(const char *start,
> > +					const char *end, u32 *num)
> 
> In kernel few functions to work with hex u32 named foo_x32(). I would rather
> use that. Besides, we spell "reverse" in full.

OK

> > +{
> > +	u32 ret = 0;
> > +	int c, i;
> > +
> > +	if (hex_to_bin(*end) < 0)
> > +		return ERR_PTR(-EINVAL);
> > +
> > +	for (i = 0; i < 32; i += 4) {
> > +		c = hex_to_bin(*end--);
> > +		if (c < 0)
> 
> Perhaps we may need similar patch for hex_to_bin() as in the commit
> 9888a588ea96 ("lib/hexdump.c: return -EINVAL in case of error in hex2bin()")
> 
> > +			return ERR_PTR(-EINVAL);
> 
> > +
> > +		ret |= c << i;
> > +
> > +		if (!in_str(start, end) || __end_of_region(*end))
> > +			goto out;
> > +	}
> > +
> > +	if (hex_to_bin(*end) >= 0)
> > +		return ERR_PTR(-EOVERFLOW);
> > +out:
> > +	*num = ret;
> > +	return end;
> > +}
> 
> -- 
> With Best Regards,
> Andy Shevchenko
> 

[PATCH 5/6] lib: add test for bitmap_parse()

[Yury Norov]

The test is derived from bitmap_parselist()

Signed-off-by: Yury Norov <ynorov@marvell.com>
---
 lib/test_bitmap.c | 102 +++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 101 insertions(+), 1 deletion(-)

diff --git a/lib/test_bitmap.c b/lib/test_bitmap.c
index d3a501f2a81a..99ad6c35d038 100644
--- a/lib/test_bitmap.c
+++ b/lib/test_bitmap.c
@@ -205,7 +205,8 @@ static void __init test_copy(void)
 	expect_eq_pbl("0-108,128-1023", bmap2, 1024);
 }
 
-#define PARSE_TIME 0x1
+#define PARSE_TIME	0x1
+#define NO_LEN		0x2
 
 struct test_bitmap_parselist{
 	const int errno;
@@ -328,6 +329,93 @@ static void __init __test_bitmap_parselist(int is_user)
 	}
 }
 
+static const unsigned long parse_test[] __initconst = {
+	BITMAP_FROM_U64(0),
+	BITMAP_FROM_U64(1),
+	BITMAP_FROM_U64(0xdeadbeef),
+	BITMAP_FROM_U64(0x100000000ULL),
+};
+
+static const unsigned long parse_test2[] __initconst = {
+	BITMAP_FROM_U64(0x100000000ULL), BITMAP_FROM_U64(0xdeadbeef),
+	BITMAP_FROM_U64(0x100000000ULL), BITMAP_FROM_U64(0xbaadf00ddeadbeef),
+	BITMAP_FROM_U64(0x100000000ULL), BITMAP_FROM_U64(0x0badf00ddeadbeef),
+};
+
+static const struct test_bitmap_parselist parse_tests[] __initconst = {
+	{0, "",				&parse_test[0 * step], 32, 0},
+	{0, " ",			&parse_test[0 * step], 32, 0},
+	{0, "0",			&parse_test[0 * step], 32, 0},
+	{0, "0\n",			&parse_test[0 * step], 32, 0},
+	{0, "1",			&parse_test[1 * step], 32, 0},
+	{0, "deadbeef",			&parse_test[2 * step], 32, 0},
+	{0, "1,0",			&parse_test[3 * step], 33, 0},
+	{0, "deadbeef,\n,0,1",		&parse_test[2 * step], 96, 0},
+
+	{0, "deadbeef,1,0",		&parse_test2[0 * 2 * step], 96, 0},
+	{0, "baadf00d,deadbeef,1,0",	&parse_test2[1 * 2 * step], 128, 0},
+	{0, "badf00d,deadbeef,1,0",	&parse_test2[2 * 2 * step], 124, 0},
+	{0, "badf00d,deadbeef,1,0",	&parse_test2[2 * 2 * step], 124, NO_LEN},
+	{0, "  badf00d,deadbeef,1,0  ",	&parse_test2[2 * 2 * step], 124, 0},
+	{0, " , badf00d,deadbeef,1,0 , ",	&parse_test2[2 * 2 * step], 124, 0},
+	{0, " , badf00d, ,, ,,deadbeef,1,0 , ",	&parse_test2[2 * 2 * step], 124, 0},
+
+	{-EINVAL,    "goodfood,deadbeef,1,0",	NULL, 128, 0},
+	{-EOVERFLOW, "3,0",			NULL, 33, 0},
+	{-EOVERFLOW, "123badf00d,deadbeef,1,0",	NULL, 128, 0},
+	{-EOVERFLOW, "badf00d,deadbeef,1,0",	NULL, 90, 0},
+	{-EOVERFLOW, "fbadf00d,deadbeef,1,0",	NULL, 95, 0},
+	{-EOVERFLOW, "badf00d,deadbeef,1,0",	NULL, 100, 0},
+};
+
+static void __init __test_bitmap_parse(int is_user)
+{
+	int i;
+	int err;
+	ktime_t time;
+	DECLARE_BITMAP(bmap, 2048);
+	char *mode = is_user ? "_user"  : "";
+
+	for (i = 0; i < ARRAY_SIZE(parse_tests); i++) {
+		struct test_bitmap_parselist test = parse_tests[i];
+
+		if (is_user) {
+			size_t len = strlen(test.in);
+			mm_segment_t orig_fs = get_fs();
+
+			set_fs(KERNEL_DS);
+			time = ktime_get();
+			err = bitmap_parse_user(test.in, len, bmap, test.nbits);
+			time = ktime_get() - time;
+			set_fs(orig_fs);
+		} else {
+			size_t len = test.flags & NO_LEN ?
+				UINT_MAX : strlen(test.in);
+			time = ktime_get();
+			err = bitmap_parse(test.in, len, bmap, test.nbits);
+			time = ktime_get() - time;
+		}
+
+		if (err != test.errno) {
+			pr_err("parse%s: %d: input is %s, errno is %d, expected %d\n",
+					mode, i, test.in, err, test.errno);
+			continue;
+		}
+
+		if (!err && test.expected
+			 && !__bitmap_equal(bmap, test.expected, test.nbits)) {
+			pr_err("parse%s: %d: input is %s, result is 0x%lx, expected 0x%lx\n",
+					mode, i, test.in, bmap[0],
+					*test.expected);
+			continue;
+		}
+
+		if (test.flags & PARSE_TIME)
+			pr_err("parse%s: %d: input is '%s' OK, Time: %llu\n",
+					mode, i, test.in, time);
+	}
+}
+
 static void __init test_bitmap_parselist(void)
 {
 	__test_bitmap_parselist(0);
@@ -338,6 +426,16 @@ static void __init test_bitmap_parselist_user(void)
 	__test_bitmap_parselist(1);
 }
 
+static void __init test_bitmap_parse(void)
+{
+	__test_bitmap_parse(0);
+}
+
+static void __init test_bitmap_parse_user(void)
+{
+	__test_bitmap_parse(1);
+}
+
 #define EXP_BYTES	(sizeof(exp) * 8)
 
 static void __init test_bitmap_arr32(void)
@@ -409,6 +507,8 @@ static void __init selftest(void)
 	test_fill_set();
 	test_copy();
 	test_bitmap_arr32();
+	test_bitmap_parse();
+	test_bitmap_parse_user();
 	test_bitmap_parselist();
 	test_bitmap_parselist_user();
 	test_mem_optimisations();
-- 
2.17.1

[PATCH 6/6] cpumask: don't calculate length of the input string

[Yury Norov]

New design of inner bitmap_parse() allows to avoid
calculating the size of a null-terminated string.

Signed-off-by: Yury Norov <ynorov@marvell.com>
---
 include/linux/cpumask.h | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

diff --git a/include/linux/cpumask.h b/include/linux/cpumask.h
index 21755471b1c3..d55d015edc58 100644
--- a/include/linux/cpumask.h
+++ b/include/linux/cpumask.h
@@ -633,9 +633,7 @@ static inline int cpumask_parselist_user(const char __user *buf, int len,
  */
 static inline int cpumask_parse(const char *buf, struct cpumask *dstp)
 {
-	unsigned int len = strchrnul(buf, '\n') - buf;
-
-	return bitmap_parse(buf, len, cpumask_bits(dstp), nr_cpumask_bits);
+	return bitmap_parse(buf, UINT_MAX, cpumask_bits(dstp), nr_cpumask_bits);
 }
 
 /**
-- 
2.17.1

Re: [PATCH 0/4] lib: rework bitmap_parse

[Andy Shevchenko]

On Sat, Apr 27, 2019 at 08:29:30PM -0700, Yury Norov wrote:
> On top of next-20190418.
> 
> Similarly to recently revisited bitmap_parselist() [1],
> bitmap_parse() is ineffective and overcomplicated.  This
> series reworks it, aligns its interface with bitmap_parselist()
> and makes usage simpler.
> 
> The series also adds a test for the function and fixes usage of it
> in cpumask_parse() according to new design - drops the calculating
> of length of an input string.
> 
> The following users would also consider to drop the length argument,
> if possible:
> drivers/block/drbd/drbd_main.c:2608
> kernel/padata.c:924
> net/core/net-sysfs.c:726
> net/core/net-sysfs.c:1309
> net/core/net-sysfs.c:1391
> 
> bitmap_parse() takes the array of numbers to be put into the map in
> the BE order which is reversed to the natural LE order for bitmaps.
> For example, to construct bitmap containing a bit on the position 42,
> we have to put a line '400,0'. Current implementation reads chunk
> one by one from the beginning ('400' before '0') and makes bitmap
> shift after each successful parse. It makes the complexity of the
> whole process as O(n^2). We can do it in reverse direction ('0'
> before '400') and avoid shifting, but it requires reverse parsing
> helpers.

Shouldn't test ceases pass on existing code base? I would rather start series
with enabling tests and then we will have a proof that before and after at
least tests have not been broken.

> 
> Tested on arm64 and BE mips.
> 
> [1] https://lkml.org/lkml/2019/4/16/66
> 
> Yury Norov (6):
>   lib/string: add strnchrnul()
>   bitops: more BITS_TO_* macros
>   lib/bitmap: make bitmap_parse_user a wrapper on bitmap_parse
>   lib: rework bitmap_parse()
>   lib: add test for bitmap_parse()
>   cpumask: don't calculate length of the input string
> 
>  include/linux/bitmap.h       |   8 +-
>  include/linux/bitops.h       |   3 +
>  include/linux/cpumask.h      |   4 +-
>  include/linux/string.h       |   3 +
>  lib/bitmap.c                 | 197 +++++++++++++++++------------------
>  lib/string.c                 |  20 ++++
>  lib/test_bitmap.c            | 102 +++++++++++++++++-
>  tools/include/linux/bitops.h |   9 +-
>  8 files changed, 230 insertions(+), 116 deletions(-)
> 
> -- 
> 2.17.1
> 

-- 
With Best Regards,
Andy Shevchenko