Re: [net-next PATCH v5 2/3] sysctl: add proc_do_large_bitmap

Re: [net-next PATCH v5 2/3] sysctl: add proc_do_large_bitmap

[Octavian Purdila]

>
> My test shows it still accepts spaces, e.g.
>
> echo '50000 50003 50005' > ip_local_reserved_ports
>
> works same as
>
> echo '50000,50003,50005' > ip_local_reserved_ports
>
> Is this expected? We will only accept commas, right?
>

Thanks for testing, I didn't saw that comming! I wanted to allow whitespaces in between the commas but it looks like I got overzealous. I can easily fix that. 

But I think its worth to keep the whitespaces in beetween, e.g. allow

$ echo '1, 2 ,3 ' >   ip_local_reserved_ports.

>
> Also, if I write an invalid value, it does reject this, but the previous
> value in that file is cleared, shouldn't we keep the previous one?
>
>

The only way I see to fix this is to return EINVAL if we detect a write with offset.

IMO we should do that for the other proc write routines as well, as otherwise ther result is confusing, e.g.

write("1 2"); write(" 3");

will set first value in the vector to 1, than second value to 2 then *first* value to 3.

I am all for it, but again, this changes userspace ABI. 

Re: [net-next PATCH v5 2/3] sysctl: add proc_do_large_bitmap

[Cong Wang]

Octavian Purdila wrote:
> 
> But I think its worth to keep the whitespaces in beetween, e.g. allow
> 
> $ echo '1, 2 ,3 ' >   ip_local_reserved_ports.


Sure.

> 
>> Also, if I write an invalid value, it does reject this, but the previous
>> value in that file is cleared, shouldn't we keep the previous one?
>>
>>
> 
> The only way I see to fix this is to return EINVAL if we detect a write with offset.


Yeah, we shouldn't continue once we find any invalid value.

> 
> IMO we should do that for the other proc write routines as well, as otherwise ther result is confusing, e.g.
> 
> write("1 2"); write(" 3");
> 
> will set first value in the vector to 1, than second value to 2 then *first* value to 3.
> 
> I am all for it, but again, this changes userspace ABI. 

Sorry, is this related with the problem I mentioned above? Both "1 2"
and " 3" are valid values.

Thanks.

Re: [net-next PATCH v5 2/3] sysctl: add proc_do_large_bitmap

[Octavian Purdila]

On Monday 22 February 2010 04:45:44 you wrote:
> Octavian Purdila wrote:
> > But I think its worth to keep the whitespaces in beetween, e.g. allow
> >
> > $ echo '1, 2 ,3 ' >   ip_local_reserved_ports.
> 
> Sure.
> 
> >> Also, if I write an invalid value, it does reject this, but the previous
> >> value in that file is cleared, shouldn't we keep the previous one?
> >
> > The only way I see to fix this is to return EINVAL if we detect a write
> > with offset.
> 
> Yeah, we shouldn't continue once we find any invalid value.
> 
> > IMO we should do that for the other proc write routines as well, as
> > otherwise ther result is confusing, e.g.
> >
> > write("1 2"); write(" 3");
> >
> > will set first value in the vector to 1, than second value to 2 then
> > *first* value to 3.
> >
> > I am all for it, but again, this changes userspace ABI.
> 
> Sorry, is this related with the problem I mentioned above? Both "1 2"
> and " 3" are valid values.
> 

Never mind, please disregard my "return EINVAL if we detect a write
with offset", it breaks the ABI in a significant way.

Here is a new version of this patch which fixes both the comma and invalid 
value issues, please give it a try.

[net-next PATCH v5 2/3] sysctl: add proc_do_large_bitmap

The new function can be used to read/write large bitmaps via /proc. A
comma separated range format is used for compact output and input
(e.g. 1,3-4,10-10).

Writing into the file will first reset the bitmap then update it
based on the given input.

Signed-off-by: Octavian Purdila <opurdila@ixiacom.com>
Cc: WANG Cong <amwang@redhat.com>
Cc: Eric W. Biederman <ebiederm@xmission.com>
---
 include/linux/sysctl.h |    2 +
 kernel/sysctl.c        |  134 
++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 136 insertions(+), 0 deletions(-)

diff --git a/include/linux/sysctl.h b/include/linux/sysctl.h
index f66014c..7bb5cb6 100644
--- a/include/linux/sysctl.h
+++ b/include/linux/sysctl.h
@@ -980,6 +980,8 @@ extern int proc_doulongvec_minmax(struct ctl_table *, int,
 				  void __user *, size_t *, loff_t *);
 extern int proc_doulongvec_ms_jiffies_minmax(struct ctl_table *table, int,
 				      void __user *, size_t *, loff_t *);
+extern int proc_do_large_bitmap(struct ctl_table *, int,
+				void __user *, size_t *, loff_t *);
 
 /*
  * Register a set of sysctl names by calling register_sysctl_table
diff --git a/kernel/sysctl.c b/kernel/sysctl.c
index 5259727..d8ea839 100644
--- a/kernel/sysctl.c
+++ b/kernel/sysctl.c
@@ -2635,6 +2635,140 @@ static int proc_do_cad_pid(struct ctl_table *table, 
int write,
 	return 0;
 }
 
+/**
+ * proc_do_large_bitmap - read/write from/to a large bitmap
+ * @table: the sysctl table
+ * @write: %TRUE if this is a write to the sysctl file
+ * @buffer: the user buffer
+ * @lenp: the size of the user buffer
+ * @ppos: file position
+ *
+ * The bitmap is stored at table->data and the bitmap length (in bits)
+ * in table->maxlen.
+ *
+ * We use a range comma separated format (e.g. 1,3-4,10-10) so that
+ * large bitmaps may be represented in a compact manner. Writing into
+ * the file will clear the bitmap then update it with the given input.
+ *
+ * Returns 0 on success.
+ */
+int proc_do_large_bitmap(struct ctl_table *table, int write,
+			 void __user *_buffer, size_t *lenp, loff_t *ppos)
+{
+	bool first = 1;
+	unsigned long *bitmap = (unsigned long *) table->data;
+	unsigned long bitmap_len = table->maxlen;
+	int left = *lenp, err = 0;
+	char __user *buffer = (char __user *) _buffer;
+	char tr_a[] = { '-', ',', 0 }, tr_b[] = { ',', 0 }, c;
+
+
+	if (!bitmap_len || !left || (*ppos && !write)) {
+		*lenp = 0;
+		return 0;
+	}
+
+	if (write) {
+		err = proc_skip_wspace(&buffer, &left);
+		while (!err && left) {
+			unsigned long val_a, val_b;
+			bool neg;
+
+			err = proc_get_ulong(&buffer, &left, tr_a,
+					     &val_a, &neg, &c);
+			if (err)
+				break;
+
+			if (val_a >= bitmap_len || neg) {
+				err = -EINVAL;
+				break;
+			}
+
+			val_b = val_a;
+			if (left && c == '-') {
+				/* skip the - */
+				buffer++; left--;
+
+				err = proc_get_ulong(&buffer, &left, tr_b,
+						     &val_b, &neg, &c);
+				if (err)
+					break;
+
+				if (val_b >= bitmap_len || neg ||
+				    val_a > val_b) {
+					err = -EINVAL;
+					break;
+				}
+			}
+
+			if (left) {
+				err = proc_skip_wspace(&buffer, &left);
+				if (err)
+					break;
+				if (left) {
+					if (get_user(c, buffer)) {
+						err = -EFAULT;
+						break;
+					}
+					if (c != ',') {
+						err = -EINVAL;
+						break;
+					}
+					/* skip the , */
+					buffer++; left--;
+				}
+			}
+
+			if (first)
+				bitmap_clear(bitmap, 0, bitmap_len);
+
+			while (val_a <= val_b)
+				set_bit(val_a++, bitmap);
+
+			first = 0;
+		}
+		if (!err)
+			err = proc_skip_wspace(&buffer, &left);
+	} else {
+		unsigned long bit_a, bit_b = 0;
+
+		while (left) {
+			bit_a = find_next_bit(bitmap, bitmap_len, bit_b);
+			if (bit_a >= bitmap_len)
+				break;
+			bit_b = find_next_zero_bit(bitmap, bitmap_len,
+						   bit_a + 1) - 1;
+
+			err = proc_put_ulong(&buffer, &left, bit_a, 0, first,
+					     ',');
+			if (err)
+				break;
+			if (bit_a != bit_b) {
+				err = proc_put_char(&buffer, &left, '-');
+				if (err)
+					break;
+				err = proc_put_ulong(&buffer, &left, bit_b, 0,
+						     1, 0);
+				if (err)
+					break;
+			}
+
+			first = 0; bit_b++;
+		}
+		if (!err)
+			err = proc_put_char(&buffer, &left, '\n');
+	}
+
+	if (first) {
+		if (err)
+			return err;
+		bitmap_clear(bitmap, 0, bitmap_len);
+	}
+	*lenp -= left;
+	*ppos += *lenp;
+	return 0;
+}
+
 #else /* CONFIG_PROC_FS */
 
 int proc_dostring(struct ctl_table *table, int write,
-- 
1.5.6.5

Re: [net-next PATCH v5 2/3] sysctl: add proc_do_large_bitmap

[Cong Wang]

Octavian Purdila wrote:
> 
> Here is a new version of this patch which fixes both the comma and invalid 
> value issues, please give it a try.
> 

Sorry, it is even worse. :(

> [net-next PATCH v5 2/3] sysctl: add proc_do_large_bitmap
> 
> The new function can be used to read/write large bitmaps via /proc. A
> comma separated range format is used for compact output and input
> (e.g. 1,3-4,10-10).
> 


Writing "50000-50100" gets EINVAL, it should be success.
Writing "50000,50100" fails too.

Please, at least, do some basic testing.

Also some comments below.

> Writing into the file will first reset the bitmap then update it
> based on the given input.
> 
> Signed-off-by: Octavian Purdila <opurdila@ixiacom.com>
> Cc: WANG Cong <amwang@redhat.com>
> Cc: Eric W. Biederman <ebiederm@xmission.com>

Please resend the whole patchset, and update the doc in patch 3/3.


> ---
>  include/linux/sysctl.h |    2 +
>  kernel/sysctl.c        |  134 
> ++++++++++++++++++++++++++++++++++++++++++++++++
>  2 files changed, 136 insertions(+), 0 deletions(-)
> 
> diff --git a/include/linux/sysctl.h b/include/linux/sysctl.h
> index f66014c..7bb5cb6 100644
> --- a/include/linux/sysctl.h
> +++ b/include/linux/sysctl.h
> @@ -980,6 +980,8 @@ extern int proc_doulongvec_minmax(struct ctl_table *, int,
>  				  void __user *, size_t *, loff_t *);
>  extern int proc_doulongvec_ms_jiffies_minmax(struct ctl_table *table, int,
>  				      void __user *, size_t *, loff_t *);
> +extern int proc_do_large_bitmap(struct ctl_table *, int,
> +				void __user *, size_t *, loff_t *);
>  
>  /*
>   * Register a set of sysctl names by calling register_sysctl_table
> diff --git a/kernel/sysctl.c b/kernel/sysctl.c
> index 5259727..d8ea839 100644
> --- a/kernel/sysctl.c
> +++ b/kernel/sysctl.c
> @@ -2635,6 +2635,140 @@ static int proc_do_cad_pid(struct ctl_table *table, 
> int write,


The above line is wrong, it should be a part of previous line.


>  	return 0;
>  }
>  
> +/**
> + * proc_do_large_bitmap - read/write from/to a large bitmap
> + * @table: the sysctl table
> + * @write: %TRUE if this is a write to the sysctl file
> + * @buffer: the user buffer
> + * @lenp: the size of the user buffer
> + * @ppos: file position
> + *
> + * The bitmap is stored at table->data and the bitmap length (in bits)
> + * in table->maxlen.
> + *
> + * We use a range comma separated format (e.g. 1,3-4,10-10) so that
> + * large bitmaps may be represented in a compact manner. Writing into
> + * the file will clear the bitmap then update it with the given input.
> + *
> + * Returns 0 on success.
> + */
> +int proc_do_large_bitmap(struct ctl_table *table, int write,
> +			 void __user *_buffer, size_t *lenp, loff_t *ppos)
> +{
> +	bool first = 1;
> +	unsigned long *bitmap = (unsigned long *) table->data;
> +	unsigned long bitmap_len = table->maxlen;
> +	int left = *lenp, err = 0;


'left' should be size_t.


> +	char __user *buffer = (char __user *) _buffer;
> +	char tr_a[] = { '-', ',', 0 }, tr_b[] = { ',', 0 }, c;
> +
> +
> +	if (!bitmap_len || !left || (*ppos && !write)) {
> +		*lenp = 0;
> +		return 0;
> +	}
> +
> +	if (write) {
> +		err = proc_skip_wspace(&buffer, &left);
> +		while (!err && left) {
> +			unsigned long val_a, val_b;
> +			bool neg;
> +
> +			err = proc_get_ulong(&buffer, &left, tr_a,
> +					     &val_a, &neg, &c);
> +			if (err)
> +				break;
> +
> +			if (val_a >= bitmap_len || neg) {
> +				err = -EINVAL;
> +				break;
> +			}
> +
> +			val_b = val_a;
> +			if (left && c == '-') {
> +				/* skip the - */
> +				buffer++; left--;
> +
> +				err = proc_get_ulong(&buffer, &left, tr_b,
> +						     &val_b, &neg, &c);
> +				if (err)
> +					break;
> +
> +				if (val_b >= bitmap_len || neg ||
> +				    val_a > val_b) {
> +					err = -EINVAL;
> +					break;
> +				}
> +			}
> +
> +			if (left) {
> +				err = proc_skip_wspace(&buffer, &left);
> +				if (err)
> +					break;
> +				if (left) {
> +					if (get_user(c, buffer)) {
> +						err = -EFAULT;
> +						break;
> +					}
> +					if (c != ',') {
> +						err = -EINVAL;
> +						break;
> +					}
> +					/* skip the , */
> +					buffer++; left--;
> +				}
> +			}
> +
> +			if (first)
> +				bitmap_clear(bitmap, 0, bitmap_len);
> +
> +			while (val_a <= val_b)
> +				set_bit(val_a++, bitmap);
> +
> +			first = 0;
> +		}
> +		if (!err)
> +			err = proc_skip_wspace(&buffer, &left);
> +	} else {
> +		unsigned long bit_a, bit_b = 0;
> +
> +		while (left) {
> +			bit_a = find_next_bit(bitmap, bitmap_len, bit_b);
> +			if (bit_a >= bitmap_len)
> +				break;
> +			bit_b = find_next_zero_bit(bitmap, bitmap_len,
> +						   bit_a + 1) - 1;
> +
> +			err = proc_put_ulong(&buffer, &left, bit_a, 0, first,
> +					     ',');
> +			if (err)
> +				break;
> +			if (bit_a != bit_b) {
> +				err = proc_put_char(&buffer, &left, '-');
> +				if (err)
> +					break;
> +				err = proc_put_ulong(&buffer, &left, bit_b, 0,
> +						     1, 0);
> +				if (err)
> +					break;
> +			}
> +
> +			first = 0; bit_b++;
> +		}
> +		if (!err)
> +			err = proc_put_char(&buffer, &left, '\n');
> +	}
> +
> +	if (first) {
> +		if (err)
> +			return err;
> +		bitmap_clear(bitmap, 0, bitmap_len);
> +	}
> +	*lenp -= left;
> +	*ppos += *lenp;
> +	return 0;
> +}
> +
>  #else /* CONFIG_PROC_FS */
>  
>  int proc_dostring(struct ctl_table *table, int write,

Re: [net-next PATCH v5 2/3] sysctl: add proc_do_large_bitmap

[Cong Wang]

Cong Wang wrote:
>> diff --git a/kernel/sysctl.c b/kernel/sysctl.c
>> index 5259727..d8ea839 100644
>> --- a/kernel/sysctl.c
>> +++ b/kernel/sysctl.c
>> @@ -2635,6 +2635,140 @@ static int proc_do_cad_pid(struct ctl_table 
>> *table, int write,
> 
> 
> The above line is wrong, it should be a part of previous line.
> 
> 

Sorry, ignore this, it should be my thunderbird's fault,
it truncated the line. I will fix it...

Re: [net-next PATCH v5 2/3] sysctl: add proc_do_large_bitmap

[Octavian Purdila]

[-- Attachment #1: Type: Text/Plain, Size: 1742 bytes --]


On Wednesday 24 February 2010 07:24:00 you wrote:
> Octavian Purdila wrote:
> > Here is a new version of this patch which fixes both the comma and
> > invalid value issues, please give it a try.
> 
> Sorry, it is even worse. :(
> 
> > [net-next PATCH v5 2/3] sysctl: add proc_do_large_bitmap
> >
> > The new function can be used to read/write large bitmaps via /proc. A
> > comma separated range format is used for compact output and input
> > (e.g. 1,3-4,10-10).
> 
> Writing "50000-50100" gets EINVAL, it should be success.
> Writing "50000,50100" fails too.
> 

Hmm, they don't fail for me :-/

> Please, at least, do some basic testing.
> 

I do test them, I've attached the current test batch I was using.

Anyways, today I've noticed that "1,2 3" does not fail and even more 
importantly the final value is "3". 

Being that I don't see a way of fixing this without not acknowledging 1,2 even 
though we will do set these values, I revisited the "1 2 3" issue. And I don't 
understand why this is actually an issue, we are just being more permissive 
(i.e. we are allowing as separators both whitespaces and ,).


> Also some comments below.
> 
> > diff --git a/kernel/sysctl.c b/kernel/sysctl.c
> > index 5259727..d8ea839 100644
> > --- a/kernel/sysctl.c
> > +++ b/kernel/sysctl.c
> > @@ -2635,6 +2635,140 @@ static int proc_do_cad_pid(struct ctl_table
> > *table, int write,
> 
> The above line is wrong, it should be a part of previous line.
> 

Probably my email client corrupted the patch, sorry about that, I will be more 
careful next time.

> > +	int left = *lenp, err = 0;
> 
> 'left' should be size_t.
> 

Will fix, thanks for catching this.

I will resend the whole patch series once we get this formatting issue 
resolved.

[-- Attachment #2: rp_test.log --]
[-- Type: text/x-log, Size: 1341 bytes --]

+ cat /proc/sys/net/ipv4/ip_local_reserved_ports
40002
+ echo 50000
+ cat /proc/sys/net/ipv4/ip_local_reserved_ports
50000
+ echo 50000-50100
+ cat /proc/sys/net/ipv4/ip_local_reserved_ports
50000-50100
+ echo
+ cat /proc/sys/net/ipv4/ip_local_reserved_ports

+ echo 40000-50000,50000-50100,50101-50101
+ cat /proc/sys/net/ipv4/ip_local_reserved_ports
40000-50101
+ set +e
+ echo 40000a
rp_test.sh: line 12: echo: write error: Invalid argument
+ cat /proc/sys/net/ipv4/ip_local_reserved_ports
40000-50101
+ echo 40000-
rp_test.sh: line 14: echo: write error: Invalid argument
+ cat /proc/sys/net/ipv4/ip_local_reserved_ports
40000-50101
+ echo 70000
rp_test.sh: line 16: echo: write error: Invalid argument
+ cat /proc/sys/net/ipv4/ip_local_reserved_ports
40000-50101
+ echo 40000-30000
rp_test.sh: line 18: echo: write error: Invalid argument
+ cat /proc/sys/net/ipv4/ip_local_reserved_ports
40000-50101
+ echo 40000,
+ cat /proc/sys/net/ipv4/ip_local_reserved_ports
40000
+ echo 40000,40001,40000b
rp_test.sh: line 22: echo: write error: Invalid argument
+ cat /proc/sys/net/ipv4/ip_local_reserved_ports
40000-40001
+ echo 40000 40001 40002
rp_test.sh: line 24: echo: write error: Invalid argument
+ cat /proc/sys/net/ipv4/ip_local_reserved_ports
40000-40001
+ echo 40000,40001 40002
+ cat /proc/sys/net/ipv4/ip_local_reserved_ports
40002

[-- Attachment #3: rp_test.sh --]
[-- Type: application/x-shellscript, Size: 1373 bytes --]

Re: [net-next PATCH v5 2/3] sysctl: add proc_do_large_bitmap

[Cong Wang]

Octavian Purdila wrote:
> On Wednesday 24 February 2010 07:24:00 you wrote:
>> Octavian Purdila wrote:
>>> Here is a new version of this patch which fixes both the comma and
>>> invalid value issues, please give it a try.
>> Sorry, it is even worse. :(
>>
>>> [net-next PATCH v5 2/3] sysctl: add proc_do_large_bitmap
>>>
>>> The new function can be used to read/write large bitmaps via /proc. A
>>> comma separated range format is used for compact output and input
>>> (e.g. 1,3-4,10-10).
>> Writing "50000-50100" gets EINVAL, it should be success.
>> Writing "50000,50100" fails too.
>>
> 
> Hmm, they don't fail for me :-/
> 
>> Please, at least, do some basic testing.
>>
> 
> I do test them, I've attached the current test batch I was using.
> 
> Anyways, today I've noticed that "1,2 3" does not fail and even more 
> importantly the final value is "3". 
> 
> Being that I don't see a way of fixing this without not acknowledging 1,2 even 
> though we will do set these values, I revisited the "1 2 3" issue. And I don't 
> understand why this is actually an issue, we are just being more permissive 
> (i.e. we are allowing as separators both whitespaces and ,).
> 
> 

Oops, after rechecking my test case, it is actually my test case's
fault. Sorry for this. I will fix my test case and run it again.


> 
> I will resend the whole patch series once we get this formatting issue 
> resolved.
> 

Thanks much!

Re: [net-next PATCH v5 2/3] sysctl: add proc_do_large_bitmap

[Cong Wang]

Cong Wang wrote:
> Octavian Purdila wrote:
>> On Wednesday 24 February 2010 07:24:00 you wrote:
>>> Octavian Purdila wrote:
>>>> Here is a new version of this patch which fixes both the comma and
>>>> invalid value issues, please give it a try.
>>> Sorry, it is even worse. :(
>>>
>>>> [net-next PATCH v5 2/3] sysctl: add proc_do_large_bitmap
>>>>
>>>> The new function can be used to read/write large bitmaps via /proc. A
>>>> comma separated range format is used for compact output and input
>>>> (e.g. 1,3-4,10-10).
>>> Writing "50000-50100" gets EINVAL, it should be success.
>>> Writing "50000,50100" fails too.
>>>
>>
>> Hmm, they don't fail for me :-/
>>
>>> Please, at least, do some basic testing.
>>>
>>
>> I do test them, I've attached the current test batch I was using.
>>
>> Anyways, today I've noticed that "1,2 3" does not fail and even more 
>> importantly the final value is "3".
>> Being that I don't see a way of fixing this without not acknowledging 
>> 1,2 even though we will do set these values, I revisited the "1 2 3" 
>> issue. And I don't understand why this is actually an issue, we are 
>> just being more permissive (i.e. we are allowing as separators both 
>> whitespaces and ,).
>>
>>
> 
> Oops, after rechecking my test case, it is actually my test case's
> fault. Sorry for this. I will fix my test case and run it again.
> 

Hi,

Still a small problem, if I do write(fd, "50000,50100", 12) I will
get a return value of 11, which should mean 11 bytes are written,
however, actually only the first 6 bytes are accepted.

The rest looks better now.

Or am I missing something here? :)

Thanks!

Re: [net-next PATCH v5 2/3] sysctl: add proc_do_large_bitmap

[Octavian Purdila]

> I think one of them is enough, since we already chose commas, why
> do we need to add spaces? If you have some strong reason to add it,
> I have no objections.
>

It is just for simpler implementation. It is actually harder to restrict the separator to only commas insted of allowing both spaces and commas, because I rely on functions used for the integer vector handling. 

Maybe I should change those functions to be more generic and thus to allow more stricter input, but I am not sure if its worth it. Isn't a more permissive input format desirable?

Re: [net-next PATCH v5 2/3] sysctl: add proc_do_large_bitmap

[Cong Wang]

Octavian Purdila wrote:
>> I think one of them is enough, since we already chose commas, why
>> do we need to add spaces? If you have some strong reason to add it,
>> I have no objections.
>>
> 
> It is just for simpler implementation. It is actually harder to restrict the separator to only commas insted of allowing both spaces and commas, because I rely on functions used for the integer vector handling. 
> 
> Maybe I should change those functions to be more generic and thus to allow more stricter input, but I am not sure if its worth it. Isn't a more permissive input format desirable?

I think you should ask Eric, since he proposed using commas. :)
I am fine with either of them.

Thanks.

Re: [net-next PATCH v5 2/3] sysctl: add proc_do_large_bitmap

[Octavian Purdila]

>
> Hi,
>
> Still a small problem, if I do write(fd, "50000,50100", 12) I will
> get a return value of 11, which should mean 11 bytes are written,
> however, actually only the first 6 bytes are accepted.
>
> The rest looks better now.
>
> Or am I missing something here? :)
>

Will take a look at this a bit later today, thanks for testing.

In the meanwhile what are your thougths on the "1 2 3" issue, are you OK with accepting spaces as well as commas as separators?

Re: [net-next PATCH v5 2/3] sysctl: add proc_do_large_bitmap

[Cong Wang]

Octavian Purdila wrote:
>> Hi,
>>
>> Still a small problem, if I do write(fd, "50000,50100", 12) I will
>> get a return value of 11, which should mean 11 bytes are written,
>> however, actually only the first 6 bytes are accepted.
>>
>> The rest looks better now.
>>
>> Or am I missing something here? :)
>>
> 
> Will take a look at this a bit later today, thanks for testing.
> 
> In the meanwhile what are your thougths on the "1 2 3" issue, are you OK with accepting spaces as well as commas as separators?

I think one of them is enough, since we already chose commas, why
do we need to add spaces? If you have some strong reason to add it,
I have no objections.


Thanks.

[net-next PATCH v5 2/3] sysctl: add proc_do_large_bitmap

[Octavian Purdila]

The new function can be used to read/write large bitmaps via /proc. A
comma separated range format is used for compact output and input
(e.g. 1,3-4,10-10).

Writing into the file will first reset the bitmap then update it
based on the given input.

Signed-off-by: Octavian Purdila <opurdila@ixiacom.com>
Cc: WANG Cong <amwang@redhat.com>
Cc: Eric W. Biederman <ebiederm@xmission.com>
---
 include/linux/sysctl.h |    2 +
 kernel/sysctl.c        |  122 ++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 124 insertions(+), 0 deletions(-)

diff --git a/include/linux/sysctl.h b/include/linux/sysctl.h
index f66014c..7bb5cb6 100644
--- a/include/linux/sysctl.h
+++ b/include/linux/sysctl.h
@@ -980,6 +980,8 @@ extern int proc_doulongvec_minmax(struct ctl_table *, int,
 				  void __user *, size_t *, loff_t *);
 extern int proc_doulongvec_ms_jiffies_minmax(struct ctl_table *table, int,
 				      void __user *, size_t *, loff_t *);
+extern int proc_do_large_bitmap(struct ctl_table *, int,
+				void __user *, size_t *, loff_t *);
 
 /*
  * Register a set of sysctl names by calling register_sysctl_table
diff --git a/kernel/sysctl.c b/kernel/sysctl.c
index 5259727..ef2c13d 100644
--- a/kernel/sysctl.c
+++ b/kernel/sysctl.c
@@ -2635,6 +2635,128 @@ static int proc_do_cad_pid(struct ctl_table *table, int write,
 	return 0;
 }
 
+/**
+ * proc_do_large_bitmap - read/write from/to a large bitmap
+ * @table: the sysctl table
+ * @write: %TRUE if this is a write to the sysctl file
+ * @buffer: the user buffer
+ * @lenp: the size of the user buffer
+ * @ppos: file position
+ *
+ * The bitmap is stored at table->data and the bitmap length (in bits)
+ * in table->maxlen.
+ *
+ * We use a range comma separated format (e.g. 1,3-4,10-10) so that
+ * large bitmaps may be represented in a compact manner. Writing into
+ * the file will clear the bitmap then update it with the given input.
+ *
+ * Returns 0 on success.
+ */
+int proc_do_large_bitmap(struct ctl_table *table, int write,
+			 void __user *_buffer, size_t *lenp, loff_t *ppos)
+{
+	bool first = 1;
+	unsigned long *bitmap = (unsigned long *) table->data;
+	unsigned long bitmap_len = table->maxlen;
+	int left = *lenp, err = 0;
+	char __user *buffer = (char __user *) _buffer;
+	char tr_a[] = { '-', ',', 0 }, tr_b[] = { ',', 0 }, c;
+
+
+	if (!bitmap_len || !left || (*ppos && !write)) {
+		*lenp = 0;
+		return 0;
+	}
+
+	if (write) {
+		bitmap_clear(bitmap, 0, bitmap_len);
+		err = proc_skip_wspace(&buffer, &left);
+		while (!err && left) {
+			unsigned long val_a, val_b;
+			bool neg;
+
+			err = proc_get_ulong(&buffer, &left, tr_a,
+					     &val_a, &neg, &c);
+			if (err)
+				break;
+
+			if (val_a >= bitmap_len || neg) {
+				err = -EINVAL;
+				break;
+			}
+
+			if (!left || c != '-') {
+				if (c == ',') {
+					buffer++;
+					left--;
+				}
+				val_b = val_a;
+				goto update;
+			}
+
+			/* skip the - */
+			buffer++; left--;
+
+			err = proc_get_ulong(&buffer, &left, tr_b,
+					     &val_b, &neg, &c);
+			if (err)
+				break;
+
+			if (val_b >= bitmap_len || neg || val_a > val_b) {
+				err = -EINVAL;
+				break;
+			}
+
+			if (left && c == ',') {
+				buffer++;
+				left--;
+			}
+
+update:
+			while (val_a <= val_b)
+				set_bit(val_a++, bitmap);
+
+			first = 0; val_b++;
+		}
+		if (!err)
+			err = proc_skip_wspace(&buffer, &left);
+	} else {
+		unsigned long bit_a, bit_b = 0;
+
+		while (left) {
+			bit_a = find_next_bit(bitmap, bitmap_len, bit_b);
+			if (bit_a >= bitmap_len)
+				break;
+			bit_b = find_next_zero_bit(bitmap, bitmap_len,
+						   bit_a + 1) - 1;
+
+			err = proc_put_ulong(&buffer, &left, bit_a, 0, first,
+					     ',');
+			if (err)
+				break;
+			if (bit_a != bit_b) {
+				err = proc_put_char(&buffer, &left, '-');
+				if (err)
+					break;
+				err = proc_put_ulong(&buffer, &left, bit_b, 0,
+						     1, 0);
+				if (err)
+					break;
+			}
+
+			first = 0; bit_b++;
+		}
+		if (!err)
+			err = proc_put_char(&buffer, &left, '\n');
+	}
+
+	if (write && first)
+		return err;
+	*lenp -= left;
+	*ppos += *lenp;
+	return 0;
+}
+
 #else /* CONFIG_PROC_FS */
 
 int proc_dostring(struct ctl_table *table, int write,
-- 
1.5.6.5

Re: [net-next PATCH v5 2/3] sysctl: add proc_do_large_bitmap

[Cong Wang]

Octavian Purdila wrote:
> The new function can be used to read/write large bitmaps via /proc. A
> comma separated range format is used for compact output and input
> (e.g. 1,3-4,10-10).
> 
> Writing into the file will first reset the bitmap then update it
> based on the given input.
> 
> Signed-off-by: Octavian Purdila <opurdila@ixiacom.com>
> Cc: WANG Cong <amwang@redhat.com>
> Cc: Eric W. Biederman <ebiederm@xmission.com>
> ---
>  include/linux/sysctl.h |    2 +
>  kernel/sysctl.c        |  122 ++++++++++++++++++++++++++++++++++++++++++++++++
>  2 files changed, 124 insertions(+), 0 deletions(-)
> 
> diff --git a/include/linux/sysctl.h b/include/linux/sysctl.h
> index f66014c..7bb5cb6 100644
> --- a/include/linux/sysctl.h
> +++ b/include/linux/sysctl.h
> @@ -980,6 +980,8 @@ extern int proc_doulongvec_minmax(struct ctl_table *, int,
>  				  void __user *, size_t *, loff_t *);
>  extern int proc_doulongvec_ms_jiffies_minmax(struct ctl_table *table, int,
>  				      void __user *, size_t *, loff_t *);
> +extern int proc_do_large_bitmap(struct ctl_table *, int,
> +				void __user *, size_t *, loff_t *);
>  
>  /*
>   * Register a set of sysctl names by calling register_sysctl_table
> diff --git a/kernel/sysctl.c b/kernel/sysctl.c
> index 5259727..ef2c13d 100644
> --- a/kernel/sysctl.c
> +++ b/kernel/sysctl.c
> @@ -2635,6 +2635,128 @@ static int proc_do_cad_pid(struct ctl_table *table, int write,
>  	return 0;
>  }
>  
> +/**
> + * proc_do_large_bitmap - read/write from/to a large bitmap
> + * @table: the sysctl table
> + * @write: %TRUE if this is a write to the sysctl file
> + * @buffer: the user buffer
> + * @lenp: the size of the user buffer
> + * @ppos: file position
> + *
> + * The bitmap is stored at table->data and the bitmap length (in bits)
> + * in table->maxlen.
> + *
> + * We use a range comma separated format (e.g. 1,3-4,10-10) so that
> + * large bitmaps may be represented in a compact manner. Writing into
> + * the file will clear the bitmap then update it with the given input.

My test shows it still accepts spaces, e.g.

echo '50000 50003 50005' > ip_local_reserved_ports

works same as

echo '50000,50003,50005' > ip_local_reserved_ports

Is this expected? We will only accept commas, right?


Also, if I write an invalid value, it does reject this, but the previous
value in that file is cleared, shouldn't we keep the previous one?