RFC: Partial IP4 syntax

RFC: Partial IP4 syntax

[Simon Lodal]

While writing yet another IP4 address parser I came up with an idea for 
a more compact IP address/mask notation, to make stuff simpler to write 
and read.

The idea is to just leave out irrelevant octets. There are 3 forms of 
partial addresses:
1) .b.c.d | .c.d | .d (leading dot): Incomplete at beginning.
2) a..d | a..c.d | a.b..d (double dot): Incomplete in the middle.
3) a | a. | a.b | a.b. | a.b.c | a.b.c. (trailing dot or missing 
octets): Incomplete at end.

In each case the position and number of missing octets can be deduced 
and zeroes inserted for them. The parser then calculates a default mask 
which only covers the octets that were defined.

Examples:
10 = 10.0.0.0/8
10. = 10.0.0.0/8
10.0 = 10.0.0.0/16
10.0. = 10.0.0.0/16
.4 = 0.0.0.4/0.0.0.255
.3.4 = 0.0.3.4/0.0.255.255
1..4 = 1.0.0.4/255.0.0.255
.2.3. = invalid, can only be incomplete at one end

I know IP address syntax should not change every day. But this will not 
break or exclude old syntax. I do not see it clashing with other/future 
syntax.

Only minor problem I know of is when parsing netmask which can be in 
cidr or dotted quad form: /32 could be interpreted as a partial address, 
expanded to 32.0.0.0. But it is simple to special case this; make it a 
cidr mask when there is only one octet and it is <=32, otherwise 
interpret as dotted quad (partial or not).

What do you think? Old hat?


Simon

Re: RFC: Partial IP4 syntax

[Ben Efros]

Simon Lodal wrote:

> Examples:
> 10 = 10.0.0.0/8

<SNIP>

> Iknow IP address syntax should not change every day. But this will not 
> break or exclude old syntax. I do not see it clashing with 
> other/future syntax.

Wrong.  IP numbers can also be written in 32-bit unsigned notation.  In 
your first example:
10 being  10.0.0.0/8
10 is actually treated as unsigned 32bit int, meaning you're address is 
actually 0.0.0.10
Try this command and see for yourself:
"ping 5000"
Notice how ping is actually sending to "0.0.19.136" ?

This is done because 32-bit addresses should be representable as a 32bit 
number and not just as a string representation of the address for a 
variety of reasons.

Unfortunately your idea would break a LOT of very useful tools and ways 
of representing valid ip numbers.

Ben

Re: RFC: Partial IP4 syntax

[Simon Lodal]

What tools are passing IP's literally as full 32bits? I understand why 
programs (especially in script languages) might do it internally, but 
between programs?

Funny, works with ping, but not ssh. Works with iptables for --src and 
--dst (surprise). But not for SNAT --to-source.

Just looked at RFC 1123, it only talks about dotted decimal 
representation. The 32bit form is not even documented anywhere afaik, 
and honestly I do not remember seeing it.

If it is really a problem I agree my scheme will break it. Could 
probably be solved by changing the default to incomplete-at-beginning, 
so 10 = .10 = 0.0.0.10.


Simon


>> Iknow IP address syntax should not change every day. But this will not 
>> break or exclude old syntax. I do not see it clashing with 
>> other/future syntax.
> 
> 
> Wrong.  IP numbers can also be written in 32-bit unsigned notation.  In 
> your first example:
> 10 being  10.0.0.0/8
> 10 is actually treated as unsigned 32bit int, meaning you're address is 
> actually 0.0.0.10
> Try this command and see for yourself:
> "ping 5000"
> Notice how ping is actually sending to "0.0.19.136" ?
> 
> This is done because 32-bit addresses should be representable as a 32bit 
> number and not just as a string representation of the address for a 
> variety of reasons.
> 
> Unfortunately your idea would break a LOT of very useful tools and ways 
> of representing valid ip numbers.
> 
> Ben

Re: RFC: Partial IP4 syntax

[Henrik Nordstrom]

On Wed, 29 Sep 2004, Simon Lodal wrote:

> What tools are passing IP's literally as full 32bits? I understand why 
> programs (especially in script languages) might do it internally, but between 
> programs?

IETF has not defined such ascii representation of IPv4 addresses, but the 
BSD libc inet_aton function does and this has been inherited by very many 
systems and tools.

This notation is being depreated as it does not serve a very useful 
purpose and can be somewhat confusing, but it exists and conflicting with 
it would be a bad idea.

> If it is really a problem I agree my scheme will break it. Could probably be 
> solved by changing the default to incomplete-at-beginning, so 10 = .10 = 
> 0.0.0.10.

Not without breaking existing "industry standards" on how IP numbers can 
be typed.

You could use 10. for the 10.0.0.0/8 network.

Regards
Henrik

Re: RFC: Partial IP4 syntax

[Simon Lodal]

Henrik Nordstrom skrev:
> IETF has not defined such ascii representation of IPv4 addresses, but 
> the BSD libc inet_aton function does and this has been inherited by very 
> many systems and tools.
> 
> This notation is being depreated as it does not serve a very useful 
> purpose and can be somewhat confusing, but it exists and conflicting 
> with it would be a bad idea.
Ok, let's see where we may get compatibility problems:
1) iptables input: iptables' support for full 32bit format addresses is 
both lacking and undocumented and could disappear without notice. So if 
any programs use it they should rather be fixed.
2) iptables output: There is a potential problem here, if programs parse 
iptables or iptables-save output, and they know about the full 32bit form.

iptables -L is for human eyes while iptables-save is for machine 
parsing. So what if we make iptables -L print in any format it likes 
(not intended for machine parsing anyway), but have iptables-save always 
print addresses in full dotted quad?

>> If it is really a problem I agree my scheme will break it. Could 
>> probably be solved by changing the default to incomplete-at-beginning, 
>> so 10 = .10 = 0.0.0.10.
> 
> 
> Not without breaking existing "industry standards" on how IP numbers can 
> be typed.
Why? I certainly do not want to break anything, formalized or not, only 
extend.

> You could use 10. for the 10.0.0.0/8 network.
That is what I propose.

The question is how to interpret a single number. Implicitly append or 
prepend a dot? Or interpret as full 32bit notation? Or ignore it?


Simon

Re: RFC: Partial IP4 syntax

[Henrik Nordstrom]

On Wed, 29 Sep 2004, Simon Lodal wrote:

> 1) iptables input: iptables' support for full 32bit format addresses is both 
> lacking and undocumented and could disappear without notice. So if any 
> programs use it they should rather be fixed.

I repead once again, introducing a new notation looking the same as an 
older well established (even if depreated) notation but with a different 
meaning is a terribly bad idea.

> iptables -L is for human eyes while iptables-save is for machine parsing. So 
> what if we make iptables -L print in any format it likes (not intended for 
> machine parsing anyway), but have iptables-save always print addresses in 
> full dotted quad?

Having iptables -L output anything else than quad dotted format is an even 
worse idea. Why outputting any other notation than the official standard?

> The question is how to interpret a single number. Implicitly append or 
> prepend a dot? Or interpret as full 32bit notation? Or ignore it?

Depends on if it is a CIDR number or not.

The following syntaxes I see as acceptable


quad dotted IP, hex or dec

N dotted IP (less than quad), hex or dec notation

CIDR notation
   N octets (up to four) / masksize. Only decimal.

   10/8 == 10.0.0.0/8

Mask notation

   quad or N dotted IP / netmask in quad or N dotted IP form. hex or dec.

To differentiate between CIDR and Mask notation when the mask is specified 
using a single number use the <=32 magics.


I do not find 10. as suitable shorthand for 10.0.0.0/8 even if this form 
is currently not in use in any of the established notations. The problem 
with 10. is that this could just as well be a partially typed IP address 
where the administrator meant to enter more information but forgot. These 
things happens more often than one would think.

Regards
Henrik

Re: RFC: Partial IP4 syntax

[Simon Lodal]

Henrik Nordstrom skrev:
> I repead once again, introducing a new notation looking the same as an 
> older well established (even if depreated) notation but with a different 
> meaning is a terribly bad idea.

Agree, and I am trying to avoid it! By only adding previously unused syntax.

But I still feel that programs relying on undocumented, partly working 
features should know they are in trouble.


>> iptables -L is for human eyes while iptables-save is for machine 
>> parsing. So what if we make iptables -L print in any format it likes 
>> (not intended for machine parsing anyway), but have iptables-save 
>> always print addresses in full dotted quad?
> 
> Having iptables -L output anything else than quad dotted format is an 
> even worse idea. Why outputting any other notation than the official 
> standard?

Because iptables -nvL lines are already horribly long. Anything making 
them shorter is good for readability.

There is no point in trying to parse iptables -L output since all the 
modules already output random stuff in the right column. So we should 
rather try to make it more readable for humans.


>> The question is how to interpret a single number. Implicitly append or 
>> prepend a dot? Or interpret as full 32bit notation? Or ignore it?
> 
> Depends on if it is a CIDR number or not.

Requires backwards parsing? Not good.


> The following syntaxes I see as acceptable
> 
> 
> quad dotted IP, hex or dec

Hex in iptables -L output would cause more confusion and anger than my 
proposals, I believe.

Hex on input ok except noone uses it.


> N dotted IP (less than quad), hex or dec notation
> 
> CIDR notation
>   N octets (up to four) / masksize. Only decimal.
> 
>   10/8 == 10.0.0.0/8

In other mail you say 1/0.255.255.255 = 0.0.0.1/0.255.255.255, inconsistent?

Again question is: When address is only one number, is it interpreted as 
first or last octet, if any? (ie. append dot, prepend dot, or bark?)

Or do you really mean that:
cidr mask => interpret as first octet
dotted mask => interpret as last octet


> Mask notation
> 
>   quad or N dotted IP / netmask in quad or N dotted IP form. hex or dec.
> 
> To differentiate between CIDR and Mask notation when the mask is 
> specified using a single number use the <=32 magics.

 From all this I do not see how you interpret N dotted IP or mask, or 
how it provides the flexibility I am aiming for.

As I understand it you rule out numbers >255, I agree to that.

BTW iptables currently parses 10.20 as 10.0.0.20, but 10.20/24 becomes 
10.0.0.0/24, so apparently the common understanding is not so common.


> I do not find 10. as suitable shorthand for 10.0.0.0/8 even if this form 
> is currently not in use in any of the established notations. The problem 
> with 10. is that this could just as well be a partially typed IP address 
> where the administrator meant to enter more information but forgot. 
> These things happens more often than one would think.

If that is a concern (and I agree it is) then we should *start* by 
ruling out all the short forms and big decimal forms outlined in other 
mails! I could just as well have forgotten a few dots in 1001122, who knows?

I believe my proposal will make partial addresses (and masks) easier to 
understand since the dot(s) explicitly show where the missing octets 
are. Contrary to N dotted form without extra dots.

May not have been clear but the idea is of course that dotted masks can 
be partial as well, fx. "10./255.", "0.0.1.0/.255.0" etc.


Simon

Re: RFC: Partial IP4 syntax

[Cedric Blancher]

Le mer 29/09/2004 à 20:45, Simon Lodal a écrit :
> Because iptables -nvL lines are already horribly long. Anything making 
> them shorter is good for readability.

If having them more readable means have them confusing, I prefer keep
reading long lines.


-- 
http://www.netexit.com/~sid/
PGP KeyID: 157E98EE FingerPrint: FA62226DA9E72FA8AECAA240008B480E157E98EE
>> Hi! I'm your friendly neighbourhood signature virus.
>> Copy me to your signature file and help me spread!

Re: RFC: Partial IP4 syntax

[Simon Lodal]

Cedric Blancher skrev:
>>Because iptables -nvL lines are already horribly long. Anything making 
>>them shorter is good for readability.
> 
> If having them more readable means have them confusing, I prefer keep
> reading long lines.

The whole point is to make things easier to write and to read.

People also had to get used to cidr masks, habits play a role. As does 
interest in improvements.


Simon

Re: RFC: Partial IP4 syntax

[Henrik Nordstrom]

On Wed, 29 Sep 2004, Simon Lodal wrote:

>> Having iptables -L output anything else than quad dotted format is an even 
>> worse idea. Why outputting any other notation than the official standard?
>
> Because iptables -nvL lines are already horribly long. Anything making them 
> shorter is good for readability.

Not if the output introduces notations the user is not used to see, and 
even worse if different from what the user entered in the first place.

>> The following syntaxes I see as acceptable
>> 
>> 
>> quad dotted IP, hex or dec
>
> Hex in iptables -L output would cause more confusion and anger than my 
> proposals, I believe.
>
> Hex on input ok except noone uses it.
>
>
>> N dotted IP (less than quad), hex or dec notation
>> 
>> CIDR notation
>>   N octets (up to four) / masksize. Only decimal.
>> 
>>   10/8 == 10.0.0.0/8
>
> In other mail you say 1/0.255.255.255 = 0.0.0.1/0.255.255.255, inconsistent?

Not at all. Your examples is not in CIDR notation as you specify a 
netmask, not a mask size.

> Again question is: When address is only one number, is it interpreted as 
> first or last octet, if any? (ie. append dot, prepend dot, or bark?)

Depends on the whole expression

   10   is a HOST address  (0.0.0.10/255.255.255.255)

   10/0.255.255.255 is a HOST address pattern (0.0.0.10/0.255.255.255)

   10/8 is a network address in CIDR notation. (10.0.0.0/255.0.0.0)

> Or do you really mean that:
> cidr mask => interpret as first octet
> dotted mask => interpret as last octet

Yes.

>> Mask notation
>> 
>>   quad or N dotted IP / netmask in quad or N dotted IP form. hex or dec.
>> 
>> To differentiate between CIDR and Mask notation when the mask is specified 
>> using a single number use the <=32 magics.
>
> From all this I do not see how you interpret N dotted IP or mask, or how it 
> provides the flexibility I am aiming for.

X/0 to X/32 is CIDR with mask size.

X/anythingelse is a ip + netmask.

> BTW iptables currently parses 10.20 as 10.0.0.20, but 10.20/24 becomes 
> 10.0.0.0/24, so apparently the common understanding is not so common.

Depends on the context of the tool. iptables parses IP addresses, not 
networks.

> If that is a concern (and I agree it is) then we should *start* by ruling out 
> all the short forms and big decimal forms outlined in other mails! I could 
> just as well have forgotten a few dots in 1001122, who knows?

Entirely fine by me, but there may be people relying on the current 
syntax and used to using it.

Regards
Henrik

Re: RFC: Partial IP4 syntax

[Henrik Nordstrom]

On Wed, 29 Sep 2004, Simon Lodal wrote:

> I know IP address syntax should not change every day. But this will not break 
> or exclude old syntax. I do not see it clashing with other/future syntax.

There already is an old BSD syntax similar this which many people are used 
to, but with a slightly different meaning

10.1 is the same as 10.0.0.1

10.59470 is the same as 10.0.232.78

2888886350 is the same as 172.48.232.78

not very useful for netfilter thou as here one mostly wants to specify 
networks, not hosts which means that the address must always end in .0

> Only minor problem I know of is when parsing netmask which can be in cidr or 
> dotted quad form: /32 could be interpreted as a partial address, expanded to 
> 32.0.0.0. But it is simple to special case this; make it a cidr mask when 
> there is only one octet and it is <=32, otherwise interpret as dotted quad 
> (partial or not).

This is how most people do this. These numbers does not make sense as 
binary netmasks, and if you need them you can always use the quad dotted 
form.

Regards
Henrik

Re: RFC: Partial IP4 syntax

[Simon Lodal]

Henrik Nordstrom skrev:
> There already is an old BSD syntax similar this which many people are 
> used to, but with a slightly different meaning
> 
> 10.1 is the same as 10.0.0.1
Interesting, thanks.

When people talk about the 10.44 network, I assume they append .0.0, not 
insert them in the middle. My proposal does that with a double dot, eg. 
10..44 = 10.0.0.44.

The old syntax could actually be supported if we always require 
leading/trailing/double dot. In other words:
1) .b.c.d | .c.d | .d
2) a..d | a..c.d | a.b..d
3) a. | a.b. | a.b.c.

They all require special dotting, so none of them clash with an address 
that just lacks some octets, eg. "10.1". We can then choose to support 
that in it's own way.

Note that "10.1" and "10..1" would both expand to 10.0.0.1 but have 
different defaults netmasks: /32 vs. 255.0.0.255.

> 10.59470 is the same as 10.0.232.78
> 
> 2888886350 is the same as 172.48.232.78
I understand if that is being deprecated.

They could at least have used hex!

> not very useful for netfilter thou as here one mostly wants to specify 
> networks, not hosts which means that the address must always end in .0
I also want compact syntax for groups of similar hosts in different 
nets. I have a bunch of class C nets internally, in each .1 is the 
router, I want to match traffic directly to/from them: 
"10.0.0.1/255.0.0.255". It is possible but looks ugly, so I want 
simplification, in this case "10..1". Parsed result is the same.

What I also want is a way to only specify the least significant bits. If 
I do not care about the first octet (10), the above would just become ".1".


Simon

Re: RFC: Partial IP4 syntax

[Henrik Nordstrom]

On Wed, 29 Sep 2004, Simon Lodal wrote:

> When people talk about the 10.44 network, I assume they append .0.0, not 
> insert them in the middle. My proposal does that with a double dot, eg. 
> 10..44 = 10.0.0.44.

There is a well established meaning in the computing industry of 10.44 to 
mean 10.0.0.44, not 10.44.0.0.

There is also a well established meaning in the routing business of 
10.44/24 to mean 10.44.0.0/24, but then only in CIDR notation with a mask 
size (not netmask), and certainly not without the mask size.

There is no one using double dots that I know of.

Guessing the netmask size is generally bad and should be avoided imho. 
Lots of problems have been caused by computers guessing netmasks based on 
zeroes..

>> 10.59470 is the same as 10.0.232.78
>> 
>> 2888886350 is the same as 172.48.232.78
> I understand if that is being deprecated.
>
> They could at least have used hex!

Many allows hex notation using 0xXX syntax per element in the IP address..

0x54.0x63.0xFE.0xEE
0x5463FEEE
0x54.0x63FEEE
84.6553326
84.99.254.238
84.99.13094

is all the same IP address. You can try it with ping if in doubt.

> What I also want is a way to only specify the least significant bits. If I do 
> not care about the first octet (10), the above would just become ".1".

Which with the already established notation is simply 1/0.255.255.255 
which matches *.0.0.1

Regards
Henrik

Re: RFC: Partial IP4 syntax

[Simon Lodal]

Henrik Nordstrom skrev:
> On Wed, 29 Sep 2004, Simon Lodal wrote:
> 
>> When people talk about the 10.44 network, I assume they append .0.0, 
>> not insert them in the middle. My proposal does that with a double 
>> dot, eg. 10..44 = 10.0.0.44.
> 
> There is a well established meaning in the computing industry of 10.44 
> to mean 10.0.0.44, not 10.44.0.0.

Really?


> There is also a well established meaning in the routing business of 
> 10.44/24 to mean 10.44.0.0/24, but then only in CIDR notation with a 
> mask size (not netmask), and certainly not without the mask size.
> 
> There is no one using double dots that I know of.

Just why I think we can use it (same goes for leading and trailing dots).

The double dot is similar to double colon in ipv6, so it's meaning 
should not be surprising.


> Guessing the netmask size is generally bad and should be avoided imho. 
> Lots of problems have been caused by computers guessing netmasks based 
> on zeroes..

Agree, but I calculate netmask based on *missing* octets. Different.


>>> 10.59470 is the same as 10.0.232.78
>>>
>>> 2888886350 is the same as 172.48.232.78
>>
>> I understand if that is being deprecated.
>>
>> They could at least have used hex!
(should never have said that :) (what about binary and octal?)
> 
> 
> Many allows hex notation using 0xXX syntax per element in the IP address..
> 
> 0x54.0x63.0xFE.0xEE
> 0x5463FEEE
> 0x54.0x63FEEE
> 84.6553326
> 84.99.254.238
> 84.99.13094
> 
> is all the same IP address. You can try it with ping if in doubt.

Ok but hey is ping just a testbed for strange notations? What other 
programs (and users) actually accept all these notations? And I am 
talking real world use on Linux (as long as iptables is not ported to bsd).

iptables has never accepted all these syntax variants, or even 
documented that a few of them might work. So what do we lose by still 
not doing it?

Anyway nothing I propose would break any of these.


>> What I also want is a way to only specify the least significant bits. 
>> If I do not care about the first octet (10), the above would just 
>> become ".1".
> 
> Which with the already established notation is simply 1/0.255.255.255 
> which matches *.0.0.1
strlen(".1") = 2
strlen("1/0.0.0.255") = 11

That's why.


Simon

Re: RFC: Partial IP4 syntax

[Henrik Nordstrom]

On Wed, 29 Sep 2004, Simon Lodal wrote:

> Henrik Nordstrom skrev:
>> On Wed, 29 Sep 2004, Simon Lodal wrote:
>> 
>>> When people talk about the 10.44 network, I assume they append .0.0, not 
>>> insert them in the middle. My proposal does that with a double dot, eg. 
>>> 10..44 = 10.0.0.44.
>> 
>> There is a well established meaning in the computing industry of 10.44 to 
>> mean 10.0.0.44, not 10.44.0.0.
>
> Really?

Yes, really. It has been like this since BSD networking first saw the 
light.

> Ok but hey is ping just a testbed for strange notations? What other programs 
> (and users) actually accept all these notations? And I am talking real world 
> use on Linux (as long as iptables is not ported to bsd).

Quite many programs accept this notation.

> strlen(".1") = 2
> strlen("1/0.0.0.255") = 11
>
> That's why.

Not a reason imho.

If you had said that .1 is a whole lot easier to understand as an 
administrator than 0.0.0.1/0.0.0.255 then there may be valid arguments.

With this kind of IP expressions extremely rare I actually prefer to have 
a long syntax for it to make sure it is not done by mistake. Shorter is 
not neccesarily better unless it is something which is typed very very 
often.

Regards
Henrik