64bit MARK

64bit MARK

[François Delawarde]

Hello,

Would it be possible to have 64bit MARKs with 64bit architectures? 

Apparently they are defined as "unsigned long" in linux headers, but
iptables limits them to 32bit.

Thanks,
François.

Re: 64bit MARK

[John A. Sullivan III]

On Thu, 2009-10-29 at 18:05 +0100, François Delawarde wrote:
> Hello,
> 
> Would it be possible to have 64bit MARKs with 64bit architectures? 
> 
> Apparently they are defined as "unsigned long" in linux headers, but
> iptables limits them to 32bit.
<snip>
Just as an aside, we've had problems using the top 16 bits of the 32 bit
marks.  The OpenSWAN implementation on SnapGear devices changes the top
16 bits all on its own with two separate bugs - one with compression
enabled and another regardless of compression- John
-- 
John A. Sullivan III
Open Source Development Corporation
+1 207-985-7880
jsullivan@opensourcedevel.com

http://www.spiritualoutreach.com
Making Christianity intelligible to secular society

Re: 64bit MARK

[Patrick McHardy]

John A. Sullivan III wrote:
> Just as an aside, we've had problems using the top 16 bits of the 32 bit
> marks.  The OpenSWAN implementation on SnapGear devices changes the top
> 16 bits all on its own with two separate bugs - one with compression
> enabled and another regardless of compression- John

You should talk to the *swan people, their kernel implementation
is seriously fscked.

Re: 64bit MARK

[John A. Sullivan III]

On Thu, 2009-10-29 at 23:33 +0100, Patrick McHardy wrote:
> John A. Sullivan III wrote:
> > Just as an aside, we've had problems using the top 16 bits of the 32 bit
> > marks.  The OpenSWAN implementation on SnapGear devices changes the top
> > 16 bits all on its own with two separate bugs - one with compression
> > enabled and another regardless of compression- John
> 
> You should talk to the *swan people, their kernel implementation
> is seriously fscked.
I believe the excellent folks at SnapGear are doing just that.  They
were the ones with the skills to find it.  I was only able to point out
the symptoms - John
-- 
John A. Sullivan III
Open Source Development Corporation
+1 207-985-7880
jsullivan@opensourcedevel.com

http://www.spiritualoutreach.com
Making Christianity intelligible to secular society

Re: 64bit MARK

[Patrick McHardy]

Fran����������������������������������������������� wrote:
> Hello,
> 
> Would it be possible to have 64bit MARKs with 64bit architectures? 
> 
> Apparently they are defined as "unsigned long" in linux headers, but
> iptables limits them to 32bit.

Thats from far far back. The unsigned long only exists in compatibility
structures, all new revisions use 32 bit. This is deliberate to make
sure they're same-sized on all architectures and rulesets don't behave
differently.

Re: 64bit MARK

[François Delawarde]

On Thu, 2009-10-29 at 23:30 +0100, Patrick McHardy wrote:
> Fran����������������������������������������������� wrote:
> > Hello,
> > 
> > Would it be possible to have 64bit MARKs with 64bit architectures? 
> > 
> > Apparently they are defined as "unsigned long" in linux headers, but
> > iptables limits them to 32bit.
> 
> Thats from far far back. The unsigned long only exists in compatibility
> structures, all new revisions use 32 bit. This is deliberate to make
> sure they're same-sized on all architectures and rulesets don't behave
> differently.

It might be an uncommon case, but many things I do need MARKs and 32bits
is just not enough!

For example, for input traffic I do marking for:
- ip rules (sometimes A LOT of rules)
- tc filters (sometimes A LOT of filters)
- gateway selection (ip rules for multipath)
- marking input interface to use in IMQ

Any ideas then for someone needing more than 32bits MARK? I could use
TOS bits (usable with ip rules and tc filters), but it's just so ugly,
and would screw in any network using it...

Could the other 32 bits be used as a MARK "category" feature available
only for 64bit arch?

Thanks,
François.

Re: 64bit MARK

[Patrick McHardy]

Fran����������������������������������������������� wrote:
> On Thu, 2009-10-29 at 23:30 +0100, Patrick McHardy wrote:
>>> Would it be possible to have 64bit MARKs with 64bit architectures? 
>>>
>>> Apparently they are defined as "unsigned long" in linux headers, but
>>> iptables limits them to 32bit.
>> Thats from far far back. The unsigned long only exists in compatibility
>> structures, all new revisions use 32 bit. This is deliberate to make
>> sure they're same-sized on all architectures and rulesets don't behave
>> differently.
> 
> It might be an uncommon case, but many things I do need MARKs and 32bits
> is just not enough!
> 
> For example, for input traffic I do marking for:
> - ip rules (sometimes A LOT of rules)
> - tc filters (sometimes A LOT of filters)
> - gateway selection (ip rules for multipath)
> - marking input interface to use in IMQ
> 
> Any ideas then for someone needing more than 32bits MARK? I could use
> TOS bits (usable with ip rules and tc filters), but it's just so ugly,
> and would screw in any network using it...
> 
> Could the other 32 bits be used as a MARK "category" feature available
> only for 64bit arch?

No, as I said, rulesets are supposed to behave similar on all
architectures. Since neither routing rules nor TC classifiers
support more than 32 bit, it would be useless anyways. But you
can usually deal with this by organizing your marking rules
properly.

First, all of the above support using masks for mark values,
which might be helpful. For routing and gateway selection you
only need the mark values at PREROUTING and OUTPUT, so you
can reuse them afterwards. TC marking is best done in
POSTROUTING since that doesn't interfere with routing. If
for some reason you want to do TC classification earlier than
that, you can use the CLASSIFY target, which doesn't use
mark values but priority values.

Re: 64bit MARK

[François Delawarde]

On Fri, 2009-10-30 at 10:20 +0100, Patrick McHardy wrote:
> Fran����������������������������������������������� wrote:
> > On Thu, 2009-10-29 at 23:30 +0100, Patrick McHardy wrote:
> >>> Would it be possible to have 64bit MARKs with 64bit architectures? 
> >>>
> >>> Apparently they are defined as "unsigned long" in linux headers, but
> >>> iptables limits them to 32bit.
> >> Thats from far far back. The unsigned long only exists in compatibility
> >> structures, all new revisions use 32 bit. This is deliberate to make
> >> sure they're same-sized on all architectures and rulesets don't behave
> >> differently.
> > 
> > It might be an uncommon case, but many things I do need MARKs and 32bits
> > is just not enough!
> > 
> > For example, for input traffic I do marking for:
> > - ip rules (sometimes A LOT of rules)
> > - tc filters (sometimes A LOT of filters)
> > - gateway selection (ip rules for multipath)
> > - marking input interface to use in IMQ
> > 
> > Any ideas then for someone needing more than 32bits MARK? I could use
> > TOS bits (usable with ip rules and tc filters), but it's just so ugly,
> > and would screw in any network using it...
> > 
> > Could the other 32 bits be used as a MARK "category" feature available
> > only for 64bit arch?
> 
> No, as I said, rulesets are supposed to behave similar on all
> architectures. Since neither routing rules nor TC classifiers
> support more than 32 bit, it would be useless anyways. But you
> can usually deal with this by organizing your marking rules
> properly.
> 
> First, all of the above support using masks for mark values,
> which might be helpful. For routing and gateway selection you
> only need the mark values at PREROUTING and OUTPUT, so you
> can reuse them afterwards. TC marking is best done in
> POSTROUTING since that doesn't interfere with routing. If
> for some reason you want to do TC classification earlier than
> that, you can use the CLASSIFY target, which doesn't use
> mark values but priority values.

Agreed, POSTROUTING is not a problem, it's more incoming traffic. I
already play a lot with masks, the problem being that I use IMQ device
that needs tc marking in PREROUTING and unfortunately does not work with
CLASSIFY.

I have systems with LOTS of ifaces, gateways and filters/rule which work
very nicely in multipath.

Currently, I'm using the 32 bits as follows:
Input Iface = 5 bits (up to 32 ifaces)
Gateway = 5 bits (up to 32 gateways)
IMQ tc mark = 11 bits (up to 2048 filters)
ip rule filters = 11 bits (up to 2048 rules)

It is quite OK for now, but I would have loved to have margin for more
rules or new features in the future (I'd like to play with NFQUEUE
things for Authentication, IDS, ... and more bits would have been cool).

Thanks for the answer, I think I can make it trying to make more use of
u32 filters and other things. If you have any other suggestions or
ideas, I would be happy to hear them!

Saludos,
François.

Re: 64bit MARK

[Patrick McHardy]

Fran��������������������������������������������� wrote:
> On Fri, 2009-10-30 at 10:20 +0100, Patrick McHardy wrote:
>> Fran����������������������������������������������� wrote:
>>> On Thu, 2009-10-29 at 23:30 +0100, Patrick McHardy wrote:
>>>>> Would it be possible to have 64bit MARKs with 64bit architectures? 
>>>>>
>>>>> Apparently they are defined as "unsigned long" in linux headers, but
>>>>> iptables limits them to 32bit.
>>>> Thats from far far back. The unsigned long only exists in compatibility
>>>> structures, all new revisions use 32 bit. This is deliberate to make
>>>> sure they're same-sized on all architectures and rulesets don't behave
>>>> differently.
>>> It might be an uncommon case, but many things I do need MARKs and 32bits
>>> is just not enough!
>>>
>>> For example, for input traffic I do marking for:
>>> - ip rules (sometimes A LOT of rules)
>>> - tc filters (sometimes A LOT of filters)
>>> - gateway selection (ip rules for multipath)
>>> - marking input interface to use in IMQ
>>>
>>> Any ideas then for someone needing more than 32bits MARK? I could use
>>> TOS bits (usable with ip rules and tc filters), but it's just so ugly,
>>> and would screw in any network using it...
>>>
>>> Could the other 32 bits be used as a MARK "category" feature available
>>> only for 64bit arch?
>> No, as I said, rulesets are supposed to behave similar on all
>> architectures. Since neither routing rules nor TC classifiers
>> support more than 32 bit, it would be useless anyways. But you
>> can usually deal with this by organizing your marking rules
>> properly.
>>
>> First, all of the above support using masks for mark values,
>> which might be helpful. For routing and gateway selection you
>> only need the mark values at PREROUTING and OUTPUT, so you
>> can reuse them afterwards. TC marking is best done in
>> POSTROUTING since that doesn't interfere with routing. If
>> for some reason you want to do TC classification earlier than
>> that, you can use the CLASSIFY target, which doesn't use
>> mark values but priority values.
> 
> Agreed, POSTROUTING is not a problem, it's more incoming traffic. I
> already play a lot with masks, the problem being that I use IMQ device
> that needs tc marking in PREROUTING and unfortunately does not work with
> CLASSIFY.
> 
> I have systems with LOTS of ifaces, gateways and filters/rule which work
> very nicely in multipath.
> 
> Currently, I'm using the 32 bits as follows:
> Input Iface = 5 bits (up to 32 ifaces)
> Gateway = 5 bits (up to 32 gateways)
> IMQ tc mark = 11 bits (up to 2048 filters)
> ip rule filters = 11 bits (up to 2048 rules)
> 
> It is quite OK for now, but I would have loved to have margin for more
> rules or new features in the future (I'd like to play with NFQUEUE
> things for Authentication, IDS, ... and more bits would have been cool).
> 
> Thanks for the answer, I think I can make it trying to make more use of
> u32 filters and other things. If you have any other suggestions or
> ideas, I would be happy to hear them!

Well, one more. You don't need any mark values for the input interface,
both routing rules and TC classifiers (flow, route, meta) support iif
classification directly.

Re: 64bit MARK

[François Delawarde]

On Fri, 2009-10-30 at 10:55 +0100, Patrick McHardy wrote:
> Well, one more. You don't need any mark values for the input interface,
> both routing rules and TC classifiers (flow, route, meta) support iif
> classification directly.

I unfortunately do not need it for routing rules which works nice, but
for TC classifiers, and apparently it does not work with those, maybe I
don't know how to use them.

# tc filter add dev imq0 parent 1: iif eth0 flowid 1:2
Unknown filter "iif", hence option "eth0" is unparsable

François.

Re: 64bit MARK

[Patrick McHardy]

Fran��������������������������������������������� wrote:
> On Fri, 2009-10-30 at 10:55 +0100, Patrick McHardy wrote:
>> Well, one more. You don't need any mark values for the input interface,
>> both routing rules and TC classifiers (flow, route, meta) support iif
>> classification directly.
> 
> I unfortunately do not need it for routing rules which works nice, but
> for TC classifiers, and apparently it does not work with those, maybe I
> don't know how to use them.
> 
> # tc filter add dev imq0 parent 1: iif eth0 flowid 1:2
> Unknown filter "iif", hence option "eth0" is unparsable

tc filter add ... route fromif ... classid ...

should work.

Re: 64bit MARK

[François Delawarde]

On Fri, 2009-10-30 at 11:17 +0100, Patrick McHardy wrote:
> Fran��������������������������������������������� wrote:
> > On Fri, 2009-10-30 at 10:55 +0100, Patrick McHardy wrote:
> >> Well, one more. You don't need any mark values for the input interface,
> >> both routing rules and TC classifiers (flow, route, meta) support iif
> >> classification directly.
> > 
> > I unfortunately do not need it for routing rules which works nice, but
> > for TC classifiers, and apparently it does not work with those, maybe I
> > don't know how to use them.
> > 
> > # tc filter add dev imq0 parent 1: iif eth0 flowid 1:2
> > Unknown filter "iif", hence option "eth0" is unparsable
> 
> tc filter add ... route fromif ... classid ...
> 
> should work.

You're my heroe, I just gained 5 out of the precious 32bits!!

Thanks a lot,
François.