libpcap and tc filters

libpcap and tc filters

[Adam Katz]

Hi Everyone

I'm sorry for littering the mailing list with this question, but no
other place could help me..

I'm attempting to use tc to shape traffic sent using libpcap, I'm
doing this for a research project. I have a classful scheduler with
several classes, to this scheduler I attach a few filters based on
destination tcp ports.

My problem is this: When sending packets using a normal userland
socket, the filters work and I see the appropriate traffic entering
the right class. BUT when sending packets with libpcap, all packets
end up in the scheduler's default band as if the filters simply refuse
to work.

Can anyone suggest a solution?

Re: libpcap and tc filters

[Eric Dumazet]

Le lundi 04 juillet 2011 à 10:38 +0300, Adam Katz a écrit :
> Hi Everyone
> 
> I'm sorry for littering the mailing list with this question, but no
> other place could help me..
> 
> I'm attempting to use tc to shape traffic sent using libpcap, I'm
> doing this for a research project. I have a classful scheduler with
> several classes, to this scheduler I attach a few filters based on
> destination tcp ports.
> 
> My problem is this: When sending packets using a normal userland
> socket, the filters work and I see the appropriate traffic entering
> the right class. BUT when sending packets with libpcap, all packets
> end up in the scheduler's default band as if the filters simply refuse
> to work.
> 
> Can anyone suggest a solution?
> --

You already asked this yesterday.

No need to repost.

4th July is not the day many people will answer you...

What gives :

tc -s -d filter show dev eth0

Re: libpcap and tc filters

[jamal]

Capture tcpdump for both scenario that works and one
that doesnt.
Make sure the filters match your failing scenario.

cheers,
jamal

On Mon, 2011-07-04 at 10:38 +0300, Adam Katz wrote:
> Hi Everyone
> 
> I'm sorry for littering the mailing list with this question, but no
> other place could help me..
> 
> I'm attempting to use tc to shape traffic sent using libpcap, I'm
> doing this for a research project. I have a classful scheduler with
> several classes, to this scheduler I attach a few filters based on
> destination tcp ports.
> 
> My problem is this: When sending packets using a normal userland
> socket, the filters work and I see the appropriate traffic entering
> the right class. BUT when sending packets with libpcap, all packets
> end up in the scheduler's default band as if the filters simply refuse
> to work.
> 
> Can anyone suggest a solution?
> --
> To unsubscribe from this list: send the line "unsubscribe netdev" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: libpcap and tc filters

[Adam Katz]

Ok, I just tried this:

I've opened www.example.com using a browser while capturing with
wireshark. TC placed all port 80 packets into the 1:1 as required. I
then physically plugged the nic to a different socket, one that isn't
connected to the internet (i did this so I wont get any server
responses to the packets i'm sending) and then I replayed the capture
of me opening www.example.com

The second time, none of the packets ended up in 1:1 - they all went
to the default class despite being the EXACT same traffic.

On Mon, Jul 4, 2011 at 2:11 PM, jamal <hadi@cyberus.ca> wrote:
>
> Capture tcpdump for both scenario that works and one
> that doesnt.
> Make sure the filters match your failing scenario.
>
> cheers,
> jamal
>
> On Mon, 2011-07-04 at 10:38 +0300, Adam Katz wrote:
>> Hi Everyone
>>
>> I'm sorry for littering the mailing list with this question, but no
>> other place could help me..
>>
>> I'm attempting to use tc to shape traffic sent using libpcap, I'm
>> doing this for a research project. I have a classful scheduler with
>> several classes, to this scheduler I attach a few filters based on
>> destination tcp ports.
>>
>> My problem is this: When sending packets using a normal userland
>> socket, the filters work and I see the appropriate traffic entering
>> the right class. BUT when sending packets with libpcap, all packets
>> end up in the scheduler's default band as if the filters simply refuse
>> to work.
>>
>> Can anyone suggest a solution?
>> --
>> To unsubscribe from this list: send the line "unsubscribe netdev" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
>
>

Re: libpcap and tc filters

[Adam Katz]

here's a more concrete example:

An example configuration:

	sudo tc qdisc add dev eth1 root handle 1: prio priomap 2 2 2 2 2 2 2
2 2 2 2 2 2 2 2 2
	sudo tc qdisc add dev eth1 parent 1:1 handle 10: pfifo 	
	sudo tc qdisc add dev eth1 parent 1:2 handle 20: pfifo
	sudo tc qdisc add dev eth1 parent 1:3 handle 30: pfifo

	sudo tc filter add dev eth1 protocol ip parent 1: prio 1 u32 match ip
dport 22 0xffff flowid 1:1
	sudo tc filter add dev eth1 protocol ip parent 1: prio 1 u32 match ip
sport 22 0xffff flowid 1:1
	sudo tc filter add dev eth1 protocol ip parent 1: prio 2 u32 match ip
dport 80 0xffff flowid 1:2
	sudo tc filter add dev eth1 protocol ip parent 1: prio 2 u32 match ip
sport 80 0xffff flowid 1:2

I then used scp to copy a small file between computers while capturing
with wireshark:

http://dl.dropbox.com/u/3237005/port22example.pcap

and later I replayed the same capture using tcpreplay.
When using scp, the packets once again ended up where they should be
(1:1 in this configuration). With tcpreplay they ended up in the
default 1:3

On Mon, Jul 4, 2011 at 3:06 PM, jamal <hadi@cyberus.ca> wrote:
> On Mon, 2011-07-04 at 15:00 +0300, Adam Katz wrote:
>> Ok, I just tried this:
>>
>> I've opened www.example.com using a browser while capturing with
>> wireshark. TC placed all port 80 packets into the 1:1 as required. I
>> then physically plugged the nic to a different socket, one that isn't
>> connected to the internet (i did this so I wont get any server
>> responses to the packets i'm sending) and then I replayed the capture
>> of me opening www.example.com
>>
>> The second time, none of the packets ended up in 1:1 - they all went
>> to the default class despite being the EXACT same traffic.
>>
>
> Please post a small sample of the tcpdumps and the tc rules you used.
>
> cheers,
> jamal
>
>



On Mon, Jul 4, 2011 at 3:01 PM, Adam Katz <adamkatz0@gmail.com> wrote:
> Ok, I just tried this:
>
> I've opened www.example.com using a browser while capturing with
> wireshark. TC placed all port 80 packets into the 1:1 as required. I
> then physically plugged the nic to a different socket, one that isn't
> connected to the internet (i did this so I wont get any server
> responses to the packets i'm sending) and then I replayed the capture
> of me opening www.example.com
>
> The second time, none of the packets ended up in 1:1 - they all went
> to the default class despite being the EXACT same traffic.
>
> On Mon, Jul 4, 2011 at 2:11 PM, jamal <hadi@cyberus.ca> wrote:
>>
>> Capture tcpdump for both scenario that works and one
>> that doesnt.
>> Make sure the filters match your failing scenario.
>>
>> cheers,
>> jamal
>>
>> On Mon, 2011-07-04 at 10:38 +0300, Adam Katz wrote:
>>> Hi Everyone
>>>
>>> I'm sorry for littering the mailing list with this question, but no
>>> other place could help me..
>>>
>>> I'm attempting to use tc to shape traffic sent using libpcap, I'm
>>> doing this for a research project. I have a classful scheduler with
>>> several classes, to this scheduler I attach a few filters based on
>>> destination tcp ports.
>>>
>>> My problem is this: When sending packets using a normal userland
>>> socket, the filters work and I see the appropriate traffic entering
>>> the right class. BUT when sending packets with libpcap, all packets
>>> end up in the scheduler's default band as if the filters simply refuse
>>> to work.
>>>
>>> Can anyone suggest a solution?
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe netdev" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
>>
>>
>

Re: libpcap and tc filters

[jamal]

On Mon, 2011-07-04 at 15:37 +0300, Adam Katz wrote:
> here's a more concrete example:
> 
> An example configuration:
> 
> 	sudo tc qdisc add dev eth1 root handle 1: prio priomap 2 2 2 2 2 2 2
> 2 2 2 2 2 2 2 2 2
> 	sudo tc qdisc add dev eth1 parent 1:1 handle 10: pfifo 	
> 	sudo tc qdisc add dev eth1 parent 1:2 handle 20: pfifo
> 	sudo tc qdisc add dev eth1 parent 1:3 handle 30: pfifo
> 	sudo tc filter add dev eth1 protocol ip parent 1: prio 1 u32 match ip
> dport 22 0xffff flowid 1:1
> 	sudo tc filter add dev eth1 protocol ip parent 1: prio 1 u32 match ip
> sport 22 0xffff flowid 1:1
> 	sudo tc filter add dev eth1 protocol ip parent 1: prio 2 u32 match ip
> dport 80 0xffff flowid 1:2
> 	sudo tc filter add dev eth1 protocol ip parent 1: prio 2 u32 match ip
> sport 80 0xffff flowid 1:2


looks fine.

> I then used scp to copy a small file between computers while capturing
> with wireshark:
> 
> http://dl.dropbox.com/u/3237005/port22example.pcap
> 
> and later I replayed the same capture using tcpreplay.
> When using scp, the packets once again ended up where they should be
> (1:1 in this configuration). With tcpreplay they ended up in the
> default 1:3

Where is the capture from tcpreplay? What i was asking is you validate
that the capture before and what is sent out by tcprelay look the same.
Can you please do that?
It is possible because your filters are not matched they end up on your
default queue based on tos value.

If you have your kernel compiled with CONFIG_CLS_U32_PERF you should
see when the filters get hit as well
(do something like sudo tc -s filter ls dev eth1)


cheers,
jamal

Re: libpcap and tc filters

[Adam Katz]

ok, I checked now and the packets sent by tcpreplay are identical to
the ones captured originally by wireshark.

I'm using the stock ubuntu 10.04 kernel that wasn't compiled with
CONFIG_CLS_U32_PERF so sudo tc -s filter ls dev eth1 shows nothing
useful (and i'm not sure that recompiling the entire kernel is worth
it to tell me what I already know - that these packets missed the
filters... but i'm willing to do it if you think that'll help).
Anyway, I suspect the problem to be something else - I suspect that
the packets sent using tcpreplay simply skip the filters in the kernel
and are being injected somewhere afterwards. But this theory is
problematic since I find it strange that the packets do end up in the
default queue after all - hence they ARE seen by tc and they don't
skip tc entirely.

On Mon, Jul 4, 2011 at 4:05 PM, jamal <hadi@cyberus.ca> wrote:
> On Mon, 2011-07-04 at 15:37 +0300, Adam Katz wrote:
>> here's a more concrete example:
>>
>> An example configuration:
>>
>>       sudo tc qdisc add dev eth1 root handle 1: prio priomap 2 2 2 2 2 2 2
>> 2 2 2 2 2 2 2 2 2
>>       sudo tc qdisc add dev eth1 parent 1:1 handle 10: pfifo
>>       sudo tc qdisc add dev eth1 parent 1:2 handle 20: pfifo
>>       sudo tc qdisc add dev eth1 parent 1:3 handle 30: pfifo
>>       sudo tc filter add dev eth1 protocol ip parent 1: prio 1 u32 match ip
>> dport 22 0xffff flowid 1:1
>>       sudo tc filter add dev eth1 protocol ip parent 1: prio 1 u32 match ip
>> sport 22 0xffff flowid 1:1
>>       sudo tc filter add dev eth1 protocol ip parent 1: prio 2 u32 match ip
>> dport 80 0xffff flowid 1:2
>>       sudo tc filter add dev eth1 protocol ip parent 1: prio 2 u32 match ip
>> sport 80 0xffff flowid 1:2
>
>
> looks fine.
>
>> I then used scp to copy a small file between computers while capturing
>> with wireshark:
>>
>> http://dl.dropbox.com/u/3237005/port22example.pcap
>>
>> and later I replayed the same capture using tcpreplay.
>> When using scp, the packets once again ended up where they should be
>> (1:1 in this configuration). With tcpreplay they ended up in the
>> default 1:3
>
> Where is the capture from tcpreplay? What i was asking is you validate
> that the capture before and what is sent out by tcprelay look the same.
> Can you please do that?
> It is possible because your filters are not matched they end up on your
> default queue based on tos value.
>
> If you have your kernel compiled with CONFIG_CLS_U32_PERF you should
> see when the filters get hit as well
> (do something like sudo tc -s filter ls dev eth1)
>
>
> cheers,
> jamal
>
>
>
>

Re: libpcap and tc filters

[jamal]

On Mon, 2011-07-04 at 16:24 +0300, Adam Katz wrote:
> ok, I checked now and the packets sent by tcpreplay are identical to
> the ones captured originally by wireshark.

Ok - thanks for removing that variable.

> I'm using the stock ubuntu 10.04 kernel that wasn't compiled with
> CONFIG_CLS_U32_PERF so sudo tc -s filter ls dev eth1 shows nothing
> useful (and i'm not sure that recompiling the entire kernel is worth
> it to tell me what I already know - that these packets missed the
> filters... but i'm willing to do it if you think that'll help).

Not necessary as long as you can tell where the packets end up.

> Anyway, I suspect the problem to be something else - I suspect that
> the packets sent using tcpreplay simply skip the filters in the kernel
> and are being injected somewhere afterwards. But this theory is
> problematic since I find it strange that the packets do end up in the
> default queue after all - hence they ARE seen by tc and they don't
> skip tc entirely.

I am not sure off top of my head why that would happen. I will try later
to install tcpreplay and reproduce your test.

cheers,
jamal

Re: libpcap and tc filters

[Adam Katz]

thanks a lot
I can tell you I'm not the first one to have this problem, but it
doesn't seem to be common... but that's probably because people
usually don't try to shape traffic sent using libpcap.

I found this post from 2003 on lartc with the exact same problem but
with no replies:
http://mailman.ds9a.nl/pipermail/lartc/2003q4/011004.html


On Mon, Jul 4, 2011 at 5:06 PM, jamal <hadi@cyberus.ca> wrote:
> On Mon, 2011-07-04 at 16:24 +0300, Adam Katz wrote:
>> ok, I checked now and the packets sent by tcpreplay are identical to
>> the ones captured originally by wireshark.
>
> Ok - thanks for removing that variable.
>
>> I'm using the stock ubuntu 10.04 kernel that wasn't compiled with
>> CONFIG_CLS_U32_PERF so sudo tc -s filter ls dev eth1 shows nothing
>> useful (and i'm not sure that recompiling the entire kernel is worth
>> it to tell me what I already know - that these packets missed the
>> filters... but i'm willing to do it if you think that'll help).
>
> Not necessary as long as you can tell where the packets end up.
>
>> Anyway, I suspect the problem to be something else - I suspect that
>> the packets sent using tcpreplay simply skip the filters in the kernel
>> and are being injected somewhere afterwards. But this theory is
>> problematic since I find it strange that the packets do end up in the
>> default queue after all - hence they ARE seen by tc and they don't
>> skip tc entirely.
>
> I am not sure off top of my head why that would happen. I will try later
> to install tcpreplay and reproduce your test.
>
> cheers,
> jamal
>
>

Re: libpcap and tc filters

[jamal]

On Mon, 2011-07-04 at 17:16 +0300, Adam Katz wrote:
> thanks a lot
> I can tell you I'm not the first one to have this problem, but it
> doesn't seem to be common... but that's probably because people
> usually don't try to shape traffic sent using libpcap.
> 
> I found this post from 2003 on lartc with the exact same problem but
> with no replies:
> http://mailman.ds9a.nl/pipermail/lartc/2003q4/011004.html

I downloaded tcpreplay and reproduced the issue with your rules.
Will look into it..

cheers,
jamal

Re: libpcap and tc filters

[jamal]

On Tue, 2011-07-05 at 06:56 -0400, jamal wrote:

> I downloaded tcpreplay and reproduced the issue with your rules.
> Will look into it..

Ok - found out whats going on. 
tcprelay sendpacket_open_pf() does bind to ETH_P_ALL.
You are sending IP packets (the name tcpreplay is misleading,
this thing replays anything). 
Your filters are for ip packets as in:
---
sudo tc filter add dev eth0 protocol ip parent 1: prio 1 u32 match ip
dport 22 0xffff flowid 1:1
---

You have two options:

1) If you change that to capture ETH_P_ALL it works.
i.e

---
sudo tc filter add dev eth0 protocol all parent 1: prio 1 u32 match ip
dport 22 0xffff flowid 1:1
---

Of course this is nasty if you are in a busy network, because _all_
packets not just ip will look at your filters. If it is just an
experimental setup, it may be a non-issue

2) Change tcpreplay to take an additional option so it binds to 
ETH_P_IP (and default stays as is today). The authors of the app
may not like that option - but it is sensible if you know you are
replaying ip packets.

cheers,
jamal

Re: libpcap and tc filters

[Adam Katz]

well, first of all, thanks A LOT for your effort.

second, I just took at the libpcap source code and it seems it's using
the same ETH_P_ALL option when binding to an interface. So based on
what you're saying, the same solution of patching libpcap and
replacing ETH_P_ALL with  ETH_P_IP should also make these rules work
with traffic sent using pure libpcap or any libpcap - based
application.

On Tue, Jul 5, 2011 at 3:47 PM, jamal <hadi@cyberus.ca> wrote:
> On Tue, 2011-07-05 at 06:56 -0400, jamal wrote:
>
>> I downloaded tcpreplay and reproduced the issue with your rules.
>> Will look into it..
>
> Ok - found out whats going on.
> tcprelay sendpacket_open_pf() does bind to ETH_P_ALL.
> You are sending IP packets (the name tcpreplay is misleading,
> this thing replays anything).
> Your filters are for ip packets as in:
> ---
> sudo tc filter add dev eth0 protocol ip parent 1: prio 1 u32 match ip
> dport 22 0xffff flowid 1:1
> ---
>
> You have two options:
>
> 1) If you change that to capture ETH_P_ALL it works.
> i.e
>
> ---
> sudo tc filter add dev eth0 protocol all parent 1: prio 1 u32 match ip
> dport 22 0xffff flowid 1:1
> ---
>
> Of course this is nasty if you are in a busy network, because _all_
> packets not just ip will look at your filters. If it is just an
> experimental setup, it may be a non-issue
>
> 2) Change tcpreplay to take an additional option so it binds to
> ETH_P_IP (and default stays as is today). The authors of the app
> may not like that option - but it is sensible if you know you are
> replaying ip packets.
>
> cheers,
> jamal
>
>
>
>

Re: libpcap and tc filters

[jamal]

On Tue, 2011-07-05 at 16:07 +0300, Adam Katz wrote:

> second, I just took at the libpcap source code and it seems it's using
> the same ETH_P_ALL option when binding to an interface. So based on
> what you're saying, the same solution of patching libpcap and
> replacing ETH_P_ALL with  ETH_P_IP should also make these rules work
> with traffic sent using pure libpcap or any libpcap - based
> application.

ETH_P_ALL makes sense if you are unsure it is going to be IP. So i would
change/optimize apps only for IP if they are intended to deal with IP
only (same for ARP etc).
In your case, it seems it is tcp only - which runs on top of IP. So
it makes sense to do it for that specific use case etc.

cheers,
jamal

Re: libpcap and tc filters

[Adam Katz]

Yes. I understand the difference between ETH_P_ALL and ETH_P_IP...

Jamal, I've now tested both solutions - changing the rule to "protocol
all" and patching tcpreplay to use ETH_P_IP and both produced the
exact same problem as before...


On Tue, Jul 5, 2011 at 4:56 PM, jamal <hadi@cyberus.ca> wrote:
> On Tue, 2011-07-05 at 16:07 +0300, Adam Katz wrote:
>
>> second, I just took at the libpcap source code and it seems it's using
>> the same ETH_P_ALL option when binding to an interface. So based on
>> what you're saying, the same solution of patching libpcap and
>> replacing ETH_P_ALL with  ETH_P_IP should also make these rules work
>> with traffic sent using pure libpcap or any libpcap - based
>> application.
>
> ETH_P_ALL makes sense if you are unsure it is going to be IP. So i would
> change/optimize apps only for IP if they are intended to deal with IP
> only (same for ARP etc).
> In your case, it seems it is tcp only - which runs on top of IP. So
> it makes sense to do it for that specific use case etc.
>
> cheers,
> jamal
>
>
>

Re: libpcap and tc filters

[jamal]

On Tue, 2011-07-05 at 17:21 +0300, Adam Katz wrote:
> Yes. I understand the difference between ETH_P_ALL and ETH_P_IP...
> 
> Jamal, I've now tested both solutions - changing the rule to "protocol
> all" and patching tcpreplay to use ETH_P_IP and both produced the
> exact same problem as before...

Sorry - dont have much time to chase further, but it works for me.

---
hadi@mojatatu10:~$ sudo tc qdisc del dev eth0 root handle 1:
RTNETLINK answers: Invalid argument
hadi@mojatatu10:~$ sudo tc qdisc add dev eth0 root handle 1: prio
priomap 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
hadi@mojatatu10:~$ sudo tc qdisc add dev eth0 parent 1:1 handle 10:
pfifo  
hadi@mojatatu10:~$ sudo tc qdisc add dev eth0 parent 1:2 handle 20:
pfifo
hadi@mojatatu10:~$ sudo tc qdisc add dev eth0 parent 1:3 handle 30:
pfifo
hadi@mojatatu10:~$ sudo tc filter add dev eth0 protocol all parent 1:
prio 1 u32 match ip dport 22 0xffff flowid 1:1 action ok
hadi@mojatatu10:~$ sudo tc -s filter ls dev eth0
filter parent 1: protocol all pref 1 u32 
filter parent 1: protocol all pref 1 u32 fh 800: ht divisor 1 
filter parent 1: protocol all pref 1 u32 fh 800::800 order 2048 key ht
800 bkt 0 flowid 1:1 
  match 00000016/0000ffff at 20
	action order 1: gact action pass
	 random type none pass val 0
	 index 1 ref 1 bind 1 installed 15 sec used 15 sec
 	Action statistics:
	Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
	backlog 0b 0p requeues 0 

Note - the "OK" action is just a place holder to count packets.
Now replay Adam's pcap file:

hadi@mojatatu10:~/Downloads$ sudo tcpreplay
--intf1=eth0 ./port22example.pcap

sending out eth0 
processing file: ./port22example.pcap
Actual: 50 packets (11594 bytes) sent in 3.66 seconds
Rated: 3167.8 bps, 0.02 Mbps, 13.66 pps
Statistics for network device: eth0
	Attempted packets:         50
	Successful packets:        50
	Failed packets:            0
	Retried packets (ENOBUFS): 0
	Retried packets (EAGAIN):  0

I dont have any ssh running on this maching. So
lets check to see if anything was captured by the filter.

-----
hadi@mojatatu10:~$ sudo tc -s filter ls dev eth0
filter parent 1: protocol all pref 1 u32 
filter parent 1: protocol all pref 1 u32 fh 800: ht divisor 1 
filter parent 1: protocol all pref 1 u32 fh 800::800 order 2048 key ht
800 bkt 0 flowid 1:1 
  match 00000016/0000ffff at 20
	action order 1: gact action pass
	 random type none pass val 0
	 index 1 ref 1 bind 1 installed 76 sec used 1 sec
 	Action statistics:
	Sent 7763 bytes 26 pkt (dropped 0, overlimits 0 requeues 0) 
	backlog 0b 0p requeues 0 
------

cheers,
jamal

> 
> On Tue, Jul 5, 2011 at 4:56 PM, jamal <hadi@cyberus.ca> wrote:
> > On Tue, 2011-07-05 at 16:07 +0300, Adam Katz wrote:
> >
> >> second, I just took at the libpcap source code and it seems it's using
> >> the same ETH_P_ALL option when binding to an interface. So based on
> >> what you're saying, the same solution of patching libpcap and
> >> replacing ETH_P_ALL with  ETH_P_IP should also make these rules work
> >> with traffic sent using pure libpcap or any libpcap - based
> >> application.
> >
> > ETH_P_ALL makes sense if you are unsure it is going to be IP. So i would
> > change/optimize apps only for IP if they are intended to deal with IP
> > only (same for ARP etc).
> > In your case, it seems it is tcp only - which runs on top of IP. So
> > it makes sense to do it for that specific use case etc.
> >
> > cheers,
> > jamal
> >
> >
> >

Re: libpcap and tc filters

[Adam Katz]

strange.
I've now tried the exact same configuration and it simply refuses to
work. Maybe your tcpreplay is configured differently...

What distro are you using? What kernel? What version of libpcap?


On Tue, Jul 5, 2011 at 5:41 PM, jamal <hadi@cyberus.ca> wrote:
> On Tue, 2011-07-05 at 17:21 +0300, Adam Katz wrote:
>> Yes. I understand the difference between ETH_P_ALL and ETH_P_IP...
>>
>> Jamal, I've now tested both solutions - changing the rule to "protocol
>> all" and patching tcpreplay to use ETH_P_IP and both produced the
>> exact same problem as before...
>
> Sorry - dont have much time to chase further, but it works for me.
>
> ---
> hadi@mojatatu10:~$ sudo tc qdisc del dev eth0 root handle 1:
> RTNETLINK answers: Invalid argument
> hadi@mojatatu10:~$ sudo tc qdisc add dev eth0 root handle 1: prio
> priomap 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
> hadi@mojatatu10:~$ sudo tc qdisc add dev eth0 parent 1:1 handle 10:
> pfifo
> hadi@mojatatu10:~$ sudo tc qdisc add dev eth0 parent 1:2 handle 20:
> pfifo
> hadi@mojatatu10:~$ sudo tc qdisc add dev eth0 parent 1:3 handle 30:
> pfifo
> hadi@mojatatu10:~$ sudo tc filter add dev eth0 protocol all parent 1:
> prio 1 u32 match ip dport 22 0xffff flowid 1:1 action ok
> hadi@mojatatu10:~$ sudo tc -s filter ls dev eth0
> filter parent 1: protocol all pref 1 u32
> filter parent 1: protocol all pref 1 u32 fh 800: ht divisor 1
> filter parent 1: protocol all pref 1 u32 fh 800::800 order 2048 key ht
> 800 bkt 0 flowid 1:1
>  match 00000016/0000ffff at 20
>        action order 1: gact action pass
>         random type none pass val 0
>         index 1 ref 1 bind 1 installed 15 sec used 15 sec
>        Action statistics:
>        Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
>        backlog 0b 0p requeues 0
>
> Note - the "OK" action is just a place holder to count packets.
> Now replay Adam's pcap file:
>
> hadi@mojatatu10:~/Downloads$ sudo tcpreplay
> --intf1=eth0 ./port22example.pcap
>
> sending out eth0
> processing file: ./port22example.pcap
> Actual: 50 packets (11594 bytes) sent in 3.66 seconds
> Rated: 3167.8 bps, 0.02 Mbps, 13.66 pps
> Statistics for network device: eth0
>        Attempted packets:         50
>        Successful packets:        50
>        Failed packets:            0
>        Retried packets (ENOBUFS): 0
>        Retried packets (EAGAIN):  0
>
> I dont have any ssh running on this maching. So
> lets check to see if anything was captured by the filter.
>
> -----
> hadi@mojatatu10:~$ sudo tc -s filter ls dev eth0
> filter parent 1: protocol all pref 1 u32
> filter parent 1: protocol all pref 1 u32 fh 800: ht divisor 1
> filter parent 1: protocol all pref 1 u32 fh 800::800 order 2048 key ht
> 800 bkt 0 flowid 1:1
>  match 00000016/0000ffff at 20
>        action order 1: gact action pass
>         random type none pass val 0
>         index 1 ref 1 bind 1 installed 76 sec used 1 sec
>        Action statistics:
>        Sent 7763 bytes 26 pkt (dropped 0, overlimits 0 requeues 0)
>        backlog 0b 0p requeues 0
> ------
>
> cheers,
> jamal
>
>>
>> On Tue, Jul 5, 2011 at 4:56 PM, jamal <hadi@cyberus.ca> wrote:
>> > On Tue, 2011-07-05 at 16:07 +0300, Adam Katz wrote:
>> >
>> >> second, I just took at the libpcap source code and it seems it's using
>> >> the same ETH_P_ALL option when binding to an interface. So based on
>> >> what you're saying, the same solution of patching libpcap and
>> >> replacing ETH_P_ALL with  ETH_P_IP should also make these rules work
>> >> with traffic sent using pure libpcap or any libpcap - based
>> >> application.
>> >
>> > ETH_P_ALL makes sense if you are unsure it is going to be IP. So i would
>> > change/optimize apps only for IP if they are intended to deal with IP
>> > only (same for ARP etc).
>> > In your case, it seems it is tcp only - which runs on top of IP. So
>> > it makes sense to do it for that specific use case etc.
>> >
>> > cheers,
>> > jamal
>> >
>> >
>> >
>
>
>

Re: libpcap and tc filters

[Eric Dumazet]

Le mardi 05 juillet 2011 à 18:16 +0300, Adam Katz a écrit :
> strange.
> I've now tried the exact same configuration and it simply refuses to
> work. Maybe your tcpreplay is configured differently...
> 
> What distro are you using? What kernel? What version of libpcap?

I did the same tests here and it works correctly for me.

latest kernel 3.0-rc6

# /usr/local/bin/tcpreplay -V
tcpreplay version: 3.4.4 (build 2450)
Copyright 2000-2010 by Aaron Turner <aturner at synfin dot net>
Cache file supported: 04
Not compiled with libdnet.
Compiled against libpcap: 1.1.1
64 bit packet counters: enabled
Verbose printing via tcpdump: enabled
Packet editing: disabled
Fragroute engine: disabled
Injection method: PF_PACKET send()

Re: libpcap and tc filters

[Adam Katz]

bah
I tried many times on both my ubuntu 10.04 boxes (2.6.32-32-generic)
one 32 bit, the other 64 and in both cases it simply refused to work.
I tried it with several different versions of libpcap (just in case)
as well as two different versions of tcpreplay (3.4.4 and 3.4.3) and
iproute (ubuntu 10.04 default version and the latest 2.6.39). One of
the machines had several NICs from different manufacturers so I tried
it on both too - and nothing!

However, once I moved to a third machine which is a virtual box with
fedora core 15 (2.6.38.6-26) - it suddenly worked.
In all cases I used the same pcap file and configuration.

unless someone suggests a better solution, it seems like i'll be
adhering to the first law of engineering - "if it works, don't fix it"
and simply replace install fedora core 15 instead of ubuntu.

On Tue, Jul 5, 2011 at 7:14 PM, Eric Dumazet <eric.dumazet@gmail.com> wrote:
> Le mardi 05 juillet 2011 à 18:16 +0300, Adam Katz a écrit :
>> strange.
>> I've now tried the exact same configuration and it simply refuses to
>> work. Maybe your tcpreplay is configured differently...
>>
>> What distro are you using? What kernel? What version of libpcap?
>
> I did the same tests here and it works correctly for me.
>
> latest kernel 3.0-rc6
>
> # /usr/local/bin/tcpreplay -V
> tcpreplay version: 3.4.4 (build 2450)
> Copyright 2000-2010 by Aaron Turner <aturner at synfin dot net>
> Cache file supported: 04
> Not compiled with libdnet.
> Compiled against libpcap: 1.1.1
> 64 bit packet counters: enabled
> Verbose printing via tcpdump: enabled
> Packet editing: disabled
> Fragroute engine: disabled
> Injection method: PF_PACKET send()
>
>
>

Re: libpcap and tc filters

[jamal]

On Tue, 2011-07-05 at 19:54 +0300, Adam Katz wrote:

> 
> unless someone suggests a better solution, it seems like i'll be
> adhering to the first law of engineering - "if it works, don't fix it"
> and simply replace install fedora core 15 instead of ubuntu.

My feeling is it has nothing to do with distro. Probably
your tcpreplay?

I am running ubuntu 10.04, but own compiled kernel (3.0-rc4). 
tcpreplay is older than Eric's version:
--
tcpreplay version: 3.4.3 (build 2375)
Copyright 2001-2009 by Aaron Turner <aturner at synfin dot net>
Cache file supported: 04
Not compiled with libdnet.
Compiled against libpcap: 1.0.0
64 bit packet counters: enabled
Verbose printing via tcpdump: enabled
Packet editing: disabled
Fragroute engine: disabled
Injection method: PF_PACKET send()
---

cheers,
jamal

Re: libpcap and tc filters

[Adam Katz]

I don't think so. I tried different versions of tcpreplay.

The solution didn't work on 3 different ubuntu 10.04 machines with
2.6.32-32 (actually one of them had 2.6.32-31) but it DID work on a
fedora core 15 virtual machine with a newer kernel as well as a second
virtual machine with on which I installed stock ubuntu 11.04 just for
this test. It has 2.6.38 i think. On the last three machines (the
third ubuntu 10.04 as well as fc15 and ubuntu 11.04 tcpreplay came
straight from the repository).

Just to make myself clear - earlier, I tried the solution on a 10.04
physical box with various versions of tcpreplay, libpcap, and
iproute2. nothing worked.


On Tue, Jul 5, 2011 at 10:19 PM, jamal <hadi@cyberus.ca> wrote:
> On Tue, 2011-07-05 at 19:54 +0300, Adam Katz wrote:
>
>>
>> unless someone suggests a better solution, it seems like i'll be
>> adhering to the first law of engineering - "if it works, don't fix it"
>> and simply replace install fedora core 15 instead of ubuntu.
>
> My feeling is it has nothing to do with distro. Probably
> your tcpreplay?
>
> I am running ubuntu 10.04, but own compiled kernel (3.0-rc4).
> tcpreplay is older than Eric's version:
> --
> tcpreplay version: 3.4.3 (build 2375)
> Copyright 2001-2009 by Aaron Turner <aturner at synfin dot net>
> Cache file supported: 04
> Not compiled with libdnet.
> Compiled against libpcap: 1.0.0
> 64 bit packet counters: enabled
> Verbose printing via tcpdump: enabled
> Packet editing: disabled
> Fragroute engine: disabled
> Injection method: PF_PACKET send()
> ---
>
> cheers,
> jamal
>
>

libpcap and tc filters

[Adam Katz]

Hi Everyone!

I'm attempting to use tc to shape traffic sent using libpcap (more
precisely tcpreplay, based on libpcap). I'm doing this for a research
project.

Ii have a prio scheduler with a default band 2:

tc qdisc add dev eth0 root handle 1: prio priomap 2 2 2 2 2 2 2 2 2 2
2 2 2 2 2 2

with two filters attached to it:

tc filter add dev eth0 protocol ip parent 1: prio 1 u32 match ip dport
80 0xffff flowid 1:1
tc filter add dev eth0 protocol ip parent 1: prio 2 u32 match ip dport
443 0xffff flowid 1:2

(disregard the port numbers, they're here just as an example)

When sending actual traffic, the filters work and I see the
appropriate traffic entering the right class. BUT when replaying
captured traffic (with the appropriate port numbers) over eth0 using
tcpreplay, all packets end up in the default band 2 as if the filters
simply refuse to work.

Can anyone suggest a solution?

Thanks!