pktgen patch available for perusal.

pktgen patch available for perusal.

[Ben Greear]

I've completed the first pass of my changes to pktgen in 2.6.18.
Many of these features are probably DOA based on previous conversations,
but perhaps this will help someone....

Changes:
* use a nano-second timer based on the scheduler timer (TSC) for relative times, instead of get_time_of_day.
* Don't busy-spin:
    * add hook to netdevice wake tx queue to re-start when queue was stopped.
    * add logic to pktgen to register/unregister this hook.
    * accumulate delay and sleep interruptible on a queue when 1ms of sleep has accumulated
      or when the associated netdev tx-queues for pktgen devices needing to transmit are asleep.
    (This all assumes that HZ is 1000, could probably work with other HZ with minor work.)
* Add receive logic to dev.c so that pktgen can receive packets and report on various stats
   including latency, packet loss, jitter, etc.
* Remove the if-lock, primarily by making the 'add-interface' logic deferred to the
   thread's main loop.  It's likely I missed a corner case or two here, hopefully I'll
   find them all in testing...
* Add ioctl to grab all of the pktgen stats in one call.

The pktgen.[ch] patch is here:  http://www.candelatech.com/oss/pktgen.patch

The full patch, including the callback hooks (and several other features)
is located here:  http://www.candelatech.com/oss/candela_2.6.18.patch

Thanks,
Ben

-- 
Ben Greear <greearb@candelatech.com>
Candela Technologies Inc  http://www.candelatech.com

pktgen patch available for perusal.

[Robert Olsson]

Ben Greear writes:
 > I've completed the first pass of my changes to pktgen in 2.6.18.
 > Many of these features are probably DOA based on previous conversations,
 > but perhaps this will help someone....

 Thanks. Well sometimes there is a need to capture and drop pkts and various 
 points, so it handy to have code fragments ready and just add the hook when 
 it's needed in driver or rx softirq etc.

 > Changes:
 > * use a nano-second timer based on the scheduler timer (TSC) for relative times, instead of get_time_of_day.

 Any chance you extract this one so we can compare with get_time_of_day. This pops up
 in the profiles and TX numbers are not what used to be.

 Cheers.
					--ro

Re: pktgen patch available for perusal.

[Ben Greear]

Robert Olsson wrote:
> Ben Greear writes:
>  > I've completed the first pass of my changes to pktgen in 2.6.18.
>  > Many of these features are probably DOA based on previous conversations,
>  > but perhaps this will help someone....
> 
>  Thanks. Well sometimes there is a need to capture and drop pkts and various 
>  points, so it handy to have code fragments ready and just add the hook when 
>  it's needed in driver or rx softirq etc.

I'd be thrilled to have the receive logic go into pktgen, even if it was #if 0 with a comment
showing how to patch dev.c to get it working.  It would make my out-of-tree patch smaller
and should help others who are doing research and driver development...

If you'd accept a patch like this, please let me know.

>  > Changes:
>  > * use a nano-second timer based on the scheduler timer (TSC) for relative times, instead of get_time_of_day.
> 
>  Any chance you extract this one so we can compare with get_time_of_day. This pops up
>  in the profiles and TX numbers are not what used to be.

I'll work on that.

Thanks,
Ben

> 
>  Cheers.
> 					--ro
> -
> To unsubscribe from this list: send the line "unsubscribe netdev" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html


-- 
Ben Greear <greearb@candelatech.com>
Candela Technologies Inc  http://www.candelatech.com

Re: pktgen patch available for perusal.

[Robert Olsson]

Ben Greear writes:

 > I'd be thrilled to have the receive logic go into pktgen, even if it was #if 0 with a comment
 > showing how to patch dev.c to get it working.  It would make my out-of-tree patch smaller
 > and should help others who are doing research and driver development...


 Just an idea... RX part is pretty separate from TX. How about if we keep the 
 this code in a small seprate optional module? It can developed into some general 
 RX probe.

 Cheers.
						--ro

Re: pktgen patch available for perusal.

[Ben Greear]

Robert Olsson wrote:
> Ben Greear writes:
>
>  > I'd be thrilled to have the receive logic go into pktgen, even if it was #if 0 with a comment
>  > showing how to patch dev.c to get it working.  It would make my out-of-tree patch smaller
>  > and should help others who are doing research and driver development...
>
>
>  Just an idea... RX part is pretty separate from TX. How about if we keep the 
>  this code in a small seprate optional module? It can developed into some general 
>  RX probe.
>   
It requires a hook in dev.c, or at least that is the only way I can 
think to implement it.
I don't think that hook is going to be allowed into the kernel, so the 
best I was hoping for
was to have the code in pktgen with #if 0 and let users patch their 
kernel to add the
hook...

Ben

>  Cheers.
> 						--ro
> -
> To unsubscribe from this list: send the line "unsubscribe netdev" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>   


-- 
Ben Greear <greearb@candelatech.com> 
Candela Technologies Inc  http://www.candelatech.com

Re: pktgen patch available for perusal.

[Robert Olsson]

Ben Greear writes:

 > It requires a hook in dev.c, or at least that is the only way I can 
 > think to implement it.

Well the hook be placed along the packet path even in drivers. In tulip I didn't 
even take packet of the ring in some experiments. 

And there plenty of existing hooks already i.e PRE_ROUTE?

 > I don't think that hook is going to be allowed into the kernel, so the 
 > best I was hoping for  was to have the code in pktgen with #if 0 and let 
 > users patch their kernel to add the hook...

Right so what I was trying to say was rather having #if 0 and dead code in 
pktgen it could be kept separate. 

Cheers.
						--ro

Re: pktgen patch available for perusal.

[Ben Greear]

Robert Olsson wrote:
> Ben Greear writes:
> 
>  > It requires a hook in dev.c, or at least that is the only way I can 
>  > think to implement it.
> 
> Well the hook be placed along the packet path even in drivers. In tulip I didn't 
> even take packet of the ring in some experiments. 
> 
> And there plenty of existing hooks already i.e PRE_ROUTE?

For my particular application the hook needs to be right
after the bridge hook.  My dev.c hook looks like this:

#if defined(CONFIG_NET_PKTGEN) || defined(CONFIG_NET_PKTGEN_MODULE)
#include "pktgen.h"

#warning "Compiling dev.c for pktgen.";

int (*handle_pktgen_hook)(struct sk_buff *skb) = NULL;
EXPORT_SYMBOL(handle_pktgen_hook);

static __inline__ int handle_pktgen_rcv(struct sk_buff* skb) {
	if (handle_pktgen_hook) {
		return handle_pktgen_hook(skb);
	}
	return -1;
}
#endif

.....

#if defined(CONFIG_NET_PKTGEN) || defined(CONFIG_NET_PKTGEN_MODULE)
	if ((skb->dev->pkt_dev) &&
	    (handle_pktgen_rcv(skb) >= 0)) {
		/* Pktgen may consume the packet, no need to send
		 * to further protocols.
		 */
		goto out;
	}
#endif


If the rx-hook logic is already in pktgen module, and it's symbol
exported, perhaps a second hook module could be written that
would just be the bridge between a driver or a PRE-ROUTE hook
and pktgen?  The hook module would probably not be much larger
than the code above...


Thanks,
Ben

-- 
Ben Greear <greearb@candelatech.com>
Candela Technologies Inc  http://www.candelatech.com

Re: pktgen patch available for perusal.

[jamal]

On Wed, 2006-01-11 at 11:11 -0800, Ben Greear wrote:

> I'd be thrilled to have the receive logic go into pktgen, even if 
> it was #if 0 with a comment
> showing how to patch dev.c to get it working.  It would make my 
> out-of-tree patch smaller
> and should help others who are doing research and driver development...
> 

I use pktgen extensively these days for ipsec testing. I also record
stats for pkts i receive using a tc action drop. Very simple and works
great.  
Ben, you keep insisting on doing this hook (every 3 months) because you
dont want to invest time to do it with netlink (I thought you sold this
as part of your product - the investment of your time is really not that
high). If you tell me what the packet trigger is, I will send you a
script ;->

Robert, I have started writing some generic netlink messaging to replace
the /proc that i will send your way. I need to get consensus on some
tunnel-mode IPSEC patches first then i will send the about three
patchsets your way (to replace the old ones i sent earlier).

cheers,
jamal

Re: pktgen patch available for perusal.

[Ben Greear]

jamal wrote:
> On Wed, 2006-01-11 at 11:11 -0800, Ben Greear wrote:
>
>   
>> I'd be thrilled to have the receive logic go into pktgen, even if 
>> it was #if 0 with a comment
>> showing how to patch dev.c to get it working.  It would make my 
>> out-of-tree patch smaller
>> and should help others who are doing research and driver development...
>>
>>     
>
> I use pktgen extensively these days for ipsec testing. I also record
> stats for pkts i receive using a tc action drop. Very simple and works
> great.  
> Ben, you keep insisting on doing this hook (every 3 months) because you
> dont want to invest time to do it with netlink (I thought you sold this
> as part of your product - the investment of your time is really not that
> high). If you tell me what the packet trigger is, I will send you a
> script ;->
>   
Please do send a script.  I match based on this method below.   Probably 
you only
need the part that checks for the MAGIC, the rest can be in a method 
called after
you match in your script?

/* Returns < 0 if the skb is not a pktgen buffer. */
int pktgen_receive(struct sk_buff* skb) {
        /* See if we have a pktgen packet */
    /* TODO:  Add support for detecting IPv6, TCP packets too.  This 
will only
     * catch UDP at the moment. --Ben
     */
    /*printk("pktgen-rcv, skb->len: %d\n", skb->len);*/
   
        if ((skb->len >= (20 + 8 + sizeof(struct pktgen_hdr))) &&
            (skb->protocol == __constant_htons(ETH_P_IP))) {
        struct pktgen_hdr* pgh;
               
                /* It's IP, and long enough, lets check the magic number.
                 * TODO:  This is a hack not always guaranteed to catch 
the right
                 * packets.
                 */
       
        /*printk("Length & protocol passed, skb->data: %p, raw: %p\n",
          skb->data, skb->h.raw);*/
      
                pgh = (struct pktgen_hdr*)(skb->data + 20 + 8);
       
                /*
                tmp = (char*)(skb->data);
                for (i = 0; i<90; i++) {
                        printk("%02hx ", tmp[i]);
                        if (((i + 1) % 15) == 0) {
                                printk("\n");
                        }
                }
                printk("\n");
                */
               
                if (pgh->pgh_magic == __constant_ntohl(PKTGEN_MAGIC)) {
                        struct net_device* dev = skb->dev;
                        struct pktgen_dev* pkt_dev;
                        __u32 seq = ntohl(pgh->seq_num);

            // TODO:  Need lock..maybe
            pkt_dev = dev->pkt_dev;
                       
                        if (!pkt_dev) {
                return -1;
                        }

                        pkt_dev->pkts_rcvd++;
                        pkt_dev->bytes_rcvd += ((skb->tail - 
skb->mac.raw) + 4); /* +4 for the checksum */
                       
                        /* Check for out-of-sequence packets */
                        if (pkt_dev->last_seq_rcvd == seq) {
                                pkt_dev->dup_rcvd++;
                                pkt_dev->dup_since_incr++;
                        }
                        else {
                                __s64 rx;
                                __s64 tx;
                                struct timeval txtv;
                if (skb->tstamp.off_sec || skb->tstamp.off_usec) {
                    skb_get_timestamp(skb, &txtv);
                }
                else {
                    do_gettimeofday(&txtv);
                    skb_set_timestamp(skb, &txtv);
                }
                                rx = tv_to_us(&txtv);

                                txtv.tv_usec = ntohl(pgh->tv_usec);
                                txtv.tv_sec = ntohl(pgh->tv_sec);
                                tx = tv_to_us(&txtv);
                                record_latency(pkt_dev, rx - tx);
                               
                                if ((pkt_dev->last_seq_rcvd + 1) == seq) {
                                        if ((pkt_dev->peer_clone_skb > 1) &&
                                            (pkt_dev->peer_clone_skb > 
(pkt_dev->dup_since_incr + 1))) {
                                               
                                                pkt_dev->seq_gap_rcvd += 
(pkt_dev->peer_clone_skb -
                                                                       
pkt_dev->dup_since_incr - 1);
                                        }
                                        /* Great, in order...all is well */
                                }
                                else if (pkt_dev->last_seq_rcvd < seq) {
                                        /* sequence gap, means we 
dropped a pkt most likely */
                                        if (pkt_dev->peer_clone_skb > 1) {
                                                /* We dropped more than 
one sequence number's worth,
                                                 * and if we're using 
clone_skb, then this is quite
                                                 * a few.  This number 
still will not be exact, but
                                                 * it will be closer.
                                                 */
                                                pkt_dev->seq_gap_rcvd += 
(((seq - pkt_dev->last_seq_rcvd) *
                                                                        
pkt_dev->peer_clone_skb) -
                                                                       
pkt_dev->dup_since_incr);
                                        }
                                        else {
                                                pkt_dev->seq_gap_rcvd += 
(seq - pkt_dev->last_seq_rcvd - 1);
                                        }
                                }
                                else {
                                        pkt_dev->ooo_rcvd++; /* 
out-of-order */
                                }
                               
                                pkt_dev->dup_since_incr = 0;
                        }
                        pkt_dev->last_seq_rcvd = seq;
                        kfree_skb(skb);
                        if (debug > 1) {
                                printk("done with pktgen_receive, free'd 
pkt\n");
                        }
                        return 0;
                }
        }
        return -1; /* Let another protocol handle it, it's not for us! */
}/* pktgen_receive */

> Robert, I have started writing some generic netlink messaging to replace
> the /proc that i will send your way. I need to get consensus on some
> tunnel-mode IPSEC patches first then i will send the about three
> patchsets your way (to replace the old ones i sent earlier).
>
> cheers,
> jamal
>
> -
> To unsubscribe from this list: send the line "unsubscribe netdev" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>   


-- 
Ben Greear <greearb@candelatech.com> 
Candela Technologies Inc  http://www.candelatech.com

Re: pktgen patch available for perusal.

[jamal]

On Sat, 2006-04-11 at 09:29 -0800, Ben Greear wrote:
> jamal wrote:
   
> Please do send a script.  I match based on this method below.   Probably 
> you only
> need the part that checks for the MAGIC, 

What i do in my case is send to the SUT to UDP port 9. The SUT loops
back the packet to me after processing and i just have a simple check in
the traffic generator of:

a) Packet = IPV4, IP address matches what i want, port =9, 
b) If this matches i  account and drop it. This means all other traffic
(like ssh etc goes through fine).

I dont really need to check for length because _i know_ udp port 9 to
the SUT is test traffic.

For such a setup, the script would be as simple as (assuming SUT hooked
to eth0):

--
#interested in incoming packets on eth0
tc qdisc add dev eth0 ingress
#interested in packets coming from SUT(10.0.0.21) on UDP port 9
tc filter add dev eth0 parent ffff: protocol ip prio 6 u32 \
match ip src 10.0.0.21/32 \
match udp port 9 0xffff \
flowid 1:16 \
action drop
---

When you do a listing of this filter you will see accounting info.
You list by saying:
---
tc -s filter ls dev eth0 parent ffff:
---

You can ofcourse add many of these based on other header info.

It seems to me your magic header is inside the UDP packet, correct?
In that case you will have to play with matches since you can specify
arbitraty offsets and values inside the packet.

Of course you can do this from your application instead of using tc.
I think that will be the best place to control this and sync with
pktgen sending.

> the rest can be in a method 
> called after you match in your script?
> 

If the packet/byte counters that the drop action provides are not good
enough, you can write yourself a little module that will do the
accounting the way you want it. For example this will be necessary to
the out-of-sequence cheques below. Timestamps are already being updated
Look at net/sched/act_simple.c and its associated user space code in
iproute2.

Now, Ben - are you listening really this time or am i wasting my time
for the nth time giving you all these details? ;->

If you are listening then start with:

1) Do a simple test with just udp traffic as above, doing simple
accounting. This helps you to get a feel on how things work.
2) modify the matching rules to match your magic cookie
3) write a simple action invoked by your matching rules and use tc to
add it to the policy.
4) integrate in your app now that you know what you are doing.

If you follow these steps or a variant-of i will spend time and help.

cheers,
jamal

Re: pktgen patch available for perusal.

[Ben Greear]

jamal wrote:
> On Sat, 2006-04-11 at 09:29 -0800, Ben Greear wrote:
>   

> You can ofcourse add many of these based on other header info.
>
> It seems to me your magic header is inside the UDP packet, correct?
> In that case you will have to play with matches since you can specify
> arbitraty offsets and values inside the packet.
>   
Yes, I want to be able to randomize source/dest MAC, IP, and IP Port, so 
I need to match only on the
magic cookie.  I'll soon add support to deal with TCP packets as well, 
but that is only going to require
calculating a different offset to look for the magic value.
> Of course you can do this from your application instead of using tc.
> I think that will be the best place to control this and sync with
> pktgen sending.
>   
You mean the equiv of calling system(tc filter .....) from the app, or 
do you mean something else
entirely?
>> the rest can be in a method 
>> called after you match in your script?
>>
>>     
>
> If the packet/byte counters that the drop action provides are not good
> enough, you can write yourself a little module that will do the
> accounting the way you want it. For example this will be necessary to
> the out-of-sequence cheques below. Timestamps are already being updated
> Look at net/sched/act_simple.c and its associated user space code in
> iproute2.
>
> Now, Ben - are you listening really this time or am i wasting my time
> for the nth time giving you all these details? ;->
>   
You always give details that are pertinent to your setup and not to 
mine, and you
always have some 'small' step that is 'write a kernel module and hack on 
tc' if you
want it to do something other than the generic thing.
> If you are listening then start with:
>
> 1) Do a simple test with just udp traffic as above, doing simple
> accounting. This helps you to get a feel on how things work.
> 2) modify the matching rules to match your magic cookie
>   
These first two do not look too difficult.  I certainly need more 
control/stats than
what the generic counters would be, however, so I assume I need to move 
to step 3.
> 3) write a simple action invoked by your matching rules and use tc to
> add it to the policy.
>   
I have exactly no idea of what you mean by this.  How do I get this tc 
framework to
send the packet with 32-bit value FOO at offset X to some method called 
pktgen_rx_pkt(skb)
in the pktgen module?  That is all I want in a hook:  The ability to 
match on a pkt and send it
to my kernel module and to let my kernel module and have it traverse the 
stack no farther
unless my kernel module decides to re-insert it for some reason.

If this takes significant work, then please don't waste more time trying 
to talk me
into this, as my small patch to dev.c already does exactly what I need 
and the code
is very easy to understand.

If this is easy, how about you write the module and make the tc 
changes.  I'll work on any
changes needed in pktgen and polish up a nice patch for Robert et 
al....  Then you can get
the latency distributions, OOO, Dup, Dropped and other counters that my 
pktgen logic
provides for your own testing purposes...

Thanks,
Ben

-- 
Ben Greear <greearb@candelatech.com> 
Candela Technologies Inc  http://www.candelatech.com

Re: pktgen patch available for perusal.

[Robert Olsson]

jamal writes:

 > If you are listening then start with:
 > 
 > 1) Do a simple test with just udp traffic as above, doing simple
 > accounting. This helps you to get a feel on how things work.
 > 2) modify the matching rules to match your magic cookie
 > 3) write a simple action invoked by your matching rules and use tc to
 > add it to the policy.
 > 4) integrate in your app now that you know what you are doing.

 Yes. Sounds like simple and general solution. No call-backs, no #ifdef's
 no extra modules. Just a little recipe in pktgen.txt

 Cheers.
					--ro
 

Re: pktgen patch available for perusal.

[Robert Olsson]

Ben Greear writes:
 > > Changes:
 > > * use a nano-second timer based on the scheduler timer (TSC) for relative times, instead of get_time_of_day.

Seems I missed to set tsc as clocksource. It makes a difference. Performance is 
normal and I'm less confused.

e1000 82546GB @ 1.6 GHz Opteron. Kernel 2.6.19-rc1_Bifrost_-g18e199c6-dirty 

echo acpi_pm    > /sys/devices/system/clocksource/clocksource0/current_clocksource   

psize pps
---------
60  556333
124  526942
252  452981
508  234996
1020  119748
1496  82248

echo tsc        > /sys/devices/system/clocksource/clocksource0/current_clocksource 

psize pps
---------
60  819914
124  747286
252  452975
508  234993
1020  119749
1496  82247

Cheers.	
						--ro