How can I calculate/estimate pps(packet per seocond) and bps(bit per second) in DPDK pktg

How can I calculate/estimate pps(packet per seocond) and bps(bit per second) in DPDK pktg

[최익성]

Dear DPDK experts. 
 
Thank you very much for your excellent work and great contributions.
 
I have a question about pps(packet per seocond) and bps(bit per second) estimation in DPDK pktgen.
 
I can't find the source code which calculates pps and bps for received packets in the DPDK pktgen source code.
 
Would you give any information/algorithm to estimate pps and bps?
 
Thank you very much. 
 
Sincerely Yours,
 
Ick-Sung Choi.
 

Re: How can I calculate/estimate pps(packet per seocond) and bps(bit per second) in DPDK pktg

[Wiles, Keith]

On 11/3/15, 12:25 AM, "dev on behalf of 최익성" <dev-bounces@dpdk.org on behalf of pnk003@naver.com> wrote:

>Dear DPDK experts. 
> 
>Thank you very much for your excellent work and great contributions.
> 
>I have a question about pps(packet per seocond) and bps(bit per second) estimation in DPDK pktgen.
> 
>I can't find the source code which calculates pps and bps for received packets in the DPDK pktgen source code.
> 
>Would you give any information/algorithm to estimate pps and bps?

Please look in the app/pktgen.c file for two functions pktgen_wire_size() and ptkgen_packet_rate(). The first function calculates the wire size of the packet and the second one calculates the packet rate for transmission. These two are used to determine the rate to send packets and the amount of time between delays.

The next part is how you convert the RX/Tx to pps and bps. I have macros that convert the packets per second to bits per second. The code to get the per second values is located in app/pktgen-stat.c and use a timer to grab the values and calculate the delta per second.

The two macros you are looking for are iBitsTotal(info->rate_stats), oBitsTotal(info->rate_stats) in the app/pktgen-stat.c line 297.

Here are the macros in app/pktgen.h

INTER_FRAME_GAP         = 12,
PKT_PREAMBLE_SIZE       = 8,
    FCS_SIZE                = 4,


#define iBitsTotal(_x) \
    (((_x.ipackets * (INTER_FRAME_GAP + PKT_PREAMBLE_SIZE)) + _x.ibytes) << 3)
#define oBitsTotal(_x) \
    (((_x.opackets * (INTER_FRAME_GAP + PKT_PREAMBLE_SIZE)) + _x.obytes) << 3)


Hmm, I just noticed I did not include the FCS bytes. Does the NIC include FCS bytes in the counters? Need to verify that one and if not then it becomes a bit more complex.

But that is how I calculate the values, if you see a problem let me know.

> 
>Thank you very much. 
> 
>Sincerely Yours,
> 
>Ick-Sung Choi.
> 
>


Regards,
Keith

Re: How can I calculate/estimate pps(packet per seocond) and bps(bit per second) in DPDK pktg

[Van Haaren, Harry]

Hi Keith,

> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Wiles, Keith
<snip>
> Hmm, I just noticed I did not include the FCS bytes. Does the NIC include FCS bytes in the
> counters? Need to verify that one and if not then it becomes a bit more complex.

The Intel NICs count packet sizes inclusive of CRC / FCS, from eg the ixgbe/82599 datasheet:
"This register includes bytes received in a packet from the <Destination Address> field through the <CRC> field, inclusively."

-Harry

Re: How can I calculate/estimate pps(packet per seocond) and bps(bit per second) in DPDK pktg

[Wiles, Keith]

On 11/3/15, 8:30 AM, "Van Haaren, Harry" <harry.van.haaren@intel.com> wrote:

>Hi Keith,
>
>> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Wiles, Keith
><snip>
>> Hmm, I just noticed I did not include the FCS bytes. Does the NIC include FCS bytes in the
>> counters? Need to verify that one and if not then it becomes a bit more complex.
>
>The Intel NICs count packet sizes inclusive of CRC / FCS, from eg the ixgbe/82599 datasheet:
>"This register includes bytes received in a packet from the <Destination Address> field through the <CRC> field, inclusively."

Thanks I assumed I had known that at the time :-)
>
>-Harry
>


Regards,
Keith

Re: How can I calculate/estimate pps(packet per seocond) and bps(bit per second) in DPDK pktg

[Polehn, Mike A]

I used the following code snip-it with the i40e device, with 1 second sample time had very high accuracy for IPv4 UDP packets:

#define FLOWD_PERF_PACKET_OVERHEAD 24  /* CRC + Preamble + SOF + Interpacket gap */
#define FLOWD_REF_NETWORK_SPEED   10e9

double Ave_Bytes_per_Packet, Data_Rate, Net_Rate;
uint64_t Bits;
uint64_t Bytes = pFlow->flow.n_bytes - pMatch_Prev->flow.n_bytes;
uint64_t Packets = pFlow->flow.n_packets - pMatch_Prev->flow.n_packets;
uint64_t Time_us = pFlow->flow.flow_time_us - pMatch_Prev->flow.flow_time_us;

if (Bytes == 0)
	Ave_Bytes_per_Packet = 0.0;
else
	Ave_Bytes_per_Packet = ((double)Bytes / (double)Packets) + 4.0;

Bits = (Bytes + (Packets*FLOWD_PERF_PACKET_OVERHEAD)) * 8;
if (Bits == 0)
	Data_Rate = 0.0;
else
	Data_Rate = (((double)Bits) / Time_us) * 1e6;

if (Data_Rate == 0.0)
	Net_Rate = 0.0;
else
	Net_Rate = Data_Rate / FLOWD_REF_NETWORK_SPEED;

For packet rate: double pk_rate = (((double)Packets)/ ((double)Time_us)) * 1e6;

To calculate elapsed time in DPDK app, used CPU counter (will not work if counter is being modified):

Initialization:
double flow_time_scale_us;
...
flow_time_scale_us = 1e6/rte_get_tsc_hz();

Elapsed time (uSec) example: 

elapse_us = (rte_rdtsc() - entry->tsc_first_packet) *
	flow_time_scale_us; /* calc total elapsed us */

-----Original Message-----
From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Wiles, Keith
Sent: Tuesday, November 3, 2015 6:33 AM
To: Van Haaren, Harry; ???; dev@dpdk.org
Subject: Re: [dpdk-dev] How can I calculate/estimate pps(packet per seocond) and bps(bit per second) in DPDK pktg

On 11/3/15, 8:30 AM, "Van Haaren, Harry" <harry.van.haaren@intel.com> wrote:

>Hi Keith,
>
>> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Wiles, Keith
><snip>
>> Hmm, I just noticed I did not include the FCS bytes. Does the NIC 
>> include FCS bytes in the counters? Need to verify that one and if not then it becomes a bit more complex.
>
>The Intel NICs count packet sizes inclusive of CRC / FCS, from eg the ixgbe/82599 datasheet:
>"This register includes bytes received in a packet from the <Destination Address> field through the <CRC> field, inclusively."

Thanks I assumed I had known that at the time :-)
>
>-Harry
>


Regards,
Keith

Re: How can I calculate/estimate pps(packet per seocond) and bps(bit per second) in DPDK pktg

[Wiles, Keith]

On 11/3/15, 9:59 AM, "Polehn, Mike A" <mike.a.polehn@intel.com> wrote:

>I used the following code snip-it with the i40e device, with 1 second sample time had very high accuracy for IPv4 UDP packets:
>
>#define FLOWD_PERF_PACKET_OVERHEAD 24  /* CRC + Preamble + SOF + Interpacket gap */
>#define FLOWD_REF_NETWORK_SPEED   10e9
>
>double Ave_Bytes_per_Packet, Data_Rate, Net_Rate;
>uint64_t Bits;
>uint64_t Bytes = pFlow->flow.n_bytes - pMatch_Prev->flow.n_bytes;
>uint64_t Packets = pFlow->flow.n_packets - pMatch_Prev->flow.n_packets;
>uint64_t Time_us = pFlow->flow.flow_time_us - pMatch_Prev->flow.flow_time_us;
>
>if (Bytes == 0)
>	Ave_Bytes_per_Packet = 0.0;
>else
>	Ave_Bytes_per_Packet = ((double)Bytes / (double)Packets) + 4.0;
>
>Bits = (Bytes + (Packets*FLOWD_PERF_PACKET_OVERHEAD)) * 8;
>if (Bits == 0)
>	Data_Rate = 0.0;
>else
>	Data_Rate = (((double)Bits) / Time_us) * 1e6;
>
>if (Data_Rate == 0.0)
>	Net_Rate = 0.0;
>else
>	Net_Rate = Data_Rate / FLOWD_REF_NETWORK_SPEED;
>
>For packet rate: double pk_rate = (((double)Packets)/ ((double)Time_us)) * 1e6;
>
>To calculate elapsed time in DPDK app, used CPU counter (will not work if counter is being modified):
>
>Initialization:
>double flow_time_scale_us;
>...
>flow_time_scale_us = 1e6/rte_get_tsc_hz();
>
>Elapsed time (uSec) example: 
>
>elapse_us = (rte_rdtsc() - entry->tsc_first_packet) *
>	flow_time_scale_us; /* calc total elapsed us */

Looks reasonable I assume the n_bytes does not include FCS as is not the case with the NIC counters.

Also I decided to avoid using double’s in my code and just used 64bit registers and integer math :-) 

>
>-----Original Message-----
>From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Wiles, Keith
>Sent: Tuesday, November 3, 2015 6:33 AM
>To: Van Haaren, Harry; ???; dev@dpdk.org
>Subject: Re: [dpdk-dev] How can I calculate/estimate pps(packet per seocond) and bps(bit per second) in DPDK pktg
>
>On 11/3/15, 8:30 AM, "Van Haaren, Harry" <harry.van.haaren@intel.com> wrote:
>
>>Hi Keith,
>>
>>> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Wiles, Keith
>><snip>
>>> Hmm, I just noticed I did not include the FCS bytes. Does the NIC 
>>> include FCS bytes in the counters? Need to verify that one and if not then it becomes a bit more complex.
>>
>>The Intel NICs count packet sizes inclusive of CRC / FCS, from eg the ixgbe/82599 datasheet:
>>"This register includes bytes received in a packet from the <Destination Address> field through the <CRC> field, inclusively."
>
>Thanks I assumed I had known that at the time :-)
>>
>>-Harry
>>
>
>
>Regards,
>Keith
>
>
>
>
>


Regards,
Keith

Re: How can I calculate/estimate pps(packet per seocond) and bps(bit per second) in DPDK pktg

[Polehn, Mike A]

Accessing registers on the NIC has very high access latency and will often stall the CPU in waiting for response 
especially with multiple register reads and high throughput packet data also being transferred. The size value was 
derived from the NIC writing a value to the descriptor table which as then written to the packet buffer. The 
bitrate calculation included the FCS/CRC has packet overhead and the packet size was 4 bytes short. 

The inclusion or exclusion of the FCS on receive might be a programmable option. For tx, it might be a flag set
in the TX descriptor table either use FCS in packet buffer or calculate it on the fly. Where you get the numbers
and initialization may affect the calculation. 

A very important rating for CPU is it's FLOPs performance. Most all modern CPUs do single cycle floating point 
multiplies (divides are done with shifts and adds and are clock per set bit in float mantissa or in int). Conversion 
to and from floating point are often done in parallel with other operations, which makes using integer math 
not always faster. Often additional checks for edge conditions and adjustments needed with integer 
processing loses the gain but all depends on exact algorithm and end scaling.  Being able to do high 
quality integer processing is a good skill, especially when doing work like signal processing.

-----Original Message-----
From: Wiles, Keith 
Sent: Tuesday, November 3, 2015 11:01 AM
To: Polehn, Mike A; Van Haaren, Harry; ???; dev@dpdk.org
Subject: Re: [dpdk-dev] How can I calculate/estimate pps(packet per seocond) and bps(bit per second) in DPDK pktg

On 11/3/15, 9:59 AM, "Polehn, Mike A" <mike.a.polehn@intel.com> wrote:

>I used the following code snip-it with the i40e device, with 1 second sample time had very high accuracy for IPv4 UDP packets:
>
>#define FLOWD_PERF_PACKET_OVERHEAD 24  /* CRC + Preamble + SOF + Interpacket gap */
>#define FLOWD_REF_NETWORK_SPEED   10e9
>
>double Ave_Bytes_per_Packet, Data_Rate, Net_Rate; uint64_t Bits; 
>uint64_t Bytes = pFlow->flow.n_bytes - pMatch_Prev->flow.n_bytes; 
>uint64_t Packets = pFlow->flow.n_packets - pMatch_Prev->flow.n_packets; 
>uint64_t Time_us = pFlow->flow.flow_time_us - 
>pMatch_Prev->flow.flow_time_us;
>
>if (Bytes == 0)
>	Ave_Bytes_per_Packet = 0.0;
>else
>	Ave_Bytes_per_Packet = ((double)Bytes / (double)Packets) + 4.0;
>
>Bits = (Bytes + (Packets*FLOWD_PERF_PACKET_OVERHEAD)) * 8; if (Bits == 
>0)
>	Data_Rate = 0.0;
>else
>	Data_Rate = (((double)Bits) / Time_us) * 1e6;
>
>if (Data_Rate == 0.0)
>	Net_Rate = 0.0;
>else
>	Net_Rate = Data_Rate / FLOWD_REF_NETWORK_SPEED;
>
>For packet rate: double pk_rate = (((double)Packets)/ 
>((double)Time_us)) * 1e6;
>
>To calculate elapsed time in DPDK app, used CPU counter (will not work if counter is being modified):
>
>Initialization:
>double flow_time_scale_us;
>...
>flow_time_scale_us = 1e6/rte_get_tsc_hz();
>
>Elapsed time (uSec) example: 
>
>elapse_us = (rte_rdtsc() - entry->tsc_first_packet) *
>	flow_time_scale_us; /* calc total elapsed us */

Looks reasonable I assume the n_bytes does not include FCS as is not the case with the NIC counters.

Also I decided to avoid using double’s in my code and just used 64bit registers and integer math :-) 

>
>-----Original Message-----
>From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Wiles, Keith
>Sent: Tuesday, November 3, 2015 6:33 AM
>To: Van Haaren, Harry; ???; dev@dpdk.org
>Subject: Re: [dpdk-dev] How can I calculate/estimate pps(packet per 
>seocond) and bps(bit per second) in DPDK pktg
>
>On 11/3/15, 8:30 AM, "Van Haaren, Harry" <harry.van.haaren@intel.com> wrote:
>
>>Hi Keith,
>>
>>> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Wiles, Keith
>><snip>
>>> Hmm, I just noticed I did not include the FCS bytes. Does the NIC 
>>> include FCS bytes in the counters? Need to verify that one and if not then it becomes a bit more complex.
>>
>>The Intel NICs count packet sizes inclusive of CRC / FCS, from eg the ixgbe/82599 datasheet:
>>"This register includes bytes received in a packet from the <Destination Address> field through the <CRC> field, inclusively."
>
>Thanks I assumed I had known that at the time :-)
>>
>>-Harry
>>
>
>
>Regards,
>Keith
>
>
>
>
>


Regards,
Keith

Re: How can I calculate/estimate pps(packet per seocond) and bps(bit per second) in DPDK pktg

[최익성]

 Dear  Wiles, Keith ,  Van Haaren, Harry,  Polehn, Mike A,  Stephen Hemminger, Kyle Larose, and DPDK experts. 
 
I really appreciate for your precious answers and advices.
 
I will find and study the corresponding codes and CRC checking.
 
 
Last night, I tried to estimate bps and pps by using the following code.
 
 
// rte_distributor_process() gets 64 mbufs packets at a time.
// rte_distributor_process() gets packets from Intel® 82599ES 10 Gigabit Ethernet 2 port Controller (2 10gbE ports).
 
int  rte_distributor_process(struct rte_distributor *d, struct rte_mbuf **mbufs, unsigned num_mbufs)
{
        uint64_t ticks_per_ms = rte_get_tsc_hz()/1000 ;
        uint64_t ticks_per_s = rte_get_tsc_hz() ;
        uint64_t ticks_per_s_div_8 = rte_get_tsc_hz()/8 ;
        uint64_t cur_tsc = 0, last_tsc = 0, sum_len, bps, pps ;
 
        cur_tsc = rte_rdtsc();
 
        sum_len = 0 ; 
        for (l=0; l < num_mbufs; l++ ) { sum_len += mbufs[l]->pkt_len ; }
 
        if ((cur_tsc - last_tsc)!=0) { 
               bps = (sum_len * ticks_per_s_div_8 ) / (cur_tsc - last_tsc) ; 
               pps = num_mbufs * ticks_per_s / (cur_tsc - last_tsc) ; 
        } else bps = pps = 0 ;
 
        last_tsc = cur_tsc ;
}
 
I got  max. bit per second = 6,835,440,833 for 20 Gbps 1500 bytes packet traffic, and got max. bit per second = 6,808,524,220 for 2 Gbps 1500 bytes packet traffic.
 
I guess there can be packet burst, however the estimated value has too many errors.
 
I will try the methods you proposed. 
 
Thank you very much.
 
Sincerely Yours,
 
Ick-Sung Choi.

-----Original Message-----
From: "Polehn, Mike A"<mike.a.polehn@intel.com> 
To: "Wiles, Keith"<keith.wiles@intel.com>; "Van Haaren, Harry"<harry.van.haaren@intel.com>; "???"<pnk003@naver.com>; "dev@dpdk.org"<dev@dpdk.org>; 
Cc: 
Sent: 2015-11-04 (수) 00:59:34
Subject: RE: [dpdk-dev] How can I calculate/estimate pps(packet per seocond) and bps(bit per second) in DPDK pktg
 
I used the following code snip-it with the i40e device, with 1 second sample time had very high accuracy for IPv4 UDP packets:



#define FLOWD_PERF_PACKET_OVERHEAD 24  /* CRC + Preamble + SOF + Interpacket gap */

#define FLOWD_REF_NETWORK_SPEED   10e9



double Ave_Bytes_per_Packet, Data_Rate, Net_Rate;

uint64_t Bits;

uint64_t Bytes = pFlow->flow.n_bytes - pMatch_Prev->flow.n_bytes;

uint64_t Packets = pFlow->flow.n_packets - pMatch_Prev->flow.n_packets;

uint64_t Time_us = pFlow->flow.flow_time_us - pMatch_Prev->flow.flow_time_us;



if (Bytes == 0)

	Ave_Bytes_per_Packet = 0.0;

else

	Ave_Bytes_per_Packet = ((double)Bytes / (double)Packets) + 4.0;



Bits = (Bytes + (Packets*FLOWD_PERF_PACKET_OVERHEAD)) * 8;

if (Bits == 0)

	Data_Rate = 0.0;

else

	Data_Rate = (((double)Bits) / Time_us) * 1e6;



if (Data_Rate == 0.0)

	Net_Rate = 0.0;

else

	Net_Rate = Data_Rate / FLOWD_REF_NETWORK_SPEED;



For packet rate: double pk_rate = (((double)Packets)/ ((double)Time_us)) * 1e6;



To calculate elapsed time in DPDK app, used CPU counter (will not work if counter is being modified):



Initialization:

double flow_time_scale_us;

...

flow_time_scale_us = 1e6/rte_get_tsc_hz();



Elapsed time (uSec) example: 



elapse_us = (rte_rdtsc() - entry->tsc_first_packet) *

	flow_time_scale_us; /* calc total elapsed us */



-----Original Message-----

From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Wiles, Keith

Sent: Tuesday, November 3, 2015 6:33 AM

To: Van Haaren, Harry; ???; dev@dpdk.org

Subject: Re: [dpdk-dev] How can I calculate/estimate pps(packet per seocond) and bps(bit per second) in DPDK pktg



On 11/3/15, 8:30 AM, "Van Haaren, Harry" <harry.van.haaren@intel.com> wrote:



>Hi Keith,

>

>> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Wiles, Keith

><snip>

>> Hmm, I just noticed I did not include the FCS bytes. Does the NIC 

>> include FCS bytes in the counters? Need to verify that one and if not then it becomes a bit more complex.

>

>The Intel NICs count packet sizes inclusive of CRC / FCS, from eg the ixgbe/82599 datasheet:

>"This register includes bytes received in a packet from the <Destination Address> field through the <CRC> field, inclusively."



Thanks I assumed I had known that at the time :-)

>

>-Harry

>





Regards,

Keith

Re: How can I calculate/estimate pps(packet per seocond) and bps(bit per second) in DPDK pktg

[Polehn, Mike A]

The change in tsc value from rte_rdtsc() needs to be multiplied by the scale to convert from clocks to get change in seconds.
For example from below:

elapse_us = (rte_rdtsc() - entry->tsc_first_packet) * flow_time_scale_us;

The bit rate requires the number of bytes passed in the time period then adjusted by the overhead of the number of packets transferred in the time period.

#define FLOWD_PERF_PACKET_OVERHEAD 24 /* CRC + Preamble + SOF + Interpacket gap */

Bits = (Bytes + (Packets*FLOWD_PERF_PACKET_OVERHEAD)) * 8;
Data_Rate = (((double)Bits) / Time_us) * 1e6;

Integer math is very tricky and often is not any faster than floating point math when using multiplies except on the very low performance processors.

Mike

From: 최익성 [mailto:pnk003@naver.com]
Sent: Tuesday, November 3, 2015 5:45 PM
To: Polehn, Mike A; Wiles, Keith; Van Haaren, Harry; dev@dpdk.org
Subject: RE: [dpdk-dev] How can I calculate/estimate pps(packet per seocond) and bps(bit per second) in DPDK pktg


Dear  Wiles, Keith ,  Van Haaren, Harry,  Polehn, Mike A,  Stephen Hemminger, Kyle Larose, and DPDK experts.



I really appreciate for your precious answers and advices.



I will find and study the corresponding codes and CRC checking.





Last night, I tried to estimate bps and pps by using the following code.





// rte_distributor_process() gets 64 mbufs packets at a time.

// rte_distributor_process() gets packets from Intel® 82599ES 10 Gigabit Ethernet 2 port Controller (2 10gbE ports).



int  rte_distributor_process(struct rte_distributor *d, struct rte_mbuf **mbufs, unsigned num_mbufs)

{

        uint64_t ticks_per_ms = rte_get_tsc_hz()/1000 ;

        uint64_t ticks_per_s = rte_get_tsc_hz() ;

        uint64_t ticks_per_s_div_8 = rte_get_tsc_hz()/8 ;

        uint64_t cur_tsc = 0, last_tsc = 0, sum_len, bps, pps ;



        cur_tsc = rte_rdtsc();



        sum_len = 0 ;

        for (l=0; l < num_mbufs; l++ ) { sum_len += mbufs[l]->pkt_len ; }



        if ((cur_tsc - last_tsc)!=0) {

               bps = (sum_len * ticks_per_s_div_8 ) / (cur_tsc - last_tsc) ;

               pps = num_mbufs * ticks_per_s / (cur_tsc - last_tsc) ;

        } else bps = pps = 0 ;



        last_tsc = cur_tsc ;

}



I got  max. bit per second = 6,835,440,833 for 20 Gbps 1500 bytes packet traffic, and got max. bit per second = 6,808,524,220 for 2 Gbps 1500 bytes packet traffic.



I guess there can be packet burst, however the estimated value has too many errors.



I will try the methods you proposed.



Thank you very much.



Sincerely Yours,



Ick-Sung Choi.


-----Original Message-----
From: "Polehn, Mike A"<mike.a.polehn@intel.com<mailto:mike.a.polehn@intel.com>>
To: "Wiles, Keith"<keith.wiles@intel.com<mailto:keith.wiles@intel.com>>; "Van Haaren, Harry"<harry.van.haaren@intel.com<mailto:harry.van.haaren@intel.com>>; "???"<pnk003@naver.com<mailto:pnk003@naver.com>>; "dev@dpdk.org<mailto:dev@dpdk.org>"<dev@dpdk.org<mailto:dev@dpdk.org>>;
Cc:
Sent: 2015-11-04 (수) 00:59:34
Subject: RE: [dpdk-dev] How can I calculate/estimate pps(packet per seocond) and bps(bit per second) in DPDK pktg

I used the following code snip-it with the i40e device, with 1 second sample time had very high accuracy for IPv4 UDP packets:

#define FLOWD_PERF_PACKET_OVERHEAD 24 /* CRC + Preamble + SOF + Interpacket gap */
#define FLOWD_REF_NETWORK_SPEED 10e9

double Ave_Bytes_per_Packet, Data_Rate, Net_Rate;
uint64_t Bits;
uint64_t Bytes = pFlow->flow.n_bytes - pMatch_Prev->flow.n_bytes;
uint64_t Packets = pFlow->flow.n_packets - pMatch_Prev->flow.n_packets;
uint64_t Time_us = pFlow->flow.flow_time_us - pMatch_Prev->flow.flow_time_us;

if (Bytes == 0)
Ave_Bytes_per_Packet = 0.0;
else
Ave_Bytes_per_Packet = ((double)Bytes / (double)Packets) + 4.0;

Bits = (Bytes + (Packets*FLOWD_PERF_PACKET_OVERHEAD)) * 8;
if (Bits == 0)
Data_Rate = 0.0;
else
Data_Rate = (((double)Bits) / Time_us) * 1e6;

if (Data_Rate == 0.0)
Net_Rate = 0.0;
else
Net_Rate = Data_Rate / FLOWD_REF_NETWORK_SPEED;

For packet rate: double pk_rate = (((double)Packets)/ ((double)Time_us)) * 1e6;

To calculate elapsed time in DPDK app, used CPU counter (will not work if counter is being modified):

Initialization:
double flow_time_scale_us;
...
flow_time_scale_us = 1e6/rte_get_tsc_hz();

Elapsed time (uSec) example:

elapse_us = (rte_rdtsc() - entry->tsc_first_packet) *
flow_time_scale_us; /* calc total elapsed us */

-----Original Message-----
From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Wiles, Keith
Sent: Tuesday, November 3, 2015 6:33 AM
To: Van Haaren, Harry; ???; dev@dpdk.org<mailto:dev@dpdk.org>
Subject: Re: [dpdk-dev] How can I calculate/estimate pps(packet per seocond) and bps(bit per second) in DPDK pktg

On 11/3/15, 8:30 AM, "Van Haaren, Harry" <harry.van.haaren@intel.com<mailto:harry.van.haaren@intel.com>> wrote:

>Hi Keith,
>
>> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Wiles, Keith
><snip>
>> Hmm, I just noticed I did not include the FCS bytes. Does the NIC
>> include FCS bytes in the counters? Need to verify that one and if not then it becomes a bit more complex.
>
>The Intel NICs count packet sizes inclusive of CRC / FCS, from eg the ixgbe/82599 datasheet:
>"This register includes bytes received in a packet from the <Destination Address> field through the <CRC> field, inclusively."

Thanks I assumed I had known that at the time :-)
>
>-Harry
>


Regards,
Keith



[http://mail.naver.com/readReceipt/notify/?img=b9KqKAIOWNISaAMrFomsaAtmF6uqpxUqFAICpAEXKrMXMxUqF430K6t9tzFXp6UmFLl5WLl51zlqDBFdp6d5MreRhoRR16R074u5bzenp6C5tz05%2Bzkn74FTWt%3D%3D.gif]

Re: How can I calculate/estimate pps(packet per seocond) and bps(bit per second) in DPDK pktg

[최익성]

 Dear Polehn, Mike A and DPDK experts. 
 
Thank you very much for your previous advice and answer.
 
I used the method you proposed.
 
I dont't understand that the data rate includes preamble and inter-packet gap since the preamble and the inter-packet gap are not packet data.
 
There are significant packet bursts in DPDK packet stream.
 
I used average value of gps/pps history (128 history).
 
It works well.
 
Thank you very much.
 
Sincerely Yours,
 
Ick-Sung Choi.
 
 
-----Original Message-----
From: "Polehn, Mike A"<mike.a.polehn@intel.com> 
To: "???"<pnk003@naver.com>; "Wiles, Keith"<keith.wiles@intel.com>; "Van Haaren, Harry"<harry.van.haaren@intel.com>; "dev@dpdk.org"<dev@dpdk.org>; 
Cc: 
Sent: 2015-11-04 (수) 23:21:54
Subject: RE: [dpdk-dev] How can I calculate/estimate pps(packet per seocond) and bps(bit per second) in DPDK pktg
 








The change in tsc value from rte_rdtsc() needs to be multiplied by the scale to convert from clocks to get change in seconds.

For example from below:

 

elapse_us = (rte_rdtsc() - entry->tsc_first_packet) * flow_time_scale_us;

 

The bit rate requires the number of bytes passed in the time period then adjusted by the overhead of the number of packets transferred in the time period.

 

#define FLOWD_PERF_PACKET_OVERHEAD 24 /* CRC + Preamble + SOF + Interpacket gap */






Bits = (Bytes + (Packets*FLOWD_PERF_PACKET_OVERHEAD)) * 8;

Data_Rate = (((double)Bits) / Time_us) * 1e6;






Integer math is very tricky and often is not any faster than floating point math when using multiplies except on the very low performance processors.

 

Mike

 



From:
최익성 [mailto:pnk003@naver.com]


Sent: Tuesday, November 3, 2015 5:45 PM

To: Polehn, Mike A; Wiles, Keith; Van Haaren, Harry; dev@dpdk.org

Subject: RE: [dpdk-dev] How can I calculate/estimate pps(packet per seocond) and bps(bit per second) in DPDK pktg



 


Dear  Wiles, Keith ,  Van Haaren, Harry,  Polehn, Mike A,  Stephen Hemminger, Kyle Larose, and DPDK experts. 

 

I really appreciate for your precious answers and advices.

 

I will find and study the corresponding codes and CRC checking.

 

 

Last night, I tried to estimate bps and pps by using the following code.

 

 

// rte_distributor_process() gets 64 mbufs packets at a time.

// rte_distributor_process() gets packets from Intel® 82599ES 10 Gigabit Ethernet 2 port Controller (2 10gbE ports).

 

int  rte_distributor_process(struct rte_distributor *d, struct rte_mbuf **mbufs, unsigned num_mbufs)

{

        uint64_t ticks_per_ms = rte_get_tsc_hz()/1000 ;

        uint64_t ticks_per_s = rte_get_tsc_hz() ;

        uint64_t ticks_per_s_div_8 = rte_get_tsc_hz()/8 ;

        uint64_t cur_tsc = 0, last_tsc = 0, sum_len, bps, pps ;

 

        cur_tsc = rte_rdtsc();

 

        sum_len = 0 ; 

        for (l=0; l < num_mbufs; l++ ) { sum_len += mbufs[l]->pkt_len ; }

 

        if ((cur_tsc - last_tsc)!=0) { 

               bps = (sum_len * ticks_per_s_div_8 ) / (cur_tsc - last_tsc) ; 

               pps = num_mbufs * ticks_per_s / (cur_tsc - last_tsc) ; 

        } else bps = pps = 0 ;

 

        last_tsc = cur_tsc ;

}

 

I got  max. bit per second = 6,835,440,833 for 20 Gbps 1500 bytes packet traffic, and got max. bit per second = 6,808,524,220 for 2 Gbps 1500 bytes packet traffic.

 

I guess there can be packet burst, however the estimated value has too many errors.

 

I will try the methods you proposed. 

 

Thank you very much.

 

Sincerely Yours,

 

Ick-Sung Choi.


 


-----Original Message-----

From: "Polehn, Mike A"<mike.a.polehn@intel.com>


To: "Wiles, Keith"<keith.wiles@intel.com>; "Van Haaren, Harry"<harry.van.haaren@intel.com>; "???"<pnk003@naver.com>; "dev@dpdk.org"<dev@dpdk.org>;


Cc: 

Sent: 2015-11-04 (수) 00:59:34

Subject: RE: [dpdk-dev] How can I calculate/estimate pps(packet per seocond) and bps(bit per second) in DPDK pktg

 

I used the following code snip-it with the i40e device, with 1 second sample time had very high accuracy for IPv4 UDP packets:




#define FLOWD_PERF_PACKET_OVERHEAD 24 /* CRC + Preamble + SOF + Interpacket gap */


#define FLOWD_REF_NETWORK_SPEED 10e9 



double Ave_Bytes_per_Packet, Data_Rate, Net_Rate; 

uint64_t Bits; 

uint64_t Bytes = pFlow->flow.n_bytes - pMatch_Prev->flow.n_bytes; 

uint64_t Packets = pFlow->flow.n_packets - pMatch_Prev->flow.n_packets; 

uint64_t Time_us = pFlow->flow.flow_time_us - pMatch_Prev->flow.flow_time_us; 



if (Bytes == 0) 

Ave_Bytes_per_Packet = 0.0; 

else 

Ave_Bytes_per_Packet = ((double)Bytes / (double)Packets) + 4.0; 



Bits = (Bytes + (Packets*FLOWD_PERF_PACKET_OVERHEAD)) * 8; 

if (Bits == 0) 

Data_Rate = 0.0; 

else 

Data_Rate = (((double)Bits) / Time_us) * 1e6; 



if (Data_Rate == 0.0) 

Net_Rate = 0.0; 

else 

Net_Rate = Data_Rate / FLOWD_REF_NETWORK_SPEED; 



For packet rate: double pk_rate = (((double)Packets)/ ((double)Time_us)) * 1e6; 



To calculate elapsed time in DPDK app, used CPU counter (will not work if counter is being modified):




Initialization: 

double flow_time_scale_us; 

... 

flow_time_scale_us = 1e6/rte_get_tsc_hz(); 



Elapsed time (uSec) example: 



elapse_us = (rte_rdtsc() - entry->tsc_first_packet) * 

flow_time_scale_us; /* calc total elapsed us */ 



-----Original Message----- 

From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Wiles, Keith


Sent: Tuesday, November 3, 2015 6:33 AM 

To: Van Haaren, Harry; ???; dev@dpdk.org 

Subject: Re: [dpdk-dev] How can I calculate/estimate pps(packet per seocond) and bps(bit per second) in DPDK pktg




On 11/3/15, 8:30 AM, "Van Haaren, Harry" <harry.van.haaren@intel.com> wrote:




>Hi Keith, 

> 

>> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Wiles, Keith


><snip> 

>> Hmm, I just noticed I did not include the FCS bytes. Does the NIC 

>> include FCS bytes in the counters? Need to verify that one and if not then it becomes a bit more complex.


> 

>The Intel NICs count packet sizes inclusive of CRC / FCS, from eg the ixgbe/82599 datasheet:


>"This register includes bytes received in a packet from the <Destination Address> field through the <CRC> field, inclusively."




Thanks I assumed I had known that at the time :-) 

> 

>-Harry 

> 





Regards, 

Keith 
















 

Re: How can I calculate/estimate pps(packet per seocond) and bps(bit per second) in DPDK pktg

[Stephen Hemminger]

On Tue, 3 Nov 2015 14:30:41 +0000
"Van Haaren, Harry" <harry.van.haaren@intel.com> wrote:

> Hi Keith,
> 
> > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Wiles, Keith  
> <snip>
> > Hmm, I just noticed I did not include the FCS bytes. Does the NIC include FCS bytes in the
> > counters? Need to verify that one and if not then it becomes a bit more complex.  
> 
> The Intel NICs count packet sizes inclusive of CRC / FCS, from eg the ixgbe/82599 datasheet:
> "This register includes bytes received in a packet from the <Destination Address> field through the <CRC> field, inclusively."
> 
> -Harry

IMHO this is a bug. Other drivers don't include the CRC, and the Intel driver
only includes CRC in count for one direction, and depends on value of stripping flag.

I sent a patch to fix this because our customers didn't like it when Rx != Tx bytes
but there was somebody who liked including CRC.

It really is a Cisco versus the world thing. Juniper/Linux/BSD all do NOT include
CRC in counters and therefore that is what should be done.

Re: How can I calculate/estimate pps(packet per seocond) and bps(bit per second) in DPDK pktg

[Kyle Larose]

On Tue, Nov 3, 2015 at 5:05 PM, Stephen Hemminger
<stephen@networkplumber.org> wrote:

>
> IMHO this is a bug. Other drivers don't include the CRC, and the Intel driver
> only includes CRC in count for one direction, and depends on value of stripping flag.
>
> I sent a patch to fix this because our customers didn't like it when Rx != Tx bytes
> but there was somebody who liked including CRC.
>
> It really is a Cisco versus the world thing. Juniper/Linux/BSD all do NOT include
> CRC in counters and therefore that is what should be done.
>

Another option is to make whether or not the NIC counts the CRC in its
byte counters configurable, when supported, and also retrievable. I'm
concerned about the case where a NIC doesn't even have an option to
control whether or not it counts the CRC, and it *does* count it. In
that case, any software running on that NIC will behave
inconsistently. If it knew that it counted the CRC, it could adjust
for it.

If we put the option in now, then software written now could deal with
it gracefully. Combined with the ability to configure it, this may
satisfy use cases where knowing the full frame size is useful (for
example when looking at bit rates with small packets. 4 bytes is a big
difference for a 64-byte frame).

Of course, this may not be a problem worth solving. But, I figure it's
worth considering.

Re: How can I calculate/estimate pps(packet per seocond) and bps(bit per second) in DPDK pktg

[Stephen Hemminger]

On Tue, 3 Nov 2015 17:18:31 -0500
Kyle Larose <eomereadig@gmail.com> wrote:

> On Tue, Nov 3, 2015 at 5:05 PM, Stephen Hemminger
> <stephen@networkplumber.org> wrote:
> 
> >
> > IMHO this is a bug. Other drivers don't include the CRC, and the Intel driver
> > only includes CRC in count for one direction, and depends on value of stripping flag.
> >
> > I sent a patch to fix this because our customers didn't like it when Rx != Tx bytes
> > but there was somebody who liked including CRC.
> >
> > It really is a Cisco versus the world thing. Juniper/Linux/BSD all do NOT include
> > CRC in counters and therefore that is what should be done.
> >  
> 
> Another option is to make whether or not the NIC counts the CRC in its
> byte counters configurable, when supported, and also retrievable. I'm
> concerned about the case where a NIC doesn't even have an option to
> control whether or not it counts the CRC, and it *does* count it. In
> that case, any software running on that NIC will behave
> inconsistently. If it knew that it counted the CRC, it could adjust
> for it.

No. configuration is the enemy of usability.
Why does DPDK have to behave differently than BSD and Linux, what possible
value could this be to the end user?

> If we put the option in now, then software written now could deal with
> it gracefully. Combined with the ability to configure it, this may
> satisfy use cases where knowing the full frame size is useful (for
> example when looking at bit rates with small packets. 4 bytes is a big
> difference for a 64-byte frame).
> 
> Of course, this may not be a problem worth solving. But, I figure it's
> worth considering.

Re: How can I calculate/estimate pps(packet per seocond) and bps(bit per second) in DPDK pktg

[Kyle Larose]

On Tue, Nov 3, 2015 at 6:28 PM, Stephen Hemminger
<stephen@networkplumber.org> wrote:

>
> No. configuration is the enemy of usability.
> Why does DPDK have to behave differently than BSD and Linux, what possible
> value could this be to the end user?

I honestly can't think of any good reasons for why a user would need
this. I figured it'd be worth asking to see if anyone thought it would
be. I guess not. :P

I'm still concerned about NICs which may not support disabling CRC
counting, but there's no need to clutter the ethdev API with an extra
attribute without a clear example of such a NIC.