Tx Queue Rate Limiting

Tx Queue Rate Limiting

[Dan Brennan]

Hello,

I've been scouring through the code and documentation, experimenting 
with various register settings, etc., for some time now and I'm still 
stuck on a problem regarding tx queue rate limiting.

I have a simple packet processing application that uses three cores 
similar to qos_sched (Rx, Worker, Tx). I am running on an E5-1620 v2, 
using an 82599EB based NIC for the network interface. I am using DPDK 
1.7.1 on Debian 7 (3.2.46 kernel).

I have modified the testpmd application to utilize multiple tx queues in 
both iofwd and flowgen modes. When I start the forwarder, I observe each 
queue sending packets equally. I can issue the following command to 
alter the rate of each queue individually:

     set port N queue X rate NEWRATE

This works well, as I have observed the updated queue transmitting fewer 
(or more) packets on output (I also have measured the proportional 
change in total bit rate using a Smartbits analyzer).

The problem is that my application doesn't behave the same. I have 32 Tx 
queues. However, when I change the rate on one, it appears to affect the 
entire interface, instead of the individual queue. I looked at both port 
configs (testpmd and my application) and can't locate the discrepancy. I 
even went so far as to port the testpmd port init code to my application 
with the same results.

I have instrumented the ixgbe driver to dump registers whenever I update 
a Tx queue, and have verified all register settings are the same when I 
perform this operation in testpmd and my application. The only 
difference is that q-stats for testpmd reflect the correct change in Tx 
queue rate for the individual queues, while q-stats in my application 
indicate all Tx queues change to the same rate.

Has anyone experienced similar behavior or have the ability to shed some 
light on what I could be doing wrong? Any assistance is appreciated.

Dan