Re: [patch net-next v5 0/4] return offloaded stats as default and expose original sw stats

[Roopa Prabhu <roopa@cumulusnetworks.com>]

On 6/22/16, 10:40 PM, Jiri Pirko wrote:
> Wed, Jun 22, 2016 at 09:32:25PM CEST, roopa@cumulusnetworks.com wrote:
>> On Tue, Jun 21, 2016 at 8:15 AM, Jiri Pirko <jiri@resnulli.us> wrote:
>>> From: Jiri Pirko <jiri@mellanox.com>
>>>
>>> The problem we try to handle is about offloaded forwarded packets
>>> which are not seen by kernel. Let me try to draw it:
>>>
>>>     port1                       port2 (HW stats are counted here)
>>>       \                          /
>>>        \                        /
>>>         \                      /
>>>          --(A)---- ASIC --(B)--
>>>                     |
>>>                    (C)
>>>                     |
>>>                    CPU (SW stats are counted here)
>>>
>>>
>>> Now we have couple of flows for TX and RX (direction does not matter here):
>>>
>>> 1) port1->A->ASIC->C->CPU
>>>
>>>    For this flow, HW and SW stats are equal.
>>>
>>> 2) port1->A->ASIC->C->CPU->C->ASIC->B->port2
>>>
>>>    For this flow, HW and SW stats are equal.
>>>
>>> 3) port1->A->ASIC->B->port2
>>>
>>>    For this flow, SW stats are 0.
>>>
>>> The purpose of this patchset is to provide facility for user to
>>> find out the difference between flows 1+2 and 3. In other words, user
>>> will be able to see the statistics for the slow-path (through kernel).
>>>
>>> Also note that HW stats are what someone calls "accumulated" stats.
>>> Every packet counted by SW is also counted by HW. Not the other way around.
>>>
>>> As a default the accumulated stats (HW) will be exposed to user
>>> so the userspace apps can react properly.
>>>
>>>
>> curious, how do you plan to handle virtual device counters like vlan
>> and vxlan stats ?.
> Yes, that is another problem (1). We have to push stats up to this devices
> most probably. But that problem is orthogonal to this. To the user, you
> will still need 2 sets of stats and HW stats being default. So this
> patchset infra is going to be used as well.
hmm...But, i don't think we should start adding different tlv's hw and sw for
every stats variant we add.
>
>
>> we can't separate CPU and HW stats there. In some cases (or ASICs) HW
>> counters do
>> not include CPU generated packets....you will have to add CPU
>> generated pkt counters to the
>> hw counters for such virtual device stats.
> Can you please provide and example how that could happen?

example is the bridge vlan stats I mention below. These are usually counted
by attaching hw virtual counter resources. And CPU generated packets
in some cases maybe setup to bypass the ASIC pipeline because the CPU
has already made the required decisions. So, they may not be counted by
by such hw virtual counters.

>
>
>> example: In the switchdev model, for bridge vlan stats, when user
>> queries bridge vlan stats,
>> you will have to add the hw stats to the bridge driver vlan stats and
>> return it to the user .
> Yep, that is (1).

unless i misunderstood, this does not look like (1). In (1) you say hw stats
 already reflect sw stats. But in this case, hw counter does not include sw stats
for CPU generated packets.
>
>
>> Having a consistent model for all kinds of stats will help.