RE: [PATCH v1 net-next 2/2] net: ena: Extend customer metrics reporting support

["Arinzon, David" <darinzon@amazon.com>]

> > > I've looked into the definition of the metrics under question
> > >
> > > Based on AWS documentation
> > > (https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/monitoring-
> > networ
> > > k-performance-ena.html)
> > >
> > > bw_in_allowance_exceeded: The number of packets queued or dropped
> > because the inbound aggregate bandwidth exceeded the maximum for the
> > instance.
> > > bw_out_allowance_exceeded: The number of packets queued or
> dropped
> > because the outbound aggregate bandwidth exceeded the maximum for
> the
> > instance.
> > >
> > > Based on the netlink spec
> > > (https://docs.kernel.org/next/networking/netlink_spec/netdev.html)
> > >
> > > rx-hw-drop-ratelimits (uint)
> > > doc: Number of the packets dropped by the device due to the received
> > packets bitrate exceeding the device rate limit.
> > > tx-hw-drop-ratelimits (uint)
> > > doc: Number of the packets dropped by the device due to the transmit
> > packets bitrate exceeding the device rate limit.
> > >
> > > The AWS metrics are counting for packets dropped or queued (delayed,
> > > but
> > are sent/received with a delay), a change in these metrics is an
> > indication to customers to check their applications and workloads due
> > to risk of exceeding limits.
> > > There's no distinction between dropped and queued in these metrics,
> > therefore, they do not match the ratelimits in the netlink spec.
> > > In case there will be a separation of these metrics in the future to
> > > dropped
> > and queued, we'll be able to add the support for hw-drop-ratelimits.
> >
> > Xuan, Michael, the virtio spec calls out drops due to b/w limit being
> > exceeded, but AWS people say their NICs also count packets buffered
> > but not dropped towards a similar metric.
> >
> > I presume the virtio spec is supposed to cover the same use cases.
> On tx side, number of packets may not be queued, but may not be even
> DMAed if the rate has exceeded.
> This is hw nic implementation detail and a choice with trade-offs.
> 
> Similarly on rx, one may implement drop or queue or both (queue upto some
> limit, and drop beyond it).
> 
> > Have the stats been approved?
> Yes. it is approved last year; I have also reviewed it; It is part of the spec
> nearly 10 months ago at [1].
> GH PR is merged but GH is not updated yet.
> 
> [1] https://github.com/oasis-tcs/virtio-
> spec/commit/42f389989823039724f95bbbd243291ab0064f82
> 
> > Is it reasonable to extend the definition of the "exceeded" stats in
> > the virtio spec to cover what AWS specifies?
> Virtio may add new stats for exceeded stats in future.
> But I do not understand how AWS ENA nic is related to virtio PCI HW nic.
> 
> Should virtio implement it? may be yes. Looks useful to me.
> Should it be now in virtio spec, not sure, this depends on virtio community
> and actual hw/sw supporting it.
> 
> > Looks like PR is still open:
> > https://github.com/oasis-tcs/virtio-spec/issues/180
> Spec already has it at [1] for drops. GH PR is not upto date.

Thank you for the reply, Parav.
I've raised the query and the summary of this discussion in the above mentioned github ticket.