Re: [PATCH v3] Drivers: hv: vmbus: Expose counters for interrupts and full conditions

Stephen Hemminger
2019-01-17T17:11:15Z

> +static ssize_t channel_intr_in_full_show(const struct vmbus_channel
> *channel,
> + char *buf)
> +{
> + return sprintf(buf, "%llu\n", channel->intr_in_full);
> +}

intr_in_full is u64, which is not the same as unsigned long long.
to be correct you need a cast here.

> > diff --git a/include/linux/hyperv.h b/include/linux/hyperv.h
> > index dcb6977afce9..7e5239123276 100644
> > --- a/include/linux/hyperv.h
> > +++ b/include/linux/hyperv.h
> > @@ -751,6 +751,27 @@ struct vmbus_channel {
> > u64 interrupts; /* Host to Guest interrupts */
> > u64 sig_events; /* Guest to Host events */
> >
> > + /* Interrupt counts for 2 types of Guest to Host interrupts */
> > + u64 intr_in_full; /* in ring buffer, full to not full */
> > + u64 intr_out_empty; /* out ring buffer, empty to not empty */
> > +
> > + /*
> > + * The total number of write operations that encountered a full
> > + * outbound ring buffer.
> > + */
> > + u64 out_full_total;
> > + /*
> > + * The number of write operations that were the first to encounter a
> > + * full outbound ring buffer.
> > + */
> > + u64 out_full_first;

Adding more fields changes cache layout which can cause
additional cache miss in the hot path.

> > + /*
> > + * Indicates that a full outbound ring buffer was encountered. The flag
> > + * is set to true when a full outbound ring buffer is encountered and
> > + * set to false when a write to the outbound ring buffer is completed.
> > + */
> > + bool out_full_flag;

Discussion on kernel mailing list. Recommends against putting bool
in structures since that pads to full sizeof(int). Could this be
part of a bitfield?

> > /* Channel callback's invoked in softirq context */
> > struct tasklet_struct callback_event;
> > void (*onchannel_callback)(void *context);
> > @@ -936,6 +957,23 @@ static inline void *get_per_channel_state(struct
> > vmbus_channel *c)
> > static inline void set_channel_pending_send_size(struct vmbus_channel *c,
> > u32 size)
> > {
> > + unsigned long flags;
> > +
> > + spin_lock_irqsave(&c->outbound.ring_lock, flags);
> > +
> > + if (size) {
> > + ++c->out_full_total;
> > +
> > + if (!c->out_full_flag) {
> > + ++c->out_full_first;
> > + c->out_full_flag = true;
> > + }
> > + } else {
> > + c->out_full_flag = false;
> > + }
> > +
> > + spin_unlock_irqrestore(&c->outbound.ring_lock, flags);

If this is called often, the additional locking will impact performance.

> > c->outbound.ring_buffer->pending_send_sz = size;
> > }
> >

Could I propose another alternative.

It might be more useful to count the guest to host interaction events
rather than the ring buffer.

For example the number of calls to:
vmbus_set_event which means host exit call
vmbus_setevent fastpath using sync_set_bit
calls to rinbuffer_write that returned -EAGAIN

These would require less locking, reuse existing code paths
and not require additional state.