Re: [PATCH 2/9] IB: add a proper completion queue abstraction

Christoph Hellwig
2015-11-14T07:08:53Z

On Fri, Nov 13, 2015 at 11:25:13AM -0700, Jason Gunthorpe wrote:
> For instance, like this, not fulling draining the cq and then doing:
>
> > + completed = __ib_process_cq(cq, budget);
> > + if (completed < budget) {
> > + irq_poll_complete(&cq->iop);
> > + if (ib_req_notify_cq(cq, IB_POLL_FLAGS) > 0) {
>
> Doesn't seem entirely right? There is no point in calling
> ib_req_notify_cq if the code knows there is still stuff in the CQ and
> has already, independently, arranged for ib_poll_hander to be
> guarenteed called.

The code only calls ib_req_notify_cq if it knowns we finished earlier than
our budget.

> > + completed = __ib_process_cq(cq, IB_POLL_BUDGET_WORKQUEUE);
> > + if (completed >= IB_POLL_BUDGET_WORKQUEUE ||
> > + ib_req_notify_cq(cq, IB_POLL_FLAGS) > 0)
> > + queue_work(ib_comp_wq, &cq->work);
>
> Same comment here..

Same here - we only requeue the work item if either we processed all of
our budget, or ib_req_notify_cq with IB_CQ_REPORT_MISSED_EVENTS told
us that we need to poll again.

> I understand several drivers are not using a hard irq context for the
> comp_handler call back. Is there any way to exploit that in this new
> API so we don't have to do so many context switches? Ie if the driver
> already is using a softirq when calling comp_handler can we somehow
> just rig ib_poll_handler directly and avoid the overhead? (Future)

Let's say this API makes it possible. I still don't think moving the
whole budget and rearm logic into the LLD is necessarily a good idea
if we can avoid it.