Re: About a shortcoming of the verbs API

[Roland Dreier <rdreier-FYB4Gu1CFyUAvxtiuMwx3w@public.gmane.org>]

 > 2. Double completion processing loop
 > 
 > * Initialization:
 > ib_req_notify_cq(cq, IB_CQ_NEXT_COMP);
 > 
 > * Notification handler:
 > 
 > struct ib_wc wc;
 > do {
 >     while (ib_poll_cq(cq, 1, &wc) > 0)
 >         /* process wc */
 > } while (ib_req_notify_cq(cq, IB_CQ_NEXT_COMP |
 > IB_CQ_REPORT_MISSED_EVENTS) > 0);

This approach can be used to have race-free in-order processing of
completions using a scheme such as the NAPI processing loop used by the
IPoIB driver (with help from the core networking stack).  Essentially a
completion notification just marks the completion processing routine as
runnable, and the networking core schedules that processing routine in a
single-threaded way until the CQ is drained.

Another approach is to just always run the completion processing for a
given CQ on a single CPU and avoid locking entirely.  If you want more
CPUs to spread the work, just use multiple CQs and multiple event vectors.

 > see e.g. VipCQNotify() in the Virtual Interface Architecture
 > Specification.

I don't know of an efficient way to implement this type of "atomic
dequeue completion or enable completions" with any existing hardware.
Do you have an idea how this could be done?

 - R.
-- 
Roland Dreier <rolandd-FYB4Gu1CFyUAvxtiuMwx3w@public.gmane.org> || For corporate legal information go to:
http://www.cisco.com/web/about/doing_business/legal/cri/index.html
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html