[PATCH v4 0/4] Sending kernel pathrecord query to user cache server

kaike.wan-ral2JQCrhuEAvxtiuMwx3w
2015-06-09T14:57:13Z

From: Kaike Wan <kaike.wan-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>

A SA cache is undeniably critical for fabric scalability and performance.
In user space, the ibacm application provides a good example of pathrecord
cache for address and route resolution. With the recent implementation of
the provider architecture, ibacm offers more extensibility as a SA cache.
In kernel, ipoib implements its own small cache for pathrecords, which is
however not available for general use. Furthermore, the implementation of
a SA cache in user space offers better flexibility, larger capacity, and
more robustness for the system.

In this patch series, a mechanism is implemented to allow ib_sa to
send pathrecord query to a user application (eg ibacm) through netlink.
Potentially, this mechanism could be easily extended to other SA queries.

With a customized test implemented in rdma_cm module (not included in this
series), it was shown that the time to retrieve 1 million pathrecords
dropped from 46660 jiffies (46.66 seconds) to 16119 jiffies (or 16.119
seconds) on a two-node system, a reduction of more than 60%.

Changes since v3:
- Patch 1: added basic RESOLVE attribute types.
- Patch 4: changed the encoding of the RESOLVE request message based on
the new attribute types and the input comp_mask. Changed the response
handling by iterating all attributes.

Changes since v2:
- Redesigned the communication protocol between the kernel and user space
application. Instead of the MAD packet format, the new protocol uses
netlink message header and attributes to exchange request and
response between the kernel and user space.The design was described
here:
http://www.spinics.net/lists/linux-rdma/msg25621.html

Changes since v1:
- Move kzalloc changes into a separate patch (Patch 3).
- Remove redundant include line (Patch 4).
- Rename struct rdma_nl_resp_msg as structure ib_nl_resp_msg (Patch 4).

Kaike Wan (4):
IB/netlink: Add defines for local service requests through netlink
IB/core: Check the presence of netlink multicast group listeners
IB/sa: Allocate SA query with kzalloc
IB/sa: Route SA pathrecord query through netlink

drivers/infiniband/core/netlink.c | 8 +
drivers/infiniband/core/sa_query.c | 523 +++++++++++++++++++++++++++++++++++-
include/rdma/rdma_netlink.h | 7 +
include/uapi/rdma/rdma_netlink.h | 82 ++++++
4 files changed, 615 insertions(+), 5 deletions(-)