Re: [PATCH v4 0/4] Sending kernel pathrecord query to user cache server

[Hal Rosenstock <hal-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>]

On 6/9/2015 10:57 AM, kaike.wan-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org wrote:
> From: Kaike Wan <kaike.wan-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
> 
> A SA cache is undeniably critical for fabric scalability and performance.
> In user space, the ibacm application provides a good example of pathrecord
> cache for address and route resolution. With the recent implementation of
> the provider architecture, ibacm offers more extensibility as a SA cache.
> In kernel, ipoib implements its own small cache for pathrecords, which is
> however not available for general use. Furthermore, the implementation of
> a SA cache in user space offers better flexibility, larger capacity, and
> more robustness for the system.
> 
> In this patch series, a mechanism is implemented to allow ib_sa to
> send pathrecord query to a user application (eg ibacm) through netlink.

While this appears to address the current upstream use model for ACM
with it's multicast overlay backend where PRs are static, it does not
appear to address PR changes.

Should aging/revalidation of PRs be supported ? If so, would this make
PRs similar at "high" level to IP neighbor cache in kernel ?

-- Hal

> Potentially, this mechanism could be easily extended to other SA queries.
> 
> With a customized test implemented in rdma_cm module (not included in this
> series), it was shown that the time to retrieve 1 million pathrecords
> dropped from 46660 jiffies (46.66 seconds) to 16119 jiffies (or 16.119
> seconds) on a two-node system, a reduction of more than 60%.
> 
> Changes since v3:
> - Patch 1: added basic RESOLVE attribute types.
> - Patch 4: changed the encoding of the RESOLVE request message based on
>   the new attribute types and the input comp_mask. Changed the response
>   handling by iterating all attributes.
> 
> Changes since v2:
> - Redesigned the communication protocol between the kernel and user space
>   application. Instead of the MAD packet format, the new protocol uses 
>   netlink message header and attributes to exchange request and 
>   response between the kernel and user space.The design was described
>   here:
>   http://www.spinics.net/lists/linux-rdma/msg25621.html
> 
> Changes since v1:
> - Move kzalloc changes into a separate patch (Patch 3).
> - Remove redundant include line (Patch 4). 
> - Rename struct rdma_nl_resp_msg as structure ib_nl_resp_msg (Patch 4).
> 
> Kaike Wan (4):
>   IB/netlink: Add defines for local service requests through netlink
>   IB/core: Check the presence of netlink multicast group listeners
>   IB/sa: Allocate SA query with kzalloc
>   IB/sa: Route SA pathrecord query through netlink
> 
>  drivers/infiniband/core/netlink.c  |    8 +
>  drivers/infiniband/core/sa_query.c |  523 +++++++++++++++++++++++++++++++++++-
>  include/rdma/rdma_netlink.h        |    7 +
>  include/uapi/rdma/rdma_netlink.h   |   82 ++++++
>  4 files changed, 615 insertions(+), 5 deletions(-)
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html