From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.3 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A2825C63697 for ; Sun, 22 Nov 2020 19:22:53 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5342D20789 for ; Sun, 22 Nov 2020 19:22:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728054AbgKVTWd (ORCPT ); Sun, 22 Nov 2020 14:22:33 -0500 Received: from gentwo.org ([3.19.106.255]:37442 "EHLO gentwo.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728103AbgKVTWb (ORCPT ); Sun, 22 Nov 2020 14:22:31 -0500 Received: by gentwo.org (Postfix, from userid 1002) id 3F1583F084; Sun, 22 Nov 2020 19:22:30 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by gentwo.org (Postfix) with ESMTP id 3C70D3E9FA; Sun, 22 Nov 2020 19:22:30 +0000 (UTC) Date: Sun, 22 Nov 2020 19:22:30 +0000 (UTC) From: Christopher Lameter X-X-Sender: cl@www.lameter.com To: =?ISO-8859-15?Q?H=E5kon_Bugge?= cc: Jason Gunthorpe , Mark Haywood , OFED mailing list Subject: Re: Is there a working cache for path record and lids etc for librdmacm? In-Reply-To: <801AE4A1-7AE8-4756-8F32-5F3BFD189E2B@oracle.com> Message-ID: References: <20201117193329.GH244516@ziepe.ca> <6F632AE0-7921-4C5F-8455-F8E9390BD071@oracle.com> <801AE4A1-7AE8-4756-8F32-5F3BFD189E2B@oracle.com> User-Agent: Alpine 2.22 (DEB 394 2020-01-19) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="531401748-2007141671-1606072950=:265127" Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. --531401748-2007141671-1606072950=:265127 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8BIT On Sun, 22 Nov 2020, Håkon Bugge wrote: > > The app that we have runs in user space. Can it use the cache? Is the > > cache only in Mellanox OFED? I heard that it was removed. > > An app in user space can use the ibacm cache. If you use the default > configuration that comes with rdma-core, both address and route > resolution will be from librdmacm directly to ibacm, i.e., no kernel > involved. The ibacm options are by default installed in > /etc/rdma/ibacm_opts.cfg I have been using that. > If you set acme_plus_kernel_only to one in said config file, you app will resolve the address using the kernel neighbour cache and the route resolution will go into the kernel and then "bounce" back to user space and ibacm through NetLink. Have not seen that in the RHEL7.8 version of ibacm. > > This is an an option while building ibacm? > > Nop, runtime config option as depicted above. Must be a newer version then. > The default provided by rdma-core should work, possible requiring the option above. The one in RHEL7 will never resolve anything through the subnet manger. Evey request results here in a leftover kernel thread hanging around. Which version of ibacm do you run? --531401748-2007141671-1606072950=:265127--