From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 29E00C433DB for ; Thu, 4 Feb 2021 07:49:40 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id DE52364F53 for ; Thu, 4 Feb 2021 07:49:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234332AbhBDHth (ORCPT ); Thu, 4 Feb 2021 02:49:37 -0500 Received: from hqnvemgate24.nvidia.com ([216.228.121.143]:13850 "EHLO hqnvemgate24.nvidia.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231941AbhBDHth (ORCPT ); Thu, 4 Feb 2021 02:49:37 -0500 Received: from hqmail.nvidia.com (Not Verified[216.228.121.13]) by hqnvemgate24.nvidia.com (using TLS: TLSv1.2, AES256-SHA) id ; Wed, 03 Feb 2021 23:48:57 -0800 Received: from [10.2.50.90] (172.20.145.6) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1473.3; Thu, 4 Feb 2021 07:48:48 +0000 Subject: Re: [PATCH v16 0/4] RDMA: Add dma-buf support To: Jianxin Xiong , , CC: Doug Ledford , Jason Gunthorpe , "Leon Romanovsky" , Sumit Semwal , Christian Koenig , Daniel Vetter References: <1608067636-98073-1-git-send-email-jianxin.xiong@intel.com> From: John Hubbard Message-ID: <5e4ac17d-1654-9abc-9a14-bda223d62866@nvidia.com> Date: Wed, 3 Feb 2021 23:48:48 -0800 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:85.0) Gecko/20100101 Thunderbird/85.0 MIME-Version: 1.0 In-Reply-To: <1608067636-98073-1-git-send-email-jianxin.xiong@intel.com> Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Originating-IP: [172.20.145.6] X-ClientProxiedBy: HQMAIL101.nvidia.com (172.20.187.10) To HQMAIL107.nvidia.com (172.20.187.13) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nvidia.com; s=n1; t=1612424937; bh=alcGj/A8SKeiAR0MFZN9AYw8+7za61R+PH3RiAuD15w=; h=Subject:To:CC:References:From:Message-ID:Date:User-Agent: MIME-Version:In-Reply-To:Content-Type:Content-Language: Content-Transfer-Encoding:X-Originating-IP:X-ClientProxiedBy; b=SVrbdc1CDRz+lmunAKbsH85fK2fsA61ndMm0kioS6EWP6r/VroL39DWB9UZJmBXZu HEyeEH3h/4ZvWUBJrDxGBSKWjRZyHVS99Qac0tr5M3BCdt0vEI3wvT5bpfPGxv3nxu AQe/CDiEzz9OlHO2w8cHCpGn4rzHBbu/FcvygoiC8EAgL2VToKxOiaRnZMBTT5qR0g ajYpFGIwUWfsqrwI/eKB4KWzJH75+GqoHUPGL3h87rdlQH8GUxtCvBsNkFuTouExRs G6VfnYU8lT8x1XX5vBTnOUhcpfEtLFv6JuXhd6OLeAvCXz7uwZqKzdvyli4sRn4REI daDg9OI+PRUHQ== Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org On 12/15/20 1:27 PM, Jianxin Xiong wrote: > This patch series adds dma-buf importer role to the RDMA driver in > attempt to support RDMA using device memory such as GPU VRAM. Dma-buf is > chosen for a few reasons: first, the API is relatively simple and allows > a lot of flexibility in implementing the buffer manipulation ops. > Second, it doesn't require page structure. Third, dma-buf is already > supported in many GPU drivers. However, we are aware that existing GPU > drivers don't allow pinning device memory via the dma-buf interface. > Pinning would simply cause the backing storage to migrate to system RAM. > True peer-to-peer access is only possible using dynamic attach, which > requires on-demand paging support from the NIC to work. For this reason, > this series only works with ODP capable NICs. Hi, Looking ahead to after this patchset is merged... Are there design thoughts out there, about the future of pinning to vidmem, for this? It would allow a huge group of older GPUs and NICs and such to do p2p with this approach, and it seems like a natural next step, right? thanks, -- John Hubbard NVIDIA