Re: [PATCH rdma-core v7 4/6] pyverbs: Add dma-buf based MR support

From: Jason Gunthorpe <jgg@ziepe.ca>
To: Daniel Vetter <daniel@ffwll.ch>
Cc: Leon Romanovsky <leon@kernel.org>,
	Gal Pressman <galpress@amazon.com>,
	Jianxin Xiong <jianxin.xiong@intel.com>,
	Yishai Hadas <yishaih@nvidia.com>,
	linux-rdma <linux-rdma@vger.kernel.org>,
	Edward Srouji <edwards@nvidia.com>,
	dri-devel <dri-devel@lists.freedesktop.org>,
	Christian Koenig <christian.koenig@amd.com>,
	Doug Ledford <dledford@redhat.com>,
	Daniel Vetter <daniel.vetter@intel.com>
Subject: Re: [PATCH rdma-core v7 4/6] pyverbs: Add dma-buf based MR support
Date: Mon, 1 Feb 2021 11:29:22 -0400	[thread overview]
Message-ID: <20210201152922.GC4718@ziepe.ca> (raw)
In-Reply-To: <CAKMK7uE0kSC1si0E9D1Spkn9aW2jFJw_SH3hYC6sZL7mG6pzyg@mail.gmail.com>

On Mon, Feb 01, 2021 at 03:10:00PM +0100, Daniel Vetter wrote:
> On Mon, Feb 1, 2021 at 7:16 AM Leon Romanovsky <leon@kernel.org> wrote:
> >
> > On Sun, Jan 31, 2021 at 05:31:16PM +0200, Gal Pressman wrote:
> > > On 25/01/2021 21:57, Jianxin Xiong wrote:
> > > > Define a new sub-class of 'MR' that uses dma-buf object for the memory
> > > > region. Define a new class 'DmaBuf' as a wrapper for dma-buf allocation
> > > > mechanism implemented in C.
> > > >
> > > > Update the cmake function for cython modules to allow building modules
> > > > with mixed cython and c source files.
> > > >
> > > > Signed-off-by: Jianxin Xiong <jianxin.xiong@intel.com>
> > > >  buildlib/pyverbs_functions.cmake |  78 +++++++----
> > > >  pyverbs/CMakeLists.txt           |  11 +-
> > > >  pyverbs/dmabuf.pxd               |  15 +++
> > > >  pyverbs/dmabuf.pyx               |  73 ++++++++++
> > > >  pyverbs/dmabuf_alloc.c           | 278 +++++++++++++++++++++++++++++++++++++++
> > > >  pyverbs/dmabuf_alloc.h           |  19 +++
> > > >  pyverbs/libibverbs.pxd           |   2 +
> > > >  pyverbs/mr.pxd                   |   6 +
> > > >  pyverbs/mr.pyx                   | 105 ++++++++++++++-
> > > >  9 files changed, 557 insertions(+), 30 deletions(-)
> > > >  create mode 100644 pyverbs/dmabuf.pxd
> > > >  create mode 100644 pyverbs/dmabuf.pyx
> > > >  create mode 100644 pyverbs/dmabuf_alloc.c
> > > >  create mode 100644 pyverbs/dmabuf_alloc.h
> >
> > <...>
> >
> > > > index 0000000..05eae75
> > > > +++ b/pyverbs/dmabuf_alloc.c
> > > > @@ -0,0 +1,278 @@
> > > > +// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
> > > > +/*
> > > > + * Copyright 2020 Intel Corporation. All rights reserved. See COPYING file
> > > > + */
> > > > +
> > > > +#include <stdio.h>
> > > > +#include <stdlib.h>
> > > > +#include <stdint.h>
> > > > +#include <unistd.h>
> > > > +#include <string.h>
> > > > +#include <errno.h>
> > > > +#include <drm/drm.h>
> > > > +#include <drm/i915_drm.h>
> > > > +#include <drm/amdgpu_drm.h>
> > > > +#include <drm/radeon_drm.h>
> > >
> > > I assume these should come from the kernel headers package, right?
> >
> > This is gross, all kernel headers should be placed in kernel-headers/*
> > and "update" script needs to be extended to take drm/* files too :(.
> 
> drm kernel headers are in the libdrm package. You need that anyway for
> doing the ioctls (if you don't hand-roll the restarting yourself).
> 
> Also our userspace has gone over to just outright copying the driver
> headers. Not the generic headers, but for the rendering side of gpus,
> which is the topic here, there's really not much generic stuff.
> 
> > Jianxin, are you fixing it?
> 
> So fix is either to depend upon libdrm for building, or have copies of
> the headers included in the package for the i915/amdgpu/radeon headers
> (drm/drm.h probably not so good idea).

We should have a cmake test to not build the drm parts if it can't be
built, and pyverbs should skip the tests.

Jason