Re: [PATCH 00/16] IB/hfi1: Add a page pinning cache for PSM sdma

[Leon Romanovsky <leon-2ukJVAZIZ/Y@public.gmane.org>]

On Thu, Mar 10, 2016 at 06:56:09AM +0200, Leon Romanovsky wrote:
> On Wed, Mar 09, 2016 at 02:21:10PM -0500, Dennis Dalessandro wrote:
> > On Wed, Mar 09, 2016 at 07:07:31AM +0200, Leon Romanovsky wrote:
> > >On Tue, Mar 08, 2016 at 07:21:58PM -0500, Dennis Dalessandro wrote:
> > >>On Tue, Mar 08, 2016 at 10:56:48PM +0200, Or Gerlitz wrote:
> > >>>On Tue, Mar 8, 2016 at 9:14 PM, Dennis Dalessandro
> > >>><dennis.dalessandro-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org> wrote:
> > >>>
> > >>>>This patch series adds a performance improvement to user SDMA transfers from
> > >>>>PSM applications by caching user buffer pages after pinning them. Subsequent
> > >>>>uses of the same user buffer will not incur the cost of pinning the same pages
> > >>>>again. Additionally, the cost of unpinning the same pages is avoided.  Buffer
> > >>>>pages are unpinned when the context is torn down or when the driver determines
> > >>>>that the buffer should be evicted from the cache. Cache evictions happen when
> > >>>>there is a request for a new, uncached buffer and the current size of the cache
> > >>>>has reached a pre-defined limit.
> > >>>
> > >>>If indeed there's a need for such a pin down cache, it should have
> > >>>been implemented in the IB core, b/c both the problem and the solution
> > >>>are generic and have nothing to do with certain device driver.
> > >>
> > >>Since this is for psm it has nothing to do with the IB core. I fail to see
> > >>what that would achieve.
> > >
> > >I tend to agree with Or, your proposal goes far beyond the driver code.
> > >Do you expect that all drivers which will be needed to support PSM will
> > >implement (copy/paste) the same generic logic?
> > 
> > We should not be putting something which is specific to PSM in the IB core.
> > We feel the driver is the most appropriate place for this.
> 
> To use cache logic - yes
> To implement cache logic - no
> 
> Can you share with us the technical justification of implementing general
> memory management cache mechanism in driver and not in appropriate layer
> (IB/core and/or MM)?

Hi Doug,
I saw that you picked these patches and it is now in your github
repository. Since the original author probably missed my and Or's questions, It
will be great if you can share with us your technical view on the topic.

In addition to my question regarding appropriate layer - IB/core. MM, I see that
it can be natively implemented in user-space layer too.

Thanks.

> 
> Thanks.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html