Re: [PATCH] ib_ipoib: Scatter-Gather support in connected mode

[Doug Ledford <dledford-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>]

[-- Attachment #1: Type: text/plain, Size: 2495 bytes --]

On Wed, 2015-04-01 at 23:33 +0300, Yuval Shaia wrote:
> On Mon, Mar 23, 2015 at 11:17:49AM -0600, Jason Gunthorpe wrote:
> > On Sun, Mar 22, 2015 at 11:21:50AM +0200, Yuval Shaia wrote:
> > > On Sun, Mar 15, 2015 at 05:16:16PM +0200, Yuval Shaia wrote:
> > > > Hi,
> > > > I didn't got any further comments on this one.
> > > > Any idea why SG in CM is un-welcome?
> > > By mistake I sent a private mail only.
> > > Cc: Roland Dreier <roland-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
> > > Cc: Sean Hefty <sean.hefty-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
> > > Cc: Hal Rosenstock <hal.rosenstock-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
> > > 
> > > Your advice would be very appreciated.
> > 
> > I haven't looked in detail at the patch, but in principle, using S/G
> > when ever possible should be the default, even if this creates a
> > performance regression.
> Per my testings - no impact on performance.
> > 
> > It is well known that high order allocations are problematic in Linux
> > and should be avoided, and I also have seen systems blow up because of
> > high order IPoIB allocations.
> Yes, snapshot from oops:
> Call Trace:
>  [<ffffffff810ddf74>] __alloc_pages_nodemask+0x524/0x595
>  [<ffffffff8110da3f>] kmem_getpages+0x4f/0xf4
>  [<ffffffff8110dc12>] fallback_alloc+0x12e/0x1ce
>  [<ffffffff8110ddd3>] ____cache_alloc_node+0x121/0x134
>  [<ffffffff8110e3f3>] kmem_cache_alloc_node_notrace+0x84/0xb9
>  [<ffffffff8110e46e>] __kmalloc_node+0x46/0x73
>  [<ffffffff813b9aa8>] ? __alloc_skb+0x72/0x13d
> > 
> > That said, there may be cases where S/G is not possible, you should
> > try and get Mellanox to comment if all their offloads work on all
> > their cards when S/G is used. Work may be required to resolve any of
> > these constraints. I'd like to belive there is some reason why we've
> > been doing high order allocations for so many years.
> Old kernels (2.6.39) restrict the usage of S/G only to H/W which supports CSUM offload.
> Mellanox HCA supports CSUM offload only for UD and this is why we can find S/G support for UD but not for CM.
> With the introduction of commit ec5f061 this limitation is no longer there so no reason not to add this support.

Can you repost the patch with a cleaned up commit message that includes
a reference to this prior restriction and the fact that it no longer
exists please?

-- 
Doug Ledford <dledford-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
              GPG KeyID: 0E572FDD


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 819 bytes --]