From mboxrd@z Thu Jan 1 00:00:00 1970 From: Sagi Grimberg Subject: Re: [PATCH 7/8] xprtrdma: Split the completion queue Date: Thu, 17 Apr 2014 22:08:24 +0300 Message-ID: <535026A8.4040906@dev.mellanox.co.il> References: <20140414220041.20646.63991.stgit@manet.1015granger.net> <20140414222323.20646.66946.stgit@manet.1015granger.net> <534E7C1C.5070407@dev.mellanox.co.il> <534E8608.8030801@opengridcomputing.com> <534EA06A.7090200@dev.mellanox.co.il> <534F7D5F.1090908@dev.mellanox.co.il> Mime-Version: 1.0 Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: QUOTED-PRINTABLE Return-path: In-Reply-To: Sender: linux-nfs-owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org To: Chuck Lever Cc: Steve Wise , Linux NFS Mailing List , linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org List-Id: linux-rdma@vger.kernel.org On 4/17/2014 4:55 PM, Chuck Lever wrote: > On Apr 17, 2014, at 3:06 AM, Sagi Grimberg = wrote: > >> On 4/16/2014 9:21 PM, Chuck Lever wrote: >>> Passing a small array to ip_poll_cq() is actually easy to do, and i= s >>> exactly equivalent to a poll budget. The struct ib_wc should be tak= en >>> off the stack anyway, IMO. >>> >>> The only other example I see in 3.15 right now is IPoIB, which seem= s >>> to do exactly this. >>> >>> I=92m testing a patch now. I=92d like to start simple and make it m= ore >>> complex only if we need to. >> What array size are you using? Note that if you use a small array it= may be an overkill since >> a lot more interrupts are invoked (-> more latency). I found that fo= r a high workload a budget >> of 256/512/1024 keeps fairness and doesn't increase latency. > My array size is currently 4. It=92s a macro that can be changed easi= ly. > > By a very large majority, my workloads see only one WC per completion > upcall. However, I=92m using an older card with simple synthetic benc= hmarks. It doesn't matter until it does matter... We noticed this phenomenon under *extreme* stress. > I don=92t want to make the array large because struct ib_wc is at lea= st > 64 bytes on my systems =97 each WC array would be enormous and hardly= ever > used. Right, large WC array would be annoying... Sagi. -- To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org More majordomo info at http://vger.kernel.org/majordomo-info.html