From mboxrd@z Thu Jan 1 00:00:00 1970 From: Shirley Ma Subject: Re: [PATCH v3 00/21] NFS/RDMA client patches for 3.17 Date: Wed, 16 Jul 2014 11:22:15 -0700 Message-ID: <53C6C2D7.2040500@oracle.com> References: <59034BAD-306F-462F-A274-3EB06C0CBE47@oracle.com> <53C69EC3.1010208@oracle.com> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: Sender: linux-nfs-owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org To: Chuck Lever Cc: linux-rdma , Linux NFS Mailing List List-Id: linux-rdma@vger.kernel.org On 07/16/2014 10:57 AM, Chuck Lever wrote: > >> On Jul 16, 2014, at 11:48 AM, Shirley Ma wrote: >> >> These two patches have been significantly reduced interrupt rate by around 4 times. >> >> xprtrdma: Disable completions for FAST_REG_MR Work Requests >> xprtrdma: Disable completions for LOCAL_INV Work Requests > > Thanks Shirley! This is result applies only to FRMR, correct? Also, i'd imagine the savings would be even greater for adapters that have short page list depth? Yes, only tested FRMR with mlx4. I can hack the code to test short page page list depth to check the savings. When looking the difference between irq and softirq, it is much closer now. > >> >> Same NFS read/write workload, here are interrupts rate irq/per sec report based upon /proc/interrupts: >> >> w/o patches: >> ----------- >> PCI-MSI-edge mlx4-ib (204): 105176 >> PCI-MSI-edge mlx4-ib (204): 123650 >> PCI-MSI-edge mlx4-ib (204): 123690 >> PCI-MSI-edge mlx4-ib (204): 116554 >> PCI-MSI-edge mlx4-ib (204): 122864 >> >> And perf stat irq report: >> Performance counter stats for 'system wide': >> >> 2,131,870 irq:irq_handler_entry [100.00%] >> 2,131,870 irq:irq_handler_exit [100.00%] >> 635,587 irq:softirq_entry [100.00%] >> 635,597 irq:softirq_exit [100.00%] >> 636,155 irq:softirq_raise >> >> 25.422821792 seconds time elapsed >> >> w/i patches: >> ----------- >> PCI-MSI-edge mlx4-ib (204): 31131 >> PCI-MSI-edge mlx4-ib (204): 32958 >> PCI-MSI-edge mlx4-ib (204): 31068 >> PCI-MSI-edge mlx4-ib (204): 30236 >> PCI-MSI-edge mlx4-ib (204): 33041 >> >> And perf stat irq report: >> >> Performance counter stats for 'system wide': >> >> 653,548 irq:irq_handler_entry [100.00%] >> 653,548 irq:irq_handler_exit [100.00%] >> 568,138 irq:softirq_entry [100.00%] >> 568,148 irq:softirq_exit [100.00%] >> 568,690 irq:softirq_raise >> >> 21.675597062 seconds time elapsed >> >> Shirley >> >> -- >> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in >> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org >> More majordomo info at http://vger.kernel.org/majordomo-info.html > -- > To unsubscribe from this list: send the line "unsubscribe linux-rdma" in > the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > -- To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org More majordomo info at http://vger.kernel.org/majordomo-info.html