From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752812AbcCUICB (ORCPT ); Mon, 21 Mar 2016 04:02:01 -0400 Received: from www.linutronix.de ([62.245.132.108]:42470 "EHLO Galois.linutronix.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750965AbcCUIB4 (ORCPT ); Mon, 21 Mar 2016 04:01:56 -0400 Date: Mon, 21 Mar 2016 09:00:12 +0100 (CET) From: Thomas Gleixner To: "Huang, Ying" cc: Ingo Molnar , Peter Zijlstra , Chris Mason , Peter Zijlstra , lkp@01.org, Sebastian Andrzej Siewior , Davidlohr Bueso , Hugh Dickins , LKML , Linus Torvalds , Mel Gorman , Mel Gorman , Darren Hart , Wu Fengguang , Xiaolong Ye Subject: Re: [LKP] [lkp] [futex] 65d8fc777f: +25.6% will-it-scale.per_process_ops In-Reply-To: <87io0gh78w.fsf@yhuang-dev.intel.com> Message-ID: References: <871t7vzze1.fsf@yhuang-dev.intel.com> <20160229093708.GA21037@gmail.com> <87d1qsgvpw.fsf@yhuang-dev.intel.com> <87io0gh78w.fsf@yhuang-dev.intel.com> User-Agent: Alpine 2.11 (DEB 23 2013-08-11) MIME-Version: 1.0 Content-Type: MULTIPART/MIXED; BOUNDARY="8323329-244806905-1458547213=:3978" X-Linutronix-Spam-Score: -1.0 X-Linutronix-Spam-Level: - X-Linutronix-Spam-Status: No , -1.0 points, 5.0 required, ALL_TRUSTED=-1,SHORTCIRCUIT=-0.0001 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. --8323329-244806905-1458547213=:3978 Content-Type: TEXT/PLAIN; charset=UTF-8 Content-Transfer-Encoding: 8BIT On Mon, 21 Mar 2016, Huang, Ying wrote: > > FYI, we noticed 25.6% performance improvement due to commit > > > > 65d8fc777f6d "futex: Remove requirement for lock_page() in get_futex_key()" > > > > in the will-it-scale.per_process_ops test. > > > > will-it-scale.per_process_ops tests the futex operations for process shared > > futexes (Or whatever that test really does). > > There is a futex sub test case for will-it-scale test suite. But I got your > point, we need some description for the test case. If email is not too > limited for the full description, we will put it in some web site and > include short description and link to the full description in email. Ok. Just make sure the short description gives enough information for the casual reader. > > The commit has no significant impact on any other test in the test suite. > > Sorry, we have no enough machine power to test all test cases for each > bisect result. So we will have no such information until we find a way > to do that. Well, then I really have to ask how I should interpret the data here: 5076304 ± 0% +25.6% 6374220 ± 0% will-it-scale.per_process_ops ^^^ That's the reason why you sent the mail in the first place 1194117 ± 0% +14.4% 1366153 ± 1% will-it-scale.per_thread_ops 0.58 ± 0% -2.0% 0.57 ± 0% will-it-scale.scalability 6820 ± 0% -19.6% 5483 ± 15% meminfo.AnonHugePages 2652 ± 5% -10.4% 2375 ± 2% vmstat.system.cs 2848 ± 32% +141.2% 6870 ± 65% numa-meminfo.node1.Active(anon) 2832 ± 31% +57.6% 4465 ± 27% numa-meminfo.node1.AnonPages 15018 ± 12% -23.3% 11515 ± 15% numa-meminfo.node2.AnonPages 1214 ± 14% -22.8% 936.75 ± 20% numa-meminfo.node3.PageTables 712.25 ± 32% +141.2% 1718 ± 65% numa-vmstat.node1.nr_active_anon 708.25 ± 31% +57.7% 1116 ± 27% numa-vmstat.node1.nr_anon_pages How is this related and what should I do about this information? If it's important then I have to admit, that I fail to understand why. If it's not important then I have to ask why is this included. > > So that allows me to reproduce that test more or less with no effort. And > > that's the really important part. > > For reproducing, now we use lkp-tests tool, which includes scripts to > build the test case, run the test, collect various information, compare > the test result, with the job file attached with the report email. That > is not the easiest way, we will continuously improve it. I know and lkp-tests is a pain to work with. So please look into a way to extract the relevant binaries, so it's simple for developers to reproduce. > > You can provide nice charts and full comparison tables for all tests on a web > > site for those who are interested in large stats and pretty charts. > > > > Full results: http://wherever.you.store/your/results/test-nr/results > > Before we have a website for detailed information, we will still put > some details into report email. Ok, but please make them understandable for mere mortals. Thanks, tglx --8323329-244806905-1458547213=:3978--