From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-kernel-owner@vger.kernel.org>
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
	id S932708AbbFJIdr (ORCPT <rfc822;w@1wt.eu>);
	Wed, 10 Jun 2015 04:33:47 -0400
Received: from mail-wi0-f172.google.com ([209.85.212.172]:36216 "EHLO
	mail-wi0-f172.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org
	with ESMTP id S1753223AbbFJIdh (ORCPT
	<rfc822;linux-kernel@vger.kernel.org>);
	Wed, 10 Jun 2015 04:33:37 -0400
Date: Wed, 10 Jun 2015 10:33:32 +0200
From: Ingo Molnar <mingo@kernel.org>
To: Mel Gorman <mgorman@suse.de>
Cc: Andrew Morton <akpm@linux-foundation.org>, Rik van Riel <riel@redhat.com>,
        Hugh Dickins <hughd@google.com>, Minchan Kim <minchan@kernel.org>,
        Dave Hansen <dave.hansen@intel.com>, Andi Kleen <andi@firstfloor.org>,
        H Peter Anvin <hpa@zytor.com>,
        Linus Torvalds <torvalds@linux-foundation.org>,
        Thomas Gleixner <tglx@linutronix.de>,
        Peter Zijlstra <a.p.zijlstra@chello.nl>, Linux-MM <linux-mm@kvack.org>,
        LKML <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH 2/4] mm: Send one IPI per CPU to TLB flush all entries
 after unmapping pages
Message-ID: <20150610083332.GA25605@gmail.com>
References: <1433871118-15207-1-git-send-email-mgorman@suse.de>
 <1433871118-15207-3-git-send-email-mgorman@suse.de>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <1433871118-15207-3-git-send-email-mgorman@suse.de>
User-Agent: Mutt/1.5.23 (2014-03-12)
Sender: linux-kernel-owner@vger.kernel.org
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org


* Mel Gorman <mgorman@suse.de> wrote:

> Linear mapped reader on a 4-node machine with 64G RAM and 48 CPUs
> 
>                                         4.1.0-rc6          4.1.0-rc6
>                                           vanilla       flushfull-v6
> Ops lru-file-mmap-read-elapsed   162.88 (  0.00%)   120.81 ( 25.83%)
> 
>            4.1.0-rc6   4.1.0-rc6
>              vanillaflushfull-v6r5
> User          568.96      614.68
> System       6085.61     4226.61
> Elapsed       164.24      122.17
> 
> This is showing that the readers completed 25.83% faster with 30% less
> system CPU time. From vmstats, it is known that the vanilla kernel was
> interrupted roughly 900K times per second during the steady phase of the
> test and the patched kernel was interrupts 180K times per second.
> 
> The impact is lower on a single socket machine.
> 
>                                         4.1.0-rc6          4.1.0-rc6
>                                           vanilla       flushfull-v6
> Ops lru-file-mmap-read-elapsed    25.43 (  0.00%)    20.59 ( 19.03%)
> 
>            4.1.0-rc6    4.1.0-rc6
>              vanilla flushfull-v6
> User           59.14        58.99
> System        109.15        77.84
> Elapsed        27.32        22.31
> 
> It's still a noticeable improvement with vmstat showing interrupts went
> from roughly 500K per second to 45K per second.

Btw., I tried to compare your previous (v5) pfn-tracking numbers with these 
full-flushing numbers, and found that the IRQ rate appears to be the same:

> > From vmstats, it is known that the vanilla kernel was interrupted roughly 900K 
> > times per second during the steady phase of the test and the patched kernel 
> > was interrupts 180K times per second.

> > It's still a noticeable improvement with vmstat showing interrupts went from 
> > roughly 500K per second to 45K per second.

... is that because the batching limit in the pfn-tracking case was high enough to 
not be noticeable in the vmstat?

In the full-flushing case (v6 without patch 4) the batching limit is 'infinite', 
we'll batch as long as possible, right?

Or have I managed to get confused somewhere ...

Thanks,

	Ingo