All of lore.kernel.org
 help / color / mirror / Atom feed
From: Minchan Kim <minchan@kernel.org>
To: Nadav Amit <nadav.amit@gmail.com>
Cc: Ye Xiaolong <xiaolong.ye@intel.com>,
	"open list:MEMORY MANAGEMENT" <linux-mm@kvack.org>,
	LKML <linux-kernel@vger.kernel.org>,
	Andrew Morton <akpm@linux-foundation.org>,
	Ingo Molnar <mingo@redhat.com>,
	Russell King <linux@armlinux.org.uk>,
	Tony Luck <tony.luck@intel.com>,
	Martin Schwidefsky <schwidefsky@de.ibm.com>,
	"David S. Miller" <davem@davemloft.net>,
	Heiko Carstens <heiko.carstens@de.ibm.com>,
	Yoshinori Sato <ysato@users.sourceforge.jp>,
	Jeff Dike <jdike@addtoit.com>,
	linux-arch@vger.kernel.org, lkp@01.org
Subject: Re: [lkp-robot] [mm]  7674270022:  will-it-scale.per_process_ops -19.3% regression
Date: Thu, 10 Aug 2017 13:20:40 +0900	[thread overview]
Message-ID: <20170810042040.GA2249@bbox> (raw)
In-Reply-To: <80589593-6F0E-4421-9279-681D5B388100@gmail.com>

On Wed, Aug 09, 2017 at 09:14:50PM -0700, Nadav Amit wrote:

Hi Nadav,

< snip >

> >>>>> According to the description it is "testcase:brk increase/decrease of one
> >>>>> page”. According to the mode it spawns multiple processes, not threads.
> >>>>> 
> >>>>> Since a single page is unmapped each time, and the iTLB-loads increase
> >>>>> dramatically, I would suspect that for some reason a full TLB flush is
> >>>>> caused during do_munmap().
> >>>>> 
> >>>>> If I find some free time, I’ll try to profile the workload - but feel free
> >>>>> to beat me to it.
> >>>> 
> >>>> The root-cause appears to be that tlb_finish_mmu() does not call
> >>>> dec_tlb_flush_pending() - as it should. Any chance you can take care of it?
> >>> 
> >>> Oops, but with second looking, it seems it's not my fault. ;-)
> >>> https://marc.info/?l=linux-mm&m=150156699114088&w=2
> >>> 
> >>> Anyway, thanks for the pointing out.
> >>> xiaolong.ye, could you retest with this fix?
> >> 
> >> I've queued tests for 5 times and results show this patch (e8f682574e4 "mm:
> >> decrease tlb flush pending count in tlb_finish_mmu") does help recover the
> >> performance back.
> >> 
> >> 378005bdbac0a2ec  76742700225cad9df49f053993  e8f682574e45b6406dadfffeb4  
> >> ----------------  --------------------------  --------------------------  
> >>         %stddev      change         %stddev      change         %stddev
> >>             \          |                \          |                \  
> >>   3405093             -19%    2747088              -2%    3348752        will-it-scale.per_process_ops
> >>      1280 ±  3%        -2%       1257 ±  3%        -6%       1207        vmstat.system.cs
> >>      2702 ± 18%        11%       3002 ± 19%        17%       3156 ± 18%  numa-vmstat.node0.nr_mapped
> >>     10765 ± 18%        11%      11964 ± 19%        17%      12588 ± 18%  numa-meminfo.node0.Mapped
> >>      0.00 ± 47%       -40%       0.00 ± 45%       -84%       0.00 ± 42%  mpstat.cpu.soft%
> >> 
> >> Thanks,
> >> Xiaolong
> > 
> > Thanks for the testing!
> 
> Sorry again for screwing your patch, Minchan.

Never mind! It always happens. :)
In this chance, I really appreciates your insight/testing/cooperation!

WARNING: multiple messages have this Message-ID (diff)
From: Minchan Kim <minchan@kernel.org>
To: lkp@lists.01.org
Subject: Re: [lkp-robot] [mm] 7674270022: will-it-scale.per_process_ops -19.3% regression
Date: Thu, 10 Aug 2017 13:20:40 +0900	[thread overview]
Message-ID: <20170810042040.GA2249@bbox> (raw)
In-Reply-To: <80589593-6F0E-4421-9279-681D5B388100@gmail.com>

[-- Attachment #1: Type: text/plain, Size: 2266 bytes --]

On Wed, Aug 09, 2017 at 09:14:50PM -0700, Nadav Amit wrote:

Hi Nadav,

< snip >

> >>>>> According to the description it is "testcase:brk increase/decrease of one
> >>>>> page”. According to the mode it spawns multiple processes, not threads.
> >>>>> 
> >>>>> Since a single page is unmapped each time, and the iTLB-loads increase
> >>>>> dramatically, I would suspect that for some reason a full TLB flush is
> >>>>> caused during do_munmap().
> >>>>> 
> >>>>> If I find some free time, I’ll try to profile the workload - but feel free
> >>>>> to beat me to it.
> >>>> 
> >>>> The root-cause appears to be that tlb_finish_mmu() does not call
> >>>> dec_tlb_flush_pending() - as it should. Any chance you can take care of it?
> >>> 
> >>> Oops, but with second looking, it seems it's not my fault. ;-)
> >>> https://marc.info/?l=linux-mm&m=150156699114088&w=2
> >>> 
> >>> Anyway, thanks for the pointing out.
> >>> xiaolong.ye, could you retest with this fix?
> >> 
> >> I've queued tests for 5 times and results show this patch (e8f682574e4 "mm:
> >> decrease tlb flush pending count in tlb_finish_mmu") does help recover the
> >> performance back.
> >> 
> >> 378005bdbac0a2ec  76742700225cad9df49f053993  e8f682574e45b6406dadfffeb4  
> >> ----------------  --------------------------  --------------------------  
> >>         %stddev      change         %stddev      change         %stddev
> >>             \          |                \          |                \  
> >>   3405093             -19%    2747088              -2%    3348752        will-it-scale.per_process_ops
> >>      1280 ±  3%        -2%       1257 ±  3%        -6%       1207        vmstat.system.cs
> >>      2702 ± 18%        11%       3002 ± 19%        17%       3156 ± 18%  numa-vmstat.node0.nr_mapped
> >>     10765 ± 18%        11%      11964 ± 19%        17%      12588 ± 18%  numa-meminfo.node0.Mapped
> >>      0.00 ± 47%       -40%       0.00 ± 45%       -84%       0.00 ± 42%  mpstat.cpu.soft%
> >> 
> >> Thanks,
> >> Xiaolong
> > 
> > Thanks for the testing!
> 
> Sorry again for screwing your patch, Minchan.

Never mind! It always happens. :)
In this chance, I really appreciates your insight/testing/cooperation!

WARNING: multiple messages have this Message-ID (diff)
From: Minchan Kim <minchan@kernel.org>
To: Nadav Amit <nadav.amit@gmail.com>
Cc: Ye Xiaolong <xiaolong.ye@intel.com>,
	"open list:MEMORY MANAGEMENT" <linux-mm@kvack.org>,
	LKML <linux-kernel@vger.kernel.org>,
	Andrew Morton <akpm@linux-foundation.org>,
	Ingo Molnar <mingo@redhat.com>,
	Russell King <linux@armlinux.org.uk>,
	Tony Luck <tony.luck@intel.com>,
	Martin Schwidefsky <schwidefsky@de.ibm.com>,
	"David S. Miller" <davem@davemloft.net>,
	Heiko Carstens <heiko.carstens@de.ibm.com>,
	Yoshinori Sato <ysato@users.sourceforge.jp>,
	Jeff Dike <jdike@addtoit.com>,
	linux-arch@vger.kernel.org, lkp@01.org
Subject: Re: [lkp-robot] [mm]  7674270022:  will-it-scale.per_process_ops -19.3% regression
Date: Thu, 10 Aug 2017 13:20:40 +0900	[thread overview]
Message-ID: <20170810042040.GA2249@bbox> (raw)
In-Reply-To: <80589593-6F0E-4421-9279-681D5B388100@gmail.com>

On Wed, Aug 09, 2017 at 09:14:50PM -0700, Nadav Amit wrote:

Hi Nadav,

< snip >

> >>>>> According to the description it is "testcase:brk increase/decrease of one
> >>>>> pagea??. According to the mode it spawns multiple processes, not threads.
> >>>>> 
> >>>>> Since a single page is unmapped each time, and the iTLB-loads increase
> >>>>> dramatically, I would suspect that for some reason a full TLB flush is
> >>>>> caused during do_munmap().
> >>>>> 
> >>>>> If I find some free time, Ia??ll try to profile the workload - but feel free
> >>>>> to beat me to it.
> >>>> 
> >>>> The root-cause appears to be that tlb_finish_mmu() does not call
> >>>> dec_tlb_flush_pending() - as it should. Any chance you can take care of it?
> >>> 
> >>> Oops, but with second looking, it seems it's not my fault. ;-)
> >>> https://marc.info/?l=linux-mm&m=150156699114088&w=2
> >>> 
> >>> Anyway, thanks for the pointing out.
> >>> xiaolong.ye, could you retest with this fix?
> >> 
> >> I've queued tests for 5 times and results show this patch (e8f682574e4 "mm:
> >> decrease tlb flush pending count in tlb_finish_mmu") does help recover the
> >> performance back.
> >> 
> >> 378005bdbac0a2ec  76742700225cad9df49f053993  e8f682574e45b6406dadfffeb4  
> >> ----------------  --------------------------  --------------------------  
> >>         %stddev      change         %stddev      change         %stddev
> >>             \          |                \          |                \  
> >>   3405093             -19%    2747088              -2%    3348752        will-it-scale.per_process_ops
> >>      1280 A+-  3%        -2%       1257 A+-  3%        -6%       1207        vmstat.system.cs
> >>      2702 A+- 18%        11%       3002 A+- 19%        17%       3156 A+- 18%  numa-vmstat.node0.nr_mapped
> >>     10765 A+- 18%        11%      11964 A+- 19%        17%      12588 A+- 18%  numa-meminfo.node0.Mapped
> >>      0.00 A+- 47%       -40%       0.00 A+- 45%       -84%       0.00 A+- 42%  mpstat.cpu.soft%
> >> 
> >> Thanks,
> >> Xiaolong
> > 
> > Thanks for the testing!
> 
> Sorry again for screwing your patch, Minchan.

Never mind! It always happens. :)
In this chance, I really appreciates your insight/testing/cooperation!

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

  reply	other threads:[~2017-08-10  4:20 UTC|newest]

Thread overview: 85+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2017-08-02  0:08 [PATCH v6 0/7] fixes of TLB batching races Nadav Amit
2017-08-02  0:08 ` Nadav Amit
2017-08-02  0:08 ` [PATCH v6 1/7] mm: migrate: prevent racy access to tlb_flush_pending Nadav Amit
2017-08-02  0:08   ` Nadav Amit
2017-08-02  0:08 ` [PATCH v6 2/7] mm: migrate: fix barriers around tlb_flush_pending Nadav Amit
2017-08-02  0:08   ` Nadav Amit
2017-08-02  0:08 ` [PATCH v6 3/7] Revert "mm: numa: defer TLB flush for THP migration as long as possible" Nadav Amit
2017-08-02  0:08   ` Nadav Amit
2017-08-11 10:50   ` Peter Zijlstra
2017-08-11 10:50     ` Peter Zijlstra
2017-08-02  0:08 ` [PATCH v6 4/7] mm: refactoring TLB gathering API Nadav Amit
2017-08-02  0:08   ` Nadav Amit
2017-08-02  0:08   ` Nadav Amit
2017-08-11  9:23   ` Peter Zijlstra
2017-08-11  9:23     ` Peter Zijlstra
2017-08-11 17:12     ` Nadav Amit
2017-08-11 17:12       ` Nadav Amit
2017-08-14  0:49       ` Minchan Kim
2017-08-14  0:49         ` Minchan Kim
2017-08-02  0:08 ` [PATCH v6 5/7] mm: make tlb_flush_pending global Nadav Amit
2017-08-02  0:08   ` Nadav Amit
2017-08-02 14:28   ` kbuild test robot
2017-08-02 14:28     ` kbuild test robot
2017-08-02 23:23     ` Minchan Kim
2017-08-02 23:23       ` Minchan Kim
2017-08-02 23:27     ` Andrew Morton
2017-08-02 23:27       ` Andrew Morton
2017-08-02 23:34       ` Minchan Kim
2017-08-02 23:34         ` Minchan Kim
2017-08-03 16:40   ` kbuild test robot
2017-08-03 16:40     ` kbuild test robot
2017-08-02  0:08 ` [PATCH v6 6/7] mm: fix MADV_[FREE|DONTNEED] TLB flush miss problem Nadav Amit
2017-08-02  0:08   ` Nadav Amit
2017-08-02  0:08   ` Nadav Amit
2017-08-08  1:19   ` [lkp-robot] [mm] 7674270022: will-it-scale.per_process_ops -19.3% regression kernel test robot
2017-08-08  1:19     ` kernel test robot
2017-08-08  1:19     ` kernel test robot
2017-08-08  1:19     ` kernel test robot
2017-08-08  2:28     ` Minchan Kim
2017-08-08  2:28       ` Minchan Kim
2017-08-08  2:28       ` Minchan Kim
2017-08-08  4:23       ` Nadav Amit
2017-08-08  4:23         ` Nadav Amit
2017-08-08  4:23         ` Nadav Amit
2017-08-08  5:51         ` Nadav Amit
2017-08-08  5:51           ` Nadav Amit
2017-08-08  5:51           ` Nadav Amit
2017-08-08  8:08           ` Minchan Kim
2017-08-08  8:08             ` Minchan Kim
2017-08-08  8:08             ` Minchan Kim
2017-08-08  8:08             ` Minchan Kim
2017-08-08  8:08             ` Minchan Kim
2017-08-08  8:16             ` Nadav Amit
2017-08-08  8:16               ` Nadav Amit
2017-08-09  1:25             ` Ye Xiaolong
2017-08-09  1:25               ` Ye Xiaolong
2017-08-09  1:25               ` Ye Xiaolong
2017-08-09  1:25               ` Ye Xiaolong
2017-08-09  2:59             ` Ye Xiaolong
2017-08-09  2:59               ` Ye Xiaolong
2017-08-09  2:59               ` Ye Xiaolong
2017-08-09  2:59               ` Ye Xiaolong
2017-08-09  2:59               ` Ye Xiaolong
2017-08-10  4:13               ` Minchan Kim
2017-08-10  4:13                 ` Minchan Kim
2017-08-10  4:13                 ` Minchan Kim
2017-08-10  4:14                 ` Nadav Amit
2017-08-10  4:14                   ` Nadav Amit
2017-08-10  4:14                   ` Nadav Amit
2017-08-10  4:20                   ` Minchan Kim [this message]
2017-08-10  4:20                     ` Minchan Kim
2017-08-10  4:20                     ` Minchan Kim
2017-08-11 13:30   ` [PATCH v6 6/7] mm: fix MADV_[FREE|DONTNEED] TLB flush miss problem Peter Zijlstra
2017-08-11 13:30     ` Peter Zijlstra
2017-08-13  6:14     ` Nadav Amit
2017-08-13 12:08       ` Peter Zijlstra
2017-08-13 12:08         ` Peter Zijlstra
2017-08-13 12:08         ` Peter Zijlstra
2017-08-14  1:26     ` Minchan Kim
2017-08-14  1:26       ` Minchan Kim
2017-08-14  1:26       ` Minchan Kim
2017-08-02  0:08 ` [PATCH v6 7/7] mm: fix KSM data corruption Nadav Amit
2017-08-02  0:08   ` Nadav Amit
2017-08-02 23:26 ` [PATCH v6 0/7] fixes of TLB batching races Minchan Kim
2017-08-02 23:26   ` Minchan Kim

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20170810042040.GA2249@bbox \
    --to=minchan@kernel.org \
    --cc=akpm@linux-foundation.org \
    --cc=davem@davemloft.net \
    --cc=heiko.carstens@de.ibm.com \
    --cc=jdike@addtoit.com \
    --cc=linux-arch@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=linux@armlinux.org.uk \
    --cc=lkp@01.org \
    --cc=mingo@redhat.com \
    --cc=nadav.amit@gmail.com \
    --cc=schwidefsky@de.ibm.com \
    --cc=tony.luck@intel.com \
    --cc=xiaolong.ye@intel.com \
    --cc=ysato@users.sourceforge.jp \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.