From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qa0-f47.google.com (mail-qa0-f47.google.com [209.85.216.47]) by kanga.kvack.org (Postfix) with ESMTP id 5CF166B0038 for ; Fri, 25 Jul 2014 11:07:23 -0400 (EDT) Received: by mail-qa0-f47.google.com with SMTP id i13so4648307qae.6 for ; Fri, 25 Jul 2014 08:07:23 -0700 (PDT) Received: from service87.mimecast.com (service87.mimecast.com. [91.220.42.44]) by mx.google.com with ESMTP id b7si16762382qai.88.2014.07.25.08.07.22 for ; Fri, 25 Jul 2014 08:07:22 -0700 (PDT) From: "Wilco Dijkstra" Subject: Background page clearing Date: Fri, 25 Jul 2014 16:06:51 +0100 Message-ID: <000001cfa81a$110d15c0$33274140$@com> MIME-Version: 1.0 Content-Language: en-gb Content-Type: text/plain; charset=WINDOWS-1252 Content-Transfer-Encoding: quoted-printable Sender: owner-linux-mm@kvack.org List-ID: To: linux-mm@kvack.org Hi, I recently noticed how a Stream benchmark took 30% more time in the first i= teration due to having to clean pages in the output array. Especially clearing a huge page on a pagef= ault is a substantial overhead. It affects the cached data of the workload while it is running an= d reduces available memory bandwidth. Is there a reason Linux does not do background page clearing like other OSe= s to reduce this overhead? It would be a good fit for typical mobile workloads (bursts of hi= gh activity followed by periods of low activity). Wilco -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pa0-f53.google.com (mail-pa0-f53.google.com [209.85.220.53]) by kanga.kvack.org (Postfix) with ESMTP id 0E9126B0035 for ; Fri, 25 Jul 2014 11:20:06 -0400 (EDT) Received: by mail-pa0-f53.google.com with SMTP id kq14so6262599pab.40 for ; Fri, 25 Jul 2014 08:20:06 -0700 (PDT) Received: from mga09.intel.com (mga09.intel.com. [134.134.136.24]) by mx.google.com with ESMTP id kj10si9514970pbd.63.2014.07.25.08.20.05 for ; Fri, 25 Jul 2014 08:20:06 -0700 (PDT) Message-ID: <53D27590.2090500@intel.com> Date: Fri, 25 Jul 2014 08:19:44 -0700 From: Dave Hansen MIME-Version: 1.0 Subject: Re: Background page clearing References: <000001cfa81a$110d15c0$33274140$@com> In-Reply-To: <000001cfa81a$110d15c0$33274140$@com> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit Sender: owner-linux-mm@kvack.org List-ID: To: Wilco Dijkstra , linux-mm@kvack.org On 07/25/2014 08:06 AM, Wilco Dijkstra wrote: > Is there a reason Linux does not do background page clearing like other OSes to reduce this > overhead? It would be a good fit for typical mobile workloads (bursts of high activity followed by > periods of low activity). If the page is being allocated, it is about to be used and be brought in to the CPU's cache. If we zero it close to this use, we only pay to bring it in to the CPU's cache once. Or so goes the theory... I tried a zero-on-free implementation a year or so ago. It helped some workloads and hurt others. The gains were not large enough or widespread enough to merit pushing it in to the kernel. -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wi0-f175.google.com (mail-wi0-f175.google.com [209.85.212.175]) by kanga.kvack.org (Postfix) with ESMTP id D4E6F6B0035 for ; Fri, 25 Jul 2014 12:27:34 -0400 (EDT) Received: by mail-wi0-f175.google.com with SMTP id ho1so1285820wib.14 for ; Fri, 25 Jul 2014 09:27:32 -0700 (PDT) Received: from service88.mimecast.com (service88.mimecast.com. [195.130.217.12]) by mx.google.com with ESMTP id pg3si18866411wjb.99.2014.07.25.09.27.30 for ; Fri, 25 Jul 2014 09:27:31 -0700 (PDT) From: Wilco Dijkstra Date: Fri, 25 Jul 2014 17:27:26 +0100 Subject: RE: Background page clearing Message-ID: References: <000001cfa81a$110d15c0$33274140$@com> <53D27590.2090500@intel.com> In-Reply-To: <53D27590.2090500@intel.com> Content-Language: en-US MIME-Version: 1.0 Content-Type: text/plain; charset=WINDOWS-1252 Content-Transfer-Encoding: quoted-printable Sender: owner-linux-mm@kvack.org List-ID: To: Dave Hansen , "linux-mm@kvack.org" > On 07/25/2014 08:06 AM, Wilco Dijkstra wrote: > > Is there a reason Linux does not do background page clearing like other= OSes to reduce this > > overhead? It would be a good fit for typical mobile workloads (bursts o= f high activity > followed by > > periods of low activity). > > If the page is being allocated, it is about to be used and be brought in > to the CPU's cache. If we zero it close to this use, we only pay to > bring it in to the CPU's cache once. Or so goes the theory... I can see the reasoning for 4KB pages and small allocations (eg. stack), but would that ever be true for huge pages? > I tried a zero-on-free implementation a year or so ago. It helped some > workloads and hurt others. The gains were not large enough or > widespread enough to merit pushing it in to the kernel. Was that literally zero-on-free or zero in the background? Was the result the same for different page sizes? My guess is that the result will be different for huge pages. Wilco -- IMPORTANT NOTICE: The contents of this email and any attachments are con= fidential and may also be privileged. If you are not the intended recipient= , please notify the sender immediately and do not disclose the contents to = any other person, use it for any purpose, or store or copy the information = in any medium. Thank you. ARM Limited, Registered office 110 Fulbourn Road, Cambridge CB1 9NJ, Regist= ered in England & Wales, Company No: 2557590 ARM Holdings plc, Registered office 110 Fulbourn Road, Cambridge CB1 9NJ, R= egistered in England & Wales, Company No: 2548782 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pa0-f47.google.com (mail-pa0-f47.google.com [209.85.220.47]) by kanga.kvack.org (Postfix) with ESMTP id 5B4806B0035 for ; Fri, 25 Jul 2014 12:32:40 -0400 (EDT) Received: by mail-pa0-f47.google.com with SMTP id kx10so6364215pab.20 for ; Fri, 25 Jul 2014 09:32:40 -0700 (PDT) Received: from mga09.intel.com (mga09.intel.com. [134.134.136.24]) by mx.google.com with ESMTP id du3si4859191pdb.245.2014.07.25.09.32.39 for ; Fri, 25 Jul 2014 09:32:39 -0700 (PDT) Message-ID: <53D286A5.7050100@intel.com> Date: Fri, 25 Jul 2014 09:32:37 -0700 From: Dave Hansen MIME-Version: 1.0 Subject: Re: Background page clearing References: <000001cfa81a$110d15c0$33274140$@com> <53D27590.2090500@intel.com> In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit Sender: owner-linux-mm@kvack.org List-ID: To: Wilco Dijkstra , "linux-mm@kvack.org" On 07/25/2014 09:27 AM, Wilco Dijkstra wrote: >> On 07/25/2014 08:06 AM, Wilco Dijkstra wrote: >>> Is there a reason Linux does not do background page clearing like other OSes to reduce this >>> overhead? It would be a good fit for typical mobile workloads (bursts of high activity >> followed by >>> periods of low activity). >> >> If the page is being allocated, it is about to be used and be brought in >> to the CPU's cache. If we zero it close to this use, we only pay to >> bring it in to the CPU's cache once. Or so goes the theory... > > I can see the reasoning for 4KB pages and small allocations (eg. stack), > but would that ever be true for huge pages? Probably not, but huge pages aren't allocated and freed enough in any workload that I know of for this to make a difference for them. >> I tried a zero-on-free implementation a year or so ago. It helped some >> workloads and hurt others. The gains were not large enough or >> widespread enough to merit pushing it in to the kernel. > > Was that literally zero-on-free or zero in the background? Was the result > the same for different page sizes? My guess is that the result will be > different for huge pages. Literally zero-on-free for 4k pages only. I did it inside the per-cpu-pages lists. -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org