From mboxrd@z Thu Jan 1 00:00:00 1970 From: Rik van Riel Subject: Re: [PATCH 5/6] mm: compaction: Cache if a pageblock was scanned and no pages were isolated Date: Thu, 20 Sep 2012 14:55:22 -0400 Message-ID: <505B669A.1000306@redhat.com> References: <1348149875-29678-1-git-send-email-mgorman@suse.de> <1348149875-29678-6-git-send-email-mgorman@suse.de> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Richard Davies , Shaohua Li , Avi Kivity , QEMU-devel , KVM , Linux-MM , LKML To: Mel Gorman Return-path: In-Reply-To: <1348149875-29678-6-git-send-email-mgorman@suse.de> Sender: owner-linux-mm@kvack.org List-Id: kvm.vger.kernel.org On 09/20/2012 10:04 AM, Mel Gorman wrote: > When compaction was implemented it was known that scanning could potentially > be excessive. The ideal was that a counter be maintained for each pageblock > but maintaining this information would incur a severe penalty due to a > shared writable cache line. It has reached the point where the scanning > costs are an serious problem, particularly on long-lived systems where a > large process starts and allocates a large number of THPs at the same time. > > Instead of using a shared counter, this patch adds another bit to the > pageblock flags called PG_migrate_skip. If a pageblock is scanned by > either migrate or free scanner and 0 pages were isolated, the pageblock > is marked to be skipped in the future. When scanning, this bit is checked > before any scanning takes place and the block skipped if set. > > The main difficulty with a patch like this is "when to ignore the cached > information?" If it's ignored too often, the scanning rates will still > be excessive. If the information is too stale then allocations will fail > that might have otherwise succeeded. In this patch Big hammer, but I guess it is effective... Acked-by: Rik van Riel -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932242Ab2ITSze (ORCPT ); Thu, 20 Sep 2012 14:55:34 -0400 Received: from mx1.redhat.com ([209.132.183.28]:6439 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932166Ab2ITSza (ORCPT ); Thu, 20 Sep 2012 14:55:30 -0400 Message-ID: <505B669A.1000306@redhat.com> Date: Thu, 20 Sep 2012 14:55:22 -0400 From: Rik van Riel User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:14.0) Gecko/20120717 Thunderbird/14.0 MIME-Version: 1.0 To: Mel Gorman CC: Richard Davies , Shaohua Li , Avi Kivity , QEMU-devel , KVM , Linux-MM , LKML Subject: Re: [PATCH 5/6] mm: compaction: Cache if a pageblock was scanned and no pages were isolated References: <1348149875-29678-1-git-send-email-mgorman@suse.de> <1348149875-29678-6-git-send-email-mgorman@suse.de> In-Reply-To: <1348149875-29678-6-git-send-email-mgorman@suse.de> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 09/20/2012 10:04 AM, Mel Gorman wrote: > When compaction was implemented it was known that scanning could potentially > be excessive. The ideal was that a counter be maintained for each pageblock > but maintaining this information would incur a severe penalty due to a > shared writable cache line. It has reached the point where the scanning > costs are an serious problem, particularly on long-lived systems where a > large process starts and allocates a large number of THPs at the same time. > > Instead of using a shared counter, this patch adds another bit to the > pageblock flags called PG_migrate_skip. If a pageblock is scanned by > either migrate or free scanner and 0 pages were isolated, the pageblock > is marked to be skipped in the future. When scanning, this bit is checked > before any scanning takes place and the block skipped if set. > > The main difficulty with a patch like this is "when to ignore the cached > information?" If it's ignored too often, the scanning rates will still > be excessive. If the information is too stale then allocations will fail > that might have otherwise succeeded. In this patch Big hammer, but I guess it is effective... Acked-by: Rik van Riel From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([208.118.235.92]:35730) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1TEluQ-0001P7-6l for qemu-devel@nongnu.org; Thu, 20 Sep 2012 14:55:34 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1TEluL-0002Sc-3b for qemu-devel@nongnu.org; Thu, 20 Sep 2012 14:55:34 -0400 Received: from mx1.redhat.com ([209.132.183.28]:48936) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1TEluK-0002SL-Rs for qemu-devel@nongnu.org; Thu, 20 Sep 2012 14:55:29 -0400 Message-ID: <505B669A.1000306@redhat.com> Date: Thu, 20 Sep 2012 14:55:22 -0400 From: Rik van Riel MIME-Version: 1.0 References: <1348149875-29678-1-git-send-email-mgorman@suse.de> <1348149875-29678-6-git-send-email-mgorman@suse.de> In-Reply-To: <1348149875-29678-6-git-send-email-mgorman@suse.de> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] [PATCH 5/6] mm: compaction: Cache if a pageblock was scanned and no pages were isolated List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Mel Gorman Cc: Richard Davies , KVM , QEMU-devel , LKML , Linux-MM , Avi Kivity , Shaohua Li On 09/20/2012 10:04 AM, Mel Gorman wrote: > When compaction was implemented it was known that scanning could potentially > be excessive. The ideal was that a counter be maintained for each pageblock > but maintaining this information would incur a severe penalty due to a > shared writable cache line. It has reached the point where the scanning > costs are an serious problem, particularly on long-lived systems where a > large process starts and allocates a large number of THPs at the same time. > > Instead of using a shared counter, this patch adds another bit to the > pageblock flags called PG_migrate_skip. If a pageblock is scanned by > either migrate or free scanner and 0 pages were isolated, the pageblock > is marked to be skipped in the future. When scanning, this bit is checked > before any scanning takes place and the block skipped if set. > > The main difficulty with a patch like this is "when to ignore the cached > information?" If it's ignored too often, the scanning rates will still > be excessive. If the information is too stale then allocations will fail > that might have otherwise succeeded. In this patch Big hammer, but I guess it is effective... Acked-by: Rik van Riel