From mboxrd@z Thu Jan 1 00:00:00 1970 From: Ben Guthro Subject: Re: [PATCH] scrub pages on guest termination Date: Fri, 23 May 2008 13:01:16 -0400 Message-ID: <4836F85C.1010609@virtualiron.com> References: Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============1590010452==" Return-path: In-Reply-To: List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Mime-version: 1.0 Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: Keir Fraser Cc: xen-devel , Robert Phillips List-Id: xen-devel@lists.xenproject.org This is a multi-part message in MIME format. --===============1590010452== Content-Type: multipart/alternative; boundary="------------010400060805090404060509" This is a multi-part message in MIME format. --------------010400060805090404060509 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Yes, sorry - should have removed our terminology from the description. Node=physical machine VS=HVM guest w/ pv-on-hvm drivers Looking back at the original bug report - it seems to indicate it was migrating from a system with 2 processors to one with 8 Specifcally - from Dell Precision WorkStation 380 Processor: Intel(R) Pentium(R) D CPU 2.80GHz # of CPUs: 2 Speed: 2.8GHz to Supermicro X7DB8 Processor: Genuine Intel(R) CPU @ 2.13GHz # of CPUs: 8 Speed: 2.133 GHz Keir Fraser wrote: > The aim of the loop was to scrub enough pages in a batch that lock > contention is kept tolerably low. Even if 16 pages is not sufficient > for that, I'm surprised a 'node' (you mean a whole system, > presumably?) would appear to lock up. Maybe pages would be scrubbed > slower than we'd like, but still CPUs should be able to get the > spinlock often enough to evaluate whether they have spent 1ms in the > loop and hence get out of there. > > What sort of system were you seeing the lockup on? Does it have very > many physical CPUs? > > -- Keir > > On 23/5/08 16:00, "Ben Guthro" wrote: > > This patch solves the following problem. When a large VS > terminates, the node locks > up. The node locks up because the page_scrub_kick routine sends a > softirq to > all processors instructing them to run the page scrub code. There > they interfere > with each other as they serialize behind the page_scrub_lock. > > --------------010400060805090404060509 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Yes, sorry -  should have removed our terminology from the description.
Node=physical machine
VS=HVM guest w/ pv-on-hvm drivers
Looking back at the original bug report - it seems to indicate it was migrating from a system with 2 processors to one with 8

Specifcally - from
Dell Precision WorkStation 380
Processor:    Intel(R) Pentium(R) D CPU 2.80GHz
# of CPUs:    2
Speed:    2.8GHz

to

Supermicro X7DB8
Processor:    Genuine Intel(R) CPU @ 2.13GHz
# of CPUs:    8
Speed:    2.133 GHz




Keir Fraser wrote:
Re: [Xen-devel] [PATCH] scrub pages on guest termination The aim of the loop was to scrub enough pages in a batch that lock contention is kept tolerably low. Even if 16 pages is not sufficient for that, I’m surprised a ‘node’ (you mean a whole system, presumably?) would appear to lock up. Maybe pages would be scrubbed slower than we’d like, but still CPUs should be able to get the spinlock often enough to evaluate whether they have spent 1ms in the loop and hence get out of there.

What sort of system were you seeing the lockup on? Does it have very many physical CPUs?

 -- Keir

On 23/5/08 16:00, "Ben Guthro" <bguthro@virtualiron.com> wrote:
This patch solves the following problem.  When a large VS terminates, the node locks
up. The node locks up because the page_scrub_kick routine sends a softirq to
all processors instructing them to run the page scrub code.  There they interfere
with each other as they serialize behind the page_scrub_lock.

--------------010400060805090404060509-- --===============1590010452== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel --===============1590010452==--