From mboxrd@z Thu Jan 1 00:00:00 1970 From: "zhenzhong.duan" Subject: kernel bootup slow issue on ovm3.1.1 Date: Tue, 07 Aug 2012 15:22:50 +0800 Message-ID: <5020C24A.3060604@oracle.com> Reply-To: zhenzhong.duan@oracle.com Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============4630480789698147875==" Return-path: List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: xen-devel@lists.xensource.com, Konrad Rzeszutek Wilk , Feng Jin List-Id: xen-devel@lists.xenproject.org This is a multi-part message in MIME format. --===============4630480789698147875== Content-Type: multipart/alternative; boundary="------------060807030108080608060303" This is a multi-part message in MIME format. --------------060807030108080608060303 Content-Type: text/plain; charset=GB2312 Content-Transfer-Encoding: 7bit Hi maintainers, We meet a uek2 bootup slow issue on our ovm product(ovm3.0.3 and ovm3.1.1). The system env is an exalogic node with 24 cores + 100G mem (2 socket , 6 cores per socket, 2 HT threads per core). After boot up this node with all cores enabled, We boot a pvhvm with 12vpcus (or 24) + 90 GB + pci passthroughed device, it takes 30+ mins to boot. If we remove passthrough device from vm.cfg, bootup takes about 2 mins. If we use a small mem(eg. 10G + 24 vcpus), bootup takes about 3 mins. So a big mem + passthrough device made the worst case. If we boot this node with HT disabled from BIOS. Now only 12 cores are available. OVM on same node, same config with 12vpcus+90GB boots in 1.5 mins! After some debug, we found it's in kernel mtrr init that make this delay. mtrr_aps_init() \-> set_mtrr() \-> mtrr_work_handler() kernel spin in mtrr_work_handler. But we don't know the scene hide in the hypervisor. Why big mem + passthrough made the worst case. Is this already fixed in xen upstream? Any comments are welcome, I'll upload all data depend on your need. thanks zduan --------------060807030108080608060303 Content-Type: text/html; charset=GB2312 Content-Transfer-Encoding: 7bit Hi maintainers,

We meet a uek2 bootup slow issue on our ovm product(ovm3.0.3 and ovm3.1.1).

The system env is an exalogic node with 24 cores + 100G mem (2 socket , 6 cores per socket, 2 HT threads per core).
After boot up this node with all cores enabled,
We boot a pvhvm with 12vpcus (or 24) + 90 GB + pci passthroughed device, it takes 30+ mins to boot.
If we remove passthrough device from vm.cfg, bootup takes about 2 mins.
If we use a small mem(eg. 10G + 24 vcpus), bootup takes about 3 mins.
So a big mem + passthrough device made the worst case.

If we boot this node with HT disabled from BIOS. Now only 12 cores are available.
OVM on same node, same config with 12vpcus+90GB boots in 1.5 mins!

After some debug, we found it's in kernel mtrr init that make this delay. 
mtrr_aps_init() 
 \-> set_mtrr() 
     \-> mtrr_work_handler() 

kernel spin in mtrr_work_handler.
But we don't know the scene hide in the hypervisor. Why big mem + passthrough made the worst case.
Is this already fixed in xen upstream?
Any comments are welcome, I'll upload all data depend on your need.

thanks
zduan
--------------060807030108080608060303-- --===============4630480789698147875== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel --===============4630480789698147875==-- From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Jan Beulich" Subject: Re: kernel bootup slow issue on ovm3.1.1 Date: Tue, 07 Aug 2012 09:37:55 +0100 Message-ID: <5020F003020000780009322C@nat28.tlf.novell.com> References: <5020C24A.3060604@oracle.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <5020C24A.3060604@oracle.com> Content-Disposition: inline List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: zhenzhong.duan@oracle.com Cc: Konrad Rzeszutek Wilk , Feng Jin , xen-devel List-Id: xen-devel@lists.xenproject.org >>> On 07.08.12 at 09:22, "zhenzhong.duan" wrote: > After some debug, we found it's in kernel mtrr init that make this delay. > > mtrr_aps_init() > \-> set_mtrr() > \-> mtrr_work_handler() > > kernel spin in mtrr_work_handler. > > But we don't know the scene hide in the hypervisor. Why big mem + > passthrough made the worst case. > Is this already fixed in xen upstream? First of all it would have been useful to indicate the kernel version, since mtrr_work_handler() disappeared after 3.0. Obviously worth checking whether that change by itself already addresses your problem. Next, if you already spotted where the spinning occurs, you should also be able to tell what's going on at the other side, i.e. why the event that is being waited for isn't occurring for this long a time. Since there's a number of open coded spin loops here, knowing exactly which one each CPU is sitting in (and which ones might not be in any) is the fundamental information needed. >>From what you're telling us so far, I'd rather suspect a kernel problem, not a hypervisor one here. Jan From mboxrd@z Thu Jan 1 00:00:00 1970 From: Konrad Rzeszutek Wilk Subject: Re: kernel bootup slow issue on ovm3.1.1 Date: Tue, 7 Aug 2012 12:26:37 -0400 Message-ID: <20120807162637.GB15053@phenom.dumpdata.com> References: <5020C24A.3060604@oracle.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: Content-Disposition: inline In-Reply-To: <5020C24A.3060604@oracle.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: "zhenzhong.duan" Cc: xen-devel@lists.xensource.com, Feng Jin List-Id: xen-devel@lists.xenproject.org On Tue, Aug 07, 2012 at 03:22:50PM +0800, zhenzhong.duan wrote: > Hi maintainers, > > We meet a uek2 bootup slow issue on our ovm product(ovm3.0.3 and ovm3.1.1). > > The system env is an exalogic node with 24 cores + 100G mem (2 socket , > 6 cores per socket, 2 HT threads per core). > After boot up this node with all cores enabled, > We boot a pvhvm with 12vpcus (or 24) + 90 GB + pci passthroughed device, > it takes 30+ mins to boot. > If we remove passthrough device from vm.cfg, bootup takes about 2 mins. > If we use a small mem(eg. 10G + 24 vcpus), bootup takes about 3 mins. > So a big mem + passthrough device made the worst case. > > If we boot this node with HT disabled from BIOS. Now only 12 cores are > available. > OVM on same node, same config with 12vpcus+90GB boots in 1.5 mins! > > After some debug, we found it's in kernel mtrr init that make this delay. > > mtrr_aps_init() > \-> set_mtrr() > \-> mtrr_work_handler() > > kernel spin in mtrr_work_handler. > > But we don't know the scene hide in the hypervisor. Why big mem + > passthrough made the worst case. > Is this already fixed in xen upstream? > Any comments are welcome, I'll upload all data depend on your need. What happens if you run with a upstream version of kernel? Say v3.4.7 ? Do you see the same issue? > > thanks > zduan > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > http://lists.xen.org/xen-devel From mboxrd@z Thu Jan 1 00:00:00 1970 From: "zhenzhong.duan" Subject: Re: kernel bootup slow issue on ovm3.1.1 Date: Wed, 08 Aug 2012 17:23:51 +0800 Message-ID: <50223027.6080502@oracle.com> References: <5020C24A.3060604@oracle.com> <20120807162637.GB15053@phenom.dumpdata.com> Reply-To: zhenzhong.duan@oracle.com Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============4844915715082705569==" Return-path: In-Reply-To: <20120807162637.GB15053@phenom.dumpdata.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Konrad Rzeszutek Wilk Cc: xen-devel@lists.xensource.com, Feng Jin List-Id: xen-devel@lists.xenproject.org This is a multi-part message in MIME format. --===============4844915715082705569== Content-Type: multipart/alternative; boundary="------------060509090501000408040803" This is a multi-part message in MIME format. --------------060509090501000408040803 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit ? 2012-08-08 00:26, Konrad Rzeszutek Wilk ??: > On Tue, Aug 07, 2012 at 03:22:50PM +0800, zhenzhong.duan wrote: >> Hi maintainers, >> >> We meet a uek2 bootup slow issue on our ovm product(ovm3.0.3 and ovm3.1.1). >> >> The system env is an exalogic node with 24 cores + 100G mem (2 socket , >> 6 cores per socket, 2 HT threads per core). >> After boot up this node with all cores enabled, >> We boot a pvhvm with 12vpcus (or 24) + 90 GB + pci passthroughed device, >> it takes 30+ mins to boot. >> If we remove passthrough device from vm.cfg, bootup takes about 2 mins. >> If we use a small mem(eg. 10G + 24 vcpus), bootup takes about 3 mins. >> So a big mem + passthrough device made the worst case. >> >> If we boot this node with HT disabled from BIOS. Now only 12 cores are >> available. >> OVM on same node, same config with 12vpcus+90GB boots in 1.5 mins! >> >> After some debug, we found it's in kernel mtrr init that make this delay. >> >> mtrr_aps_init() >> \-> set_mtrr() >> \-> mtrr_work_handler() >> >> kernel spin in mtrr_work_handler. >> >> But we don't know the scene hide in the hypervisor. Why big mem + >> passthrough made the worst case. >> Is this already fixed in xen upstream? >> Any comments are welcome, I'll upload all data depend on your need. > What happens if you run with a upstream version of kernel? Say v3.4.7 ? Hi konrad, Jan, I tried 3.5.0-2.fc17.x86_64 and 3.6.0-rc1. * 3.5.0-2.fc17.x86_64 took ~30 mins.* Below is piece of fc17 dmesg: #22[ 0.002999] installing Xen timer for CPU 22 #23[ 0.002999] installing Xen timer for CPU 23 [ 1.844896] Brought up 24 CPUs [ 1.844898] Total of 24 processors activated (140449.34 BogoMIPS). *block for 30 mins here.* [ 1.899794] devtmpfs: initialized [ 1.905956] atomic64 test passed for x86-64 platform with CX8 and with SSE * 3.6.0-rc1 took more than 2 hours.* piece of dmesg: cpu 22 spinlock event irq 218 [ 1.884775] #22[ 0.001999] installing Xen timer for CPU 22 cpu 23 spinlock event irq 225 [ 1.932764] #23[ 0.001999] installing Xen timer for CPU 23 [ 1.977734] Brought up 24 CPUs [ 1.978706] smpboot: Total of 24 processors activated (140449.34 BogoMIPS) *block for more than 2 hours here.* [ 1.988859] devtmpfs: initialized [ 2.021785] dummy: [ 2.023706] NET: Registered protocol family 16 [ 2.026735] ACPI: bus type pci registered [ 2.028002] PCI: Using configuration type 1 for base access I also send a patch to lkml that can workaround this issue, but I don't know the reason of block in xen side. link: https://lkml.org/lkml/2012/8/7/50 regards zduan --------------060509090501000408040803 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit

于 2012-08-08 00:26, Konrad Rzeszutek Wilk 写道: 
On Tue, Aug 07, 2012 at 03:22:50PM +0800, zhenzhong.duan wrote:

Hi maintainers,

We meet a uek2 bootup slow issue on our ovm product(ovm3.0.3 and ovm3.1.1).

The system env is an exalogic node with 24 cores + 100G mem (2 socket ,
6 cores per socket, 2 HT threads per core).
After boot up this node with all cores enabled,
We boot a pvhvm with 12vpcus (or 24) + 90 GB + pci passthroughed device,
it takes 30+ mins to boot.
If we remove passthrough device from vm.cfg, bootup takes about 2 mins.
If we use a small mem(eg. 10G + 24 vcpus), bootup takes about 3 mins.
So a big mem + passthrough device made the worst case.

If we boot this node with HT disabled from BIOS. Now only 12 cores are
available.
OVM on same node, same config with 12vpcus+90GB boots in 1.5 mins!

After some debug, we found it's in kernel mtrr init that make this delay.

mtrr_aps_init() 
 \-> set_mtrr() 
     \-> mtrr_work_handler() 

kernel spin in mtrr_work_handler.

But we don't know the scene hide in the hypervisor. Why big mem +
passthrough made the worst case.
Is this already fixed in xen upstream?
Any comments are welcome, I'll upload all data depend on your need.

What happens if you run with a upstream version of kernel? Say v3.4.7 ?
Hi konrad, Jan,

I tried 3.5.0-2.fc17.x86_64 and 3.6.0-rc1.

3.5.0-2.fc17.x86_64 took ~30 mins.
Below is piece of fc17 dmesg:
 #22[    0.002999] installing Xen timer for CPU 22
 #23[    0.002999] installing Xen timer for CPU 23
[    1.844896] Brought up 24 CPUs
[    1.844898] Total of 24 processors activated (140449.34 BogoMIPS).
block for 30 mins here.
[    1.899794] devtmpfs: initialized
[    1.905956] atomic64 test passed for x86-64 platform with CX8 and with SSE

3.6.0-rc1 took more than 2 hours.
piece of dmesg:
cpu 22 spinlock event irq 218
[    1.884775]  #22[    0.001999] installing Xen timer for CPU 22
cpu 23 spinlock event irq 225
[    1.932764]  #23[    0.001999] installing Xen timer for CPU 23
[    1.977734] Brought up 24 CPUs
[    1.978706] smpboot: Total of 24 processors activated (140449.34 BogoMIPS)
block for more than 2 hours here.
[    1.988859] devtmpfs: initialized
[    2.021785] dummy:
[    2.023706] NET: Registered protocol family 16
[    2.026735] ACPI: bus type pci registered
[    2.028002] PCI: Using configuration type 1 for base access

I also send a patch to lkml that can workaround this issue, but I don't know the reason of block in xen side.
link: https://lkml.org/lkml/2012/8/7/50

regards
zduan

    

  


--------------060509090501000408040803--


--===============4844915715082705569==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

--===============4844915715082705569==--

From mboxrd@z Thu Jan  1 00:00:00 1970
From: "zhenzhong.duan" 
Subject: Re: kernel bootup slow issue on ovm3.1.1
Date: Wed, 08 Aug 2012 17:48:24 +0800
Message-ID: <502235E8.9040309@oracle.com>
References: <5020C24A.3060604@oracle.com>
	<5020F003020000780009322C@nat28.tlf.novell.com>
Reply-To: zhenzhong.duan@oracle.com
Mime-Version: 1.0
Content-Type: text/plain; charset="utf-8"; Format="flowed"
Content-Transfer-Encoding: base64
Return-path: 
In-Reply-To: <5020F003020000780009322C@nat28.tlf.novell.com>
List-Unsubscribe: ,
	
List-Post: 
List-Help: 
List-Subscribe: ,
	
Sender: xen-devel-bounces@lists.xen.org
Errors-To: xen-devel-bounces@lists.xen.org
To: Jan Beulich 
Cc: Konrad Rzeszutek Wilk , Feng Jin , xen-devel 
List-Id: xen-devel@lists.xenproject.org

Cgrkuo4gMjAxMi0wOC0wNyAxNjozNywgSmFuIEJldWxpY2gg5YaZ6YGTOgo+Pj4+IE9uIDA3LjA4
LjEyIGF0IDA5OjIyLCAiemhlbnpob25nLmR1YW4iPHpoZW56aG9uZy5kdWFuQG9yYWNsZS5jb20+
ICB3cm90ZToKPj4gQWZ0ZXIgc29tZSBkZWJ1Zywgd2UgZm91bmQgaXQncyBpbiBrZXJuZWwgbXRy
ciBpbml0IHRoYXQgbWFrZSB0aGlzIGRlbGF5Lgo+Pgo+PiBtdHJyX2Fwc19pbml0KCkKPj4gICBc
LT4gIHNldF9tdHJyKCkKPj4gICAgICAgXC0+ICBtdHJyX3dvcmtfaGFuZGxlcigpCj4+Cj4+IGtl
cm5lbCBzcGluIGluIG10cnJfd29ya19oYW5kbGVyLgo+Pgo+PiBCdXQgd2UgZG9uJ3Qga25vdyB0
aGUgc2NlbmUgaGlkZSBpbiB0aGUgaHlwZXJ2aXNvci4gV2h5IGJpZyBtZW0gKwo+PiBwYXNzdGhy
b3VnaCBtYWRlIHRoZSB3b3JzdCBjYXNlLgo+PiBJcyB0aGlzIGFscmVhZHkgZml4ZWQgaW4geGVu
IHVwc3RyZWFtPwo+IEZpcnN0IG9mIGFsbCBpdCB3b3VsZCBoYXZlIGJlZW4gdXNlZnVsIHRvIGlu
ZGljYXRlIHRoZSBrZXJuZWwgdmVyc2lvbiwKPiBzaW5jZSBtdHJyX3dvcmtfaGFuZGxlcigpIGRp
c2FwcGVhcmVkIGFmdGVyIDMuMC4gT2J2aW91c2x5IHdvcnRoCj4gY2hlY2tpbmcgd2hldGhlciB0
aGF0IGNoYW5nZSBieSBpdHNlbGYgYWxyZWFkeSBhZGRyZXNzZXMgeW91cgo+IHByb2JsZW0uCk5v
IGx1Y2ssIHRyaWVkIHVwc3RyZWFtIGtlcm5lbCAzLjYuMC1yYzEsIHNlZW1zIHdvcnNlLiBJdCB0
b29rIDIgaG91cnMgCnRvIGJvb3QgdXAuCj4KPiBOZXh0LCBpZiB5b3UgYWxyZWFkeSBzcG90dGVk
IHdoZXJlIHRoZSBzcGlubmluZyBvY2N1cnMsIHlvdQo+IHNob3VsZCBhbHNvIGJlIGFibGUgdG8g
dGVsbCB3aGF0J3MgZ29pbmcgb24gYXQgdGhlIG90aGVyIHNpZGUsIGkuZS4KPiB3aHkgdGhlIGV2
ZW50IHRoYXQgaXMgYmVpbmcgd2FpdGVkIGZvciBpc24ndCBvY2N1cnJpbmcgZm9yIHRoaXMKPiBs
b25nIGEgdGltZS4gU2luY2UgdGhlcmUncyBhIG51bWJlciBvZiBvcGVuIGNvZGVkIHNwaW4gbG9v
cHMKPiBoZXJlLCBrbm93aW5nIGV4YWN0bHkgd2hpY2ggb25lIGVhY2ggQ1BVIGlzIHNpdHRpbmcg
aW4gKGFuZAo+IHdoaWNoIG9uZXMgbWlnaHQgbm90IGJlIGluIGFueSkgaXMgdGhlIGZ1bmRhbWVu
dGFsIGluZm9ybWF0aW9uCj4gbmVlZGVkLgo+Cj4gIEZyb20gd2hhdCB5b3UncmUgdGVsbGluZyB1
cyBzbyBmYXIsIEknZCByYXRoZXIgc3VzcGVjdCBhIGtlcm5lbAo+IHByb2JsZW0sIG5vdCBhIGh5
cGVydmlzb3Igb25lIGhlcmUuClBlciBteSBmaW5kaW5nLCBtb3N0IG9mIHZjcHVzIHNwaW4gYXQg
c2V0X2F0b21pY2l0eV9sb2NrLiBTb21lIHNwaW4gYXQgCnN0b3BfbWFjaGluZSBhZnRlciBmaW5p
c2ggdGhlaXIgam9iLgpPbmx5IG9uZSB2Y3B1IGlzIGNhbGxpbmcgZ2VuZXJpY19zZXRfYWxsLgpJ
J20gbm90IHN1cmUgaWYgdGhlIHZjcHUgY2FsbGluZyBnZW5lcmljX3NldF9hbGwgZG9uJ3QgaGF2
ZSBoaWdoZXIgCnByaW9yaXR5IGFuZCBtYXliZSBwcmVlbXB0IGJ5IG90aGVyIHZjcHVzIGFuZCBk
b20wIGZyZXF1ZW50bHkuClRoaXMgd2FzdGUgbXVjaCB0aW1lLgo+Cj4gSmFuCj4KCl9fX19fX19f
X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fClhlbi1kZXZlbCBtYWlsaW5n
IGxpc3QKWGVuLWRldmVsQGxpc3RzLnhlbi5vcmcKaHR0cDovL2xpc3RzLnhlbi5vcmcveGVuLWRl
dmVsCg==

From mboxrd@z Thu Jan  1 00:00:00 1970
From: "Jan Beulich" 
Subject: Re: kernel bootup slow issue on ovm3.1.1
Date: Wed, 08 Aug 2012 15:43:27 +0100
Message-ID: <5022972F0200007800093A3C@nat28.tlf.novell.com>
References: <5020C24A.3060604@oracle.com>
	<20120807162637.GB15053@phenom.dumpdata.com>
	<50223027.6080502@oracle.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Return-path: 
In-Reply-To: <50223027.6080502@oracle.com>
Content-Disposition: inline
List-Unsubscribe: ,
	
List-Post: 
List-Help: 
List-Subscribe: ,
	
Sender: xen-devel-bounces@lists.xen.org
Errors-To: xen-devel-bounces@lists.xen.org
To: zhenzhong.duan@oracle.com
Cc: xen-devel@lists.xensource.com, Feng Jin , Konrad Rzeszutek Wilk 
List-Id: xen-devel@lists.xenproject.org

>>> On 08.08.12 at 11:23, "zhenzhong.duan"  wrote:
> I also send a patch to lkml that can workaround this issue, but I don't 
> know the reason of block in xen side.
> link: https://lkml.org/lkml/2012/8/7/50 

Without understanding the reason for this, I agree with hpa that
blindly changing the kernel to address this is not really a good
idea.

Jan

From mboxrd@z Thu Jan  1 00:00:00 1970
From: "Jan Beulich" 
Subject: Re: kernel bootup slow issue on ovm3.1.1
Date: Wed, 08 Aug 2012 15:47:31 +0100
Message-ID: <502298230200007800093A57@nat28.tlf.novell.com>
References: <5020C24A.3060604@oracle.com>
	<5020F003020000780009322C@nat28.tlf.novell.com>
	<502235E8.9040309@oracle.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
Return-path: 
In-Reply-To: <502235E8.9040309@oracle.com>
Content-Disposition: inline
List-Unsubscribe: ,
	
List-Post: 
List-Help: 
List-Subscribe: ,
	
Sender: xen-devel-bounces@lists.xen.org
Errors-To: xen-devel-bounces@lists.xen.org
To: zhenzhong.duan@oracle.com
Cc: Konrad Rzeszutek Wilk , Feng Jin , xen-devel 
List-Id: xen-devel@lists.xenproject.org

Pj4+IE9uIDA4LjA4LjEyIGF0IDExOjQ4LCAiemhlbnpob25nLmR1YW4iIDx6aGVuemhvbmcuZHVh
bkBvcmFjbGUuY29tPiB3cm90ZToKPiDkuo4gMjAxMi0wOC0wNyAxNjozNywgSmFuIEJldWxpY2gg
5YaZ6YGTOgo+Pj4+PiBPbiAwNy4wOC4xMiBhdCAwOToyMiwgInpoZW56aG9uZy5kdWFuIjx6aGVu
emhvbmcuZHVhbkBvcmFjbGUuY29tPiAgd3JvdGU6Cj4+PiBBZnRlciBzb21lIGRlYnVnLCB3ZSBm
b3VuZCBpdCdzIGluIGtlcm5lbCBtdHJyIGluaXQgdGhhdCBtYWtlIHRoaXMgZGVsYXkuCj4+Pgo+
Pj4gbXRycl9hcHNfaW5pdCgpCj4+PiAgIFwtPiAgc2V0X210cnIoKQo+Pj4gICAgICAgXC0+ICBt
dHJyX3dvcmtfaGFuZGxlcigpCj4+Pgo+Pj4ga2VybmVsIHNwaW4gaW4gbXRycl93b3JrX2hhbmRs
ZXIuCj4+Pgo+Pj4gQnV0IHdlIGRvbid0IGtub3cgdGhlIHNjZW5lIGhpZGUgaW4gdGhlIGh5cGVy
dmlzb3IuIFdoeSBiaWcgbWVtICsKPj4+IHBhc3N0aHJvdWdoIG1hZGUgdGhlIHdvcnN0IGNhc2Uu
Cj4+PiBJcyB0aGlzIGFscmVhZHkgZml4ZWQgaW4geGVuIHVwc3RyZWFtPwo+PiBGaXJzdCBvZiBh
bGwgaXQgd291bGQgaGF2ZSBiZWVuIHVzZWZ1bCB0byBpbmRpY2F0ZSB0aGUga2VybmVsIHZlcnNp
b24sCj4+IHNpbmNlIG10cnJfd29ya19oYW5kbGVyKCkgZGlzYXBwZWFyZWQgYWZ0ZXIgMy4wLiBP
YnZpb3VzbHkgd29ydGgKPj4gY2hlY2tpbmcgd2hldGhlciB0aGF0IGNoYW5nZSBieSBpdHNlbGYg
YWxyZWFkeSBhZGRyZXNzZXMgeW91cgo+PiBwcm9ibGVtLgo+IE5vIGx1Y2ssIHRyaWVkIHVwc3Ry
ZWFtIGtlcm5lbCAzLjYuMC1yYzEsIHNlZW1zIHdvcnNlLiBJdCB0b29rIDIgaG91cnMgCj4gdG8g
Ym9vdCB1cC4KClRoYXQncyBxdWl0ZSBhIGJpZyBzdGVwIGZyb20gMy4wLnguIEFuZCBpbiBhbm90
aGVyIHJlc3BvbnNlIHlvdQpwb2ludCBvdXQgdGhhdCAzLjYgaXMgd2F5IHdvcnNlIHRoYW4gMy41
IHdhcy4gU28gbWF5YmUgZ29pbmcKYmFjayB0byAzLjEgb3IgMy4yIG1pZ2h0IGJlIGEgYmV0dGVy
IGlkZWEgaWYgZGVidWdnaW5nIHRoZSBpc3N1ZQpkb2Vzbid0IGdldCB5b3UgYW55d2hlcmUuCgpK
YW4KCl9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fClhlbi1k
ZXZlbCBtYWlsaW5nIGxpc3QKWGVuLWRldmVsQGxpc3RzLnhlbi5vcmcKaHR0cDovL2xpc3RzLnhl
bi5vcmcveGVuLWRldmVsCg==

From mboxrd@z Thu Jan  1 00:00:00 1970
From: "Jan Beulich" 
Subject: Re: kernel bootup slow issue on ovm3.1.1
Date: Wed, 08 Aug 2012 16:01:56 +0100
Message-ID: <50229B840200007800093A73@nat28.tlf.novell.com>
References: <5020C24A.3060604@oracle.com>
	<5020F003020000780009322C@nat28.tlf.novell.com>
	<502235E8.9040309@oracle.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
Return-path: 
In-Reply-To: <502235E8.9040309@oracle.com>
Content-Disposition: inline
List-Unsubscribe: ,
	
List-Post: 
List-Help: 
List-Subscribe: ,
	
Sender: xen-devel-bounces@lists.xen.org
Errors-To: xen-devel-bounces@lists.xen.org
To: zhenzhong.duan@oracle.com
Cc: Konrad Rzeszutek Wilk , Feng Jin , xen-devel 
List-Id: xen-devel@lists.xenproject.org

Pj4+IE9uIDA4LjA4LjEyIGF0IDExOjQ4LCAiemhlbnpob25nLmR1YW4iIDx6aGVuemhvbmcuZHVh
bkBvcmFjbGUuY29tPiB3cm90ZToKPiDkuo4gMjAxMi0wOC0wNyAxNjozNywgSmFuIEJldWxpY2gg
5YaZ6YGTOgo+Pj4+PiBPbiAwNy4wOC4xMiBhdCAwOToyMiwgInpoZW56aG9uZy5kdWFuIjx6aGVu
emhvbmcuZHVhbkBvcmFjbGUuY29tPiAgd3JvdGU6Cj4+IE5leHQsIGlmIHlvdSBhbHJlYWR5IHNw
b3R0ZWQgd2hlcmUgdGhlIHNwaW5uaW5nIG9jY3VycywgeW91Cj4+IHNob3VsZCBhbHNvIGJlIGFi
bGUgdG8gdGVsbCB3aGF0J3MgZ29pbmcgb24gYXQgdGhlIG90aGVyIHNpZGUsIGkuZS4KPj4gd2h5
IHRoZSBldmVudCB0aGF0IGlzIGJlaW5nIHdhaXRlZCBmb3IgaXNuJ3Qgb2NjdXJyaW5nIGZvciB0
aGlzCj4+IGxvbmcgYSB0aW1lLiBTaW5jZSB0aGVyZSdzIGEgbnVtYmVyIG9mIG9wZW4gY29kZWQg
c3BpbiBsb29wcwo+PiBoZXJlLCBrbm93aW5nIGV4YWN0bHkgd2hpY2ggb25lIGVhY2ggQ1BVIGlz
IHNpdHRpbmcgaW4gKGFuZAo+PiB3aGljaCBvbmVzIG1pZ2h0IG5vdCBiZSBpbiBhbnkpIGlzIHRo
ZSBmdW5kYW1lbnRhbCBpbmZvcm1hdGlvbgo+PiBuZWVkZWQuCj4+Cj4+ICBGcm9tIHdoYXQgeW91
J3JlIHRlbGxpbmcgdXMgc28gZmFyLCBJJ2QgcmF0aGVyIHN1c3BlY3QgYSBrZXJuZWwKPj4gcHJv
YmxlbSwgbm90IGEgaHlwZXJ2aXNvciBvbmUgaGVyZS4KPiBQZXIgbXkgZmluZGluZywgbW9zdCBv
ZiB2Y3B1cyBzcGluIGF0IHNldF9hdG9taWNpdHlfbG9jay4KClRoZW4geW91IG5lZWQgdG8gZGV0
ZXJtaW5lIHdoYXQgdGhlIGN1cnJlbnQgb3duZXIgb2YgdGhlCmxvY2sgaXMgZG9pbmcuCgo+IFNv
bWUgc3BpbiBhdCBzdG9wX21hY2hpbmUgYWZ0ZXIgZmluaXNoIHRoZWlyIGpvYi4KCkFuZCBoZXJl
IHlvdSdkIG5lZWQgdG8gZmluZCBvdXQgd2hhdCB0aGV5J3JlIHdhaXRpbmcgZm9yLAphbmQgd2hh
dCB0aG9zZSBDUFVzIGFyZSBkb2luZy4KCj4gT25seSBvbmUgdmNwdSBpcyBjYWxsaW5nIGdlbmVy
aWNfc2V0X2FsbC4KPiBJJ20gbm90IHN1cmUgaWYgdGhlIHZjcHUgY2FsbGluZyBnZW5lcmljX3Nl
dF9hbGwgZG9uJ3QgaGF2ZSBoaWdoZXIgCj4gcHJpb3JpdHkgYW5kIG1heWJlIHByZWVtcHQgYnkg
b3RoZXIgdmNwdXMgYW5kIGRvbTAgZnJlcXVlbnRseS4KPiBUaGlzIHdhc3RlIG11Y2ggdGltZS4K
ClRoZXJlJ3Mgbm90IHRoYXQgbXVjaCBiZWluZyBkb25lIGluIGdlbmVyaWNfc2V0X2FsbCgpLCBz
byB0aGUKY29kZSBzaG91bGQgZmluaXNoIHJlYXNvbmFibHkgcXVpY2tseS4gQXJlIHlvdSBwZXJo
YXBzIGhhdmluZwptb3JlIHZDUFUtcyBpbiB0aGUgZ3Vlc3QgdGhhbiBwQ1BVLXMgdGhleSBjYW4g
cnVuIG9uPyBEb2VzCnlvdXIgaGFyZHdhcmUgc3VwcG9ydCBQYXVzZS1Mb29wLUV4aXRpbmcgKG9y
IHRoZSBBTUQKZXF1aXZhbGVudCwgZG9uJ3QgcmVjYWxsIHRoZWlyIHRlcm0gcmlnaHQgbm93KT8K
CkphbgoKX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KWGVu
LWRldmVsIG1haWxpbmcgbGlzdApYZW4tZGV2ZWxAbGlzdHMueGVuLm9yZwpodHRwOi8vbGlzdHMu
eGVuLm9yZy94ZW4tZGV2ZWwK

From mboxrd@z Thu Jan  1 00:00:00 1970
From: "zhenzhong.duan" 
Subject: Re: kernel bootup slow issue on ovm3.1.1
Date: Thu, 09 Aug 2012 17:42:38 +0800
Message-ID: <5023860E.7080908@oracle.com>
References: <5020C24A.3060604@oracle.com>
	<5020F003020000780009322C@nat28.tlf.novell.com>
	<502235E8.9040309@oracle.com>
	<50229B840200007800093A73@nat28.tlf.novell.com>
Reply-To: zhenzhong.duan@oracle.com
Mime-Version: 1.0
Content-Type: text/plain; charset="utf-8"; Format="flowed"
Content-Transfer-Encoding: base64
Return-path: 
In-Reply-To: <50229B840200007800093A73@nat28.tlf.novell.com>
List-Unsubscribe: ,
	
List-Post: 
List-Help: 
List-Subscribe: ,
	
Sender: xen-devel-bounces@lists.xen.org
Errors-To: xen-devel-bounces@lists.xen.org
To: Jan Beulich 
Cc: Konrad Rzeszutek Wilk , Feng Jin , xen-devel 
List-Id: xen-devel@lists.xenproject.org

Cgrkuo4gMjAxMi0wOC0wOCAyMzowMSwgSmFuIEJldWxpY2gg5YaZ6YGTOgo+Pj4+IE9uIDA4LjA4
LjEyIGF0IDExOjQ4LCAiemhlbnpob25nLmR1YW4iPHpoZW56aG9uZy5kdWFuQG9yYWNsZS5jb20+
ICB3cm90ZToKPj4g5LqOIDIwMTItMDgtMDcgMTY6MzcsIEphbiBCZXVsaWNoIOWGmemBkzoKPj4+
Pj4+IE9uIDA3LjA4LjEyIGF0IDA5OjIyLCAiemhlbnpob25nLmR1YW4iPHpoZW56aG9uZy5kdWFu
QG9yYWNsZS5jb20+ICAgd3JvdGU6Cj4+PiBOZXh0LCBpZiB5b3UgYWxyZWFkeSBzcG90dGVkIHdo
ZXJlIHRoZSBzcGlubmluZyBvY2N1cnMsIHlvdQo+Pj4gc2hvdWxkIGFsc28gYmUgYWJsZSB0byB0
ZWxsIHdoYXQncyBnb2luZyBvbiBhdCB0aGUgb3RoZXIgc2lkZSwgaS5lLgo+Pj4gd2h5IHRoZSBl
dmVudCB0aGF0IGlzIGJlaW5nIHdhaXRlZCBmb3IgaXNuJ3Qgb2NjdXJyaW5nIGZvciB0aGlzCj4+
PiBsb25nIGEgdGltZS4gU2luY2UgdGhlcmUncyBhIG51bWJlciBvZiBvcGVuIGNvZGVkIHNwaW4g
bG9vcHMKPj4+IGhlcmUsIGtub3dpbmcgZXhhY3RseSB3aGljaCBvbmUgZWFjaCBDUFUgaXMgc2l0
dGluZyBpbiAoYW5kCj4+PiB3aGljaCBvbmVzIG1pZ2h0IG5vdCBiZSBpbiBhbnkpIGlzIHRoZSBm
dW5kYW1lbnRhbCBpbmZvcm1hdGlvbgo+Pj4gbmVlZGVkLgo+Pj4KPj4+ICAgRnJvbSB3aGF0IHlv
dSdyZSB0ZWxsaW5nIHVzIHNvIGZhciwgSSdkIHJhdGhlciBzdXNwZWN0IGEga2VybmVsCj4+PiBw
cm9ibGVtLCBub3QgYSBoeXBlcnZpc29yIG9uZSBoZXJlLgo+PiBQZXIgbXkgZmluZGluZywgbW9z
dCBvZiB2Y3B1cyBzcGluIGF0IHNldF9hdG9taWNpdHlfbG9jay4KPiBUaGVuIHlvdSBuZWVkIHRv
IGRldGVybWluZSB3aGF0IHRoZSBjdXJyZW50IG93bmVyIG9mIHRoZQo+IGxvY2sgaXMgZG9pbmcu
CkkgYWRkIHByaW50ay50aW1lPTEgdG8ga2VybmVsIGNtZGxpbmUsIGJ1dCBkbWVzZyBkb24ndCBz
aG93IG11Y2ggaGVscC4KClsgICAgMS45Nzg3MDZdIHNtcGJvb3Q6IFRvdGFsIG9mIDI0IHByb2Nl
c3NvcnMgYWN0aXZhdGVkICgxNDA0NDkuMzQgCkJvZ29NSVBTKQooYmxvY2sgfjMwIG1pbnMpClsg
ICAgMS45ODg4NTldIGRldnRtcGZzOiBpbml0aWFsaXplZAo+Cj4+IFNvbWUgc3BpbiBhdCBzdG9w
X21hY2hpbmUgYWZ0ZXIgZmluaXNoIHRoZWlyIGpvYi4KPiBBbmQgaGVyZSB5b3UnZCBuZWVkIHRv
IGZpbmQgb3V0IHdoYXQgdGhleSdyZSB3YWl0aW5nIGZvciwKPiBhbmQgd2hhdCB0aG9zZSBDUFVz
IGFyZSBkb2luZy4KVGhleSBhcmUgd2FpdGluZyB0aGUgdmNwdSBjYWxsaW5nIGdlbmVyaWNfc2V0
X2FsbCBhbmQgdGhvc2Ugc3BpbiBhdCAKc2V0X2F0b21pY2l0eV9sb2NrLgpJbiBmYWN0LCBhbGwg
YXJlIHdhaXRpbmcgZ2VuZXJpY19zZXRfYWxsCj4KPj4gT25seSBvbmUgdmNwdSBpcyBjYWxsaW5n
IGdlbmVyaWNfc2V0X2FsbC4KPj4gSSdtIG5vdCBzdXJlIGlmIHRoZSB2Y3B1IGNhbGxpbmcgZ2Vu
ZXJpY19zZXRfYWxsIGRvbid0IGhhdmUgaGlnaGVyCj4+IHByaW9yaXR5IGFuZCBtYXliZSBwcmVl
bXB0IGJ5IG90aGVyIHZjcHVzIGFuZCBkb20wIGZyZXF1ZW50bHkuCj4+IFRoaXMgd2FzdGUgbXVj
aCB0aW1lLgo+IFRoZXJlJ3Mgbm90IHRoYXQgbXVjaCBiZWluZyBkb25lIGluIGdlbmVyaWNfc2V0
X2FsbCgpLCBzbyB0aGUKPiBjb2RlIHNob3VsZCBmaW5pc2ggcmVhc29uYWJseSBxdWlja2x5LiBB
cmUgeW91IHBlcmhhcHMgaGF2aW5nCj4gbW9yZSB2Q1BVLXMgaW4gdGhlIGd1ZXN0IHRoYW4gcENQ
VS1zIHRoZXkgY2FuIHJ1biBvbj8KU3lzdGVtIGVudiBpcyBhbiBleGFsb2dpYyBub2RlIHdpdGgg
MjQgY29yZXMgKyAxMDBHIG1lbSAoMiBzb2NrZXQgLCA2IApjb3JlcyBwZXIgc29ja2V0LCAyIEhU
IHRocmVhZHMgcGVyIGNvcmUpLgpCb290dXAgYSBwdmh2bSB3aXRoIDEydnBjdXMgKG9yIDI0KSAr
IDkwIEdCICsgcGNpIHBhc3N0aHJvdWdoZWQgZGV2aWNlLgo+ICAgRG9lcwo+IHlvdXIgaGFyZHdh
cmUgc3VwcG9ydCBQYXVzZS1Mb29wLUV4aXRpbmcgKG9yIHRoZSBBTUQKPiBlcXVpdmFsZW50LCBk
b24ndCByZWNhbGwgdGhlaXIgdGVybSByaWdodCBub3cpPwpJIGhhdmUgbm8gYWNjZXNzIHRvIHNl
cmlhbCBsaW5lLCBjb3VsZCBJIGdldCB0aGUgaW5mbyBieSBhIGNvbW1hbmQ/Ci9wcm9jL2NwdWlu
Zm8gc2hvd3MgYmVsb3c6CmNwdSBmYW1pbHkgICAgICA6IDYKbW9kZWwgICAgICAgICAgIDogNDQK
bW9kZWwgbmFtZSAgICAgIDogSW50ZWwoUikgWGVvbihSKSBDUFUgICAgICAgICAgIFg1NjcwICBA
IDIuOTNHSHoKPgo+IEphbgoKX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f
X19fX19fX18KWGVuLWRldmVsIG1haWxpbmcgbGlzdApYZW4tZGV2ZWxAbGlzdHMueGVuLm9yZwpo
dHRwOi8vbGlzdHMueGVuLm9yZy94ZW4tZGV2ZWwK

From mboxrd@z Thu Jan  1 00:00:00 1970
From: "Jan Beulich" 
Subject: Re: kernel bootup slow issue on ovm3.1.1
Date: Thu, 09 Aug 2012 11:35:34 +0100
Message-ID: <5023AE960200007800093DE8@nat28.tlf.novell.com>
References: <5020C24A.3060604@oracle.com>
	<5020F003020000780009322C@nat28.tlf.novell.com>
	<502235E8.9040309@oracle.com>
	<50229B840200007800093A73@nat28.tlf.novell.com>
	<5023860E.7080908@oracle.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
Return-path: 
In-Reply-To: <5023860E.7080908@oracle.com>
Content-Disposition: inline
List-Unsubscribe: ,
	
List-Post: 
List-Help: 
List-Subscribe: ,
	
Sender: xen-devel-bounces@lists.xen.org
Errors-To: xen-devel-bounces@lists.xen.org
To: zhenzhong.duan@oracle.com
Cc: Konrad Rzeszutek Wilk , Feng Jin , xen-devel 
List-Id: xen-devel@lists.xenproject.org

Pj4+IE9uIDA5LjA4LjEyIGF0IDExOjQyLCAiemhlbnpob25nLmR1YW4iIDx6aGVuemhvbmcuZHVh
bkBvcmFjbGUuY29tPiB3cm90ZToKPiDkuo4gMjAxMi0wOC0wOCAyMzowMSwgSmFuIEJldWxpY2gg
5YaZ6YGTOgo+Pj4+PiBPbiAwOC4wOC4xMiBhdCAxMTo0OCwgInpoZW56aG9uZy5kdWFuIjx6aGVu
emhvbmcuZHVhbkBvcmFjbGUuY29tPiAgd3JvdGU6Cj4+PiDkuo4gMjAxMi0wOC0wNyAxNjozNywg
SmFuIEJldWxpY2gg5YaZ6YGTOgo+Pj4gU29tZSBzcGluIGF0IHN0b3BfbWFjaGluZSBhZnRlciBm
aW5pc2ggdGhlaXIgam9iLgo+PiBBbmQgaGVyZSB5b3UnZCBuZWVkIHRvIGZpbmQgb3V0IHdoYXQg
dGhleSdyZSB3YWl0aW5nIGZvciwKPj4gYW5kIHdoYXQgdGhvc2UgQ1BVcyBhcmUgZG9pbmcuCj4g
VGhleSBhcmUgd2FpdGluZyB0aGUgdmNwdSBjYWxsaW5nIGdlbmVyaWNfc2V0X2FsbCBhbmQgdGhv
c2Ugc3BpbiBhdCAKPiBzZXRfYXRvbWljaXR5X2xvY2suCj4gSW4gZmFjdCwgYWxsIGFyZSB3YWl0
aW5nIGdlbmVyaWNfc2V0X2FsbAoKSSB0aGluayB3ZSdyZSBtb3ZpbmcgaW4gY2lyY2xlcyAtIHdo
YXQgaXMgdGhlIHZDUFUgY3VycmVudGx5CmdlbmVyaWNfc2V0X2FsbCgpIHRoZW4gZG9pbmc/Cgo+
PiBUaGVyZSdzIG5vdCB0aGF0IG11Y2ggYmVpbmcgZG9uZSBpbiBnZW5lcmljX3NldF9hbGwoKSwg
c28gdGhlCj4+IGNvZGUgc2hvdWxkIGZpbmlzaCByZWFzb25hYmx5IHF1aWNrbHkuIEFyZSB5b3Ug
cGVyaGFwcyBoYXZpbmcKPj4gbW9yZSB2Q1BVLXMgaW4gdGhlIGd1ZXN0IHRoYW4gcENQVS1zIHRo
ZXkgY2FuIHJ1biBvbj8KPiBTeXN0ZW0gZW52IGlzIGFuIGV4YWxvZ2ljIG5vZGUgd2l0aCAyNCBj
b3JlcyArIDEwMEcgbWVtICgyIHNvY2tldCAsIDYgCj4gY29yZXMgcGVyIHNvY2tldCwgMiBIVCB0
aHJlYWRzIHBlciBjb3JlKS4KPiBCb290dXAgYSBwdmh2bSB3aXRoIDEydnBjdXMgKG9yIDI0KSAr
IDkwIEdCICsgcGNpIHBhc3N0aHJvdWdoZWQgZGV2aWNlLgoKU28geW91J3JlIGluZGVlZCBvdmVy
LWNvbW1pdHRpbmcgdGhlIHN5c3RlbS4gSG93IG1hbnkgdkNQVS1zCmRvZXMgeW91IERvbTAgaGF2
ZT8gQXJlIHRoZXJlIGFueSBvdGhlciBWTXM/IElzIHRoZXJlIGFueQp2Q1BVIHBpbm5pbmcgaW4g
ZWZmZWN0PwoKPj4gICBEb2VzCj4+IHlvdXIgaGFyZHdhcmUgc3VwcG9ydCBQYXVzZS1Mb29wLUV4
aXRpbmcgKG9yIHRoZSBBTUQKPj4gZXF1aXZhbGVudCwgZG9uJ3QgcmVjYWxsIHRoZWlyIHRlcm0g
cmlnaHQgbm93KT8KPiBJIGhhdmUgbm8gYWNjZXNzIHRvIHNlcmlhbCBsaW5lLCBjb3VsZCBJIGdl
dCB0aGUgaW5mbyBieSBhIGNvbW1hbmQ/CgoieGwgZG1lc2ciIHJ1biBlYXJseSBlbm91Z2ggKGku
ZS4gYmVmb3JlIHRoZSBsb2cgYnVmZmVyIHdyYXBzKS4KCkphbgoKX19fX19fX19fX19fX19fX19f
X19fX19fX19fX19fX19fX19fX19fX19fX19fX18KWGVuLWRldmVsIG1haWxpbmcgbGlzdApYZW4t
ZGV2ZWxAbGlzdHMueGVuLm9yZwpodHRwOi8vbGlzdHMueGVuLm9yZy94ZW4tZGV2ZWwK

From mboxrd@z Thu Jan  1 00:00:00 1970
From: "zhenzhong.duan" 
Subject: Re: kernel bootup slow issue on ovm3.1.1
Date: Fri, 10 Aug 2012 12:40:07 +0800
Message-ID: <502490A7.7020603@oracle.com>
References: <5020C24A.3060604@oracle.com>
	<5020F003020000780009322C@nat28.tlf.novell.com>
	<502235E8.9040309@oracle.com>
	<50229B840200007800093A73@nat28.tlf.novell.com>
	<5023860E.7080908@oracle.com>
	<5023AE960200007800093DE8@nat28.tlf.novell.com>
Reply-To: zhenzhong.duan@oracle.com
Mime-Version: 1.0
Content-Type: multipart/mixed; boundary="===============2070227487450461743=="
Return-path: 
In-Reply-To: <5023AE960200007800093DE8@nat28.tlf.novell.com>
List-Unsubscribe: ,
	
List-Post: 
List-Help: 
List-Subscribe: ,
	
Sender: xen-devel-bounces@lists.xen.org
Errors-To: xen-devel-bounces@lists.xen.org
To: Jan Beulich 
Cc: Konrad Rzeszutek Wilk , Feng Jin , xen-devel 
List-Id: xen-devel@lists.xenproject.org

This is a multi-part message in MIME format.
--===============2070227487450461743==
Content-Type: multipart/alternative;
 boundary="------------090605070704070006080803"

This is a multi-part message in MIME format.
--------------090605070704070006080803
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: quoted-printable



=E4=BA=8E 2012-08-09 18:35, Jan Beulich =E5=86=99=E9=81=93:
>>>> On 09.08.12 at 11:42, "zhenzhong.duan"  w=
rote:
>> =E4=BA=8E 2012-08-08 23:01, Jan Beulich =E5=86=99=E9=81=93:
>>>>>> On 08.08.12 at 11:48, "zhenzhong.duan" =
  wrote:
>>>> =E4=BA=8E 2012-08-07 16:37, Jan Beulich =E5=86=99=E9=81=93:
>>>> Some spin at stop_machine after finish their job.
>>> And here you'd need to find out what they're waiting for,
>>> and what those CPUs are doing.
>> They are waiting the vcpu calling generic_set_all and those spin at
>> set_atomicity_lock.
>> In fact, all are waiting generic_set_all
> I think we're moving in circles - what is the vCPU currently
> generic_set_all() then doing?
Add some debug print, generic_set_all->prepare_set->write_cr0 took much=20
time,
all else are quick. set_atomicity_lock serialized this process between=20
cpus, make it worse.
One iteration:
MTRR: CPU 2
prepare_set: before read_cr0
prepare_set: before write_cr0 ------*block here*
prepare_set: before wbinvd
prepare_set: before read_cr4
prepare_set: before write_cr4
prepare_set: before __flush_tlb
prepare_set: before rdmsr
prepare_set: before wrmsr
generic_set_all: before set_mtrr_state
generic_set_all: before pat_init
post_set: before wbinvd
post_set: before wrmsr
post_set: before write_cr0
post_set: before write_cr4

>
>>> There's not that much being done in generic_set_all(), so the
>>> code should finish reasonably quickly. Are you perhaps having
>>> more vCPU-s in the guest than pCPU-s they can run on?
>> System env is an exalogic node with 24 cores + 100G mem (2 socket , 6
>> cores per socket, 2 HT threads per core).
>> Bootup a pvhvm with 12vpcus (or 24) + 90 GB + pci passthroughed device=
=2E
> So you're indeed over-committing the system. How many vCPU-s
> does you Dom0 have? Are there any other VMs? Is there any
> vCPU pinning in effect?
dom0 boot with 24 vcpus(same result with dom0_max_vcpus=3D4). No other vm=
=20
except dom0. All 24 vcpus spin from xentop result. Below is xentop clip.

       NAME  STATE   CPU(sec) CPU(%)     MEM(k) MEM(%)  MAXMEM(k)=20
MAXMEM(%) VCPUS NETS NETTX(k) NETRX(k) VBDS   VBD_OO   VBD_RD   VBD_WR =20
VBD_RSECT  VBD_WSECT
  SSID
   Domain-0 -----r      43072  158.8    2050560    2.0   no limit      =20
n/a    24    0        0        0    0        0        0       =20
0          0          0
     0
VCPUs(sec):   0:      13649s  1:       6197s  2:       4254s  3:      =20
2006s  4:       1409s
           5:        930s  6:        698s  7:        630s  8:       =20
612s  9:       2038s
          10:        544s 11:        940s 12:        556s 13:       =20
510s 14:        456s
          15:        591s 16:        438s 17:        508s 18:      =20
3350s 19:        512s
          20:        544s 21:        529s 22:        547s 23:        610s=

zduan_test -----r      13140 2234.4   92327920   91.7   92327936     =20
91.7    24    1        0        0    1        0        0       =20
0          0          0
     0
VCPUs(sec):   0:        556s  1:        551s  2:        549s  3:       =20
544s  4:        549s
           5:        545s  6:        545s  7:        547s  8:       =20
545s  9:        548s
          10:        545s 11:        546s 12:        545s 13:       =20
548s 14:        543s
          15:        544s 16:        551s 17:        545s 18:       =20
547s 19:        551s
          20:        544s 21:        549s 22:        546s 23:        545s=

>>>    Does
>>> your hardware support Pause-Loop-Exiting (or the AMD
>>> equivalent, don't recall their term right now)?
>> I have no access to serial line, could I get the info by a command?
> "xl dmesg" run early enough (i.e. before the log buffer wraps).
Below is xl dmesg result for your reference. thanks
[root@scae02cn01 zduan]# xl dmesg
  __  __            _  _    ___   ____      _____     ____  __
  \ \/ /___ _ __   | || |  / _ \ |___ \    / _ \ \   / /  \/  |
   \  // _ \ '_ \  | || |_| | | |  __) |__| | | \ \ / /| |\/| |
   /  \  __/ | | | |__   _| |_| | / __/|__| |_| |\ V / | |  | |
  /_/\_\___|_| |_|    |_|(_)___(_)_____|   \___/  \_/  |_|  |_|

(XEN) Xen version 4.0.2-OVM (mockbuild@(none)) (gcc version 4.1.2=20
20080704 (Red Hat 4.1.2-48)) Fri Dec 23 17:00:16 EST 2011
(XEN) Latest ChangeSet: unavailable
(XEN) Bootloader: GNU GRUB 0.97
(XEN) Command line: dom0_mem=3D2G
(XEN) Video information:
(XEN)  VGA is text mode 80x25, font 8x16
(XEN)  VBE/DDC methods: none; EDID transfer time: 1 seconds
(XEN)  EDID info not retrieved because no DDC retrieval method detected
(XEN) Disc information:
(XEN)  Found 1 MBR signatures
(XEN)  Found 1 EDD information structures
(XEN) Xen-e820 RAM map:
(XEN)  0000000000000000 - 0000000000099400 (usable)
(XEN)  0000000000099400 - 00000000000a0000 (reserved)
(XEN)  00000000000e0000 - 0000000000100000 (reserved)
(XEN)  0000000000100000 - 000000007f780000 (usable)
(XEN)  000000007f78e000 - 000000007f790000 type 9
(XEN)  000000007f790000 - 000000007f79e000 (ACPI data)
(XEN)  000000007f79e000 - 000000007f7d0000 (ACPI NVS)
(XEN)  000000007f7d0000 - 000000007f7e0000 (reserved)
(XEN)  000000007f7ec000 - 0000000080000000 (reserved)
(XEN)  00000000e0000000 - 00000000f0000000 (reserved)
(XEN)  00000000fee00000 - 00000000fee01000 (reserved)
(XEN)  00000000ffc00000 - 0000000100000000 (reserved)
(XEN)  0000000100000000 - 0000001880000000 (usable)
(XEN) ACPI: RSDP 000FAA40, 0024 (r2 SUN   )
(XEN) ACPI: XSDT 7F790100, 0094 (r1 SUN    Xxx70    20111011 MSFT       9=
7)
(XEN) ACPI: FACP 7F790290, 00F4 (r4 SUN    Xxx70    20111011 MSFT       9=
7)
(XEN) ACPI: DSDT 7F7905C0, 5ECF (r2 SUN    Xxx70           1 INTL 2005111=
7)
(XEN) ACPI: FACS 7F79E000, 0040
(XEN) ACPI: APIC 7F790390, 011E (r2 SUN    Xxx70    20111011 MSFT       9=
7)
(XEN) ACPI: MCFG 7F790500, 003C (r1 SUN    Xxx70    20111011 MSFT       9=
7)
(XEN) ACPI: SLIT 7F790540, 0030 (r1 SUN    Xxx70    20111011 MSFT       9=
7)
(XEN) ACPI: SPMI 7F790570, 0041 (r5 SUN    Xxx70    20111011 MSFT       9=
7)
(XEN) ACPI: OEMB 7F79E040, 00BE (r1 SUN    Xxx70    20111011 MSFT       9=
7)
(XEN) ACPI: HPET 7F79A5C0, 0038 (r1 SUN    Xxx70    20111011 MSFT       9=
7)
(XEN) ACPI: DMAR 7F79E100, 0130 (r1 SUN    Xxx70           1 MSFT       9=
7)
(XEN) ACPI: SRAT 7F79A600, 0250 (r1 SUN    Xxx70           1 INTC        =
1)
(XEN) ACPI: SSDT 7F79EF60, 0363 (r1  SUN   Xxx70          12 INTL 2005111=
7)
(XEN) ACPI: EINJ 7F79A850, 0130 (r1 SUN    Xxx70    20111011 MSFT       9=
7)
(XEN) ACPI: BERT 7F79A9E0, 0030 (r1 SUN    Xxx70    20111011 MSFT       9=
7)
(XEN) ACPI: ERST 7F79AA10, 01B0 (r1 SUN    Xxx70    20111011 MSFT       9=
7)
(XEN) ACPI: HEST 7F79ABC0, 00A8 (r1 SUN    Xxx70    20111011 MSFT       9=
7)
(XEN) System RAM: 98295MB (100654180kB)
(XEN) Domain heap initialised DMA width 32 bits
(XEN) Processor #0 6:12 APIC version 21
(XEN) Processor #2 6:12 APIC version 21
(XEN) Processor #4 6:12 APIC version 21
(XEN) Processor #16 6:12 APIC version 21
(XEN) Processor #18 6:12 APIC version 21
(XEN) Processor #20 6:12 APIC version 21
(XEN) Processor #32 6:12 APIC version 21
(XEN) Processor #34 6:12 APIC version 21
(XEN) Processor #36 6:12 APIC version 21
(XEN) Processor #48 6:12 APIC version 21
(XEN) Processor #50 6:12 APIC version 21
(XEN) Processor #52 6:12 APIC version 21
(XEN) Processor #1 6:12 APIC version 21
(XEN) Processor #3 6:12 APIC version 21
(XEN) Processor #5 6:12 APIC version 21
(XEN) Processor #17 6:12 APIC version 21
(XEN) Processor #19 6:12 APIC version 21
(XEN) Processor #21 6:12 APIC version 21
(XEN) Processor #33 6:12 APIC version 21
(XEN) Processor #35 6:12 APIC version 21
(XEN) Processor #37 6:12 APIC version 21
(XEN) Processor #49 6:12 APIC version 21
(XEN) Processor #51 6:12 APIC version 21
(XEN) Processor #53 6:12 APIC version 21
(XEN) IOAPIC[0]: apic_id 6, version 32, address 0xfec00000, GSI 0-23
(XEN) IOAPIC[1]: apic_id 7, version 32, address 0xfec8a000, GSI 24-47
(XEN) Enabling APIC mode:  Phys.  Using 2 I/O APICs
(XEN) Using scheduler: SMP Credit Scheduler (credit)
(XEN) Detected 2926.029 MHz processor.
(XEN) Initing memory sharing.
(XEN) VMX: Supported advanced features:
(XEN)  - APIC MMIO access virtualisation
(XEN)  - APIC TPR shadow
(XEN)  - Extended Page Tables (EPT)
(XEN)  - Virtual-Processor Identifiers (VPID)
(XEN)  - Virtual NMI
(XEN)  - MSR direct-access bitmap
(XEN)  - Unrestricted Guest
(XEN) EPT supports 2MB super page.
(XEN) HVM: ASIDs enabled.
(XEN) HVM: VMX enabled
(XEN) HVM: Hardware Assisted Paging detected.
(XEN) Intel VT-d Snoop Control enabled.
(XEN) Intel VT-d Dom0 DMA Passthrough not enabled.
(XEN) Intel VT-d Queued Invalidation enabled.
(XEN) Intel VT-d Interrupt Remapping enabled.
(XEN) I/O virtualisation enabled
(XEN)  - Dom0 mode: Relaxed
(XEN) Enabled directed EOI with ioapic_ack_old on!
(XEN) Total of 24 processors activated.
(XEN) ENABLING IO-APIC IRQs
(XEN)  -> Using old ACK method
(XEN) TSC is reliable, synchronization unnecessary
(XEN) Platform timer is 14.318MHz HPET
(XEN) Allocated console ring of 64 KiB.
(XEN) Brought up 24 CPUs
(XEN) *** LOADING DOMAIN 0 ***
(XEN)  Xen  kernel: 64-bit, lsb, compat32
(XEN)  Dom0 kernel: 64-bit, lsb, paddr 0x2000 -> 0x6d5000
(XEN) PHYSICAL MEMORY ARRANGEMENT:
(XEN)  Dom0 alloc.:   0000000835000000->0000000836000000 (520192 pages=20
to be allocated)
(XEN) VIRTUAL MEMORY ARRANGEMENT:
(XEN)  Loaded kernel: ffffffff80002000->ffffffff806d5000
(XEN)  Init. ramdisk: ffffffff806d5000->ffffffff80ed7400
(XEN)  Phys-Mach map: ffffea0000000000->ffffea0000400000
(XEN)  Start info:    ffffffff80ed8000->ffffffff80ed84b4
(XEN)  Page tables:   ffffffff80ed9000->ffffffff80ee4000
(XEN)  Boot stack:    ffffffff80ee4000->ffffffff80ee5000
(XEN)  TOTAL:         ffffffff80000000->ffffffff81000000
(XEN)  ENTRY ADDRESS: ffffffff80002000
(XEN) Dom0 has maximum 24 VCPUs
(XEN) Scrubbing Free RAM:=20
=2E......................................................................=
=2E......................................................................=
=2E......................................................................=
=2E......................................................................=
=2E......................................................................=
=2E......................................................................=
=2E......................................................................=
=2E......................................................................=
=2E......................................................................=
=2E......................................................................=
=2E......................................................................=
=2E......................................................................=
=2E......................................................................=
=2E.........................done.
(XEN) Xen trace buffers: disabled
(XEN) Std. Loglevel: Errors and warnings
(XEN) Guest Loglevel: Nothing (Rate-limited: Errors and warnings)
(XEN) Xen is relinquishing VGA console.
(XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch=20
input to Xen)
(XEN) Freed 168kB init memory.


--------------090605070704070006080803
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: 8bit


  
    
  
  
    

    

    于 2012-08-09 18:35, Jan Beulich 写道:
    

      

        

          

            
On 09.08.12 at 11:42, "zhenzhong.duan" <zhenzhong.duan@oracle.com> wrote:


          

        

        
于 2012-08-08 23:01, Jan Beulich 写道:


        

          

            

              

                
On 08.08.12 at 11:48, "zhenzhong.duan"<zhenzhong.duan@oracle.com>  wrote:


              

            

            
于 2012-08-07 16:37, Jan Beulich 写道:
Some spin at stop_machine after finish their job.


          

          
And here you'd need to find out what they're waiting for,
and what those CPUs are doing.


        

        
They are waiting the vcpu calling generic_set_all and those spin at 
set_atomicity_lock.
In fact, all are waiting generic_set_all


      

      
I think we're moving in circles - what is the vCPU currently
generic_set_all() then doing?

    

    Add some debug print,
      generic_set_all->prepare_set->write_cr0 took much time,

      all else are quick. set_atomicity_lock serialized this process
      between cpus, make it worse.

      One iteration:

      MTRR: CPU 2

      prepare_set: before read_cr0

      prepare_set: before write_cr0    ------block
        here

      prepare_set: before wbinvd

      prepare_set: before read_cr4

      prepare_set: before write_cr4

      prepare_set: before __flush_tlb

      prepare_set: before rdmsr

      prepare_set: before wrmsr

      generic_set_all: before set_mtrr_state

      generic_set_all: before pat_init

      post_set: before wbinvd

      post_set: before wrmsr

      post_set: before write_cr0

      post_set: before write_cr4

      

    
    

      



      

        

          
There's not that much being done in generic_set_all(), so the
code should finish reasonably quickly. Are you perhaps having
more vCPU-s in the guest than pCPU-s they can run on?


        

        
System env is an exalogic node with 24 cores + 100G mem (2 socket , 6 
cores per socket, 2 HT threads per core).
Bootup a pvhvm with 12vpcus (or 24) + 90 GB + pci passthroughed device.


      

      
So you're indeed over-committing the system. How many vCPU-s
does you Dom0 have? Are there any other VMs? Is there any
vCPU pinning in effect?

    

    dom0 boot with 24 vcpus(same result with dom0_max_vcpus=4). No
      other vm except dom0. All 24 vcpus spin from xentop result. Below
      is xentop clip.

      

            NAME  STATE   CPU(sec) CPU(%)     MEM(k) MEM(%)  MAXMEM(k)
      MAXMEM(%) VCPUS NETS NETTX(k) NETRX(k) VBDS   VBD_OO   VBD_RD  
      VBD_WR  VBD_RSECT  VBD_WSECT

       SSID

        Domain-0 -----r      43072  158.8    2050560    2.0   no
      limit       n/a    24    0        0        0    0        0       
      0        0          0          0

          0

      VCPUs(sec):   0:      13649s  1:       6197s  2:       4254s 
      3:       2006s  4:       1409s

                5:        930s  6:        698s  7:        630s 
      8:        612s  9:       2038s

               10:        544s 11:        940s 12:        556s
      13:        510s 14:        456s

               15:        591s 16:        438s 17:        508s 18:      
      3350s 19:        512s

               20:        544s 21:        529s 22:        547s
      23:        610s

      zduan_test -----r      13140 2234.4   92327920   91.7  
      92327936      91.7    24    1        0        0    1       
      0        0        0          0          0

          0

      VCPUs(sec):   0:        556s  1:        551s  2:        549s 
      3:        544s  4:        549s

                5:        545s  6:        545s  7:        547s 
      8:        545s  9:        548s

               10:        545s 11:        546s 12:        545s
      13:        548s 14:        543s

               15:        544s 16:        551s 17:        545s
      18:        547s 19:        551s

               20:        544s 21:        549s 22:        546s
      23:        545s

    

      

        

          
  Does
your hardware support Pause-Loop-Exiting (or the AMD
equivalent, don't recall their term right now)?


        

        
I have no access to serial line, could I get the info by a command?


      

      
"xl dmesg" run early enough (i.e. before the log buffer wraps).

    

    Below is xl dmesg result for your reference. thanks

    [root@scae02cn01 zduan]# xl dmesg

       __  __            _  _    ___   ____      _____     ____  __

       \ \/ /___ _ __   | || |  / _ \ |___ \    / _ \ \   / /  \/  |

        \  // _ \ '_ \  | || |_| | | |  __) |__| | | \ \ / /| |\/| |

        /  \  __/ | | | |__   _| |_| | / __/|__| |_| |\ V / | |  | |

       /_/\_\___|_| |_|    |_|(_)___(_)_____|   \___/  \_/  |_|  |_|

      

      (XEN) Xen version 4.0.2-OVM (mockbuild@(none)) (gcc version 4.1.2
      20080704 (Red Hat 4.1.2-48)) Fri Dec 23 17:00:16 EST 2011

      (XEN) Latest ChangeSet: unavailable

      (XEN) Bootloader: GNU GRUB 0.97

      (XEN) Command line: dom0_mem=2G

      (XEN) Video information:

      (XEN)  VGA is text mode 80x25, font 8x16

      (XEN)  VBE/DDC methods: none; EDID transfer time: 1 seconds

      (XEN)  EDID info not retrieved because no DDC retrieval method
      detected

      (XEN) Disc information:

      (XEN)  Found 1 MBR signatures

      (XEN)  Found 1 EDD information structures

      (XEN) Xen-e820 RAM map:

      (XEN)  0000000000000000 - 0000000000099400 (usable)

      (XEN)  0000000000099400 - 00000000000a0000 (reserved)

      (XEN)  00000000000e0000 - 0000000000100000 (reserved)

      (XEN)  0000000000100000 - 000000007f780000 (usable)

      (XEN)  000000007f78e000 - 000000007f790000 type 9

      (XEN)  000000007f790000 - 000000007f79e000 (ACPI data)

      (XEN)  000000007f79e000 - 000000007f7d0000 (ACPI NVS)

      (XEN)  000000007f7d0000 - 000000007f7e0000 (reserved)

      (XEN)  000000007f7ec000 - 0000000080000000 (reserved)

      (XEN)  00000000e0000000 - 00000000f0000000 (reserved)

      (XEN)  00000000fee00000 - 00000000fee01000 (reserved)

      (XEN)  00000000ffc00000 - 0000000100000000 (reserved)

      (XEN)  0000000100000000 - 0000001880000000 (usable)

      (XEN) ACPI: RSDP 000FAA40, 0024 (r2 SUN   )

      (XEN) ACPI: XSDT 7F790100, 0094 (r1 SUN    Xxx70    20111011
      MSFT       97)

      (XEN) ACPI: FACP 7F790290, 00F4 (r4 SUN    Xxx70    20111011
      MSFT       97)

      (XEN) ACPI: DSDT 7F7905C0, 5ECF (r2 SUN    Xxx70           1 INTL
      20051117)

      (XEN) ACPI: FACS 7F79E000, 0040

      (XEN) ACPI: APIC 7F790390, 011E (r2 SUN    Xxx70    20111011
      MSFT       97)

      (XEN) ACPI: MCFG 7F790500, 003C (r1 SUN    Xxx70    20111011
      MSFT       97)

      (XEN) ACPI: SLIT 7F790540, 0030 (r1 SUN    Xxx70    20111011
      MSFT       97)

      (XEN) ACPI: SPMI 7F790570, 0041 (r5 SUN    Xxx70    20111011
      MSFT       97)

      (XEN) ACPI: OEMB 7F79E040, 00BE (r1 SUN    Xxx70    20111011
      MSFT       97)

      (XEN) ACPI: HPET 7F79A5C0, 0038 (r1 SUN    Xxx70    20111011
      MSFT       97)

      (XEN) ACPI: DMAR 7F79E100, 0130 (r1 SUN    Xxx70           1
      MSFT       97)

      (XEN) ACPI: SRAT 7F79A600, 0250 (r1 SUN    Xxx70           1
      INTC        1)

      (XEN) ACPI: SSDT 7F79EF60, 0363 (r1  SUN   Xxx70          12 INTL
      20051117)

      (XEN) ACPI: EINJ 7F79A850, 0130 (r1 SUN    Xxx70    20111011
      MSFT       97)

      (XEN) ACPI: BERT 7F79A9E0, 0030 (r1 SUN    Xxx70    20111011
      MSFT       97)

      (XEN) ACPI: ERST 7F79AA10, 01B0 (r1 SUN    Xxx70    20111011
      MSFT       97)

      (XEN) ACPI: HEST 7F79ABC0, 00A8 (r1 SUN    Xxx70    20111011
      MSFT       97)

      (XEN) System RAM: 98295MB (100654180kB)

      (XEN) Domain heap initialised DMA width 32 bits

      (XEN) Processor #0 6:12 APIC version 21

      (XEN) Processor #2 6:12 APIC version 21

      (XEN) Processor #4 6:12 APIC version 21

      (XEN) Processor #16 6:12 APIC version 21

      (XEN) Processor #18 6:12 APIC version 21

      (XEN) Processor #20 6:12 APIC version 21

      (XEN) Processor #32 6:12 APIC version 21

      (XEN) Processor #34 6:12 APIC version 21

      (XEN) Processor #36 6:12 APIC version 21

      (XEN) Processor #48 6:12 APIC version 21

      (XEN) Processor #50 6:12 APIC version 21

      (XEN) Processor #52 6:12 APIC version 21

      (XEN) Processor #1 6:12 APIC version 21

      (XEN) Processor #3 6:12 APIC version 21

      (XEN) Processor #5 6:12 APIC version 21

      (XEN) Processor #17 6:12 APIC version 21

      (XEN) Processor #19 6:12 APIC version 21

      (XEN) Processor #21 6:12 APIC version 21

      (XEN) Processor #33 6:12 APIC version 21

      (XEN) Processor #35 6:12 APIC version 21

      (XEN) Processor #37 6:12 APIC version 21

      (XEN) Processor #49 6:12 APIC version 21

      (XEN) Processor #51 6:12 APIC version 21

      (XEN) Processor #53 6:12 APIC version 21

      (XEN) IOAPIC[0]: apic_id 6, version 32, address 0xfec00000, GSI
      0-23

      (XEN) IOAPIC[1]: apic_id 7, version 32, address 0xfec8a000, GSI
      24-47

      (XEN) Enabling APIC mode:  Phys.  Using 2 I/O APICs

      (XEN) Using scheduler: SMP Credit Scheduler (credit)

      (XEN) Detected 2926.029 MHz processor.

      (XEN) Initing memory sharing.

      (XEN) VMX: Supported advanced features:

      (XEN)  - APIC MMIO access virtualisation

      (XEN)  - APIC TPR shadow

      (XEN)  - Extended Page Tables (EPT)

      (XEN)  - Virtual-Processor Identifiers (VPID)

      (XEN)  - Virtual NMI

      (XEN)  - MSR direct-access bitmap

      (XEN)  - Unrestricted Guest

      (XEN) EPT supports 2MB super page.

      (XEN) HVM: ASIDs enabled.

      (XEN) HVM: VMX enabled

      (XEN) HVM: Hardware Assisted Paging detected.

      (XEN) Intel VT-d Snoop Control enabled.

      (XEN) Intel VT-d Dom0 DMA Passthrough not enabled.

      (XEN) Intel VT-d Queued Invalidation enabled.

      (XEN) Intel VT-d Interrupt Remapping enabled.

      (XEN) I/O virtualisation enabled

      (XEN)  - Dom0 mode: Relaxed

      (XEN) Enabled directed EOI with ioapic_ack_old on!

      (XEN) Total of 24 processors activated.

      (XEN) ENABLING IO-APIC IRQs

      (XEN)  -> Using old ACK method

      (XEN) TSC is reliable, synchronization unnecessary

      (XEN) Platform timer is 14.318MHz HPET

      (XEN) Allocated console ring of 64 KiB.

      (XEN) Brought up 24 CPUs

      (XEN) *** LOADING DOMAIN 0 ***

      (XEN)  Xen  kernel: 64-bit, lsb, compat32

      (XEN)  Dom0 kernel: 64-bit, lsb, paddr 0x2000 -> 0x6d5000

      (XEN) PHYSICAL MEMORY ARRANGEMENT:

      (XEN)  Dom0 alloc.:   0000000835000000->0000000836000000
      (520192 pages to be allocated)

      (XEN) VIRTUAL MEMORY ARRANGEMENT:

      (XEN)  Loaded kernel: ffffffff80002000->ffffffff806d5000

      (XEN)  Init. ramdisk: ffffffff806d5000->ffffffff80ed7400

      (XEN)  Phys-Mach map: ffffea0000000000->ffffea0000400000

      (XEN)  Start info:    ffffffff80ed8000->ffffffff80ed84b4

      (XEN)  Page tables:   ffffffff80ed9000->ffffffff80ee4000

      (XEN)  Boot stack:    ffffffff80ee4000->ffffffff80ee5000

      (XEN)  TOTAL:         ffffffff80000000->ffffffff81000000

      (XEN)  ENTRY ADDRESS: ffffffff80002000

      (XEN) Dom0 has maximum 24 VCPUs

      (XEN) Scrubbing Free RAM:
................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
 .....................................................................................................................................................done.

      (XEN) Xen trace buffers: disabled

      (XEN) Std. Loglevel: Errors and warnings

      (XEN) Guest Loglevel: Nothing (Rate-limited: Errors and warnings)

      (XEN) Xen is relinquishing VGA console.

      (XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to
      switch input to Xen)

      (XEN) Freed 168kB init memory.

      

    
    

    

  


--------------090605070704070006080803--


--===============2070227487450461743==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

--===============2070227487450461743==--

From mboxrd@z Thu Jan  1 00:00:00 1970
From: "Jan Beulich" 
Subject: Re: kernel bootup slow issue on ovm3.1.1
Date: Fri, 10 Aug 2012 15:22:00 +0100
Message-ID: <502535280200007800094322@nat28.tlf.novell.com>
References: <5020C24A.3060604@oracle.com>
	<5020F003020000780009322C@nat28.tlf.novell.com>
	<502235E8.9040309@oracle.com>
	<50229B840200007800093A73@nat28.tlf.novell.com>
	<5023860E.7080908@oracle.com>
	<5023AE960200007800093DE8@nat28.tlf.novell.com>
	<502490A7.7020603@oracle.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
Return-path: 
In-Reply-To: <502490A7.7020603@oracle.com>
Content-Disposition: inline
List-Unsubscribe: ,
	
List-Post: 
List-Help: 
List-Subscribe: ,
	
Sender: xen-devel-bounces@lists.xen.org
Errors-To: xen-devel-bounces@lists.xen.org
To: zhenzhong.duan@oracle.com
Cc: Konrad Rzeszutek Wilk , Feng Jin , xen-devel 
List-Id: xen-devel@lists.xenproject.org

Pj4+IE9uIDEwLjA4LjEyIGF0IDA2OjQwLCAiemhlbnpob25nLmR1YW4iIDx6aGVuemhvbmcuZHVh
bkBvcmFjbGUuY29tPiB3cm90ZToKPiDkuo4gMjAxMi0wOC0wOSAxODozNSwgSmFuIEJldWxpY2gg
5YaZ6YGTOgo+Pj4+PiBPbiAwOS4wOC4xMiBhdCAxMTo0MiwgInpoZW56aG9uZy5kdWFuIjx6aGVu
emhvbmcuZHVhbkBvcmFjbGUuY29tPiAgd3JvdGU6Cj4+PiDkuo4gMjAxMi0wOC0wOCAyMzowMSwg
SmFuIEJldWxpY2gg5YaZ6YGTOgo+Pj4+Pj4+IE9uIDA4LjA4LjEyIGF0IDExOjQ4LCAiemhlbnpo
b25nLmR1YW4iPHpoZW56aG9uZy5kdWFuQG9yYWNsZS5jb20+ICAgd3JvdGU6Cj4+Pj4+IOS6jiAy
MDEyLTA4LTA3IDE2OjM3LCBKYW4gQmV1bGljaCDlhpnpgZM6Cj4+Pj4+IFNvbWUgc3BpbiBhdCBz
dG9wX21hY2hpbmUgYWZ0ZXIgZmluaXNoIHRoZWlyIGpvYi4KPj4+PiBBbmQgaGVyZSB5b3UnZCBu
ZWVkIHRvIGZpbmQgb3V0IHdoYXQgdGhleSdyZSB3YWl0aW5nIGZvciwKPj4+PiBhbmQgd2hhdCB0
aG9zZSBDUFVzIGFyZSBkb2luZy4KPj4+IFRoZXkgYXJlIHdhaXRpbmcgdGhlIHZjcHUgY2FsbGlu
ZyBnZW5lcmljX3NldF9hbGwgYW5kIHRob3NlIHNwaW4gYXQKPj4+IHNldF9hdG9taWNpdHlfbG9j
ay4KPj4+IEluIGZhY3QsIGFsbCBhcmUgd2FpdGluZyBnZW5lcmljX3NldF9hbGwKPj4gSSB0aGlu
ayB3ZSdyZSBtb3ZpbmcgaW4gY2lyY2xlcyAtIHdoYXQgaXMgdGhlIHZDUFUgY3VycmVudGx5Cj4+
IGdlbmVyaWNfc2V0X2FsbCgpIHRoZW4gZG9pbmc/Cj4gQWRkIHNvbWUgZGVidWcgcHJpbnQsIGdl
bmVyaWNfc2V0X2FsbC0+cHJlcGFyZV9zZXQtPndyaXRlX2NyMCB0b29rIG11Y2ggCj4gdGltZSwK
PiBhbGwgZWxzZSBhcmUgcXVpY2suIHNldF9hdG9taWNpdHlfbG9jayBzZXJpYWxpemVkIHRoaXMg
cHJvY2VzcyBiZXR3ZWVuIAo+IGNwdXMsIG1ha2UgaXQgd29yc2UuCj4gT25lIGl0ZXJhdGlvbjoK
PiBNVFJSOiBDUFUgMgo+IHByZXBhcmVfc2V0OiBiZWZvcmUgcmVhZF9jcjAKPiBwcmVwYXJlX3Nl
dDogYmVmb3JlIHdyaXRlX2NyMCAtLS0tLS0qYmxvY2sgaGVyZSoKClllYWgsIHRoYXQgQ1IwIHdy
aXRlIGRpc2FibGVzIHRoZSBjYWNoZXMsIGFuZCB0aGF0J3MgcHJldHR5CmV4cGVuc2l2ZSBvbiBF
UFQgKG5vdCBzdXJlIHdoeSBOUFQgZG9lc24ndCB1c2UvbmVlZCB0aGUKc2FtZSBob29rKSB3aGVu
IHRoZSBndWVzdCBoYXMgYW55IGFjdGl2ZSBNTUlPIHJlZ2lvbnM6CnZteF9zZXRfdWNfbW9kZSgp
LCB3aGVuIEhBUCBpcyBlbmFibGVkLCBjYWxscwplcHRfY2hhbmdlX2VudHJ5X2VtdF93aXRoX3Jh
bmdlKCksIHdoaWNoIGlzIGEgd2FsayB0aHJvdWdoCnRoZSBlbnRpcmUgZ3Vlc3QgcGFnZSB0YWJs
ZXMgKGkuZS4gc2NhbGVzIHdpdGggZ3Vlc3Qgc2l6ZSwgb3IsIHRvCmJlIHByZWNpc2UsIHdpdGgg
dGhlIGhpZ2hlc3QgcG9wdWxhdGVkIEdGTikuCgpHb2luZyBiYWNrIHRvIHlvdXIgb3JpZ2luYWwg
bWFpbCwgSSB3b25kZXIgaG93ZXZlciB3aHkgdGhpcwpnZXRzIGRvbmUgYXQgYWxsLiBZb3Ugc2Fp
ZCBpdCBnb3QgdGhlcmUgdmlhCgptdHJyX2Fwc19pbml0KCkgCiBcLT4gc2V0X210cnIoKSAKICAg
ICBcLT4gbXRycl93b3JrX2hhbmRsZXIoKSAKCnlldCB0aGlzIGlzbid0IGRvbmUgdW5jb25kaXRp
b25hbGx5IC0gc2VlIHRoZSBjb21tZW50IGJlZm9yZQpjaGVja2luZyBtdHJyX2Fwc19kZWxheWVk
X2luaXQuIENhbiB5b3UgZmluZCBvdXQgd2hlcmUgdGhlCm9idmlvdXNseSBuZWNlc3NhcnkgY2Fs
bChzKSB0byBzZXRfbXRycl9hcHNfZGVsYXllZF9pbml0KCkKY29tZShzKSBmcm9tPwoKPiBwcmVw
YXJlX3NldDogYmVmb3JlIHdiaW52ZAo+IHByZXBhcmVfc2V0OiBiZWZvcmUgcmVhZF9jcjQKPiBw
cmVwYXJlX3NldDogYmVmb3JlIHdyaXRlX2NyNAo+IHByZXBhcmVfc2V0OiBiZWZvcmUgX19mbHVz
aF90bGIKPiBwcmVwYXJlX3NldDogYmVmb3JlIHJkbXNyCj4gcHJlcGFyZV9zZXQ6IGJlZm9yZSB3
cm1zcgo+IGdlbmVyaWNfc2V0X2FsbDogYmVmb3JlIHNldF9tdHJyX3N0YXRlCj4gZ2VuZXJpY19z
ZXRfYWxsOiBiZWZvcmUgcGF0X2luaXQKPiBwb3N0X3NldDogYmVmb3JlIHdiaW52ZAo+IHBvc3Rf
c2V0OiBiZWZvcmUgd3Jtc3IKPiBwb3N0X3NldDogYmVmb3JlIHdyaXRlX2NyMAo+IHBvc3Rfc2V0
OiBiZWZvcmUgd3JpdGVfY3I0Cj4gCj4+Cj4+Pj4gVGhlcmUncyBub3QgdGhhdCBtdWNoIGJlaW5n
IGRvbmUgaW4gZ2VuZXJpY19zZXRfYWxsKCksIHNvIHRoZQo+Pj4+IGNvZGUgc2hvdWxkIGZpbmlz
aCByZWFzb25hYmx5IHF1aWNrbHkuIEFyZSB5b3UgcGVyaGFwcyBoYXZpbmcKPj4+PiBtb3JlIHZD
UFUtcyBpbiB0aGUgZ3Vlc3QgdGhhbiBwQ1BVLXMgdGhleSBjYW4gcnVuIG9uPwo+Pj4gU3lzdGVt
IGVudiBpcyBhbiBleGFsb2dpYyBub2RlIHdpdGggMjQgY29yZXMgKyAxMDBHIG1lbSAoMiBzb2Nr
ZXQgLCA2Cj4+PiBjb3JlcyBwZXIgc29ja2V0LCAyIEhUIHRocmVhZHMgcGVyIGNvcmUpLgo+Pj4g
Qm9vdHVwIGEgcHZodm0gd2l0aCAxMnZwY3VzIChvciAyNCkgKyA5MCBHQiArIHBjaSBwYXNzdGhy
b3VnaGVkIGRldmljZS4KPj4gU28geW91J3JlIGluZGVlZCBvdmVyLWNvbW1pdHRpbmcgdGhlIHN5
c3RlbS4gSG93IG1hbnkgdkNQVS1zCj4+IGRvZXMgeW91IERvbTAgaGF2ZT8gQXJlIHRoZXJlIGFu
eSBvdGhlciBWTXM/IElzIHRoZXJlIGFueQo+PiB2Q1BVIHBpbm5pbmcgaW4gZWZmZWN0Pwo+IGRv
bTAgYm9vdCB3aXRoIDI0IHZjcHVzKHNhbWUgcmVzdWx0IHdpdGggZG9tMF9tYXhfdmNwdXM9NCku
IE5vIG90aGVyIHZtIAo+IGV4Y2VwdCBkb20wLiBBbGwgMjQgdmNwdXMgc3BpbiBmcm9tIHhlbnRv
cCByZXN1bHQuIEJlbG93IGlzIHhlbnRvcCBjbGlwLgoKWWVzLCB0aGlzIHdheSB5b3UgZG8gb3Zl
cmNvbW1pdCAtIDI0IGd1ZXN0IHZDUFUtcyBzcGlubmluZywgcGx1cwphbnl0aGluZyBEb20wIG1h
eSBuZWVkIHRvIGRvLgoKPj4+PiAgICBEb2VzCj4+Pj4geW91ciBoYXJkd2FyZSBzdXBwb3J0IFBh
dXNlLUxvb3AtRXhpdGluZyAob3IgdGhlIEFNRAo+Pj4+IGVxdWl2YWxlbnQsIGRvbid0IHJlY2Fs
bCB0aGVpciB0ZXJtIHJpZ2h0IG5vdyk/Cj4+PiBJIGhhdmUgbm8gYWNjZXNzIHRvIHNlcmlhbCBs
aW5lLCBjb3VsZCBJIGdldCB0aGUgaW5mbyBieSBhIGNvbW1hbmQ/Cj4+ICJ4bCBkbWVzZyIgcnVu
IGVhcmx5IGVub3VnaCAoaS5lLiBiZWZvcmUgdGhlIGxvZyBidWZmZXIgd3JhcHMpLgo+IEJlbG93
IGlzIHhsIGRtZXNnIHJlc3VsdCBmb3IgeW91ciByZWZlcmVuY2UuIHRoYW5rcwo+Li4uCj4gKFhF
TikgVk1YOiBTdXBwb3J0ZWQgYWR2YW5jZWQgZmVhdHVyZXM6Cj4gKFhFTikgIC0gQVBJQyBNTUlP
IGFjY2VzcyB2aXJ0dWFsaXNhdGlvbgo+IChYRU4pICAtIEFQSUMgVFBSIHNoYWRvdwo+IChYRU4p
ICAtIEV4dGVuZGVkIFBhZ2UgVGFibGVzIChFUFQpCj4gKFhFTikgIC0gVmlydHVhbC1Qcm9jZXNz
b3IgSWRlbnRpZmllcnMgKFZQSUQpCj4gKFhFTikgIC0gVmlydHVhbCBOTUkKPiAoWEVOKSAgLSBN
U1IgZGlyZWN0LWFjY2VzcyBiaXRtYXAKPiAoWEVOKSAgLSBVbnJlc3RyaWN0ZWQgR3Vlc3QKCkkn
bSBzb3JyeSwgSSBoYWQgZXhwZWN0ZWQgdGhpcyB0byBiZSBwcmludGVkIGhlcmUsIGJ1dCBpdCBp
c24ndC4KSGVuY2UgSSBjYW4ndCB0ZWxsIGZvciBzdXJlIHdoZXRoZXIgUExFIGlzIGltcGxlbWVu
dGVkIHRoZXJlLApidXQgZ2l2ZW4gaG93IGxvbmcgaXQgaGFzIGJlZW4gYXZhaWxhYmxlIGl0IG91
Z2h0IHRvIGJlIHdoZW4KIlVucmVzdHJpY3RlZCBHdWVzdCIgaXMgdGhlcmUgKHdoaWNoIGlpcmMg
Z290IGludHJvZHVjZWQgbXVjaApsYXRlcikuCgpKYW4KCl9fX19fX19fX19fX19fX19fX19fX19f
X19fX19fX19fX19fX19fX19fX19fX19fClhlbi1kZXZlbCBtYWlsaW5nIGxpc3QKWGVuLWRldmVs
QGxpc3RzLnhlbi5vcmcKaHR0cDovL2xpc3RzLnhlbi5vcmcveGVuLWRldmVsCg==

From mboxrd@z Thu Jan  1 00:00:00 1970
From: "zhenzhong.duan" 
Subject: Re: kernel bootup slow issue on ovm3.1.1
Date: Mon, 13 Aug 2012 15:58:35 +0800
Message-ID: <5028B3AB.7060705@oracle.com>
References: <5020C24A.3060604@oracle.com>
	<5020F003020000780009322C@nat28.tlf.novell.com>
	<502235E8.9040309@oracle.com>
	<50229B840200007800093A73@nat28.tlf.novell.com>
	<5023860E.7080908@oracle.com>
	<5023AE960200007800093DE8@nat28.tlf.novell.com>
	<502490A7.7020603@oracle.com>
	<502535280200007800094322@nat28.tlf.novell.com>
Reply-To: zhenzhong.duan@oracle.com
Mime-Version: 1.0
Content-Type: text/plain; charset="utf-8"; Format="flowed"
Content-Transfer-Encoding: base64
Return-path: 
In-Reply-To: <502535280200007800094322@nat28.tlf.novell.com>
List-Unsubscribe: ,
	
List-Post: 
List-Help: 
List-Subscribe: ,
	
Sender: xen-devel-bounces@lists.xen.org
Errors-To: xen-devel-bounces@lists.xen.org
To: Jan Beulich 
Cc: Satish Kantheti , Konrad Rzeszutek Wilk , Feng Jin , xen-devel 
List-Id: xen-devel@lists.xenproject.org

Q2Npbmcgc2F0aXNoIHdobyBmaXJzdCBmaW5kIHRoaXMgaXNzdWUuCgrkuo4gMjAxMi0wOC0xMCAy
MjoyMiwgSmFuIEJldWxpY2gg5YaZ6YGTOgo+Pj4+IE9uIDEwLjA4LjEyIGF0IDA2OjQwLCAiemhl
bnpob25nLmR1YW4iPHpoZW56aG9uZy5kdWFuQG9yYWNsZS5jb20+ICB3cm90ZToKPj4g5LqOIDIw
MTItMDgtMDkgMTg6MzUsIEphbiBCZXVsaWNoIOWGmemBkzoKPj4+Pj4+IE9uIDA5LjA4LjEyIGF0
IDExOjQyLCAiemhlbnpob25nLmR1YW4iPHpoZW56aG9uZy5kdWFuQG9yYWNsZS5jb20+ICAgd3Jv
dGU6Cj4+Pj4g5LqOIDIwMTItMDgtMDggMjM6MDEsIEphbiBCZXVsaWNoIOWGmemBkzoKPj4+Pj4+
Pj4gT24gMDguMDguMTIgYXQgMTE6NDgsICJ6aGVuemhvbmcuZHVhbiI8emhlbnpob25nLmR1YW5A
b3JhY2xlLmNvbT4gICAgd3JvdGU6Cj4+Pj4+PiDkuo4gMjAxMi0wOC0wNyAxNjozNywgSmFuIEJl
dWxpY2gg5YaZ6YGTOgo+Pj4+Pj4gU29tZSBzcGluIGF0IHN0b3BfbWFjaGluZSBhZnRlciBmaW5p
c2ggdGhlaXIgam9iLgo+Pj4+PiBBbmQgaGVyZSB5b3UnZCBuZWVkIHRvIGZpbmQgb3V0IHdoYXQg
dGhleSdyZSB3YWl0aW5nIGZvciwKPj4+Pj4gYW5kIHdoYXQgdGhvc2UgQ1BVcyBhcmUgZG9pbmcu
Cj4+Pj4gVGhleSBhcmUgd2FpdGluZyB0aGUgdmNwdSBjYWxsaW5nIGdlbmVyaWNfc2V0X2FsbCBh
bmQgdGhvc2Ugc3BpbiBhdAo+Pj4+IHNldF9hdG9taWNpdHlfbG9jay4KPj4+PiBJbiBmYWN0LCBh
bGwgYXJlIHdhaXRpbmcgZ2VuZXJpY19zZXRfYWxsCj4+PiBJIHRoaW5rIHdlJ3JlIG1vdmluZyBp
biBjaXJjbGVzIC0gd2hhdCBpcyB0aGUgdkNQVSBjdXJyZW50bHkKPj4+IGdlbmVyaWNfc2V0X2Fs
bCgpIHRoZW4gZG9pbmc/Cj4+IEFkZCBzb21lIGRlYnVnIHByaW50LCBnZW5lcmljX3NldF9hbGwt
PnByZXBhcmVfc2V0LT53cml0ZV9jcjAgdG9vayBtdWNoCj4+IHRpbWUsCj4+IGFsbCBlbHNlIGFy
ZSBxdWljay4gc2V0X2F0b21pY2l0eV9sb2NrIHNlcmlhbGl6ZWQgdGhpcyBwcm9jZXNzIGJldHdl
ZW4KPj4gY3B1cywgbWFrZSBpdCB3b3JzZS4KPj4gT25lIGl0ZXJhdGlvbjoKPj4gTVRSUjogQ1BV
IDIKPj4gcHJlcGFyZV9zZXQ6IGJlZm9yZSByZWFkX2NyMAo+PiBwcmVwYXJlX3NldDogYmVmb3Jl
IHdyaXRlX2NyMCAtLS0tLS0qYmxvY2sgaGVyZSoKPgo+IFllYWgsIHRoYXQgQ1IwIHdyaXRlIGRp
c2FibGVzIHRoZSBjYWNoZXMsIGFuZCB0aGF0J3MgcHJldHR5Cj4gZXhwZW5zaXZlIG9uIEVQVCAo
bm90IHN1cmUgd2h5IE5QVCBkb2Vzbid0IHVzZS9uZWVkIHRoZQo+IHNhbWUgaG9vaykgd2hlbiB0
aGUgZ3Vlc3QgaGFzIGFueSBhY3RpdmUgTU1JTyByZWdpb25zOgo+IHZteF9zZXRfdWNfbW9kZSgp
LCB3aGVuIEhBUCBpcyBlbmFibGVkLCBjYWxscwo+IGVwdF9jaGFuZ2VfZW50cnlfZW10X3dpdGhf
cmFuZ2UoKSwgd2hpY2ggaXMgYSB3YWxrIHRocm91Z2gKPiB0aGUgZW50aXJlIGd1ZXN0IHBhZ2Ug
dGFibGVzIChpLmUuIHNjYWxlcyB3aXRoIGd1ZXN0IHNpemUsIG9yLCB0bwo+IGJlIHByZWNpc2Us
IHdpdGggdGhlIGhpZ2hlc3QgcG9wdWxhdGVkIEdGTikuCj4KPiBHb2luZyBiYWNrIHRvIHlvdXIg
b3JpZ2luYWwgbWFpbCwgSSB3b25kZXIgaG93ZXZlciB3aHkgdGhpcwo+IGdldHMgZG9uZSBhdCBh
bGwuIFlvdSBzYWlkIGl0IGdvdCB0aGVyZSB2aWEKPgo+IG10cnJfYXBzX2luaXQoKQo+ICAgXC0+
ICBzZXRfbXRycigpCj4gICAgICAgXC0+ICBtdHJyX3dvcmtfaGFuZGxlcigpCj4KPiB5ZXQgdGhp
cyBpc24ndCBkb25lIHVuY29uZGl0aW9uYWxseSAtIHNlZSB0aGUgY29tbWVudCBiZWZvcmUKPiBj
aGVja2luZyBtdHJyX2Fwc19kZWxheWVkX2luaXQuIENhbiB5b3UgZmluZCBvdXQgd2hlcmUgdGhl
Cj4gb2J2aW91c2x5IG5lY2Vzc2FyeSBjYWxsKHMpIHRvIHNldF9tdHJyX2Fwc19kZWxheWVkX2lu
aXQoKQo+IGNvbWUocykgZnJvbT8KQXQgYm9vdHVwIHN0YWdlLCBzZXRfbXRycl9hcHNfZGVsYXll
ZF9pbml0IGlzIGNhbGxlZCBieSBuYXRpdmVfc21wX3ByZXBhcmVfY3B1cy4KbXRycl9hcHNfZGVs
YXllZF9pbml0IGlzIGFsd2F5cyBzZXQgdG8gdHVyZSBmb3IgaW50ZWwgcHJvY2Vzc29yIGluIHVw
c3RyZWFtIGNvZGUuCgo+Pj4+PiAgICAgRG9lcwo+Pj4+PiB5b3VyIGhhcmR3YXJlIHN1cHBvcnQg
UGF1c2UtTG9vcC1FeGl0aW5nIChvciB0aGUgQU1ECj4+Pj4+IGVxdWl2YWxlbnQsIGRvbid0IHJl
Y2FsbCB0aGVpciB0ZXJtIHJpZ2h0IG5vdyk/Cj4+Pj4gSSBoYXZlIG5vIGFjY2VzcyB0byBzZXJp
YWwgbGluZSwgY291bGQgSSBnZXQgdGhlIGluZm8gYnkgYSBjb21tYW5kPwo+Pj4gInhsIGRtZXNn
IiBydW4gZWFybHkgZW5vdWdoIChpLmUuIGJlZm9yZSB0aGUgbG9nIGJ1ZmZlciB3cmFwcykuCj4+
IEJlbG93IGlzIHhsIGRtZXNnIHJlc3VsdCBmb3IgeW91ciByZWZlcmVuY2UuIHRoYW5rcwo+PiAu
Li4KPj4gKFhFTikgVk1YOiBTdXBwb3J0ZWQgYWR2YW5jZWQgZmVhdHVyZXM6Cj4+IChYRU4pICAt
IEFQSUMgTU1JTyBhY2Nlc3MgdmlydHVhbGlzYXRpb24KPj4gKFhFTikgIC0gQVBJQyBUUFIgc2hh
ZG93Cj4+IChYRU4pICAtIEV4dGVuZGVkIFBhZ2UgVGFibGVzIChFUFQpCj4+IChYRU4pICAtIFZp
cnR1YWwtUHJvY2Vzc29yIElkZW50aWZpZXJzIChWUElEKQo+PiAoWEVOKSAgLSBWaXJ0dWFsIE5N
SQo+PiAoWEVOKSAgLSBNU1IgZGlyZWN0LWFjY2VzcyBiaXRtYXAKPj4gKFhFTikgIC0gVW5yZXN0
cmljdGVkIEd1ZXN0Cj4KPiBJJ20gc29ycnksIEkgaGFkIGV4cGVjdGVkIHRoaXMgdG8gYmUgcHJp
bnRlZCBoZXJlLCBidXQgaXQgaXNuJ3QuCj4gSGVuY2UgSSBjYW4ndCB0ZWxsIGZvciBzdXJlIHdo
ZXRoZXIgUExFIGlzIGltcGxlbWVudGVkIHRoZXJlLAo+IGJ1dCBnaXZlbiBob3cgbG9uZyBpdCBo
YXMgYmVlbiBhdmFpbGFibGUgaXQgb3VnaHQgdG8gYmUgd2hlbgo+ICJVbnJlc3RyaWN0ZWQgR3Vl
c3QiIGlzIHRoZXJlICh3aGljaCBpaXJjIGdvdCBpbnRyb2R1Y2VkIG11Y2gKPiBsYXRlcikuCiBG
cm9tIFZNQ1MgZHVtcCwgbG9va3MgUEFVU0UgZXhpdGluZyBpcyAwLCBQTEUgaXMgMS4KKFhFTikg
KioqIENvbnRyb2wgU3RhdGUgKioqCihYRU4pIFBpbkJhc2VkPTAwMDAwMDNmIENQVUJhc2VkPWI2
YTA2NWZlIFNlY29uZGFyeUV4ZWM9MDAwMDA0ZWIKCnpkdWFuCgpfX19fX19fX19fX19fX19fX19f
X19fX19fX19fX19fX19fX19fX19fX19fX19fXwpYZW4tZGV2ZWwgbWFpbGluZyBsaXN0Clhlbi1k
ZXZlbEBsaXN0cy54ZW4ub3JnCmh0dHA6Ly9saXN0cy54ZW4ub3JnL3hlbi1kZXZlbAo=

From mboxrd@z Thu Jan  1 00:00:00 1970
From: Tim Deegan 
Subject: Re: kernel bootup slow issue on ovm3.1.1
Date: Mon, 13 Aug 2012 10:07:57 +0100
Message-ID: <20120813090757.GA75552@ocelot.phlegethon.org>
References: <5020C24A.3060604@oracle.com>
	<5020F003020000780009322C@nat28.tlf.novell.com>
	<502235E8.9040309@oracle.com>
	<50229B840200007800093A73@nat28.tlf.novell.com>
	<5023860E.7080908@oracle.com>
	<5023AE960200007800093DE8@nat28.tlf.novell.com>
	<502490A7.7020603@oracle.com>
	<502535280200007800094322@nat28.tlf.novell.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Return-path: 
Content-Disposition: inline
In-Reply-To: <502535280200007800094322@nat28.tlf.novell.com>
List-Unsubscribe: ,
	
List-Post: 
List-Help: 
List-Subscribe: ,
	
Sender: xen-devel-bounces@lists.xen.org
Errors-To: xen-devel-bounces@lists.xen.org
To: Jan Beulich 
Cc: xen-devel , Feng Jin , zhenzhong.duan@oracle.com, Konrad Rzeszutek Wilk 
List-Id: xen-devel@lists.xenproject.org

At 15:22 +0100 on 10 Aug (1344612120), Jan Beulich wrote:
> Yeah, that CR0 write disables the caches, and that's pretty
> expensive on EPT (not sure why NPT doesn't use/need the
> same hook) when the guest has any active MMIO regions:
> vmx_set_uc_mode(), when HAP is enabled, calls
> ept_change_entry_emt_with_range(), which is a walk through
> the entire guest page tables (i.e. scales with guest size, or, to
> be precise, with the highest populated GFN).

:( That's not so great.  It can definitely be done more efficiently than
with that for() loop, and I wonder whether there isn't some better way
involving flipping a global flag somewhere.

If no EPT maintainers have commented on this by Thursday I'll look into
it then.

Tim.

From mboxrd@z Thu Jan  1 00:00:00 1970
From: "Jan Beulich" 
Subject: Re: kernel bootup slow issue on ovm3.1.1
Date: Mon, 13 Aug 2012 10:29:54 +0100
Message-ID: <5028E53202000078000946B1@nat28.tlf.novell.com>
References: <5020C24A.3060604@oracle.com>
	<5020F003020000780009322C@nat28.tlf.novell.com>
	<502235E8.9040309@oracle.com>
	<50229B840200007800093A73@nat28.tlf.novell.com>
	<5023860E.7080908@oracle.com>
	<5023AE960200007800093DE8@nat28.tlf.novell.com>
	<502490A7.7020603@oracle.com>
	<502535280200007800094322@nat28.tlf.novell.com>
	<5028B3AB.7060705@oracle.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
Return-path: 
In-Reply-To: <5028B3AB.7060705@oracle.com>
Content-Disposition: inline
List-Unsubscribe: ,
	
List-Post: 
List-Help: 
List-Subscribe: ,
	
Sender: xen-devel-bounces@lists.xen.org
Errors-To: xen-devel-bounces@lists.xen.org
To: zhenzhong.duan@oracle.com
Cc: Satish Kantheti , xen-devel , Feng Jin , Konrad Rzeszutek Wilk , Stefano Stabellini 
List-Id: xen-devel@lists.xenproject.org

Pj4+IE9uIDEzLjA4LjEyIGF0IDA5OjU4LCAiemhlbnpob25nLmR1YW4iIDx6aGVuemhvbmcuZHVh
bkBvcmFjbGUuY29tPiB3cm90ZToKPiDkuo4gMjAxMi0wOC0xMCAyMjoyMiwgSmFuIEJldWxpY2gg
5YaZ6YGTOgo+PiBHb2luZyBiYWNrIHRvIHlvdXIgb3JpZ2luYWwgbWFpbCwgSSB3b25kZXIgaG93
ZXZlciB3aHkgdGhpcwo+PiBnZXRzIGRvbmUgYXQgYWxsLiBZb3Ugc2FpZCBpdCBnb3QgdGhlcmUg
dmlhCj4+Cj4+IG10cnJfYXBzX2luaXQoKQo+PiAgIFwtPiAgc2V0X210cnIoKQo+PiAgICAgICBc
LT4gIG10cnJfd29ya19oYW5kbGVyKCkKPj4KPj4geWV0IHRoaXMgaXNuJ3QgZG9uZSB1bmNvbmRp
dGlvbmFsbHkgLSBzZWUgdGhlIGNvbW1lbnQgYmVmb3JlCj4+IGNoZWNraW5nIG10cnJfYXBzX2Rl
bGF5ZWRfaW5pdC4gQ2FuIHlvdSBmaW5kIG91dCB3aGVyZSB0aGUKPj4gb2J2aW91c2x5IG5lY2Vz
c2FyeSBjYWxsKHMpIHRvIHNldF9tdHJyX2Fwc19kZWxheWVkX2luaXQoKQo+PiBjb21lKHMpIGZy
b20/Cj4gQXQgYm9vdHVwIHN0YWdlLCBzZXRfbXRycl9hcHNfZGVsYXllZF9pbml0IGlzIGNhbGxl
ZCBieSAKPiBuYXRpdmVfc21wX3ByZXBhcmVfY3B1cy4KPiBtdHJyX2Fwc19kZWxheWVkX2luaXQg
aXMgYWx3YXlzIHNldCB0byB0dXJlIGZvciBpbnRlbCBwcm9jZXNzb3IgaW4gdXBzdHJlYW0gCj4g
Y29kZS4KCkluZGVlZCwgYW5kIHRoYXQgKGluIG9uZSBmb3JtIG9yIGFub3RoZXIpIGhhcyBiZWVu
IGRvbmUKdmlydHVhbGx5IGZvcmV2ZXIgaW4gTGludXguIEkgd29uZGVyIHdoeSB0aGUgcHJvYmxl
bSB3YXNuJ3QKbm90aWNlZCAob3IgbG9va2VkIGludG8sIGlmIGl0IHdhcyBub3RpY2VkKSBzbyBm
YXIuCgpBcyBpdCdzIGdvaW5nIHRvIGJlIHJhdGhlciBkaWZmaWN1bHQgdG8gY29udmluY2UgdGhl
IExpbnV4IGZvbGtzCnRvIGNoYW5nZSB0aGVpciBjb2RlIChwbHVzIHRoaXMgd291bGRuJ3QgaGVs
cCB3aXRoIGV4aXN0aW5nCmtlcm5lbHMgYW55d2F5KSwgd2UnbGwgbmVlZCB0byBmaW5kIGEgd2F5
IHRvIGltcHJvdmUgdGhpcyBpbgp0aGUgaHlwZXJ2aXNvci4KCk9uZSBzZWVtaW5nbHkgb3J0aG9n
b25hbCB0aGluZyB3b3VsZCBwcmVzdW1hYmx5IGhlbHAgcXVpdGUKYSBiaXQgb24gdGhlIGd1ZXN0
IHNpZGUgbmV2ZXJ0aGVsZXNzIC0gcGFyYS12aXJ0dWFsaXplZCBzcGluCmxvY2tzLiBJIGhhdmUg
bm8gaWRlYSB3aHkgdGhpcyBpcyBvbmx5IGJlaW5nIGRvbmUgd2hlbiBydW5uaW5nCnB2LCBidXQg
bm90IGZvciBwdmh2bS4gS29ucmFkLCBTdGVmYW5vPwoKSmFuCgpfX19fX19fX19fX19fX19fX19f
X19fX19fX19fX19fX19fX19fX19fX19fX19fXwpYZW4tZGV2ZWwgbWFpbGluZyBsaXN0Clhlbi1k
ZXZlbEBsaXN0cy54ZW4ub3JnCmh0dHA6Ly9saXN0cy54ZW4ub3JnL3hlbi1kZXZlbAo=

From mboxrd@z Thu Jan  1 00:00:00 1970
From: Stefano Stabellini 
Subject: Re: kernel bootup slow issue on ovm3.1.1
Date: Mon, 13 Aug 2012 12:08:21 +0100
Message-ID: 
References: <5020C24A.3060604@oracle.com>
	<5020F003020000780009322C@nat28.tlf.novell.com>
	<502235E8.9040309@oracle.com>
	<50229B840200007800093A73@nat28.tlf.novell.com>
	<5023860E.7080908@oracle.com>
	<5023AE960200007800093DE8@nat28.tlf.novell.com>
	<502490A7.7020603@oracle.com>
	<502535280200007800094322@nat28.tlf.novell.com>
	<5028B3AB.7060705@oracle.com>
	<5028E53202000078000946B1@nat28.tlf.novell.com>
Mime-Version: 1.0
Content-Type: multipart/mixed;
	boundary="1342847746-504235113-1344855878=:21096"
Return-path: 
In-Reply-To: <5028E53202000078000946B1@nat28.tlf.novell.com>
Content-ID: 
List-Unsubscribe: ,
	
List-Post: 
List-Help: 
List-Subscribe: ,
	
Sender: xen-devel-bounces@lists.xen.org
Errors-To: xen-devel-bounces@lists.xen.org
To: Jan Beulich 
Cc: Satish Kantheti , Konrad Rzeszutek Wilk , Stefano Stabellini , Feng Jin , "zhenzhong.duan@oracle.com" , xen-devel 
List-Id: xen-devel@lists.xenproject.org

--1342847746-504235113-1344855878=:21096
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8BIT
Content-ID: 

On Mon, 13 Aug 2012, Jan Beulich wrote:
> >>> On 13.08.12 at 09:58, "zhenzhong.duan"  wrote:
> > 于 2012-08-10 22:22, Jan Beulich 写道:
> >> Going back to your original mail, I wonder however why this
> >> gets done at all. You said it got there via
> >>
> >> mtrr_aps_init()
> >>   \->  set_mtrr()
> >>       \->  mtrr_work_handler()
> >>
> >> yet this isn't done unconditionally - see the comment before
> >> checking mtrr_aps_delayed_init. Can you find out where the
> >> obviously necessary call(s) to set_mtrr_aps_delayed_init()
> >> come(s) from?
> > At bootup stage, set_mtrr_aps_delayed_init is called by 
> > native_smp_prepare_cpus.
> > mtrr_aps_delayed_init is always set to ture for intel processor in upstream 
> > code.
> 
> Indeed, and that (in one form or another) has been done
> virtually forever in Linux. I wonder why the problem wasn't
> noticed (or looked into, if it was noticed) so far.
> 
> As it's going to be rather difficult to convince the Linux folks
> to change their code (plus this wouldn't help with existing
> kernels anyway), we'll need to find a way to improve this in
> the hypervisor.
> 
> One seemingly orthogonal thing would presumably help quite
> a bit on the guest side nevertheless - para-virtualized spin
> locks. I have no idea why this is only being done when running
> pv, but not for pvhvm. Konrad, Stefano?

I tried to use PV spinlocks on PV on HVM guests but I found that:

commit f10cd522c5fbfec9ae3cc01967868c9c2401ed23
Author: Stefano Stabellini 
Date:   Tue Sep 6 17:41:47 2011 +0100

    xen: disable PV spinlocks on HVM
    
    PV spinlocks cannot possibly work with the current code because they are
    enabled after pvops patching has already been done, and because PV
    spinlocks use a different data structure than native spinlocks so we
    cannot switch between them dynamically. A spinlock that has been taken
    once by the native code (__ticket_spin_lock) cannot be taken by
    __xen_spin_lock even after it has been released.
    
    Reported-and-Tested-by: Stefan Bader 
    Signed-off-by: Stefano Stabellini 
    Signed-off-by: Konrad Rzeszutek Wilk 


at that time Jeremy was finishing off his PV ticket locks series, that
has the nice side effect of making it much easier to implement PV on HVM
spin locks so I just deciced to wait and just append the following patch
to his series:

http://marc.info/?l=xen-devel&m=131846828430409&w=2

that clearly never went upstream.
--1342847746-504235113-1344855878=:21096
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

--1342847746-504235113-1344855878=:21096--

From mboxrd@z Thu Jan  1 00:00:00 1970
From: "zhenzhong.duan" 
Subject: Re: kernel bootup slow issue on ovm3.1.1
Date: Wed, 29 Aug 2012 13:19:52 +0800
Message-ID: <503DA678.4040801@oracle.com>
References: <5020C24A.3060604@oracle.com>
	<5020F003020000780009322C@nat28.tlf.novell.com>
	<502235E8.9040309@oracle.com>
	<50229B840200007800093A73@nat28.tlf.novell.com>
	<5023860E.7080908@oracle.com>
	<5023AE960200007800093DE8@nat28.tlf.novell.com>
	<502490A7.7020603@oracle.com>
	<502535280200007800094322@nat28.tlf.novell.com>
	<5028B3AB.7060705@oracle.com>
	<5028E53202000078000946B1@nat28.tlf.novell.com>
	
Reply-To: zhenzhong.duan@oracle.com
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"; Format="flowed"
Content-Transfer-Encoding: 7bit
Return-path: 
In-Reply-To: 
List-Unsubscribe: ,
	
List-Post: 
List-Help: 
List-Subscribe: ,
	
Sender: xen-devel-bounces@lists.xen.org
Errors-To: xen-devel-bounces@lists.xen.org
To: Stefano Stabellini 
Cc: Satish Kantheti , Konrad Rzeszutek Wilk , Feng Jin , Jan Beulich , xen-devel 
List-Id: xen-devel@lists.xenproject.org



On 2012-08-13 19:08, Stefano Stabellini wrote:
> On Mon, 13 Aug 2012, Jan Beulich wrote:
>
> I tried to use PV spinlocks on PV on HVM guests but I found that:
>
> commit f10cd522c5fbfec9ae3cc01967868c9c2401ed23
> Author: Stefano Stabellini
> Date:   Tue Sep 6 17:41:47 2011 +0100
>
>      xen: disable PV spinlocks on HVM
>
>      PV spinlocks cannot possibly work with the current code because they are
>      enabled after pvops patching has already been done, and because PV
>      spinlocks use a different data structure than native spinlocks so we
>      cannot switch between them dynamically. A spinlock that has been taken
>      once by the native code (__ticket_spin_lock) cannot be taken by
>      __xen_spin_lock even after it has been released.
>
>      Reported-and-Tested-by: Stefan Bader
>      Signed-off-by: Stefano Stabellini
>      Signed-off-by: Konrad Rzeszutek Wilk
>
>
> at that time Jeremy was finishing off his PV ticket locks series, that
> has the nice side effect of making it much easier to implement PV on HVM
> spin locks so I just deciced to wait and just append the following patch
> to his series:
>
> http://marc.info/?l=xen-devel&m=131846828430409&w=2
>
> that clearly never went upstream.
Hi Stefano,
Is there a schedule those patch merge to upstream?

zduan

From mboxrd@z Thu Jan  1 00:00:00 1970
From: "zhenzhong.duan" 
Subject: Re: kernel bootup slow issue on ovm3.1.1
Date: Wed, 29 Aug 2012 13:36:31 +0800
Message-ID: <503DAA5F.5030306@oracle.com>
References: <5020C24A.3060604@oracle.com>
	<5020F003020000780009322C@nat28.tlf.novell.com>
	<502235E8.9040309@oracle.com>
	<50229B840200007800093A73@nat28.tlf.novell.com>
	<5023860E.7080908@oracle.com>
	<5023AE960200007800093DE8@nat28.tlf.novell.com>
	<502490A7.7020603@oracle.com>
	<502535280200007800094322@nat28.tlf.novell.com>
	<5028B3AB.7060705@oracle.com>
	<5028E53202000078000946B1@nat28.tlf.novell.com>
Reply-To: zhenzhong.duan@oracle.com
Mime-Version: 1.0
Content-Type: text/plain; charset="utf-8"; Format="flowed"
Content-Transfer-Encoding: base64
Return-path: 
In-Reply-To: <5028E53202000078000946B1@nat28.tlf.novell.com>
List-Unsubscribe: ,
	
List-Post: 
List-Help: 
List-Subscribe: ,
	
Sender: xen-devel-bounces@lists.xen.org
Errors-To: xen-devel-bounces@lists.xen.org
To: Jan Beulich 
Cc: Satish Kantheti , xen-devel , Feng Jin , Konrad Rzeszutek Wilk , Stefano Stabellini 
List-Id: xen-devel@lists.xenproject.org

Cgrkuo4gMjAxMi0wOC0xMyAxNzoyOSwgSmFuIEJldWxpY2gg5YaZ6YGTOgo+Pj4+IE9uIDEzLjA4
LjEyIGF0IDA5OjU4LCAiemhlbnpob25nLmR1YW4iPHpoZW56aG9uZy5kdWFuQG9yYWNsZS5jb20+
ICB3cm90ZToKPj4g5LqOIDIwMTItMDgtMTAgMjI6MjIsIEphbiBCZXVsaWNoIOWGmemBkzoKPj4+
IEdvaW5nIGJhY2sgdG8geW91ciBvcmlnaW5hbCBtYWlsLCBJIHdvbmRlciBob3dldmVyIHdoeSB0
aGlzCj4+PiBnZXRzIGRvbmUgYXQgYWxsLiBZb3Ugc2FpZCBpdCBnb3QgdGhlcmUgdmlhCj4+Pgo+
Pj4gbXRycl9hcHNfaW5pdCgpCj4+PiAgICBcLT4gICBzZXRfbXRycigpCj4+PiAgICAgICAgXC0+
ICAgbXRycl93b3JrX2hhbmRsZXIoKQo+Pj4KPj4+IHlldCB0aGlzIGlzbid0IGRvbmUgdW5jb25k
aXRpb25hbGx5IC0gc2VlIHRoZSBjb21tZW50IGJlZm9yZQo+Pj4gY2hlY2tpbmcgbXRycl9hcHNf
ZGVsYXllZF9pbml0LiBDYW4geW91IGZpbmQgb3V0IHdoZXJlIHRoZQo+Pj4gb2J2aW91c2x5IG5l
Y2Vzc2FyeSBjYWxsKHMpIHRvIHNldF9tdHJyX2Fwc19kZWxheWVkX2luaXQoKQo+Pj4gY29tZShz
KSBmcm9tPwo+PiBBdCBib290dXAgc3RhZ2UsIHNldF9tdHJyX2Fwc19kZWxheWVkX2luaXQgaXMg
Y2FsbGVkIGJ5Cj4+IG5hdGl2ZV9zbXBfcHJlcGFyZV9jcHVzLgo+PiBtdHJyX2Fwc19kZWxheWVk
X2luaXQgaXMgYWx3YXlzIHNldCB0byB0dXJlIGZvciBpbnRlbCBwcm9jZXNzb3IgaW4gdXBzdHJl
YW0KPj4gY29kZS4KPiBJbmRlZWQsIGFuZCB0aGF0IChpbiBvbmUgZm9ybSBvciBhbm90aGVyKSBo
YXMgYmVlbiBkb25lCj4gdmlydHVhbGx5IGZvcmV2ZXIgaW4gTGludXguIEkgd29uZGVyIHdoeSB0
aGUgcHJvYmxlbSB3YXNuJ3QKPiBub3RpY2VkIChvciBsb29rZWQgaW50bywgaWYgaXQgd2FzIG5v
dGljZWQpIHNvIGZhci4KPgo+IEFzIGl0J3MgZ29pbmcgdG8gYmUgcmF0aGVyIGRpZmZpY3VsdCB0
byBjb252aW5jZSB0aGUgTGludXggZm9sa3MKPiB0byBjaGFuZ2UgdGhlaXIgY29kZSAocGx1cyB0
aGlzIHdvdWxkbid0IGhlbHAgd2l0aCBleGlzdGluZwo+IGtlcm5lbHMgYW55d2F5KSwgd2UnbGwg
bmVlZCB0byBmaW5kIGEgd2F5IHRvIGltcHJvdmUgdGhpcyBpbgo+IHRoZSBoeXBlcnZpc29yLgpI
aSBKYW4sIFRpbQpJcyB0aGlzIGlzc3VlIGltcHJvdmFibGUgZnJvbSB4ZW4gc2lkZT8KdGhhbmtz
CnpkdWFuCgpfX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fXwpY
ZW4tZGV2ZWwgbWFpbGluZyBsaXN0Clhlbi1kZXZlbEBsaXN0cy54ZW4ub3JnCmh0dHA6Ly9saXN0
cy54ZW4ub3JnL3hlbi1kZXZlbAo=

From mboxrd@z Thu Jan  1 00:00:00 1970
From: Stefano Stabellini 
Subject: Re: kernel bootup slow issue on ovm3.1.1
Date: Wed, 29 Aug 2012 19:28:32 +0100
Message-ID: 
References: <5020C24A.3060604@oracle.com>
	<5020F003020000780009322C@nat28.tlf.novell.com>
	<502235E8.9040309@oracle.com>
	<50229B840200007800093A73@nat28.tlf.novell.com>
	<5023860E.7080908@oracle.com>
	<5023AE960200007800093DE8@nat28.tlf.novell.com>
	<502490A7.7020603@oracle.com>
	<502535280200007800094322@nat28.tlf.novell.com>
	<5028B3AB.7060705@oracle.com>
	<5028E53202000078000946B1@nat28.tlf.novell.com>
	
	<503DA678.4040801@oracle.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Return-path: 
In-Reply-To: <503DA678.4040801@oracle.com>
List-Unsubscribe: ,
	
List-Post: 
List-Help: 
List-Subscribe: ,
	
Sender: xen-devel-bounces@lists.xen.org
Errors-To: xen-devel-bounces@lists.xen.org
To: "zhenzhong.duan" 
Cc: Satish Kantheti , Konrad Rzeszutek Wilk , Stefano Stabellini , Feng Jin , xen-devel , Jan Beulich 
List-Id: xen-devel@lists.xenproject.org

On Wed, 29 Aug 2012, zhenzhong.duan wrote:
> 
> On 2012-08-13 19:08, Stefano Stabellini wrote:
> > On Mon, 13 Aug 2012, Jan Beulich wrote:
> >
> > I tried to use PV spinlocks on PV on HVM guests but I found that:
> >
> > commit f10cd522c5fbfec9ae3cc01967868c9c2401ed23
> > Author: Stefano Stabellini
> > Date:   Tue Sep 6 17:41:47 2011 +0100
> >
> >      xen: disable PV spinlocks on HVM
> >
> >      PV spinlocks cannot possibly work with the current code because they are
> >      enabled after pvops patching has already been done, and because PV
> >      spinlocks use a different data structure than native spinlocks so we
> >      cannot switch between them dynamically. A spinlock that has been taken
> >      once by the native code (__ticket_spin_lock) cannot be taken by
> >      __xen_spin_lock even after it has been released.
> >
> >      Reported-and-Tested-by: Stefan Bader
> >      Signed-off-by: Stefano Stabellini
> >      Signed-off-by: Konrad Rzeszutek Wilk
> >
> >
> > at that time Jeremy was finishing off his PV ticket locks series, that
> > has the nice side effect of making it much easier to implement PV on HVM
> > spin locks so I just deciced to wait and just append the following patch
> > to his series:
> >
> > http://marc.info/?l=xen-devel&m=131846828430409&w=2
> >
> > that clearly never went upstream.
> Hi Stefano,
> Is there a schedule those patch merge to upstream?

They are currently being handled by the KVM guys:

https://lkml.org/lkml/2012/5/2/119

From mboxrd@z Thu Jan  1 00:00:00 1970
From: Tim Deegan 
Subject: Re: kernel bootup slow issue on ovm3.1.1
Date: Thu, 30 Aug 2012 10:03:12 +0100
Message-ID: <20120830090312.GA54290@ocelot.phlegethon.org>
References: <5020F003020000780009322C@nat28.tlf.novell.com>
	<502235E8.9040309@oracle.com>
	<50229B840200007800093A73@nat28.tlf.novell.com>
	<5023860E.7080908@oracle.com>
	<5023AE960200007800093DE8@nat28.tlf.novell.com>
	<502490A7.7020603@oracle.com>
	<502535280200007800094322@nat28.tlf.novell.com>
	<5028B3AB.7060705@oracle.com>
	<5028E53202000078000946B1@nat28.tlf.novell.com>
	<503DAA5F.5030306@oracle.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Return-path: 
Content-Disposition: inline
In-Reply-To: <503DAA5F.5030306@oracle.com>
List-Unsubscribe: ,
	
List-Post: 
List-Help: 
List-Subscribe: ,
	
Sender: xen-devel-bounces@lists.xen.org
Errors-To: xen-devel-bounces@lists.xen.org
To: "zhenzhong.duan" 
Cc: Satish Kantheti , Stefano Stabellini , Konrad Rzeszutek Wilk , Feng Jin , xen-devel , Jan Beulich 
List-Id: xen-devel@lists.xenproject.org

At 13:36 +0800 on 29 Aug (1346247391), zhenzhong.duan wrote:
> 
> 
> ??? 2012-08-13 17:29, Jan Beulich ??????:
> >>>>On 13.08.12 at 09:58, "zhenzhong.duan"  
> >>>>wrote:
> >>??? 2012-08-10 22:22, Jan Beulich ??????:
> >>>Going back to your original mail, I wonder however why this
> >>>gets done at all. You said it got there via
> >>>
> >>>mtrr_aps_init()
> >>>   \->   set_mtrr()
> >>>       \->   mtrr_work_handler()
> >>>
> >>>yet this isn't done unconditionally - see the comment before
> >>>checking mtrr_aps_delayed_init. Can you find out where the
> >>>obviously necessary call(s) to set_mtrr_aps_delayed_init()
> >>>come(s) from?
> >>At bootup stage, set_mtrr_aps_delayed_init is called by
> >>native_smp_prepare_cpus.
> >>mtrr_aps_delayed_init is always set to ture for intel processor in 
> >>upstream
> >>code.
> >Indeed, and that (in one form or another) has been done
> >virtually forever in Linux. I wonder why the problem wasn't
> >noticed (or looked into, if it was noticed) so far.
> >
> >As it's going to be rather difficult to convince the Linux folks
> >to change their code (plus this wouldn't help with existing
> >kernels anyway), we'll need to find a way to improve this in
> >the hypervisor.
> Hi Jan, Tim
> Is this issue improvable from xen side?

Probably; we're looking into the best way to address it. 

Tim.

From mboxrd@z Thu Jan  1 00:00:00 1970
From: "Jan Beulich" 
Subject: Re: kernel bootup slow issue on ovm3.1.1
Date: Fri, 31 Aug 2012 10:07:40 +0100
Message-ID: <50409AFC0200007800097BA6@nat28.tlf.novell.com>
References: <5020C24A.3060604@oracle.com>
	<5020F003020000780009322C@nat28.tlf.novell.com>
	<502235E8.9040309@oracle.com>
	<50229B840200007800093A73@nat28.tlf.novell.com>
	<5023860E.7080908@oracle.com>
	<5023AE960200007800093DE8@nat28.tlf.novell.com>
	<502490A7.7020603@oracle.com>
	<502535280200007800094322@nat28.tlf.novell.com>
	<5028B3AB.7060705@oracle.com>
	<5028E53202000078000946B1@nat28.tlf.novell.com>
	<503DAA5F.5030306@oracle.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
Return-path: 
In-Reply-To: <503DAA5F.5030306@oracle.com>
Content-Disposition: inline
List-Unsubscribe: ,
	
List-Post: 
List-Help: 
List-Subscribe: ,
	
Sender: xen-devel-bounces@lists.xen.org
Errors-To: xen-devel-bounces@lists.xen.org
To: zhenzhong.duan@oracle.com
Cc: Satish Kantheti , Konrad Rzeszutek Wilk , Feng Jin , Stefano Stabellini , Tim Deegan , xen-devel 
List-Id: xen-devel@lists.xenproject.org

Pj4+IE9uIDI5LjA4LjEyIGF0IDA3OjM2LCAiemhlbnpob25nLmR1YW4iIDx6aGVuemhvbmcuZHVh
bkBvcmFjbGUuY29tPiB3cm90ZToKCj4gCj4g5LqOIDIwMTItMDgtMTMgMTc6MjksIEphbiBCZXVs
aWNoIOWGmemBkzoKPj4+Pj4gT24gMTMuMDguMTIgYXQgMDk6NTgsICJ6aGVuemhvbmcuZHVhbiI8
emhlbnpob25nLmR1YW5Ab3JhY2xlLmNvbT4gIHdyb3RlOgo+Pj4g5LqOIDIwMTItMDgtMTAgMjI6
MjIsIEphbiBCZXVsaWNoIOWGmemBkzoKPj4+PiBHb2luZyBiYWNrIHRvIHlvdXIgb3JpZ2luYWwg
bWFpbCwgSSB3b25kZXIgaG93ZXZlciB3aHkgdGhpcwo+Pj4+IGdldHMgZG9uZSBhdCBhbGwuIFlv
dSBzYWlkIGl0IGdvdCB0aGVyZSB2aWEKPj4+Pgo+Pj4+IG10cnJfYXBzX2luaXQoKQo+Pj4+ICAg
IFwtPiAgIHNldF9tdHJyKCkKPj4+PiAgICAgICAgXC0+ICAgbXRycl93b3JrX2hhbmRsZXIoKQo+
Pj4+Cj4+Pj4geWV0IHRoaXMgaXNuJ3QgZG9uZSB1bmNvbmRpdGlvbmFsbHkgLSBzZWUgdGhlIGNv
bW1lbnQgYmVmb3JlCj4+Pj4gY2hlY2tpbmcgbXRycl9hcHNfZGVsYXllZF9pbml0LiBDYW4geW91
IGZpbmQgb3V0IHdoZXJlIHRoZQo+Pj4+IG9idmlvdXNseSBuZWNlc3NhcnkgY2FsbChzKSB0byBz
ZXRfbXRycl9hcHNfZGVsYXllZF9pbml0KCkKPj4+PiBjb21lKHMpIGZyb20/Cj4+PiBBdCBib290
dXAgc3RhZ2UsIHNldF9tdHJyX2Fwc19kZWxheWVkX2luaXQgaXMgY2FsbGVkIGJ5Cj4+PiBuYXRp
dmVfc21wX3ByZXBhcmVfY3B1cy4KPj4+IG10cnJfYXBzX2RlbGF5ZWRfaW5pdCBpcyBhbHdheXMg
c2V0IHRvIHR1cmUgZm9yIGludGVsIHByb2Nlc3NvciBpbiB1cHN0cmVhbQo+Pj4gY29kZS4KPj4g
SW5kZWVkLCBhbmQgdGhhdCAoaW4gb25lIGZvcm0gb3IgYW5vdGhlcikgaGFzIGJlZW4gZG9uZQo+
PiB2aXJ0dWFsbHkgZm9yZXZlciBpbiBMaW51eC4gSSB3b25kZXIgd2h5IHRoZSBwcm9ibGVtIHdh
c24ndAo+PiBub3RpY2VkIChvciBsb29rZWQgaW50bywgaWYgaXQgd2FzIG5vdGljZWQpIHNvIGZh
ci4KPj4KPj4gQXMgaXQncyBnb2luZyB0byBiZSByYXRoZXIgZGlmZmljdWx0IHRvIGNvbnZpbmNl
IHRoZSBMaW51eCBmb2xrcwo+PiB0byBjaGFuZ2UgdGhlaXIgY29kZSAocGx1cyB0aGlzIHdvdWxk
bid0IGhlbHAgd2l0aCBleGlzdGluZwo+PiBrZXJuZWxzIGFueXdheSksIHdlJ2xsIG5lZWQgdG8g
ZmluZCBhIHdheSB0byBpbXByb3ZlIHRoaXMgaW4KPj4gdGhlIGh5cGVydmlzb3IuCj4gSXMgdGhp
cyBpc3N1ZSBpbXByb3ZhYmxlIGZyb20geGVuIHNpZGU/CgpZZXMsIHdlJ3JlIGludmVzdGlnYXRp
bmcgb3B0aW9ucy4KCkphbgoKX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f
X19fX19fX18KWGVuLWRldmVsIG1haWxpbmcgbGlzdApYZW4tZGV2ZWxAbGlzdHMueGVuLm9yZwpo
dHRwOi8vbGlzdHMueGVuLm9yZy94ZW4tZGV2ZWwK

From mboxrd@z Thu Jan  1 00:00:00 1970
From: "zhenzhong.duan" 
Subject: Re: kernel bootup slow issue on ovm3.1.1
Date: Wed, 19 Sep 2012 10:39:45 +0800
Message-ID: <50593071.2050108@oracle.com>
References: <5020F003020000780009322C@nat28.tlf.novell.com>
	<502235E8.9040309@oracle.com>
	<50229B840200007800093A73@nat28.tlf.novell.com>
	<5023860E.7080908@oracle.com>
	<5023AE960200007800093DE8@nat28.tlf.novell.com>
	<502490A7.7020603@oracle.com>
	<502535280200007800094322@nat28.tlf.novell.com>
	<5028B3AB.7060705@oracle.com>
	<5028E53202000078000946B1@nat28.tlf.novell.com>
	<503DAA5F.5030306@oracle.com>
	<20120830090312.GA54290@ocelot.phlegethon.org>
Reply-To: zhenzhong.duan@oracle.com
Mime-Version: 1.0
Content-Type: text/plain; charset="utf-8"; Format="flowed"
Content-Transfer-Encoding: base64
Return-path: 
In-Reply-To: <20120830090312.GA54290@ocelot.phlegethon.org>
List-Unsubscribe: ,
	
List-Post: 
List-Help: 
List-Subscribe: ,
	
Sender: xen-devel-bounces@lists.xen.org
Errors-To: xen-devel-bounces@lists.xen.org
To: Tim Deegan 
Cc: Satish Kantheti , Stefano Stabellini , Konrad Rzeszutek Wilk , Feng Jin , xen-devel , Jan Beulich 
List-Id: xen-devel@lists.xenproject.org

Cgrkuo4gMjAxMi0wOC0zMCAxNzowMywgVGltIERlZWdhbiDlhpnpgZM6Cj4gQXQgMTM6MzYgKzA4
MDAgb24gMjkgQXVnICgxMzQ2MjQ3MzkxKSwgemhlbnpob25nLmR1YW4gd3JvdGU6Cj4+Cj4+ID8/
PyAyMDEyLTA4LTEzIDE3OjI5LCBKYW4gQmV1bGljaCA/Pz8/Pz86Cj4+Pj4+PiBPbiAxMy4wOC4x
MiBhdCAwOTo1OCwgInpoZW56aG9uZy5kdWFuIjx6aGVuemhvbmcuZHVhbkBvcmFjbGUuY29tPgo+
Pj4+Pj4gd3JvdGU6Cj4+Pj4gPz8/IDIwMTItMDgtMTAgMjI6MjIsIEphbiBCZXVsaWNoID8/Pz8/
PzoKPj4+Pj4gR29pbmcgYmFjayB0byB5b3VyIG9yaWdpbmFsIG1haWwsIEkgd29uZGVyIGhvd2V2
ZXIgd2h5IHRoaXMKPj4+Pj4gZ2V0cyBkb25lIGF0IGFsbC4gWW91IHNhaWQgaXQgZ290IHRoZXJl
IHZpYQo+Pj4+Pgo+Pj4+PiBtdHJyX2Fwc19pbml0KCkKPj4+Pj4gICAgXC0+ICAgIHNldF9tdHJy
KCkKPj4+Pj4gICAgICAgIFwtPiAgICBtdHJyX3dvcmtfaGFuZGxlcigpCj4+Pj4+Cj4+Pj4+IHll
dCB0aGlzIGlzbid0IGRvbmUgdW5jb25kaXRpb25hbGx5IC0gc2VlIHRoZSBjb21tZW50IGJlZm9y
ZQo+Pj4+PiBjaGVja2luZyBtdHJyX2Fwc19kZWxheWVkX2luaXQuIENhbiB5b3UgZmluZCBvdXQg
d2hlcmUgdGhlCj4+Pj4+IG9idmlvdXNseSBuZWNlc3NhcnkgY2FsbChzKSB0byBzZXRfbXRycl9h
cHNfZGVsYXllZF9pbml0KCkKPj4+Pj4gY29tZShzKSBmcm9tPwo+Pj4+IEF0IGJvb3R1cCBzdGFn
ZSwgc2V0X210cnJfYXBzX2RlbGF5ZWRfaW5pdCBpcyBjYWxsZWQgYnkKPj4+PiBuYXRpdmVfc21w
X3ByZXBhcmVfY3B1cy4KPj4+PiBtdHJyX2Fwc19kZWxheWVkX2luaXQgaXMgYWx3YXlzIHNldCB0
byB0dXJlIGZvciBpbnRlbCBwcm9jZXNzb3IgaW4KPj4+PiB1cHN0cmVhbQo+Pj4+IGNvZGUuCj4+
PiBJbmRlZWQsIGFuZCB0aGF0IChpbiBvbmUgZm9ybSBvciBhbm90aGVyKSBoYXMgYmVlbiBkb25l
Cj4+PiB2aXJ0dWFsbHkgZm9yZXZlciBpbiBMaW51eC4gSSB3b25kZXIgd2h5IHRoZSBwcm9ibGVt
IHdhc24ndAo+Pj4gbm90aWNlZCAob3IgbG9va2VkIGludG8sIGlmIGl0IHdhcyBub3RpY2VkKSBz
byBmYXIuCj4+Pgo+Pj4gQXMgaXQncyBnb2luZyB0byBiZSByYXRoZXIgZGlmZmljdWx0IHRvIGNv
bnZpbmNlIHRoZSBMaW51eCBmb2xrcwo+Pj4gdG8gY2hhbmdlIHRoZWlyIGNvZGUgKHBsdXMgdGhp
cyB3b3VsZG4ndCBoZWxwIHdpdGggZXhpc3RpbmcKPj4+IGtlcm5lbHMgYW55d2F5KSwgd2UnbGwg
bmVlZCB0byBmaW5kIGEgd2F5IHRvIGltcHJvdmUgdGhpcyBpbgo+Pj4gdGhlIGh5cGVydmlzb3Iu
Cj4+IEhpIEphbiwgVGltCj4+IElzIHRoaXMgaXNzdWUgaW1wcm92YWJsZSBmcm9tIHhlbiBzaWRl
Pwo+IFByb2JhYmx5OyB3ZSdyZSBsb29raW5nIGludG8gdGhlIGJlc3Qgd2F5IHRvIGFkZHJlc3Mg
aXQuCkhpIEphbiwgVGltCklzIHRoZXJlIGFueSBwYXRjaCBmb3IgdXMgdG8gdGVzdD8KV2UgYXJl
IGxvb2tpbmcgZm93YXJkIHRvIHlvdXIgZml4LgpPdXIgY3VzdG9tZXIgZ290IHVuc2F0aXNmaWVk
IHdpdGggbW9yZSB0aGFuIDMwIG1pbnMgb2YgYm9vdHVwIGFuZCBsb25nIAp0aW1lIHdhaXQuClJl
Z2FyZHMKemR1YW4KCl9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f
X19fClhlbi1kZXZlbCBtYWlsaW5nIGxpc3QKWGVuLWRldmVsQGxpc3RzLnhlbi5vcmcKaHR0cDov
L2xpc3RzLnhlbi5vcmcveGVuLWRldmVsCg==

From mboxrd@z Thu Jan  1 00:00:00 1970
From: "Jan Beulich" 
Subject: Re: kernel bootup slow issue on ovm3.1.1
Date: Wed, 19 Sep 2012 11:29:40 +0100
Message-ID: <5059BAB4020000780009C462@nat28.tlf.novell.com>
References: <5020F003020000780009322C@nat28.tlf.novell.com>
	<502235E8.9040309@oracle.com>
	<50229B840200007800093A73@nat28.tlf.novell.com>
	<5023860E.7080908@oracle.com>
	<5023AE960200007800093DE8@nat28.tlf.novell.com>
	<502490A7.7020603@oracle.com>
	<502535280200007800094322@nat28.tlf.novell.com>
	<5028B3AB.7060705@oracle.com>
	<5028E53202000078000946B1@nat28.tlf.novell.com>
	<503DAA5F.5030306@oracle.com>
	<20120830090312.GA54290@ocelot.phlegethon.org>
	<50593071.2050108@oracle.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
Return-path: 
In-Reply-To: <50593071.2050108@oracle.com>
Content-Disposition: inline
List-Unsubscribe: ,
	
List-Post: 
List-Help: 
List-Subscribe: ,
	
Sender: xen-devel-bounces@lists.xen.org
Errors-To: xen-devel-bounces@lists.xen.org
To: zhenzhong.duan@oracle.com
Cc: Satish Kantheti , Konrad Rzeszutek Wilk , Feng Jin , Stefano Stabellini , Tim Deegan , xen-devel 
List-Id: xen-devel@lists.xenproject.org

Pj4+IE9uIDE5LjA5LjEyIGF0IDA0OjM5LCAiemhlbnpob25nLmR1YW4iIDx6aGVuemhvbmcuZHVh
bkBvcmFjbGUuY29tPiB3cm90ZToKCj4gCj4g5LqOIDIwMTItMDgtMzAgMTc6MDMsIFRpbSBEZWVn
YW4g5YaZ6YGTOgo+PiBBdCAxMzozNiArMDgwMCBvbiAyOSBBdWcgKDEzNDYyNDczOTEpLCB6aGVu
emhvbmcuZHVhbiB3cm90ZToKPj4+Cj4+PiA/Pz8gMjAxMi0wOC0xMyAxNzoyOSwgSmFuIEJldWxp
Y2ggPz8/Pz8/Ogo+Pj4+Pj4+IE9uIDEzLjA4LjEyIGF0IDA5OjU4LCAiemhlbnpob25nLmR1YW4i
PHpoZW56aG9uZy5kdWFuQG9yYWNsZS5jb20+Cj4+Pj4+Pj4gd3JvdGU6Cj4+Pj4+ID8/PyAyMDEy
LTA4LTEwIDIyOjIyLCBKYW4gQmV1bGljaCA/Pz8/Pz86Cj4+Pj4+PiBHb2luZyBiYWNrIHRvIHlv
dXIgb3JpZ2luYWwgbWFpbCwgSSB3b25kZXIgaG93ZXZlciB3aHkgdGhpcwo+Pj4+Pj4gZ2V0cyBk
b25lIGF0IGFsbC4gWW91IHNhaWQgaXQgZ290IHRoZXJlIHZpYQo+Pj4+Pj4KPj4+Pj4+IG10cnJf
YXBzX2luaXQoKQo+Pj4+Pj4gICAgXC0+ICAgIHNldF9tdHJyKCkKPj4+Pj4+ICAgICAgICBcLT4g
ICAgbXRycl93b3JrX2hhbmRsZXIoKQo+Pj4+Pj4KPj4+Pj4+IHlldCB0aGlzIGlzbid0IGRvbmUg
dW5jb25kaXRpb25hbGx5IC0gc2VlIHRoZSBjb21tZW50IGJlZm9yZQo+Pj4+Pj4gY2hlY2tpbmcg
bXRycl9hcHNfZGVsYXllZF9pbml0LiBDYW4geW91IGZpbmQgb3V0IHdoZXJlIHRoZQo+Pj4+Pj4g
b2J2aW91c2x5IG5lY2Vzc2FyeSBjYWxsKHMpIHRvIHNldF9tdHJyX2Fwc19kZWxheWVkX2luaXQo
KQo+Pj4+Pj4gY29tZShzKSBmcm9tPwo+Pj4+PiBBdCBib290dXAgc3RhZ2UsIHNldF9tdHJyX2Fw
c19kZWxheWVkX2luaXQgaXMgY2FsbGVkIGJ5Cj4+Pj4+IG5hdGl2ZV9zbXBfcHJlcGFyZV9jcHVz
Lgo+Pj4+PiBtdHJyX2Fwc19kZWxheWVkX2luaXQgaXMgYWx3YXlzIHNldCB0byB0dXJlIGZvciBp
bnRlbCBwcm9jZXNzb3IgaW4KPj4+Pj4gdXBzdHJlYW0KPj4+Pj4gY29kZS4KPj4+PiBJbmRlZWQs
IGFuZCB0aGF0IChpbiBvbmUgZm9ybSBvciBhbm90aGVyKSBoYXMgYmVlbiBkb25lCj4+Pj4gdmly
dHVhbGx5IGZvcmV2ZXIgaW4gTGludXguIEkgd29uZGVyIHdoeSB0aGUgcHJvYmxlbSB3YXNuJ3QK
Pj4+PiBub3RpY2VkIChvciBsb29rZWQgaW50bywgaWYgaXQgd2FzIG5vdGljZWQpIHNvIGZhci4K
Pj4+Pgo+Pj4+IEFzIGl0J3MgZ29pbmcgdG8gYmUgcmF0aGVyIGRpZmZpY3VsdCB0byBjb252aW5j
ZSB0aGUgTGludXggZm9sa3MKPj4+PiB0byBjaGFuZ2UgdGhlaXIgY29kZSAocGx1cyB0aGlzIHdv
dWxkbid0IGhlbHAgd2l0aCBleGlzdGluZwo+Pj4+IGtlcm5lbHMgYW55d2F5KSwgd2UnbGwgbmVl
ZCB0byBmaW5kIGEgd2F5IHRvIGltcHJvdmUgdGhpcyBpbgo+Pj4+IHRoZSBoeXBlcnZpc29yLgo+
Pj4gSGkgSmFuLCBUaW0KPj4+IElzIHRoaXMgaXNzdWUgaW1wcm92YWJsZSBmcm9tIHhlbiBzaWRl
Pwo+PiBQcm9iYWJseTsgd2UncmUgbG9va2luZyBpbnRvIHRoZSBiZXN0IHdheSB0byBhZGRyZXNz
IGl0Lgo+Cj4gSXMgdGhlcmUgYW55IHBhdGNoIGZvciB1cyB0byB0ZXN0PwoKTm8sIHNvcnJ5LgoK
SmFuCgpfX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fXwpYZW4t
ZGV2ZWwgbWFpbGluZyBsaXN0Clhlbi1kZXZlbEBsaXN0cy54ZW4ub3JnCmh0dHA6Ly9saXN0cy54
ZW4ub3JnL3hlbi1kZXZlbAo=

From mboxrd@z Thu Jan  1 00:00:00 1970
From: Konrad Rzeszutek Wilk 
Subject: Re: kernel bootup slow issue on ovm3.1.1
Date: Mon, 29 Apr 2013 13:55:56 -0400
Message-ID: <20130429175556.GA3921@phenom.dumpdata.com>
References: <502235E8.9040309@oracle.com>
	<50229B840200007800093A73@nat28.tlf.novell.com>
	<5023860E.7080908@oracle.com>
	<5023AE960200007800093DE8@nat28.tlf.novell.com>
	<502490A7.7020603@oracle.com>
	<502535280200007800094322@nat28.tlf.novell.com>
	<5028B3AB.7060705@oracle.com>
	<5028E53202000078000946B1@nat28.tlf.novell.com>
	<503DAA5F.5030306@oracle.com>
	<20120830090312.GA54290@ocelot.phlegethon.org>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Return-path: 
Content-Disposition: inline
In-Reply-To: <20120830090312.GA54290@ocelot.phlegethon.org>
List-Unsubscribe: ,
	
List-Post: 
List-Help: 
List-Subscribe: ,
	
Sender: xen-devel-bounces@lists.xen.org
Errors-To: xen-devel-bounces@lists.xen.org
To: Tim Deegan 
Cc: Satish Kantheti , Stefano Stabellini , Feng Jin , "zhenzhong.duan" , xen-devel , Jan Beulich 
List-Id: xen-devel@lists.xenproject.org

On Thu, Aug 30, 2012 at 10:03:12AM +0100, Tim Deegan wrote:
> At 13:36 +0800 on 29 Aug (1346247391), zhenzhong.duan wrote:
> > 
> > 
> > ??? 2012-08-13 17:29, Jan Beulich ??????:
> > >>>>On 13.08.12 at 09:58, "zhenzhong.duan"  
> > >>>>wrote:
> > >>??? 2012-08-10 22:22, Jan Beulich ??????:
> > >>>Going back to your original mail, I wonder however why this
> > >>>gets done at all. You said it got there via
> > >>>
> > >>>mtrr_aps_init()
> > >>>   \->   set_mtrr()
> > >>>       \->   mtrr_work_handler()
> > >>>
> > >>>yet this isn't done unconditionally - see the comment before
> > >>>checking mtrr_aps_delayed_init. Can you find out where the
> > >>>obviously necessary call(s) to set_mtrr_aps_delayed_init()
> > >>>come(s) from?
> > >>At bootup stage, set_mtrr_aps_delayed_init is called by
> > >>native_smp_prepare_cpus.
> > >>mtrr_aps_delayed_init is always set to ture for intel processor in 
> > >>upstream
> > >>code.
> > >Indeed, and that (in one form or another) has been done
> > >virtually forever in Linux. I wonder why the problem wasn't
> > >noticed (or looked into, if it was noticed) so far.
> > >
> > >As it's going to be rather difficult to convince the Linux folks
> > >to change their code (plus this wouldn't help with existing
> > >kernels anyway), we'll need to find a way to improve this in
> > >the hypervisor.
> > Hi Jan, Tim
> > Is this issue improvable from xen side?
> 
> Probably; we're looking into the best way to address it. 
> 
> Tim.

Ping? Was there any progress on this? Thanks

From mboxrd@z Thu Jan  1 00:00:00 1970
From: George Dunlap 
Subject: Re: kernel bootup slow issue on ovm3.1.1
Date: Tue, 30 Apr 2013 11:37:47 +0100
Message-ID: 
References: <502235E8.9040309@oracle.com>
	<50229B840200007800093A73@nat28.tlf.novell.com>
	<5023860E.7080908@oracle.com>
	<5023AE960200007800093DE8@nat28.tlf.novell.com>
	<502490A7.7020603@oracle.com>
	<502535280200007800094322@nat28.tlf.novell.com>
	<5028B3AB.7060705@oracle.com>
	<5028E53202000078000946B1@nat28.tlf.novell.com>
	<503DAA5F.5030306@oracle.com>
	<20120830090312.GA54290@ocelot.phlegethon.org>
	<20130429175556.GA3921@phenom.dumpdata.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Return-path: 
In-Reply-To: <20130429175556.GA3921@phenom.dumpdata.com>
List-Unsubscribe: ,
	
List-Post: 
List-Help: 
List-Subscribe: ,
	
Sender: xen-devel-bounces@lists.xen.org
Errors-To: xen-devel-bounces@lists.xen.org
To: Konrad Rzeszutek Wilk 
Cc: Satish Kantheti , Stefano Stabellini , Feng Jin , Tim Deegan , "zhenzhong.duan" , xen-devel , Jan Beulich 
List-Id: xen-devel@lists.xenproject.org

On Mon, Apr 29, 2013 at 6:55 PM, Konrad Rzeszutek Wilk
 wrote:
> On Thu, Aug 30, 2012 at 10:03:12AM +0100, Tim Deegan wrote:
>> At 13:36 +0800 on 29 Aug (1346247391), zhenzhong.duan wrote:
>> >
>> >
>> > ??? 2012-08-13 17:29, Jan Beulich ??????:
>> > >>>>On 13.08.12 at 09:58, "zhenzhong.duan"
>> > >>>>wrote:
>> > >>??? 2012-08-10 22:22, Jan Beulich ??????:
>> > >>>Going back to your original mail, I wonder however why this
>> > >>>gets done at all. You said it got there via
>> > >>>
>> > >>>mtrr_aps_init()
>> > >>>   \->   set_mtrr()
>> > >>>       \->   mtrr_work_handler()
>> > >>>
>> > >>>yet this isn't done unconditionally - see the comment before
>> > >>>checking mtrr_aps_delayed_init. Can you find out where the
>> > >>>obviously necessary call(s) to set_mtrr_aps_delayed_init()
>> > >>>come(s) from?
>> > >>At bootup stage, set_mtrr_aps_delayed_init is called by
>> > >>native_smp_prepare_cpus.
>> > >>mtrr_aps_delayed_init is always set to ture for intel processor in
>> > >>upstream
>> > >>code.
>> > >Indeed, and that (in one form or another) has been done
>> > >virtually forever in Linux. I wonder why the problem wasn't
>> > >noticed (or looked into, if it was noticed) so far.
>> > >
>> > >As it's going to be rather difficult to convince the Linux folks
>> > >to change their code (plus this wouldn't help with existing
>> > >kernels anyway), we'll need to find a way to improve this in
>> > >the hypervisor.
>> > Hi Jan, Tim
>> > Is this issue improvable from xen side?
>>
>> Probably; we're looking into the best way to address it.
>>
>> Tim.
>
> Ping? Was there any progress on this? Thanks

Does this need to be added to our tracking list?

 -George