KVM/ia64 > 3GB or RAM?

KVM/ia64 > 3GB or RAM?

[Jes Sorensen]

Hi,

Does anyone remember how far back it was for KVM on ia64 to still work
with more than 3GB of memory assigned to the virtual machine?

Thanks,
Jes

Re: KVM/ia64 > 3GB or RAM?

[Jes Sorensen]

>>>>> "Jes" = Jes Sorensen <jes@sgi.com> writes:

Jes> Hi, Does anyone remember how far back it was for KVM on ia64 to
Jes> still work with more than 3GB of memory assigned to the virtual
Jes> machine?

Hi,

I have done some more investiation on this problem and found that
things go wrong the moment we try and register any memory outside the
4GB window.

In addition I noticed that even though I register the > 4GB memory
from say 0x400000000 the firmware image still seems to think it starts
at 0x100000000  - anyone can tell me how the firmware gets it's
addresses?

Thanks,
Jes

RE: KVM/ia64 > 3GB or RAM?

[Zhang, Xiantao]

Jes Sorensen wrote:
>>>>>> "Jes" = Jes Sorensen <jes@sgi.com> writes:
> 
> Jes> Hi, Does anyone remember how far back it was for KVM on ia64 to
> Jes> still work with more than 3GB of memory assigned to the virtual
> Jes> machine?
> 
> Hi,
> 
> I have done some more investiation on this problem and found that
> things go wrong the moment we try and register any memory outside the
> 4GB window.
> 
> In addition I noticed that even though I register the > 4GB memory
> from say 0x400000000 the firmware image still seems to think it starts
> at 0x100000000  - anyone can tell me how the firmware gets it's
> addresses?
Hi, Jes
	Seems I missed some mails from kvm/ia64 list, because my mailing
address is unsubscribed by unknow reason days ago. For guest's physical
address space,  0-3G is conventional ram, 3G-3.5G is allocated for mmio
address sapce. 4G- higher address is also ram. The memory size is
transferred to firmware at creating guest time.  In firmware, it also
has the knowledge about guest memory layout from memory size, so it will
know where is ram and where is mmio...   I have a fix about this issue,
and you should can get it in mailing list. :)
Thanks
Xiantao

Re: KVM/ia64 > 3GB or RAM?

[Jes Sorensen]

Zhang, Xiantao wrote:
> Hi, Jes
> 	Seems I missed some mails from kvm/ia64 list, because my mailing
> address is unsubscribed by unknow reason days ago. For guest's physical
> address space,  0-3G is conventional ram, 3G-3.5G is allocated for mmio
> address sapce. 4G- higher address is also ram. The memory size is
> transferred to firmware at creating guest time.  In firmware, it also
> has the knowledge about guest memory layout from memory size, so it will
> know where is ram and where is mmio...   I have a fix about this issue,
> and you should can get it in mailing list. :)
> Thanks
> Xiantao

Hi Xiantao,

Hmmm I didn't realize this was hardcoded into the firmware image :-(
at least that explains why changing the addresses fails so badly.

I still have major problems with more than 3GB of memory though, the
guest gets memory corruption and crashes fast. It happens as soon as
anything is mapped above 4GB, as little as 2MB extra is enough to break
it.

Cheers,
Jes

RE: KVM/ia64 > 3GB or RAM?

[Zhang, Xiantao]

[-- Attachment #1: Type: text/plain, Size: 1672 bytes --]

 Hi, Jes
     Maybe what you met is the same issue with that is found in xen
side, and see the attached mail to get the detail.  It should be a
firmware bug and Tristan should be working on it.  Guests may get crash
when configured with special size of memory, such as 3073, 3074 etc...
But anyway if you configure guests with memory >4000M, guest should have
no issue.  BTW, please make sure you have applied the attached patch.  
Thanks
Xiantao


-----Original Message-----
From: Jes Sorensen [mailto:jes@sgi.com] 
Sent: Tuesday, September 23, 2008 6:45 PM
To: Zhang, Xiantao
Cc: kvm-ia64@vger.kernel.org
Subject: Re: KVM/ia64 > 3GB or RAM?

Zhang, Xiantao wrote:
> Hi, Jes
> 	Seems I missed some mails from kvm/ia64 list, because my mailing
> address is unsubscribed by unknow reason days ago. For guest's
physical
> address space,  0-3G is conventional ram, 3G-3.5G is allocated for
mmio
> address sapce. 4G- higher address is also ram. The memory size is
> transferred to firmware at creating guest time.  In firmware, it also
> has the knowledge about guest memory layout from memory size, so it
will
> know where is ram and where is mmio...   I have a fix about this
issue,
> and you should can get it in mailing list. :)
> Thanks
> Xiantao

Hi Xiantao,

Hmmm I didn't realize this was hardcoded into the firmware image :-(
at least that explains why changing the addresses fails so badly.

I still have major problems with more than 3GB of memory though, the
guest gets memory corruption and crashes fast. It happens as soon as
anything is mapped above 4GB, as little as 2MB extra is enough to break
it.

Cheers,
Jes

[-- Attachment #2: 0001-kvm-ia64-Qemu-Fix-Guest-boot-issue-with-3G-memor.patch --]
[-- Type: application/octet-stream, Size: 920 bytes --]

From a6703684b67518ca614bbd2c23060d8f502136ce Mon Sep 17 00:00:00 2001
From: Xiantao Zhang <xiantao.zhang@intel.com>
Date: Thu, 18 Sep 2008 14:07:00 +0800
Subject: [PATCH] kvm/ia64: Qemu : Fix Guest boot issue with >3G memory.

I have fixed it before, but the patch was removed by 77c9148ba4a8 accidently,
when deal with merge conflicts.
Signed-off-by: Xiantao Zhang <xiantao.zhang@intel.com>
---
 qemu/exec.c |    2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)

diff --git a/qemu/exec.c b/qemu/exec.c
index bf037f0..7cb811b 100644
--- a/qemu/exec.c
+++ b/qemu/exec.c
@@ -82,6 +82,8 @@
 #define TARGET_PHYS_ADDR_SPACE_BITS 42
 #elif defined(TARGET_I386) && !defined(USE_KQEMU)
 #define TARGET_PHYS_ADDR_SPACE_BITS 36
+#elif defined(TARGET_IA64)
+#define TARGET_PHYS_ADDR_SPACE_BITS 36
 #else
 /* Note: for compatibility with kqemu, we use 32 bits for x86_64 */
 #define TARGET_PHYS_ADDR_SPACE_BITS 32
-- 
1.5.1


[-- Attachment #3: Type: message/rfc822, Size: 5010 bytes --]

From: "Akio Takebe" <takebe_akio@jp.fujitsu.com>
To: "Zhang, Jingke" <jingke.zhang@intel.com>
Cc: <xen-ia64-devel@lists.xensource.com>
Subject: Re: [Xen-ia64-devel] RE: VTI will crash with memory=3073M
Date: Tue, 2 Sep 2008 13:19:31 +0800
Message-ID: <48BCCCE3.2020104@jp.fujitsu.com>

Zhang, Jingke wrote:
> Tristan Gingold wrote:
>> On Tue, Sep 02, 2008 at 11:43:58AM +0800, Zhang, Jingke wrote:
>>> Hi Tristan,
>>>     We found VTI guest with 3073M (memory is a little more than 3G)
>>>     can not be booted up. After some investigation, our engineer
>>> found some pages are ruined by unknow reason.  A EFI driver was
>>> loaded to 4G+, and its initilization code used the ruined pages, so
>>> bug out. I used latest openGFW binary (xenia64-gfw-126.bin). This
>>> issue is very easy to reproduce. Could you please help to look at
>>> it? Thank you very much!
>> Hi,
>>
>> Can you explain what do you mean by 'ruined' ?
>>
>> Tristan.
> 
> Hi Tristan,
>     We found some code pages are polluted by an EFI driver (currently, we did not locate which driver), and leads to the issue. Thanks!
I guess the same problem.
http://lists.xensource.com/archives/html/xen-ia64-devel/2008-07/msg00200.html

Best Regards,

Akio Takebe

_______________________________________________
Xen-ia64-devel mailing list
Xen-ia64-devel@lists.xensource.com
http://lists.xensource.com/xen-ia64-devel

Re: KVM/ia64 > 3GB or RAM?

[Jes Sorensen]

Zhang, Xiantao wrote:
>  Hi, Jes
>      Maybe what you met is the same issue with that is found in xen
> side, and see the attached mail to get the detail.  It should be a
> firmware bug and Tristan should be working on it.  Guests may get crash
> when configured with special size of memory, such as 3073, 3074 etc...
> But anyway if you configure guests with memory >4000M, guest should have
> no issue.  BTW, please make sure you have applied the attached patch.  
> Thanks
> Xiantao

Thats it!

Now I can boot a 4G guest - before anything over 3072M would explode.

Lets get that patch to Avi asap.

Thanks,
Jes

RE: KVM/ia64 > 3GB or RAM?

[Zhang, Xiantao]

[-- Attachment #1: Type: text/plain, Size: 921 bytes --]

Jes Sorensen wrote:
> Zhang, Xiantao wrote:
>>  Hi, Jes
>>      Maybe what you met is the same issue with that is found in xen
>> side, and see the attached mail to get the detail.  It should be a
>> firmware bug and Tristan should be working on it.  Guests may get
>> crash when configured with special size of memory, such as 3073,
>> 3074 etc... But anyway if you configure guests with memory >4000M,
>> guest should have no issue.  BTW, please make sure you have applied
>> the attached patch. Thanks Xiantao
> 
> Thats it!
> 
> Now I can boot a 4G guest - before anything over 3072M would explode.
> 
> Lets get that patch to Avi asap.

I have posted it to mailing list last week, but seem Avi hasn't picked
it up yet. 
Avi, 
    Could you help to apply the attached patch ? It is missing after
your merge with Qemu upstream, and it breaks ia64 guests to configure >
3G ram. 
Thanks
Xiantao

[-- Attachment #2: 0001-kvm-ia64-Qemu-Fix-Guest-boot-issue-with-3G-memor.patch --]
[-- Type: application/octet-stream, Size: 920 bytes --]

From a6703684b67518ca614bbd2c23060d8f502136ce Mon Sep 17 00:00:00 2001
From: Xiantao Zhang <xiantao.zhang@intel.com>
Date: Thu, 18 Sep 2008 14:07:00 +0800
Subject: [PATCH] kvm/ia64: Qemu : Fix Guest boot issue with >3G memory.

I have fixed it before, but the patch was removed by 77c9148ba4a8 accidently,
when deal with merge conflicts.
Signed-off-by: Xiantao Zhang <xiantao.zhang@intel.com>
---
 qemu/exec.c |    2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)

diff --git a/qemu/exec.c b/qemu/exec.c
index bf037f0..7cb811b 100644
--- a/qemu/exec.c
+++ b/qemu/exec.c
@@ -82,6 +82,8 @@
 #define TARGET_PHYS_ADDR_SPACE_BITS 42
 #elif defined(TARGET_I386) && !defined(USE_KQEMU)
 #define TARGET_PHYS_ADDR_SPACE_BITS 36
+#elif defined(TARGET_IA64)
+#define TARGET_PHYS_ADDR_SPACE_BITS 36
 #else
 /* Note: for compatibility with kqemu, we use 32 bits for x86_64 */
 #define TARGET_PHYS_ADDR_SPACE_BITS 32
-- 
1.5.1