[PATCH] Patchset to enable kvm/ia64 support.

[PATCH] Patchset to enable kvm/ia64 support.

[Zhang, Xiantao]

[-- Attachment #1: Type: text/plain, Size: 3603 bytes --]

Hi, all 
As you know, in the past two months, commnity worked out cross-arch
framework support for kvm.  Thank you all for your contributions !
Especially for Avi's quick response in mailing list! ! 
Currently, following userspace suppport, we have completed code rebase
for kvm/ia64 support according to new framework.  With its support, we
can boot up windows/ Linux guests with open guest firmware, and get
reasonable performance compared with other virtualization projects.  The
current status are as following:
1. Both UP Linux & Windows  are OK, but short of SMP guest support. (see
attached screen snapshort)
2. Stability and performance are also good!
3. Short of save/restore live migration support. 
4. Short of host swapping support.

Kvm/ia64 architecture also has two parts, one is kvm module, and the
other one is GVMM module.  The responsibility of kvm module is similar
with x86's, and you can refer to kvm's docs for more information. GVMM
module is mainly focusing on performance-critical component
virutlization, such as processor and memory virtulization.  They are
working together to build virtualization environment for virtual
machines. I attached a figure to better understand  their workflow. From
the graph, we can see transition code is used for world switch. In this
transition code, it completes the address space switch, and this is a
must, because processor hardware doesn't provide address switch
machanism due to large register set in Itanium processors.  Now, GVMM
runs in isolated address space with host side, here we use
0xd000000000000000 to avoid possbile confilicts with host side.   

In this mode, there are two kinds of vm exits, one is called
light-weight exits, such as memory tlb miss,  priviliged register
access. They can be handled in GVMM side, and doen't need to switch back
to host side.  The other one is heavey-weight exit, they need to switch
back to host side, such as IO operations, Physical external interrupts,
firmware emulation, guest IPI and so on.  You know,  light-weight exits
should be dominant for all exit events.  Even if heavey-weight exit,  it
only consumes hundreds itc cycles.  So ,it doesn't have big performance
impact, and our performance data also confirms our guess.  

In all, let's work togerther to make it run better and better. 

But, anyway, this is the first time for us to see kvm boot up on non-x86
processors. :)

Now, it's time for us sending the code out for review and check-in.  We
are appreciating any comments ! :)
[1/17] Adding kvm compile support in Kconfig and Makefile file
[2/17] Add kvm.h, kvm_para.h and kvm_host.h for ia64. 
[3/17] Add kvm-ia64.c to support kvm module on Itanium processors. 
[4/17] Add firmware virtualization support. 
[5/17] Add TR operation routines.
[6/17] Add some header files for kvm use.
[7/17] Add GVMM interfaces, it provides GVMM information for host use.
[8/17] TLB virtulization
[9/17] Add MMIO decoder support.
[10/17] Add Interruption vector table for GVMM
[11/17] Add trampoline code for host/guest mode switch.
[12/17] Add processor virtulization support
[13/17] Some optimizations for virtulization faults.
[14/17] Generating offset values for assembly code use. 
[15/17] Guest interruption injection support.
[16/17] Add Kconfig for choosing kvm support on IA64.
[17/17] Add Makefile for compiling source files. 

In addition, current linux upstream tree is not fully friendly to kvm,
so a kernel patch is prepared for kvm, and will be sending to linux-ia64
mailing list.

Thanks. 
Xiantao 


[-- Attachment #2: kvm.JPG --]
[-- Type: image/jpeg, Size: 27144 bytes --]

[-- Attachment #3: win.JPG --]
[-- Type: image/jpeg, Size: 23010 bytes --]

[-- Attachment #4: linux.JPG --]
[-- Type: image/jpeg, Size: 34506 bytes --]

[-- Attachment #5: Type: text/plain, Size: 308 bytes --]

-------------------------------------------------------------------------
SF.Net email is sponsored by:
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services
for just about anything Open Source.
http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace

[-- Attachment #6: Type: text/plain, Size: 186 bytes --]

_______________________________________________
kvm-devel mailing list
kvm-devel-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org
https://lists.sourceforge.net/lists/listinfo/kvm-devel

Re: [PATCH] Patchset to enable kvm/ia64 support.

[Izik Eidus]

On Wed, 2007-12-19 at 20:18 +0800, Zhang, Xiantao wrote:
> 
> Now, it's time for us sending the code out for review and check-in.  We
> are appreciating any comments ! :)
> [1/17] Adding kvm compile support in Kconfig and Makefile file
> [2/17] Add kvm.h, kvm_para.h and kvm_host.h for ia64. 
> [3/17] Add kvm-ia64.c to support kvm module on Itanium processors. 
> [4/17] Add firmware virtualization support. 
> [5/17] Add TR operation routines.
> [6/17] Add some header files for kvm use.
> [7/17] Add GVMM interfaces, it provides GVMM information for host use.
> [8/17] TLB virtulization

i think this patch is missing?



-------------------------------------------------------------------------
SF.Net email is sponsored by:
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services
for just about anything Open Source.
http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace

Re: [PATCH] Patchset to enable kvm/ia64 support.

[Zhang, Xiantao]

Izik Eidus wrote:
> On Wed, 2007-12-19 at 20:18 +0800, Zhang, Xiantao wrote:
>> 
>> Now, it's time for us sending the code out for review and check-in. 
>> We are appreciating any comments ! :)
>> [1/17] Adding kvm compile support in Kconfig and Makefile file
>> [2/17] Add kvm.h, kvm_para.h and kvm_host.h for ia64.
>> [3/17] Add kvm-ia64.c to support kvm module on Itanium processors.
>> [4/17] Add firmware virtualization support.
>> [5/17] Add TR operation routines.
>> [6/17] Add some header files for kvm use.
>> [7/17] Add GVMM interfaces, it provides GVMM information for host
>> use. [8/17] TLB virtulization
> 
> i think this patch is missing?

I have sent out them, maybe need wait more minutes :)
Xiantao

-------------------------------------------------------------------------
SF.Net email is sponsored by:
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services
for just about anything Open Source.
http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace

Re: [PATCH] Patchset to enable kvm/ia64 support.

[Amit Shah]

On Wednesday 19 December 2007 17:48:14 Zhang, Xiantao wrote:
> Hi, all
> As you know, in the past two months, commnity worked out cross-arch
> framework support for kvm.  Thank you all for your contributions !
> Especially for Avi's quick response in mailing list! !
> Currently, following userspace suppport, we have completed code rebase
> for kvm/ia64 support according to new framework.  With its support, we
> can boot up windows/ Linux guests with open guest firmware, and get
> reasonable performance compared with other virtualization projects.  The
> current status are as following:
> 1. Both UP Linux & Windows  are OK, but short of SMP guest support. (see
> attached screen snapshort)
> 2. Stability and performance are also good!
> 3. Short of save/restore live migration support.
> 4. Short of host swapping support.

> Kvm/ia64 architecture also has two parts, one is kvm module, and the
> other one is GVMM module.  The responsibility of kvm module is similar
> with x86's, and you can refer to kvm's docs for more information. GVMM
> module is mainly focusing on performance-critical component
> virutlization, such as processor and memory virtulization.  They are
> working together to build virtualization environment for virtual
> machines. I attached a figure to better understand  their workflow. From
> the graph, we can see transition code is used for world switch. In this
> transition code, it completes the address space switch, and this is a
> must, because processor hardware doesn't provide address switch
> machanism due to large register set in Itanium processors.  Now, GVMM
> runs in isolated address space with host side, here we use
> 0xd000000000000000 to avoid possbile confilicts with host side.
>
> In this mode, there are two kinds of vm exits, one is called
> light-weight exits, such as memory tlb miss,  priviliged register
> access. They can be handled in GVMM side, and doen't need to switch back
> to host side.  The other one is heavey-weight exit, they need to switch
> back to host side, such as IO operations, Physical external interrupts,
> firmware emulation, guest IPI and so on.  You know,  light-weight exits
> should be dominant for all exit events.  Even if heavey-weight exit,  it
> only consumes hundreds itc cycles.  So ,it doesn't have big performance
> impact, and our performance data also confirms our guess.
>
> In all, let's work togerther to make it run better and better.
>
> But, anyway, this is the first time for us to see kvm boot up on non-x86
> processors. :)

Great! Looks like we'll soon have support for more architectures :-)

Can you please also put this documentation on the wiki? Some of the parts that 
are not expected to change much can also be put in Documentation/, with the 
rest being a pointer to the wiki.

Since the 0/nn patches aren't committed, such documentation can be easily 
lost.

> Now, it's time for us sending the code out for review and check-in.  We
> are appreciating any comments ! :)
> [1/17] Adding kvm compile support in Kconfig and Makefile file

Please order patches so that after applying each one, the kernel is buildable. 
This one adds entries for arch/ia64/kvm/Kconfig, but those files aren't added 
yet. Compiles will fail and bisecting will just add more frustration while 
hunting down that elusive bug.

> [2/17] Add kvm.h, kvm_para.h and kvm_host.h for ia64.
> [3/17] Add kvm-ia64.c to support kvm module on Itanium processors.

I see code that's commented out. Do you plan to send in cleanup patches later?

Also, have you run these through checkpatch.pl? It looks like there'll be a 
few complaints.

> [4/17] Add firmware virtualization support.

Can you explain (in the git commit message) what this is? There are also a few 
typos, like pal_logical_to_physica() (missing 'l' for physical).

Also, please add #defines for the constants used.

Indentation is also a problem in some places.

Mostly, these comments apply to all the patches in this series.

> [5/17] Add TR operation routines.
> [6/17] Add some header files for kvm use.
> [7/17] Add GVMM interfaces, it provides GVMM information for host use.
> [8/17] TLB virtulization
> [9/17] Add MMIO decoder support.
> [10/17] Add Interruption vector table for GVMM
> [11/17] Add trampoline code for host/guest mode switch.
> [12/17] Add processor virtulization support
> [13/17] Some optimizations for virtulization faults.
> [14/17] Generating offset values for assembly code use.
> [15/17] Guest interruption injection support.
> [16/17] Add Kconfig for choosing kvm support on IA64.
> [17/17] Add Makefile for compiling source files.
>
> In addition, current linux upstream tree is not fully friendly to kvm,
> so a kernel patch is prepared for kvm, and will be sending to linux-ia64
> mailing list.

Why don't you send these patches to the ia64 list as well? I think there can 
be better review of the code from the people who understand the arch.

>
> Thanks.
> Xiantao

Amit

-------------------------------------------------------------------------
SF.Net email is sponsored by:
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services
for just about anything Open Source.
http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace

Re: [PATCH] Patchset to enable kvm/ia64 support.

[Zhang, Xiantao]

Amit Shah wrote:
> On Wednesday 19 December 2007 17:48:14 Zhang, Xiantao wrote:
>> Hi, all
>> As you know, in the past two months, commnity worked out cross-arch
>> framework support for kvm.  Thank you all for your contributions !
>> Especially for Avi's quick response in mailing list! !
>> Currently, following userspace suppport, we have completed code
>> rebase for kvm/ia64 support according to new framework.  With its
>> support, we can boot up windows/ Linux guests with open guest
>> firmware, and get reasonable performance compared with other
>> virtualization projects.  The current status are as following:
>> 1. Both UP Linux & Windows  are OK, but short of SMP guest support.
>> (see attached screen snapshort) 
>> 2. Stability and performance are also good!
>> 3. Short of save/restore live migration support.
>> 4. Short of host swapping support.
> 
>> Kvm/ia64 architecture also has two parts, one is kvm module, and the
>> other one is GVMM module.  The responsibility of kvm module is
>> similar with x86's, and you can refer to kvm's docs for more
>> information. GVMM module is mainly focusing on performance-critical
>> component virutlization, such as processor and memory virtulization.
>> They are working together to build virtualization environment for
>> virtual machines. I attached a figure to better understand  their
>> workflow. From the graph, we can see transition code is used for
>> world switch. In this transition code, it completes the address
>> space switch, and this is a must, because processor hardware doesn't
>> provide address switch machanism due to large register set in
>> Itanium processors.  Now, GVMM runs in isolated address space with
>> host side, here we use 0xd000000000000000 to avoid possbile
>> confilicts with host side. 
>> 
>> In this mode, there are two kinds of vm exits, one is called
>> light-weight exits, such as memory tlb miss,  priviliged register
>> access. They can be handled in GVMM side, and doen't need to switch
>> back to host side.  The other one is heavey-weight exit, they need
>> to switch back to host side, such as IO operations, Physical
>> external interrupts, firmware emulation, guest IPI and so on.  You
>> know,  light-weight exits should be dominant for all exit events. 
>> Even if heavey-weight exit,  it only consumes hundreds itc cycles. 
>> So ,it doesn't have big performance impact, and our performance data
>> also confirms our guess. 
>> 
>> In all, let's work togerther to make it run better and better.
>> 
>> But, anyway, this is the first time for us to see kvm boot up on
>> non-x86 processors. :)
> 
> Great! Looks like we'll soon have support for more architectures :-)
> 
> Can you please also put this documentation on the wiki? Some of the
> parts that are not expected to change much can also be put in
> Documentation/, with the rest being a pointer to the wiki.
> 
> Since the 0/nn patches aren't committed, such documentation can be
> easily lost.

I will update it. Thank you! :)

>> Now, it's time for us sending the code out for review and check-in. 
>> We are appreciating any comments ! :)
>> [1/17] Adding kvm compile support in Kconfig and Makefile file
> 
> Please order patches so that after applying each one, the kernel is
> buildable. This one adds entries for arch/ia64/kvm/Kconfig, but those
> files aren't added yet. Compiles will fail and bisecting will just
> add more frustration while hunting down that elusive bug.

OK, this should be the last one. And others has no order requirement, I
think. 

>> [2/17] Add kvm.h, kvm_para.h and kvm_host.h for ia64.
>> [3/17] Add kvm-ia64.c to support kvm module on Itanium processors.
> 
> I see code that's commented out. Do you plan to send in cleanup
> patches later? 

Yes, some code need to keep it there for some hints, since some
functionality doesn't implment completely yet.

> Also, have you run these through checkpatch.pl? It looks like
> there'll be a few complaints.

I have checked it with this scripts.  Only some warnings didn't to
solve, such as 
If (xx) {
	xxxxxxxx
}
It will report an WARNING: braces {} are not necessary for single
statement blocks. Do we need to clean up them ?

But for assembly code, seems this script is not smart, and report
inexistent style issues. 

>> [4/17] Add firmware virtualization support.
> 
> Can you explain (in the git commit message) what this is? There are
> also a few typos, like pal_logical_to_physica() (missing 'l' for
> physical). 

Oh, that should be a typo. I will check it again. Thank you!

> Also, please add #defines for the constants used.
> 
> Indentation is also a problem in some places.
> 
> Mostly, these comments apply to all the patches in this series.

>> [5/17] Add TR operation routines.
>> [6/17] Add some header files for kvm use.
>> [7/17] Add GVMM interfaces, it provides GVMM information for host
>> use. [8/17] TLB virtulization [9/17] Add MMIO decoder support.
>> [10/17] Add Interruption vector table for GVMM
>> [11/17] Add trampoline code for host/guest mode switch.
>> [12/17] Add processor virtulization support
>> [13/17] Some optimizations for virtulization faults.
>> [14/17] Generating offset values for assembly code use.
>> [15/17] Guest interruption injection support.
>> [16/17] Add Kconfig for choosing kvm support on IA64.
>> [17/17] Add Makefile for compiling source files.
>> 
>> In addition, current linux upstream tree is not fully friendly to
>> kvm, so a kernel patch is prepared for kvm, and will be sending to
>> linux-ia64 mailing list.
> 
> Why don't you send these patches to the ia64 list as well? I think
> there can be better review of the code from the people who understand
> the arch. 

Good idea! I will send out an updated version, and also cc to ia64 list
:)

Xiantao


-------------------------------------------------------------------------
SF.Net email is sponsored by:
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services
for just about anything Open Source.
http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace