[PATCH 0/6][RFC] kvmppc: paravirtualization interface

[PATCH 0/6][RFC] kvmppc: paravirtualization interface

[ehrhardt]

From: Christian Ehrhardt <ehrhardt@linux.vnet.ibm.com>

This patch series implements a paravirtualization interface using:
- the device tree mechanism to pass hypervisor informations to the guest
- hypercalls for guest->host calls
- an example exploiter of that interface (magic page)
This is work in progress, but working so far. I just start to really exploit
the fuctionality behind the magic page mechanism therefor I can't provide any
performance improvements so far, but it is evolved enough for RFC and to start
the standardization discussion.

The used hypercall ABI was already discussed on the embedded-hypervisor mailing
list and is available at http://kvm.qumranet.com/kvmwiki/PowerPC_Hypercall_ABI

The device tree format used here (=base for the discussions on
embedded-hypervisor) is the following.
- A node "hypervisor" to show the general availability of some hypervisor data
- flags for features like the example "feature,pv-magicpage"
  setting 1 = available, everything else = unavailable
- Some features might need to pass more data and can use an entry in the
  device tree like the example of "data,pv-magicpage-size"

Parties on cc:
linuxppc-dev@ozlabs.org
  The patches affect code in the generic powerpc boot&setup so I would be
  happy about comments if the hooks are ok that way.
embedded-hypervisor@power.org
  This power.org TSC discusses about standardization of the virtualization
  interfaces. This patch series is perfectly suited due to it's simple changes
  to start the discussion about the device tree there.
kvm-ppc@vger.kernel.org
  The code is made for kvm on powerpc which lives on this list.

[patches in series]
Subject: [PATCH 1/6] kvmppc: read device tree hypervisor node infrastructure
  Providing the guest functionality to read hypervisor features from the
  device tree and adding the basic hook to the powerpc boot6setup code
Subject: [PATCH 2/6] kvmppc: add hypercall infrastructure - host part
Subject: [PATCH 3/6] kvmppc: add hypercall infrastructure - guest part
  patch 2&3 add the hypercall infrastruture as mentioned above
Subject: [PATCH 4/6] kvmppc: magic page hypercall - host part
Subject: [PATCH 5/6] kvmppc: magic page paravirtualization - guest part
  patch 4&5 add the magic page mechanism which will later on be used for
  binary rewriting the guest.
Subject: [PATCH 6/6] kvmppc: kvm-userspace: device tree modification for magicpage
  This connects host and guest reading host capabilities and modifying the
  device tree passed to the guest accordingly 

---
[diffstat]
 arch/powerpc/kernel/kvm.c            |   48 +++++++++++++++++++++++++++++++++++
 arch/powerpc/kvm/emulate.c           |    5 +++
 b/arch/powerpc/kernel/Makefile       |    2 +
 b/arch/powerpc/kernel/kvm.c          |   30 +++++++++++++++++++++
 b/arch/powerpc/kernel/setup_32.c     |    3 ++
 b/arch/powerpc/kvm/emulate.c         |   27 +++++++++++++++++++
 b/arch/powerpc/kvm/powerpc.c         |   18 ++++++++++++-
 b/arch/powerpc/platforms/44x/Kconfig |    7 +++++
 b/include/asm-powerpc/kvm_para.h     |   37 ++++++++++++++++++++++++--
 b/include/linux/kvm.h                |    6 ++++
 b/libkvm/libkvm-powerpc.c            |    6 ++++
 b/libkvm/libkvm.h                    |    6 ++++
 b/qemu/hw/device_tree.c              |   10 +++++++
 b/qemu/hw/device_tree.h              |    1
 b/qemu/hw/ppc440_bamboo.c            |   15 ++++++++++
 b/qemu/qemu-kvm-powerpc.c            |    5 +++
 b/qemu/qemu-kvm.h                    |    1
 include/asm-powerpc/kvm_para.h       |   47 +++++++++++++++++++++++++++++++++-
 18 files changed, 269 insertions(+), 5 deletions(-)

[PATCH 1/6] kvmppc: read device tree hypervisor node infrastructure

[ehrhardt]

From: Christian Ehrhardt <ehrhardt@linux.vnet.ibm.com>

This patch adds the guest portion of the device tree based host->guest
communication. Using the device tree infrastructure this patch implements
kvm_para_available and kvm_arch_para_features (in this patch just the
infrastructure, no specific feature registered).

Signed-off-by: Christian Ehrhardt <ehrhardt@linux.vnet.ibm.com>
---

[diffstat]
 arch/powerpc/kernel/Makefile       |    2 ++
 arch/powerpc/kernel/kvm.c          |   30 ++++++++++++++++++++++++++++++
 arch/powerpc/kernel/setup_32.c     |    3 +++
 arch/powerpc/platforms/44x/Kconfig |    7 +++++++
 include/asm-powerpc/kvm_para.h     |   37 ++++++++++++++++++++++++++++++++++---
 5 files changed, 76 insertions(+), 3 deletions(-)

[diff]

diff --git a/arch/powerpc/kernel/Makefile b/arch/powerpc/kernel/Makefile
--- a/arch/powerpc/kernel/Makefile
+++ b/arch/powerpc/kernel/Makefile
@@ -80,6 +80,8 @@
 
 obj-$(CONFIG_8XX_MINIMAL_FPEMU) += softemu8xx.o
 
+obj-$(CONFIG_KVM_GUEST)		+= kvm.o
+
 ifneq ($(CONFIG_PPC_INDIRECT_IO),y)
 obj-y				+= iomap.o
 endif
diff --git a/arch/powerpc/kernel/kvm.c b/arch/powerpc/kernel/kvm.c
new file mode 100644
--- /dev/null
+++ b/arch/powerpc/kernel/kvm.c
@@ -0,0 +1,30 @@
+/*
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License, version 2, as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301, USA.
+ *
+ * Copyright IBM Corp. 2008
+ *
+ * Authors:
+ * 	Hollis Blanchard <hollisb@us.ibm.com>
+ * 	Christian Ehrhardt <ehrhardt@de.ibm.com>
+ */
+
+#include <linux/percpu.h>
+#include <linux/mm.h>
+#include <linux/kvm_para.h>
+
+void __init kvm_guest_init(void)
+{
+	if (!kvm_para_available())
+		return;
+}
diff --git a/arch/powerpc/kernel/setup_32.c b/arch/powerpc/kernel/setup_32.c
--- a/arch/powerpc/kernel/setup_32.c
+++ b/arch/powerpc/kernel/setup_32.c
@@ -17,6 +17,7 @@
 #include <linux/cpu.h>
 #include <linux/console.h>
 #include <linux/lmb.h>
+#include <linux/kvm_para.h>
 
 #include <asm/io.h>
 #include <asm/prom.h>
@@ -319,5 +320,7 @@
 		ppc_md.setup_arch();
 	if ( ppc_md.progress ) ppc_md.progress("arch: exit", 0x3eab);
 
+	kvm_guest_init();
+
 	paging_init();
 }
diff --git a/arch/powerpc/platforms/44x/Kconfig b/arch/powerpc/platforms/44x/Kconfig
--- a/arch/powerpc/platforms/44x/Kconfig
+++ b/arch/powerpc/platforms/44x/Kconfig
@@ -152,3 +152,10 @@
 # 44x errata/workaround config symbols, selected by the CPU models above
 config IBM440EP_ERR42
 	bool
+
+config KVM_GUEST
+	bool "KVM Guest support"
+	depends on EXPERIMENTAL
+	help
+	This option enables various optimizations for running under the KVM
+	hypervisor.
diff --git a/include/asm-powerpc/kvm_para.h b/include/asm-powerpc/kvm_para.h
--- a/include/asm-powerpc/kvm_para.h
+++ b/include/asm-powerpc/kvm_para.h
@@ -14,7 +14,9 @@
  *
  * Copyright IBM Corp. 2008
  *
- * Authors: Hollis Blanchard <hollisb@us.ibm.com>
+ * Authors:
+ * 	Hollis Blanchard <hollisb@us.ibm.com>
+ * 	Christian Ehrhardt <ehrhardt@de.ibm.com>
  */
 
 #ifndef __POWERPC_KVM_PARA_H__
@@ -22,15 +24,44 @@
 
 #ifdef __KERNEL__
 
+#include <linux/of.h>
+
+static struct kvmppc_para_features {
+	char *dtcell;
+	int feature;
+} para_features[] = {
+};
+
 static inline int kvm_para_available(void)
 {
-	return 0;
+	struct device_node *dn;
+
+	dn = of_find_node_by_path("/hypervisor");
+
+	return !!dn;
 }
 
 static inline unsigned int kvm_arch_para_features(void)
 {
-	return 0;
+	struct device_node *dn;
+	const int *dtval;
+	unsigned int features = 0;
+	int i;
+
+	dn = of_find_node_by_path("/hypervisor");
+	if (!dn)
+		return 0;
+
+	for (i = 0; i < ARRAY_SIZE(para_features)-1; i++) {
+		dtval = of_get_property(dn, para_features[i].dtcell, NULL);
+		if (dtval && *dtval == 1)
+			features |= (1 << para_features[i].feature);
+	}
+
+	return features;
 }
+
+void kvm_guest_init(void);
 
 #endif /* __KERNEL__ */
 

Re: [PATCH 1/6] kvmppc: read device tree hypervisor node infrastructure

[Tony Breeds]

On Wed, Jul 23, 2008 at 10:36:42AM +0200, ehrhardt@linux.vnet.ibm.com wrote:

Hi Christian,
	A few comments inlined ...

> diff --git a/include/asm-powerpc/kvm_para.h b/include/asm-powerpc/kvm_para.h
> --- a/include/asm-powerpc/kvm_para.h
> +++ b/include/asm-powerpc/kvm_para.h
> @@ -14,7 +14,9 @@
>   *
>   * Copyright IBM Corp. 2008
>   *
> - * Authors: Hollis Blanchard <hollisb@us.ibm.com>
> + * Authors:
> + * 	Hollis Blanchard <hollisb@us.ibm.com>
> + * 	Christian Ehrhardt <ehrhardt@de.ibm.com>
>   */
>  
>  #ifndef __POWERPC_KVM_PARA_H__
> @@ -22,15 +24,44 @@
>  
>  #ifdef __KERNEL__
>  
> +#include <linux/of.h>
> +
> +static struct kvmppc_para_features {
> +	char *dtcell;
> +	int feature;
> +} para_features[] = {
> +};
> +
>  static inline int kvm_para_available(void)
>  {
> -	return 0;
> +	struct device_node *dn;
> +
> +	dn = of_find_node_by_path("/hypervisor");

You need an of_node_put(dn);

> +
> +	return !!dn;
>  }
>  
>  static inline unsigned int kvm_arch_para_features(void)
>  {
> -	return 0;
> +	struct device_node *dn;
> +	const int *dtval;
> +	unsigned int features = 0;
> +	int i;
> +
> +	dn = of_find_node_by_path("/hypervisor");
> +	if (!dn)
> +		return 0;
> +
> +	for (i = 0; i < ARRAY_SIZE(para_features)-1; i++) {

Why -1?  Isn't ARRAY_SIZE(para_features) adequate?

> +		dtval = of_get_property(dn, para_features[i].dtcell, NULL);
> +		if (dtval && *dtval == 1)
> +			features |= (1 << para_features[i].feature);
> +	}
> +

You need an of_node_put(dn);

> +	return features;
>  }

Yours Tony

  linux.conf.au    http://www.marchsouth.org/
  Jan 19 - 24 2009 The Australian Linux Technical Conference!

Re: [PATCH 1/6] kvmppc: read device tree hypervisor node infrastructure

[Christian Ehrhardt]

Tony Breeds wrote:
> On Wed, Jul 23, 2008 at 10:36:42AM +0200, ehrhardt@linux.vnet.ibm.com wrote:
>
> Hi Christian,
> 	A few comments inlined ...
>
>   
[...]
>> +
>>  static inline int kvm_para_available(void)
>>  {
>> -	return 0;
>> +	struct device_node *dn;
>> +
>> +	dn = of_find_node_by_path("/hypervisor");
>>     
>
> You need an of_node_put(dn);
>
>   
I just looked at the linux/of.h and did not see that I have to free it 
again.
Thanks for the hint, I inserted both calls.
>> +
>> +	return !!dn;
>>  }
>>  
>>  static inline unsigned int kvm_arch_para_features(void)
>>  {
>> -	return 0;
>> +	struct device_node *dn;
>> +	const int *dtval;
>> +	unsigned int features = 0;
>> +	int i;
>> +
>> +	dn = of_find_node_by_path("/hypervisor");
>> +	if (!dn)
>> +		return 0;
>> +
>> +	for (i = 0; i < ARRAY_SIZE(para_features)-1; i++) {
>>     
>
> Why -1?  Isn't ARRAY_SIZE(para_features) adequate?
>   

yeah I already had this, bit the change was folded into the wrong patch, 
fixed now

[...]
> Yours Tony
>
>   linux.conf.au    http://www.marchsouth.org/
>   Jan 19 - 24 2009 The Australian Linux Technical Conference!
>
>   


-- 

Grüsse / regards, 
Christian Ehrhardt
IBM Linux Technology Center, Open Virtualization

[PATCH 2/6] kvmppc: add hypercall infrastructure - host part

[ehrhardt]

From: Christian Ehrhardt <ehrhardt@linux.vnet.ibm.com>

This adds the host portion of the hypercall infrastructure which receives
the guest calls - no specific hcall function is implemented in this patch.

Signed-off-by: Christian Ehrhardt <ehrhardt@linux.vnet.ibm.com>
---

[diffstat]
 arch/powerpc/kvm/emulate.c     |   27 +++++++++++++++++++++++++++
 include/asm-powerpc/kvm_para.h |    2 ++
 2 files changed, 29 insertions(+)

[diff]
diff --git a/arch/powerpc/kvm/emulate.c b/arch/powerpc/kvm/emulate.c
--- a/arch/powerpc/kvm/emulate.c
+++ b/arch/powerpc/kvm/emulate.c
@@ -203,6 +203,24 @@
 	kvmppc_set_msr(vcpu, vcpu->arch.srr1);
 }
 
+static int kvmppc_do_hypercall(struct kvm_vcpu *vcpu)
+{
+	int ret = 0;
+
+	switch (vcpu->arch.gpr[0]) {
+	default:
+		printk(KERN_ERR"unknown hypercall %d\n", vcpu->arch.gpr[0]);
+		kvmppc_dump_vcpu(vcpu);
+		ret = -ENOSYS;
+	}
+
+	vcpu->arch.gpr[11] = ret;
+	vcpu->arch.pc += 4; /* Advance past hypercall instruction. */
+
+	return ret;
+}
+
+
 /* XXX to do:
  * lhax
  * lhaux
@@ -232,6 +250,15 @@
 	int advance = 1;
 
 	switch (get_op(inst)) {
+	case 0:
+		if (inst == KVM_HYPERCALL_BIN) {
+			kvmppc_do_hypercall(vcpu);
+			advance = 0; /* kvmppc_do_hypercall handles the PC. */
+		} else {
+			printk(KERN_ERR"unknown op %d\n", get_op(inst));
+			emulated = EMULATE_FAIL;
+		}
+		break;
 	case 3:                                                 /* trap */
 		printk("trap!\n");
 		kvmppc_queue_exception(vcpu, BOOKE_INTERRUPT_PROGRAM);
diff --git a/include/asm-powerpc/kvm_para.h b/include/asm-powerpc/kvm_para.h
--- a/include/asm-powerpc/kvm_para.h
+++ b/include/asm-powerpc/kvm_para.h
@@ -22,6 +22,8 @@
 
 #ifdef __KERNEL__
 
+#define KVM_HYPERCALL_BIN 0x03ffffff
+
 static inline int kvm_para_available(void)
 {
 	return 0;

Re: [PATCH 2/6] kvmppc: add hypercall infrastructure - host part

[Tony Breeds]

On Wed, Jul 23, 2008 at 10:36:43AM +0200, ehrhardt@linux.vnet.ibm.com wrote:
> From: Christian Ehrhardt <ehrhardt@linux.vnet.ibm.com>

Hi Christian,
	A few comments 

> diff --git a/arch/powerpc/kvm/emulate.c b/arch/powerpc/kvm/emulate.c
> --- a/arch/powerpc/kvm/emulate.c
> +++ b/arch/powerpc/kvm/emulate.c
> @@ -203,6 +203,24 @@
>  	kvmppc_set_msr(vcpu, vcpu->arch.srr1);
>  }
>  
> +static int kvmppc_do_hypercall(struct kvm_vcpu *vcpu)
> +{
> +	int ret = 0;
> +
> +	switch (vcpu->arch.gpr[0]) {
> +	default:
> +		printk(KERN_ERR"unknown hypercall %d\n", vcpu->arch.gpr[0]);

I think the preffered style is printk(KERN_ERR "...)  You've made the
same style mistake in most of you printk()'s in your other patches
aswell.

Yours Tony

  linux.conf.au    http://www.marchsouth.org/
  Jan 19 - 24 2009 The Australian Linux Technical Conference!

Re: [PATCH 2/6] kvmppc: add hypercall infrastructure - host part

[Geert Uytterhoeven]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 1229 bytes --]

On Thu, 24 Jul 2008, Tony Breeds wrote:
> On Wed, Jul 23, 2008 at 10:36:43AM +0200, ehrhardt@linux.vnet.ibm.com wrote:
> > From: Christian Ehrhardt <ehrhardt@linux.vnet.ibm.com>
> > diff --git a/arch/powerpc/kvm/emulate.c b/arch/powerpc/kvm/emulate.c
> > --- a/arch/powerpc/kvm/emulate.c
> > +++ b/arch/powerpc/kvm/emulate.c
> > @@ -203,6 +203,24 @@
> >  	kvmppc_set_msr(vcpu, vcpu->arch.srr1);
> >  }
> >  
> > +static int kvmppc_do_hypercall(struct kvm_vcpu *vcpu)
> > +{
> > +	int ret = 0;
> > +
> > +	switch (vcpu->arch.gpr[0]) {
> > +	default:
> > +		printk(KERN_ERR"unknown hypercall %d\n", vcpu->arch.gpr[0]);
> 
> I think the preffered style is printk(KERN_ERR "...)  You've made the
> same style mistake in most of you printk()'s in your other patches
> aswell.

Note that these days people use pr_err() instead.

With kind regards,

Geert Uytterhoeven
Software Architect

Sony Techsoft Centre Europe
The Corporate Village · Da Vincilaan 7-D1 · B-1935 Zaventem · Belgium

Phone:    +32 (0)2 700 8453
Fax:      +32 (0)2 700 8622
E-mail:   Geert.Uytterhoeven@eu.sony.com
Internet: http://www.sony-europe.com/

A division of Sony Europe (Belgium) N.V.
VAT BE 0413.825.160 · RPR Brussels
Fortis 293-0376800-10 GEBA-BE-BB

[PATCH 3/6] kvmppc: add hypercall infrastructure - guest part

[ehrhardt]

From: Christian Ehrhardt <ehrhardt@linux.vnet.ibm.com>

This adds the guest portion of the hypercall infrastructure, basically an
illegal instruction with a defined layout.
See http://kvm.qumranet.com/kvmwiki/PowerPC_Hypercall_ABI for more detail
on the hypercall ABI for powerpc.

Signed-off-by: Christian Ehrhardt <ehrhardt@linux.vnet.ibm.com>
---

[diffstat]
 kvm_para.h |   16 ++++++++++++++++
 1 file changed, 16 insertions(+)

[diff]
diff --git a/include/asm-powerpc/kvm_para.h b/include/asm-powerpc/kvm_para.h
--- a/include/asm-powerpc/kvm_para.h
+++ b/include/asm-powerpc/kvm_para.h
@@ -25,6 +25,8 @@
 #ifdef __KERNEL__
 
 #include <linux/of.h>
+
+#define KVM_HYPERCALL_BIN 0x03ffffff
 
 static struct kvmppc_para_features {
 	char *dtcell;
@@ -63,6 +65,20 @@
 
 void kvm_guest_init(void);
 
+static inline long kvm_hypercall1(unsigned int nr, unsigned long p1)
+{
+	register unsigned long hcall asm ("r0") = nr;
+	register unsigned long arg1 asm ("r3") = p1;
+	register long ret asm ("r11");
+
+	asm volatile(".long %1"
+			: "=r"(ret)
+			: "i"(KVM_HYPERCALL_BIN), "r"(hcall), "r"(arg1)
+			: "r4", "r5", "r6", "r7", "r8",
+			  "r9", "r10", "r12", "cc");
+	return ret;
+}
+
 #endif /* __KERNEL__ */
 
 #endif /* __POWERPC_KVM_PARA_H__ */

Re: [PATCH 3/6] kvmppc: add hypercall infrastructure - guest part

[Tony Breeds]

On Wed, Jul 23, 2008 at 10:36:44AM +0200, ehrhardt@linux.vnet.ibm.com wrote:
> From: Christian Ehrhardt <ehrhardt@linux.vnet.ibm.com>

Hi Christian,
> 
> This adds the guest portion of the hypercall infrastructure, basically an
> illegal instruction with a defined layout.
> See http://kvm.qumranet.com/kvmwiki/PowerPC_Hypercall_ABI for more detail
> on the hypercall ABI for powerpc.
> 
> Signed-off-by: Christian Ehrhardt <ehrhardt@linux.vnet.ibm.com>
> ---
> 
> [diffstat]
>  kvm_para.h |   16 ++++++++++++++++
>  1 file changed, 16 insertions(+)
> 
> [diff]
> diff --git a/include/asm-powerpc/kvm_para.h b/include/asm-powerpc/kvm_para.h
> --- a/include/asm-powerpc/kvm_para.h
> +++ b/include/asm-powerpc/kvm_para.h
> @@ -25,6 +25,8 @@
>  #ifdef __KERNEL__
>  
>  #include <linux/of.h>
> +
> +#define KVM_HYPERCALL_BIN 0x03ffffff

Ummm didn't you add this in patch 2 of 6?

Yours Tony

  linux.conf.au    http://www.marchsouth.org/
  Jan 19 - 24 2009 The Australian Linux Technical Conference!

Re: [PATCH 3/6] kvmppc: add hypercall infrastructure - guest part

[Christian Ehrhardt]

Tony Breeds wrote:
> On Wed, Jul 23, 2008 at 10:36:44AM +0200, ehrhardt@linux.vnet.ibm.com wrote:
>   
>> From: Christian Ehrhardt <ehrhardt@linux.vnet.ibm.com>
>>     
>
> Hi Christian,
>   
>> This adds the guest portion of the hypercall infrastructure, basically an
>> illegal instruction with a defined layout.
>> See http://kvm.qumranet.com/kvmwiki/PowerPC_Hypercall_ABI for more detail
>> on the hypercall ABI for powerpc.
>>
>> Signed-off-by: Christian Ehrhardt <ehrhardt@linux.vnet.ibm.com>
>> ---
>>
>> [diffstat]
>>  kvm_para.h |   16 ++++++++++++++++
>>  1 file changed, 16 insertions(+)
>>
>> [diff]
>> diff --git a/include/asm-powerpc/kvm_para.h b/include/asm-powerpc/kvm_para.h
>> --- a/include/asm-powerpc/kvm_para.h
>> +++ b/include/asm-powerpc/kvm_para.h
>> @@ -25,6 +25,8 @@
>>  #ifdef __KERNEL__
>>  
>>  #include <linux/of.h>
>> +
>> +#define KVM_HYPERCALL_BIN 0x03ffffff
>>     
>
> Ummm didn't you add this in patch 2 of 6?
>   
This is just because I initially wanted to split Host & Guest patch series.
I need to separate my patches a bit more anyway for the next submission 
thanks for pointing out this duplication.

> Yours Tony
>
>   linux.conf.au    http://www.marchsouth.org/
>   Jan 19 - 24 2009 The Australian Linux Technical Conference!
>
>   


-- 

Grüsse / regards, 
Christian Ehrhardt
IBM Linux Technology Center, Open Virtualization

[PATCH 4/6] kvmppc: magic page hypercall - host part

[ehrhardt]

From: Christian Ehrhardt <ehrhardt@linux.vnet.ibm.com>

This adds the host part of the magic page registration. This is a memory
area of the guest granted to the host.
The patch just introduces the infrastruture to receive the guest paddr.
This is work in progress and it is intended to later on use this memory
as storage area a guest can read unprivileged (using binary rewriting to
change privileges instructions).

Signed-off-by: Christian Ehrhardt <ehrhardt@linux.vnet.ibm.com>
---

[diffstat]
 arch/powerpc/kvm/emulate.c     |    5 +++++
 arch/powerpc/kvm/powerpc.c     |   18 +++++++++++++++++-
 include/asm-powerpc/kvm_para.h |    2 ++
 include/linux/kvm.h            |    6 ++++++
 4 files changed, 30 insertions(+), 1 deletion(-)

[diff]
diff --git a/arch/powerpc/kvm/emulate.c b/arch/powerpc/kvm/emulate.c
--- a/arch/powerpc/kvm/emulate.c
+++ b/arch/powerpc/kvm/emulate.c
@@ -208,6 +208,11 @@
 	int ret = 0;
 
 	switch (vcpu->arch.gpr[0]) {
+	case KVM_HCALL_RESERVE_MAGICPAGE:
+		/* FIXME TODO implement the real fuctionality using that */
+		printk(KERN_ERR"%s - receive magicpage address 0x%x\n",
+			__func__, vcpu->arch.gpr[3]);
+		break;
 	default:
 		printk(KERN_ERR"unknown hypercall %d\n", vcpu->arch.gpr[0]);
 		kvmppc_dump_vcpu(vcpu);
diff --git a/arch/powerpc/kvm/powerpc.c b/arch/powerpc/kvm/powerpc.c
--- a/arch/powerpc/kvm/powerpc.c
+++ b/arch/powerpc/kvm/powerpc.c
@@ -148,6 +148,9 @@
 	case KVM_CAP_COALESCED_MMIO:
 		r = KVM_COALESCED_MMIO_PAGE_OFFSET;
 		break;
+	case KVM_CAP_PPCPV_MAGICPAGE:
+		r = 1;
+		break;
 	default:
 		r = 0;
 		break;
@@ -159,7 +162,20 @@
 long kvm_arch_dev_ioctl(struct file *filp,
                         unsigned int ioctl, unsigned long arg)
 {
-	return -EINVAL;
+	long r = -EINVAL;
+
+	switch (ioctl) {
+	case KVM_GET_PPCPV_MAGICPAGE_SIZE:
+		r = -EINVAL;
+		if (arg)
+			goto out;
+		r = 1024;
+		break;
+	default:
+		r = -EINVAL;
+	}
+out:
+	return r;
 }
 
 int kvm_arch_set_memory_region(struct kvm *kvm,
diff --git a/include/asm-powerpc/kvm_para.h b/include/asm-powerpc/kvm_para.h
--- a/include/asm-powerpc/kvm_para.h
+++ b/include/asm-powerpc/kvm_para.h
@@ -24,6 +24,8 @@
 
 #define KVM_HYPERCALL_BIN 0x03ffffff
 
+#define KVM_HCALL_RESERVE_MAGICPAGE	0
+
 static inline int kvm_para_available(void)
 {
 	return 0;
diff --git a/include/linux/kvm.h b/include/linux/kvm.h
--- a/include/linux/kvm.h
+++ b/include/linux/kvm.h
@@ -365,6 +365,11 @@
 #define KVM_TRACE_PAUSE           _IO(KVMIO,  0x07)
 #define KVM_TRACE_DISABLE         _IO(KVMIO,  0x08)
 /*
+ * ioctls for powerpc paravirtualization extensions
+ */
+#define KVM_GET_PPCPV_MAGICPAGE_SIZE       _IO(KVMIO,   0x09)
+
+/*
  * Extension capability list.
  */
 #define KVM_CAP_IRQCHIP	  0
@@ -382,6 +387,7 @@
 #define KVM_CAP_PV_MMU 13
 #define KVM_CAP_MP_STATE 14
 #define KVM_CAP_COALESCED_MMIO 15
+#define KVM_CAP_PPCPV_MAGICPAGE 16
 
 /*
  * ioctls for VM fds

Re: [PATCH 4/6] kvmppc: magic page hypercall - host part

[Tony Breeds]

On Wed, Jul 23, 2008 at 10:36:45AM +0200, ehrhardt@linux.vnet.ibm.com wrote:

Hi Christian,

>  long kvm_arch_dev_ioctl(struct file *filp,
>                          unsigned int ioctl, unsigned long arg)
>  {
> -	return -EINVAL;
> +	long r = -EINVAL;
> +
> +	switch (ioctl) {
> +	case KVM_GET_PPCPV_MAGICPAGE_SIZE:
> +		r = -EINVAL;

Not needed you set it on the declaration.

> +		if (arg)
> +			goto out;
> +		r = 1024;

Ummm what does 1024 represent? can it me #defined? or at least add a
comment.

> +		break;
> +	default:
> +		r = -EINVAL;

Not needed you set it on the declaration.

> +	}
> +out:
> +	return r;
>  }

Yours Tony

  linux.conf.au    http://www.marchsouth.org/
  Jan 19 - 24 2009 The Australian Linux Technical Conference!

[PATCH 5/6] kvmppc: magic page paravirtualization - guest part

[ehrhardt]

From: Christian Ehrhardt <ehrhardt@linux.vnet.ibm.com>

This patch adds the guest handling for the magic page mechanism. A Hypervisor
can modify the device tree passed to the guest. Using that already existing
interface a guest can simply detect available hypervisor features and agree
on the supported ones using hypercalls.
In this example it is checked for the feature switch "feature,pv-magicpage"
in the hypervisor node and additional data which represents the size the
hypervisor requests in "data,pv-magicpage-size".
When the guest read that data and wants to support it the memory is allocated
and passed to the hypervisor using the KVM_HCALL_RESERVE_MAGICPAGE hypercall.

Signed-off-by: Christian Ehrhardt <ehrhardt@linux.vnet.ibm.com>
---

[diffstat]
 arch/powerpc/kernel/kvm.c      |   48 +++++++++++++++++++++++++++++++++++++++++
 include/asm-powerpc/kvm_para.h |   27 ++++++++++++++++++++++-
 2 files changed, 74 insertions(+), 1 deletion(-)

[diff]
diff --git a/arch/powerpc/kernel/kvm.c b/arch/powerpc/kernel/kvm.c
--- a/arch/powerpc/kernel/kvm.c
+++ b/arch/powerpc/kernel/kvm.c
@@ -22,9 +22,57 @@
 #include <linux/percpu.h>
 #include <linux/mm.h>
 #include <linux/kvm_para.h>
+#include <linux/bootmem.h>
+
+/*
+ * this is guest memory granted to the hypervisor;
+ * the hypervisor can place data in this area and rewrite
+ * privileged instructions to read from this area without
+ * trapping.
+ * Only the Hypervisor needs to be aware of the structure layout
+ * which makes the guest more felxible - the guest only guarantees
+ * the size which is requested by the hypervisor and read from a
+ * device tree entry.
+ */
+void *kvm_magicpage;
+
+static void __init kvmppc_register_magic_page(void)
+{
+	unsigned long paddr;
+	int size;
+	long err;
+
+	size = kvmppc_pv_read_data(KVM_PVDATA_MAGICPAGE_SIZE);
+	if (size < 0) {
+		printk(KERN_ERR"%s: couldn't read size for kvmppc style "
+			"paravirtualization support (got %d)\n",
+			__func__, size);
+		return;
+	}
+
+	/* FIXME Guest SMP needs that percpu
+	 * On SMP we might also need a free implementation */
+	kvm_magicpage = alloc_bootmem(size);
+	if (!kvm_magicpage) {
+		printk(KERN_ERR"%s - failed to allocate %d bytes\n",
+			 __func__, size);
+		return;
+	}
+
+	paddr = (unsigned long)__pa(kvm_magicpage);
+	err = kvm_hypercall1(KVM_HCALL_RESERVE_MAGICPAGE, paddr);
+	if (err)
+		printk(KERN_ERR"%s: couldn't register magic page\n", __func__);
+	else
+		printk(KERN_NOTICE"%s: registered %d bytes for "
+			"virtualization support\n", __func__, size);
+}
 
 void __init kvm_guest_init(void)
 {
 	if (!kvm_para_available())
 		return;
+
+	if (kvm_para_has_feature(KVM_FEATURE_PPCPV_MAGICPAGE))
+		kvmppc_register_magic_page();
 }
diff --git a/include/asm-powerpc/kvm_para.h b/include/asm-powerpc/kvm_para.h
--- a/include/asm-powerpc/kvm_para.h
+++ b/include/asm-powerpc/kvm_para.h
@@ -28,10 +28,18 @@
 
 #define KVM_HYPERCALL_BIN 0x03ffffff
 
+#define KVM_HCALL_RESERVE_MAGICPAGE	0
+
+#define KVM_PVDATA_MAGICPAGE_SIZE	"data,pv-magicpage-size"
+
+/* List of PV features supported, returned as a bitfield */
+#define KVM_FEATURE_PPCPV_MAGICPAGE	0
+
 static struct kvmppc_para_features {
 	char *dtcell;
 	int feature;
 } para_features[] = {
+	{ "feature,pv-magicpage", KVM_FEATURE_PPCPV_MAGICPAGE }
 };
 
 static inline int kvm_para_available(void)
@@ -54,13 +62,30 @@
 	if (!dn)
 		return 0;
 
-	for (i = 0; i < ARRAY_SIZE(para_features)-1; i++) {
+	for (i = 0; i < ARRAY_SIZE(para_features); i++) {
 		dtval = of_get_property(dn, para_features[i].dtcell, NULL);
 		if (dtval && *dtval == 1)
 			features |= (1 << para_features[i].feature);
 	}
 
 	return features;
+}
+
+/* reads the specified data field out of the hypervisor node */
+static inline int kvmppc_pv_read_data(char *dtcell)
+{
+	struct device_node *dn;
+	const int *dtval;
+
+	dn = of_find_node_by_path("/hypervisor");
+	if (!dn)
+		return -EINVAL;
+
+	dtval = of_get_property(dn, dtcell, NULL);
+	if (dtval)
+		return *dtval;
+	else
+		return -EINVAL;
 }
 
 void kvm_guest_init(void);

Re: [PATCH 5/6] kvmppc: magic page paravirtualization - guest part

[Tony Breeds]

On Wed, Jul 23, 2008 at 10:36:46AM +0200, ehrhardt@linux.vnet.ibm.com wrote:

Hi Christian,

<snip>

> +/*
> + * this is guest memory granted to the hypervisor;
> + * the hypervisor can place data in this area and rewrite
> + * privileged instructions to read from this area without
> + * trapping.
> + * Only the Hypervisor needs to be aware of the structure layout
> + * which makes the guest more felxible - the guest only guarantees
> + * the size which is requested by the hypervisor and read from a
> + * device tree entry.
> + */
> +void *kvm_magicpage;

static?

<snip>

> +/* reads the specified data field out of the hypervisor node */
> +static inline int kvmppc_pv_read_data(char *dtcell)
> +{
> +	struct device_node *dn;
> +	const int *dtval;
> +
> +	dn = of_find_node_by_path("/hypervisor");
> +	if (!dn)
> +		return -EINVAL;
> +
> +	dtval = of_get_property(dn, dtcell, NULL);
> +	if (dtval)
> +		return *dtval;
> +	else
> +		return -EINVAL;

You need an of_node_put(dn) in this function somewhere.

Yours Tony

  linux.conf.au    http://www.marchsouth.org/
  Jan 19 - 24 2009 The Australian Linux Technical Conference!

[PATCH 6/6] kvmppc: kvm-userspace: device tree modification for magicpage

[ehrhardt]

From: Christian Ehrhardt <ehrhardt@linux.vnet.ibm.com>

This patch to kvm-userspace connects the other host & guest patches in this
series. On guest initialization it checks the hosts capabilities for the
magicpage mechanism. If available the device tree passed to the guest gets the
"hypervisor" node added and in that node the feature flag and the requested
magic page size (read from the host kernel via an ioctl) is stored.

Signed-off-by: Christian Ehrhardt <ehrhardt@linux.vnet.ibm.com>
---

[diffstat]
 libkvm/libkvm-powerpc.c |    6 ++++++
 libkvm/libkvm.h         |    6 ++++++
 qemu/hw/device_tree.c   |   10 ++++++++++
 qemu/hw/device_tree.h   |    1 +
 qemu/hw/ppc440_bamboo.c |   15 +++++++++++++++
 qemu/qemu-kvm-powerpc.c |    5 +++++
 qemu/qemu-kvm.h         |    1 +
 7 files changed, 44 insertions(+)

[diff]
diff --git a/libkvm/libkvm-powerpc.c b/libkvm/libkvm-powerpc.c
--- a/libkvm/libkvm-powerpc.c
+++ b/libkvm/libkvm-powerpc.c
@@ -19,6 +19,7 @@
 
 #include "libkvm.h"
 #include "kvm-powerpc.h"
+#include <sys/ioctl.h>
 #include <errno.h>
 #include <stdio.h>
 #include <inttypes.h>
@@ -105,6 +106,11 @@
 	return 0;
 }
 
+int kvm_get_magicpage_size(kvm_context_t kvm)
+{
+	return ioctl(kvm->fd, KVM_GET_PPCPV_MAGICPAGE_SIZE, 0);
+}
+
 int kvm_arch_run(struct kvm_run *run, kvm_context_t kvm, int vcpu)
 {
 	int ret = 0;
diff --git a/libkvm/libkvm.h b/libkvm/libkvm.h
--- a/libkvm/libkvm.h
+++ b/libkvm/libkvm.h
@@ -639,6 +639,12 @@
 
 #endif
 
+#ifdef KVM_CAP_PPCPV_MAGICPAGE
+
+int kvm_get_magicpage_size(kvm_context_t kvm);
+
+#endif
+
 int kvm_translate(kvm_context_t kvm, int vcpu, struct kvm_translation *tr);
 
 #endif
diff --git a/qemu/hw/device_tree.c b/qemu/hw/device_tree.c
--- a/qemu/hw/device_tree.c
+++ b/qemu/hw/device_tree.c
@@ -190,4 +190,14 @@
 		exit(1);
 	}
 }
+
+void dt_add_subnode(void *fdt, const char *name, char *node_path)
+{
+	int offset;
+	offset = get_offset_of_node(fdt, node_path);
+	if (fdt_add_subnode(fdt, offset, name) < 0) {
+		printf("Unable to create device tree node '%s'\n", name);
+		exit(1);
+	}
+}
 #endif
diff --git a/qemu/hw/device_tree.h b/qemu/hw/device_tree.h
--- a/qemu/hw/device_tree.h
+++ b/qemu/hw/device_tree.h
@@ -23,4 +23,5 @@
 		uint32_t *val_array, int size);
 void dt_string(void *fdt, char *node_path, char *property,
 		char *string);
+void dt_add_subnode(void *fdt, const char *name, char *node_path);
 #endif
diff --git a/qemu/hw/ppc440_bamboo.c b/qemu/hw/ppc440_bamboo.c
--- a/qemu/hw/ppc440_bamboo.c
+++ b/qemu/hw/ppc440_bamboo.c
@@ -51,6 +51,7 @@
 	uint32_t cpu_freq;
 	uint32_t timebase_freq;
 	uint32_t mem_reg_property[]={0, 0, ram_size};
+	int pv_magicpage_size;
 
 	printf("%s: START\n", __func__);
 
@@ -167,6 +168,20 @@
 	dt_cell(fdt, "/chosen", "linux,initrd-end",
 				(initrd_base + initrd_size));
 	dt_string(fdt, "/chosen", "bootargs", (char *)kernel_cmdline);
+
+	if (kvm_enabled()
+	    && kvm_qemu_check_extension(KVM_CAP_PPCPV_MAGICPAGE)) {
+		pv_magicpage_size = kvmppc_pv_get_magicpage_size();
+		if (pv_magicpage_size < 0) {
+			fprintf(stderr, "%s: error reading magic page size\n",
+				 __func__);
+			exit(1);
+		}
+		dt_add_subnode(fdt, "hypervisor", "/");
+		dt_cell(fdt, "/hypervisor", "feature,pv-magicpage", 1);
+		dt_cell(fdt, "/hypervisor", "data,pv-magicpage-size",
+			pv_magicpage_size);
+	}
 #endif
 
 	if (kvm_enabled()) {
diff --git a/qemu/qemu-kvm-powerpc.c b/qemu/qemu-kvm-powerpc.c
--- a/qemu/qemu-kvm-powerpc.c
+++ b/qemu/qemu-kvm-powerpc.c
@@ -214,6 +214,11 @@
     return 0; /* XXX ignore failed DCR ops */
 }
 
+int kvmppc_pv_get_magicpage_size(void)
+{
+	return kvm_get_magicpage_size(kvm_context);
+}
+
 int mmukvm_get_physical_address(CPUState *env, mmu_ctx_t *ctx,
                                 target_ulong eaddr, int rw, int access_type)
 {
diff --git a/qemu/qemu-kvm.h b/qemu/qemu-kvm.h
--- a/qemu/qemu-kvm.h
+++ b/qemu/qemu-kvm.h
@@ -86,6 +86,7 @@
 #ifdef TARGET_PPC
 int handle_powerpc_dcr_read(int vcpu, uint32_t dcrn, uint32_t *data);
 int handle_powerpc_dcr_write(int vcpu,uint32_t dcrn, uint32_t data);
+int kvmppc_pv_get_magicpage_size();
 #endif
 
 #if !defined(SYS_signalfd)

Re: [PATCH 0/6][RFC] kvmppc: paravirtualization interface

[Tony Breeds]

On Wed, Jul 23, 2008 at 10:36:41AM +0200, ehrhardt@linux.vnet.ibm.com wrote:
> From: Christian Ehrhardt <ehrhardt@linux.vnet.ibm.com>
> 
> This patch series implements a paravirtualization interface using:
> - the device tree mechanism to pass hypervisor informations to the guest
> - hypercalls for guest->host calls
> - an example exploiter of that interface (magic page)
> This is work in progress, but working so far. I just start to really exploit
> the fuctionality behind the magic page mechanism therefor I can't provide any
> performance improvements so far, but it is evolved enough for RFC and to start
> the standardization discussion.

Are you aiming this for the current merge window, ie for 2.6.27?

Yours Tony

  linux.conf.au    http://www.marchsouth.org/
  Jan 19 - 24 2009 The Australian Linux Technical Conference!

Re: [PATCH 0/6][RFC] kvmppc: paravirtualization interface

[Christian Ehrhardt]

Tony Breeds wrote:
> On Wed, Jul 23, 2008 at 10:36:41AM +0200, ehrhardt@linux.vnet.ibm.com wrote:
>   
>> From: Christian Ehrhardt <ehrhardt@linux.vnet.ibm.com>
>>
>> This patch series implements a paravirtualization interface using:
>> - the device tree mechanism to pass hypervisor informations to the guest
>> - hypercalls for guest->host calls
>> - an example exploiter of that interface (magic page)
>> This is work in progress, but working so far. I just start to really exploit
>> the fuctionality behind the magic page mechanism therefor I can't provide any
>> performance improvements so far, but it is evolved enough for RFC and to start
>> the standardization discussion.
>>     
>
> Are you aiming this for the current merge window, ie for 2.6.27?
>   
The aim is not really fixed. It would be nice to get into 2.6.27, but 
since I can't yet expect how long it takes ...

Actually the guest patches would already go through reviews and 
upstream, due to the fact that the guest code changes are not that (the 
major part of the implementation will go over kvmppc -> kvm upstream).
But since I want to discuss about the standardization on the embedded 
hypervisor list first, the naming of the device tree entries are not 
fixed yet.
Therefor I can't yet define which kernel version merge window I'll 
target/reach.

btw - embedded hypervisor - I got advised that this is a closed list 
which I forgot.
Sorry for all who got bounces on a replay-all action. The next version 
of the patch series will go to the involved open source lists only and a 
separate more standardization than patch style mail series to embedded 
hypervisor.

> Yours Tony
>
>   linux.conf.au    http://www.marchsouth.org/
>   Jan 19 - 24 2009 The Australian Linux Technical Conference!
>   
-- 

Grüsse / regards, 
Christian Ehrhardt
IBM Linux Technology Center, Open Virtualization

Re: [PATCH 0/6][RFC] kvmppc: paravirtualization interface

[Tony Breeds]

On Thu, Jul 24, 2008 at 10:17:31AM +0200, Christian Ehrhardt wrote:

> The aim is not really fixed. It would be nice to get into 2.6.27, but 
> since I can't yet expect how long it takes ...

Ahh okay, Id say given that we're 2/3rds through the merge window then
2.8.28 is a safer target.
 
> Actually the guest patches would already go through reviews and 
> upstream, due to the fact that the guest code changes are not that (the 
> major part of the implementation will go over kvmppc -> kvm upstream).
> But since I want to discuss about the standardization on the embedded 
> hypervisor list first, the naming of the device tree entries are not 
> fixed yet.

Okay.  When that's done we can take another look at them.

> Therefor I can't yet define which kernel version merge window I'll 
> target/reach.
> 
> btw - embedded hypervisor - I got advised that this is a closed list 
> which I forgot.

... snipped from this reply

Yours Tony

  linux.conf.au    http://www.marchsouth.org/
  Jan 19 - 24 2009 The Australian Linux Technical Conference!