[patch] dynamic nr online cpus

[patch] dynamic nr online cpus

[Jes Sorensen]

[-- Attachment #1: Type: text/plain, Size: 248 bytes --]

Hi,

Working on allowing larger max vcpus I'd like to put in this patch that
calculates the number of online cpus. It's a building block for allowing
the larger number of max vcpus without crippling performance for the
smaller cases.

Cheers,
Jes


[-- Attachment #2: 5000-kvm-ia64-nr-online-vcpu.patch --]
[-- Type: text/plain, Size: 3508 bytes --]

Account for number of online cpus and use that in loops iterating over
the list of vpus. This patch is a building block, in the work to allow
for larger max number of vcpus.

A copy of the number of online cpus is stored in vcpu 0's private data
area to allow the number to be visible from the KVM module.

Signed-off-by: Jes Sorensen <jes@sgi.com>

---
 arch/ia64/include/asm/kvm_host.h |    3 +++
 arch/ia64/kvm/kvm-ia64.c         |   20 ++++++++++++++++----
 arch/ia64/kvm/vcpu.c             |    2 +-
 3 files changed, 20 insertions(+), 5 deletions(-)

Index: linux-2.6.git/arch/ia64/include/asm/kvm_host.h
===================================================================
--- linux-2.6.git.orig/arch/ia64/include/asm/kvm_host.h
+++ linux-2.6.git/arch/ia64/include/asm/kvm_host.h
@@ -377,6 +377,7 @@
 	int last_run_cpu;
 	int vmm_tr_slot;
 	int vm_tr_slot;
+	int online_cpus;
 
 #define KVM_MP_STATE_RUNNABLE          0
 #define KVM_MP_STATE_UNINITIALIZED     1
@@ -470,6 +471,8 @@
 	unsigned long	metaphysical_rr4;
 	unsigned long	vmm_init_rr;
 
+	int		online_cpus;
+
 	struct kvm_ioapic *vioapic;
 	struct kvm_vm_stat stat;
 	struct kvm_sal_data rdv_sal_data;
Index: linux-2.6.git/arch/ia64/kvm/kvm-ia64.c
===================================================================
--- linux-2.6.git.orig/arch/ia64/kvm/kvm-ia64.c
+++ linux-2.6.git/arch/ia64/kvm/kvm-ia64.c
@@ -316,7 +316,7 @@
 	union ia64_lid lid;
 	int i;
 
-	for (i = 0; i < KVM_MAX_VCPUS; i++) {
+	for (i = 0; i < kvm->arch.online_cpus; i++) {
 		if (kvm->vcpus[i]) {
 			lid.val = VCPU_LID(kvm->vcpus[i]);
 			if (lid.id == id && lid.eid == eid)
@@ -390,7 +390,7 @@
 
 	call_data.ptc_g_data = p->u.ptc_g_data;
 
-	for (i = 0; i < KVM_MAX_VCPUS; i++) {
+	for (i = 0; i < kvm->arch.online_cpus; i++) {
 		if (!kvm->vcpus[i] || kvm->vcpus[i]->arch.mp_state ==
 						KVM_MP_STATE_UNINITIALIZED ||
 					vcpu == kvm->vcpus[i])
@@ -825,6 +825,8 @@
 		return ERR_PTR(-ENOMEM);
 	kvm_init_vm(kvm);
 
+	kvm->arch.online_cpus = 0; /* xxx hack me harder xxx */
+
 	return kvm;
 
 }
@@ -1186,7 +1188,7 @@
 
 		/*Initialize itc offset for vcpus*/
 		itc_offset = 0UL - ia64_getreg(_IA64_REG_AR_ITC);
-		for (i = 0; i < KVM_MAX_VCPUS; i++) {
+		for (i = 0; i < kvm->arch.online_cpus; i++) {
 			v = (struct kvm_vcpu *)((char *)vcpu +
 					sizeof(struct kvm_vcpu_data) * i);
 			v->arch.itc_offset = itc_offset;
@@ -1320,6 +1322,16 @@
 		goto fail;
 	}
 
+	kvm->arch.online_cpus++;
+	/*
+	 * For vcpu 0, kvm->vcpus hasn't been assigned yet, special case it
+	 */
+	if (!id)
+		vcpu->arch.online_cpus = kvm->arch.online_cpus;
+	else
+		kvm->vcpus[0]->arch.online_cpus = kvm->arch.online_cpus;
+
+	printk(KERN_INFO "arch.online_cpus %i\n", kvm->arch.online_cpus);
 	return vcpu;
 fail:
 	return ERR_PTR(r);
@@ -1766,7 +1778,7 @@
 	struct kvm_vcpu *lvcpu = kvm->vcpus[0];
 	int i;
 
-	for (i = 1; i < KVM_MAX_VCPUS; i++) {
+	for (i = 1; i < kvm->arch.online_cpus; i++) {
 		if (!kvm->vcpus[i])
 			continue;
 		if (lvcpu->arch.xtp > kvm->vcpus[i]->arch.xtp)
Index: linux-2.6.git/arch/ia64/kvm/vcpu.c
===================================================================
--- linux-2.6.git.orig/arch/ia64/kvm/vcpu.c
+++ linux-2.6.git/arch/ia64/kvm/vcpu.c
@@ -816,7 +816,7 @@
 	unsigned long vitv = VCPU(vcpu, itv);
 
 	if (vcpu->vcpu_id == 0) {
-		for (i = 0; i < KVM_MAX_VCPUS; i++) {
+		for (i = 0; i < vcpu->arch.online_cpus; i++) {
 			v = (struct kvm_vcpu *)((char *)vcpu +
 					sizeof(struct kvm_vcpu_data) * i);
 			VMX(v, itc_offset) = itc_offset;

RE: [patch] dynamic nr online cpus

[Zhang, Xiantao]

Hi, Jes
  This patch is okay to me, and still have some minor comments below, and expect them to address in your final patch.  Thanks!

 1. For readability, I prefer to use  online_vcpus intead of online_cpus 
 2. I don't think it is necessary to save a copy in vcpu's private data, just use the following code to get the kvm pointer and access its non-pointer fileds in vmm module, 
    struct kvm *get_kvm() {
         return (struct kvm*) VM_BASE; 
    }

    After getting kvm pointer, you can use it for vcpu.c's change.
Xiantao 

Jes Sorensen wrote:
> Hi,
> 
> Working on allowing larger max vcpus I'd like to put in this patch
> that calculates the number of online cpus. It's a building block for
> allowing the larger number of max vcpus without crippling performance
> for the smaller cases.
> 
> Cheers,
> Jes

Re: [patch] dynamic nr online cpus

[Jes Sorensen]

[-- Attachment #1: Type: text/plain, Size: 1352 bytes --]

Hi Xiantao,

Here's an update version of the online_vcpus patch. I agree with you
that the name online_vcpus is more appropriate and I used your trick
to get to struct kvm, so it's a lot cleaner than the old version.

I renamed VM_BASE to KVM_VM_BASE as it seems a more correct name to me
and has less chance of clashing with something. In addition the
parenthesis I added are required to avoid unexpected results from CPP
when you typecast it to struct kvm *.

Cheers,
Jes

Zhang, Xiantao wrote:
> Hi, Jes
>   This patch is okay to me, and still have some minor comments below, and expect them to address in your final patch.  Thanks!
> 
>  1. For readability, I prefer to use  online_vcpus intead of online_cpus 
>  2. I don't think it is necessary to save a copy in vcpu's private data, just use the following code to get the kvm pointer and access its non-pointer fileds in vmm module, 
>     struct kvm *get_kvm() {
>          return (struct kvm*) VM_BASE; 
>     }
> 
>     After getting kvm pointer, you can use it for vcpu.c's change.
> Xiantao 
> 
> Jes Sorensen wrote:
>> Hi,
>>
>> Working on allowing larger max vcpus I'd like to put in this patch
>> that calculates the number of online cpus. It's a building block for
>> allowing the larger number of max vcpus without crippling performance
>> for the smaller cases.
>>
>> Cheers,
>> Jes


[-- Attachment #2: 5000-kvm-ia64-nr-online-vcpu.patch --]
[-- Type: text/plain, Size: 3720 bytes --]

Account for number of online cpus and use that in loops iterating over
the list of vpus instead of scanning the full array unconditionally.
This patch is a building block to facilitate allowing to bump up
the size of MAX_VCPUS significantly.

Signed-off-by: Jes Sorensen <jes@sgi.com>

---
 arch/ia64/include/asm/kvm_host.h |   10 ++++++----
 arch/ia64/kvm/kvm-ia64.c         |   12 ++++++++----
 arch/ia64/kvm/vcpu.c             |    5 ++++-
 3 files changed, 18 insertions(+), 9 deletions(-)

Index: linux-2.6.git/arch/ia64/include/asm/kvm_host.h
===================================================================
--- linux-2.6.git.orig/arch/ia64/include/asm/kvm_host.h
+++ linux-2.6.git/arch/ia64/include/asm/kvm_host.h
@@ -163,10 +163,10 @@
 	struct kvm_vcpu_data vcpu_data[KVM_MAX_VCPUS];
 };
 
-#define VCPU_BASE(n)	KVM_VM_DATA_BASE + \
-				offsetof(struct kvm_vm_data, vcpu_data[n])
-#define VM_BASE		KVM_VM_DATA_BASE + \
-				offsetof(struct kvm_vm_data, kvm_vm_struct)
+#define VCPU_BASE(n)	(KVM_VM_DATA_BASE + \
+				offsetof(struct kvm_vm_data, vcpu_data[n]))
+#define KVM_VM_BASE	(KVM_VM_DATA_BASE + \
+				offsetof(struct kvm_vm_data, kvm_vm_struct))
 #define KVM_MEM_DIRTY_LOG_BASE	KVM_VM_DATA_BASE + \
 				offsetof(struct kvm_vm_data, kvm_mem_dirty_log)
 
@@ -479,6 +479,8 @@
 	unsigned long	metaphysical_rr4;
 	unsigned long	vmm_init_rr;
 
+	int		online_vcpus;
+
 	struct kvm_ioapic *vioapic;
 	struct kvm_vm_stat stat;
 	struct kvm_sal_data rdv_sal_data;
Index: linux-2.6.git/arch/ia64/kvm/kvm-ia64.c
===================================================================
--- linux-2.6.git.orig/arch/ia64/kvm/kvm-ia64.c
+++ linux-2.6.git/arch/ia64/kvm/kvm-ia64.c
@@ -316,7 +316,7 @@
 	union ia64_lid lid;
 	int i;
 
-	for (i = 0; i < KVM_MAX_VCPUS; i++) {
+	for (i = 0; i < kvm->arch.online_vcpus; i++) {
 		if (kvm->vcpus[i]) {
 			lid.val = VCPU_LID(kvm->vcpus[i]);
 			if (lid.id == id && lid.eid == eid)
@@ -390,7 +390,7 @@
 
 	call_data.ptc_g_data = p->u.ptc_g_data;
 
-	for (i = 0; i < KVM_MAX_VCPUS; i++) {
+	for (i = 0; i < kvm->arch.online_vcpus; i++) {
 		if (!kvm->vcpus[i] || kvm->vcpus[i]->arch.mp_state ==
 						KVM_MP_STATE_UNINITIALIZED ||
 					vcpu == kvm->vcpus[i])
@@ -825,6 +825,8 @@
 		return ERR_PTR(-ENOMEM);
 	kvm_init_vm(kvm);
 
+	kvm->arch.online_vcpus = 0;
+
 	return kvm;
 
 }
@@ -1186,7 +1188,7 @@
 
 		/*Initialize itc offset for vcpus*/
 		itc_offset = 0UL - ia64_getreg(_IA64_REG_AR_ITC);
-		for (i = 0; i < KVM_MAX_VCPUS; i++) {
+		for (i = 0; i < kvm->arch.online_vcpus; i++) {
 			v = (struct kvm_vcpu *)((char *)vcpu +
 					sizeof(struct kvm_vcpu_data) * i);
 			v->arch.itc_offset = itc_offset;
@@ -1320,6 +1322,8 @@
 		goto fail;
 	}
 
+	kvm->arch.online_vcpus++;
+
 	return vcpu;
 fail:
 	return ERR_PTR(r);
@@ -1766,7 +1770,7 @@
 	struct kvm_vcpu *lvcpu = kvm->vcpus[0];
 	int i;
 
-	for (i = 1; i < KVM_MAX_VCPUS; i++) {
+	for (i = 1; i < kvm->arch.online_vcpus; i++) {
 		if (!kvm->vcpus[i])
 			continue;
 		if (lvcpu->arch.xtp > kvm->vcpus[i]->arch.xtp)
Index: linux-2.6.git/arch/ia64/kvm/vcpu.c
===================================================================
--- linux-2.6.git.orig/arch/ia64/kvm/vcpu.c
+++ linux-2.6.git/arch/ia64/kvm/vcpu.c
@@ -811,12 +811,15 @@
 static void vcpu_set_itc(struct kvm_vcpu *vcpu, u64 val)
 {
 	struct kvm_vcpu *v;
+	struct kvm *kvm;
 	int i;
 	long itc_offset = val - ia64_getreg(_IA64_REG_AR_ITC);
 	unsigned long vitv = VCPU(vcpu, itv);
 
+	kvm = (struct kvm *)KVM_VM_BASE;
+
 	if (vcpu->vcpu_id == 0) {
-		for (i = 0; i < KVM_MAX_VCPUS; i++) {
+		for (i = 0; i < kvm->arch.online_vcpus; i++) {
 			v = (struct kvm_vcpu *)((char *)vcpu +
 					sizeof(struct kvm_vcpu_data) * i);
 			VMX(v, itc_offset) = itc_offset;

RE: [patch] dynamic nr online cpus

[Zhang, Xiantao]

The version is Okay to me. Thanks for your work!:)

Avi, 
	Please help to applied it, Thanks! 

Acked-by : Xiantao Zhang  <xiantao.zhang@intel.com>

-----Original Message-----
From: Jes Sorensen [mailto:jes@sgi.com] 
Sent: Wednesday, January 21, 2009 10:17 PM
To: Zhang, Xiantao
Cc: kvm-ia64@vger.kernel.org
Subject: Re: [patch] dynamic nr online cpus

Hi Xiantao,

Here's an update version of the online_vcpus patch. I agree with you
that the name online_vcpus is more appropriate and I used your trick
to get to struct kvm, so it's a lot cleaner than the old version.

I renamed VM_BASE to KVM_VM_BASE as it seems a more correct name to me
and has less chance of clashing with something. In addition the
parenthesis I added are required to avoid unexpected results from CPP
when you typecast it to struct kvm *.

Cheers,
Jes

Zhang, Xiantao wrote:
> Hi, Jes
>   This patch is okay to me, and still have some minor comments below, and expect them to address in your final patch.  Thanks!
> 
>  1. For readability, I prefer to use  online_vcpus intead of online_cpus 
>  2. I don't think it is necessary to save a copy in vcpu's private data, just use the following code to get the kvm pointer and access its non-pointer fileds in vmm module, 
>     struct kvm *get_kvm() {
>          return (struct kvm*) VM_BASE; 
>     }
> 
>     After getting kvm pointer, you can use it for vcpu.c's change.
> Xiantao 
> 
> Jes Sorensen wrote:
>> Hi,
>>
>> Working on allowing larger max vcpus I'd like to put in this patch
>> that calculates the number of online cpus. It's a building block for
>> allowing the larger number of max vcpus without crippling performance
>> for the smaller cases.
>>
>> Cheers,
>> Jes

Re: [patch] dynamic nr online cpus

[Avi Kivity]

Jes Sorensen wrote:
> Hi Xiantao,
>
> Here's an update version of the online_vcpus patch. I agree with you
> that the name online_vcpus is more appropriate and I used your trick
> to get to struct kvm, so it's a lot cleaner than the old version.
>
> I renamed VM_BASE to KVM_VM_BASE as it seems a more correct name to me
> and has less chance of clashing with something. In addition the
> parenthesis I added are required to avoid unexpected results from CPP
> when you typecast it to struct kvm *.
>

Applied, thanks.

-- 
error compiling committee.c: too many arguments to function