[PATCH] make x86_64 vcpu hotplug work like i386

[Ryan Grimm <grimm@us.ibm.com>]

hi,

i386 vcpu hotplug seems to work reliably but x86_64 does not and i think
i have discovered why.  in x86-64, a cpu within a domu can be removed with
vcpu-set but subsequent calls do nothing.

after xenwatch_thread grabs the event triggered by the write to the
store, it calls the registered handler and never comes back.

eventually, __cpu_die in drivers/xen/core/smpboot.c spins while waiting for the
hypervisor to report that the vcpu is down:

while (HYPERVISOR_vcpu_op(VCPUOP_is_up, cpu, NULL)) {
    current->state = TASK_UNINTERRUPTIBLE;
    schedule_timeout(HZ/10);
}

the critical difference is that play_dead in arch/i386/process-xen.c and
arch/x86_64/process-xen.c differ.  the i386 version makes a VCPUOP_down
call to the hypervisor while the x86_64 version schedules a
SCHEDOP_yield among other things.

plopping the i386 version (patch below) into x86_64/process-xen.c makes
hotplugging in x86_64 behavior like i386.  does anyone know why the
x86_64 play_dead function is in the current state?

thanks,
ryan

Signed-off-by: Ryan Grimm <grimm@us.ibm.com>
  
diff -r 974ed9f73641 linux-2.6-xen-sparse/arch/x86_64/kernel/process-xen.c
--- a/linux-2.6-xen-sparse/arch/x86_64/kernel/process-xen.c	Wed Feb  8 16:27:32 2006
+++ b/linux-2.6-xen-sparse/arch/x86_64/kernel/process-xen.c	Wed Feb  8 09:32:46 2006
@@ -53,6 +53,7 @@
 #include <asm/kdebug.h>
 #include <xen/interface/dom0_ops.h>
 #include <xen/interface/physdev.h>
+#include <xen/interface/vcpu.h>
 #include <asm/desc.h>
 #include <asm/proto.h>
 #include <asm/hardirq.h>
@@ -143,22 +144,7 @@
 /* We halt the CPU with physical CPU hotplug */
 static inline void play_dead(void)
 {
-	idle_task_exit();
-	wbinvd();
-	mb();
-	/* Ack it */
-	__get_cpu_var(cpu_state) = CPU_DEAD;
-
-	/* We shouldn't have to disable interrupts while dead, but
-	 * some interrupts just don't seem to go away, and this makes
-	 * it "work" for testing purposes. */
-	/* Death loop */
-	while (__get_cpu_var(cpu_state) != CPU_UP_PREPARE)
-		HYPERVISOR_sched_op(SCHEDOP_yield, 0);
-
-	local_irq_disable();
-	__flush_tlb_all();
-	cpu_set(smp_processor_id(), cpu_online_map);
+	HYPERVISOR_vcpu_op(VCPUOP_down, smp_processor_id(), NULL);
 	local_irq_enable();
 }
 #else