public inbox for linux-ia64@vger.kernel.org
 help / color / mirror / Atom feed
* Xen and the Art of Linux/ia64 Virtualization
@ 2005-06-29 20:16 Magenheimer, Dan (HP Labs Fort Collins)
  2005-06-29 20:22 ` Magenheimer, Dan (HP Labs Fort Collins)
                   ` (19 more replies)
  0 siblings, 20 replies; 21+ messages in thread
From: Magenheimer, Dan (HP Labs Fort Collins) @ 2005-06-29 20:16 UTC (permalink / raw)
  To: linux-ia64

[-- Attachment #1: Type: text/plain, Size: 3420 bytes --]

Most of you probably have already heard of Xen because it
was mentioned prominently (and often incorrectly) in many
of the press releases for Fedora Core 4 and Linux 2.6.12.
In short, Xen is a virtualization technology that allows
a single machine to simultaneously and efficiently run
multiple fully-isolated operating systems -- even different
distributions, different Linux kernel versions, or even
non-Linux operating systems (e.g. FreeBSD).  It can also do
other cool tricks such as live migration from one machine
to another, efficient virtual I/O to hide underlying storage
implementations, and varying how many CPUs are assigned to
the multiple running kernels.  Xen runs today on i386,
is nearing completin on x86_64, and is being ported to
ia64 and ppc.  For more information, see
the Xen web page (google it) or I can try to answer some
general questions in this thread if you prefer.

One principal concept of Xen, called "paravirtualization"
is that a few relatively minor changes to the kernel
can make it perform much more efficiently, in fact
near native performance.  There has already been some
discussion on lkml about how these changes might find their
way into Linux/x86, e.g. with a config option.

I am the principal developer and maintainer of the Xen port
to ia64, which is currently approaching its Beta release, and
currently has contributors from Intel, SGI, CERN, and Bull.
I have also been co-developing "xenlinux" for ia64, the
paravirtualized version of Linux/ia64.  Like Xen/x86, the
required linux changes are small but -- unlike Xen/x86 --
the required changes for Linux/ia64 are entirely archdep;
no common files need to be changed.  Also unlike Xen/x86,
Linux/ia64 with CONFIG_XEN enabled runs successfully on Xen,
but also run natively on hardware (called "transparent
paravirtualization").  This should be appealing to Linux/ia64
distributors as different kernels need not be shipped for
running on Xen or running native.

I would like to submit a series of CONFIG_XEN patches to
this list for review over the next few weeks.  The first
patch will be a few syntactic changes that will make
subsequent patches easier.  For example, there are a number of
places where ia64_getreg(REG_X) is used where abstracting
at a slightly higher level, e.g. with ia64_get_regX() would
make CONFIG_XEN easier.

For preliminary discussion, I have attached the current
patch for existing files for 2.6.11.  There are many other
files required (in arch/ia64/xen, include/asm-ia64/xen, and
drivers/xen) for everything to work, but I'd like to
initiate discussion and review on existing Linux/ia64 files
that would change.  Note that the patch contains many ifdef's;
with a more intrusive patch, most of the ifdef's could/should
be eliminated.  But for now, the ifdef's serve to more clearly
identify what the required changes are.

Having never submitted patches to Linux/ia64 (except a few
minor ones indirectly via David Mosberger), I thought it might
be a good idea to start by soliciting discussion.
So comments please!  After collecting and absorbing input,
I will work on a 2.6.12 patch.

For those who can't wait, a pre-beta version of Xen/ia64 and
the transparently paravirtualized 2.6.11 xenlinux/ia64 can be
found at xen-ia64.bkbits.net (and soon in a repository for
a different SCM).

Thanks!
Dan Magenheimer
HP Labs


[-- Attachment #2: xenlinux.diff --]
[-- Type: application/octet-stream, Size: 11747 bytes --]

===== include/asm-ia64/delay.h 1.10 vs 1.12 =====
--- 1.10/include/asm-ia64/delay.h	Thu Dec 16 12:57:15 2004
+++ 1.12/include/asm-ia64/delay.h	Wed Jun 29 13:46:08 2005
@@ -20,12 +20,17 @@
 #include <asm/intrinsics.h>
 #include <asm/processor.h>
 
+#ifdef CONFIG_XEN
+extern void xen_set_itm (unsigned long);
+#define ia64_set_itm(val) xen_set_itm(val)
+#else
 static __inline__ void
 ia64_set_itm (unsigned long val)
 {
 	ia64_setreg(_IA64_REG_CR_ITM, val);
 	ia64_srlz_d();
 }
+#endif
 
 static __inline__ unsigned long
 ia64_get_itm (void)
===== arch/ia64/ia32/ia32_signal.c 1.35 vs 1.36 =====
--- 1.35/arch/ia64/ia32/ia32_signal.c	Tue Jan 25 13:23:45 2005
+++ 1.36/arch/ia64/ia32/ia32_signal.c	Wed May  4 14:16:59 2005
@@ -667,7 +667,11 @@
 	/*
 	 *  `eflags' is in an ar register for this context
 	 */
+#ifdef CONFIG_XEN
+	flag = xen_get_eflag();
+#else
 	flag = ia64_getreg(_IA64_REG_AR_EFLAG);
+#endif
 	err |= __put_user((unsigned int)flag, &sc->eflags);
 	err |= __put_user(regs->r12, &sc->esp_at_signal);
 	err |= __put_user((regs->r17 >> 16) & 0xffff, (unsigned int __user *)&sc->ss);
@@ -755,10 +759,18 @@
 		 *  IA32 process's context.
 		 */
 		err |= __get_user(tmpflags, &sc->eflags);
+#ifdef CONFIG_XEN
+		flag = xen_get_eflag();
+#else
 		flag = ia64_getreg(_IA64_REG_AR_EFLAG);
+#endif
 		flag &= ~0x40DD5;
 		flag |= (tmpflags & 0x40DD5);
+#ifdef CONFIG_XEN
+		xen_set_eflag(flag);
+#else
 		ia64_setreg(_IA64_REG_AR_EFLAG, flag);
+#endif
 
 		regs->r1 = -1;	/* disable syscall checks, r1 is orig_eax */
 	}
===== arch/ia64/Makefile 1.74 vs 1.80 =====
--- 1.74/arch/ia64/Makefile	Fri Jan 28 16:32:45 2005
+++ 1.80/arch/ia64/Makefile	Tue Jun 28 16:59:42 2005
@@ -11,6 +11,8 @@
 NM := $(CROSS_COMPILE)nm -B
 READELF := $(CROSS_COMPILE)readelf
 
+NOSTDINC_FLAGS += -Iinclude/asm-xen
+
 export AWK
 
 CHECKFLAGS	+= -m64 -D__ia64=1 -D__ia64__=1 -D_LP64 -D__LP64__
@@ -57,9 +59,13 @@
 core-$(CONFIG_IA64_HP_ZX1)	+= arch/ia64/dig/
 core-$(CONFIG_IA64_HP_ZX1_SWIOTLB) += arch/ia64/dig/
 core-$(CONFIG_IA64_SGI_SN2)	+= arch/ia64/sn/
+core-$(CONFIG_XEN)		+= arch/ia64/xen/
 
 drivers-$(CONFIG_PCI)		+= arch/ia64/pci/
+ifneq ($(CONFIG_XEN),y)
 drivers-$(CONFIG_IA64_HP_SIM)	+= arch/ia64/hp/sim/
+endif
+drivers-$(CONFIG_XEN)		+= arch/ia64/hp/sim/
 drivers-$(CONFIG_IA64_HP_ZX1)	+= arch/ia64/hp/common/ arch/ia64/hp/zx1/
 drivers-$(CONFIG_IA64_HP_ZX1_SWIOTLB) += arch/ia64/hp/common/ arch/ia64/hp/zx1/
 drivers-$(CONFIG_IA64_GENERIC)	+= arch/ia64/hp/common/ arch/ia64/hp/zx1/ arch/ia64/hp/sim/ arch/ia64/sn/
===== arch/ia64/kernel/setup.c 1.86 vs 1.90 =====
--- 1.86/arch/ia64/kernel/setup.c	Wed Feb  2 13:21:27 2005
+++ 1.90/arch/ia64/kernel/setup.c	Wed Jun 29 13:46:07 2005
@@ -1,3 +1,4 @@
+
 /*
  * Architecture-specific setup.
  *
@@ -270,6 +271,16 @@
 static inline int __init
 early_console_setup (char *cmdline)
 {
+#ifdef CONFIG_XEN
+#ifndef CONFIG_IA64_HP_SIM
+	extern int running_on_xen;
+	if (running_on_xen) {
+		extern struct console hpsim_cons;
+		register_console(&hpsim_cons);
+		return 0;
+	}
+#endif
+#else
 #ifdef CONFIG_SERIAL_SGI_L1_CONSOLE
 	{
 		extern int sn_serial_console_early_setup(void);
@@ -284,6 +295,7 @@
 #ifdef CONFIG_SERIAL_8250_CONSOLE
 	if (!early_serial_console_init(cmdline))
 		return 0;
+#endif
 #endif
 
 	return -1;
===== arch/ia64/kernel/entry.S 1.80 vs edited =====
--- 1.80/arch/ia64/kernel/entry.S	Fri Jan 28 16:15:49 2005
+++ edited/arch/ia64/kernel/entry.S	Wed Jun 29 12:45:24 2005
@@ -184,6 +184,12 @@
 GLOBAL_ENTRY(ia64_switch_to)
 	.prologue
 	alloc r16=ar.pfs,1,0,0,0
+#ifdef CONFIG_XEN
+	movl r22=running_on_xen;;
+	ld4 r22=[r22];;
+	cmp.ne p7,p0=r22,r0
+(p7)	br.cond.spnt.few xen_switch_to;;
+#endif
 	DO_SAVE_SWITCH_STACK
 	.body
 
@@ -376,7 +382,11 @@
  *	- b7 holds address to return to
  *	- must not touch r8-r11
  */
+#ifdef CONFIG_XEN
+GLOBAL_ENTRY(load_switch_stack)
+#else
 ENTRY(load_switch_stack)
+#endif
 	.prologue
 	.altrp b7
 
@@ -671,6 +681,12 @@
  */
 ENTRY(ia64_leave_syscall)
 	PT_REGS_UNWIND_INFO(0)
+#ifdef CONFIG_XEN
+	movl r22=running_on_xen;;
+	ld4 r22=[r22];;
+	cmp.ne p7,p0=r22,r0
+(p7)	br.cond.spnt.few xen_leave_syscall;;
+#endif
 	/*
 	 * work.need_resched etc. mustn't get changed by this CPU before it returns to
 	 * user- or fsys-mode, hence we disable interrupts early on.
@@ -787,6 +803,12 @@
 #endif /* CONFIG_IA32_SUPPORT */
 GLOBAL_ENTRY(ia64_leave_kernel)
 	PT_REGS_UNWIND_INFO(0)
+#ifdef CONFIG_XEN
+	movl r22=running_on_xen;;
+	ld4 r22=[r22];;
+	cmp.ne p7,p0=r22,r0
+(p7)	br.cond.spnt.few xen_leave_kernel;;
+#endif
 	/*
 	 * work.need_resched etc. mustn't get changed by this CPU before it returns to
 	 * user- or fsys-mode, hence we disable interrupts early on.
@@ -1171,7 +1193,11 @@
 	 * be set up by the caller.  We declare 8 input registers so the system call
 	 * args get preserved, in case we need to restart a system call.
 	 */
+#ifdef CONFIG_XEN
+GLOBAL_ENTRY(notify_resume_user)
+#else
 ENTRY(notify_resume_user)
+#endif
 	.prologue ASM_UNW_PRLG_RP|ASM_UNW_PRLG_PFS, ASM_UNW_PRLG_GRSAVE(8)
 	alloc loc1=ar.pfs,8,2,3,0 // preserve all eight input regs in case of syscall restart!
 	mov r9=ar.unat
===== include/asm-ia64/processor.h 1.72 vs 1.79 =====
--- 1.72/include/asm-ia64/processor.h	Wed Jan 26 11:01:41 2005
+++ 1.79/include/asm-ia64/processor.h	Wed Jun 29 13:46:08 2005
@@ -551,12 +551,17 @@
 	ia64_srlz_i();
 }
 
+#ifdef CONFIG_XEN
+extern void xen_eoi (void);
+#define ia64_eoi xen_eoi
+#else
 static inline void
 ia64_eoi (void)
 {
 	ia64_setreg(_IA64_REG_CR_EOI, 0);
 	ia64_srlz_d();
 }
+#endif
 
 #define cpu_relax()	ia64_hint(ia64_hint_pause)
 
@@ -621,6 +626,10 @@
 #define current_text_addr() \
 	({ void *_pc; _pc = (void *)ia64_getreg(_IA64_REG_IP); _pc; })
 
+#ifdef CONFIG_XEN
+extern __u64 xen_get_ivr (void);
+#define ia64_get_ivr xen_get_ivr
+#else
 static inline __u64
 ia64_get_ivr (void)
 {
@@ -630,6 +639,7 @@
 	ia64_srlz_d();
 	return r;
 }
+#endif
 
 static inline void
 ia64_set_dbr (__u64 regnum, __u64 value)
@@ -658,6 +668,11 @@
 	return (w >> n) | (w << (64 - n));
 }
 
+#ifdef CONFIG_XEN
+#define ia64_get_tpr()	(ia64_getreg(_IA64_REG_CR_TPR))
+#define ia64_set_tpr(vector)	ia64_setreg(_IA64_REG_CR_TPR,vector)
+#endif
+
 #define ia64_rotl(w,n)	ia64_rotr((w), (64) - (n))
 
 /*
@@ -694,5 +709,9 @@
 extern unsigned long boot_option_idle_override;
 
 #endif /* !__ASSEMBLY__ */
+
+#ifdef CONFIG_XEN
+#include <asm/xen/processor.h>
+#endif
 
 #endif /* _ASM_IA64_PROCESSOR_H */
===== arch/ia64/kernel/irq_ia64.c 1.25 vs 1.31 =====
--- 1.25/arch/ia64/kernel/irq_ia64.c	Sat Jan 22 16:54:49 2005
+++ 1.31/arch/ia64/kernel/irq_ia64.c	Wed Jun 29 13:46:07 2005
@@ -141,11 +141,19 @@
 	 * to kernel stack overflows).
 	 */
 	irq_enter();
+#ifdef CONFIG_XEN
+	saved_tpr = ia64_get_tpr();
+#else
 	saved_tpr = ia64_getreg(_IA64_REG_CR_TPR);
+#endif
 	ia64_srlz_d();
 	while (vector != IA64_SPURIOUS_INT_VECTOR) {
 		if (!IS_RESCHEDULE(vector)) {
+#ifdef CONFIG_XEN
+			ia64_set_tpr(vector);
+#else
 			ia64_setreg(_IA64_REG_CR_TPR, vector);
+#endif
 			ia64_srlz_d();
 
 			__do_IRQ(local_vector_to_irq(vector), regs);
@@ -154,7 +162,11 @@
 			 * Disable interrupts and send EOI:
 			 */
 			local_irq_disable();
+#ifdef CONFIG_XEN
+			ia64_set_tpr(saved_tpr);
+#else
 			ia64_setreg(_IA64_REG_CR_TPR, saved_tpr);
+#endif
 		}
 		ia64_eoi();
 		vector = ia64_get_ivr();
===== include/asm-ia64/system.h 1.48 vs 1.50 =====
--- 1.48/include/asm-ia64/system.h	Tue Jan  4 19:48:18 2005
+++ 1.50/include/asm-ia64/system.h	Thu May 12 11:10:16 2005
@@ -181,6 +181,10 @@
 	(__ia64_id_flags & IA64_PSR_I) == 0;	\
 })
 
+#ifdef CONFIG_XEN
+#include <asm/xen/system.h>
+#endif
+
 #ifdef __KERNEL__
 
 #define prepare_to_switch()    do { } while(0)
===== arch/ia64/hp/sim/Makefile 1.8 vs 1.9 =====
--- 1.8/arch/ia64/hp/sim/Makefile	Sat Aug 16 16:58:46 2003
+++ 1.9/arch/ia64/hp/sim/Makefile	Fri Apr 22 16:52:50 2005
@@ -13,4 +13,6 @@
 obj-$(CONFIG_HP_SIMETH)	+= simeth.o
 obj-$(CONFIG_HP_SIMSERIAL) += simserial.o
 obj-$(CONFIG_HP_SIMSERIAL_CONSOLE) += hpsim_console.o
+obj-$(CONFIG_XEN) += simserial.o
+obj-$(CONFIG_XEN) += hpsim_console.o
 obj-$(CONFIG_HP_SIMSCSI) += simscsi.o
===== arch/ia64/kernel/head.S 1.31 vs 1.34 =====
--- 1.31/arch/ia64/kernel/head.S	Fri Jan 28 16:50:13 2005
+++ 1.34/arch/ia64/kernel/head.S	Tue Jun 28 16:59:42 2005
@@ -240,6 +240,10 @@
 
 	// This is executed by the bootstrap processor (bsp) only:
 
+#ifdef CONFIG_XEN
+	br.call.sptk.many rp=early_xen_setup
+.retx:
+#endif
 #ifdef CONFIG_IA64_FW_EMU
 	// initialize PAL & SAL emulator:
 	br.call.sptk.many rp=sys_fw_init
===== drivers/acpi/motherboard.c 1.6 vs 1.7 =====
--- 1.6/drivers/acpi/motherboard.c	Wed Nov 10 15:57:35 2004
+++ 1.7/drivers/acpi/motherboard.c	Fri Apr 22 16:47:41 2005
@@ -120,6 +120,9 @@
 static void __init
 acpi_reserve_resources (void)
 {
+#ifdef CONFIG_XEN
+	if (!acpi_gbl_FADT) return;
+#endif
 	if (acpi_gbl_FADT->xpm1a_evt_blk.address && acpi_gbl_FADT->pm1_evt_len)
 		request_region(acpi_gbl_FADT->xpm1a_evt_blk.address, 
 			acpi_gbl_FADT->pm1_evt_len, "PM1a_EVT_BLK");
===== arch/ia64/Kconfig 1.85 vs 1.86 =====
--- 1.85/arch/ia64/Kconfig	Fri Jan 28 16:32:25 2005
+++ 1.86/arch/ia64/Kconfig	Fri Apr 22 16:52:50 2005
@@ -46,6 +46,10 @@
 	bool
 	default y
 
+config XEN
+	bool
+	default y
+
 choice
 	prompt "System type"
 	default IA64_GENERIC
===== arch/ia64/kernel/pal.S 1.10 vs 1.13 =====
--- 1.10/arch/ia64/kernel/pal.S	Fri Jul  2 05:27:05 2004
+++ 1.13/arch/ia64/kernel/pal.S	Wed Jun 29 13:46:07 2005
@@ -16,6 +16,9 @@
 #include <asm/processor.h>
 
 	.data
+#ifdef CONFIG_XEN
+	.globl pal_entry_point
+#endif
 pal_entry_point:
 	data8 ia64_pal_default_handler
 	.text
@@ -55,6 +58,12 @@
  */
 GLOBAL_ENTRY(ia64_pal_call_static)
 	.prologue ASM_UNW_PRLG_RP|ASM_UNW_PRLG_PFS, ASM_UNW_PRLG_GRSAVE(5)
+#ifdef CONFIG_XEN
+	movl r22=running_on_xen;;
+	ld4 r22=[r22];;
+	cmp.ne p7,p0=r22,r0
+(p7)	br.cond.spnt.many xen_pal_call_static;;
+#endif
 	alloc loc1 = ar.pfs,5,5,0,0
 	movl loc2 = pal_entry_point
 1:	{
@@ -73,7 +82,6 @@
 	mov loc0 = rp
 	.body
 	mov r30 = in2
-
 (p6)	rsm psr.i | psr.ic
 	mov r31 = in3
 	mov b7 = loc2
===== include/asm-ia64/ia32.h 1.31 vs 1.32 =====
--- 1.31/include/asm-ia64/ia32.h	Wed Oct  6 23:59:48 2004
+++ 1.32/include/asm-ia64/ia32.h	Wed May  4 14:16:59 2005
@@ -19,6 +19,10 @@
 extern int ia32_exception (struct pt_regs *regs, unsigned long isr);
 extern int ia32_intercept (struct pt_regs *regs, unsigned long isr);
 extern int ia32_clone_tls (struct task_struct *child, struct pt_regs *childregs);
+#ifdef CONFIG_XEN
+extern unsigned long xen_get_eflag(void);
+extern void xen_set_eflag(unsigned long);
+#endif
 
 # endif /* !CONFIG_IA32_SUPPORT */
 
===== arch/ia64/ia32/ia32_support.c 1.20 vs 1.22 =====
--- 1.20/arch/ia64/ia32/ia32_support.c	Wed Oct  6 23:55:23 2004
+++ 1.22/arch/ia64/ia32/ia32_support.c	Fri May  6 15:23:00 2005
@@ -100,7 +100,11 @@
 void
 ia32_save_state (struct task_struct *t)
 {
+#ifdef CONFIG_XEN
+	t->thread.eflag = xen_get_eflag();
+#else
 	t->thread.eflag = ia64_getreg(_IA64_REG_AR_EFLAG);
+#endif
 	t->thread.fsr   = ia64_getreg(_IA64_REG_AR_FSR);
 	t->thread.fcr   = ia64_getreg(_IA64_REG_AR_FCR);
 	t->thread.fir   = ia64_getreg(_IA64_REG_AR_FIR);
@@ -122,7 +126,11 @@
 	fdr = t->thread.fdr;
 	tssd = load_desc(_TSS);					/* TSSD */
 
+#ifdef CONFIG_XEN
+	xen_set_eflag(eflag);
+#else
 	ia64_setreg(_IA64_REG_AR_EFLAG, eflag);
+#endif
 	ia64_setreg(_IA64_REG_AR_FSR, fsr);
 	ia64_setreg(_IA64_REG_AR_FCR, fcr);
 	ia64_setreg(_IA64_REG_AR_FIR, fir);
===== drivers/Makefile 1.54 vs 1.56 =====
--- 1.54/drivers/Makefile	Mon Jan 17 11:06:18 2005
+++ 1.56/drivers/Makefile	Wed May 25 14:22:47 2005
@@ -63,3 +63,4 @@
 obj-$(CONFIG_INFINIBAND)	+= infiniband/
 obj-y				+= firmware/
 obj-$(CONFIG_CRYPTO)		+= crypto/
+obj-$(CONFIG_XEN)		+= xen/

^ permalink raw reply	[flat|nested] 21+ messages in thread
end of thread, other threads:[~2005-08-31 15:52 UTC | newest]

Thread overview: 21+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2005-06-29 20:16 Xen and the Art of Linux/ia64 Virtualization Magenheimer, Dan (HP Labs Fort Collins)
2005-06-29 20:22 ` Magenheimer, Dan (HP Labs Fort Collins)
2005-07-04 15:24 ` Christoph Hellwig
2005-07-04 18:26 ` Magenheimer, Dan (HP Labs Fort Collins)
2005-07-05 20:20 ` Magenheimer, Dan (HP Labs Fort Collins)
2005-07-06 20:46 ` Magenheimer, Dan (HP Labs Fort Collins)
2005-07-07 22:18 ` Gerald Pfeifer
2005-07-07 23:57 ` Magenheimer, Dan (HP Labs Fort Collins)
2005-07-11 15:09 ` Magenheimer, Dan (HP Labs Fort Collins)
2005-07-14 21:17 ` Magenheimer, Dan (HP Labs Fort Collins)
2005-07-16  1:15 ` Christoph Hellwig
2005-07-16 12:55 ` Magenheimer, Dan (HP Labs Fort Collins)
2005-07-28 22:38 ` Magenheimer, Dan (HP Labs Fort Collins)
2005-07-28 23:13 ` david mosberger
2005-07-28 23:27 ` Magenheimer, Dan (HP Labs Fort Collins)
2005-08-12 22:09 ` Magenheimer, Dan (HP Labs Fort Collins)
2005-08-22 21:45 ` Arun Sharma
2005-08-22 22:02 ` Magenheimer, Dan (HP Labs Fort Collins)
2005-08-22 22:22 ` Arun Sharma
2005-08-29 19:12 ` Magenheimer, Dan (HP Labs Fort Collins)
2005-08-31 15:52 ` Magenheimer, Dan (HP Labs Fort Collins)

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox