public inbox for linux-ia64@vger.kernel.org
 help / color / mirror / Atom feed
* Xen and the Art of Linux/ia64 Virtualization
@ 2005-06-29 20:16 Magenheimer, Dan (HP Labs Fort Collins)
  2005-06-29 20:22 ` Magenheimer, Dan (HP Labs Fort Collins)
                   ` (19 more replies)
  0 siblings, 20 replies; 21+ messages in thread
From: Magenheimer, Dan (HP Labs Fort Collins) @ 2005-06-29 20:16 UTC (permalink / raw)
  To: linux-ia64

[-- Attachment #1: Type: text/plain, Size: 3420 bytes --]

Most of you probably have already heard of Xen because it
was mentioned prominently (and often incorrectly) in many
of the press releases for Fedora Core 4 and Linux 2.6.12.
In short, Xen is a virtualization technology that allows
a single machine to simultaneously and efficiently run
multiple fully-isolated operating systems -- even different
distributions, different Linux kernel versions, or even
non-Linux operating systems (e.g. FreeBSD).  It can also do
other cool tricks such as live migration from one machine
to another, efficient virtual I/O to hide underlying storage
implementations, and varying how many CPUs are assigned to
the multiple running kernels.  Xen runs today on i386,
is nearing completin on x86_64, and is being ported to
ia64 and ppc.  For more information, see
the Xen web page (google it) or I can try to answer some
general questions in this thread if you prefer.

One principal concept of Xen, called "paravirtualization"
is that a few relatively minor changes to the kernel
can make it perform much more efficiently, in fact
near native performance.  There has already been some
discussion on lkml about how these changes might find their
way into Linux/x86, e.g. with a config option.

I am the principal developer and maintainer of the Xen port
to ia64, which is currently approaching its Beta release, and
currently has contributors from Intel, SGI, CERN, and Bull.
I have also been co-developing "xenlinux" for ia64, the
paravirtualized version of Linux/ia64.  Like Xen/x86, the
required linux changes are small but -- unlike Xen/x86 --
the required changes for Linux/ia64 are entirely archdep;
no common files need to be changed.  Also unlike Xen/x86,
Linux/ia64 with CONFIG_XEN enabled runs successfully on Xen,
but also run natively on hardware (called "transparent
paravirtualization").  This should be appealing to Linux/ia64
distributors as different kernels need not be shipped for
running on Xen or running native.

I would like to submit a series of CONFIG_XEN patches to
this list for review over the next few weeks.  The first
patch will be a few syntactic changes that will make
subsequent patches easier.  For example, there are a number of
places where ia64_getreg(REG_X) is used where abstracting
at a slightly higher level, e.g. with ia64_get_regX() would
make CONFIG_XEN easier.

For preliminary discussion, I have attached the current
patch for existing files for 2.6.11.  There are many other
files required (in arch/ia64/xen, include/asm-ia64/xen, and
drivers/xen) for everything to work, but I'd like to
initiate discussion and review on existing Linux/ia64 files
that would change.  Note that the patch contains many ifdef's;
with a more intrusive patch, most of the ifdef's could/should
be eliminated.  But for now, the ifdef's serve to more clearly
identify what the required changes are.

Having never submitted patches to Linux/ia64 (except a few
minor ones indirectly via David Mosberger), I thought it might
be a good idea to start by soliciting discussion.
So comments please!  After collecting and absorbing input,
I will work on a 2.6.12 patch.

For those who can't wait, a pre-beta version of Xen/ia64 and
the transparently paravirtualized 2.6.11 xenlinux/ia64 can be
found at xen-ia64.bkbits.net (and soon in a repository for
a different SCM).

Thanks!
Dan Magenheimer
HP Labs


[-- Attachment #2: xenlinux.diff --]
[-- Type: application/octet-stream, Size: 11747 bytes --]

===== include/asm-ia64/delay.h 1.10 vs 1.12 =====
--- 1.10/include/asm-ia64/delay.h	Thu Dec 16 12:57:15 2004
+++ 1.12/include/asm-ia64/delay.h	Wed Jun 29 13:46:08 2005
@@ -20,12 +20,17 @@
 #include <asm/intrinsics.h>
 #include <asm/processor.h>
 
+#ifdef CONFIG_XEN
+extern void xen_set_itm (unsigned long);
+#define ia64_set_itm(val) xen_set_itm(val)
+#else
 static __inline__ void
 ia64_set_itm (unsigned long val)
 {
 	ia64_setreg(_IA64_REG_CR_ITM, val);
 	ia64_srlz_d();
 }
+#endif
 
 static __inline__ unsigned long
 ia64_get_itm (void)
===== arch/ia64/ia32/ia32_signal.c 1.35 vs 1.36 =====
--- 1.35/arch/ia64/ia32/ia32_signal.c	Tue Jan 25 13:23:45 2005
+++ 1.36/arch/ia64/ia32/ia32_signal.c	Wed May  4 14:16:59 2005
@@ -667,7 +667,11 @@
 	/*
 	 *  `eflags' is in an ar register for this context
 	 */
+#ifdef CONFIG_XEN
+	flag = xen_get_eflag();
+#else
 	flag = ia64_getreg(_IA64_REG_AR_EFLAG);
+#endif
 	err |= __put_user((unsigned int)flag, &sc->eflags);
 	err |= __put_user(regs->r12, &sc->esp_at_signal);
 	err |= __put_user((regs->r17 >> 16) & 0xffff, (unsigned int __user *)&sc->ss);
@@ -755,10 +759,18 @@
 		 *  IA32 process's context.
 		 */
 		err |= __get_user(tmpflags, &sc->eflags);
+#ifdef CONFIG_XEN
+		flag = xen_get_eflag();
+#else
 		flag = ia64_getreg(_IA64_REG_AR_EFLAG);
+#endif
 		flag &= ~0x40DD5;
 		flag |= (tmpflags & 0x40DD5);
+#ifdef CONFIG_XEN
+		xen_set_eflag(flag);
+#else
 		ia64_setreg(_IA64_REG_AR_EFLAG, flag);
+#endif
 
 		regs->r1 = -1;	/* disable syscall checks, r1 is orig_eax */
 	}
===== arch/ia64/Makefile 1.74 vs 1.80 =====
--- 1.74/arch/ia64/Makefile	Fri Jan 28 16:32:45 2005
+++ 1.80/arch/ia64/Makefile	Tue Jun 28 16:59:42 2005
@@ -11,6 +11,8 @@
 NM := $(CROSS_COMPILE)nm -B
 READELF := $(CROSS_COMPILE)readelf
 
+NOSTDINC_FLAGS += -Iinclude/asm-xen
+
 export AWK
 
 CHECKFLAGS	+= -m64 -D__ia64=1 -D__ia64__=1 -D_LP64 -D__LP64__
@@ -57,9 +59,13 @@
 core-$(CONFIG_IA64_HP_ZX1)	+= arch/ia64/dig/
 core-$(CONFIG_IA64_HP_ZX1_SWIOTLB) += arch/ia64/dig/
 core-$(CONFIG_IA64_SGI_SN2)	+= arch/ia64/sn/
+core-$(CONFIG_XEN)		+= arch/ia64/xen/
 
 drivers-$(CONFIG_PCI)		+= arch/ia64/pci/
+ifneq ($(CONFIG_XEN),y)
 drivers-$(CONFIG_IA64_HP_SIM)	+= arch/ia64/hp/sim/
+endif
+drivers-$(CONFIG_XEN)		+= arch/ia64/hp/sim/
 drivers-$(CONFIG_IA64_HP_ZX1)	+= arch/ia64/hp/common/ arch/ia64/hp/zx1/
 drivers-$(CONFIG_IA64_HP_ZX1_SWIOTLB) += arch/ia64/hp/common/ arch/ia64/hp/zx1/
 drivers-$(CONFIG_IA64_GENERIC)	+= arch/ia64/hp/common/ arch/ia64/hp/zx1/ arch/ia64/hp/sim/ arch/ia64/sn/
===== arch/ia64/kernel/setup.c 1.86 vs 1.90 =====
--- 1.86/arch/ia64/kernel/setup.c	Wed Feb  2 13:21:27 2005
+++ 1.90/arch/ia64/kernel/setup.c	Wed Jun 29 13:46:07 2005
@@ -1,3 +1,4 @@
+
 /*
  * Architecture-specific setup.
  *
@@ -270,6 +271,16 @@
 static inline int __init
 early_console_setup (char *cmdline)
 {
+#ifdef CONFIG_XEN
+#ifndef CONFIG_IA64_HP_SIM
+	extern int running_on_xen;
+	if (running_on_xen) {
+		extern struct console hpsim_cons;
+		register_console(&hpsim_cons);
+		return 0;
+	}
+#endif
+#else
 #ifdef CONFIG_SERIAL_SGI_L1_CONSOLE
 	{
 		extern int sn_serial_console_early_setup(void);
@@ -284,6 +295,7 @@
 #ifdef CONFIG_SERIAL_8250_CONSOLE
 	if (!early_serial_console_init(cmdline))
 		return 0;
+#endif
 #endif
 
 	return -1;
===== arch/ia64/kernel/entry.S 1.80 vs edited =====
--- 1.80/arch/ia64/kernel/entry.S	Fri Jan 28 16:15:49 2005
+++ edited/arch/ia64/kernel/entry.S	Wed Jun 29 12:45:24 2005
@@ -184,6 +184,12 @@
 GLOBAL_ENTRY(ia64_switch_to)
 	.prologue
 	alloc r16=ar.pfs,1,0,0,0
+#ifdef CONFIG_XEN
+	movl r22=running_on_xen;;
+	ld4 r22=[r22];;
+	cmp.ne p7,p0=r22,r0
+(p7)	br.cond.spnt.few xen_switch_to;;
+#endif
 	DO_SAVE_SWITCH_STACK
 	.body
 
@@ -376,7 +382,11 @@
  *	- b7 holds address to return to
  *	- must not touch r8-r11
  */
+#ifdef CONFIG_XEN
+GLOBAL_ENTRY(load_switch_stack)
+#else
 ENTRY(load_switch_stack)
+#endif
 	.prologue
 	.altrp b7
 
@@ -671,6 +681,12 @@
  */
 ENTRY(ia64_leave_syscall)
 	PT_REGS_UNWIND_INFO(0)
+#ifdef CONFIG_XEN
+	movl r22=running_on_xen;;
+	ld4 r22=[r22];;
+	cmp.ne p7,p0=r22,r0
+(p7)	br.cond.spnt.few xen_leave_syscall;;
+#endif
 	/*
 	 * work.need_resched etc. mustn't get changed by this CPU before it returns to
 	 * user- or fsys-mode, hence we disable interrupts early on.
@@ -787,6 +803,12 @@
 #endif /* CONFIG_IA32_SUPPORT */
 GLOBAL_ENTRY(ia64_leave_kernel)
 	PT_REGS_UNWIND_INFO(0)
+#ifdef CONFIG_XEN
+	movl r22=running_on_xen;;
+	ld4 r22=[r22];;
+	cmp.ne p7,p0=r22,r0
+(p7)	br.cond.spnt.few xen_leave_kernel;;
+#endif
 	/*
 	 * work.need_resched etc. mustn't get changed by this CPU before it returns to
 	 * user- or fsys-mode, hence we disable interrupts early on.
@@ -1171,7 +1193,11 @@
 	 * be set up by the caller.  We declare 8 input registers so the system call
 	 * args get preserved, in case we need to restart a system call.
 	 */
+#ifdef CONFIG_XEN
+GLOBAL_ENTRY(notify_resume_user)
+#else
 ENTRY(notify_resume_user)
+#endif
 	.prologue ASM_UNW_PRLG_RP|ASM_UNW_PRLG_PFS, ASM_UNW_PRLG_GRSAVE(8)
 	alloc loc1=ar.pfs,8,2,3,0 // preserve all eight input regs in case of syscall restart!
 	mov r9=ar.unat
===== include/asm-ia64/processor.h 1.72 vs 1.79 =====
--- 1.72/include/asm-ia64/processor.h	Wed Jan 26 11:01:41 2005
+++ 1.79/include/asm-ia64/processor.h	Wed Jun 29 13:46:08 2005
@@ -551,12 +551,17 @@
 	ia64_srlz_i();
 }
 
+#ifdef CONFIG_XEN
+extern void xen_eoi (void);
+#define ia64_eoi xen_eoi
+#else
 static inline void
 ia64_eoi (void)
 {
 	ia64_setreg(_IA64_REG_CR_EOI, 0);
 	ia64_srlz_d();
 }
+#endif
 
 #define cpu_relax()	ia64_hint(ia64_hint_pause)
 
@@ -621,6 +626,10 @@
 #define current_text_addr() \
 	({ void *_pc; _pc = (void *)ia64_getreg(_IA64_REG_IP); _pc; })
 
+#ifdef CONFIG_XEN
+extern __u64 xen_get_ivr (void);
+#define ia64_get_ivr xen_get_ivr
+#else
 static inline __u64
 ia64_get_ivr (void)
 {
@@ -630,6 +639,7 @@
 	ia64_srlz_d();
 	return r;
 }
+#endif
 
 static inline void
 ia64_set_dbr (__u64 regnum, __u64 value)
@@ -658,6 +668,11 @@
 	return (w >> n) | (w << (64 - n));
 }
 
+#ifdef CONFIG_XEN
+#define ia64_get_tpr()	(ia64_getreg(_IA64_REG_CR_TPR))
+#define ia64_set_tpr(vector)	ia64_setreg(_IA64_REG_CR_TPR,vector)
+#endif
+
 #define ia64_rotl(w,n)	ia64_rotr((w), (64) - (n))
 
 /*
@@ -694,5 +709,9 @@
 extern unsigned long boot_option_idle_override;
 
 #endif /* !__ASSEMBLY__ */
+
+#ifdef CONFIG_XEN
+#include <asm/xen/processor.h>
+#endif
 
 #endif /* _ASM_IA64_PROCESSOR_H */
===== arch/ia64/kernel/irq_ia64.c 1.25 vs 1.31 =====
--- 1.25/arch/ia64/kernel/irq_ia64.c	Sat Jan 22 16:54:49 2005
+++ 1.31/arch/ia64/kernel/irq_ia64.c	Wed Jun 29 13:46:07 2005
@@ -141,11 +141,19 @@
 	 * to kernel stack overflows).
 	 */
 	irq_enter();
+#ifdef CONFIG_XEN
+	saved_tpr = ia64_get_tpr();
+#else
 	saved_tpr = ia64_getreg(_IA64_REG_CR_TPR);
+#endif
 	ia64_srlz_d();
 	while (vector != IA64_SPURIOUS_INT_VECTOR) {
 		if (!IS_RESCHEDULE(vector)) {
+#ifdef CONFIG_XEN
+			ia64_set_tpr(vector);
+#else
 			ia64_setreg(_IA64_REG_CR_TPR, vector);
+#endif
 			ia64_srlz_d();
 
 			__do_IRQ(local_vector_to_irq(vector), regs);
@@ -154,7 +162,11 @@
 			 * Disable interrupts and send EOI:
 			 */
 			local_irq_disable();
+#ifdef CONFIG_XEN
+			ia64_set_tpr(saved_tpr);
+#else
 			ia64_setreg(_IA64_REG_CR_TPR, saved_tpr);
+#endif
 		}
 		ia64_eoi();
 		vector = ia64_get_ivr();
===== include/asm-ia64/system.h 1.48 vs 1.50 =====
--- 1.48/include/asm-ia64/system.h	Tue Jan  4 19:48:18 2005
+++ 1.50/include/asm-ia64/system.h	Thu May 12 11:10:16 2005
@@ -181,6 +181,10 @@
 	(__ia64_id_flags & IA64_PSR_I) == 0;	\
 })
 
+#ifdef CONFIG_XEN
+#include <asm/xen/system.h>
+#endif
+
 #ifdef __KERNEL__
 
 #define prepare_to_switch()    do { } while(0)
===== arch/ia64/hp/sim/Makefile 1.8 vs 1.9 =====
--- 1.8/arch/ia64/hp/sim/Makefile	Sat Aug 16 16:58:46 2003
+++ 1.9/arch/ia64/hp/sim/Makefile	Fri Apr 22 16:52:50 2005
@@ -13,4 +13,6 @@
 obj-$(CONFIG_HP_SIMETH)	+= simeth.o
 obj-$(CONFIG_HP_SIMSERIAL) += simserial.o
 obj-$(CONFIG_HP_SIMSERIAL_CONSOLE) += hpsim_console.o
+obj-$(CONFIG_XEN) += simserial.o
+obj-$(CONFIG_XEN) += hpsim_console.o
 obj-$(CONFIG_HP_SIMSCSI) += simscsi.o
===== arch/ia64/kernel/head.S 1.31 vs 1.34 =====
--- 1.31/arch/ia64/kernel/head.S	Fri Jan 28 16:50:13 2005
+++ 1.34/arch/ia64/kernel/head.S	Tue Jun 28 16:59:42 2005
@@ -240,6 +240,10 @@
 
 	// This is executed by the bootstrap processor (bsp) only:
 
+#ifdef CONFIG_XEN
+	br.call.sptk.many rp=early_xen_setup
+.retx:
+#endif
 #ifdef CONFIG_IA64_FW_EMU
 	// initialize PAL & SAL emulator:
 	br.call.sptk.many rp=sys_fw_init
===== drivers/acpi/motherboard.c 1.6 vs 1.7 =====
--- 1.6/drivers/acpi/motherboard.c	Wed Nov 10 15:57:35 2004
+++ 1.7/drivers/acpi/motherboard.c	Fri Apr 22 16:47:41 2005
@@ -120,6 +120,9 @@
 static void __init
 acpi_reserve_resources (void)
 {
+#ifdef CONFIG_XEN
+	if (!acpi_gbl_FADT) return;
+#endif
 	if (acpi_gbl_FADT->xpm1a_evt_blk.address && acpi_gbl_FADT->pm1_evt_len)
 		request_region(acpi_gbl_FADT->xpm1a_evt_blk.address, 
 			acpi_gbl_FADT->pm1_evt_len, "PM1a_EVT_BLK");
===== arch/ia64/Kconfig 1.85 vs 1.86 =====
--- 1.85/arch/ia64/Kconfig	Fri Jan 28 16:32:25 2005
+++ 1.86/arch/ia64/Kconfig	Fri Apr 22 16:52:50 2005
@@ -46,6 +46,10 @@
 	bool
 	default y
 
+config XEN
+	bool
+	default y
+
 choice
 	prompt "System type"
 	default IA64_GENERIC
===== arch/ia64/kernel/pal.S 1.10 vs 1.13 =====
--- 1.10/arch/ia64/kernel/pal.S	Fri Jul  2 05:27:05 2004
+++ 1.13/arch/ia64/kernel/pal.S	Wed Jun 29 13:46:07 2005
@@ -16,6 +16,9 @@
 #include <asm/processor.h>
 
 	.data
+#ifdef CONFIG_XEN
+	.globl pal_entry_point
+#endif
 pal_entry_point:
 	data8 ia64_pal_default_handler
 	.text
@@ -55,6 +58,12 @@
  */
 GLOBAL_ENTRY(ia64_pal_call_static)
 	.prologue ASM_UNW_PRLG_RP|ASM_UNW_PRLG_PFS, ASM_UNW_PRLG_GRSAVE(5)
+#ifdef CONFIG_XEN
+	movl r22=running_on_xen;;
+	ld4 r22=[r22];;
+	cmp.ne p7,p0=r22,r0
+(p7)	br.cond.spnt.many xen_pal_call_static;;
+#endif
 	alloc loc1 = ar.pfs,5,5,0,0
 	movl loc2 = pal_entry_point
 1:	{
@@ -73,7 +82,6 @@
 	mov loc0 = rp
 	.body
 	mov r30 = in2
-
 (p6)	rsm psr.i | psr.ic
 	mov r31 = in3
 	mov b7 = loc2
===== include/asm-ia64/ia32.h 1.31 vs 1.32 =====
--- 1.31/include/asm-ia64/ia32.h	Wed Oct  6 23:59:48 2004
+++ 1.32/include/asm-ia64/ia32.h	Wed May  4 14:16:59 2005
@@ -19,6 +19,10 @@
 extern int ia32_exception (struct pt_regs *regs, unsigned long isr);
 extern int ia32_intercept (struct pt_regs *regs, unsigned long isr);
 extern int ia32_clone_tls (struct task_struct *child, struct pt_regs *childregs);
+#ifdef CONFIG_XEN
+extern unsigned long xen_get_eflag(void);
+extern void xen_set_eflag(unsigned long);
+#endif
 
 # endif /* !CONFIG_IA32_SUPPORT */
 
===== arch/ia64/ia32/ia32_support.c 1.20 vs 1.22 =====
--- 1.20/arch/ia64/ia32/ia32_support.c	Wed Oct  6 23:55:23 2004
+++ 1.22/arch/ia64/ia32/ia32_support.c	Fri May  6 15:23:00 2005
@@ -100,7 +100,11 @@
 void
 ia32_save_state (struct task_struct *t)
 {
+#ifdef CONFIG_XEN
+	t->thread.eflag = xen_get_eflag();
+#else
 	t->thread.eflag = ia64_getreg(_IA64_REG_AR_EFLAG);
+#endif
 	t->thread.fsr   = ia64_getreg(_IA64_REG_AR_FSR);
 	t->thread.fcr   = ia64_getreg(_IA64_REG_AR_FCR);
 	t->thread.fir   = ia64_getreg(_IA64_REG_AR_FIR);
@@ -122,7 +126,11 @@
 	fdr = t->thread.fdr;
 	tssd = load_desc(_TSS);					/* TSSD */
 
+#ifdef CONFIG_XEN
+	xen_set_eflag(eflag);
+#else
 	ia64_setreg(_IA64_REG_AR_EFLAG, eflag);
+#endif
 	ia64_setreg(_IA64_REG_AR_FSR, fsr);
 	ia64_setreg(_IA64_REG_AR_FCR, fcr);
 	ia64_setreg(_IA64_REG_AR_FIR, fir);
===== drivers/Makefile 1.54 vs 1.56 =====
--- 1.54/drivers/Makefile	Mon Jan 17 11:06:18 2005
+++ 1.56/drivers/Makefile	Wed May 25 14:22:47 2005
@@ -63,3 +63,4 @@
 obj-$(CONFIG_INFINIBAND)	+= infiniband/
 obj-y				+= firmware/
 obj-$(CONFIG_CRYPTO)		+= crypto/
+obj-$(CONFIG_XEN)		+= xen/

^ permalink raw reply	[flat|nested] 21+ messages in thread

* RE: Xen and the Art of Linux/ia64 Virtualization
  2005-06-29 20:16 Xen and the Art of Linux/ia64 Virtualization Magenheimer, Dan (HP Labs Fort Collins)
@ 2005-06-29 20:22 ` Magenheimer, Dan (HP Labs Fort Collins)
  2005-07-04 15:24 ` Christoph Hellwig
                   ` (18 subsequent siblings)
  19 siblings, 0 replies; 21+ messages in thread
From: Magenheimer, Dan (HP Labs Fort Collins) @ 2005-06-29 20:22 UTC (permalink / raw)
  To: linux-ia64

Forgot to add the diffstat...

> -----Original Message-----
> From: Magenheimer, Dan (HP Labs Fort Collins) 
> Sent: Wednesday, June 29, 2005 2:17 PM
> To: linux-ia64@vger.kernel.org
> Subject: Xen and the Art of Linux/ia64 Virtualization

 arch/ia64/Kconfig             |    4 ++++
 arch/ia64/Makefile            |    6 ++++++
 arch/ia64/hp/sim/Makefile     |    2 ++
 arch/ia64/ia32/ia32_signal.c  |   12 ++++++++++++
 arch/ia64/ia32/ia32_support.c |    8 ++++++++
 arch/ia64/kernel/entry.S      |   26 ++++++++++++++++++++++++++
 arch/ia64/kernel/head.S       |    4 ++++
 arch/ia64/kernel/irq_ia64.c   |   12 ++++++++++++
 arch/ia64/kernel/pal.S        |   10 +++++++++-
 arch/ia64/kernel/setup.c      |   12 ++++++++++++
 drivers/Makefile              |    1 +
 drivers/acpi/motherboard.c    |    3 +++
 include/asm-ia64/delay.h      |    5 +++++
 include/asm-ia64/ia32.h       |    4 ++++
 include/asm-ia64/processor.h  |   19 +++++++++++++++++++
 include/asm-ia64/system.h     |    4 ++++

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: Xen and the Art of Linux/ia64 Virtualization
  2005-06-29 20:16 Xen and the Art of Linux/ia64 Virtualization Magenheimer, Dan (HP Labs Fort Collins)
  2005-06-29 20:22 ` Magenheimer, Dan (HP Labs Fort Collins)
@ 2005-07-04 15:24 ` Christoph Hellwig
  2005-07-04 18:26 ` Magenheimer, Dan (HP Labs Fort Collins)
                   ` (17 subsequent siblings)
  19 siblings, 0 replies; 21+ messages in thread
From: Christoph Hellwig @ 2005-07-04 15:24 UTC (permalink / raw)
  To: linux-ia64

On Wed, Jun 29, 2005 at 01:16:34PM -0700, Magenheimer, Dan (HP Labs Fort Collins) wrote:
> to ia64, which is currently approaching its Beta release, and
> currently has contributors from Intel, SGI, CERN, and Bull.
> I have also been co-developing "xenlinux" for ia64, the
> paravirtualized version of Linux/ia64.  Like Xen/x86, the
> required linux changes are small but -- unlike Xen/x86 --
> the required changes for Linux/ia64 are entirely archdep;
> no common files need to be changed.  Also unlike Xen/x86,
> Linux/ia64 with CONFIG_XEN enabled runs successfully on Xen,
> but also run natively on hardware (called "transparent
> paravirtualization").  This should be appealing to Linux/ia64
> distributors as different kernels need not be shipped for
> running on Xen or running native.

Very cool, and I think xen/x86 will have to adopt that model
aswell if it wants to get in mainline.  Maybe you can clean up
the ia64 patches soon enough so we get xen/ia64 merged before
xen/x86?

> I would like to submit a series of CONFIG_XEN patches to
> this list for review over the next few weeks.  The first
> patch will be a few syntactic changes that will make
> subsequent patches easier.  For example, there are a number of
> places where ia64_getreg(REG_X) is used where abstracting
> at a slightly higher level, e.g. with ia64_get_regX() would
> make CONFIG_XEN easier.

Yes, absolutely.  That's always a good thing and the ifdef mess
in the current patch is quite bad.

> For preliminary discussion, I have attached the current
> patch for existing files for 2.6.11.  There are many other
> files required (in arch/ia64/xen, include/asm-ia64/xen, and
> drivers/xen) for everything to work, but I'd like to
> initiate discussion and review on existing Linux/ia64 files
> that would change.

Can you please post those bits aswell, at least the per-arch bits -
the last time I saw a patch for drivers/xen the code there was
really horrible, but I hope it has changed by now.  I also think
IA64 Xen arch bits can be merged before it, we support botting with
initrd and out of tree drivers quite well these days.

> For those who can't wait, a pre-beta version of Xen/ia64 and
> the transparently paravirtualized 2.6.11 xenlinux/ia64 can be
> found at xen-ia64.bkbits.net (and soon in a repository for
> a different SCM).

A patch would be much nicer..

--- 1.10/include/asm-ia64/delay.h	Thu Dec 16 12:57:15 2004
+++ 1.12/include/asm-ia64/delay.h	Wed Jun 29 13:46:08 2005
@@ -20,12 +20,17 @@
 #include <asm/intrinsics.h>
 #include <asm/processor.h>
 
+#ifdef CONFIG_XEN
+extern void xen_set_itm (unsigned long);
+#define ia64_set_itm(val) xen_set_itm(val)

Any reason you don't call the Xen version ia64_set_itm directly?

+#ifdef CONFIG_XEN
+	flag = xen_get_eflag();
+#else
 	flag = ia64_getreg(_IA64_REG_AR_EFLAG);
+#endif

So this is repeated all over, and an ia64_get_eflag() would be
cool (I suspect you're already planning it, but I'd like to offer
my opinion on the abstractions anyway, feel free to disacard them
if they're boring and duplicate what you already plan)

--- 1.74/arch/ia64/Makefile	Fri Jan 28 16:32:45 2005
+++ 1.80/arch/ia64/Makefile	Tue Jun 28 16:59:42 2005
@@ -11,6 +11,8 @@
 NM := $(CROSS_COMPILE)nm -B
 READELF := $(CROSS_COMPILE)readelf
 
+NOSTDINC_FLAGS += -Iinclude/asm-xen

Please don't do that.  Use '#include <asm-xen/foo.h>', but in reality
that stuff shouldn't be in include/asm-xen anyway but include/xen/.

 drivers-$(CONFIG_PCI)		+= arch/ia64/pci/
+ifneq ($(CONFIG_XEN),y)
 drivers-$(CONFIG_IA64_HP_SIM)	+= arch/ia64/hp/sim/
+endif
+drivers-$(CONFIG_XEN)		+= arch/ia64/hp/sim/

I think you can þave two drivers- lines for different symbols and let
the build system sort it out. Atleast it works for obj-, if it doesn't
work for drivers- ask the build system maintainers for a little
help.

Any particular reason you use the HP SIM bits but Xen/x86 doesn't?
their far less messy then what xen/x86 uses at least, and a lot simpler..

 /*
  * Architecture-specific setup.
  *
@@ -270,6 +271,16 @@
 static inline int __init
 early_console_setup (char *cmdline)
 {
+#ifdef CONFIG_XEN
+#ifndef CONFIG_IA64_HP_SIM
+	extern int running_on_xen;
+	if (running_on_xen) {
+		extern struct console hpsim_cons;
+		register_console(&hpsim_cons);
+		return 0;
+	}
+#endif

why the #ifndef CONFIG_IA64_HP_SIM?  

=== include/asm-ia64/processor.h 1.72 vs 1.79 ==--- 1.72/include/asm-ia64/processor.h	Wed Jan 26 11:01:41 2005
+++ 1.79/include/asm-ia64/processor.h	Wed Jun 29 13:46:08 2005
@@ -551,12 +551,17 @@
 	ia64_srlz_i();
 }
 
+#ifdef CONFIG_XEN
+extern void xen_eoi (void);
+#define ia64_eoi xen_eoi
+#else
 static inline void
 ia64_eoi (void)
 {
 	ia64_setreg(_IA64_REG_CR_EOI, 0);
 	ia64_srlz_d();
 }
+#endif

Same comment as for xen_set_itm().
 
 #define cpu_relax()	ia64_hint(ia64_hint_pause)
 
+#ifdef CONFIG_XEN
+extern __u64 xen_get_ivr (void);
+#define ia64_get_ivr xen_get_ivr
+#else
 static inline __u64
 ia64_get_ivr (void)
 {
@@ -630,6 +639,7 @@
 	ia64_srlz_d();
 	return r;
 }
+#endif

Dito. It might also be useful to regroup the inlines so all that
are overriden by Xen are in a single place.
 
 #endif /* !__ASSEMBLY__ */
+
+#ifdef CONFIG_XEN
+#include <asm/xen/processor.h>
+#endif

This is a bit ugly.  How big is asm/xen/processor.h? Can you just inline
it here, or maybe some of this should become machvecs?
 
--- 1.6/drivers/acpi/motherboard.c	Wed Nov 10 15:57:35 2004
+++ 1.7/drivers/acpi/motherboard.c	Fri Apr 22 16:47:41 2005
@@ -120,6 +120,9 @@
 static void __init
 acpi_reserve_resources (void)
 {
+#ifdef CONFIG_XEN
+	if (!acpi_gbl_FADT) return;
+#endif

Should be unconditional.

--- 1.85/arch/ia64/Kconfig	Fri Jan 28 16:32:25 2005
+++ 1.86/arch/ia64/Kconfig	Fri Apr 22 16:52:50 2005
@@ -46,6 +46,10 @@
 	bool
 	default y
 
+config XEN
+	bool
+	default y

You should make it user-selectable by adding a prompt text, and while
you're at it also add a help text.


^ permalink raw reply	[flat|nested] 21+ messages in thread

* RE: Xen and the Art of Linux/ia64 Virtualization
  2005-06-29 20:16 Xen and the Art of Linux/ia64 Virtualization Magenheimer, Dan (HP Labs Fort Collins)
  2005-06-29 20:22 ` Magenheimer, Dan (HP Labs Fort Collins)
  2005-07-04 15:24 ` Christoph Hellwig
@ 2005-07-04 18:26 ` Magenheimer, Dan (HP Labs Fort Collins)
  2005-07-05 20:20 ` Magenheimer, Dan (HP Labs Fort Collins)
                   ` (16 subsequent siblings)
  19 siblings, 0 replies; 21+ messages in thread
From: Magenheimer, Dan (HP Labs Fort Collins) @ 2005-07-04 18:26 UTC (permalink / raw)
  To: linux-ia64

Thanks for the reply and very useful feedback!

> > but also run natively on hardware (called "transparent
> > paravirtualization").  This should be appealing to Linux/ia64
> 
> Very cool, and I think xen/x86 will have to adopt that model
> aswell if it wants to get in mainline.  Maybe you can clean up
> the ia64 patches soon enough so we get xen/ia64 merged before
> xen/x86?

Yes, but this is much harder on Xen/x86 as the memory model is
somewhat different on Xen/x86 than on Linux/x86.  On Xen/ia64,
the differences can be completely hidden from Linux because
all mapping is done through the TLB (which requires a privileged
instruction that can be trapped).  The Xen/x86 guys have proposed
some ideas (e.g. a special linker that links in both Xen and
standard Linux routines and selects at runtime) but I don't think
anyone has tried to do it.

In any case, I agree that getting the xen/ia64 changes merged
in can be done prior to the xen/x86 changes.

> > I would like to submit a series of CONFIG_XEN patches to
> > this list for review over the next few weeks.  The first
> > patch will be a few syntactic changes that will make
> > subsequent patches easier.  For example, there are a number of
> > places where ia64_getreg(REG_X) is used where abstracting
> > at a slightly higher level, e.g. with ia64_get_regX() would
> > make CONFIG_XEN easier.
> 
> Yes, absolutely.  That's always a good thing and the ifdef mess
> in the current patch is quite bad.

Understood.  This was just a "get it to work" patch and intended
to solicit useful feedback (like yours below).
 
> > For preliminary discussion, I have attached the current
> > patch for existing files for 2.6.11.  There are many other
> > files required (in arch/ia64/xen, include/asm-ia64/xen, and
> > drivers/xen) for everything to work, but I'd like to
> > initiate discussion and review on existing Linux/ia64 files
> > that would change.
> 
> Can you please post those bits aswell, at least the per-arch bits -
> the last time I saw a patch for drivers/xen the code there was
> really horrible, but I hope it has changed by now.  I also think
> IA64 Xen arch bits can be merged before it, we support botting with
> initrd and out of tree drivers quite well these days.

The code in drivers/xen is still x86-specific.  We are working
on merging in some changes to accommodate ia64 (and other future
architectures) but they're not quite ready yet.

The arch/ia64/xen code includes a 95% identical IVT and 95%
identical ia64_leave_kernel/ia64_leave_syscall and a few others.
(The switch between the original and xen versions is handled
dynamically because the impact on the real IVT would be very
ugly.)  These are still evolving somewhat as I tune performance.
I can send you the current versions (off-list) if you like.

> --- 1.10/include/asm-ia64/delay.h	Thu Dec 16 12:57:15 2004
> +++ 1.12/include/asm-ia64/delay.h	Wed Jun 29 13:46:08 2005
> @@ -20,12 +20,17 @@
>  #include <asm/intrinsics.h>
>  #include <asm/processor.h>
>  
> +#ifdef CONFIG_XEN
> +extern void xen_set_itm (unsigned long);
> +#define ia64_set_itm(val) xen_set_itm(val)
> 
> Any reason you don't call the Xen version ia64_set_itm directly?

Do you mean just change the code in arch/ia64/time.c?  Actually,
what I was going to try was to move this out of delay.h and
into processor.h (or privop.h, see below).
 
> +#ifdef CONFIG_XEN
> +	flag = xen_get_eflag();
> +#else
>  	flag = ia64_getreg(_IA64_REG_AR_EFLAG);
> +#endif
> 
> So this is repeated all over, and an ia64_get_eflag() would be
> cool (I suspect you're already planning it, but I'd like to offer
> my opinion on the abstractions anyway, feel free to disacard them
> if they're boring and duplicate what you already plan)

Yep, that was the direction I am headed.  I'm glad you are
offering your opinion as it helps to reinforce the direction.
However, I am sure I will get some of the details wrong (or at
least that there will be varied opinions about the "right" way).

> --- 1.74/arch/ia64/Makefile	Fri Jan 28 16:32:45 2005
> +++ 1.80/arch/ia64/Makefile	Tue Jun 28 16:59:42 2005
> @@ -11,6 +11,8 @@
>  NM := $(CROSS_COMPILE)nm -B
>  READELF := $(CROSS_COMPILE)readelf
>  
> +NOSTDINC_FLAGS += -Iinclude/asm-xen
> 
> Please don't do that.  Use '#include <asm-xen/foo.h>', but in reality
> that stuff shouldn't be in include/asm-xen anyway but include/xen/.

Yeah the Xen team is still debating how best to structure the xen
code in the linux tree.  Hopefully that will be resolved at OLS.
In the meantime, I have a problem because there are two "asm" trees
that need to be include'd.  IOW, the NOSTDINC_FLAGS should go away.

>  drivers-$(CONFIG_PCI)		+= arch/ia64/pci/
> +ifneq ($(CONFIG_XEN),y)
>  drivers-$(CONFIG_IA64_HP_SIM)	+= arch/ia64/hp/sim/
> +endif
> +drivers-$(CONFIG_XEN)		+= arch/ia64/hp/sim/
> 
> I think you can þave two drivers- lines for different symbols and let
> the build system sort it out. Atleast it works for obj-, if it doesn't
> work for drivers- ask the build system maintainers for a little
> help.
> 
> Any particular reason you use the HP SIM bits but Xen/x86 doesn't?
> their far less messy then what xen/x86 uses at least, and a 
> lot simpler..

Actually I'm using both (the x86-based and the HP SIM).  One of my
side projects is to turn the HP SIM code into a ia64 CONFIG_ option rather
than a machvec (similar to what Greg Edwards just did with the
SGI simulator).

>  /*
>   * Architecture-specific setup.
>   *
> @@ -270,6 +271,16 @@
>  static inline int __init
>  early_console_setup (char *cmdline)
>  {
> +#ifdef CONFIG_XEN
> +#ifndef CONFIG_IA64_HP_SIM
> +	extern int running_on_xen;
> +	if (running_on_xen) {
> +		extern struct console hpsim_cons;
> +		register_console(&hpsim_cons);
> +		return 0;
> +	}
> +#endif
> 
> why the #ifndef CONFIG_IA64_HP_SIM?  

Because I am doing some debugging on the HP simulator :-) and CONFIG_IA64_HP_SIM
is currently required to build a Linux that runs on the simulator.  Hopefully
this will become unnecessary.

> === include/asm-ia64/processor.h 1.72 vs 1.79 ==> --- 1.72/include/asm-ia64/processor.h	Wed Jan 26 11:01:41 2005
> +#define ia64_eoi xen_eoi
> 
> Same comment as for xen_set_itm().
>  
> +#define ia64_get_ivr xen_get_ivr
> 
> Dito. It might also be useful to regroup the inlines so all that
> are overriden by Xen are in a single place.

Yes.  Actually, does it make sense to pull these into a separate
include file (e.g. asm/privop.h)?  Then I think I can get rid
of all CONFIG_XEN in processor.h... and privop.h may be more
appropriate since it is privileged operations that Xen is primarily
concerned with.
  
>  #endif /* !__ASSEMBLY__ */
> +
> +#ifdef CONFIG_XEN
> +#include <asm/xen/processor.h>
> +#endif
> 
> This is a bit ugly.  How big is asm/xen/processor.h? Can you 
> just inline
> it here, or maybe some of this should become machvecs?

I don't think machvec is an option as CONFIG_XEN should work
regardless of the machvec.

xen/processor.h is not real big but it is ugly

> --- 1.6/drivers/acpi/motherboard.c	Wed Nov 10 15:57:35 2004
> +++ 1.7/drivers/acpi/motherboard.c	Fri Apr 22 16:47:41 2005
> @@ -120,6 +120,9 @@
>  static void __init
>  acpi_reserve_resources (void)
>  {
> +#ifdef CONFIG_XEN
> +	if (!acpi_gbl_FADT) return;
> +#endif
> 
> Should be unconditional.

Agreed.  Should I submit a patch to the ACPI maintainer?

> --- 1.85/arch/ia64/Kconfig	Fri Jan 28 16:32:25 2005
> +++ 1.86/arch/ia64/Kconfig	Fri Apr 22 16:52:50 2005
> @@ -46,6 +46,10 @@
>  	bool
>  	default y
>  
> +config XEN
> +	bool
> +	default y
> 
> You should make it user-selectable by adding a prompt text, and while
> you're at it also add a help text.

Will do.  Haven't gotten around to many aesthetics yet as I was primarily
focused on proving that transparent paravirtualization would work.

Thanks again for the great feedback.  After I make another round
of cleanups, I will send the patch (at least phase 1) for another
review.

Dan

^ permalink raw reply	[flat|nested] 21+ messages in thread

* RE: Xen and the Art of Linux/ia64 Virtualization
  2005-06-29 20:16 Xen and the Art of Linux/ia64 Virtualization Magenheimer, Dan (HP Labs Fort Collins)
                   ` (2 preceding siblings ...)
  2005-07-04 18:26 ` Magenheimer, Dan (HP Labs Fort Collins)
@ 2005-07-05 20:20 ` Magenheimer, Dan (HP Labs Fort Collins)
  2005-07-06 20:46 ` Magenheimer, Dan (HP Labs Fort Collins)
                   ` (15 subsequent siblings)
  19 siblings, 0 replies; 21+ messages in thread
From: Magenheimer, Dan (HP Labs Fort Collins) @ 2005-07-05 20:20 UTC (permalink / raw)
  To: linux-ia64

[-- Attachment #1: Type: text/plain, Size: 647 bytes --]

Thanks to excellent feedback from David Mosberger and
Christophe Hellwig, I have greatly cleaned up the attached
patch by using some very nice abstractions.  In fact,
most of the bulk in the attached patch results from
moving some code from asm-ia64/{delay,processor}.h to
a new file, asm-ia64/privop.h.  And the number of
CONFIG_XEN ifdefs is dramatically reduced.

More feedback appreciated (including comments about how
close this might be to be ready for submission to Tony).

Thanks,
Dan Magenheimer

P.S. Anybody who is attending OLS who wants to talk about
Xen (and specifically Xen/ia64)?  I'll be there... send
me email.

[-- Attachment #2: xenia64.patch --]
[-- Type: application/octet-stream, Size: 17023 bytes --]

diff -Naur linux-2.6.12/arch/ia64/hp/sim/Makefile linux-2.6.12-XenPatch/arch/ia64/hp/sim/Makefile
--- linux-2.6.12/arch/ia64/hp/sim/Makefile	2005-06-17 13:48:29.000000000 -0600
+++ linux-2.6.12-XenPatch/arch/ia64/hp/sim/Makefile	2005-07-05 13:27:34.000000000 -0600
@@ -14,3 +14,5 @@
 obj-$(CONFIG_HP_SIMSERIAL) += simserial.o
 obj-$(CONFIG_HP_SIMSERIAL_CONSOLE) += hpsim_console.o
 obj-$(CONFIG_HP_SIMSCSI) += simscsi.o
+obj-$(CONFIG_XEN) += simserial.o
+obj-$(CONFIG_XEN) += hpsim_console.o
diff -Naur linux-2.6.12/arch/ia64/ia32/ia32_support.c linux-2.6.12-XenPatch/arch/ia64/ia32/ia32_support.c
--- linux-2.6.12/arch/ia64/ia32/ia32_support.c	2005-06-17 13:48:29.000000000 -0600
+++ linux-2.6.12-XenPatch/arch/ia64/ia32/ia32_support.c	2005-07-05 11:50:03.000000000 -0600
@@ -100,7 +100,7 @@
 void
 ia32_save_state (struct task_struct *t)
 {
-	t->thread.eflag = ia64_getreg(_IA64_REG_AR_EFLAG);
+	t->thread.eflag = ia64_get_eflag();
 	t->thread.fsr   = ia64_getreg(_IA64_REG_AR_FSR);
 	t->thread.fcr   = ia64_getreg(_IA64_REG_AR_FCR);
 	t->thread.fir   = ia64_getreg(_IA64_REG_AR_FIR);
@@ -122,7 +122,7 @@
 	fdr = t->thread.fdr;
 	tssd = load_desc(_TSS);					/* TSSD */
 
-	ia64_setreg(_IA64_REG_AR_EFLAG, eflag);
+	ia64_set_eflag(eflag);
 	ia64_setreg(_IA64_REG_AR_FSR, fsr);
 	ia64_setreg(_IA64_REG_AR_FCR, fcr);
 	ia64_setreg(_IA64_REG_AR_FIR, fir);
diff -Naur linux-2.6.12/arch/ia64/kernel/entry.S linux-2.6.12-XenPatch/arch/ia64/kernel/entry.S
--- linux-2.6.12/arch/ia64/kernel/entry.S	2005-06-17 13:48:29.000000000 -0600
+++ linux-2.6.12-XenPatch/arch/ia64/kernel/entry.S	2005-07-05 13:25:27.000000000 -0600
@@ -181,7 +181,7 @@
  *	called.  The code starting at .map relies on this.  The rest of the code
  *	doesn't care about the interrupt masking status.
  */
-GLOBAL_ENTRY(ia64_switch_to)
+GLOBAL_ENTRY(__ia64_switch_to)
 	.prologue
 	alloc r16=ar.pfs,1,0,0,0
 	DO_SAVE_SWITCH_STACK
@@ -235,7 +235,7 @@
 	;;
 	itr.d dtr[r25]=r23		// wire in new mapping...
 	br.cond.sptk .done
-END(ia64_switch_to)
+END(__ia64_switch_to)
 
 /*
  * Note that interrupts are enabled during save_switch_stack and load_switch_stack.  This
@@ -376,7 +376,7 @@
  *	- b7 holds address to return to
  *	- must not touch r8-r11
  */
-ENTRY(load_switch_stack)
+GLOBAL_ENTRY(load_switch_stack)
 	.prologue
 	.altrp b7
 
@@ -586,7 +586,7 @@
 	nop.i 0
 	br.call.sptk.many rp=syscall_trace_leave // give parent a chance to catch return value
 }
-.ret4:	br.cond.sptk ia64_leave_kernel
+.ret4:	br.cond.sptk __ia64_leave_kernel
 END(ia64_strace_leave_kernel)
 
 GLOBAL_ENTRY(ia64_ret_from_clone)
@@ -785,7 +785,7 @@
 END(ia64_ret_from_ia32_execve)
 	// fall through
 #endif /* CONFIG_IA32_SUPPORT */
-GLOBAL_ENTRY(ia64_leave_kernel)
+GLOBAL_ENTRY(__ia64_leave_kernel)
 	PT_REGS_UNWIND_INFO(0)
 	/*
 	 * work.need_resched etc. mustn't get changed by this CPU before it returns to
@@ -1131,7 +1131,7 @@
 	ld8 r10=[r3]
 	br.cond.sptk.many .work_processed_syscall	// re-check
 
-END(ia64_leave_kernel)
+END(__ia64_leave_kernel)
 
 ENTRY(handle_syscall_error)
 	/*
@@ -1171,7 +1171,7 @@
 	 * be set up by the caller.  We declare 8 input registers so the system call
 	 * args get preserved, in case we need to restart a system call.
 	 */
-ENTRY(notify_resume_user)
+GLOBAL_ENTRY(notify_resume_user)
 	.prologue ASM_UNW_PRLG_RP|ASM_UNW_PRLG_PFS, ASM_UNW_PRLG_GRSAVE(8)
 	alloc loc1=ar.pfs,8,2,3,0 // preserve all eight input regs in case of syscall restart!
 	mov r9=ar.unat
@@ -1259,7 +1259,7 @@
 	adds sp=16,sp
 	;;
 	ld8 r9=[sp]				// load new ar.unat
-	mov.sptk b7=r8,ia64_leave_kernel
+	mov.sptk b7=r8,__ia64_leave_kernel
 	;;
 	mov ar.unat=r9
 	br.many b7
diff -Naur linux-2.6.12/arch/ia64/kernel/head.S linux-2.6.12-XenPatch/arch/ia64/kernel/head.S
--- linux-2.6.12/arch/ia64/kernel/head.S	2005-06-17 13:48:29.000000000 -0600
+++ linux-2.6.12-XenPatch/arch/ia64/kernel/head.S	2005-07-05 13:20:53.000000000 -0600
@@ -370,6 +370,9 @@
 
 	// This is executed by the bootstrap processor (bsp) only:
 
+#ifdef CONFIG_XEN
+	br.call.sptk.many rp=early_xen_setup
+#endif
 #ifdef CONFIG_IA64_FW_EMU
 	// initialize PAL & SAL emulator:
 	br.call.sptk.many rp=sys_fw_init
diff -Naur linux-2.6.12/arch/ia64/kernel/irq_ia64.c linux-2.6.12-XenPatch/arch/ia64/kernel/irq_ia64.c
--- linux-2.6.12/arch/ia64/kernel/irq_ia64.c	2005-06-17 13:48:29.000000000 -0600
+++ linux-2.6.12-XenPatch/arch/ia64/kernel/irq_ia64.c	2005-07-05 11:42:38.000000000 -0600
@@ -151,11 +151,11 @@
 	 * to kernel stack overflows).
 	 */
 	irq_enter();
-	saved_tpr = ia64_getreg(_IA64_REG_CR_TPR);
+	saved_tpr = ia64_get_tpr();
 	ia64_srlz_d();
 	while (vector != IA64_SPURIOUS_INT_VECTOR) {
 		if (!IS_RESCHEDULE(vector)) {
-			ia64_setreg(_IA64_REG_CR_TPR, vector);
+			ia64_set_tpr(vector);
 			ia64_srlz_d();
 
 			__do_IRQ(local_vector_to_irq(vector), regs);
@@ -164,7 +164,7 @@
 			 * Disable interrupts and send EOI:
 			 */
 			local_irq_disable();
-			ia64_setreg(_IA64_REG_CR_TPR, saved_tpr);
+			ia64_set_tpr(saved_tpr);
 		}
 		ia64_eoi();
 		vector = ia64_get_ivr();
@@ -191,7 +191,7 @@
 	vector = ia64_get_ivr();
 
 	 irq_enter();
-	 saved_tpr = ia64_getreg(_IA64_REG_CR_TPR);
+	 saved_tpr = ia64_get_tpr();
 	 ia64_srlz_d();
 
 	 /*
@@ -199,7 +199,7 @@
 	  */
 	while (vector != IA64_SPURIOUS_INT_VECTOR) {
 		if (!IS_RESCHEDULE(vector)) {
-			ia64_setreg(_IA64_REG_CR_TPR, vector);
+			ia64_set_tpr(vector);
 			ia64_srlz_d();
 
 			/*
@@ -215,7 +215,7 @@
 			 * Disable interrupts and send EOI
 			 */
 			local_irq_disable();
-			ia64_setreg(_IA64_REG_CR_TPR, saved_tpr);
+			ia64_set_tpr(saved_tpr);
 		}
 		ia64_eoi();
 		vector = ia64_get_ivr();
diff -Naur linux-2.6.12/arch/ia64/kernel/pal.S linux-2.6.12-XenPatch/arch/ia64/kernel/pal.S
--- linux-2.6.12/arch/ia64/kernel/pal.S	2005-06-17 13:48:29.000000000 -0600
+++ linux-2.6.12-XenPatch/arch/ia64/kernel/pal.S	2005-07-05 13:21:33.000000000 -0600
@@ -16,6 +16,7 @@
 #include <asm/processor.h>
 
 	.data
+	.globl pal_entry_point
 pal_entry_point:
 	data8 ia64_pal_default_handler
 	.text
diff -Naur linux-2.6.12/arch/ia64/kernel/setup.c linux-2.6.12-XenPatch/arch/ia64/kernel/setup.c
--- linux-2.6.12/arch/ia64/kernel/setup.c	2005-06-17 13:48:29.000000000 -0600
+++ linux-2.6.12-XenPatch/arch/ia64/kernel/setup.c	2005-07-05 13:17:01.000000000 -0600
@@ -273,6 +273,9 @@
 static inline int __init
 early_console_setup (char *cmdline)
 {
+#ifdef CONFIG_XEN
+	early_xen_console_setup(cmdline);
+#endif
 #ifdef CONFIG_SERIAL_SGI_L1_CONSOLE
 	{
 		extern int sn_serial_console_early_setup(void);
diff -Naur linux-2.6.12/arch/ia64/Makefile linux-2.6.12-XenPatch/arch/ia64/Makefile
--- linux-2.6.12/arch/ia64/Makefile	2005-06-17 13:48:29.000000000 -0600
+++ linux-2.6.12-XenPatch/arch/ia64/Makefile	2005-07-05 13:30:58.000000000 -0600
@@ -57,6 +57,7 @@
 core-$(CONFIG_IA64_HP_ZX1)	+= arch/ia64/dig/
 core-$(CONFIG_IA64_HP_ZX1_SWIOTLB) += arch/ia64/dig/
 core-$(CONFIG_IA64_SGI_SN2)	+= arch/ia64/sn/
+core-$(CONFIG_XEN)		+= arch/ia64/xen/ arch/ia64/hp/sim/
 
 drivers-$(CONFIG_PCI)		+= arch/ia64/pci/
 drivers-$(CONFIG_IA64_HP_SIM)	+= arch/ia64/hp/sim/
diff -Naur linux-2.6.12/include/asm-ia64/delay.h linux-2.6.12-XenPatch/include/asm-ia64/delay.h
--- linux-2.6.12/include/asm-ia64/delay.h	2005-06-17 13:48:29.000000000 -0600
+++ linux-2.6.12-XenPatch/include/asm-ia64/delay.h	2005-07-05 10:56:19.000000000 -0600
@@ -20,59 +20,6 @@
 #include <asm/intrinsics.h>
 #include <asm/processor.h>
 
-static __inline__ void
-ia64_set_itm (unsigned long val)
-{
-	ia64_setreg(_IA64_REG_CR_ITM, val);
-	ia64_srlz_d();
-}
-
-static __inline__ unsigned long
-ia64_get_itm (void)
-{
-	unsigned long result;
-
-	result = ia64_getreg(_IA64_REG_CR_ITM);
-	ia64_srlz_d();
-	return result;
-}
-
-static __inline__ void
-ia64_set_itv (unsigned long val)
-{
-	ia64_setreg(_IA64_REG_CR_ITV, val);
-	ia64_srlz_d();
-}
-
-static __inline__ unsigned long
-ia64_get_itv (void)
-{
-	return ia64_getreg(_IA64_REG_CR_ITV);
-}
-
-static __inline__ void
-ia64_set_itc (unsigned long val)
-{
-	ia64_setreg(_IA64_REG_AR_ITC, val);
-	ia64_srlz_d();
-}
-
-static __inline__ unsigned long
-ia64_get_itc (void)
-{
-	unsigned long result;
-
-	result = ia64_getreg(_IA64_REG_AR_ITC);
-	ia64_barrier();
-#ifdef CONFIG_ITANIUM
-	while (unlikely((__s32) result == -1)) {
-		result = ia64_getreg(_IA64_REG_AR_ITC);
-		ia64_barrier();
-	}
-#endif
-	return result;
-}
-
 extern void ia64_delay_loop (unsigned long loops);
 
 static __inline__ void
diff -Naur linux-2.6.12/include/asm-ia64/privop.h linux-2.6.12-XenPatch/include/asm-ia64/privop.h
--- linux-2.6.12/include/asm-ia64/privop.h	1969-12-31 17:00:00.000000000 -0700
+++ linux-2.6.12-XenPatch/include/asm-ia64/privop.h	2005-07-05 13:23:31.000000000 -0600
@@ -0,0 +1,159 @@
+#ifndef _ASM_IA64_PRIVOP_H
+#define _ASM_IA64_PRIVOP_H
+
+/*
+ * Copyright (C) 2005 Hewlett-Packard Co
+ *	Dan Magenheimer <dan.magenheimer@hp.com>
+ * Reorganization from asm-ia64/processor.h
+ *
+ */
+
+#include <linux/config.h>
+
+#include <asm/intrinsics.h>
+
+#ifndef __ASSEMBLY__
+extern void ia64_getreg_unknown_kr (void);
+extern void ia64_setreg_unknown_kr (void);
+
+#define __ia64_get_kr(regnum)					\
+({								\
+	unsigned long r = 0;					\
+								\
+	switch (regnum) {					\
+	    case 0: r = ia64_getreg(_IA64_REG_AR_KR0); break;	\
+	    case 1: r = ia64_getreg(_IA64_REG_AR_KR1); break;	\
+	    case 2: r = ia64_getreg(_IA64_REG_AR_KR2); break;	\
+	    case 3: r = ia64_getreg(_IA64_REG_AR_KR3); break;	\
+	    case 4: r = ia64_getreg(_IA64_REG_AR_KR4); break;	\
+	    case 5: r = ia64_getreg(_IA64_REG_AR_KR5); break;	\
+	    case 6: r = ia64_getreg(_IA64_REG_AR_KR6); break;	\
+	    case 7: r = ia64_getreg(_IA64_REG_AR_KR7); break;	\
+	    default: ia64_getreg_unknown_kr(); break;		\
+	}							\
+	r;							\
+})
+
+#define __ia64_set_kr(regnum, r) 					\
+({								\
+	switch (regnum) {					\
+	    case 0: ia64_setreg(_IA64_REG_AR_KR0, r); break;	\
+	    case 1: ia64_setreg(_IA64_REG_AR_KR1, r); break;	\
+	    case 2: ia64_setreg(_IA64_REG_AR_KR2, r); break;	\
+	    case 3: ia64_setreg(_IA64_REG_AR_KR3, r); break;	\
+	    case 4: ia64_setreg(_IA64_REG_AR_KR4, r); break;	\
+	    case 5: ia64_setreg(_IA64_REG_AR_KR5, r); break;	\
+	    case 6: ia64_setreg(_IA64_REG_AR_KR6, r); break;	\
+	    case 7: ia64_setreg(_IA64_REG_AR_KR7, r); break;	\
+	    default: ia64_setreg_unknown_kr(); break;		\
+	}							\
+})
+
+static inline unsigned long
+__ia64_get_ivr (void)
+{
+	unsigned long r;
+	ia64_srlz_d();
+	r = ia64_getreg(_IA64_REG_CR_IVR);
+	ia64_srlz_d();
+	return r;
+}
+
+static inline void
+__ia64_eoi (void)
+{
+	ia64_setreg(_IA64_REG_CR_EOI, 0);
+	ia64_srlz_d();
+}
+
+#define __cpu_relax()	ia64_hint(ia64_hint_pause)
+
+#define __ia64_get_tpr(vector)	ia64_getreg(_IA64_REG_CR_TPR)
+#define __ia64_set_tpr(vector)	ia64_setreg(_IA64_REG_CR_TPR,vector)
+
+#define __ia64_get_eflag()	ia64_getreg(_IA64_REG_AR_EFLAG)
+#define __ia64_set_eflag(eflag)	ia64_setreg(_IA64_REG_AR_EFLAG,eflag)
+
+#define	__ia64_get_psr_i()	ia64_getreg(_IA64_REG_PSR)
+#define __ia64_rsm_i()		ia64_rsm(IA64_PSR_I)
+#define __ia64_ssm_i()		ia64_ssm(IA64_PSR_I)
+#define	__ia64_local_irq_restore(x)	ia64_intrin_local_irq_restore(x)
+
+static __inline__ void
+__ia64_set_itm (unsigned long val)
+{
+	ia64_setreg(_IA64_REG_CR_ITM, val);
+	ia64_srlz_d();
+}
+
+static __inline__ unsigned long
+ia64_get_itm (void)
+{
+	unsigned long result;
+
+	result = ia64_getreg(_IA64_REG_CR_ITM);
+	ia64_srlz_d();
+	return result;
+}
+
+static __inline__ void
+ia64_set_itv (unsigned long val)
+{
+	ia64_setreg(_IA64_REG_CR_ITV, val);
+	ia64_srlz_d();
+}
+
+static __inline__ unsigned long
+ia64_get_itv (void)
+{
+	return ia64_getreg(_IA64_REG_CR_ITV);
+}
+
+static __inline__ void
+ia64_set_itc (unsigned long val)
+{
+	ia64_setreg(_IA64_REG_AR_ITC, val);
+	ia64_srlz_d();
+}
+
+static __inline__ unsigned long
+ia64_get_itc (void)
+{
+	unsigned long result;
+
+	result = ia64_getreg(_IA64_REG_AR_ITC);
+	ia64_barrier();
+#ifdef CONFIG_ITANIUM
+	while (unlikely((__s32) result == -1)) {
+		result = ia64_getreg(_IA64_REG_AR_ITC);
+		ia64_barrier();
+	}
+#endif
+	return result;
+}
+#endif /* !__ASSEMBLY__ */
+
+#ifdef CONFIG_XEN
+#include <asm/xen/privop.h>
+#else
+#define ia64_get_kr(regnum)		__ia64_get_kr(regnum)
+#define ia64_set_kr(regnum, r)		__ia64_set_kr(regnum,r)
+#define ia64_get_ivr			__ia64_get_ivr
+#define ia64_eoi			__ia64_eoi
+#define ia64_set_itm(val)		__ia64_set_itm(val)
+#define cpu_relax			__cpu_relax
+#define ia64_get_tpr			__ia64_get_tpr
+#define ia64_set_tpr(vector)		__ia64_set_tpr(vector)
+#define ia64_get_eflag			__ia64_get_eflag
+#define ia64_set_eflag(eflag)		__ia64_set_eflag(eflag)
+#define ia64_pal_halt_light		__ia64_pal_halt_light
+#define	ia64_get_psr_i			__ia64_get_psr_i
+#define ia64_rsm_i			__ia64_rsm_i
+#define ia64_ssm_i			__ia64_ssm_i
+#define	ia64_local_irq_restore(x)	__ia64_local_irq_restore(x)
+#define	ia64_leave_kernel		__ia64_leave_kernel
+#define	ia64_switch_to			__ia64_switch_to
+#define	ia64_pal_call_static		__ia64_pal_call_static
+#endif
+
+#endif /* _ASM_IA64_PRIVOP_H */
diff -Naur linux-2.6.12/include/asm-ia64/processor.h linux-2.6.12-XenPatch/include/asm-ia64/processor.h
--- linux-2.6.12/include/asm-ia64/processor.h	2005-06-17 13:48:29.000000000 -0600
+++ linux-2.6.12-XenPatch/include/asm-ia64/processor.h	2005-07-05 11:51:07.000000000 -0600
@@ -19,6 +19,7 @@
 #include <asm/kregs.h>
 #include <asm/ptrace.h>
 #include <asm/ustack.h>
+#include <asm/privop.h>
 
 /* Our arch specific arch_init_sched_domain is in arch/ia64/kernel/domain.c */
 #define ARCH_HAS_SCHED_DOMAIN
@@ -362,42 +363,6 @@
 /* Return stack pointer of blocked task TSK.  */
 #define KSTK_ESP(tsk)  ((tsk)->thread.ksp)
 
-extern void ia64_getreg_unknown_kr (void);
-extern void ia64_setreg_unknown_kr (void);
-
-#define ia64_get_kr(regnum)					\
-({								\
-	unsigned long r = 0;					\
-								\
-	switch (regnum) {					\
-	    case 0: r = ia64_getreg(_IA64_REG_AR_KR0); break;	\
-	    case 1: r = ia64_getreg(_IA64_REG_AR_KR1); break;	\
-	    case 2: r = ia64_getreg(_IA64_REG_AR_KR2); break;	\
-	    case 3: r = ia64_getreg(_IA64_REG_AR_KR3); break;	\
-	    case 4: r = ia64_getreg(_IA64_REG_AR_KR4); break;	\
-	    case 5: r = ia64_getreg(_IA64_REG_AR_KR5); break;	\
-	    case 6: r = ia64_getreg(_IA64_REG_AR_KR6); break;	\
-	    case 7: r = ia64_getreg(_IA64_REG_AR_KR7); break;	\
-	    default: ia64_getreg_unknown_kr(); break;		\
-	}							\
-	r;							\
-})
-
-#define ia64_set_kr(regnum, r) 					\
-({								\
-	switch (regnum) {					\
-	    case 0: ia64_setreg(_IA64_REG_AR_KR0, r); break;	\
-	    case 1: ia64_setreg(_IA64_REG_AR_KR1, r); break;	\
-	    case 2: ia64_setreg(_IA64_REG_AR_KR2, r); break;	\
-	    case 3: ia64_setreg(_IA64_REG_AR_KR3, r); break;	\
-	    case 4: ia64_setreg(_IA64_REG_AR_KR4, r); break;	\
-	    case 5: ia64_setreg(_IA64_REG_AR_KR5, r); break;	\
-	    case 6: ia64_setreg(_IA64_REG_AR_KR6, r); break;	\
-	    case 7: ia64_setreg(_IA64_REG_AR_KR7, r); break;	\
-	    default: ia64_setreg_unknown_kr(); break;		\
-	}							\
-})
-
 /*
  * The following three macros can't be inline functions because we don't have struct
  * task_struct at this point.
@@ -554,15 +519,6 @@
 }
 
 static inline void
-ia64_eoi (void)
-{
-	ia64_setreg(_IA64_REG_CR_EOI, 0);
-	ia64_srlz_d();
-}
-
-#define cpu_relax()	ia64_hint(ia64_hint_pause)
-
-static inline void
 ia64_set_lrr0 (unsigned long val)
 {
 	ia64_setreg(_IA64_REG_CR_LRR0, val);
@@ -623,16 +579,6 @@
 #define current_text_addr() \
 	({ void *_pc; _pc = (void *)ia64_getreg(_IA64_REG_IP); _pc; })
 
-static inline __u64
-ia64_get_ivr (void)
-{
-	__u64 r;
-	ia64_srlz_d();
-	r = ia64_getreg(_IA64_REG_CR_IVR);
-	ia64_srlz_d();
-	return r;
-}
-
 static inline void
 ia64_set_dbr (__u64 regnum, __u64 value)
 {
diff -Naur linux-2.6.12/include/asm-ia64/system.h linux-2.6.12-XenPatch/include/asm-ia64/system.h
--- linux-2.6.12/include/asm-ia64/system.h	2005-06-17 13:48:29.000000000 -0600
+++ linux-2.6.12-XenPatch/include/asm-ia64/system.h	2005-07-05 12:44:19.000000000 -0600
@@ -124,18 +124,18 @@
 #define __local_irq_save(x)			\
 do {						\
 	ia64_stop();				\
-	(x) = ia64_getreg(_IA64_REG_PSR);	\
+	(x) = ia64_get_psr_i();			\
 	ia64_stop();				\
-	ia64_rsm(IA64_PSR_I);			\
+	ia64_rsm_i();				\
 } while (0)
 
 #define __local_irq_disable()			\
 do {						\
 	ia64_stop();				\
-	ia64_rsm(IA64_PSR_I);			\
+	ia64_rsm_i();				\
 } while (0)
 
-#define __local_irq_restore(x)	ia64_intrin_local_irq_restore((x) & IA64_PSR_I)
+#define __local_irq_restore(x)	ia64_local_irq_restore((x) & IA64_PSR_I)
 
 #ifdef CONFIG_IA64_DEBUG_IRQ
 
@@ -171,8 +171,8 @@
 # define local_irq_restore(x)	__local_irq_restore(x)
 #endif /* !CONFIG_IA64_DEBUG_IRQ */
 
-#define local_irq_enable()	({ ia64_stop(); ia64_ssm(IA64_PSR_I); ia64_srlz_d(); })
-#define local_save_flags(flags)	({ ia64_stop(); (flags) = ia64_getreg(_IA64_REG_PSR); })
+#define local_irq_enable()	({ ia64_stop(); ia64_ssm_i(); ia64_srlz_d(); })
+#define local_save_flags(flags)	({ ia64_stop(); (flags) = ia64_get_psr_i(); })
 
 #define irqs_disabled()				\
 ({						\

^ permalink raw reply	[flat|nested] 21+ messages in thread

* RE: Xen and the Art of Linux/ia64 Virtualization
  2005-06-29 20:16 Xen and the Art of Linux/ia64 Virtualization Magenheimer, Dan (HP Labs Fort Collins)
                   ` (3 preceding siblings ...)
  2005-07-05 20:20 ` Magenheimer, Dan (HP Labs Fort Collins)
@ 2005-07-06 20:46 ` Magenheimer, Dan (HP Labs Fort Collins)
  2005-07-07 22:18 ` Gerald Pfeifer
                   ` (14 subsequent siblings)
  19 siblings, 0 replies; 21+ messages in thread
From: Magenheimer, Dan (HP Labs Fort Collins) @ 2005-07-06 20:46 UTC (permalink / raw)
  To: linux-ia64

[-- Attachment #1: Type: text/plain, Size: 2357 bytes --]

The patch I posted yesterday had a couple of bugs.
This version applies cleanly against 2.6.12 and
(when augmented with additional files from various
Xen subdirectories) has been booted both on Xen/ia64
and on hardware.

Note that a patch to drivers/acpi/motherboard.c (that
allows for acpi to be enabled on a stubbed acpi tree
without a kernel null pointer dereference!) is required
and has been submitted separately to the linux-acpi list.

With the exception of three short CONFIG_XEN
ifdefs, the vast majority of changes in this patch
are code rearrangement to enable a number of routines
and defines to add one level of abstraction.

Comments and feedback would be much appreciated!

 arch/ia64/Kconfig             |    7 +
 arch/ia64/Makefile            |    1 
 arch/ia64/hp/sim/Makefile     |    2 
 arch/ia64/ia32/elfcore32.h    |    2 
 arch/ia64/ia32/ia32_signal.c  |    6 -
 arch/ia64/ia32/ia32_support.c |    4 -
 arch/ia64/kernel/entry.S      |   30 ++++---
 arch/ia64/kernel/head.S       |    4 +
 arch/ia64/kernel/irq_ia64.c   |   12 +--
 arch/ia64/kernel/pal.S        |    5 -
 arch/ia64/kernel/setup.c      |    3 
 include/asm-ia64/delay.h      |   53 -------------
 include/asm-ia64/privop.h     |  160
++++++++++++++++++++++++++++++++++++++++++
 include/asm-ia64/processor.h  |   56 --------------
 include/asm-ia64/system.h     |   12 +--
 15 files changed, 217 insertions(+), 140 deletions(-)

> -----Original Message-----
> From: Magenheimer, Dan (HP Labs Fort Collins) 
> Sent: Tuesday, July 05, 2005 2:21 PM
> To: linux-ia64@vger.kernel.org
> Subject: RE: Xen and the Art of Linux/ia64 Virtualization
> 
> Thanks to excellent feedback from David Mosberger and
> Christophe Hellwig, I have greatly cleaned up the attached
> patch by using some very nice abstractions.  In fact,
> most of the bulk in the attached patch results from
> moving some code from asm-ia64/{delay,processor}.h to
> a new file, asm-ia64/privop.h.  And the number of
> CONFIG_XEN ifdefs is dramatically reduced.
> 
> More feedback appreciated (including comments about how
> close this might be to be ready for submission to Tony).
> 
> Thanks,
> Dan Magenheimer
> 
> P.S. Anybody who is attending OLS who wants to talk about
> Xen (and specifically Xen/ia64)?  I'll be there... send
> me email.
> 

[-- Attachment #2: xenia64.patch --]
[-- Type: application/octet-stream, Size: 20918 bytes --]

diff -Naur linux-2.6.12/arch/ia64/hp/sim/Makefile linux-2.6.12-XenPatch/arch/ia64/hp/sim/Makefile
--- linux-2.6.12/arch/ia64/hp/sim/Makefile	2005-06-17 13:48:29.000000000 -0600
+++ linux-2.6.12-XenPatch/arch/ia64/hp/sim/Makefile	2005-07-05 13:27:34.000000000 -0600
@@ -14,3 +14,5 @@
 obj-$(CONFIG_HP_SIMSERIAL) += simserial.o
 obj-$(CONFIG_HP_SIMSERIAL_CONSOLE) += hpsim_console.o
 obj-$(CONFIG_HP_SIMSCSI) += simscsi.o
+obj-$(CONFIG_XEN) += simserial.o
+obj-$(CONFIG_XEN) += hpsim_console.o
diff -Naur linux-2.6.12/arch/ia64/ia32/elfcore32.h linux-2.6.12-XenPatch/arch/ia64/ia32/elfcore32.h
--- linux-2.6.12/arch/ia64/ia32/elfcore32.h	2005-06-17 13:48:29.000000000 -0600
+++ linux-2.6.12-XenPatch/arch/ia64/ia32/elfcore32.h	2005-07-06 13:49:54.000000000 -0600
@@ -82,7 +82,7 @@
 	pr_reg[11] = regs->r1; 				\
 	pr_reg[12] = regs->cr_iip;			\
 	pr_reg[13] = regs->r17 & 0xffff;		\
-	pr_reg[14] = ia64_getreg(_IA64_REG_AR_EFLAG);	\
+	pr_reg[14] = ia64_get_eflag();			\
 	pr_reg[15] = regs->r12;				\
 	pr_reg[16] = (regs->r17 >> 16) & 0xffff;
 
diff -Naur linux-2.6.12/arch/ia64/ia32/ia32_signal.c linux-2.6.12-XenPatch/arch/ia64/ia32/ia32_signal.c
--- linux-2.6.12/arch/ia64/ia32/ia32_signal.c	2005-06-17 13:48:29.000000000 -0600
+++ linux-2.6.12-XenPatch/arch/ia64/ia32/ia32_signal.c	2005-07-06 13:47:51.000000000 -0600
@@ -666,7 +666,7 @@
 	/*
 	 *  `eflags' is in an ar register for this context
 	 */
-	flag = ia64_getreg(_IA64_REG_AR_EFLAG);
+	flag = ia64_get_eflag();
 	err |= __put_user((unsigned int)flag, &sc->eflags);
 	err |= __put_user(regs->r12, &sc->esp_at_signal);
 	err |= __put_user((regs->r17 >> 16) & 0xffff, (unsigned int __user *)&sc->ss);
@@ -754,10 +754,10 @@
 		 *  IA32 process's context.
 		 */
 		err |= __get_user(tmpflags, &sc->eflags);
-		flag = ia64_getreg(_IA64_REG_AR_EFLAG);
+		flag = ia64_get_eflag();
 		flag &= ~0x40DD5;
 		flag |= (tmpflags & 0x40DD5);
-		ia64_setreg(_IA64_REG_AR_EFLAG, flag);
+		ia64_set_eflag(flag);
 
 		regs->r1 = -1;	/* disable syscall checks, r1 is orig_eax */
 	}
diff -Naur linux-2.6.12/arch/ia64/ia32/ia32_support.c linux-2.6.12-XenPatch/arch/ia64/ia32/ia32_support.c
--- linux-2.6.12/arch/ia64/ia32/ia32_support.c	2005-06-17 13:48:29.000000000 -0600
+++ linux-2.6.12-XenPatch/arch/ia64/ia32/ia32_support.c	2005-07-05 11:50:03.000000000 -0600
@@ -100,7 +100,7 @@
 void
 ia32_save_state (struct task_struct *t)
 {
-	t->thread.eflag = ia64_getreg(_IA64_REG_AR_EFLAG);
+	t->thread.eflag = ia64_get_eflag();
 	t->thread.fsr   = ia64_getreg(_IA64_REG_AR_FSR);
 	t->thread.fcr   = ia64_getreg(_IA64_REG_AR_FCR);
 	t->thread.fir   = ia64_getreg(_IA64_REG_AR_FIR);
@@ -122,7 +122,7 @@
 	fdr = t->thread.fdr;
 	tssd = load_desc(_TSS);					/* TSSD */
 
-	ia64_setreg(_IA64_REG_AR_EFLAG, eflag);
+	ia64_set_eflag(eflag);
 	ia64_setreg(_IA64_REG_AR_FSR, fsr);
 	ia64_setreg(_IA64_REG_AR_FCR, fcr);
 	ia64_setreg(_IA64_REG_AR_FIR, fir);
diff -Naur linux-2.6.12/arch/ia64/Kconfig linux-2.6.12-XenPatch/arch/ia64/Kconfig
--- linux-2.6.12/arch/ia64/Kconfig	2005-06-17 13:48:29.000000000 -0600
+++ linux-2.6.12-XenPatch/arch/ia64/Kconfig	2005-07-06 10:50:20.000000000 -0600
@@ -46,6 +46,13 @@
 	bool
 	default y
 
+config XEN
+	bool
+	default n
+	help
+	  Enable Xen hypervisor support.  Resulting kernel runs
+	  both as a guest OS on Xen and natively on hardware.
+
 config SCHED_NO_NO_OMIT_FRAME_POINTER
 	bool
 	default y
diff -Naur linux-2.6.12/arch/ia64/kernel/entry.S linux-2.6.12-XenPatch/arch/ia64/kernel/entry.S
--- linux-2.6.12/arch/ia64/kernel/entry.S	2005-06-17 13:48:29.000000000 -0600
+++ linux-2.6.12-XenPatch/arch/ia64/kernel/entry.S	2005-07-06 14:21:07.000000000 -0600
@@ -181,7 +181,7 @@
  *	called.  The code starting at .map relies on this.  The rest of the code
  *	doesn't care about the interrupt masking status.
  */
-GLOBAL_ENTRY(ia64_switch_to)
+GLOBAL_ENTRY(__ia64_switch_to)
 	.prologue
 	alloc r16=ar.pfs,1,0,0,0
 	DO_SAVE_SWITCH_STACK
@@ -235,7 +235,7 @@
 	;;
 	itr.d dtr[r25]=r23		// wire in new mapping...
 	br.cond.sptk .done
-END(ia64_switch_to)
+END(__ia64_switch_to)
 
 /*
  * Note that interrupts are enabled during save_switch_stack and load_switch_stack.  This
@@ -376,7 +376,7 @@
  *	- b7 holds address to return to
  *	- must not touch r8-r11
  */
-ENTRY(load_switch_stack)
+GLOBAL_ENTRY(load_switch_stack)
 	.prologue
 	.altrp b7
 
@@ -586,7 +586,7 @@
 	nop.i 0
 	br.call.sptk.many rp=syscall_trace_leave // give parent a chance to catch return value
 }
-.ret4:	br.cond.sptk ia64_leave_kernel
+.ret4:	br.cond.sptk __ia64_leave_kernel
 END(ia64_strace_leave_kernel)
 
 GLOBAL_ENTRY(ia64_ret_from_clone)
@@ -623,8 +623,11 @@
 	adds r2=PT(R8)+16,sp			// r2 = &pt_regs.r8
 	mov r10=r0				// clear error indication in r10
 (p7)	br.cond.spnt handle_syscall_error	// handle potential syscall failure
+	;;
+	// don't fall through, ia64_leave_syscall may be #define'd
+	br.cond.sptk.few ia64_leave_syscall
+	;;
 END(ia64_ret_from_syscall)
-	// fall through
 /*
  * ia64_leave_syscall(): Same as ia64_leave_kernel, except that it doesn't
  *	need to switch to bank 0 and doesn't restore the scratch registers.
@@ -669,7 +672,7 @@
  *	      ar.csd: cleared
  *	      ar.ssd: cleared
  */
-ENTRY(ia64_leave_syscall)
+GLOBAL_ENTRY(__ia64_leave_syscall)
 	PT_REGS_UNWIND_INFO(0)
 	/*
 	 * work.need_resched etc. mustn't get changed by this CPU before it returns to
@@ -770,7 +773,7 @@
 	mov.m ar.ccv=r0		// clear ar.ccv
 (pNonSys) br.cond.dpnt.many dont_preserve_current_frame
 	br.cond.sptk.many rbs_switch
-END(ia64_leave_syscall)
+END(__ia64_leave_syscall)
 
 #ifdef CONFIG_IA32_SUPPORT
 GLOBAL_ENTRY(ia64_ret_from_ia32_execve)
@@ -782,10 +785,13 @@
 	st8.spill [r2]=r8	// store return value in slot for r8 and set unat bit
 	.mem.offset 8,0
 	st8.spill [r3]=r0	// clear error indication in slot for r10 and set unat bit
+	;;
+	// don't fall through, ia64_leave_kernel may be #define'd
+	br.cond.sptk.few ia64_leave_kernel
+	;;
 END(ia64_ret_from_ia32_execve)
-	// fall through
 #endif /* CONFIG_IA32_SUPPORT */
-GLOBAL_ENTRY(ia64_leave_kernel)
+GLOBAL_ENTRY(__ia64_leave_kernel)
 	PT_REGS_UNWIND_INFO(0)
 	/*
 	 * work.need_resched etc. mustn't get changed by this CPU before it returns to
@@ -1131,7 +1137,7 @@
 	ld8 r10=[r3]
 	br.cond.sptk.many .work_processed_syscall	// re-check
 
-END(ia64_leave_kernel)
+END(__ia64_leave_kernel)
 
 ENTRY(handle_syscall_error)
 	/*
@@ -1171,7 +1177,7 @@
 	 * be set up by the caller.  We declare 8 input registers so the system call
 	 * args get preserved, in case we need to restart a system call.
 	 */
-ENTRY(notify_resume_user)
+GLOBAL_ENTRY(notify_resume_user)
 	.prologue ASM_UNW_PRLG_RP|ASM_UNW_PRLG_PFS, ASM_UNW_PRLG_GRSAVE(8)
 	alloc loc1=ar.pfs,8,2,3,0 // preserve all eight input regs in case of syscall restart!
 	mov r9=ar.unat
@@ -1259,7 +1265,7 @@
 	adds sp=16,sp
 	;;
 	ld8 r9=[sp]				// load new ar.unat
-	mov.sptk b7=r8,ia64_leave_kernel
+	mov.sptk b7=r8,__ia64_leave_kernel
 	;;
 	mov ar.unat=r9
 	br.many b7
diff -Naur linux-2.6.12/arch/ia64/kernel/head.S linux-2.6.12-XenPatch/arch/ia64/kernel/head.S
--- linux-2.6.12/arch/ia64/kernel/head.S	2005-06-17 13:48:29.000000000 -0600
+++ linux-2.6.12-XenPatch/arch/ia64/kernel/head.S	2005-07-06 11:28:16.000000000 -0600
@@ -370,6 +370,10 @@
 
 	// This is executed by the bootstrap processor (bsp) only:
 
+#ifdef CONFIG_XEN
+	br.call.sptk.many rp=early_xen_setup
+	;;
+#endif
 #ifdef CONFIG_IA64_FW_EMU
 	// initialize PAL & SAL emulator:
 	br.call.sptk.many rp=sys_fw_init
diff -Naur linux-2.6.12/arch/ia64/kernel/irq_ia64.c linux-2.6.12-XenPatch/arch/ia64/kernel/irq_ia64.c
--- linux-2.6.12/arch/ia64/kernel/irq_ia64.c	2005-06-17 13:48:29.000000000 -0600
+++ linux-2.6.12-XenPatch/arch/ia64/kernel/irq_ia64.c	2005-07-05 11:42:38.000000000 -0600
@@ -151,11 +151,11 @@
 	 * to kernel stack overflows).
 	 */
 	irq_enter();
-	saved_tpr = ia64_getreg(_IA64_REG_CR_TPR);
+	saved_tpr = ia64_get_tpr();
 	ia64_srlz_d();
 	while (vector != IA64_SPURIOUS_INT_VECTOR) {
 		if (!IS_RESCHEDULE(vector)) {
-			ia64_setreg(_IA64_REG_CR_TPR, vector);
+			ia64_set_tpr(vector);
 			ia64_srlz_d();
 
 			__do_IRQ(local_vector_to_irq(vector), regs);
@@ -164,7 +164,7 @@
 			 * Disable interrupts and send EOI:
 			 */
 			local_irq_disable();
-			ia64_setreg(_IA64_REG_CR_TPR, saved_tpr);
+			ia64_set_tpr(saved_tpr);
 		}
 		ia64_eoi();
 		vector = ia64_get_ivr();
@@ -191,7 +191,7 @@
 	vector = ia64_get_ivr();
 
 	 irq_enter();
-	 saved_tpr = ia64_getreg(_IA64_REG_CR_TPR);
+	 saved_tpr = ia64_get_tpr();
 	 ia64_srlz_d();
 
 	 /*
@@ -199,7 +199,7 @@
 	  */
 	while (vector != IA64_SPURIOUS_INT_VECTOR) {
 		if (!IS_RESCHEDULE(vector)) {
-			ia64_setreg(_IA64_REG_CR_TPR, vector);
+			ia64_set_tpr(vector);
 			ia64_srlz_d();
 
 			/*
@@ -215,7 +215,7 @@
 			 * Disable interrupts and send EOI
 			 */
 			local_irq_disable();
-			ia64_setreg(_IA64_REG_CR_TPR, saved_tpr);
+			ia64_set_tpr(saved_tpr);
 		}
 		ia64_eoi();
 		vector = ia64_get_ivr();
diff -Naur linux-2.6.12/arch/ia64/kernel/pal.S linux-2.6.12-XenPatch/arch/ia64/kernel/pal.S
--- linux-2.6.12/arch/ia64/kernel/pal.S	2005-06-17 13:48:29.000000000 -0600
+++ linux-2.6.12-XenPatch/arch/ia64/kernel/pal.S	2005-07-06 10:44:15.000000000 -0600
@@ -16,6 +16,7 @@
 #include <asm/processor.h>
 
 	.data
+	.globl pal_entry_point
 pal_entry_point:
 	data8 ia64_pal_default_handler
 	.text
@@ -53,7 +54,7 @@
  * in4	       1 ==> clear psr.ic,  0 ==> don't clear psr.ic
  *
  */
-GLOBAL_ENTRY(ia64_pal_call_static)
+GLOBAL_ENTRY(__ia64_pal_call_static)
 	.prologue ASM_UNW_PRLG_RP|ASM_UNW_PRLG_PFS, ASM_UNW_PRLG_GRSAVE(5)
 	alloc loc1 = ar.pfs,5,5,0,0
 	movl loc2 = pal_entry_point
@@ -90,7 +91,7 @@
 	;;
 	srlz.d				// seralize restoration of psr.l
 	br.ret.sptk.many b0
-END(ia64_pal_call_static)
+END(__ia64_pal_call_static)
 
 /*
  * Make a PAL call using the stacked registers calling convention.
diff -Naur linux-2.6.12/arch/ia64/kernel/setup.c linux-2.6.12-XenPatch/arch/ia64/kernel/setup.c
--- linux-2.6.12/arch/ia64/kernel/setup.c	2005-06-17 13:48:29.000000000 -0600
+++ linux-2.6.12-XenPatch/arch/ia64/kernel/setup.c	2005-07-06 10:43:15.000000000 -0600
@@ -273,6 +273,9 @@
 static inline int __init
 early_console_setup (char *cmdline)
 {
+#ifdef CONFIG_XEN
+	if (!early_xen_console_setup(cmdline)) return 0;
+#endif
 #ifdef CONFIG_SERIAL_SGI_L1_CONSOLE
 	{
 		extern int sn_serial_console_early_setup(void);
diff -Naur linux-2.6.12/arch/ia64/Makefile linux-2.6.12-XenPatch/arch/ia64/Makefile
--- linux-2.6.12/arch/ia64/Makefile	2005-06-17 13:48:29.000000000 -0600
+++ linux-2.6.12-XenPatch/arch/ia64/Makefile	2005-07-05 13:30:58.000000000 -0600
@@ -57,6 +57,7 @@
 core-$(CONFIG_IA64_HP_ZX1)	+= arch/ia64/dig/
 core-$(CONFIG_IA64_HP_ZX1_SWIOTLB) += arch/ia64/dig/
 core-$(CONFIG_IA64_SGI_SN2)	+= arch/ia64/sn/
+core-$(CONFIG_XEN)		+= arch/ia64/xen/ arch/ia64/hp/sim/
 
 drivers-$(CONFIG_PCI)		+= arch/ia64/pci/
 drivers-$(CONFIG_IA64_HP_SIM)	+= arch/ia64/hp/sim/
diff -Naur linux-2.6.12/include/asm-ia64/delay.h linux-2.6.12-XenPatch/include/asm-ia64/delay.h
--- linux-2.6.12/include/asm-ia64/delay.h	2005-06-17 13:48:29.000000000 -0600
+++ linux-2.6.12-XenPatch/include/asm-ia64/delay.h	2005-07-05 10:56:19.000000000 -0600
@@ -20,59 +20,6 @@
 #include <asm/intrinsics.h>
 #include <asm/processor.h>
 
-static __inline__ void
-ia64_set_itm (unsigned long val)
-{
-	ia64_setreg(_IA64_REG_CR_ITM, val);
-	ia64_srlz_d();
-}
-
-static __inline__ unsigned long
-ia64_get_itm (void)
-{
-	unsigned long result;
-
-	result = ia64_getreg(_IA64_REG_CR_ITM);
-	ia64_srlz_d();
-	return result;
-}
-
-static __inline__ void
-ia64_set_itv (unsigned long val)
-{
-	ia64_setreg(_IA64_REG_CR_ITV, val);
-	ia64_srlz_d();
-}
-
-static __inline__ unsigned long
-ia64_get_itv (void)
-{
-	return ia64_getreg(_IA64_REG_CR_ITV);
-}
-
-static __inline__ void
-ia64_set_itc (unsigned long val)
-{
-	ia64_setreg(_IA64_REG_AR_ITC, val);
-	ia64_srlz_d();
-}
-
-static __inline__ unsigned long
-ia64_get_itc (void)
-{
-	unsigned long result;
-
-	result = ia64_getreg(_IA64_REG_AR_ITC);
-	ia64_barrier();
-#ifdef CONFIG_ITANIUM
-	while (unlikely((__s32) result == -1)) {
-		result = ia64_getreg(_IA64_REG_AR_ITC);
-		ia64_barrier();
-	}
-#endif
-	return result;
-}
-
 extern void ia64_delay_loop (unsigned long loops);
 
 static __inline__ void
diff -Naur linux-2.6.12/include/asm-ia64/privop.h linux-2.6.12-XenPatch/include/asm-ia64/privop.h
--- linux-2.6.12/include/asm-ia64/privop.h	1969-12-31 17:00:00.000000000 -0700
+++ linux-2.6.12-XenPatch/include/asm-ia64/privop.h	2005-07-06 10:42:20.000000000 -0600
@@ -0,0 +1,160 @@
+#ifndef _ASM_IA64_PRIVOP_H
+#define _ASM_IA64_PRIVOP_H
+
+/*
+ * Copyright (C) 2005 Hewlett-Packard Co
+ *	Dan Magenheimer <dan.magenheimer@hp.com>
+ * Reorganization from asm-ia64/processor.h
+ *
+ */
+
+#include <linux/config.h>
+
+#include <asm/intrinsics.h>
+
+#ifndef __ASSEMBLY__
+extern void ia64_getreg_unknown_kr (void);
+extern void ia64_setreg_unknown_kr (void);
+
+#define __ia64_get_kr(regnum)					\
+({								\
+	unsigned long r = 0;					\
+								\
+	switch (regnum) {					\
+	    case 0: r = ia64_getreg(_IA64_REG_AR_KR0); break;	\
+	    case 1: r = ia64_getreg(_IA64_REG_AR_KR1); break;	\
+	    case 2: r = ia64_getreg(_IA64_REG_AR_KR2); break;	\
+	    case 3: r = ia64_getreg(_IA64_REG_AR_KR3); break;	\
+	    case 4: r = ia64_getreg(_IA64_REG_AR_KR4); break;	\
+	    case 5: r = ia64_getreg(_IA64_REG_AR_KR5); break;	\
+	    case 6: r = ia64_getreg(_IA64_REG_AR_KR6); break;	\
+	    case 7: r = ia64_getreg(_IA64_REG_AR_KR7); break;	\
+	    default: ia64_getreg_unknown_kr(); break;		\
+	}							\
+	r;							\
+})
+
+#define __ia64_set_kr(regnum,r)					\
+({								\
+	switch (regnum) {					\
+	    case 0: ia64_setreg(_IA64_REG_AR_KR0, r); break;	\
+	    case 1: ia64_setreg(_IA64_REG_AR_KR1, r); break;	\
+	    case 2: ia64_setreg(_IA64_REG_AR_KR2, r); break;	\
+	    case 3: ia64_setreg(_IA64_REG_AR_KR3, r); break;	\
+	    case 4: ia64_setreg(_IA64_REG_AR_KR4, r); break;	\
+	    case 5: ia64_setreg(_IA64_REG_AR_KR5, r); break;	\
+	    case 6: ia64_setreg(_IA64_REG_AR_KR6, r); break;	\
+	    case 7: ia64_setreg(_IA64_REG_AR_KR7, r); break;	\
+	    default: ia64_setreg_unknown_kr(); break;		\
+	}							\
+})
+
+static inline unsigned long
+__ia64_get_ivr (void)
+{
+	unsigned long r;
+	ia64_srlz_d();
+	r = ia64_getreg(_IA64_REG_CR_IVR);
+	ia64_srlz_d();
+	return r;
+}
+
+static inline void
+__ia64_eoi (void)
+{
+	ia64_setreg(_IA64_REG_CR_EOI, 0);
+	ia64_srlz_d();
+}
+
+#define __cpu_relax()	ia64_hint(ia64_hint_pause)
+
+#define __ia64_get_tpr(vector)	ia64_getreg(_IA64_REG_CR_TPR)
+#define __ia64_set_tpr(vector)	ia64_setreg(_IA64_REG_CR_TPR,vector)
+
+#define __ia64_get_eflag()	ia64_getreg(_IA64_REG_AR_EFLAG)
+#define __ia64_set_eflag(eflag)	ia64_setreg(_IA64_REG_AR_EFLAG,eflag)
+
+#define	__ia64_get_psr_i()	ia64_getreg(_IA64_REG_PSR)
+#define __ia64_rsm_i()		ia64_rsm(IA64_PSR_I)
+#define __ia64_ssm_i()		ia64_ssm(IA64_PSR_I)
+#define	__ia64_local_irq_restore(x)	ia64_intrin_local_irq_restore(x)
+
+static __inline__ void
+__ia64_set_itm (unsigned long val)
+{
+	ia64_setreg(_IA64_REG_CR_ITM, val);
+	ia64_srlz_d();
+}
+
+static __inline__ unsigned long
+ia64_get_itm (void)
+{
+	unsigned long result;
+
+	result = ia64_getreg(_IA64_REG_CR_ITM);
+	ia64_srlz_d();
+	return result;
+}
+
+static __inline__ void
+ia64_set_itv (unsigned long val)
+{
+	ia64_setreg(_IA64_REG_CR_ITV, val);
+	ia64_srlz_d();
+}
+
+static __inline__ unsigned long
+ia64_get_itv (void)
+{
+	return ia64_getreg(_IA64_REG_CR_ITV);
+}
+
+static __inline__ void
+ia64_set_itc (unsigned long val)
+{
+	ia64_setreg(_IA64_REG_AR_ITC, val);
+	ia64_srlz_d();
+}
+
+static __inline__ unsigned long
+ia64_get_itc (void)
+{
+	unsigned long result;
+
+	result = ia64_getreg(_IA64_REG_AR_ITC);
+	ia64_barrier();
+#ifdef CONFIG_ITANIUM
+	while (unlikely((__s32) result == -1)) {
+		result = ia64_getreg(_IA64_REG_AR_ITC);
+		ia64_barrier();
+	}
+#endif
+	return result;
+}
+#endif /* !__ASSEMBLY__ */
+
+#ifdef CONFIG_XEN
+#include <asm/xen/privop.h>
+#else
+#define ia64_get_kr(regnum)		__ia64_get_kr(regnum)
+#define ia64_set_kr(regnum, r)		__ia64_set_kr(regnum,r)
+#define ia64_get_ivr			__ia64_get_ivr
+#define ia64_eoi			__ia64_eoi
+#define ia64_set_itm(val)		__ia64_set_itm(val)
+#define cpu_relax			__cpu_relax
+#define ia64_get_tpr			__ia64_get_tpr
+#define ia64_set_tpr(vector)		__ia64_set_tpr(vector)
+#define ia64_get_eflag			__ia64_get_eflag
+#define ia64_set_eflag(eflag)		__ia64_set_eflag(eflag)
+#define ia64_pal_halt_light		__ia64_pal_halt_light
+#define	ia64_get_psr_i			__ia64_get_psr_i
+#define ia64_rsm_i			__ia64_rsm_i
+#define ia64_ssm_i			__ia64_ssm_i
+#define	ia64_local_irq_restore(x)	__ia64_local_irq_restore(x)
+#define	ia64_leave_kernel		__ia64_leave_kernel
+#define	ia64_leave_syscall		__ia64_leave_syscall
+#define	ia64_switch_to			__ia64_switch_to
+#define	ia64_pal_call_static		__ia64_pal_call_static
+#endif
+
+#endif /* _ASM_IA64_PRIVOP_H */
diff -Naur linux-2.6.12/include/asm-ia64/processor.h linux-2.6.12-XenPatch/include/asm-ia64/processor.h
--- linux-2.6.12/include/asm-ia64/processor.h	2005-06-17 13:48:29.000000000 -0600
+++ linux-2.6.12-XenPatch/include/asm-ia64/processor.h	2005-07-05 11:51:07.000000000 -0600
@@ -19,6 +19,7 @@
 #include <asm/kregs.h>
 #include <asm/ptrace.h>
 #include <asm/ustack.h>
+#include <asm/privop.h>
 
 /* Our arch specific arch_init_sched_domain is in arch/ia64/kernel/domain.c */
 #define ARCH_HAS_SCHED_DOMAIN
@@ -362,42 +363,6 @@
 /* Return stack pointer of blocked task TSK.  */
 #define KSTK_ESP(tsk)  ((tsk)->thread.ksp)
 
-extern void ia64_getreg_unknown_kr (void);
-extern void ia64_setreg_unknown_kr (void);
-
-#define ia64_get_kr(regnum)					\
-({								\
-	unsigned long r = 0;					\
-								\
-	switch (regnum) {					\
-	    case 0: r = ia64_getreg(_IA64_REG_AR_KR0); break;	\
-	    case 1: r = ia64_getreg(_IA64_REG_AR_KR1); break;	\
-	    case 2: r = ia64_getreg(_IA64_REG_AR_KR2); break;	\
-	    case 3: r = ia64_getreg(_IA64_REG_AR_KR3); break;	\
-	    case 4: r = ia64_getreg(_IA64_REG_AR_KR4); break;	\
-	    case 5: r = ia64_getreg(_IA64_REG_AR_KR5); break;	\
-	    case 6: r = ia64_getreg(_IA64_REG_AR_KR6); break;	\
-	    case 7: r = ia64_getreg(_IA64_REG_AR_KR7); break;	\
-	    default: ia64_getreg_unknown_kr(); break;		\
-	}							\
-	r;							\
-})
-
-#define ia64_set_kr(regnum, r) 					\
-({								\
-	switch (regnum) {					\
-	    case 0: ia64_setreg(_IA64_REG_AR_KR0, r); break;	\
-	    case 1: ia64_setreg(_IA64_REG_AR_KR1, r); break;	\
-	    case 2: ia64_setreg(_IA64_REG_AR_KR2, r); break;	\
-	    case 3: ia64_setreg(_IA64_REG_AR_KR3, r); break;	\
-	    case 4: ia64_setreg(_IA64_REG_AR_KR4, r); break;	\
-	    case 5: ia64_setreg(_IA64_REG_AR_KR5, r); break;	\
-	    case 6: ia64_setreg(_IA64_REG_AR_KR6, r); break;	\
-	    case 7: ia64_setreg(_IA64_REG_AR_KR7, r); break;	\
-	    default: ia64_setreg_unknown_kr(); break;		\
-	}							\
-})
-
 /*
  * The following three macros can't be inline functions because we don't have struct
  * task_struct at this point.
@@ -554,15 +519,6 @@
 }
 
 static inline void
-ia64_eoi (void)
-{
-	ia64_setreg(_IA64_REG_CR_EOI, 0);
-	ia64_srlz_d();
-}
-
-#define cpu_relax()	ia64_hint(ia64_hint_pause)
-
-static inline void
 ia64_set_lrr0 (unsigned long val)
 {
 	ia64_setreg(_IA64_REG_CR_LRR0, val);
@@ -623,16 +579,6 @@
 #define current_text_addr() \
 	({ void *_pc; _pc = (void *)ia64_getreg(_IA64_REG_IP); _pc; })
 
-static inline __u64
-ia64_get_ivr (void)
-{
-	__u64 r;
-	ia64_srlz_d();
-	r = ia64_getreg(_IA64_REG_CR_IVR);
-	ia64_srlz_d();
-	return r;
-}
-
 static inline void
 ia64_set_dbr (__u64 regnum, __u64 value)
 {
diff -Naur linux-2.6.12/include/asm-ia64/system.h linux-2.6.12-XenPatch/include/asm-ia64/system.h
--- linux-2.6.12/include/asm-ia64/system.h	2005-06-17 13:48:29.000000000 -0600
+++ linux-2.6.12-XenPatch/include/asm-ia64/system.h	2005-07-05 12:44:19.000000000 -0600
@@ -124,18 +124,18 @@
 #define __local_irq_save(x)			\
 do {						\
 	ia64_stop();				\
-	(x) = ia64_getreg(_IA64_REG_PSR);	\
+	(x) = ia64_get_psr_i();			\
 	ia64_stop();				\
-	ia64_rsm(IA64_PSR_I);			\
+	ia64_rsm_i();				\
 } while (0)
 
 #define __local_irq_disable()			\
 do {						\
 	ia64_stop();				\
-	ia64_rsm(IA64_PSR_I);			\
+	ia64_rsm_i();				\
 } while (0)
 
-#define __local_irq_restore(x)	ia64_intrin_local_irq_restore((x) & IA64_PSR_I)
+#define __local_irq_restore(x)	ia64_local_irq_restore((x) & IA64_PSR_I)
 
 #ifdef CONFIG_IA64_DEBUG_IRQ
 
@@ -171,8 +171,8 @@
 # define local_irq_restore(x)	__local_irq_restore(x)
 #endif /* !CONFIG_IA64_DEBUG_IRQ */
 
-#define local_irq_enable()	({ ia64_stop(); ia64_ssm(IA64_PSR_I); ia64_srlz_d(); })
-#define local_save_flags(flags)	({ ia64_stop(); (flags) = ia64_getreg(_IA64_REG_PSR); })
+#define local_irq_enable()	({ ia64_stop(); ia64_ssm_i(); ia64_srlz_d(); })
+#define local_save_flags(flags)	({ ia64_stop(); (flags) = ia64_get_psr_i(); })
 
 #define irqs_disabled()				\
 ({						\

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: Xen and the Art of Linux/ia64 Virtualization
  2005-06-29 20:16 Xen and the Art of Linux/ia64 Virtualization Magenheimer, Dan (HP Labs Fort Collins)
                   ` (4 preceding siblings ...)
  2005-07-06 20:46 ` Magenheimer, Dan (HP Labs Fort Collins)
@ 2005-07-07 22:18 ` Gerald Pfeifer
  2005-07-07 23:57 ` Magenheimer, Dan (HP Labs Fort Collins)
                   ` (13 subsequent siblings)
  19 siblings, 0 replies; 21+ messages in thread
From: Gerald Pfeifer @ 2005-07-07 22:18 UTC (permalink / raw)
  To: linux-ia64

On Wed, 29 Jun 2005, Magenheimer, Dan (HP Labs Fort Collins) wrote:
> Also unlike Xen/x86, Linux/ia64 with CONFIG_XEN enabled runs 
> successfully on Xen, but also run natively on hardware (called 
> "transparent paravirtualization").

That's neat indeed.

> This should be appealing to Linux/ia64 distributors as different
> kernels need not be shipped for running on Xen or running native.

What's the performance impact for using transparanent paravirtualization? 
Is it sufficiently low that distributors really can consider using a 
single kernel for both native and Xen "hosting"?

Gerald

^ permalink raw reply	[flat|nested] 21+ messages in thread

* RE: Xen and the Art of Linux/ia64 Virtualization
  2005-06-29 20:16 Xen and the Art of Linux/ia64 Virtualization Magenheimer, Dan (HP Labs Fort Collins)
                   ` (5 preceding siblings ...)
  2005-07-07 22:18 ` Gerald Pfeifer
@ 2005-07-07 23:57 ` Magenheimer, Dan (HP Labs Fort Collins)
  2005-07-11 15:09 ` Magenheimer, Dan (HP Labs Fort Collins)
                   ` (12 subsequent siblings)
  19 siblings, 0 replies; 21+ messages in thread
From: Magenheimer, Dan (HP Labs Fort Collins) @ 2005-07-07 23:57 UTC (permalink / raw)
  To: linux-ia64

> On Wed, 29 Jun 2005, Magenheimer, Dan (HP Labs Fort Collins) wrote:
> > Also unlike Xen/x86, Linux/ia64 with CONFIG_XEN enabled runs 
> > successfully on Xen, but also run natively on hardware (called 
> > "transparent paravirtualization").
> 
> That's neat indeed.

Thanks!
 
> > This should be appealing to Linux/ia64 distributors as different
> > kernels need not be shipped for running on Xen or running native.
> 
> What's the performance impact for using transparanent 
> paravirtualization? 
> Is it sufficiently low that distributors really can consider using a 
> single kernel for both native and Xen "hosting"?

I believe so.  No benchmarks yet but I'm guessing it will
be barely measurable.  If someone who has experience measuring
very small differences in benchmarks wants to give it a try,
I would be happy to help!

Dan


^ permalink raw reply	[flat|nested] 21+ messages in thread

* RE: Xen and the Art of Linux/ia64 Virtualization
  2005-06-29 20:16 Xen and the Art of Linux/ia64 Virtualization Magenheimer, Dan (HP Labs Fort Collins)
                   ` (6 preceding siblings ...)
  2005-07-07 23:57 ` Magenheimer, Dan (HP Labs Fort Collins)
@ 2005-07-11 15:09 ` Magenheimer, Dan (HP Labs Fort Collins)
  2005-07-14 21:17 ` Magenheimer, Dan (HP Labs Fort Collins)
                   ` (11 subsequent siblings)
  19 siblings, 0 replies; 21+ messages in thread
From: Magenheimer, Dan (HP Labs Fort Collins) @ 2005-07-11 15:09 UTC (permalink / raw)
  To: linux-ia64

> On Wed, 29 Jun 2005, Magenheimer, Dan (HP Labs Fort Collins) wrote:
> > Also unlike Xen/x86, Linux/ia64 with CONFIG_XEN enabled runs 
> > successfully on Xen, but also run natively on hardware (called 
> > "transparent paravirtualization").
> 
> That's neat indeed.
> 
> > This should be appealing to Linux/ia64 distributors as different
> > kernels need not be shipped for running on Xen or running native.
> 
> What's the performance impact for using transparanent 
> paravirtualization? 
> Is it sufficiently low that distributors really can consider using a 
> single kernel for both native and Xen "hosting"?

I ran a test over the weekend comparing kernel.org 2.6.12
compiled for a zx1 (rx2600) vs the same with transparent
paravirtualization enabled.  The test is compiling linux-2.6.12
(several times... I interrupted the second run) in
single-user mode.

As you can see, the performance impact is "in the noise".

If someone wants to run a benchmark that can be measured
with better precision than 1 part in 1000, please let
me know.

Dan

==
transparent paravirtualization
[djm@localhost djm]$ egrep '^real|^user|^sys' *048
05.07.11.071048:real    18m42.709s
05.07.11.071048:user    18m26.588s
05.07.11.071048:sys     0m25.431s
05.07.11.071048:real    18m33.756s
05.07.11.071048:user    18m27.412s
05.07.11.071048:sys     0m25.480s
05.07.11.071048:real    18m34.987s
05.07.11.071048:user    18m28.532s
05.07.11.071048:sys     0m25.700s
05.07.11.071048:real    18m35.458s
05.07.11.071048:user    18m28.976s
05.07.11.071048:sys     0m25.502s

native 2.6.12
[djm@localhost djm]$ egrep '^real|^user|^sys' *753
05.07.10.180753:real    18m43.805s
05.07.10.180753:user    18m27.958s
05.07.10.180753:sys     0m25.181s
05.07.10.180753:real    18m34.283s
05.07.10.180753:user    18m27.864s
05.07.10.180753:sys     0m25.414s
05.07.10.180753:real    18m34.513s
05.07.10.180753:user    18m28.247s
05.07.10.180753:sys     0m25.362s

^ permalink raw reply	[flat|nested] 21+ messages in thread

* RE: Xen and the Art of Linux/ia64 Virtualization
  2005-06-29 20:16 Xen and the Art of Linux/ia64 Virtualization Magenheimer, Dan (HP Labs Fort Collins)
                   ` (7 preceding siblings ...)
  2005-07-11 15:09 ` Magenheimer, Dan (HP Labs Fort Collins)
@ 2005-07-14 21:17 ` Magenheimer, Dan (HP Labs Fort Collins)
  2005-07-16  1:15 ` Christoph Hellwig
                   ` (10 subsequent siblings)
  19 siblings, 0 replies; 21+ messages in thread
From: Magenheimer, Dan (HP Labs Fort Collins) @ 2005-07-14 21:17 UTC (permalink / raw)
  To: linux-ia64

[-- Attachment #1: Type: text/plain, Size: 2782 bytes --]

Revision 3.  Incorporates some more feedback and
a minor bug fix.

Dan

> -----Original Message-----
> From: Magenheimer, Dan (HP Labs Fort Collins) 
> Sent: Wednesday, July 06, 2005 2:47 PM
> To: 'linux-ia64@vger.kernel.org'
> Subject: RE: Xen and the Art of Linux/ia64 Virtualization
> 
> The patch I posted yesterday had a couple of bugs.
> This version applies cleanly against 2.6.12 and
> (when augmented with additional files from various
> Xen subdirectories) has been booted both on Xen/ia64
> and on hardware.
> 
> Note that a patch to drivers/acpi/motherboard.c (that
> allows for acpi to be enabled on a stubbed acpi tree
> without a kernel null pointer dereference!) is required
> and has been submitted separately to the linux-acpi list.
> 
> With the exception of three short CONFIG_XEN
> ifdefs, the vast majority of changes in this patch
> are code rearrangement to enable a number of routines
> and defines to add one level of abstraction.
> 
> Comments and feedback would be much appreciated!
> 
>  arch/ia64/Kconfig             |    7 +
>  arch/ia64/Makefile            |    1 
>  arch/ia64/hp/sim/Makefile     |    2 
>  arch/ia64/ia32/elfcore32.h    |    2 
>  arch/ia64/ia32/ia32_signal.c  |    6 -
>  arch/ia64/ia32/ia32_support.c |    4 -
>  arch/ia64/kernel/entry.S      |   30 ++++---
>  arch/ia64/kernel/head.S       |    4 +
>  arch/ia64/kernel/irq_ia64.c   |   12 +--
>  arch/ia64/kernel/pal.S        |    5 -
>  arch/ia64/kernel/setup.c      |    3 
>  include/asm-ia64/delay.h      |   53 -------------
>  include/asm-ia64/privop.h     |  160 
> ++++++++++++++++++++++++++++++++++++++++++
>  include/asm-ia64/processor.h  |   56 --------------
>  include/asm-ia64/system.h     |   12 +--
>  15 files changed, 217 insertions(+), 140 deletions(-)
> 
> > -----Original Message-----
> > From: Magenheimer, Dan (HP Labs Fort Collins) 
> > Sent: Tuesday, July 05, 2005 2:21 PM
> > To: linux-ia64@vger.kernel.org
> > Subject: RE: Xen and the Art of Linux/ia64 Virtualization
> > 
> > Thanks to excellent feedback from David Mosberger and
> > Christophe Hellwig, I have greatly cleaned up the attached
> > patch by using some very nice abstractions.  In fact,
> > most of the bulk in the attached patch results from
> > moving some code from asm-ia64/{delay,processor}.h to
> > a new file, asm-ia64/privop.h.  And the number of
> > CONFIG_XEN ifdefs is dramatically reduced.
> > 
> > More feedback appreciated (including comments about how
> > close this might be to be ready for submission to Tony).
> > 
> > Thanks,
> > Dan Magenheimer
> > 
> > P.S. Anybody who is attending OLS who wants to talk about
> > Xen (and specifically Xen/ia64)?  I'll be there... send
> > me email.
> > 
> 

[-- Attachment #2: xenia64.patch.final --]
[-- Type: application/octet-stream, Size: 20973 bytes --]

diff -Naur linux-2.6.12/arch/ia64/hp/sim/Makefile linux-2.6.12-XenPatch/arch/ia64/hp/sim/Makefile
--- linux-2.6.12/arch/ia64/hp/sim/Makefile	2005-06-17 13:48:29.000000000 -0600
+++ linux-2.6.12-XenPatch/arch/ia64/hp/sim/Makefile	2005-07-14 14:35:54.000000000 -0600
@@ -14,3 +14,5 @@
 obj-$(CONFIG_HP_SIMSERIAL) += simserial.o
 obj-$(CONFIG_HP_SIMSERIAL_CONSOLE) += hpsim_console.o
 obj-$(CONFIG_HP_SIMSCSI) += simscsi.o
+obj-$(CONFIG_XEN) += simserial.o
+obj-$(CONFIG_XEN) += hpsim_console.o
diff -Naur linux-2.6.12/arch/ia64/ia32/elfcore32.h linux-2.6.12-XenPatch/arch/ia64/ia32/elfcore32.h
--- linux-2.6.12/arch/ia64/ia32/elfcore32.h	2005-06-17 13:48:29.000000000 -0600
+++ linux-2.6.12-XenPatch/arch/ia64/ia32/elfcore32.h	2005-07-14 14:35:54.000000000 -0600
@@ -82,7 +82,7 @@
 	pr_reg[11] = regs->r1; 				\
 	pr_reg[12] = regs->cr_iip;			\
 	pr_reg[13] = regs->r17 & 0xffff;		\
-	pr_reg[14] = ia64_getreg(_IA64_REG_AR_EFLAG);	\
+	pr_reg[14] = ia64_get_eflag();			\
 	pr_reg[15] = regs->r12;				\
 	pr_reg[16] = (regs->r17 >> 16) & 0xffff;
 
diff -Naur linux-2.6.12/arch/ia64/ia32/ia32_signal.c linux-2.6.12-XenPatch/arch/ia64/ia32/ia32_signal.c
--- linux-2.6.12/arch/ia64/ia32/ia32_signal.c	2005-06-17 13:48:29.000000000 -0600
+++ linux-2.6.12-XenPatch/arch/ia64/ia32/ia32_signal.c	2005-07-14 14:35:54.000000000 -0600
@@ -666,7 +666,7 @@
 	/*
 	 *  `eflags' is in an ar register for this context
 	 */
-	flag = ia64_getreg(_IA64_REG_AR_EFLAG);
+	flag = ia64_get_eflag();
 	err |= __put_user((unsigned int)flag, &sc->eflags);
 	err |= __put_user(regs->r12, &sc->esp_at_signal);
 	err |= __put_user((regs->r17 >> 16) & 0xffff, (unsigned int __user *)&sc->ss);
@@ -754,10 +754,10 @@
 		 *  IA32 process's context.
 		 */
 		err |= __get_user(tmpflags, &sc->eflags);
-		flag = ia64_getreg(_IA64_REG_AR_EFLAG);
+		flag = ia64_get_eflag();
 		flag &= ~0x40DD5;
 		flag |= (tmpflags & 0x40DD5);
-		ia64_setreg(_IA64_REG_AR_EFLAG, flag);
+		ia64_set_eflag(flag);
 
 		regs->r1 = -1;	/* disable syscall checks, r1 is orig_eax */
 	}
diff -Naur linux-2.6.12/arch/ia64/ia32/ia32_support.c linux-2.6.12-XenPatch/arch/ia64/ia32/ia32_support.c
--- linux-2.6.12/arch/ia64/ia32/ia32_support.c	2005-06-17 13:48:29.000000000 -0600
+++ linux-2.6.12-XenPatch/arch/ia64/ia32/ia32_support.c	2005-07-14 14:35:54.000000000 -0600
@@ -100,7 +100,7 @@
 void
 ia32_save_state (struct task_struct *t)
 {
-	t->thread.eflag = ia64_getreg(_IA64_REG_AR_EFLAG);
+	t->thread.eflag = ia64_get_eflag();
 	t->thread.fsr   = ia64_getreg(_IA64_REG_AR_FSR);
 	t->thread.fcr   = ia64_getreg(_IA64_REG_AR_FCR);
 	t->thread.fir   = ia64_getreg(_IA64_REG_AR_FIR);
@@ -122,7 +122,7 @@
 	fdr = t->thread.fdr;
 	tssd = load_desc(_TSS);					/* TSSD */
 
-	ia64_setreg(_IA64_REG_AR_EFLAG, eflag);
+	ia64_set_eflag(eflag);
 	ia64_setreg(_IA64_REG_AR_FSR, fsr);
 	ia64_setreg(_IA64_REG_AR_FCR, fcr);
 	ia64_setreg(_IA64_REG_AR_FIR, fir);
diff -Naur linux-2.6.12/arch/ia64/Kconfig linux-2.6.12-XenPatch/arch/ia64/Kconfig
--- linux-2.6.12/arch/ia64/Kconfig	2005-06-17 13:48:29.000000000 -0600
+++ linux-2.6.12-XenPatch/arch/ia64/Kconfig	2005-07-14 14:35:54.000000000 -0600
@@ -46,6 +46,13 @@
 	bool
 	default y
 
+config XEN
+	bool
+	default n
+	help
+	  Enable Xen hypervisor support.  Resulting kernel runs
+	  both as a guest OS on Xen and natively on hardware.
+
 config SCHED_NO_NO_OMIT_FRAME_POINTER
 	bool
 	default y
diff -Naur linux-2.6.12/arch/ia64/kernel/entry.S linux-2.6.12-XenPatch/arch/ia64/kernel/entry.S
--- linux-2.6.12/arch/ia64/kernel/entry.S	2005-06-17 13:48:29.000000000 -0600
+++ linux-2.6.12-XenPatch/arch/ia64/kernel/entry.S	2005-07-14 14:35:58.000000000 -0600
@@ -181,7 +181,7 @@
  *	called.  The code starting at .map relies on this.  The rest of the code
  *	doesn't care about the interrupt masking status.
  */
-GLOBAL_ENTRY(ia64_switch_to)
+GLOBAL_ENTRY(__ia64_switch_to)
 	.prologue
 	alloc r16=ar.pfs,1,0,0,0
 	DO_SAVE_SWITCH_STACK
@@ -235,7 +235,7 @@
 	;;
 	itr.d dtr[r25]=r23		// wire in new mapping...
 	br.cond.sptk .done
-END(ia64_switch_to)
+END(__ia64_switch_to)
 
 /*
  * Note that interrupts are enabled during save_switch_stack and load_switch_stack.  This
@@ -376,7 +376,7 @@
  *	- b7 holds address to return to
  *	- must not touch r8-r11
  */
-ENTRY(load_switch_stack)
+GLOBAL_ENTRY(load_switch_stack)
 	.prologue
 	.altrp b7
 
@@ -500,7 +500,7 @@
 	 * because some system calls (such as ia64_execve) directly
 	 * manipulate ar.pfs.
 	 */
-GLOBAL_ENTRY(ia64_trace_syscall)
+GLOBAL_ENTRY(__ia64_trace_syscall)
 	PT_REGS_UNWIND_INFO(0)
 	/*
 	 * We need to preserve the scratch registers f6-f11 in case the system
@@ -570,7 +570,7 @@
 (p6)	mov r10=-1
 (p6)	mov r8=r9
 	br.cond.sptk .strace_save_retval
-END(ia64_trace_syscall)
+END(__ia64_trace_syscall)
 
 	/*
 	 * When traced and returning from sigreturn, we invoke syscall_trace but then
@@ -623,8 +623,11 @@
 	adds r2=PT(R8)+16,sp			// r2 = &pt_regs.r8
 	mov r10=r0				// clear error indication in r10
 (p7)	br.cond.spnt handle_syscall_error	// handle potential syscall failure
+	;;
+	// don't fall through, ia64_leave_syscall may be #define'd
+	br.cond.sptk.few ia64_leave_syscall
+	;;
 END(ia64_ret_from_syscall)
-	// fall through
 /*
  * ia64_leave_syscall(): Same as ia64_leave_kernel, except that it doesn't
  *	need to switch to bank 0 and doesn't restore the scratch registers.
@@ -669,7 +672,7 @@
  *	      ar.csd: cleared
  *	      ar.ssd: cleared
  */
-ENTRY(ia64_leave_syscall)
+GLOBAL_ENTRY(__ia64_leave_syscall)
 	PT_REGS_UNWIND_INFO(0)
 	/*
 	 * work.need_resched etc. mustn't get changed by this CPU before it returns to
@@ -770,7 +773,7 @@
 	mov.m ar.ccv=r0		// clear ar.ccv
 (pNonSys) br.cond.dpnt.many dont_preserve_current_frame
 	br.cond.sptk.many rbs_switch
-END(ia64_leave_syscall)
+END(__ia64_leave_syscall)
 
 #ifdef CONFIG_IA32_SUPPORT
 GLOBAL_ENTRY(ia64_ret_from_ia32_execve)
@@ -782,10 +785,13 @@
 	st8.spill [r2]=r8	// store return value in slot for r8 and set unat bit
 	.mem.offset 8,0
 	st8.spill [r3]=r0	// clear error indication in slot for r10 and set unat bit
+	;;
+	// don't fall through, ia64_leave_kernel may be #define'd
+	br.cond.sptk.few ia64_leave_kernel
+	;;
 END(ia64_ret_from_ia32_execve)
-	// fall through
 #endif /* CONFIG_IA32_SUPPORT */
-GLOBAL_ENTRY(ia64_leave_kernel)
+GLOBAL_ENTRY(__ia64_leave_kernel)
 	PT_REGS_UNWIND_INFO(0)
 	/*
 	 * work.need_resched etc. mustn't get changed by this CPU before it returns to
@@ -1131,7 +1137,7 @@
 	ld8 r10=[r3]
 	br.cond.sptk.many .work_processed_syscall	// re-check
 
-END(ia64_leave_kernel)
+END(__ia64_leave_kernel)
 
 ENTRY(handle_syscall_error)
 	/*
@@ -1171,7 +1177,7 @@
 	 * be set up by the caller.  We declare 8 input registers so the system call
 	 * args get preserved, in case we need to restart a system call.
 	 */
-ENTRY(notify_resume_user)
+GLOBAL_ENTRY(notify_resume_user)
 	.prologue ASM_UNW_PRLG_RP|ASM_UNW_PRLG_PFS, ASM_UNW_PRLG_GRSAVE(8)
 	alloc loc1=ar.pfs,8,2,3,0 // preserve all eight input regs in case of syscall restart!
 	mov r9=ar.unat
diff -Naur linux-2.6.12/arch/ia64/kernel/head.S linux-2.6.12-XenPatch/arch/ia64/kernel/head.S
--- linux-2.6.12/arch/ia64/kernel/head.S	2005-06-17 13:48:29.000000000 -0600
+++ linux-2.6.12-XenPatch/arch/ia64/kernel/head.S	2005-07-14 14:35:54.000000000 -0600
@@ -370,6 +370,10 @@
 
 	// This is executed by the bootstrap processor (bsp) only:
 
+#ifdef CONFIG_XEN
+	br.call.sptk.many rp=early_xen_setup
+	;;
+#endif
 #ifdef CONFIG_IA64_FW_EMU
 	// initialize PAL & SAL emulator:
 	br.call.sptk.many rp=sys_fw_init
diff -Naur linux-2.6.12/arch/ia64/kernel/irq_ia64.c linux-2.6.12-XenPatch/arch/ia64/kernel/irq_ia64.c
--- linux-2.6.12/arch/ia64/kernel/irq_ia64.c	2005-06-17 13:48:29.000000000 -0600
+++ linux-2.6.12-XenPatch/arch/ia64/kernel/irq_ia64.c	2005-07-14 14:35:54.000000000 -0600
@@ -151,11 +151,11 @@
 	 * to kernel stack overflows).
 	 */
 	irq_enter();
-	saved_tpr = ia64_getreg(_IA64_REG_CR_TPR);
+	saved_tpr = ia64_get_tpr();
 	ia64_srlz_d();
 	while (vector != IA64_SPURIOUS_INT_VECTOR) {
 		if (!IS_RESCHEDULE(vector)) {
-			ia64_setreg(_IA64_REG_CR_TPR, vector);
+			ia64_set_tpr(vector);
 			ia64_srlz_d();
 
 			__do_IRQ(local_vector_to_irq(vector), regs);
@@ -164,7 +164,7 @@
 			 * Disable interrupts and send EOI:
 			 */
 			local_irq_disable();
-			ia64_setreg(_IA64_REG_CR_TPR, saved_tpr);
+			ia64_set_tpr(saved_tpr);
 		}
 		ia64_eoi();
 		vector = ia64_get_ivr();
@@ -191,7 +191,7 @@
 	vector = ia64_get_ivr();
 
 	 irq_enter();
-	 saved_tpr = ia64_getreg(_IA64_REG_CR_TPR);
+	 saved_tpr = ia64_get_tpr();
 	 ia64_srlz_d();
 
 	 /*
@@ -199,7 +199,7 @@
 	  */
 	while (vector != IA64_SPURIOUS_INT_VECTOR) {
 		if (!IS_RESCHEDULE(vector)) {
-			ia64_setreg(_IA64_REG_CR_TPR, vector);
+			ia64_set_tpr(vector);
 			ia64_srlz_d();
 
 			/*
@@ -215,7 +215,7 @@
 			 * Disable interrupts and send EOI
 			 */
 			local_irq_disable();
-			ia64_setreg(_IA64_REG_CR_TPR, saved_tpr);
+			ia64_set_tpr(saved_tpr);
 		}
 		ia64_eoi();
 		vector = ia64_get_ivr();
diff -Naur linux-2.6.12/arch/ia64/kernel/pal.S linux-2.6.12-XenPatch/arch/ia64/kernel/pal.S
--- linux-2.6.12/arch/ia64/kernel/pal.S	2005-06-17 13:48:29.000000000 -0600
+++ linux-2.6.12-XenPatch/arch/ia64/kernel/pal.S	2005-07-14 14:35:54.000000000 -0600
@@ -16,6 +16,7 @@
 #include <asm/processor.h>
 
 	.data
+	.globl pal_entry_point
 pal_entry_point:
 	data8 ia64_pal_default_handler
 	.text
@@ -53,7 +54,7 @@
  * in4	       1 ==> clear psr.ic,  0 ==> don't clear psr.ic
  *
  */
-GLOBAL_ENTRY(ia64_pal_call_static)
+GLOBAL_ENTRY(__ia64_pal_call_static)
 	.prologue ASM_UNW_PRLG_RP|ASM_UNW_PRLG_PFS, ASM_UNW_PRLG_GRSAVE(5)
 	alloc loc1 = ar.pfs,5,5,0,0
 	movl loc2 = pal_entry_point
@@ -90,7 +91,7 @@
 	;;
 	srlz.d				// seralize restoration of psr.l
 	br.ret.sptk.many b0
-END(ia64_pal_call_static)
+END(__ia64_pal_call_static)
 
 /*
  * Make a PAL call using the stacked registers calling convention.
diff -Naur linux-2.6.12/arch/ia64/kernel/setup.c linux-2.6.12-XenPatch/arch/ia64/kernel/setup.c
--- linux-2.6.12/arch/ia64/kernel/setup.c	2005-06-17 13:48:29.000000000 -0600
+++ linux-2.6.12-XenPatch/arch/ia64/kernel/setup.c	2005-07-14 14:35:54.000000000 -0600
@@ -273,6 +273,9 @@
 static inline int __init
 early_console_setup (char *cmdline)
 {
+#ifdef CONFIG_XEN
+	if (!early_xen_console_setup(cmdline)) return 0;
+#endif
 #ifdef CONFIG_SERIAL_SGI_L1_CONSOLE
 	{
 		extern int sn_serial_console_early_setup(void);
diff -Naur linux-2.6.12/arch/ia64/Makefile linux-2.6.12-XenPatch/arch/ia64/Makefile
--- linux-2.6.12/arch/ia64/Makefile	2005-06-17 13:48:29.000000000 -0600
+++ linux-2.6.12-XenPatch/arch/ia64/Makefile	2005-07-14 14:35:54.000000000 -0600
@@ -57,6 +57,7 @@
 core-$(CONFIG_IA64_HP_ZX1)	+= arch/ia64/dig/
 core-$(CONFIG_IA64_HP_ZX1_SWIOTLB) += arch/ia64/dig/
 core-$(CONFIG_IA64_SGI_SN2)	+= arch/ia64/sn/
+core-$(CONFIG_XEN)		+= arch/ia64/xen/ arch/ia64/hp/sim/
 
 drivers-$(CONFIG_PCI)		+= arch/ia64/pci/
 drivers-$(CONFIG_IA64_HP_SIM)	+= arch/ia64/hp/sim/
diff -Naur linux-2.6.12/include/asm-ia64/delay.h linux-2.6.12-XenPatch/include/asm-ia64/delay.h
--- linux-2.6.12/include/asm-ia64/delay.h	2005-06-17 13:48:29.000000000 -0600
+++ linux-2.6.12-XenPatch/include/asm-ia64/delay.h	2005-07-14 14:35:54.000000000 -0600
@@ -20,59 +20,6 @@
 #include <asm/intrinsics.h>
 #include <asm/processor.h>
 
-static __inline__ void
-ia64_set_itm (unsigned long val)
-{
-	ia64_setreg(_IA64_REG_CR_ITM, val);
-	ia64_srlz_d();
-}
-
-static __inline__ unsigned long
-ia64_get_itm (void)
-{
-	unsigned long result;
-
-	result = ia64_getreg(_IA64_REG_CR_ITM);
-	ia64_srlz_d();
-	return result;
-}
-
-static __inline__ void
-ia64_set_itv (unsigned long val)
-{
-	ia64_setreg(_IA64_REG_CR_ITV, val);
-	ia64_srlz_d();
-}
-
-static __inline__ unsigned long
-ia64_get_itv (void)
-{
-	return ia64_getreg(_IA64_REG_CR_ITV);
-}
-
-static __inline__ void
-ia64_set_itc (unsigned long val)
-{
-	ia64_setreg(_IA64_REG_AR_ITC, val);
-	ia64_srlz_d();
-}
-
-static __inline__ unsigned long
-ia64_get_itc (void)
-{
-	unsigned long result;
-
-	result = ia64_getreg(_IA64_REG_AR_ITC);
-	ia64_barrier();
-#ifdef CONFIG_ITANIUM
-	while (unlikely((__s32) result == -1)) {
-		result = ia64_getreg(_IA64_REG_AR_ITC);
-		ia64_barrier();
-	}
-#endif
-	return result;
-}
-
 extern void ia64_delay_loop (unsigned long loops);
 
 static __inline__ void
diff -Naur linux-2.6.12/include/asm-ia64/privop.h linux-2.6.12-XenPatch/include/asm-ia64/privop.h
--- linux-2.6.12/include/asm-ia64/privop.h	1969-12-31 17:00:00.000000000 -0700
+++ linux-2.6.12-XenPatch/include/asm-ia64/privop.h	2005-07-14 14:35:54.000000000 -0600
@@ -0,0 +1,160 @@
+#ifndef _ASM_IA64_PRIVOP_H
+#define _ASM_IA64_PRIVOP_H
+
+/*
+ * Copyright (C) 2005 Hewlett-Packard Co
+ *	Dan Magenheimer <dan.magenheimer@hp.com>
+ * Reorganization from asm-ia64/processor.h
+ *
+ */
+
+#include <linux/config.h>
+
+#include <asm/intrinsics.h>
+
+#ifndef __ASSEMBLY__
+extern void ia64_getreg_unknown_kr (void);
+extern void ia64_setreg_unknown_kr (void);
+
+#define __ia64_get_kr(regnum)					\
+({								\
+	unsigned long r = 0;					\
+								\
+	switch (regnum) {					\
+	    case 0: r = ia64_getreg(_IA64_REG_AR_KR0); break;	\
+	    case 1: r = ia64_getreg(_IA64_REG_AR_KR1); break;	\
+	    case 2: r = ia64_getreg(_IA64_REG_AR_KR2); break;	\
+	    case 3: r = ia64_getreg(_IA64_REG_AR_KR3); break;	\
+	    case 4: r = ia64_getreg(_IA64_REG_AR_KR4); break;	\
+	    case 5: r = ia64_getreg(_IA64_REG_AR_KR5); break;	\
+	    case 6: r = ia64_getreg(_IA64_REG_AR_KR6); break;	\
+	    case 7: r = ia64_getreg(_IA64_REG_AR_KR7); break;	\
+	    default: ia64_getreg_unknown_kr(); break;		\
+	}							\
+	r;							\
+})
+
+#define __ia64_set_kr(regnum,r)					\
+({								\
+	switch (regnum) {					\
+	    case 0: ia64_setreg(_IA64_REG_AR_KR0, r); break;	\
+	    case 1: ia64_setreg(_IA64_REG_AR_KR1, r); break;	\
+	    case 2: ia64_setreg(_IA64_REG_AR_KR2, r); break;	\
+	    case 3: ia64_setreg(_IA64_REG_AR_KR3, r); break;	\
+	    case 4: ia64_setreg(_IA64_REG_AR_KR4, r); break;	\
+	    case 5: ia64_setreg(_IA64_REG_AR_KR5, r); break;	\
+	    case 6: ia64_setreg(_IA64_REG_AR_KR6, r); break;	\
+	    case 7: ia64_setreg(_IA64_REG_AR_KR7, r); break;	\
+	    default: ia64_setreg_unknown_kr(); break;		\
+	}							\
+})
+
+static inline unsigned long
+__ia64_get_ivr (void)
+{
+	unsigned long r;
+	ia64_srlz_d();
+	r = ia64_getreg(_IA64_REG_CR_IVR);
+	ia64_srlz_d();
+	return r;
+}
+
+static inline void
+__ia64_eoi (void)
+{
+	ia64_setreg(_IA64_REG_CR_EOI, 0);
+	ia64_srlz_d();
+}
+
+#define __cpu_relax()	ia64_hint(ia64_hint_pause)
+
+#define __ia64_get_tpr(vector)	ia64_getreg(_IA64_REG_CR_TPR)
+#define __ia64_set_tpr(vector)	ia64_setreg(_IA64_REG_CR_TPR,vector)
+
+#define __ia64_get_eflag()	ia64_getreg(_IA64_REG_AR_EFLAG)
+#define __ia64_set_eflag(eflag)	ia64_setreg(_IA64_REG_AR_EFLAG,eflag)
+
+#define	__ia64_get_psr_i()	ia64_getreg(_IA64_REG_PSR)
+#define __ia64_rsm_i()		ia64_rsm(IA64_PSR_I)
+#define __ia64_ssm_i()		ia64_ssm(IA64_PSR_I)
+#define	__ia64_local_irq_restore(x)	ia64_intrin_local_irq_restore(x)
+
+static __inline__ void
+__ia64_set_itm (unsigned long val)
+{
+	ia64_setreg(_IA64_REG_CR_ITM, val);
+	ia64_srlz_d();
+}
+
+static __inline__ unsigned long
+ia64_get_itm (void)
+{
+	unsigned long result;
+
+	result = ia64_getreg(_IA64_REG_CR_ITM);
+	ia64_srlz_d();
+	return result;
+}
+
+static __inline__ void
+ia64_set_itv (unsigned long val)
+{
+	ia64_setreg(_IA64_REG_CR_ITV, val);
+	ia64_srlz_d();
+}
+
+static __inline__ unsigned long
+ia64_get_itv (void)
+{
+	return ia64_getreg(_IA64_REG_CR_ITV);
+}
+
+static __inline__ void
+ia64_set_itc (unsigned long val)
+{
+	ia64_setreg(_IA64_REG_AR_ITC, val);
+	ia64_srlz_d();
+}
+
+static __inline__ unsigned long
+ia64_get_itc (void)
+{
+	unsigned long result;
+
+	result = ia64_getreg(_IA64_REG_AR_ITC);
+	ia64_barrier();
+#ifdef CONFIG_ITANIUM
+	while (unlikely((__s32) result == -1)) {
+		result = ia64_getreg(_IA64_REG_AR_ITC);
+		ia64_barrier();
+	}
+#endif
+	return result;
+}
+#endif /* !__ASSEMBLY__ */
+
+#ifdef CONFIG_XEN
+#include <asm/xen/privop.h>
+#else
+#define ia64_get_kr(regnum)		__ia64_get_kr(regnum)
+#define ia64_set_kr(regnum, r)		__ia64_set_kr(regnum,r)
+#define ia64_get_ivr			__ia64_get_ivr
+#define ia64_eoi			__ia64_eoi
+#define ia64_set_itm(val)		__ia64_set_itm(val)
+#define cpu_relax			__cpu_relax
+#define ia64_get_tpr			__ia64_get_tpr
+#define ia64_set_tpr(vector)		__ia64_set_tpr(vector)
+#define ia64_get_eflag			__ia64_get_eflag
+#define ia64_set_eflag(eflag)		__ia64_set_eflag(eflag)
+#define ia64_pal_halt_light		__ia64_pal_halt_light
+#define	ia64_get_psr_i			__ia64_get_psr_i
+#define ia64_rsm_i			__ia64_rsm_i
+#define ia64_ssm_i			__ia64_ssm_i
+#define	ia64_local_irq_restore(x)	__ia64_local_irq_restore(x)
+#define	ia64_leave_kernel		__ia64_leave_kernel
+#define	ia64_leave_syscall		__ia64_leave_syscall
+#define	ia64_switch_to			__ia64_switch_to
+#define	ia64_pal_call_static		__ia64_pal_call_static
+#endif
+
+#endif /* _ASM_IA64_PRIVOP_H */
diff -Naur linux-2.6.12/include/asm-ia64/processor.h linux-2.6.12-XenPatch/include/asm-ia64/processor.h
--- linux-2.6.12/include/asm-ia64/processor.h	2005-06-17 13:48:29.000000000 -0600
+++ linux-2.6.12-XenPatch/include/asm-ia64/processor.h	2005-07-14 14:35:54.000000000 -0600
@@ -19,6 +19,7 @@
 #include <asm/kregs.h>
 #include <asm/ptrace.h>
 #include <asm/ustack.h>
+#include <asm/privop.h>
 
 /* Our arch specific arch_init_sched_domain is in arch/ia64/kernel/domain.c */
 #define ARCH_HAS_SCHED_DOMAIN
@@ -362,42 +363,6 @@
 /* Return stack pointer of blocked task TSK.  */
 #define KSTK_ESP(tsk)  ((tsk)->thread.ksp)
 
-extern void ia64_getreg_unknown_kr (void);
-extern void ia64_setreg_unknown_kr (void);
-
-#define ia64_get_kr(regnum)					\
-({								\
-	unsigned long r = 0;					\
-								\
-	switch (regnum) {					\
-	    case 0: r = ia64_getreg(_IA64_REG_AR_KR0); break;	\
-	    case 1: r = ia64_getreg(_IA64_REG_AR_KR1); break;	\
-	    case 2: r = ia64_getreg(_IA64_REG_AR_KR2); break;	\
-	    case 3: r = ia64_getreg(_IA64_REG_AR_KR3); break;	\
-	    case 4: r = ia64_getreg(_IA64_REG_AR_KR4); break;	\
-	    case 5: r = ia64_getreg(_IA64_REG_AR_KR5); break;	\
-	    case 6: r = ia64_getreg(_IA64_REG_AR_KR6); break;	\
-	    case 7: r = ia64_getreg(_IA64_REG_AR_KR7); break;	\
-	    default: ia64_getreg_unknown_kr(); break;		\
-	}							\
-	r;							\
-})
-
-#define ia64_set_kr(regnum, r) 					\
-({								\
-	switch (regnum) {					\
-	    case 0: ia64_setreg(_IA64_REG_AR_KR0, r); break;	\
-	    case 1: ia64_setreg(_IA64_REG_AR_KR1, r); break;	\
-	    case 2: ia64_setreg(_IA64_REG_AR_KR2, r); break;	\
-	    case 3: ia64_setreg(_IA64_REG_AR_KR3, r); break;	\
-	    case 4: ia64_setreg(_IA64_REG_AR_KR4, r); break;	\
-	    case 5: ia64_setreg(_IA64_REG_AR_KR5, r); break;	\
-	    case 6: ia64_setreg(_IA64_REG_AR_KR6, r); break;	\
-	    case 7: ia64_setreg(_IA64_REG_AR_KR7, r); break;	\
-	    default: ia64_setreg_unknown_kr(); break;		\
-	}							\
-})
-
 /*
  * The following three macros can't be inline functions because we don't have struct
  * task_struct at this point.
@@ -554,15 +519,6 @@
 }
 
 static inline void
-ia64_eoi (void)
-{
-	ia64_setreg(_IA64_REG_CR_EOI, 0);
-	ia64_srlz_d();
-}
-
-#define cpu_relax()	ia64_hint(ia64_hint_pause)
-
-static inline void
 ia64_set_lrr0 (unsigned long val)
 {
 	ia64_setreg(_IA64_REG_CR_LRR0, val);
@@ -623,16 +579,6 @@
 #define current_text_addr() \
 	({ void *_pc; _pc = (void *)ia64_getreg(_IA64_REG_IP); _pc; })
 
-static inline __u64
-ia64_get_ivr (void)
-{
-	__u64 r;
-	ia64_srlz_d();
-	r = ia64_getreg(_IA64_REG_CR_IVR);
-	ia64_srlz_d();
-	return r;
-}
-
 static inline void
 ia64_set_dbr (__u64 regnum, __u64 value)
 {
diff -Naur linux-2.6.12/include/asm-ia64/system.h linux-2.6.12-XenPatch/include/asm-ia64/system.h
--- linux-2.6.12/include/asm-ia64/system.h	2005-06-17 13:48:29.000000000 -0600
+++ linux-2.6.12-XenPatch/include/asm-ia64/system.h	2005-07-14 14:35:54.000000000 -0600
@@ -124,18 +124,18 @@
 #define __local_irq_save(x)			\
 do {						\
 	ia64_stop();				\
-	(x) = ia64_getreg(_IA64_REG_PSR);	\
+	(x) = ia64_get_psr_i();			\
 	ia64_stop();				\
-	ia64_rsm(IA64_PSR_I);			\
+	ia64_rsm_i();				\
 } while (0)
 
 #define __local_irq_disable()			\
 do {						\
 	ia64_stop();				\
-	ia64_rsm(IA64_PSR_I);			\
+	ia64_rsm_i();				\
 } while (0)
 
-#define __local_irq_restore(x)	ia64_intrin_local_irq_restore((x) & IA64_PSR_I)
+#define __local_irq_restore(x)	ia64_local_irq_restore((x) & IA64_PSR_I)
 
 #ifdef CONFIG_IA64_DEBUG_IRQ
 
@@ -171,8 +171,8 @@
 # define local_irq_restore(x)	__local_irq_restore(x)
 #endif /* !CONFIG_IA64_DEBUG_IRQ */
 
-#define local_irq_enable()	({ ia64_stop(); ia64_ssm(IA64_PSR_I); ia64_srlz_d(); })
-#define local_save_flags(flags)	({ ia64_stop(); (flags) = ia64_getreg(_IA64_REG_PSR); })
+#define local_irq_enable()	({ ia64_stop(); ia64_ssm_i(); ia64_srlz_d(); })
+#define local_save_flags(flags)	({ ia64_stop(); (flags) = ia64_get_psr_i(); })
 
 #define irqs_disabled()				\
 ({						\

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: Xen and the Art of Linux/ia64 Virtualization
  2005-06-29 20:16 Xen and the Art of Linux/ia64 Virtualization Magenheimer, Dan (HP Labs Fort Collins)
                   ` (8 preceding siblings ...)
  2005-07-14 21:17 ` Magenheimer, Dan (HP Labs Fort Collins)
@ 2005-07-16  1:15 ` Christoph Hellwig
  2005-07-16 12:55 ` Magenheimer, Dan (HP Labs Fort Collins)
                   ` (9 subsequent siblings)
  19 siblings, 0 replies; 21+ messages in thread
From: Christoph Hellwig @ 2005-07-16  1:15 UTC (permalink / raw)
  To: linux-ia64

On Thu, Jul 14, 2005 at 02:17:47PM -0700, Magenheimer, Dan (HP Labs Fort Collins) wrote:
> Revision 3.  Incorporates some more feedback and
> a minor bug fix.

Looks nice.  One thing that bothers me is that privop.h is unsymmetic,
I'd rather see the Xen variants of the routines in asm/privop.h aswell.
Also in the early console setup please put the return on a separate line
from the if clause.

It would be very nice if you could submit the additional abstractions
as a first patch as soon as possible and post a full patch ontop of it
later that includes the new files aswell.


^ permalink raw reply	[flat|nested] 21+ messages in thread

* RE: Xen and the Art of Linux/ia64 Virtualization
  2005-06-29 20:16 Xen and the Art of Linux/ia64 Virtualization Magenheimer, Dan (HP Labs Fort Collins)
                   ` (9 preceding siblings ...)
  2005-07-16  1:15 ` Christoph Hellwig
@ 2005-07-16 12:55 ` Magenheimer, Dan (HP Labs Fort Collins)
  2005-07-28 22:38 ` Magenheimer, Dan (HP Labs Fort Collins)
                   ` (8 subsequent siblings)
  19 siblings, 0 replies; 21+ messages in thread
From: Magenheimer, Dan (HP Labs Fort Collins) @ 2005-07-16 12:55 UTC (permalink / raw)
  To: linux-ia64

> Looks nice.  One thing that bothers me is that privop.h is unsymmetic,
> I'd rather see the Xen variants of the routines in 
> asm/privop.h aswell.

Thanks!  Do you mean like:

#ifdef CONFIG_XEN
#include <asm/xen/privop.h>
#define privop1 xen_privop1
#define privop2(x) xen_privop2(x)
 :
#else
#define privop1 __privop1
#define privop2(x) __privop2(x)
 :
#endif

> Also in the early console setup please put the return on a 
> separate line
> from the if clause.

OK, will do in Rev4.

> It would be very nice if you could submit the additional abstractions
> as a first patch as soon as possible and post a full patch ontop of it
> later that includes the new files aswell.

That's the plan.

Thanks very much for the feedback!
Dan

^ permalink raw reply	[flat|nested] 21+ messages in thread

* RE: Xen and the Art of Linux/ia64 Virtualization
  2005-06-29 20:16 Xen and the Art of Linux/ia64 Virtualization Magenheimer, Dan (HP Labs Fort Collins)
                   ` (10 preceding siblings ...)
  2005-07-16 12:55 ` Magenheimer, Dan (HP Labs Fort Collins)
@ 2005-07-28 22:38 ` Magenheimer, Dan (HP Labs Fort Collins)
  2005-07-28 23:13 ` david mosberger
                   ` (7 subsequent siblings)
  19 siblings, 0 replies; 21+ messages in thread
From: Magenheimer, Dan (HP Labs Fort Collins) @ 2005-07-28 22:38 UTC (permalink / raw)
  To: linux-ia64

[-- Attachment #1: Type: text/plain, Size: 3433 bytes --]

Revision 4.  Minor formatting change suggested by
Christoph.  Also, no change for "symmetry" suggested
by Christoph as Tony wanted to leave as is. 

Tony I think this is now ready to apply.  If there is
anything else needed, please let me know.

Thanks,
Dan Magenheimer

> -----Original Message-----
> From: Magenheimer, Dan (HP Labs Fort Collins) 
> Sent: Thursday, July 14, 2005 3:18 PM
> To: 'linux-ia64@vger.kernel.org'
> Subject: RE: Xen and the Art of Linux/ia64 Virtualization
> 
> Revision 3.  Incorporates some more feedback and
> a minor bug fix.
> 
> Dan
> 
> > -----Original Message-----
> > From: Magenheimer, Dan (HP Labs Fort Collins) 
> > Sent: Wednesday, July 06, 2005 2:47 PM
> > To: 'linux-ia64@vger.kernel.org'
> > Subject: RE: Xen and the Art of Linux/ia64 Virtualization
> > 
> > The patch I posted yesterday had a couple of bugs.
> > This version applies cleanly against 2.6.12 and
> > (when augmented with additional files from various
> > Xen subdirectories) has been booted both on Xen/ia64
> > and on hardware.
> > 
> > Note that a patch to drivers/acpi/motherboard.c (that
> > allows for acpi to be enabled on a stubbed acpi tree
> > without a kernel null pointer dereference!) is required
> > and has been submitted separately to the linux-acpi list.
> > 
> > With the exception of three short CONFIG_XEN
> > ifdefs, the vast majority of changes in this patch
> > are code rearrangement to enable a number of routines
> > and defines to add one level of abstraction.
> > 
> > Comments and feedback would be much appreciated!
> > 
> >  arch/ia64/Kconfig             |    7 +
> >  arch/ia64/Makefile            |    1 
> >  arch/ia64/hp/sim/Makefile     |    2 
> >  arch/ia64/ia32/elfcore32.h    |    2 
> >  arch/ia64/ia32/ia32_signal.c  |    6 -
> >  arch/ia64/ia32/ia32_support.c |    4 -
> >  arch/ia64/kernel/entry.S      |   30 ++++---
> >  arch/ia64/kernel/head.S       |    4 +
> >  arch/ia64/kernel/irq_ia64.c   |   12 +--
> >  arch/ia64/kernel/pal.S        |    5 -
> >  arch/ia64/kernel/setup.c      |    3 
> >  include/asm-ia64/delay.h      |   53 -------------
> >  include/asm-ia64/privop.h     |  160 
> > ++++++++++++++++++++++++++++++++++++++++++
> >  include/asm-ia64/processor.h  |   56 --------------
> >  include/asm-ia64/system.h     |   12 +--
> >  15 files changed, 217 insertions(+), 140 deletions(-)
> > 
> > > -----Original Message-----
> > > From: Magenheimer, Dan (HP Labs Fort Collins) 
> > > Sent: Tuesday, July 05, 2005 2:21 PM
> > > To: linux-ia64@vger.kernel.org
> > > Subject: RE: Xen and the Art of Linux/ia64 Virtualization
> > > 
> > > Thanks to excellent feedback from David Mosberger and
> > > Christophe Hellwig, I have greatly cleaned up the attached
> > > patch by using some very nice abstractions.  In fact,
> > > most of the bulk in the attached patch results from
> > > moving some code from asm-ia64/{delay,processor}.h to
> > > a new file, asm-ia64/privop.h.  And the number of
> > > CONFIG_XEN ifdefs is dramatically reduced.
> > > 
> > > More feedback appreciated (including comments about how
> > > close this might be to be ready for submission to Tony).
> > > 
> > > Thanks,
> > > Dan Magenheimer
> > > 
> > > P.S. Anybody who is attending OLS who wants to talk about
> > > Xen (and specifically Xen/ia64)?  I'll be there... send
> > > me email.
> > > 
> > 
> 

[-- Attachment #2: xenia64.patch.rev4 --]
[-- Type: application/octet-stream, Size: 20977 bytes --]

diff -Naur linux-2.6.12/arch/ia64/hp/sim/Makefile linux-2.6.12-XenPatch/arch/ia64/hp/sim/Makefile
--- linux-2.6.12/arch/ia64/hp/sim/Makefile	2005-06-17 13:48:29.000000000 -0600
+++ linux-2.6.12-XenPatch/arch/ia64/hp/sim/Makefile	2005-07-28 16:25:20.000000000 -0600
@@ -14,3 +14,5 @@
 obj-$(CONFIG_HP_SIMSERIAL) += simserial.o
 obj-$(CONFIG_HP_SIMSERIAL_CONSOLE) += hpsim_console.o
 obj-$(CONFIG_HP_SIMSCSI) += simscsi.o
+obj-$(CONFIG_XEN) += simserial.o
+obj-$(CONFIG_XEN) += hpsim_console.o
diff -Naur linux-2.6.12/arch/ia64/ia32/elfcore32.h linux-2.6.12-XenPatch/arch/ia64/ia32/elfcore32.h
--- linux-2.6.12/arch/ia64/ia32/elfcore32.h	2005-06-17 13:48:29.000000000 -0600
+++ linux-2.6.12-XenPatch/arch/ia64/ia32/elfcore32.h	2005-07-28 16:25:20.000000000 -0600
@@ -82,7 +82,7 @@
 	pr_reg[11] = regs->r1; 				\
 	pr_reg[12] = regs->cr_iip;			\
 	pr_reg[13] = regs->r17 & 0xffff;		\
-	pr_reg[14] = ia64_getreg(_IA64_REG_AR_EFLAG);	\
+	pr_reg[14] = ia64_get_eflag();			\
 	pr_reg[15] = regs->r12;				\
 	pr_reg[16] = (regs->r17 >> 16) & 0xffff;
 
diff -Naur linux-2.6.12/arch/ia64/ia32/ia32_signal.c linux-2.6.12-XenPatch/arch/ia64/ia32/ia32_signal.c
--- linux-2.6.12/arch/ia64/ia32/ia32_signal.c	2005-06-17 13:48:29.000000000 -0600
+++ linux-2.6.12-XenPatch/arch/ia64/ia32/ia32_signal.c	2005-07-28 16:25:20.000000000 -0600
@@ -666,7 +666,7 @@
 	/*
 	 *  `eflags' is in an ar register for this context
 	 */
-	flag = ia64_getreg(_IA64_REG_AR_EFLAG);
+	flag = ia64_get_eflag();
 	err |= __put_user((unsigned int)flag, &sc->eflags);
 	err |= __put_user(regs->r12, &sc->esp_at_signal);
 	err |= __put_user((regs->r17 >> 16) & 0xffff, (unsigned int __user *)&sc->ss);
@@ -754,10 +754,10 @@
 		 *  IA32 process's context.
 		 */
 		err |= __get_user(tmpflags, &sc->eflags);
-		flag = ia64_getreg(_IA64_REG_AR_EFLAG);
+		flag = ia64_get_eflag();
 		flag &= ~0x40DD5;
 		flag |= (tmpflags & 0x40DD5);
-		ia64_setreg(_IA64_REG_AR_EFLAG, flag);
+		ia64_set_eflag(flag);
 
 		regs->r1 = -1;	/* disable syscall checks, r1 is orig_eax */
 	}
diff -Naur linux-2.6.12/arch/ia64/ia32/ia32_support.c linux-2.6.12-XenPatch/arch/ia64/ia32/ia32_support.c
--- linux-2.6.12/arch/ia64/ia32/ia32_support.c	2005-06-17 13:48:29.000000000 -0600
+++ linux-2.6.12-XenPatch/arch/ia64/ia32/ia32_support.c	2005-07-28 16:25:20.000000000 -0600
@@ -100,7 +100,7 @@
 void
 ia32_save_state (struct task_struct *t)
 {
-	t->thread.eflag = ia64_getreg(_IA64_REG_AR_EFLAG);
+	t->thread.eflag = ia64_get_eflag();
 	t->thread.fsr   = ia64_getreg(_IA64_REG_AR_FSR);
 	t->thread.fcr   = ia64_getreg(_IA64_REG_AR_FCR);
 	t->thread.fir   = ia64_getreg(_IA64_REG_AR_FIR);
@@ -122,7 +122,7 @@
 	fdr = t->thread.fdr;
 	tssd = load_desc(_TSS);					/* TSSD */
 
-	ia64_setreg(_IA64_REG_AR_EFLAG, eflag);
+	ia64_set_eflag(eflag);
 	ia64_setreg(_IA64_REG_AR_FSR, fsr);
 	ia64_setreg(_IA64_REG_AR_FCR, fcr);
 	ia64_setreg(_IA64_REG_AR_FIR, fir);
diff -Naur linux-2.6.12/arch/ia64/Kconfig linux-2.6.12-XenPatch/arch/ia64/Kconfig
--- linux-2.6.12/arch/ia64/Kconfig	2005-06-17 13:48:29.000000000 -0600
+++ linux-2.6.12-XenPatch/arch/ia64/Kconfig	2005-07-28 16:25:20.000000000 -0600
@@ -46,6 +46,13 @@
 	bool
 	default y
 
+config XEN
+	bool
+	default n
+	help
+	  Enable Xen hypervisor support.  Resulting kernel runs
+	  both as a guest OS on Xen and natively on hardware.
+
 config SCHED_NO_NO_OMIT_FRAME_POINTER
 	bool
 	default y
diff -Naur linux-2.6.12/arch/ia64/kernel/entry.S linux-2.6.12-XenPatch/arch/ia64/kernel/entry.S
--- linux-2.6.12/arch/ia64/kernel/entry.S	2005-06-17 13:48:29.000000000 -0600
+++ linux-2.6.12-XenPatch/arch/ia64/kernel/entry.S	2005-07-28 16:25:20.000000000 -0600
@@ -181,7 +181,7 @@
  *	called.  The code starting at .map relies on this.  The rest of the code
  *	doesn't care about the interrupt masking status.
  */
-GLOBAL_ENTRY(ia64_switch_to)
+GLOBAL_ENTRY(__ia64_switch_to)
 	.prologue
 	alloc r16=ar.pfs,1,0,0,0
 	DO_SAVE_SWITCH_STACK
@@ -235,7 +235,7 @@
 	;;
 	itr.d dtr[r25]=r23		// wire in new mapping...
 	br.cond.sptk .done
-END(ia64_switch_to)
+END(__ia64_switch_to)
 
 /*
  * Note that interrupts are enabled during save_switch_stack and load_switch_stack.  This
@@ -376,7 +376,7 @@
  *	- b7 holds address to return to
  *	- must not touch r8-r11
  */
-ENTRY(load_switch_stack)
+GLOBAL_ENTRY(load_switch_stack)
 	.prologue
 	.altrp b7
 
@@ -500,7 +500,7 @@
 	 * because some system calls (such as ia64_execve) directly
 	 * manipulate ar.pfs.
 	 */
-GLOBAL_ENTRY(ia64_trace_syscall)
+GLOBAL_ENTRY(__ia64_trace_syscall)
 	PT_REGS_UNWIND_INFO(0)
 	/*
 	 * We need to preserve the scratch registers f6-f11 in case the system
@@ -570,7 +570,7 @@
 (p6)	mov r10=-1
 (p6)	mov r8=r9
 	br.cond.sptk .strace_save_retval
-END(ia64_trace_syscall)
+END(__ia64_trace_syscall)
 
 	/*
 	 * When traced and returning from sigreturn, we invoke syscall_trace but then
@@ -623,8 +623,11 @@
 	adds r2=PT(R8)+16,sp			// r2 = &pt_regs.r8
 	mov r10=r0				// clear error indication in r10
 (p7)	br.cond.spnt handle_syscall_error	// handle potential syscall failure
+	;;
+	// don't fall through, ia64_leave_syscall may be #define'd
+	br.cond.sptk.few ia64_leave_syscall
+	;;
 END(ia64_ret_from_syscall)
-	// fall through
 /*
  * ia64_leave_syscall(): Same as ia64_leave_kernel, except that it doesn't
  *	need to switch to bank 0 and doesn't restore the scratch registers.
@@ -669,7 +672,7 @@
  *	      ar.csd: cleared
  *	      ar.ssd: cleared
  */
-ENTRY(ia64_leave_syscall)
+GLOBAL_ENTRY(__ia64_leave_syscall)
 	PT_REGS_UNWIND_INFO(0)
 	/*
 	 * work.need_resched etc. mustn't get changed by this CPU before it returns to
@@ -770,7 +773,7 @@
 	mov.m ar.ccv=r0		// clear ar.ccv
 (pNonSys) br.cond.dpnt.many dont_preserve_current_frame
 	br.cond.sptk.many rbs_switch
-END(ia64_leave_syscall)
+END(__ia64_leave_syscall)
 
 #ifdef CONFIG_IA32_SUPPORT
 GLOBAL_ENTRY(ia64_ret_from_ia32_execve)
@@ -782,10 +785,13 @@
 	st8.spill [r2]=r8	// store return value in slot for r8 and set unat bit
 	.mem.offset 8,0
 	st8.spill [r3]=r0	// clear error indication in slot for r10 and set unat bit
+	;;
+	// don't fall through, ia64_leave_kernel may be #define'd
+	br.cond.sptk.few ia64_leave_kernel
+	;;
 END(ia64_ret_from_ia32_execve)
-	// fall through
 #endif /* CONFIG_IA32_SUPPORT */
-GLOBAL_ENTRY(ia64_leave_kernel)
+GLOBAL_ENTRY(__ia64_leave_kernel)
 	PT_REGS_UNWIND_INFO(0)
 	/*
 	 * work.need_resched etc. mustn't get changed by this CPU before it returns to
@@ -1131,7 +1137,7 @@
 	ld8 r10=[r3]
 	br.cond.sptk.many .work_processed_syscall	// re-check
 
-END(ia64_leave_kernel)
+END(__ia64_leave_kernel)
 
 ENTRY(handle_syscall_error)
 	/*
@@ -1171,7 +1177,7 @@
 	 * be set up by the caller.  We declare 8 input registers so the system call
 	 * args get preserved, in case we need to restart a system call.
 	 */
-ENTRY(notify_resume_user)
+GLOBAL_ENTRY(notify_resume_user)
 	.prologue ASM_UNW_PRLG_RP|ASM_UNW_PRLG_PFS, ASM_UNW_PRLG_GRSAVE(8)
 	alloc loc1=ar.pfs,8,2,3,0 // preserve all eight input regs in case of syscall restart!
 	mov r9=ar.unat
diff -Naur linux-2.6.12/arch/ia64/kernel/head.S linux-2.6.12-XenPatch/arch/ia64/kernel/head.S
--- linux-2.6.12/arch/ia64/kernel/head.S	2005-06-17 13:48:29.000000000 -0600
+++ linux-2.6.12-XenPatch/arch/ia64/kernel/head.S	2005-07-28 16:25:20.000000000 -0600
@@ -370,6 +370,10 @@
 
 	// This is executed by the bootstrap processor (bsp) only:
 
+#ifdef CONFIG_XEN
+	br.call.sptk.many rp=early_xen_setup
+	;;
+#endif
 #ifdef CONFIG_IA64_FW_EMU
 	// initialize PAL & SAL emulator:
 	br.call.sptk.many rp=sys_fw_init
diff -Naur linux-2.6.12/arch/ia64/kernel/irq_ia64.c linux-2.6.12-XenPatch/arch/ia64/kernel/irq_ia64.c
--- linux-2.6.12/arch/ia64/kernel/irq_ia64.c	2005-06-17 13:48:29.000000000 -0600
+++ linux-2.6.12-XenPatch/arch/ia64/kernel/irq_ia64.c	2005-07-28 16:25:20.000000000 -0600
@@ -151,11 +151,11 @@
 	 * to kernel stack overflows).
 	 */
 	irq_enter();
-	saved_tpr = ia64_getreg(_IA64_REG_CR_TPR);
+	saved_tpr = ia64_get_tpr();
 	ia64_srlz_d();
 	while (vector != IA64_SPURIOUS_INT_VECTOR) {
 		if (!IS_RESCHEDULE(vector)) {
-			ia64_setreg(_IA64_REG_CR_TPR, vector);
+			ia64_set_tpr(vector);
 			ia64_srlz_d();
 
 			__do_IRQ(local_vector_to_irq(vector), regs);
@@ -164,7 +164,7 @@
 			 * Disable interrupts and send EOI:
 			 */
 			local_irq_disable();
-			ia64_setreg(_IA64_REG_CR_TPR, saved_tpr);
+			ia64_set_tpr(saved_tpr);
 		}
 		ia64_eoi();
 		vector = ia64_get_ivr();
@@ -191,7 +191,7 @@
 	vector = ia64_get_ivr();
 
 	 irq_enter();
-	 saved_tpr = ia64_getreg(_IA64_REG_CR_TPR);
+	 saved_tpr = ia64_get_tpr();
 	 ia64_srlz_d();
 
 	 /*
@@ -199,7 +199,7 @@
 	  */
 	while (vector != IA64_SPURIOUS_INT_VECTOR) {
 		if (!IS_RESCHEDULE(vector)) {
-			ia64_setreg(_IA64_REG_CR_TPR, vector);
+			ia64_set_tpr(vector);
 			ia64_srlz_d();
 
 			/*
@@ -215,7 +215,7 @@
 			 * Disable interrupts and send EOI
 			 */
 			local_irq_disable();
-			ia64_setreg(_IA64_REG_CR_TPR, saved_tpr);
+			ia64_set_tpr(saved_tpr);
 		}
 		ia64_eoi();
 		vector = ia64_get_ivr();
diff -Naur linux-2.6.12/arch/ia64/kernel/pal.S linux-2.6.12-XenPatch/arch/ia64/kernel/pal.S
--- linux-2.6.12/arch/ia64/kernel/pal.S	2005-06-17 13:48:29.000000000 -0600
+++ linux-2.6.12-XenPatch/arch/ia64/kernel/pal.S	2005-07-28 16:25:20.000000000 -0600
@@ -16,6 +16,7 @@
 #include <asm/processor.h>
 
 	.data
+	.globl pal_entry_point
 pal_entry_point:
 	data8 ia64_pal_default_handler
 	.text
@@ -53,7 +54,7 @@
  * in4	       1 ==> clear psr.ic,  0 ==> don't clear psr.ic
  *
  */
-GLOBAL_ENTRY(ia64_pal_call_static)
+GLOBAL_ENTRY(__ia64_pal_call_static)
 	.prologue ASM_UNW_PRLG_RP|ASM_UNW_PRLG_PFS, ASM_UNW_PRLG_GRSAVE(5)
 	alloc loc1 = ar.pfs,5,5,0,0
 	movl loc2 = pal_entry_point
@@ -90,7 +91,7 @@
 	;;
 	srlz.d				// seralize restoration of psr.l
 	br.ret.sptk.many b0
-END(ia64_pal_call_static)
+END(__ia64_pal_call_static)
 
 /*
  * Make a PAL call using the stacked registers calling convention.
diff -Naur linux-2.6.12/arch/ia64/kernel/setup.c linux-2.6.12-XenPatch/arch/ia64/kernel/setup.c
--- linux-2.6.12/arch/ia64/kernel/setup.c	2005-06-17 13:48:29.000000000 -0600
+++ linux-2.6.12-XenPatch/arch/ia64/kernel/setup.c	2005-07-28 16:25:50.000000000 -0600
@@ -273,6 +273,10 @@
 static inline int __init
 early_console_setup (char *cmdline)
 {
+#ifdef CONFIG_XEN
+	if (!early_xen_console_setup(cmdline))
+		return 0;
+#endif
 #ifdef CONFIG_SERIAL_SGI_L1_CONSOLE
 	{
 		extern int sn_serial_console_early_setup(void);
diff -Naur linux-2.6.12/arch/ia64/Makefile linux-2.6.12-XenPatch/arch/ia64/Makefile
--- linux-2.6.12/arch/ia64/Makefile	2005-06-17 13:48:29.000000000 -0600
+++ linux-2.6.12-XenPatch/arch/ia64/Makefile	2005-07-28 16:25:20.000000000 -0600
@@ -57,6 +57,7 @@
 core-$(CONFIG_IA64_HP_ZX1)	+= arch/ia64/dig/
 core-$(CONFIG_IA64_HP_ZX1_SWIOTLB) += arch/ia64/dig/
 core-$(CONFIG_IA64_SGI_SN2)	+= arch/ia64/sn/
+core-$(CONFIG_XEN)		+= arch/ia64/xen/ arch/ia64/hp/sim/
 
 drivers-$(CONFIG_PCI)		+= arch/ia64/pci/
 drivers-$(CONFIG_IA64_HP_SIM)	+= arch/ia64/hp/sim/
diff -Naur linux-2.6.12/include/asm-ia64/delay.h linux-2.6.12-XenPatch/include/asm-ia64/delay.h
--- linux-2.6.12/include/asm-ia64/delay.h	2005-06-17 13:48:29.000000000 -0600
+++ linux-2.6.12-XenPatch/include/asm-ia64/delay.h	2005-07-28 16:25:20.000000000 -0600
@@ -20,59 +20,6 @@
 #include <asm/intrinsics.h>
 #include <asm/processor.h>
 
-static __inline__ void
-ia64_set_itm (unsigned long val)
-{
-	ia64_setreg(_IA64_REG_CR_ITM, val);
-	ia64_srlz_d();
-}
-
-static __inline__ unsigned long
-ia64_get_itm (void)
-{
-	unsigned long result;
-
-	result = ia64_getreg(_IA64_REG_CR_ITM);
-	ia64_srlz_d();
-	return result;
-}
-
-static __inline__ void
-ia64_set_itv (unsigned long val)
-{
-	ia64_setreg(_IA64_REG_CR_ITV, val);
-	ia64_srlz_d();
-}
-
-static __inline__ unsigned long
-ia64_get_itv (void)
-{
-	return ia64_getreg(_IA64_REG_CR_ITV);
-}
-
-static __inline__ void
-ia64_set_itc (unsigned long val)
-{
-	ia64_setreg(_IA64_REG_AR_ITC, val);
-	ia64_srlz_d();
-}
-
-static __inline__ unsigned long
-ia64_get_itc (void)
-{
-	unsigned long result;
-
-	result = ia64_getreg(_IA64_REG_AR_ITC);
-	ia64_barrier();
-#ifdef CONFIG_ITANIUM
-	while (unlikely((__s32) result == -1)) {
-		result = ia64_getreg(_IA64_REG_AR_ITC);
-		ia64_barrier();
-	}
-#endif
-	return result;
-}
-
 extern void ia64_delay_loop (unsigned long loops);
 
 static __inline__ void
diff -Naur linux-2.6.12/include/asm-ia64/privop.h linux-2.6.12-XenPatch/include/asm-ia64/privop.h
--- linux-2.6.12/include/asm-ia64/privop.h	1969-12-31 17:00:00.000000000 -0700
+++ linux-2.6.12-XenPatch/include/asm-ia64/privop.h	2005-07-28 16:25:20.000000000 -0600
@@ -0,0 +1,160 @@
+#ifndef _ASM_IA64_PRIVOP_H
+#define _ASM_IA64_PRIVOP_H
+
+/*
+ * Copyright (C) 2005 Hewlett-Packard Co
+ *	Dan Magenheimer <dan.magenheimer@hp.com>
+ * Reorganization from asm-ia64/processor.h
+ *
+ */
+
+#include <linux/config.h>
+
+#include <asm/intrinsics.h>
+
+#ifndef __ASSEMBLY__
+extern void ia64_getreg_unknown_kr (void);
+extern void ia64_setreg_unknown_kr (void);
+
+#define __ia64_get_kr(regnum)					\
+({								\
+	unsigned long r = 0;					\
+								\
+	switch (regnum) {					\
+	    case 0: r = ia64_getreg(_IA64_REG_AR_KR0); break;	\
+	    case 1: r = ia64_getreg(_IA64_REG_AR_KR1); break;	\
+	    case 2: r = ia64_getreg(_IA64_REG_AR_KR2); break;	\
+	    case 3: r = ia64_getreg(_IA64_REG_AR_KR3); break;	\
+	    case 4: r = ia64_getreg(_IA64_REG_AR_KR4); break;	\
+	    case 5: r = ia64_getreg(_IA64_REG_AR_KR5); break;	\
+	    case 6: r = ia64_getreg(_IA64_REG_AR_KR6); break;	\
+	    case 7: r = ia64_getreg(_IA64_REG_AR_KR7); break;	\
+	    default: ia64_getreg_unknown_kr(); break;		\
+	}							\
+	r;							\
+})
+
+#define __ia64_set_kr(regnum,r)					\
+({								\
+	switch (regnum) {					\
+	    case 0: ia64_setreg(_IA64_REG_AR_KR0, r); break;	\
+	    case 1: ia64_setreg(_IA64_REG_AR_KR1, r); break;	\
+	    case 2: ia64_setreg(_IA64_REG_AR_KR2, r); break;	\
+	    case 3: ia64_setreg(_IA64_REG_AR_KR3, r); break;	\
+	    case 4: ia64_setreg(_IA64_REG_AR_KR4, r); break;	\
+	    case 5: ia64_setreg(_IA64_REG_AR_KR5, r); break;	\
+	    case 6: ia64_setreg(_IA64_REG_AR_KR6, r); break;	\
+	    case 7: ia64_setreg(_IA64_REG_AR_KR7, r); break;	\
+	    default: ia64_setreg_unknown_kr(); break;		\
+	}							\
+})
+
+static inline unsigned long
+__ia64_get_ivr (void)
+{
+	unsigned long r;
+	ia64_srlz_d();
+	r = ia64_getreg(_IA64_REG_CR_IVR);
+	ia64_srlz_d();
+	return r;
+}
+
+static inline void
+__ia64_eoi (void)
+{
+	ia64_setreg(_IA64_REG_CR_EOI, 0);
+	ia64_srlz_d();
+}
+
+#define __cpu_relax()	ia64_hint(ia64_hint_pause)
+
+#define __ia64_get_tpr(vector)	ia64_getreg(_IA64_REG_CR_TPR)
+#define __ia64_set_tpr(vector)	ia64_setreg(_IA64_REG_CR_TPR,vector)
+
+#define __ia64_get_eflag()	ia64_getreg(_IA64_REG_AR_EFLAG)
+#define __ia64_set_eflag(eflag)	ia64_setreg(_IA64_REG_AR_EFLAG,eflag)
+
+#define	__ia64_get_psr_i()	ia64_getreg(_IA64_REG_PSR)
+#define __ia64_rsm_i()		ia64_rsm(IA64_PSR_I)
+#define __ia64_ssm_i()		ia64_ssm(IA64_PSR_I)
+#define	__ia64_local_irq_restore(x)	ia64_intrin_local_irq_restore(x)
+
+static __inline__ void
+__ia64_set_itm (unsigned long val)
+{
+	ia64_setreg(_IA64_REG_CR_ITM, val);
+	ia64_srlz_d();
+}
+
+static __inline__ unsigned long
+ia64_get_itm (void)
+{
+	unsigned long result;
+
+	result = ia64_getreg(_IA64_REG_CR_ITM);
+	ia64_srlz_d();
+	return result;
+}
+
+static __inline__ void
+ia64_set_itv (unsigned long val)
+{
+	ia64_setreg(_IA64_REG_CR_ITV, val);
+	ia64_srlz_d();
+}
+
+static __inline__ unsigned long
+ia64_get_itv (void)
+{
+	return ia64_getreg(_IA64_REG_CR_ITV);
+}
+
+static __inline__ void
+ia64_set_itc (unsigned long val)
+{
+	ia64_setreg(_IA64_REG_AR_ITC, val);
+	ia64_srlz_d();
+}
+
+static __inline__ unsigned long
+ia64_get_itc (void)
+{
+	unsigned long result;
+
+	result = ia64_getreg(_IA64_REG_AR_ITC);
+	ia64_barrier();
+#ifdef CONFIG_ITANIUM
+	while (unlikely((__s32) result == -1)) {
+		result = ia64_getreg(_IA64_REG_AR_ITC);
+		ia64_barrier();
+	}
+#endif
+	return result;
+}
+#endif /* !__ASSEMBLY__ */
+
+#ifdef CONFIG_XEN
+#include <asm/xen/privop.h>
+#else
+#define ia64_get_kr(regnum)		__ia64_get_kr(regnum)
+#define ia64_set_kr(regnum, r)		__ia64_set_kr(regnum,r)
+#define ia64_get_ivr			__ia64_get_ivr
+#define ia64_eoi			__ia64_eoi
+#define ia64_set_itm(val)		__ia64_set_itm(val)
+#define cpu_relax			__cpu_relax
+#define ia64_get_tpr			__ia64_get_tpr
+#define ia64_set_tpr(vector)		__ia64_set_tpr(vector)
+#define ia64_get_eflag			__ia64_get_eflag
+#define ia64_set_eflag(eflag)		__ia64_set_eflag(eflag)
+#define ia64_pal_halt_light		__ia64_pal_halt_light
+#define	ia64_get_psr_i			__ia64_get_psr_i
+#define ia64_rsm_i			__ia64_rsm_i
+#define ia64_ssm_i			__ia64_ssm_i
+#define	ia64_local_irq_restore(x)	__ia64_local_irq_restore(x)
+#define	ia64_leave_kernel		__ia64_leave_kernel
+#define	ia64_leave_syscall		__ia64_leave_syscall
+#define	ia64_switch_to			__ia64_switch_to
+#define	ia64_pal_call_static		__ia64_pal_call_static
+#endif
+
+#endif /* _ASM_IA64_PRIVOP_H */
diff -Naur linux-2.6.12/include/asm-ia64/processor.h linux-2.6.12-XenPatch/include/asm-ia64/processor.h
--- linux-2.6.12/include/asm-ia64/processor.h	2005-06-17 13:48:29.000000000 -0600
+++ linux-2.6.12-XenPatch/include/asm-ia64/processor.h	2005-07-28 16:25:20.000000000 -0600
@@ -19,6 +19,7 @@
 #include <asm/kregs.h>
 #include <asm/ptrace.h>
 #include <asm/ustack.h>
+#include <asm/privop.h>
 
 /* Our arch specific arch_init_sched_domain is in arch/ia64/kernel/domain.c */
 #define ARCH_HAS_SCHED_DOMAIN
@@ -362,42 +363,6 @@
 /* Return stack pointer of blocked task TSK.  */
 #define KSTK_ESP(tsk)  ((tsk)->thread.ksp)
 
-extern void ia64_getreg_unknown_kr (void);
-extern void ia64_setreg_unknown_kr (void);
-
-#define ia64_get_kr(regnum)					\
-({								\
-	unsigned long r = 0;					\
-								\
-	switch (regnum) {					\
-	    case 0: r = ia64_getreg(_IA64_REG_AR_KR0); break;	\
-	    case 1: r = ia64_getreg(_IA64_REG_AR_KR1); break;	\
-	    case 2: r = ia64_getreg(_IA64_REG_AR_KR2); break;	\
-	    case 3: r = ia64_getreg(_IA64_REG_AR_KR3); break;	\
-	    case 4: r = ia64_getreg(_IA64_REG_AR_KR4); break;	\
-	    case 5: r = ia64_getreg(_IA64_REG_AR_KR5); break;	\
-	    case 6: r = ia64_getreg(_IA64_REG_AR_KR6); break;	\
-	    case 7: r = ia64_getreg(_IA64_REG_AR_KR7); break;	\
-	    default: ia64_getreg_unknown_kr(); break;		\
-	}							\
-	r;							\
-})
-
-#define ia64_set_kr(regnum, r) 					\
-({								\
-	switch (regnum) {					\
-	    case 0: ia64_setreg(_IA64_REG_AR_KR0, r); break;	\
-	    case 1: ia64_setreg(_IA64_REG_AR_KR1, r); break;	\
-	    case 2: ia64_setreg(_IA64_REG_AR_KR2, r); break;	\
-	    case 3: ia64_setreg(_IA64_REG_AR_KR3, r); break;	\
-	    case 4: ia64_setreg(_IA64_REG_AR_KR4, r); break;	\
-	    case 5: ia64_setreg(_IA64_REG_AR_KR5, r); break;	\
-	    case 6: ia64_setreg(_IA64_REG_AR_KR6, r); break;	\
-	    case 7: ia64_setreg(_IA64_REG_AR_KR7, r); break;	\
-	    default: ia64_setreg_unknown_kr(); break;		\
-	}							\
-})
-
 /*
  * The following three macros can't be inline functions because we don't have struct
  * task_struct at this point.
@@ -554,15 +519,6 @@
 }
 
 static inline void
-ia64_eoi (void)
-{
-	ia64_setreg(_IA64_REG_CR_EOI, 0);
-	ia64_srlz_d();
-}
-
-#define cpu_relax()	ia64_hint(ia64_hint_pause)
-
-static inline void
 ia64_set_lrr0 (unsigned long val)
 {
 	ia64_setreg(_IA64_REG_CR_LRR0, val);
@@ -623,16 +579,6 @@
 #define current_text_addr() \
 	({ void *_pc; _pc = (void *)ia64_getreg(_IA64_REG_IP); _pc; })
 
-static inline __u64
-ia64_get_ivr (void)
-{
-	__u64 r;
-	ia64_srlz_d();
-	r = ia64_getreg(_IA64_REG_CR_IVR);
-	ia64_srlz_d();
-	return r;
-}
-
 static inline void
 ia64_set_dbr (__u64 regnum, __u64 value)
 {
diff -Naur linux-2.6.12/include/asm-ia64/system.h linux-2.6.12-XenPatch/include/asm-ia64/system.h
--- linux-2.6.12/include/asm-ia64/system.h	2005-06-17 13:48:29.000000000 -0600
+++ linux-2.6.12-XenPatch/include/asm-ia64/system.h	2005-07-28 16:25:20.000000000 -0600
@@ -124,18 +124,18 @@
 #define __local_irq_save(x)			\
 do {						\
 	ia64_stop();				\
-	(x) = ia64_getreg(_IA64_REG_PSR);	\
+	(x) = ia64_get_psr_i();			\
 	ia64_stop();				\
-	ia64_rsm(IA64_PSR_I);			\
+	ia64_rsm_i();				\
 } while (0)
 
 #define __local_irq_disable()			\
 do {						\
 	ia64_stop();				\
-	ia64_rsm(IA64_PSR_I);			\
+	ia64_rsm_i();				\
 } while (0)
 
-#define __local_irq_restore(x)	ia64_intrin_local_irq_restore((x) & IA64_PSR_I)
+#define __local_irq_restore(x)	ia64_local_irq_restore((x) & IA64_PSR_I)
 
 #ifdef CONFIG_IA64_DEBUG_IRQ
 
@@ -171,8 +171,8 @@
 # define local_irq_restore(x)	__local_irq_restore(x)
 #endif /* !CONFIG_IA64_DEBUG_IRQ */
 
-#define local_irq_enable()	({ ia64_stop(); ia64_ssm(IA64_PSR_I); ia64_srlz_d(); })
-#define local_save_flags(flags)	({ ia64_stop(); (flags) = ia64_getreg(_IA64_REG_PSR); })
+#define local_irq_enable()	({ ia64_stop(); ia64_ssm_i(); ia64_srlz_d(); })
+#define local_save_flags(flags)	({ ia64_stop(); (flags) = ia64_get_psr_i(); })
 
 #define irqs_disabled()				\
 ({						\

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: Xen and the Art of Linux/ia64 Virtualization
  2005-06-29 20:16 Xen and the Art of Linux/ia64 Virtualization Magenheimer, Dan (HP Labs Fort Collins)
                   ` (11 preceding siblings ...)
  2005-07-28 22:38 ` Magenheimer, Dan (HP Labs Fort Collins)
@ 2005-07-28 23:13 ` david mosberger
  2005-07-28 23:27 ` Magenheimer, Dan (HP Labs Fort Collins)
                   ` (6 subsequent siblings)
  19 siblings, 0 replies; 21+ messages in thread
From: david mosberger @ 2005-07-28 23:13 UTC (permalink / raw)
  To: linux-ia64

Hi Dan,

I was hoping someone else would bring this up, but alas....

Why do you need to split getreg into get_eflag etc?  By definition,
wherever you call get_eflag, you have a getreg(R) call with R a
constant (and, in this case, R=_IA64_REG_AR_EFLAG).  Thus, you can
just have a Xen variant of getreg() which checks if R is a constant
and, if so, redirects the call to the appropriate code for special
registers such as EFLAG.

  --david

On 7/28/05, Magenheimer, Dan (HP Labs Fort Collins)
<dan.magenheimer@hp.com> wrote:
> Revision 4.  Minor formatting change suggested by
> Christoph.  Also, no change for "symmetry" suggested
> by Christoph as Tony wanted to leave as is.
> 
> Tony I think this is now ready to apply.  If there is
> anything else needed, please let me know.
> 
> Thanks,
> Dan Magenheimer
> 
> > -----Original Message-----
> > From: Magenheimer, Dan (HP Labs Fort Collins)
> > Sent: Thursday, July 14, 2005 3:18 PM
> > To: 'linux-ia64@vger.kernel.org'
> > Subject: RE: Xen and the Art of Linux/ia64 Virtualization
> >
> > Revision 3.  Incorporates some more feedback and
> > a minor bug fix.
> >
> > Dan
> >
> > > -----Original Message-----
> > > From: Magenheimer, Dan (HP Labs Fort Collins)
> > > Sent: Wednesday, July 06, 2005 2:47 PM
> > > To: 'linux-ia64@vger.kernel.org'
> > > Subject: RE: Xen and the Art of Linux/ia64 Virtualization
> > >
> > > The patch I posted yesterday had a couple of bugs.
> > > This version applies cleanly against 2.6.12 and
> > > (when augmented with additional files from various
> > > Xen subdirectories) has been booted both on Xen/ia64
> > > and on hardware.
> > >
> > > Note that a patch to drivers/acpi/motherboard.c (that
> > > allows for acpi to be enabled on a stubbed acpi tree
> > > without a kernel null pointer dereference!) is required
> > > and has been submitted separately to the linux-acpi list.
> > >
> > > With the exception of three short CONFIG_XEN
> > > ifdefs, the vast majority of changes in this patch
> > > are code rearrangement to enable a number of routines
> > > and defines to add one level of abstraction.
> > >
> > > Comments and feedback would be much appreciated!
> > >
> > >  arch/ia64/Kconfig             |    7 +
> > >  arch/ia64/Makefile            |    1
> > >  arch/ia64/hp/sim/Makefile     |    2
> > >  arch/ia64/ia32/elfcore32.h    |    2
> > >  arch/ia64/ia32/ia32_signal.c  |    6 -
> > >  arch/ia64/ia32/ia32_support.c |    4 -
> > >  arch/ia64/kernel/entry.S      |   30 ++++---
> > >  arch/ia64/kernel/head.S       |    4 +
> > >  arch/ia64/kernel/irq_ia64.c   |   12 +--
> > >  arch/ia64/kernel/pal.S        |    5 -
> > >  arch/ia64/kernel/setup.c      |    3
> > >  include/asm-ia64/delay.h      |   53 -------------
> > >  include/asm-ia64/privop.h     |  160
> > > ++++++++++++++++++++++++++++++++++++++++++
> > >  include/asm-ia64/processor.h  |   56 --------------
> > >  include/asm-ia64/system.h     |   12 +--
> > >  15 files changed, 217 insertions(+), 140 deletions(-)
> > >
> > > > -----Original Message-----
> > > > From: Magenheimer, Dan (HP Labs Fort Collins)
> > > > Sent: Tuesday, July 05, 2005 2:21 PM
> > > > To: linux-ia64@vger.kernel.org
> > > > Subject: RE: Xen and the Art of Linux/ia64 Virtualization
> > > >
> > > > Thanks to excellent feedback from David Mosberger and
> > > > Christophe Hellwig, I have greatly cleaned up the attached
> > > > patch by using some very nice abstractions.  In fact,
> > > > most of the bulk in the attached patch results from
> > > > moving some code from asm-ia64/{delay,processor}.h to
> > > > a new file, asm-ia64/privop.h.  And the number of
> > > > CONFIG_XEN ifdefs is dramatically reduced.
> > > >
> > > > More feedback appreciated (including comments about how
> > > > close this might be to be ready for submission to Tony).
> > > >
> > > > Thanks,
> > > > Dan Magenheimer
> > > >
> > > > P.S. Anybody who is attending OLS who wants to talk about
> > > > Xen (and specifically Xen/ia64)?  I'll be there... send
> > > > me email.
> > > >
> > >
> >
> 
> 
> 


-- 
Mosberger Consulting LLC, voice/fax: 510-744-9372,
http://www.mosberger-consulting.com/
35706 Runckel Lane, Fremont, CA 94536

^ permalink raw reply	[flat|nested] 21+ messages in thread

* RE: Xen and the Art of Linux/ia64 Virtualization
  2005-06-29 20:16 Xen and the Art of Linux/ia64 Virtualization Magenheimer, Dan (HP Labs Fort Collins)
                   ` (12 preceding siblings ...)
  2005-07-28 23:13 ` david mosberger
@ 2005-07-28 23:27 ` Magenheimer, Dan (HP Labs Fort Collins)
  2005-08-12 22:09 ` Magenheimer, Dan (HP Labs Fort Collins)
                   ` (5 subsequent siblings)
  19 siblings, 0 replies; 21+ messages in thread
From: Magenheimer, Dan (HP Labs Fort Collins) @ 2005-07-28 23:27 UTC (permalink / raw)
  To: linux-ia64

A reasonable possibility.

There seems to be two different ways to "get" architected
control/application registers, using a named routine
(e.g. ia64_get_kr()) or using the more generic getreg.
I'm not sure which is preferable to you kernel developers
but I think the named routine approach is more clear and
easy to read.  I'd like to make uses of "privilege-sensitive"
instructions and registers _more_ obvious rather than less,
as this should help all virtualization efforts, not just
Xen.

It would certainly minimize change to add a Xen variant
of getreg but I'd argue the proposed approach (changing
getreg usages to named routines) is preferable.

However if there is a consensus to use getreg, I can re-do
the patch.

Dan

> -----Original Message-----
> From: david mosberger [mailto:dmosberger@gmail.com] 
> Sent: Thursday, July 28, 2005 5:13 PM
> To: Magenheimer, Dan (HP Labs Fort Collins)
> Cc: linux-ia64@vger.kernel.org; tony.luck@intel.com
> Subject: Re: Xen and the Art of Linux/ia64 Virtualization
> 
> Hi Dan,
> 
> I was hoping someone else would bring this up, but alas....
> 
> Why do you need to split getreg into get_eflag etc?  By definition,
> wherever you call get_eflag, you have a getreg(R) call with R a
> constant (and, in this case, R=_IA64_REG_AR_EFLAG).  Thus, you can
> just have a Xen variant of getreg() which checks if R is a constant
> and, if so, redirects the call to the appropriate code for special
> registers such as EFLAG.
> 
>   --david
> 
> On 7/28/05, Magenheimer, Dan (HP Labs Fort Collins)
> <dan.magenheimer@hp.com> wrote:
> > Revision 4.  Minor formatting change suggested by
> > Christoph.  Also, no change for "symmetry" suggested
> > by Christoph as Tony wanted to leave as is.
> > 
> > Tony I think this is now ready to apply.  If there is
> > anything else needed, please let me know.
> > 
> > Thanks,
> > Dan Magenheimer
> > 
> > > -----Original Message-----
> > > From: Magenheimer, Dan (HP Labs Fort Collins)
> > > Sent: Thursday, July 14, 2005 3:18 PM
> > > To: 'linux-ia64@vger.kernel.org'
> > > Subject: RE: Xen and the Art of Linux/ia64 Virtualization
> > >
> > > Revision 3.  Incorporates some more feedback and
> > > a minor bug fix.
> > >
> > > Dan
> > >
> > > > -----Original Message-----
> > > > From: Magenheimer, Dan (HP Labs Fort Collins)
> > > > Sent: Wednesday, July 06, 2005 2:47 PM
> > > > To: 'linux-ia64@vger.kernel.org'
> > > > Subject: RE: Xen and the Art of Linux/ia64 Virtualization
> > > >
> > > > The patch I posted yesterday had a couple of bugs.
> > > > This version applies cleanly against 2.6.12 and
> > > > (when augmented with additional files from various
> > > > Xen subdirectories) has been booted both on Xen/ia64
> > > > and on hardware.
> > > >
> > > > Note that a patch to drivers/acpi/motherboard.c (that
> > > > allows for acpi to be enabled on a stubbed acpi tree
> > > > without a kernel null pointer dereference!) is required
> > > > and has been submitted separately to the linux-acpi list.
> > > >
> > > > With the exception of three short CONFIG_XEN
> > > > ifdefs, the vast majority of changes in this patch
> > > > are code rearrangement to enable a number of routines
> > > > and defines to add one level of abstraction.
> > > >
> > > > Comments and feedback would be much appreciated!
> > > >
> > > >  arch/ia64/Kconfig             |    7 +
> > > >  arch/ia64/Makefile            |    1
> > > >  arch/ia64/hp/sim/Makefile     |    2
> > > >  arch/ia64/ia32/elfcore32.h    |    2
> > > >  arch/ia64/ia32/ia32_signal.c  |    6 -
> > > >  arch/ia64/ia32/ia32_support.c |    4 -
> > > >  arch/ia64/kernel/entry.S      |   30 ++++---
> > > >  arch/ia64/kernel/head.S       |    4 +
> > > >  arch/ia64/kernel/irq_ia64.c   |   12 +--
> > > >  arch/ia64/kernel/pal.S        |    5 -
> > > >  arch/ia64/kernel/setup.c      |    3
> > > >  include/asm-ia64/delay.h      |   53 -------------
> > > >  include/asm-ia64/privop.h     |  160
> > > > ++++++++++++++++++++++++++++++++++++++++++
> > > >  include/asm-ia64/processor.h  |   56 --------------
> > > >  include/asm-ia64/system.h     |   12 +--
> > > >  15 files changed, 217 insertions(+), 140 deletions(-)
> > > >
> > > > > -----Original Message-----
> > > > > From: Magenheimer, Dan (HP Labs Fort Collins)
> > > > > Sent: Tuesday, July 05, 2005 2:21 PM
> > > > > To: linux-ia64@vger.kernel.org
> > > > > Subject: RE: Xen and the Art of Linux/ia64 Virtualization
> > > > >
> > > > > Thanks to excellent feedback from David Mosberger and
> > > > > Christophe Hellwig, I have greatly cleaned up the attached
> > > > > patch by using some very nice abstractions.  In fact,
> > > > > most of the bulk in the attached patch results from
> > > > > moving some code from asm-ia64/{delay,processor}.h to
> > > > > a new file, asm-ia64/privop.h.  And the number of
> > > > > CONFIG_XEN ifdefs is dramatically reduced.
> > > > >
> > > > > More feedback appreciated (including comments about how
> > > > > close this might be to be ready for submission to Tony).
> > > > >
> > > > > Thanks,
> > > > > Dan Magenheimer
> > > > >
> > > > > P.S. Anybody who is attending OLS who wants to talk about
> > > > > Xen (and specifically Xen/ia64)?  I'll be there... send
> > > > > me email.
> > > > >
> > > >
> > >
> > 
> > 
> > 
> 
> 
> -- 
> Mosberger Consulting LLC, voice/fax: 510-744-9372,
> http://www.mosberger-consulting.com/
> 35706 Runckel Lane, Fremont, CA 94536
> 

^ permalink raw reply	[flat|nested] 21+ messages in thread

* RE: Xen and the Art of Linux/ia64 Virtualization
  2005-06-29 20:16 Xen and the Art of Linux/ia64 Virtualization Magenheimer, Dan (HP Labs Fort Collins)
                   ` (13 preceding siblings ...)
  2005-07-28 23:27 ` Magenheimer, Dan (HP Labs Fort Collins)
@ 2005-08-12 22:09 ` Magenheimer, Dan (HP Labs Fort Collins)
  2005-08-22 21:45 ` Arun Sharma
                   ` (4 subsequent siblings)
  19 siblings, 0 replies; 21+ messages in thread
From: Magenheimer, Dan (HP Labs Fort Collins) @ 2005-08-12 22:09 UTC (permalink / raw)
  To: linux-ia64

[-- Attachment #1: Type: text/plain, Size: 4582 bytes --]

Revision 5.  Incorporates abstraction changes to ia64_getreg
and ia64_setreg as suggested by David Mosberger and Tony
Luck.  Also generalized a bit so that it will be easy to
support other virtualization software that may come along.
This patch is currently against 2.6.12 (thus for review only).

Signed-off-by: Dan Magenheimer <dan.magenheimer@hp.com>

Tony, I think this is now ready to go.  I suppose the best
thing would be for me to regenerate this against 2.6.13
as soon as it comes out?

I've also attached include/asm-ia64/xen/privop.h.  Note
that this is not quite final but is provided for now as
a well-documented (para)virtualization example which
should answer many questions.

Dan

> -----Original Message-----
> From: Magenheimer, Dan (HP Labs Fort Collins) 
> Sent: Thursday, July 28, 2005 4:39 PM
> To: 'linux-ia64@vger.kernel.org'
> Cc: 'tony.luck@intel.com'
> Subject: RE: Xen and the Art of Linux/ia64 Virtualization
> 
> Revision 4.  Minor formatting change suggested by
> Christoph.  Also, no change for "symmetry" suggested
> by Christoph as Tony wanted to leave as is. 
> 
> Tony I think this is now ready to apply.  If there is
> anything else needed, please let me know.
> 
> Thanks,
> Dan Magenheimer
> 
> > -----Original Message-----
> > From: Magenheimer, Dan (HP Labs Fort Collins) 
> > Sent: Thursday, July 14, 2005 3:18 PM
> > To: 'linux-ia64@vger.kernel.org'
> > Subject: RE: Xen and the Art of Linux/ia64 Virtualization
> > 
> > Revision 3.  Incorporates some more feedback and
> > a minor bug fix.
> > 
> > Dan
> > 
> > > -----Original Message-----
> > > From: Magenheimer, Dan (HP Labs Fort Collins) 
> > > Sent: Wednesday, July 06, 2005 2:47 PM
> > > To: 'linux-ia64@vger.kernel.org'
> > > Subject: RE: Xen and the Art of Linux/ia64 Virtualization
> > > 
> > > The patch I posted yesterday had a couple of bugs.
> > > This version applies cleanly against 2.6.12 and
> > > (when augmented with additional files from various
> > > Xen subdirectories) has been booted both on Xen/ia64
> > > and on hardware.
> > > 
> > > Note that a patch to drivers/acpi/motherboard.c (that
> > > allows for acpi to be enabled on a stubbed acpi tree
> > > without a kernel null pointer dereference!) is required
> > > and has been submitted separately to the linux-acpi list.
> > > 
> > > With the exception of three short CONFIG_XEN
> > > ifdefs, the vast majority of changes in this patch
> > > are code rearrangement to enable a number of routines
> > > and defines to add one level of abstraction.
> > > 
> > > Comments and feedback would be much appreciated!
> > > 
> > >  arch/ia64/Kconfig             |    7 +
> > >  arch/ia64/Makefile            |    1 
> > >  arch/ia64/hp/sim/Makefile     |    2 
> > >  arch/ia64/ia32/elfcore32.h    |    2 
> > >  arch/ia64/ia32/ia32_signal.c  |    6 -
> > >  arch/ia64/ia32/ia32_support.c |    4 -
> > >  arch/ia64/kernel/entry.S      |   30 ++++---
> > >  arch/ia64/kernel/head.S       |    4 +
> > >  arch/ia64/kernel/irq_ia64.c   |   12 +--
> > >  arch/ia64/kernel/pal.S        |    5 -
> > >  arch/ia64/kernel/setup.c      |    3 
> > >  include/asm-ia64/delay.h      |   53 -------------
> > >  include/asm-ia64/privop.h     |  160 
> > > ++++++++++++++++++++++++++++++++++++++++++
> > >  include/asm-ia64/processor.h  |   56 --------------
> > >  include/asm-ia64/system.h     |   12 +--
> > >  15 files changed, 217 insertions(+), 140 deletions(-)
> > > 
> > > > -----Original Message-----
> > > > From: Magenheimer, Dan (HP Labs Fort Collins) 
> > > > Sent: Tuesday, July 05, 2005 2:21 PM
> > > > To: linux-ia64@vger.kernel.org
> > > > Subject: RE: Xen and the Art of Linux/ia64 Virtualization
> > > > 
> > > > Thanks to excellent feedback from David Mosberger and
> > > > Christophe Hellwig, I have greatly cleaned up the attached
> > > > patch by using some very nice abstractions.  In fact,
> > > > most of the bulk in the attached patch results from
> > > > moving some code from asm-ia64/{delay,processor}.h to
> > > > a new file, asm-ia64/privop.h.  And the number of
> > > > CONFIG_XEN ifdefs is dramatically reduced.
> > > > 
> > > > More feedback appreciated (including comments about how
> > > > close this might be to be ready for submission to Tony).
> > > > 
> > > > Thanks,
> > > > Dan Magenheimer
> > > > 
> > > > P.S. Anybody who is attending OLS who wants to talk about
> > > > Xen (and specifically Xen/ia64)?  I'll be there... send
> > > > me email.
> > > > 
> > > 
> > 
> 

[-- Attachment #2: xenia64.patch.rev5 --]
[-- Type: application/octet-stream, Size: 21539 bytes --]

diff -Naur linux-2.6.12/arch/ia64/hp/sim/Makefile linux-2.6.12-XenPatch/arch/ia64/hp/sim/Makefile
--- linux-2.6.12/arch/ia64/hp/sim/Makefile	2005-06-17 13:48:29.000000000 -0600
+++ linux-2.6.12-XenPatch/arch/ia64/hp/sim/Makefile	2005-08-12 16:06:00.000000000 -0600
@@ -14,3 +14,5 @@
 obj-$(CONFIG_HP_SIMSERIAL) += simserial.o
 obj-$(CONFIG_HP_SIMSERIAL_CONSOLE) += hpsim_console.o
 obj-$(CONFIG_HP_SIMSCSI) += simscsi.o
+obj-$(CONFIG_XEN) += simserial.o
+obj-$(CONFIG_XEN) += hpsim_console.o
diff -Naur linux-2.6.12/arch/ia64/Kconfig linux-2.6.12-XenPatch/arch/ia64/Kconfig
--- linux-2.6.12/arch/ia64/Kconfig	2005-06-17 13:48:29.000000000 -0600
+++ linux-2.6.12-XenPatch/arch/ia64/Kconfig	2005-08-12 16:06:00.000000000 -0600
@@ -46,6 +46,13 @@
 	bool
 	default y
 
+config XEN
+	bool
+	default n
+	help
+	  Enable Xen hypervisor support.  Resulting kernel runs
+	  both as a guest OS on Xen and natively on hardware.
+
 config SCHED_NO_NO_OMIT_FRAME_POINTER
 	bool
 	default y
diff -Naur linux-2.6.12/arch/ia64/kernel/entry.S linux-2.6.12-XenPatch/arch/ia64/kernel/entry.S
--- linux-2.6.12/arch/ia64/kernel/entry.S	2005-06-17 13:48:29.000000000 -0600
+++ linux-2.6.12-XenPatch/arch/ia64/kernel/entry.S	2005-08-12 16:06:00.000000000 -0600
@@ -181,7 +181,7 @@
  *	called.  The code starting at .map relies on this.  The rest of the code
  *	doesn't care about the interrupt masking status.
  */
-GLOBAL_ENTRY(ia64_switch_to)
+GLOBAL_ENTRY(__ia64_switch_to)
 	.prologue
 	alloc r16=ar.pfs,1,0,0,0
 	DO_SAVE_SWITCH_STACK
@@ -235,7 +235,7 @@
 	;;
 	itr.d dtr[r25]=r23		// wire in new mapping...
 	br.cond.sptk .done
-END(ia64_switch_to)
+END(__ia64_switch_to)
 
 /*
  * Note that interrupts are enabled during save_switch_stack and load_switch_stack.  This
@@ -376,7 +376,7 @@
  *	- b7 holds address to return to
  *	- must not touch r8-r11
  */
-ENTRY(load_switch_stack)
+GLOBAL_ENTRY(load_switch_stack)
 	.prologue
 	.altrp b7
 
@@ -500,7 +500,7 @@
 	 * because some system calls (such as ia64_execve) directly
 	 * manipulate ar.pfs.
 	 */
-GLOBAL_ENTRY(ia64_trace_syscall)
+GLOBAL_ENTRY(__ia64_trace_syscall)
 	PT_REGS_UNWIND_INFO(0)
 	/*
 	 * We need to preserve the scratch registers f6-f11 in case the system
@@ -570,7 +570,7 @@
 (p6)	mov r10=-1
 (p6)	mov r8=r9
 	br.cond.sptk .strace_save_retval
-END(ia64_trace_syscall)
+END(__ia64_trace_syscall)
 
 	/*
 	 * When traced and returning from sigreturn, we invoke syscall_trace but then
@@ -623,8 +623,11 @@
 	adds r2=PT(R8)+16,sp			// r2 = &pt_regs.r8
 	mov r10=r0				// clear error indication in r10
 (p7)	br.cond.spnt handle_syscall_error	// handle potential syscall failure
+	;;
+	// don't fall through, ia64_leave_syscall may be #define'd
+	br.cond.sptk.few ia64_leave_syscall
+	;;
 END(ia64_ret_from_syscall)
-	// fall through
 /*
  * ia64_leave_syscall(): Same as ia64_leave_kernel, except that it doesn't
  *	need to switch to bank 0 and doesn't restore the scratch registers.
@@ -669,7 +672,7 @@
  *	      ar.csd: cleared
  *	      ar.ssd: cleared
  */
-ENTRY(ia64_leave_syscall)
+GLOBAL_ENTRY(__ia64_leave_syscall)
 	PT_REGS_UNWIND_INFO(0)
 	/*
 	 * work.need_resched etc. mustn't get changed by this CPU before it returns to
@@ -770,7 +773,7 @@
 	mov.m ar.ccv=r0		// clear ar.ccv
 (pNonSys) br.cond.dpnt.many dont_preserve_current_frame
 	br.cond.sptk.many rbs_switch
-END(ia64_leave_syscall)
+END(__ia64_leave_syscall)
 
 #ifdef CONFIG_IA32_SUPPORT
 GLOBAL_ENTRY(ia64_ret_from_ia32_execve)
@@ -782,10 +785,13 @@
 	st8.spill [r2]=r8	// store return value in slot for r8 and set unat bit
 	.mem.offset 8,0
 	st8.spill [r3]=r0	// clear error indication in slot for r10 and set unat bit
+	;;
+	// don't fall through, ia64_leave_kernel may be #define'd
+	br.cond.sptk.few ia64_leave_kernel
+	;;
 END(ia64_ret_from_ia32_execve)
-	// fall through
 #endif /* CONFIG_IA32_SUPPORT */
-GLOBAL_ENTRY(ia64_leave_kernel)
+GLOBAL_ENTRY(__ia64_leave_kernel)
 	PT_REGS_UNWIND_INFO(0)
 	/*
 	 * work.need_resched etc. mustn't get changed by this CPU before it returns to
@@ -1131,7 +1137,7 @@
 	ld8 r10=[r3]
 	br.cond.sptk.many .work_processed_syscall	// re-check
 
-END(ia64_leave_kernel)
+END(__ia64_leave_kernel)
 
 ENTRY(handle_syscall_error)
 	/*
@@ -1171,7 +1177,7 @@
 	 * be set up by the caller.  We declare 8 input registers so the system call
 	 * args get preserved, in case we need to restart a system call.
 	 */
-ENTRY(notify_resume_user)
+GLOBAL_ENTRY(notify_resume_user)
 	.prologue ASM_UNW_PRLG_RP|ASM_UNW_PRLG_PFS, ASM_UNW_PRLG_GRSAVE(8)
 	alloc loc1=ar.pfs,8,2,3,0 // preserve all eight input regs in case of syscall restart!
 	mov r9=ar.unat
diff -Naur linux-2.6.12/arch/ia64/kernel/head.S linux-2.6.12-XenPatch/arch/ia64/kernel/head.S
--- linux-2.6.12/arch/ia64/kernel/head.S	2005-06-17 13:48:29.000000000 -0600
+++ linux-2.6.12-XenPatch/arch/ia64/kernel/head.S	2005-08-12 16:06:00.000000000 -0600
@@ -370,6 +370,10 @@
 
 	// This is executed by the bootstrap processor (bsp) only:
 
+#ifdef CONFIG_XEN
+	br.call.sptk.many rp=early_xen_setup
+	;;
+#endif
 #ifdef CONFIG_IA64_FW_EMU
 	// initialize PAL & SAL emulator:
 	br.call.sptk.many rp=sys_fw_init
diff -Naur linux-2.6.12/arch/ia64/kernel/pal.S linux-2.6.12-XenPatch/arch/ia64/kernel/pal.S
--- linux-2.6.12/arch/ia64/kernel/pal.S	2005-06-17 13:48:29.000000000 -0600
+++ linux-2.6.12-XenPatch/arch/ia64/kernel/pal.S	2005-08-12 16:06:00.000000000 -0600
@@ -16,6 +16,7 @@
 #include <asm/processor.h>
 
 	.data
+	.globl pal_entry_point
 pal_entry_point:
 	data8 ia64_pal_default_handler
 	.text
@@ -53,7 +54,7 @@
  * in4	       1 ==> clear psr.ic,  0 ==> don't clear psr.ic
  *
  */
-GLOBAL_ENTRY(ia64_pal_call_static)
+GLOBAL_ENTRY(__ia64_pal_call_static)
 	.prologue ASM_UNW_PRLG_RP|ASM_UNW_PRLG_PFS, ASM_UNW_PRLG_GRSAVE(5)
 	alloc loc1 = ar.pfs,5,5,0,0
 	movl loc2 = pal_entry_point
@@ -90,7 +91,7 @@
 	;;
 	srlz.d				// seralize restoration of psr.l
 	br.ret.sptk.many b0
-END(ia64_pal_call_static)
+END(__ia64_pal_call_static)
 
 /*
  * Make a PAL call using the stacked registers calling convention.
diff -Naur linux-2.6.12/arch/ia64/kernel/setup.c linux-2.6.12-XenPatch/arch/ia64/kernel/setup.c
--- linux-2.6.12/arch/ia64/kernel/setup.c	2005-06-17 13:48:29.000000000 -0600
+++ linux-2.6.12-XenPatch/arch/ia64/kernel/setup.c	2005-08-12 16:06:00.000000000 -0600
@@ -273,6 +273,10 @@
 static inline int __init
 early_console_setup (char *cmdline)
 {
+#ifdef CONFIG_XEN
+	if (!early_xen_console_setup(cmdline))
+		return 0;
+#endif
 #ifdef CONFIG_SERIAL_SGI_L1_CONSOLE
 	{
 		extern int sn_serial_console_early_setup(void);
diff -Naur linux-2.6.12/arch/ia64/Makefile linux-2.6.12-XenPatch/arch/ia64/Makefile
--- linux-2.6.12/arch/ia64/Makefile	2005-06-17 13:48:29.000000000 -0600
+++ linux-2.6.12-XenPatch/arch/ia64/Makefile	2005-08-12 16:06:00.000000000 -0600
@@ -57,6 +57,7 @@
 core-$(CONFIG_IA64_HP_ZX1)	+= arch/ia64/dig/
 core-$(CONFIG_IA64_HP_ZX1_SWIOTLB) += arch/ia64/dig/
 core-$(CONFIG_IA64_SGI_SN2)	+= arch/ia64/sn/
+core-$(CONFIG_XEN)		+= arch/ia64/xen/ arch/ia64/hp/sim/
 
 drivers-$(CONFIG_PCI)		+= arch/ia64/pci/
 drivers-$(CONFIG_IA64_HP_SIM)	+= arch/ia64/hp/sim/
diff -Naur linux-2.6.12/include/asm-ia64/gcc_intrin.h linux-2.6.12-XenPatch/include/asm-ia64/gcc_intrin.h
--- linux-2.6.12/include/asm-ia64/gcc_intrin.h	2005-06-17 13:48:29.000000000 -0600
+++ linux-2.6.12-XenPatch/include/asm-ia64/gcc_intrin.h	2005-08-12 16:06:00.000000000 -0600
@@ -26,7 +26,7 @@
 
 register unsigned long ia64_r13 asm ("r13") __attribute_used__;
 
-#define ia64_setreg(regnum, val)						\
+#define __ia64_setreg(regnum, val)						\
 ({										\
 	switch (regnum) {							\
 	    case _IA64_REG_PSR_L:						\
@@ -55,7 +55,7 @@
 	}									\
 })
 
-#define ia64_getreg(regnum)							\
+#define __ia64_getreg(regnum)							\
 ({										\
 	__u64 ia64_intri_res;							\
 										\
@@ -92,7 +92,7 @@
 
 #define ia64_hint_pause 0
 
-#define ia64_hint(mode)						\
+#define __ia64_hint(mode)						\
 ({								\
 	switch (mode) {						\
 	case ia64_hint_pause:					\
@@ -374,7 +374,7 @@
 
 #define ia64_invala() asm volatile ("invala" ::: "memory")
 
-#define ia64_thash(addr)							\
+#define __ia64_thash(addr)							\
 ({										\
 	__u64 ia64_intri_res;							\
 	asm volatile ("thash %0=%1" : "=r"(ia64_intri_res) : "r" (addr));	\
@@ -394,18 +394,18 @@
 
 #define ia64_nop(x)	asm volatile ("nop %0"::"i"(x));
 
-#define ia64_itci(addr)	asm volatile ("itc.i %0;;" :: "r"(addr) : "memory")
+#define __ia64_itci(addr)	asm volatile ("itc.i %0;;" :: "r"(addr) : "memory")
 
-#define ia64_itcd(addr)	asm volatile ("itc.d %0;;" :: "r"(addr) : "memory")
+#define __ia64_itcd(addr)	asm volatile ("itc.d %0;;" :: "r"(addr) : "memory")
 
 
-#define ia64_itri(trnum, addr) asm volatile ("itr.i itr[%0]=%1"				\
+#define __ia64_itri(trnum, addr) asm volatile ("itr.i itr[%0]=%1"			\
 					     :: "r"(trnum), "r"(addr) : "memory")
 
-#define ia64_itrd(trnum, addr) asm volatile ("itr.d dtr[%0]=%1"				\
+#define __ia64_itrd(trnum, addr) asm volatile ("itr.d dtr[%0]=%1"			\
 					     :: "r"(trnum), "r"(addr) : "memory")
 
-#define ia64_tpa(addr)								\
+#define __ia64_tpa(addr)							\
 ({										\
 	__u64 ia64_pa;								\
 	asm volatile ("tpa %0 = %1" : "=r"(ia64_pa) : "r"(addr) : "memory");	\
@@ -415,22 +415,22 @@
 #define __ia64_set_dbr(index, val)						\
 	asm volatile ("mov dbr[%0]=%1" :: "r"(index), "r"(val) : "memory")
 
-#define ia64_set_ibr(index, val)						\
+#define __ia64_set_ibr(index, val)						\
 	asm volatile ("mov ibr[%0]=%1" :: "r"(index), "r"(val) : "memory")
 
-#define ia64_set_pkr(index, val)						\
+#define __ia64_set_pkr(index, val)						\
 	asm volatile ("mov pkr[%0]=%1" :: "r"(index), "r"(val) : "memory")
 
-#define ia64_set_pmc(index, val)						\
+#define __ia64_set_pmc(index, val)						\
 	asm volatile ("mov pmc[%0]=%1" :: "r"(index), "r"(val) : "memory")
 
-#define ia64_set_pmd(index, val)						\
+#define __ia64_set_pmd(index, val)						\
 	asm volatile ("mov pmd[%0]=%1" :: "r"(index), "r"(val) : "memory")
 
-#define ia64_set_rr(index, val)							\
+#define __ia64_set_rr(index, val)							\
 	asm volatile ("mov rr[%0]=%1" :: "r"(index), "r"(val) : "memory");
 
-#define ia64_get_cpuid(index)								\
+#define __ia64_get_cpuid(index)								\
 ({											\
 	__u64 ia64_intri_res;								\
 	asm volatile ("mov %0=cpuid[%r1]" : "=r"(ia64_intri_res) : "rO"(index));	\
@@ -444,21 +444,21 @@
 	ia64_intri_res;								\
 })
 
-#define ia64_get_ibr(index)							\
+#define __ia64_get_ibr(index)							\
 ({										\
 	__u64 ia64_intri_res;							\
 	asm volatile ("mov %0=ibr[%1]" : "=r"(ia64_intri_res) : "r"(index));	\
 	ia64_intri_res;								\
 })
 
-#define ia64_get_pkr(index)							\
+#define __ia64_get_pkr(index)							\
 ({										\
 	__u64 ia64_intri_res;							\
 	asm volatile ("mov %0=pkr[%1]" : "=r"(ia64_intri_res) : "r"(index));	\
 	ia64_intri_res;								\
 })
 
-#define ia64_get_pmc(index)							\
+#define __ia64_get_pmc(index)							\
 ({										\
 	__u64 ia64_intri_res;							\
 	asm volatile ("mov %0=pmc[%1]" : "=r"(ia64_intri_res) : "r"(index));	\
@@ -466,48 +466,48 @@
 })
 
 
-#define ia64_get_pmd(index)							\
+#define __ia64_get_pmd(index)							\
 ({										\
 	__u64 ia64_intri_res;							\
 	asm volatile ("mov %0=pmd[%1]" : "=r"(ia64_intri_res) : "r"(index));	\
 	ia64_intri_res;								\
 })
 
-#define ia64_get_rr(index)							\
+#define __ia64_get_rr(index)							\
 ({										\
 	__u64 ia64_intri_res;							\
 	asm volatile ("mov %0=rr[%1]" : "=r"(ia64_intri_res) : "r" (index));	\
 	ia64_intri_res;								\
 })
 
-#define ia64_fc(addr)	asm volatile ("fc %0" :: "r"(addr) : "memory")
+#define __ia64_fc(addr)	asm volatile ("fc %0" :: "r"(addr) : "memory")
 
 
 #define ia64_sync_i()	asm volatile (";; sync.i" ::: "memory")
 
-#define ia64_ssm(mask)	asm volatile ("ssm %0":: "i"((mask)) : "memory")
-#define ia64_rsm(mask)	asm volatile ("rsm %0":: "i"((mask)) : "memory")
+#define __ia64_ssm(mask)	asm volatile ("ssm %0":: "i"((mask)) : "memory")
+#define __ia64_rsm(mask)	asm volatile ("rsm %0":: "i"((mask)) : "memory")
 #define ia64_sum(mask)	asm volatile ("sum %0":: "i"((mask)) : "memory")
 #define ia64_rum(mask)	asm volatile ("rum %0":: "i"((mask)) : "memory")
 
-#define ia64_ptce(addr)	asm volatile ("ptc.e %0" :: "r"(addr))
+#define __ia64_ptce(addr)	asm volatile ("ptc.e %0" :: "r"(addr))
 
-#define ia64_ptcga(addr, size)							\
+#define __ia64_ptcga(addr, size)							\
 do {										\
 	asm volatile ("ptc.ga %0,%1" :: "r"(addr), "r"(size) : "memory");	\
 	ia64_dv_serialize_data();						\
 } while (0)
 
-#define ia64_ptcl(addr, size)							\
+#define __ia64_ptcl(addr, size)							\
 do {										\
 	asm volatile ("ptc.l %0,%1" :: "r"(addr), "r"(size) : "memory");	\
 	ia64_dv_serialize_data();						\
 } while (0)
 
-#define ia64_ptri(addr, size)						\
+#define __ia64_ptri(addr, size)						\
 	asm volatile ("ptr.i %0,%1" :: "r"(addr), "r"(size) : "memory")
 
-#define ia64_ptrd(addr, size)						\
+#define __ia64_ptrd(addr, size)						\
 	asm volatile ("ptr.d %0,%1" :: "r"(addr), "r"(size) : "memory")
 
 /* Values for lfhint in ia64_lfetch and ia64_lfetch_fault */
@@ -589,7 +589,7 @@
         }								\
 })
 
-#define ia64_intrin_local_irq_restore(x)			\
+#define __ia64_intrin_local_irq_restore(x)			\
 do {								\
 	asm volatile (";;   cmp.ne p6,p7=%0,r0;;"		\
 		      "(p6) ssm psr.i;"				\
@@ -598,4 +598,6 @@
 		      :: "r"((x)) : "p6", "p7", "memory");	\
 } while (0)
 
+#define __ia64_get_psr_i()	(__ia64_getreg(_IA64_REG_PSR) & 0x4000UL)
+
 #endif /* _ASM_IA64_GCC_INTRIN_H */
diff -Naur linux-2.6.12/include/asm-ia64/intel_intrin.h linux-2.6.12-XenPatch/include/asm-ia64/intel_intrin.h
--- linux-2.6.12/include/asm-ia64/intel_intrin.h	2005-06-17 13:48:29.000000000 -0600
+++ linux-2.6.12-XenPatch/include/asm-ia64/intel_intrin.h	2005-08-12 16:06:00.000000000 -0600
@@ -119,10 +119,10 @@
 		 	 * intrinsic
 		 	 */
 
-#define ia64_getreg		__getReg
-#define ia64_setreg		__setReg
+#define __ia64_getreg		__getReg
+#define __ia64_setreg		__setReg
 
-#define ia64_hint(x)
+#define __ia64_hint(x)
 
 #define ia64_mux1_brcst	 0
 #define ia64_mux1_mix		 8
@@ -135,16 +135,16 @@
 #define ia64_getf_exp		__getf_exp
 #define ia64_shrp		_m64_shrp
 
-#define ia64_tpa		__tpa
+#define __ia64_tpa		__tpa
 #define ia64_invala		__invala
 #define ia64_invala_gr		__invala_gr
 #define ia64_invala_fr		__invala_fr
 #define ia64_nop		__nop
 #define ia64_sum		__sum
-#define ia64_ssm		__ssm
+#define __ia64_ssm		__ssm
 #define ia64_rum		__rum
-#define ia64_rsm		__rsm
-#define ia64_fc 		__fc
+#define __ia64_rsm		__rsm
+#define __ia64_fc 		__fc
 
 #define ia64_ldfs		__ldfs
 #define ia64_ldfd		__ldfd
@@ -182,24 +182,24 @@
 
 #define __ia64_set_dbr(index, val)	\
 		__setIndReg(_IA64_REG_INDR_DBR, index, val)
-#define ia64_set_ibr(index, val)	\
+#define __ia64_set_ibr(index, val)	\
 		__setIndReg(_IA64_REG_INDR_IBR, index, val)
-#define ia64_set_pkr(index, val)	\
+#define __ia64_set_pkr(index, val)	\
 		__setIndReg(_IA64_REG_INDR_PKR, index, val)
-#define ia64_set_pmc(index, val)	\
+#define __ia64_set_pmc(index, val)	\
 		__setIndReg(_IA64_REG_INDR_PMC, index, val)
-#define ia64_set_pmd(index, val)	\
+#define __ia64_set_pmd(index, val)	\
 		__setIndReg(_IA64_REG_INDR_PMD, index, val)
-#define ia64_set_rr(index, val)	\
+#define __ia64_set_rr(index, val)	\
 		__setIndReg(_IA64_REG_INDR_RR, index, val)
 
-#define ia64_get_cpuid(index) 	__getIndReg(_IA64_REG_INDR_CPUID, index)
+#define __ia64_get_cpuid(index) 	__getIndReg(_IA64_REG_INDR_CPUID, index)
 #define __ia64_get_dbr(index) 	__getIndReg(_IA64_REG_INDR_DBR, index)
-#define ia64_get_ibr(index) 	__getIndReg(_IA64_REG_INDR_IBR, index)
-#define ia64_get_pkr(index) 	__getIndReg(_IA64_REG_INDR_PKR, index)
-#define ia64_get_pmc(index) 	__getIndReg(_IA64_REG_INDR_PMC, index)
-#define ia64_get_pmd(index)  	__getIndReg(_IA64_REG_INDR_PMD, index)
-#define ia64_get_rr(index) 	__getIndReg(_IA64_REG_INDR_RR, index)
+#define __ia64_get_ibr(index) 	__getIndReg(_IA64_REG_INDR_IBR, index)
+#define __ia64_get_pkr(index) 	__getIndReg(_IA64_REG_INDR_PKR, index)
+#define __ia64_get_pmc(index) 	__getIndReg(_IA64_REG_INDR_PMC, index)
+#define __ia64_get_pmd(index)  	__getIndReg(_IA64_REG_INDR_PMD, index)
+#define __ia64_get_rr(index) 	__getIndReg(_IA64_REG_INDR_RR, index)
 
 #define ia64_srlz_d		__dsrlz
 #define ia64_srlz_i		__isrlz
@@ -218,18 +218,18 @@
 #define ia64_ld8_acq		__ld8_acq
 
 #define ia64_sync_i		__synci
-#define ia64_thash		__thash
-#define ia64_ttag		__ttag
-#define ia64_itcd		__itcd
-#define ia64_itci		__itci
-#define ia64_itrd		__itrd
-#define ia64_itri		__itri
-#define ia64_ptce		__ptce
-#define ia64_ptcl		__ptcl
-#define ia64_ptcg		__ptcg
-#define ia64_ptcga		__ptcga
-#define ia64_ptri		__ptri
-#define ia64_ptrd		__ptrd
+#define __ia64_thash		__thash
+#define __ia64_ttag		__ttag
+#define __ia64_itcd		__itcd
+#define __ia64_itci		__itci
+#define __ia64_itrd		__itrd
+#define __ia64_itri		__itri
+#define __ia64_ptce		__ptce
+#define __ia64_ptcl		__ptcl
+#define __ia64_ptcg		__ptcg
+#define __ia64_ptcga		__ptcga
+#define __ia64_ptri		__ptri
+#define __ia64_ptrd		__ptrd
 #define ia64_dep_mi		_m64_dep_mi
 
 /* Values for lfhint in __lfetch and __lfetch_fault */
@@ -244,14 +244,16 @@
 #define ia64_lfetch_fault	__lfetch_fault
 #define ia64_lfetch_fault_excl	__lfetch_fault_excl
 
-#define ia64_intrin_local_irq_restore(x)		\
+#define __ia64_intrin_local_irq_restore(x)		\
 do {							\
 	if ((x) != 0) {					\
-		ia64_ssm(IA64_PSR_I);			\
+		__ia64_ssm(IA64_PSR_I);			\
 		ia64_srlz_d();				\
 	} else {					\
-		ia64_rsm(IA64_PSR_I);			\
+		__ia64_rsm(IA64_PSR_I);			\
 	}						\
 } while (0)
 
+#define __ia64_get_psr_i()	(__ia64_getreg(_IA64_REG_PSR) & 0x4000UL)
+
 #endif /* _ASM_IA64_INTEL_INTRIN_H */
diff -Naur linux-2.6.12/include/asm-ia64/pal.h linux-2.6.12-XenPatch/include/asm-ia64/pal.h
--- linux-2.6.12/include/asm-ia64/pal.h	2005-06-17 13:48:29.000000000 -0600
+++ linux-2.6.12-XenPatch/include/asm-ia64/pal.h	2005-08-12 16:06:00.000000000 -0600
@@ -79,6 +79,7 @@
 #ifndef __ASSEMBLY__
 
 #include <linux/types.h>
+#include <asm/processor.h>
 #include <asm/fpu.h>
 
 /*
diff -Naur linux-2.6.12/include/asm-ia64/privop.h linux-2.6.12-XenPatch/include/asm-ia64/privop.h
--- linux-2.6.12/include/asm-ia64/privop.h	1969-12-31 17:00:00.000000000 -0700
+++ linux-2.6.12-XenPatch/include/asm-ia64/privop.h	2005-08-12 16:06:35.000000000 -0600
@@ -0,0 +1,60 @@
+#ifndef _ASM_IA64_PRIVOP_H
+#define _ASM_IA64_PRIVOP_H
+
+/*
+ * Copyright (C) 2005 Hewlett-Packard Co
+ *	Dan Magenheimer <dan.magenheimer@hp.com>
+ *
+ */
+
+#include <linux/config.h>
+#ifdef CONFIG_XEN
+#include <asm/xen/privop.h>
+#endif
+
+#ifndef __ASSEMBLY
+
+#ifndef IA64_PARAVIRTUALIZED
+
+#define ia64_getreg			__ia64_getreg
+#define ia64_setreg			__ia64_setreg
+#define ia64_hint			__ia64_hint
+#define ia64_thash			__ia64_thash
+#define ia64_itci			__ia64_itci
+#define ia64_itcd			__ia64_itcd
+#define ia64_itri			__ia64_itri
+#define ia64_itrd			__ia64_itrd
+#define ia64_tpa			__ia64_tpa
+#define ia64_set_ibr			__ia64_set_ibr
+#define ia64_set_pkr			__ia64_set_pkr
+#define ia64_set_pmc			__ia64_set_pmc
+#define ia64_set_pmd			__ia64_set_pmd
+#define ia64_set_rr			__ia64_set_rr
+#define ia64_get_cpuid			__ia64_get_cpuid
+#define ia64_get_ibr			__ia64_get_ibr
+#define ia64_get_pkr			__ia64_get_pkr
+#define ia64_get_pmc			__ia64_get_pmc
+#define ia64_get_pmd			__ia64_get_pmd
+#define ia64_get_rr			__ia64_get_rr
+#define ia64_fc				__ia64_fc
+#define ia64_ssm			__ia64_ssm
+#define ia64_rsm			__ia64_rsm
+#define ia64_ptce			__ia64_ptce
+#define ia64_ptcga			__ia64_ptcga
+#define ia64_ptcl			__ia64_ptcl
+#define ia64_ptri			__ia64_ptri
+#define ia64_ptrd			__ia64_ptrd
+#define	ia64_get_psr_i			__ia64_get_psr_i
+#define ia64_intrin_local_irq_restore	__ia64_intrin_local_irq_restore
+#define ia64_pal_halt_light		__ia64_pal_halt_light
+#define	ia64_leave_kernel		__ia64_leave_kernel
+#define	ia64_leave_syscall		__ia64_leave_syscall
+#define	ia64_trace_syscall		__ia64_trace_syscall
+#define	ia64_switch_to			__ia64_switch_to
+#define	ia64_pal_call_static		__ia64_pal_call_static
+
+#endif /* !IA64_PARAVIRTUALIZED */
+
+#endif /* !__ASSEMBLY */
+
+#endif /* _ASM_IA64_PRIVOP_H */
diff -Naur linux-2.6.12/include/asm-ia64/processor.h linux-2.6.12-XenPatch/include/asm-ia64/processor.h
--- linux-2.6.12/include/asm-ia64/processor.h	2005-06-17 13:48:29.000000000 -0600
+++ linux-2.6.12-XenPatch/include/asm-ia64/processor.h	2005-08-12 16:06:00.000000000 -0600
@@ -19,6 +19,7 @@
 #include <asm/kregs.h>
 #include <asm/ptrace.h>
 #include <asm/ustack.h>
+#include <asm/privop.h>
 
 /* Our arch specific arch_init_sched_domain is in arch/ia64/kernel/domain.c */
 #define ARCH_HAS_SCHED_DOMAIN
diff -Naur linux-2.6.12/include/asm-ia64/system.h linux-2.6.12-XenPatch/include/asm-ia64/system.h
--- linux-2.6.12/include/asm-ia64/system.h	2005-06-17 13:48:29.000000000 -0600
+++ linux-2.6.12-XenPatch/include/asm-ia64/system.h	2005-08-12 16:06:00.000000000 -0600
@@ -124,7 +124,7 @@
 #define __local_irq_save(x)			\
 do {						\
 	ia64_stop();				\
-	(x) = ia64_getreg(_IA64_REG_PSR);	\
+	(x) = ia64_get_psr_i();			\
 	ia64_stop();				\
 	ia64_rsm(IA64_PSR_I);			\
 } while (0)
@@ -172,7 +172,7 @@
 #endif /* !CONFIG_IA64_DEBUG_IRQ */
 
 #define local_irq_enable()	({ ia64_stop(); ia64_ssm(IA64_PSR_I); ia64_srlz_d(); })
-#define local_save_flags(flags)	({ ia64_stop(); (flags) = ia64_getreg(_IA64_REG_PSR); })
+#define local_save_flags(flags)	({ ia64_stop(); (flags) = ia64_get_psr_i(); })
 
 #define irqs_disabled()				\
 ({						\

[-- Attachment #3: privop.h --]
[-- Type: application/octet-stream, Size: 10273 bytes --]

#ifndef _ASM_IA64_XEN_PRIVOP_H
#define _ASM_IA64_XEN_PRIVOP_H

/*
 * Copyright (C) 2005 Hewlett-Packard Co
 *	Dan Magenheimer <dan.magenheimer@hp.com>
 *
 * Paravirtualizations of privileged operations for Xen/ia64
 *
 */


#include <asm/xen/asm-xsi-offsets.h>

#define IA64_PARAVIRTUALIZED

#ifndef __ASSEMBLY__

/* The global running_on_xen is initialized to zero but an early
 * check is made to see if the Xen is beneath Linux and if so the
 * global is turned on.  Any code where the semantics differ
 * depending on whether it is running on Xen or not checks this
 * global and acts accordingly.  In this way, the resultant kernel
 * is "transparently paravirtualized", meaning the binary runs
 * without change both on top of Xen and on bare metal.
 * What about performance, you ask?  Yes, there is some very small
 * loss of performance, but because this variable is almost always
 * in cache, the loss is negligible. */
extern int running_on_xen;

/************************************************/
/* Instructions paravirtualized for correctness */
/************************************************/

/* "fc" and "thash" are privilege-sensitive instructions, meaning they
 *  may have different semantics depending on whether they are executed
 *  at PL0 vs PL!=0.  When paravirtualized, these instructions mustn't
 *  be allowed to execute directly, lest incorrect semantics result. */
extern unsigned long xen_fc(unsigned long addr);
#define ia64_fc(addr)			xen_fc((unsigned long)(addr))
extern unsigned long xen_thash(unsigned long addr);
#define ia64_thash(addr)		xen_thash((unsigned long)(addr))
/* Note that "ttag" and "cover" are also privilege-sensitive; "ttag"
 * is not currently used (though it may be in a long-format VHPT system!)
 * and the semantics of cover only change if psr.ic is off which is very
 * rare (and currently non-existent outside of assembly code */

/* There are also privilege-sensitive registers.  These registers are
 * readable at any privilege level but only writable at PL0. */
extern unsigned long xen_get_cpuid(int index);
#define	ia64_get_cpuid(i)		xen_get_cpuid(i)
extern unsigned long xen_get_pmd(int index);
#define	ia64_get_pmd(i)			xen_get_pmd(i)
extern unsigned long xen_get_eflag(void);	/* see xen_ia64_getreg */
extern void xen_set_eflag(unsigned long);	/* see xen_ia64_setreg */

/************************************************/
/* Instructions paravirtualized for performance */
/************************************************/

/* Xen uses memory-mapped virtual privileged registers for access to many
 * performance-sensitive privileged registers.  (The memory location for
 * these is currently a constant defined in asm-xsi-offsets.h but will
 * likely change to be an offset relative to a pointer.)  Some, like the
 * processor status register (psr), are broken up into multiple memory
 * locations.  Others, like "pend", are abstractions based on privileged
 * registers. "Pend" is guaranteed to be set if reading cr.ivr would
 * return a (non-spurious) interrupt. Many of these memory-mapped virtual
 * privileged registers are used in assembly but only a few need to be
 * accessed from C. */
#define xen_get_virtual_psr_i()		(*(int *)(XSI_PSR_I))
#define xen_set_virtual_psr_i(_val)	({ *(int *)(XSI_PSR_I) = _val ? 1:0; })
#define xen_set_virtual_psr_ic(_val)	({ *(int *)(XSI_PSR_IC) = _val ? 1:0; })
#define xen_get_virtual_pend()		(*(int *)(XSI_PEND))

/* Hyperprivops are "break" instructions with a well-defined API.
 * In particular, the virtual psr.ic bit must be off; in this way
 * it is guaranteed to never conflict with a linux break instruction.
 * Normally, this is done in a xen stub but this one is frequent enough
 * that we inline it */
#define	XEN_HYPER_SSM_I			asm("break 0x7");

#define xen_hyper_ssm_i()						\
({									\
	xen_set_virtual_psr_i(0);					\
	xen_set_virtual_psr_ic(0);					\
	XEN_HYPER_SSM_I;						\
})

/* kernel register paravirtualization may soon go away */
#define xen_get_kr(regnum) (((unsigned long *)(XSI_KR0))[regnum])
#define xen_set_kr(regnum,val) ((((unsigned long *)(XSI_KR0))[regnum]) = val)

/* turning off interrupts can be paravirtualized simply by writing
 * to a memory-mapped virtual psr.i bit (implemented as a 16-bit bool) */
#define xen_rsm_i()	xen_set_virtual_psr_i(0)

/* turning on interrupts is a bit more complicated.. write to the
 * memory-mapped virtual psr.i bit first (to avoid race condition),
 * then if any interrupts were pending, we have to execute a hyperprivop
 * to ensure the pending interrupt gets delivered; else we're done! */
#define xen_ssm_i()							\
({									\
	int old = xen_get_virtual_psr_i();				\
	xen_set_virtual_psr_i(1);					\
	if (!old && xen_get_virtual_pend()) xen_hyper_ssm_i();		\
})

#define xen_ia64_intrin_local_irq_restore(x)				\
{									\
     if (running_on_xen) {						\
	if ((x) & IA64_PSR_I) { xen_ssm_i(); }				\
	else { xen_rsm_i(); }						\
    }									\
    else __ia64_intrin_local_irq_restore((x));				\
}

#define	xen_get_psr_i()							\
(									\
	(running_on_xen) ?						\
		(xen_get_virtual_psr_i() ? IA64_PSR_I : 0)		\
		: __ia64_get_psr_i()					\
)

#define xen_ia64_ssm(mask)						\
{									\
	if ((mask)==IA64_PSR_I) {					\
		if (running_on_xen) { xen_ssm_i(); }			\
		else { __ia64_ssm(mask); }				\
	}								\
	else { __ia64_ssm(mask); }					\
}

#define xen_ia64_rsm(mask)						\
{									\
	if ((mask)==IA64_PSR_I) {					\
		if (running_on_xen) { xen_rsm_i(); }			\
		else { __ia64_rsm(mask); }				\
	}								\
	else { __ia64_rsm(mask); }					\
}


/* Although all privileged operations can be left to trap and will
 * be properly handled by Xen, some are frequent enough that we use
 * hyperprivops for performance. */

extern unsigned long xen_get_ivr(void);
extern unsigned long xen_get_tpr(void);
extern void xen_set_itm(unsigned long);
extern void xen_set_tpr(unsigned long);
extern void xen_eoi(void);
extern void xen_set_rr(unsigned long index, unsigned long val);
extern unsigned long xen_get_rr(unsigned long index);

/* Note: It may look wrong to test for running_on_xen in each case.
 * However regnum is always a constant so, as written, the compiler
 * eliminates the switch statement, whereas running_on_xen must be
 * tested dynamically. */
#define xen_ia64_getreg(regnum)						\
({									\
	__u64 ia64_intri_res;						\
									\
	switch(regnum) {						\
	case _IA64_REG_AR_KR0 ... _IA64_REG_AR_KR7:			\
		ia64_intri_res = (running_on_xen) ?			\
			xen_get_kr((regnum-_IA64_REG_AR_KR0)) :		\
			__ia64_getreg(regnum);				\
		break;							\
	case _IA64_REG_CR_IVR:						\
		ia64_intri_res = (running_on_xen) ?			\
			xen_get_ivr() :					\
			__ia64_getreg(regnum);				\
		break;							\
	case _IA64_REG_CR_TPR:						\
		ia64_intri_res = (running_on_xen) ?			\
			xen_get_tpr() :					\
			__ia64_getreg(regnum);				\
		break;							\
	case _IA64_REG_AR_EFLAG:					\
		ia64_intri_res = (running_on_xen) ?			\
			xen_get_eflag() :				\
			__ia64_getreg(regnum);				\
		break;							\
	default:							\
		ia64_intri_res = __ia64_getreg(regnum);			\
		break;							\
	}								\
	ia64_intri_res;							\
})

#define xen_ia64_setreg(regnum,val)					\
({									\
	switch(regnum) {						\
	case _IA64_REG_AR_KR0 ... _IA64_REG_AR_KR7:			\
		(running_on_xen) ?					\
			xen_set_kr((regnum-_IA64_REG_AR_KR0), val) :	\
			__ia64_setreg(regnum,val);			\
		break;							\
	case _IA64_REG_CR_ITM:						\
		(running_on_xen) ?					\
			xen_set_itm(val) :				\
			__ia64_setreg(regnum,val);			\
		break;							\
	case _IA64_REG_CR_TPR:						\
		(running_on_xen) ?					\
			xen_set_tpr(val) :				\
			__ia64_setreg(regnum,val);			\
		break;							\
	case _IA64_REG_CR_EOI:						\
		(running_on_xen) ?					\
			xen_eoi() :					\
			__ia64_setreg(regnum,val);			\
		break;							\
	case _IA64_REG_AR_EFLAG:					\
		(running_on_xen) ?					\
			xen_set_eflag(val) :				\
			__ia64_setreg(regnum,val);			\
		break;							\
	default:							\
		__ia64_setreg(regnum,val);				\
		break;							\
	}								\
})

#define ia64_ssm			xen_ia64_ssm
#define ia64_rsm			xen_ia64_rsm
#define ia64_intrin_local_irq_restore	xen_ia64_intrin_local_irq_restore
#define	ia64_ptcga			xen_ptcga
#define	ia64_set_rr(index,val)		xen_set_rr(index,val)
#define	ia64_get_rr(index)		xen_get_rr(index)
#define ia64_getreg			xen_ia64_getreg
#define ia64_setreg			xen_ia64_setreg
#define	ia64_get_psr_i			xen_get_psr_i

/* the remainder of these are not performance-sensitive so its
 * OK to not paravirtualize and just take a privop trap and emulate */
#define ia64_hint			__ia64_hint
#define ia64_set_pmd			__ia64_set_pmd
#define ia64_itci			__ia64_itci
#define ia64_itcd			__ia64_itcd
#define ia64_itri			__ia64_itri
#define ia64_itrd			__ia64_itrd
#define ia64_tpa			__ia64_tpa
#define ia64_set_ibr			__ia64_set_ibr
#define ia64_set_pkr			__ia64_set_pkr
#define ia64_set_pmc			__ia64_set_pmc
#define ia64_get_ibr			__ia64_get_ibr
#define ia64_get_pkr			__ia64_get_pkr
#define ia64_get_pmc			__ia64_get_pmc
#define ia64_ptce			__ia64_ptce
#define ia64_ptcl			__ia64_ptcl
#define ia64_ptri			__ia64_ptri
#define ia64_ptrd			__ia64_ptrd

#endif /* !__ASSEMBLY__ */

#ifdef __ASSEMBLY__
#define	XEN_HYPER_RFI			break 0x1
#define	XEN_HYPER_RSM_PSR_DT		break 0x2
#define	XEN_HYPER_SSM_PSR_DT		break 0x3
#define	XEN_HYPER_COVER			break 0x4
#define	XEN_HYPER_ITC_D			break 0x5
#define	XEN_HYPER_ITC_I			break 0x6
#define	XEN_HYPER_SSM_I			break 0x7
#define	XEN_HYPER_GET_IVR		break 0x8
#define	XEN_HYPER_GET_TPR		break 0x9
#define	XEN_HYPER_SET_TPR		break 0xa
#define	XEN_HYPER_EOI			break 0xb
#define	XEN_HYPER_SET_ITM		break 0xc
#define	XEN_HYPER_THASH			break 0xd
#define	XEN_HYPER_PTC_GA		break 0xe
#define	XEN_HYPER_ITR_D			break 0xf
#define	XEN_HYPER_GET_RR		break 0x10
#define	XEN_HYPER_SET_RR		break 0x11
#endif

/* these routines utilize privilege-sensitive or performance-sensitive
 * privileged instructions so the code must be replaced with
 * paravirtualized versions */
#define ia64_pal_halt_light		xen_pal_halt_light
#define	ia64_leave_kernel		xen_leave_kernel
#define	ia64_leave_syscall		xen_leave_syscall
#define	ia64_trace_syscall		xen_trace_syscall
#define	ia64_switch_to			xen_switch_to
#define	ia64_pal_call_static		xen_pal_call_static

#endif /* _ASM_IA64_XEN_PRIVOP_H */

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: Xen and the Art of Linux/ia64 Virtualization
  2005-06-29 20:16 Xen and the Art of Linux/ia64 Virtualization Magenheimer, Dan (HP Labs Fort Collins)
                   ` (14 preceding siblings ...)
  2005-08-12 22:09 ` Magenheimer, Dan (HP Labs Fort Collins)
@ 2005-08-22 21:45 ` Arun Sharma
  2005-08-22 22:02 ` Magenheimer, Dan (HP Labs Fort Collins)
                   ` (3 subsequent siblings)
  19 siblings, 0 replies; 21+ messages in thread
From: Arun Sharma @ 2005-08-22 21:45 UTC (permalink / raw)
  To: linux-ia64

Magenheimer, Dan (HP Labs Fort Collins) wrote:

Hi Dan,

> Revision 5.  Incorporates abstraction changes to ia64_getreg
> and ia64_setreg as suggested by David Mosberger and Tony
> Luck.  Also generalized a bit so that it will be easy to
> support other virtualization software that may come along.

You mean at compile time? Runtime would be interesting as well, but that 
would mean moving the macros out of line.

> This patch is currently against 2.6.12 (thus for review only).
> 
> Signed-off-by: Dan Magenheimer <dan.magenheimer@hp.com>
> 

The patch looks pretty clean to me (except for the CONFIG_XEN comment I 
sent to xen-merge). But the bigger question we haven't answered is:

Is an instruction level approach sufficient to build a high performance 
hypervisor?

This question is being actively debated on the x86 side as well.  If the 
answer turns out to be "no" and if after doing some performance 
analysis, we conclude that higher level primitives are needed, then some 
of these changes would not be needed (because those privilege sensitive 
instructions have been replaced by a higher level primitive).

	-Arun

^ permalink raw reply	[flat|nested] 21+ messages in thread

* RE: Xen and the Art of Linux/ia64 Virtualization
  2005-06-29 20:16 Xen and the Art of Linux/ia64 Virtualization Magenheimer, Dan (HP Labs Fort Collins)
                   ` (15 preceding siblings ...)
  2005-08-22 21:45 ` Arun Sharma
@ 2005-08-22 22:02 ` Magenheimer, Dan (HP Labs Fort Collins)
  2005-08-22 22:22 ` Arun Sharma
                   ` (2 subsequent siblings)
  19 siblings, 0 replies; 21+ messages in thread
From: Magenheimer, Dan (HP Labs Fort Collins) @ 2005-08-22 22:02 UTC (permalink / raw)
  To: linux-ia64

Thanks for the review Arun. 

> You mean at compile time? Runtime would be interesting as...

The transparent paravirtualization approach I use could be easily
extended to support more than one hypervisor selected at
runtime.

> But the bigger question we haven't answered is:
> Is an instruction level approach sufficient to build a high 
> performance hypervisor?

Define "high performance hypervisor"... Would "within a
few percent of native" qualify?  Xen/ia64 admittedly hasn't
gone through a wide range of performance tests but
domain0* currently compiles linux at only 4% slower than
native and I expect this to get closer to 2% with some more
work (and without additional changes to the patch).  A domU*
guest will be slower due to I/O overhead but I/O is already
using higher level primitives (the same ones as x86).

Dan

* For those not familiar with Xen terminology:
  domain0: the "host" or "service" virtual machine
  domU: a "guest" virtual machine 

> -----Original Message-----
> From: Arun Sharma [mailto:arun.sharma@intel.com] 
> Sent: Monday, August 22, 2005 3:46 PM
> To: Magenheimer, Dan (HP Labs Fort Collins)
> Cc: linux-ia64@vger.kernel.org
> Subject: Re: Xen and the Art of Linux/ia64 Virtualization
> 
> Magenheimer, Dan (HP Labs Fort Collins) wrote:
> 
> Hi Dan,
> 
> > Revision 5.  Incorporates abstraction changes to ia64_getreg
> > and ia64_setreg as suggested by David Mosberger and Tony
> > Luck.  Also generalized a bit so that it will be easy to
> > support other virtualization software that may come along.
> 
> You mean at compile time? Runtime would be interesting as 
> well, but that 
> would mean moving the macros out of line.
> 
> > This patch is currently against 2.6.12 (thus for review only).
> > 
> > Signed-off-by: Dan Magenheimer <dan.magenheimer@hp.com>
> > 
> 
> The patch looks pretty clean to me (except for the CONFIG_XEN 
> comment I 
> sent to xen-merge). But the bigger question we haven't answered is:
> 
> Is an instruction level approach sufficient to build a high 
> performance 
> hypervisor?
> 
> This question is being actively debated on the x86 side as 
> well.  If the 
> answer turns out to be "no" and if after doing some performance 
> analysis, we conclude that higher level primitives are 
> needed, then some 
> of these changes would not be needed (because those privilege 
> sensitive 
> instructions have been replaced by a higher level primitive).
> 
> 	-Arun
> 

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: Xen and the Art of Linux/ia64 Virtualization
  2005-06-29 20:16 Xen and the Art of Linux/ia64 Virtualization Magenheimer, Dan (HP Labs Fort Collins)
                   ` (16 preceding siblings ...)
  2005-08-22 22:02 ` Magenheimer, Dan (HP Labs Fort Collins)
@ 2005-08-22 22:22 ` Arun Sharma
  2005-08-29 19:12 ` Magenheimer, Dan (HP Labs Fort Collins)
  2005-08-31 15:52 ` Magenheimer, Dan (HP Labs Fort Collins)
  19 siblings, 0 replies; 21+ messages in thread
From: Arun Sharma @ 2005-08-22 22:22 UTC (permalink / raw)
  To: linux-ia64

Magenheimer, Dan (HP Labs Fort Collins) wrote:
> 
> Define "high performance hypervisor"... Would "within a
> few percent of native" qualify?  Xen/ia64 admittedly hasn't
> gone through a wide range of performance tests but
> domain0* currently compiles linux at only 4% slower than
> native and I expect this to get closer to 2% with some more
> work (and without additional changes to the patch).  A domU*
> guest will be slower due to I/O overhead but I/O is already
> using higher level primitives (the same ones as x86).

That's certainly impressive! SMP, more I/O intensive workloads and domU 
would be interesting as well.

I think xenlinux/ia64 is assuming that the memory allocated to dom0 is 
machine contiguous. This is not supported by the balloon driver and the 
netfront driver in drivers/xen. They change the guest physical -> 
machine physical mapping for dom0 at runtime.

Dealing with the non-contiguity might impose some I/O performance 
overheads, but that's orthogonal to the "is the instruction level 
approach sufficient?" or "is it a good first step?" debate.

All I'm saying is that if there is a divergence on this question from 
x86, sharing common driver code might become an issue.

	-Arun

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: Xen and the Art of Linux/ia64 Virtualization
  2005-06-29 20:16 Xen and the Art of Linux/ia64 Virtualization Magenheimer, Dan (HP Labs Fort Collins)
                   ` (17 preceding siblings ...)
  2005-08-22 22:22 ` Arun Sharma
@ 2005-08-29 19:12 ` Magenheimer, Dan (HP Labs Fort Collins)
  2005-08-31 15:52 ` Magenheimer, Dan (HP Labs Fort Collins)
  19 siblings, 0 replies; 21+ messages in thread
From: Magenheimer, Dan (HP Labs Fort Collins) @ 2005-08-29 19:12 UTC (permalink / raw)
  To: linux-ia64

[-- Attachment #1: Type: text/plain, Size: 5251 bytes --]

Final revision, this one regen'd against 2.6.13.
Incorporates some minor feedback from Chris Wright
and Arun Sharma.

Tony, please apply!

Thanks,
Dan Magenheimer
HP Labs

> -----Original Message-----
> From: Magenheimer, Dan (HP Labs Fort Collins) 
> Sent: Friday, August 12, 2005 4:09 PM
> To: 'linux-ia64@vger.kernel.org'
> Cc: 'tony.luck@intel.com'
> Subject: RE: Xen and the Art of Linux/ia64 Virtualization
> 
> Revision 5.  Incorporates abstraction changes to ia64_getreg
> and ia64_setreg as suggested by David Mosberger and Tony
> Luck.  Also generalized a bit so that it will be easy to
> support other virtualization software that may come along.
> This patch is currently against 2.6.12 (thus for review only).
> 
> Signed-off-by: Dan Magenheimer <dan.magenheimer@hp.com>
> 
> Tony, I think this is now ready to go.  I suppose the best
> thing would be for me to regenerate this against 2.6.13
> as soon as it comes out?
> 
> I've also attached include/asm-ia64/xen/privop.h.  Note
> that this is not quite final but is provided for now as
> a well-documented (para)virtualization example which
> should answer many questions.
> 
> Dan
> 
> > -----Original Message-----
> > From: Magenheimer, Dan (HP Labs Fort Collins) 
> > Sent: Thursday, July 28, 2005 4:39 PM
> > To: 'linux-ia64@vger.kernel.org'
> > Cc: 'tony.luck@intel.com'
> > Subject: RE: Xen and the Art of Linux/ia64 Virtualization
> > 
> > Revision 4.  Minor formatting change suggested by
> > Christoph.  Also, no change for "symmetry" suggested
> > by Christoph as Tony wanted to leave as is. 
> > 
> > Tony I think this is now ready to apply.  If there is
> > anything else needed, please let me know.
> > 
> > Thanks,
> > Dan Magenheimer
> > 
> > > -----Original Message-----
> > > From: Magenheimer, Dan (HP Labs Fort Collins) 
> > > Sent: Thursday, July 14, 2005 3:18 PM
> > > To: 'linux-ia64@vger.kernel.org'
> > > Subject: RE: Xen and the Art of Linux/ia64 Virtualization
> > > 
> > > Revision 3.  Incorporates some more feedback and
> > > a minor bug fix.
> > > 
> > > Dan
> > > 
> > > > -----Original Message-----
> > > > From: Magenheimer, Dan (HP Labs Fort Collins) 
> > > > Sent: Wednesday, July 06, 2005 2:47 PM
> > > > To: 'linux-ia64@vger.kernel.org'
> > > > Subject: RE: Xen and the Art of Linux/ia64 Virtualization
> > > > 
> > > > The patch I posted yesterday had a couple of bugs.
> > > > This version applies cleanly against 2.6.12 and
> > > > (when augmented with additional files from various
> > > > Xen subdirectories) has been booted both on Xen/ia64
> > > > and on hardware.
> > > > 
> > > > Note that a patch to drivers/acpi/motherboard.c (that
> > > > allows for acpi to be enabled on a stubbed acpi tree
> > > > without a kernel null pointer dereference!) is required
> > > > and has been submitted separately to the linux-acpi list.
> > > > 
> > > > With the exception of three short CONFIG_XEN
> > > > ifdefs, the vast majority of changes in this patch
> > > > are code rearrangement to enable a number of routines
> > > > and defines to add one level of abstraction.
> > > > 
> > > > Comments and feedback would be much appreciated!
> > > > 
> > > >  arch/ia64/Kconfig             |    7 +
> > > >  arch/ia64/Makefile            |    1 
> > > >  arch/ia64/hp/sim/Makefile     |    2 
> > > >  arch/ia64/ia32/elfcore32.h    |    2 
> > > >  arch/ia64/ia32/ia32_signal.c  |    6 -
> > > >  arch/ia64/ia32/ia32_support.c |    4 -
> > > >  arch/ia64/kernel/entry.S      |   30 ++++---
> > > >  arch/ia64/kernel/head.S       |    4 +
> > > >  arch/ia64/kernel/irq_ia64.c   |   12 +--
> > > >  arch/ia64/kernel/pal.S        |    5 -
> > > >  arch/ia64/kernel/setup.c      |    3 
> > > >  include/asm-ia64/delay.h      |   53 -------------
> > > >  include/asm-ia64/privop.h     |  160 
> > > > ++++++++++++++++++++++++++++++++++++++++++
> > > >  include/asm-ia64/processor.h  |   56 --------------
> > > >  include/asm-ia64/system.h     |   12 +--
> > > >  15 files changed, 217 insertions(+), 140 deletions(-)
> > > > 
> > > > > -----Original Message-----
> > > > > From: Magenheimer, Dan (HP Labs Fort Collins) 
> > > > > Sent: Tuesday, July 05, 2005 2:21 PM
> > > > > To: linux-ia64@vger.kernel.org
> > > > > Subject: RE: Xen and the Art of Linux/ia64 Virtualization
> > > > > 
> > > > > Thanks to excellent feedback from David Mosberger and
> > > > > Christophe Hellwig, I have greatly cleaned up the attached
> > > > > patch by using some very nice abstractions.  In fact,
> > > > > most of the bulk in the attached patch results from
> > > > > moving some code from asm-ia64/{delay,processor}.h to
> > > > > a new file, asm-ia64/privop.h.  And the number of
> > > > > CONFIG_XEN ifdefs is dramatically reduced.
> > > > > 
> > > > > More feedback appreciated (including comments about how
> > > > > close this might be to be ready for submission to Tony).
> > > > > 
> > > > > Thanks,
> > > > > Dan Magenheimer
> > > > > 
> > > > > P.S. Anybody who is attending OLS who wants to talk about
> > > > > Xen (and specifically Xen/ia64)?  I'll be there... send
> > > > > me email.
> > > > > 
> > > > 
> > > 
> > 
> 

[-- Attachment #2: xenia64-2.6.13.patch --]
[-- Type: application/octet-stream, Size: 20979 bytes --]

diff -Naur linux-2.6.13/arch/ia64/Kconfig linux-2.6.13-Xenpatch/arch/ia64/Kconfig
--- linux-2.6.13/arch/ia64/Kconfig	2005-08-28 17:41:01.000000000 -0600
+++ linux-2.6.13-Xenpatch/arch/ia64/Kconfig	2005-08-29 10:39:15.000000000 -0600
@@ -46,6 +46,13 @@
 	bool
 	default y
 
+config XEN
+	bool
+	default n
+	help
+	  Enable Xen hypervisor support.  Resulting kernel runs
+	  both as a guest OS on Xen and natively on hardware.
+
 config SCHED_NO_NO_OMIT_FRAME_POINTER
 	bool
 	default y
diff -Naur linux-2.6.13/arch/ia64/kernel/entry.S linux-2.6.13-Xenpatch/arch/ia64/kernel/entry.S
--- linux-2.6.13/arch/ia64/kernel/entry.S	2005-08-28 17:41:01.000000000 -0600
+++ linux-2.6.13-Xenpatch/arch/ia64/kernel/entry.S	2005-08-29 10:41:55.000000000 -0600
@@ -181,7 +181,7 @@
  *	called.  The code starting at .map relies on this.  The rest of the code
  *	doesn't care about the interrupt masking status.
  */
-GLOBAL_ENTRY(ia64_switch_to)
+GLOBAL_ENTRY(__ia64_switch_to)
 	.prologue
 	alloc r16=ar.pfs,1,0,0,0
 	DO_SAVE_SWITCH_STACK
@@ -235,7 +235,7 @@
 	;;
 	itr.d dtr[r25]=r23		// wire in new mapping...
 	br.cond.sptk .done
-END(ia64_switch_to)
+END(__ia64_switch_to)
 
 /*
  * Note that interrupts are enabled during save_switch_stack and load_switch_stack.  This
@@ -376,7 +376,7 @@
  *	- b7 holds address to return to
  *	- must not touch r8-r11
  */
-ENTRY(load_switch_stack)
+GLOBAL_ENTRY(load_switch_stack)
 	.prologue
 	.altrp b7
 
@@ -488,7 +488,7 @@
 	 * because some system calls (such as ia64_execve) directly
 	 * manipulate ar.pfs.
 	 */
-GLOBAL_ENTRY(ia64_trace_syscall)
+GLOBAL_ENTRY(__ia64_trace_syscall)
 	PT_REGS_UNWIND_INFO(0)
 	/*
 	 * We need to preserve the scratch registers f6-f11 in case the system
@@ -558,7 +558,7 @@
 (p6)	mov r10=-1
 (p6)	mov r8=r9
 	br.cond.sptk .strace_save_retval
-END(ia64_trace_syscall)
+END(__ia64_trace_syscall)
 
 	/*
 	 * When traced and returning from sigreturn, we invoke syscall_trace but then
@@ -611,8 +611,11 @@
 	adds r2=PT(R8)+16,sp			// r2 = &pt_regs.r8
 	mov r10=r0				// clear error indication in r10
 (p7)	br.cond.spnt handle_syscall_error	// handle potential syscall failure
+	;;
+	// don't fall through, ia64_leave_syscall may be #define'd
+	br.cond.sptk.few ia64_leave_syscall
+	;;
 END(ia64_ret_from_syscall)
-	// fall through
 /*
  * ia64_leave_syscall(): Same as ia64_leave_kernel, except that it doesn't
  *	need to switch to bank 0 and doesn't restore the scratch registers.
@@ -657,7 +660,7 @@
  *	      ar.csd: cleared
  *	      ar.ssd: cleared
  */
-ENTRY(ia64_leave_syscall)
+GLOBAL_ENTRY(__ia64_leave_syscall)
 	PT_REGS_UNWIND_INFO(0)
 	/*
 	 * work.need_resched etc. mustn't get changed by this CPU before it returns to
@@ -765,7 +768,7 @@
 	mov.m ar.ssd=r0			// M2   clear ar.ssd
 	mov f11=f0			// F    clear f11
 	br.cond.sptk.many rbs_switch	// B
-END(ia64_leave_syscall)
+END(__ia64_leave_syscall)
 
 #ifdef CONFIG_IA32_SUPPORT
 GLOBAL_ENTRY(ia64_ret_from_ia32_execve)
@@ -777,10 +780,13 @@
 	st8.spill [r2]=r8	// store return value in slot for r8 and set unat bit
 	.mem.offset 8,0
 	st8.spill [r3]=r0	// clear error indication in slot for r10 and set unat bit
+	;;
+	// don't fall through, ia64_leave_kernel may be #define'd
+	br.cond.sptk.few ia64_leave_kernel
+	;;
 END(ia64_ret_from_ia32_execve)
-	// fall through
 #endif /* CONFIG_IA32_SUPPORT */
-GLOBAL_ENTRY(ia64_leave_kernel)
+GLOBAL_ENTRY(__ia64_leave_kernel)
 	PT_REGS_UNWIND_INFO(0)
 	/*
 	 * work.need_resched etc. mustn't get changed by this CPU before it returns to
@@ -1125,7 +1131,7 @@
 	ld8 r10=[r3]
 	br.cond.sptk.many .work_processed_syscall	// re-check
 
-END(ia64_leave_kernel)
+END(__ia64_leave_kernel)
 
 ENTRY(handle_syscall_error)
 	/*
@@ -1165,7 +1171,7 @@
 	 * be set up by the caller.  We declare 8 input registers so the system call
 	 * args get preserved, in case we need to restart a system call.
 	 */
-ENTRY(notify_resume_user)
+GLOBAL_ENTRY(notify_resume_user)
 	.prologue ASM_UNW_PRLG_RP|ASM_UNW_PRLG_PFS, ASM_UNW_PRLG_GRSAVE(8)
 	alloc loc1=ar.pfs,8,2,3,0 // preserve all eight input regs in case of syscall restart!
 	mov r9=ar.unat
diff -Naur linux-2.6.13/arch/ia64/kernel/head.S linux-2.6.13-Xenpatch/arch/ia64/kernel/head.S
--- linux-2.6.13/arch/ia64/kernel/head.S	2005-08-28 17:41:01.000000000 -0600
+++ linux-2.6.13-Xenpatch/arch/ia64/kernel/head.S	2005-08-29 10:39:15.000000000 -0600
@@ -370,6 +370,10 @@
 
 	// This is executed by the bootstrap processor (bsp) only:
 
+#ifdef CONFIG_XEN
+	br.call.sptk.many rp=early_xen_setup
+	;;
+#endif
 #ifdef CONFIG_IA64_FW_EMU
 	// initialize PAL & SAL emulator:
 	br.call.sptk.many rp=sys_fw_init
diff -Naur linux-2.6.13/arch/ia64/kernel/pal.S linux-2.6.13-Xenpatch/arch/ia64/kernel/pal.S
--- linux-2.6.13/arch/ia64/kernel/pal.S	2005-08-28 17:41:01.000000000 -0600
+++ linux-2.6.13-Xenpatch/arch/ia64/kernel/pal.S	2005-08-29 10:39:15.000000000 -0600
@@ -16,6 +16,7 @@
 #include <asm/processor.h>
 
 	.data
+	.globl pal_entry_point
 pal_entry_point:
 	data8 ia64_pal_default_handler
 	.text
@@ -53,7 +54,7 @@
  * in4	       1 ==> clear psr.ic,  0 ==> don't clear psr.ic
  *
  */
-GLOBAL_ENTRY(ia64_pal_call_static)
+GLOBAL_ENTRY(__ia64_pal_call_static)
 	.prologue ASM_UNW_PRLG_RP|ASM_UNW_PRLG_PFS, ASM_UNW_PRLG_GRSAVE(5)
 	alloc loc1 = ar.pfs,5,5,0,0
 	movl loc2 = pal_entry_point
@@ -90,7 +91,7 @@
 	;;
 	srlz.d				// seralize restoration of psr.l
 	br.ret.sptk.many b0
-END(ia64_pal_call_static)
+END(__ia64_pal_call_static)
 
 /*
  * Make a PAL call using the stacked registers calling convention.
diff -Naur linux-2.6.13/arch/ia64/kernel/setup.c linux-2.6.13-Xenpatch/arch/ia64/kernel/setup.c
--- linux-2.6.13/arch/ia64/kernel/setup.c	2005-08-28 17:41:01.000000000 -0600
+++ linux-2.6.13-Xenpatch/arch/ia64/kernel/setup.c	2005-08-29 10:42:39.000000000 -0600
@@ -287,6 +287,10 @@
 {
 	int earlycons = 0;
 
+#ifdef CONFIG_XEN
+	if (!early_xen_console_setup(cmdline))
+		earlycons++;
+#endif
 #ifdef CONFIG_SERIAL_SGI_L1_CONSOLE
 	{
 		extern int sn_serial_console_early_setup(void);
diff -Naur linux-2.6.13/arch/ia64/Makefile linux-2.6.13-Xenpatch/arch/ia64/Makefile
--- linux-2.6.13/arch/ia64/Makefile	2005-08-28 17:41:01.000000000 -0600
+++ linux-2.6.13-Xenpatch/arch/ia64/Makefile	2005-08-29 10:39:15.000000000 -0600
@@ -57,6 +57,7 @@
 core-$(CONFIG_IA64_HP_ZX1)	+= arch/ia64/dig/
 core-$(CONFIG_IA64_HP_ZX1_SWIOTLB) += arch/ia64/dig/
 core-$(CONFIG_IA64_SGI_SN2)	+= arch/ia64/sn/
+core-$(CONFIG_XEN)		+= arch/ia64/xen/
 
 drivers-$(CONFIG_PCI)		+= arch/ia64/pci/
 drivers-$(CONFIG_IA64_HP_SIM)	+= arch/ia64/hp/sim/
diff -Naur linux-2.6.13/include/asm-ia64/gcc_intrin.h linux-2.6.13-Xenpatch/include/asm-ia64/gcc_intrin.h
--- linux-2.6.13/include/asm-ia64/gcc_intrin.h	2005-08-28 17:41:01.000000000 -0600
+++ linux-2.6.13-Xenpatch/include/asm-ia64/gcc_intrin.h	2005-08-29 10:39:15.000000000 -0600
@@ -26,7 +26,7 @@
 
 register unsigned long ia64_r13 asm ("r13") __attribute_used__;
 
-#define ia64_setreg(regnum, val)						\
+#define __ia64_setreg(regnum, val)						\
 ({										\
 	switch (regnum) {							\
 	    case _IA64_REG_PSR_L:						\
@@ -55,7 +55,7 @@
 	}									\
 })
 
-#define ia64_getreg(regnum)							\
+#define __ia64_getreg(regnum)							\
 ({										\
 	__u64 ia64_intri_res;							\
 										\
@@ -92,7 +92,7 @@
 
 #define ia64_hint_pause 0
 
-#define ia64_hint(mode)						\
+#define __ia64_hint(mode)						\
 ({								\
 	switch (mode) {						\
 	case ia64_hint_pause:					\
@@ -374,7 +374,7 @@
 
 #define ia64_invala() asm volatile ("invala" ::: "memory")
 
-#define ia64_thash(addr)							\
+#define __ia64_thash(addr)							\
 ({										\
 	__u64 ia64_intri_res;							\
 	asm volatile ("thash %0=%1" : "=r"(ia64_intri_res) : "r" (addr));	\
@@ -394,18 +394,18 @@
 
 #define ia64_nop(x)	asm volatile ("nop %0"::"i"(x));
 
-#define ia64_itci(addr)	asm volatile ("itc.i %0;;" :: "r"(addr) : "memory")
+#define __ia64_itci(addr)	asm volatile ("itc.i %0;;" :: "r"(addr) : "memory")
 
-#define ia64_itcd(addr)	asm volatile ("itc.d %0;;" :: "r"(addr) : "memory")
+#define __ia64_itcd(addr)	asm volatile ("itc.d %0;;" :: "r"(addr) : "memory")
 
 
-#define ia64_itri(trnum, addr) asm volatile ("itr.i itr[%0]=%1"				\
+#define __ia64_itri(trnum, addr) asm volatile ("itr.i itr[%0]=%1"			\
 					     :: "r"(trnum), "r"(addr) : "memory")
 
-#define ia64_itrd(trnum, addr) asm volatile ("itr.d dtr[%0]=%1"				\
+#define __ia64_itrd(trnum, addr) asm volatile ("itr.d dtr[%0]=%1"			\
 					     :: "r"(trnum), "r"(addr) : "memory")
 
-#define ia64_tpa(addr)								\
+#define __ia64_tpa(addr)							\
 ({										\
 	__u64 ia64_pa;								\
 	asm volatile ("tpa %0 = %1" : "=r"(ia64_pa) : "r"(addr) : "memory");	\
@@ -415,22 +415,22 @@
 #define __ia64_set_dbr(index, val)						\
 	asm volatile ("mov dbr[%0]=%1" :: "r"(index), "r"(val) : "memory")
 
-#define ia64_set_ibr(index, val)						\
+#define __ia64_set_ibr(index, val)						\
 	asm volatile ("mov ibr[%0]=%1" :: "r"(index), "r"(val) : "memory")
 
-#define ia64_set_pkr(index, val)						\
+#define __ia64_set_pkr(index, val)						\
 	asm volatile ("mov pkr[%0]=%1" :: "r"(index), "r"(val) : "memory")
 
-#define ia64_set_pmc(index, val)						\
+#define __ia64_set_pmc(index, val)						\
 	asm volatile ("mov pmc[%0]=%1" :: "r"(index), "r"(val) : "memory")
 
-#define ia64_set_pmd(index, val)						\
+#define __ia64_set_pmd(index, val)						\
 	asm volatile ("mov pmd[%0]=%1" :: "r"(index), "r"(val) : "memory")
 
-#define ia64_set_rr(index, val)							\
+#define __ia64_set_rr(index, val)							\
 	asm volatile ("mov rr[%0]=%1" :: "r"(index), "r"(val) : "memory");
 
-#define ia64_get_cpuid(index)								\
+#define __ia64_get_cpuid(index)								\
 ({											\
 	__u64 ia64_intri_res;								\
 	asm volatile ("mov %0=cpuid[%r1]" : "=r"(ia64_intri_res) : "rO"(index));	\
@@ -444,21 +444,21 @@
 	ia64_intri_res;								\
 })
 
-#define ia64_get_ibr(index)							\
+#define __ia64_get_ibr(index)							\
 ({										\
 	__u64 ia64_intri_res;							\
 	asm volatile ("mov %0=ibr[%1]" : "=r"(ia64_intri_res) : "r"(index));	\
 	ia64_intri_res;								\
 })
 
-#define ia64_get_pkr(index)							\
+#define __ia64_get_pkr(index)							\
 ({										\
 	__u64 ia64_intri_res;							\
 	asm volatile ("mov %0=pkr[%1]" : "=r"(ia64_intri_res) : "r"(index));	\
 	ia64_intri_res;								\
 })
 
-#define ia64_get_pmc(index)							\
+#define __ia64_get_pmc(index)							\
 ({										\
 	__u64 ia64_intri_res;							\
 	asm volatile ("mov %0=pmc[%1]" : "=r"(ia64_intri_res) : "r"(index));	\
@@ -466,48 +466,48 @@
 })
 
 
-#define ia64_get_pmd(index)							\
+#define __ia64_get_pmd(index)							\
 ({										\
 	__u64 ia64_intri_res;							\
 	asm volatile ("mov %0=pmd[%1]" : "=r"(ia64_intri_res) : "r"(index));	\
 	ia64_intri_res;								\
 })
 
-#define ia64_get_rr(index)							\
+#define __ia64_get_rr(index)							\
 ({										\
 	__u64 ia64_intri_res;							\
 	asm volatile ("mov %0=rr[%1]" : "=r"(ia64_intri_res) : "r" (index));	\
 	ia64_intri_res;								\
 })
 
-#define ia64_fc(addr)	asm volatile ("fc %0" :: "r"(addr) : "memory")
+#define __ia64_fc(addr)	asm volatile ("fc %0" :: "r"(addr) : "memory")
 
 
 #define ia64_sync_i()	asm volatile (";; sync.i" ::: "memory")
 
-#define ia64_ssm(mask)	asm volatile ("ssm %0":: "i"((mask)) : "memory")
-#define ia64_rsm(mask)	asm volatile ("rsm %0":: "i"((mask)) : "memory")
+#define __ia64_ssm(mask)	asm volatile ("ssm %0":: "i"((mask)) : "memory")
+#define __ia64_rsm(mask)	asm volatile ("rsm %0":: "i"((mask)) : "memory")
 #define ia64_sum(mask)	asm volatile ("sum %0":: "i"((mask)) : "memory")
 #define ia64_rum(mask)	asm volatile ("rum %0":: "i"((mask)) : "memory")
 
-#define ia64_ptce(addr)	asm volatile ("ptc.e %0" :: "r"(addr))
+#define __ia64_ptce(addr)	asm volatile ("ptc.e %0" :: "r"(addr))
 
-#define ia64_ptcga(addr, size)							\
+#define __ia64_ptcga(addr, size)							\
 do {										\
 	asm volatile ("ptc.ga %0,%1" :: "r"(addr), "r"(size) : "memory");	\
 	ia64_dv_serialize_data();						\
 } while (0)
 
-#define ia64_ptcl(addr, size)							\
+#define __ia64_ptcl(addr, size)							\
 do {										\
 	asm volatile ("ptc.l %0,%1" :: "r"(addr), "r"(size) : "memory");	\
 	ia64_dv_serialize_data();						\
 } while (0)
 
-#define ia64_ptri(addr, size)						\
+#define __ia64_ptri(addr, size)						\
 	asm volatile ("ptr.i %0,%1" :: "r"(addr), "r"(size) : "memory")
 
-#define ia64_ptrd(addr, size)						\
+#define __ia64_ptrd(addr, size)						\
 	asm volatile ("ptr.d %0,%1" :: "r"(addr), "r"(size) : "memory")
 
 /* Values for lfhint in ia64_lfetch and ia64_lfetch_fault */
@@ -589,7 +589,7 @@
         }								\
 })
 
-#define ia64_intrin_local_irq_restore(x)			\
+#define __ia64_intrin_local_irq_restore(x)			\
 do {								\
 	asm volatile (";;   cmp.ne p6,p7=%0,r0;;"		\
 		      "(p6) ssm psr.i;"				\
@@ -598,4 +598,6 @@
 		      :: "r"((x)) : "p6", "p7", "memory");	\
 } while (0)
 
+#define __ia64_get_psr_i()	(__ia64_getreg(_IA64_REG_PSR) & 0x4000UL)
+
 #endif /* _ASM_IA64_GCC_INTRIN_H */
diff -Naur linux-2.6.13/include/asm-ia64/intel_intrin.h linux-2.6.13-Xenpatch/include/asm-ia64/intel_intrin.h
--- linux-2.6.13/include/asm-ia64/intel_intrin.h	2005-08-28 17:41:01.000000000 -0600
+++ linux-2.6.13-Xenpatch/include/asm-ia64/intel_intrin.h	2005-08-29 10:39:15.000000000 -0600
@@ -119,10 +119,10 @@
 		 	 * intrinsic
 		 	 */
 
-#define ia64_getreg		__getReg
-#define ia64_setreg		__setReg
+#define __ia64_getreg		__getReg
+#define __ia64_setreg		__setReg
 
-#define ia64_hint(x)
+#define __ia64_hint(x)
 
 #define ia64_mux1_brcst	 0
 #define ia64_mux1_mix		 8
@@ -135,16 +135,16 @@
 #define ia64_getf_exp		__getf_exp
 #define ia64_shrp		_m64_shrp
 
-#define ia64_tpa		__tpa
+#define __ia64_tpa		__tpa
 #define ia64_invala		__invala
 #define ia64_invala_gr		__invala_gr
 #define ia64_invala_fr		__invala_fr
 #define ia64_nop		__nop
 #define ia64_sum		__sum
-#define ia64_ssm		__ssm
+#define __ia64_ssm		__ssm
 #define ia64_rum		__rum
-#define ia64_rsm		__rsm
-#define ia64_fc 		__fc
+#define __ia64_rsm		__rsm
+#define __ia64_fc 		__fc
 
 #define ia64_ldfs		__ldfs
 #define ia64_ldfd		__ldfd
@@ -182,24 +182,24 @@
 
 #define __ia64_set_dbr(index, val)	\
 		__setIndReg(_IA64_REG_INDR_DBR, index, val)
-#define ia64_set_ibr(index, val)	\
+#define __ia64_set_ibr(index, val)	\
 		__setIndReg(_IA64_REG_INDR_IBR, index, val)
-#define ia64_set_pkr(index, val)	\
+#define __ia64_set_pkr(index, val)	\
 		__setIndReg(_IA64_REG_INDR_PKR, index, val)
-#define ia64_set_pmc(index, val)	\
+#define __ia64_set_pmc(index, val)	\
 		__setIndReg(_IA64_REG_INDR_PMC, index, val)
-#define ia64_set_pmd(index, val)	\
+#define __ia64_set_pmd(index, val)	\
 		__setIndReg(_IA64_REG_INDR_PMD, index, val)
-#define ia64_set_rr(index, val)	\
+#define __ia64_set_rr(index, val)	\
 		__setIndReg(_IA64_REG_INDR_RR, index, val)
 
-#define ia64_get_cpuid(index) 	__getIndReg(_IA64_REG_INDR_CPUID, index)
+#define __ia64_get_cpuid(index) 	__getIndReg(_IA64_REG_INDR_CPUID, index)
 #define __ia64_get_dbr(index) 	__getIndReg(_IA64_REG_INDR_DBR, index)
-#define ia64_get_ibr(index) 	__getIndReg(_IA64_REG_INDR_IBR, index)
-#define ia64_get_pkr(index) 	__getIndReg(_IA64_REG_INDR_PKR, index)
-#define ia64_get_pmc(index) 	__getIndReg(_IA64_REG_INDR_PMC, index)
-#define ia64_get_pmd(index)  	__getIndReg(_IA64_REG_INDR_PMD, index)
-#define ia64_get_rr(index) 	__getIndReg(_IA64_REG_INDR_RR, index)
+#define __ia64_get_ibr(index) 	__getIndReg(_IA64_REG_INDR_IBR, index)
+#define __ia64_get_pkr(index) 	__getIndReg(_IA64_REG_INDR_PKR, index)
+#define __ia64_get_pmc(index) 	__getIndReg(_IA64_REG_INDR_PMC, index)
+#define __ia64_get_pmd(index)  	__getIndReg(_IA64_REG_INDR_PMD, index)
+#define __ia64_get_rr(index) 	__getIndReg(_IA64_REG_INDR_RR, index)
 
 #define ia64_srlz_d		__dsrlz
 #define ia64_srlz_i		__isrlz
@@ -218,18 +218,18 @@
 #define ia64_ld8_acq		__ld8_acq
 
 #define ia64_sync_i		__synci
-#define ia64_thash		__thash
-#define ia64_ttag		__ttag
-#define ia64_itcd		__itcd
-#define ia64_itci		__itci
-#define ia64_itrd		__itrd
-#define ia64_itri		__itri
-#define ia64_ptce		__ptce
-#define ia64_ptcl		__ptcl
-#define ia64_ptcg		__ptcg
-#define ia64_ptcga		__ptcga
-#define ia64_ptri		__ptri
-#define ia64_ptrd		__ptrd
+#define __ia64_thash		__thash
+#define __ia64_ttag		__ttag
+#define __ia64_itcd		__itcd
+#define __ia64_itci		__itci
+#define __ia64_itrd		__itrd
+#define __ia64_itri		__itri
+#define __ia64_ptce		__ptce
+#define __ia64_ptcl		__ptcl
+#define __ia64_ptcg		__ptcg
+#define __ia64_ptcga		__ptcga
+#define __ia64_ptri		__ptri
+#define __ia64_ptrd		__ptrd
 #define ia64_dep_mi		_m64_dep_mi
 
 /* Values for lfhint in __lfetch and __lfetch_fault */
@@ -244,14 +244,16 @@
 #define ia64_lfetch_fault	__lfetch_fault
 #define ia64_lfetch_fault_excl	__lfetch_fault_excl
 
-#define ia64_intrin_local_irq_restore(x)		\
+#define __ia64_intrin_local_irq_restore(x)		\
 do {							\
 	if ((x) != 0) {					\
-		ia64_ssm(IA64_PSR_I);			\
+		__ia64_ssm(IA64_PSR_I);			\
 		ia64_srlz_d();				\
 	} else {					\
-		ia64_rsm(IA64_PSR_I);			\
+		__ia64_rsm(IA64_PSR_I);			\
 	}						\
 } while (0)
 
+#define __ia64_get_psr_i()	(__ia64_getreg(_IA64_REG_PSR) & 0x4000UL)
+
 #endif /* _ASM_IA64_INTEL_INTRIN_H */
diff -Naur linux-2.6.13/include/asm-ia64/pal.h linux-2.6.13-Xenpatch/include/asm-ia64/pal.h
--- linux-2.6.13/include/asm-ia64/pal.h	2005-08-28 17:41:01.000000000 -0600
+++ linux-2.6.13-Xenpatch/include/asm-ia64/pal.h	2005-08-29 10:39:15.000000000 -0600
@@ -79,6 +79,7 @@
 #ifndef __ASSEMBLY__
 
 #include <linux/types.h>
+#include <asm/processor.h>
 #include <asm/fpu.h>
 
 /*
diff -Naur linux-2.6.13/include/asm-ia64/privop.h linux-2.6.13-Xenpatch/include/asm-ia64/privop.h
--- linux-2.6.13/include/asm-ia64/privop.h	1969-12-31 17:00:00.000000000 -0700
+++ linux-2.6.13-Xenpatch/include/asm-ia64/privop.h	2005-08-29 10:39:15.000000000 -0600
@@ -0,0 +1,60 @@
+#ifndef _ASM_IA64_PRIVOP_H
+#define _ASM_IA64_PRIVOP_H
+
+/*
+ * Copyright (C) 2005 Hewlett-Packard Co
+ *	Dan Magenheimer <dan.magenheimer@hp.com>
+ *
+ */
+
+#include <linux/config.h>
+#ifdef CONFIG_XEN
+#include <asm/xen/privop.h>
+#endif
+
+#ifndef __ASSEMBLY
+
+#ifndef IA64_PARAVIRTUALIZED
+
+#define ia64_getreg			__ia64_getreg
+#define ia64_setreg			__ia64_setreg
+#define ia64_hint			__ia64_hint
+#define ia64_thash			__ia64_thash
+#define ia64_itci			__ia64_itci
+#define ia64_itcd			__ia64_itcd
+#define ia64_itri			__ia64_itri
+#define ia64_itrd			__ia64_itrd
+#define ia64_tpa			__ia64_tpa
+#define ia64_set_ibr			__ia64_set_ibr
+#define ia64_set_pkr			__ia64_set_pkr
+#define ia64_set_pmc			__ia64_set_pmc
+#define ia64_set_pmd			__ia64_set_pmd
+#define ia64_set_rr			__ia64_set_rr
+#define ia64_get_cpuid			__ia64_get_cpuid
+#define ia64_get_ibr			__ia64_get_ibr
+#define ia64_get_pkr			__ia64_get_pkr
+#define ia64_get_pmc			__ia64_get_pmc
+#define ia64_get_pmd			__ia64_get_pmd
+#define ia64_get_rr			__ia64_get_rr
+#define ia64_fc				__ia64_fc
+#define ia64_ssm			__ia64_ssm
+#define ia64_rsm			__ia64_rsm
+#define ia64_ptce			__ia64_ptce
+#define ia64_ptcga			__ia64_ptcga
+#define ia64_ptcl			__ia64_ptcl
+#define ia64_ptri			__ia64_ptri
+#define ia64_ptrd			__ia64_ptrd
+#define	ia64_get_psr_i			__ia64_get_psr_i
+#define ia64_intrin_local_irq_restore	__ia64_intrin_local_irq_restore
+#define ia64_pal_halt_light		__ia64_pal_halt_light
+#define	ia64_leave_kernel		__ia64_leave_kernel
+#define	ia64_leave_syscall		__ia64_leave_syscall
+#define	ia64_trace_syscall		__ia64_trace_syscall
+#define	ia64_switch_to			__ia64_switch_to
+#define	ia64_pal_call_static		__ia64_pal_call_static
+
+#endif /* !IA64_PARAVIRTUALIZED */
+
+#endif /* !__ASSEMBLY */
+
+#endif /* _ASM_IA64_PRIVOP_H */
diff -Naur linux-2.6.13/include/asm-ia64/processor.h linux-2.6.13-Xenpatch/include/asm-ia64/processor.h
--- linux-2.6.13/include/asm-ia64/processor.h	2005-08-28 17:41:01.000000000 -0600
+++ linux-2.6.13-Xenpatch/include/asm-ia64/processor.h	2005-08-29 10:39:15.000000000 -0600
@@ -19,6 +19,7 @@
 #include <asm/kregs.h>
 #include <asm/ptrace.h>
 #include <asm/ustack.h>
+#include <asm/privop.h>
 
 /* Our arch specific arch_init_sched_domain is in arch/ia64/kernel/domain.c */
 #define ARCH_HAS_SCHED_DOMAIN
diff -Naur linux-2.6.13/include/asm-ia64/system.h linux-2.6.13-Xenpatch/include/asm-ia64/system.h
--- linux-2.6.13/include/asm-ia64/system.h	2005-08-28 17:41:01.000000000 -0600
+++ linux-2.6.13-Xenpatch/include/asm-ia64/system.h	2005-08-29 10:39:15.000000000 -0600
@@ -124,7 +124,7 @@
 #define __local_irq_save(x)			\
 do {						\
 	ia64_stop();				\
-	(x) = ia64_getreg(_IA64_REG_PSR);	\
+	(x) = ia64_get_psr_i();			\
 	ia64_stop();				\
 	ia64_rsm(IA64_PSR_I);			\
 } while (0)
@@ -172,7 +172,7 @@
 #endif /* !CONFIG_IA64_DEBUG_IRQ */
 
 #define local_irq_enable()	({ ia64_stop(); ia64_ssm(IA64_PSR_I); ia64_srlz_d(); })
-#define local_save_flags(flags)	({ ia64_stop(); (flags) = ia64_getreg(_IA64_REG_PSR); })
+#define local_save_flags(flags)	({ ia64_stop(); (flags) = ia64_get_psr_i(); })
 
 #define irqs_disabled()				\
 ({						\

^ permalink raw reply	[flat|nested] 21+ messages in thread

* RE: Xen and the Art of Linux/ia64 Virtualization
  2005-06-29 20:16 Xen and the Art of Linux/ia64 Virtualization Magenheimer, Dan (HP Labs Fort Collins)
                   ` (18 preceding siblings ...)
  2005-08-29 19:12 ` Magenheimer, Dan (HP Labs Fort Collins)
@ 2005-08-31 15:52 ` Magenheimer, Dan (HP Labs Fort Collins)
  19 siblings, 0 replies; 21+ messages in thread
From: Magenheimer, Dan (HP Labs Fort Collins) @ 2005-08-31 15:52 UTC (permalink / raw)
  To: linux-ia64

> On 8/30/05, Keith Owens <kaos@sgi.com> wrote:
> > Tony, can you give some indication on your plans for my MCA/INIT
> > rewrite?  Will you send it to Linus in the next couple of weeks or
will
> > it have to wait for 2.6.15 to open?

Same question as as Keith but for the virtualization prep patch...
the patch is strictly syntactic with !CONFIG_XEN but
is a necessary foundation for which to build Xen/ia64 on top of
in the future and has been generalized so that it should serve
well for yet-to-be-defined non-Xen paravirtualization approaches.
It's been reviewed by the key Linux/ia64 developers (with
feedback/changes incorporated) and also by Chris Wright who is
working with Linus to get a similar (but orthogonal) set of
changes into Linux/x86.

Is it time yet?

Thanks,
Dan

> Sent: Monday, August 29, 2005 1:12 PM
> To: linux-ia64@vger.kernel.org; tony.luck@intel.com
> Subject: Re: Xen and the Art of Linux/ia64 Virtualization
> 
> Final revision, this one regen'd against 2.6.13.
> Incorporates some minor feedback from Chris Wright
> and Arun Sharma.
> 
> Tony, please apply!
> 
> Thanks,
> Dan Magenheimer
> HP Labs
> 
> > -----Original Message-----
> > From: Magenheimer, Dan (HP Labs Fort Collins) 
> > Sent: Friday, August 12, 2005 4:09 PM
> > To: 'linux-ia64@vger.kernel.org'
> > Cc: 'tony.luck@intel.com'
> > Subject: RE: Xen and the Art of Linux/ia64 Virtualization
> > 
> > Revision 5.  Incorporates abstraction changes to ia64_getreg
> > and ia64_setreg as suggested by David Mosberger and Tony
> > Luck.  Also generalized a bit so that it will be easy to
> > support other virtualization software that may come along.
> > This patch is currently against 2.6.12 (thus for review only).
> > 
> > Signed-off-by: Dan Magenheimer <dan.magenheimer@hp.com>
> > 
> > Tony, I think this is now ready to go.  I suppose the best
> > thing would be for me to regenerate this against 2.6.13
> > as soon as it comes out?
> > 
> > I've also attached include/asm-ia64/xen/privop.h.  Note
> > that this is not quite final but is provided for now as
> > a well-documented (para)virtualization example which
> > should answer many questions.
> > 
> > Dan
> > 
> > > -----Original Message-----
> > > From: Magenheimer, Dan (HP Labs Fort Collins) 
> > > Sent: Thursday, July 28, 2005 4:39 PM
> > > To: 'linux-ia64@vger.kernel.org'
> > > Cc: 'tony.luck@intel.com'
> > > Subject: RE: Xen and the Art of Linux/ia64 Virtualization
> > > 
> > > Revision 4.  Minor formatting change suggested by
> > > Christoph.  Also, no change for "symmetry" suggested
> > > by Christoph as Tony wanted to leave as is. 
> > > 
> > > Tony I think this is now ready to apply.  If there is
> > > anything else needed, please let me know.
> > > 
> > > Thanks,
> > > Dan Magenheimer
> > > 
> > > > -----Original Message-----
> > > > From: Magenheimer, Dan (HP Labs Fort Collins) 
> > > > Sent: Thursday, July 14, 2005 3:18 PM
> > > > To: 'linux-ia64@vger.kernel.org'
> > > > Subject: RE: Xen and the Art of Linux/ia64 Virtualization
> > > > 
> > > > Revision 3.  Incorporates some more feedback and
> > > > a minor bug fix.
> > > > 
> > > > Dan
> > > > 
> > > > > -----Original Message-----
> > > > > From: Magenheimer, Dan (HP Labs Fort Collins) 
> > > > > Sent: Wednesday, July 06, 2005 2:47 PM
> > > > > To: 'linux-ia64@vger.kernel.org'
> > > > > Subject: RE: Xen and the Art of Linux/ia64 Virtualization
> > > > > 
> > > > > The patch I posted yesterday had a couple of bugs.
> > > > > This version applies cleanly against 2.6.12 and
> > > > > (when augmented with additional files from various
> > > > > Xen subdirectories) has been booted both on Xen/ia64
> > > > > and on hardware.
> > > > > 
> > > > > Note that a patch to drivers/acpi/motherboard.c (that
> > > > > allows for acpi to be enabled on a stubbed acpi tree
> > > > > without a kernel null pointer dereference!) is required
> > > > > and has been submitted separately to the linux-acpi list.
> > > > > 
> > > > > With the exception of three short CONFIG_XEN
> > > > > ifdefs, the vast majority of changes in this patch
> > > > > are code rearrangement to enable a number of routines
> > > > > and defines to add one level of abstraction.
> > > > > 
> > > > > Comments and feedback would be much appreciated!
> > > > > 
> > > > >  arch/ia64/Kconfig             |    7 +
> > > > >  arch/ia64/Makefile            |    1 
> > > > >  arch/ia64/hp/sim/Makefile     |    2 
> > > > >  arch/ia64/ia32/elfcore32.h    |    2 
> > > > >  arch/ia64/ia32/ia32_signal.c  |    6 -
> > > > >  arch/ia64/ia32/ia32_support.c |    4 -
> > > > >  arch/ia64/kernel/entry.S      |   30 ++++---
> > > > >  arch/ia64/kernel/head.S       |    4 +
> > > > >  arch/ia64/kernel/irq_ia64.c   |   12 +--
> > > > >  arch/ia64/kernel/pal.S        |    5 -
> > > > >  arch/ia64/kernel/setup.c      |    3 
> > > > >  include/asm-ia64/delay.h      |   53 -------------
> > > > >  include/asm-ia64/privop.h     |  160 
> > > > > ++++++++++++++++++++++++++++++++++++++++++
> > > > >  include/asm-ia64/processor.h  |   56 --------------
> > > > >  include/asm-ia64/system.h     |   12 +--
> > > > >  15 files changed, 217 insertions(+), 140 deletions(-)
> > > > > 
> > > > > > -----Original Message-----
> > > > > > From: Magenheimer, Dan (HP Labs Fort Collins) 
> > > > > > Sent: Tuesday, July 05, 2005 2:21 PM
> > > > > > To: linux-ia64@vger.kernel.org
> > > > > > Subject: RE: Xen and the Art of Linux/ia64 Virtualization
> > > > > > 
> > > > > > Thanks to excellent feedback from David Mosberger and
> > > > > > Christophe Hellwig, I have greatly cleaned up the attached
> > > > > > patch by using some very nice abstractions.  In fact,
> > > > > > most of the bulk in the attached patch results from
> > > > > > moving some code from asm-ia64/{delay,processor}.h to
> > > > > > a new file, asm-ia64/privop.h.  And the number of
> > > > > > CONFIG_XEN ifdefs is dramatically reduced.
> > > > > > 
> > > > > > More feedback appreciated (including comments about how
> > > > > > close this might be to be ready for submission to Tony).
> > > > > > 
> > > > > > Thanks,
> > > > > > Dan Magenheimer
> > > > > > 
> > > > > > P.S. Anybody who is attending OLS who wants to talk about
> > > > > > Xen (and specifically Xen/ia64)?  I'll be there... send
> > > > > > me email.
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 

^ permalink raw reply	[flat|nested] 21+ messages in thread

end of thread, other threads:[~2005-08-31 15:52 UTC | newest]

Thread overview: 21+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2005-06-29 20:16 Xen and the Art of Linux/ia64 Virtualization Magenheimer, Dan (HP Labs Fort Collins)
2005-06-29 20:22 ` Magenheimer, Dan (HP Labs Fort Collins)
2005-07-04 15:24 ` Christoph Hellwig
2005-07-04 18:26 ` Magenheimer, Dan (HP Labs Fort Collins)
2005-07-05 20:20 ` Magenheimer, Dan (HP Labs Fort Collins)
2005-07-06 20:46 ` Magenheimer, Dan (HP Labs Fort Collins)
2005-07-07 22:18 ` Gerald Pfeifer
2005-07-07 23:57 ` Magenheimer, Dan (HP Labs Fort Collins)
2005-07-11 15:09 ` Magenheimer, Dan (HP Labs Fort Collins)
2005-07-14 21:17 ` Magenheimer, Dan (HP Labs Fort Collins)
2005-07-16  1:15 ` Christoph Hellwig
2005-07-16 12:55 ` Magenheimer, Dan (HP Labs Fort Collins)
2005-07-28 22:38 ` Magenheimer, Dan (HP Labs Fort Collins)
2005-07-28 23:13 ` david mosberger
2005-07-28 23:27 ` Magenheimer, Dan (HP Labs Fort Collins)
2005-08-12 22:09 ` Magenheimer, Dan (HP Labs Fort Collins)
2005-08-22 21:45 ` Arun Sharma
2005-08-22 22:02 ` Magenheimer, Dan (HP Labs Fort Collins)
2005-08-22 22:22 ` Arun Sharma
2005-08-29 19:12 ` Magenheimer, Dan (HP Labs Fort Collins)
2005-08-31 15:52 ` Magenheimer, Dan (HP Labs Fort Collins)

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox