Re: Simple patches for Linux as a guest OS in a plex86 VM (please consider)

[Kevin Lawton <kevinlawton2001@yahoo.com>]

[-- Attachment #1: Type: text/plain, Size: 1111 bytes --]

OK, here's my re-submit of patches, after some great
clean-up/simplification ideas from Andrew Morton and Andi Kleen.

There are now no mods to any *.{c,S,h} files.  Here's
a summary of the mods:

  o Added 'Documentation/x86-hal.txt'
  o Simple mods to 'Makefile'
  o Added menu entry to 'arch/i386/Kconfig'
  o Simple mod to 'arch/i386/boot/Makefile'
  o Added 'include/asm-i386/if.h'

Only when compiling after enabling this new option,
the Makefile forces inclusion of the 'if.h' file, which
macro'izes the use of PUSHF/POPF instructions, substituting
a few lines of code which fix the broken-ness of such
logic when running with PVI (protected mode virtual interrupts).
No of this would be necessary if the VME (for vm86) semantics
were carried over to PVI.

Also, I'm including the 'Documentation/x86-hal.txt' to
state my case for getting these mods in sooner than later,
and to familiarize folks with what they're for.  My bad,
not explaining it better the 1st time.

-Kevin

__________________________________________________
Do you Yahoo!?
New DSL Internet Access from SBC & Yahoo!
http://sbc.yahoo.com

[-- Attachment #2: linux-2.5.59-hal.diffs --]
[-- Type: application/octet-stream, Size: 12095 bytes --]

diff -urN linux-2.5.59/Documentation/x86-hal.txt linux-2.5.59-hal/Documentation/x86-hal.txt
--- linux-2.5.59/Documentation/x86-hal.txt	Wed Dec 31 19:00:00 1969
+++ linux-2.5.59-hal/Documentation/x86-hal.txt	Wed Jan 22 18:37:44 2003
@@ -0,0 +1,114 @@
+x86 Hardware Abstraction Layer
+==============================
+
+Maintained by Kevin P. Lawton <kevinlawton2001@yahoo.com>
+Last updated: Wed Jan 22 17:05:05 EST 2003
+
+The goal of the x86 hardware abstraction layer is to provide
+a way to run the mainstream x86 Linux kernel as a guest OS, in
+a lightweight Virtual Machine (VM), provided by a new cut-down
+plex86 port.  For reference, see the section on plex86.
+
+The idea being, to have the ability to configure/compile x86 Linux
+"out-of-the-box", disabling lots of unnecessary host-oriented
+hardware support, and running it unpatched inside a special
+VM container.
+
+Because the VM understands that a VM'able guest OS is running,
+_many_ of the heavy x86 VM techniques are unnessary, and only
+the core set of IO functionality needs to be emulated; things
+like the interrupt controller, timers, straight text-mode
+VGA frame buffer, and other things which are very lightweight
+to emulate.  References to these devices can be easily monitored
+by a VM monitor, without modification to the Linux kernel.
+
+The functionality of heavy IO devices like disk/network, are
+best handled by efficient loadable kernel modules designed for
+Linux as a guest.  These modules can communicate a hardware
+abstraction layer protocol to the host OS outside the VM
+to handle the IO workload independently by the host OS,
+which is likely Linux (of course!).
+
+
+Why a Linux VM?
+===============
+
+  o Ability to have multiple Linux installations on one machine
+    at your disposal, with various software configuations.
+    This is great for developers, and call-in support centers
+    who need to have a matrix of OS/application versions
+    available.
+
+  o Security.  There are tons of applications here.  Firewalling
+    of certain applications, honey-pots, ...  I'll let the
+    security guys fill in this category.
+
+  o Provisioning of server resources.  Having the ability to
+    consolidate servers to optimize the use of each CPU.  And
+    to dynamically spin up servers on-demand.  Lots of
+    consoliation and load-balancing plays here.
+
+  o Ability to log into a specific/personal VM "context"
+    from across the Internet, but to have the VM centrally
+    located (and maintained) on servers.  The "context"
+    is the particular Linux install with all of the
+    user-specific files.
+
+
+What is plex86?
+===============
+
+Plex86 is an x86 virtual machine project.  Though it's a separate
+entity, it is currently a subdirectory of "bochs", an x86 PC
+emulation project which can be found on
+
+  http://bochs.sourceforge.net/
+
+Originally, the goals of plex86 were to run any guest OS
+inside the VM, which required some heavy trickery.  I
+completely gutted plex86, and now it is an incredibly
+small and lightweight x86 VM, meant to run code which
+is "VM friendly".  Or in other words, if you don't need to
+run non-Linux, you don't need 99% of the VM baggage.
+
+Plex86 is not ready yet for running Linux as a guest, but I do
+have it working in tandem with bochs to run Linux user-level
+code inside the VM.
+
+
+How much does it cost?
+======================
+
+Nothing, it's Open Source.  Linux is obviously a GPL'd program.
+Plex86 is an LGPL'd kernel module.  Though it's a separate
+
+
+What kind of modifications to the Linux kernel are needed?
+==========================================================
+
+The ideal answer would be none.  The goal is to run the
+actual x86 codebase, with the aim for this to be usable in
+enterprise-level settings.  Thus, it's much more appealing
+not to have modifications, but to leverage all the
+testing that the main codebase goes through.
+
+So far, there are just a couple x86 instructions which
+can be easily macro'ized to change their behaviour, relating
+to manipulation of the interrupt flag.  This is due to
+a deficiency in the handling of code running with Protected
+mode Virtual Interrupts (PVI).  Thus far, the code impact
+is one extra header file, and a few lines to conditionally
+include it!
+
+I am _really_ hoping these small mods make it into the main
+kernel base before Linux 2.6.0.  I believe there will be
+commercial thrust behind running Linux VMs, and would like
+to see the ability to compile kernel source included with
+the major Linux distros, as-is with no extra patches.
+
+For kernel modules which are developed to facilitate a
+Linux VM communicating HAL protocol to the host Linux,
+I have no religion.  They can be maintained wherever makes
+sense.
+
+-Kevin
diff -urN linux-2.5.59/Makefile linux-2.5.59-hal/Makefile
--- linux-2.5.59/Makefile	Wed Jan 22 12:25:58 2003
+++ linux-2.5.59-hal/Makefile	Wed Jan 22 22:36:00 2003
@@ -264,6 +264,18 @@
 CFLAGS		+= -fomit-frame-pointer
 endif
 
+ifdef CONFIG_X86_HAL
+# On x86, if compiling for the Hardware Abstraction Layer
+# (running Linux as a guest OS in a Virtual Machine),
+# we need to insert some asm macros which redefine
+# the behaviour of instructions which modify the
+# interrupt flag.  You are probably not configuring for
+# this mode.  For more info, read 'Documentation/x86-hal.txt'.
+# This needs to be the first include any module sees.
+CFLAGS		+= -include include/asm/if.h
+AFLAGS		+= -include include/asm/if.h
+endif
+
 #
 # INSTALL_PATH specifies where to place the updated kernel and system map
 # images.  Uncomment if you want to place them anywhere other than root.
@@ -409,6 +421,10 @@
 #	their vmlinux.lds.S file
 
 AFLAGS_vmlinux.lds.o += -P -C -U$(ARCH)
+
+ifdef CONFIG_X86_HAL
+AFLAGS_vmlinux.lds.o += -DNO_X86_HAL_INCLUDES
+endif
 
 arch/$(ARCH)/vmlinux.lds.s: %.s: %.S scripts FORCE
 	$(call if_changed_dep,as_s_S)
diff -urN linux-2.5.59/arch/i386/Kconfig linux-2.5.59-hal/arch/i386/Kconfig
--- linux-2.5.59/arch/i386/Kconfig	Wed Jan 22 12:25:59 2003
+++ linux-2.5.59-hal/arch/i386/Kconfig	Wed Jan 22 23:41:11 2003
@@ -1661,3 +1661,20 @@
 	bool
 	depends on SMP
 	default y
+
+menu "Hardware Abstraction Layer"
+
+config X86_HAL
+	bool "Compile for HAL (experimental)"
+	depends on !HIGHMEM && !SMP && !M486 && !M386 && !X86_UP_APIC && !X86_UP_IOAPIC
+	help
+    Experimental: Say Y here _only_ if you are compiling Linux to run inside
+    an experimental Hardware Abstraction Layer.  This is only useful
+    for running Linux as a guest OS in a special virtual machine.
+    Say N here unless you absolutely know what this is for.  This
+    will break Linux running on a real machine for sure.  This option
+    has the following configuration constraints: Pentium or higher,
+    uniprocessor mode, no apic support.  For more info see
+    <file:Documentation/x86-hal.txt>.
+
+endmenu
diff -urN linux-2.5.59/arch/i386/boot/Makefile linux-2.5.59-hal/arch/i386/boot/Makefile
--- linux-2.5.59/arch/i386/boot/Makefile	Tue Jan 14 00:58:35 2003
+++ linux-2.5.59-hal/arch/i386/boot/Makefile	Wed Jan 22 22:52:40 2003
@@ -32,6 +32,10 @@
 
 host-progs	:= tools/build
 
+ifdef CONFIG_X86_HAL
+AFLAGS += -DNO_X86_HAL_INCLUDES
+endif
+
 # ---------------------------------------------------------------------------
 
 $(obj)/zImage:  IMAGE_OFFSET := 0x1000
diff -urN linux-2.5.59/include/asm-i386/if.h linux-2.5.59-hal/include/asm-i386/if.h
--- linux-2.5.59/include/asm-i386/if.h	Wed Dec 31 19:00:00 1969
+++ linux-2.5.59-hal/include/asm-i386/if.h	Wed Jan 22 22:35:02 2003
@@ -0,0 +1,157 @@
+#ifndef NO_X86_HAL_INCLUDES
+/*
+ *	Routines to manipulate the interrupt flag (EFLAGS.IF).  Abstracting
+ *	them, makes it easier to compile for a hardware abstraction layer (HAL).
+ *
+ *	(c) 2003 Kevin P. Lawton <kevinlawton2001@yahoo.com>
+ */
+
+#ifndef _LINUX_ASM_IF_H_
+#define _LINUX_ASM_IF_H_
+
+
+/* NOTE: This file is directly included via the "-include include/asm/if.h"
+ * option conditionally added to the compiler flags by the top Makefile when
+ * CONFIG_X86_HAL is defined.  None of the header files have been included
+ * at this point.
+ */
+
+/* For a HAL (Hardware Abstraction Layer) compile, the kernel is executed
+ * at PL=3 using PVI (Protected mode Virtual Interrupts), which are the
+ * analog to VME for v86 code.  For some reason, The Creators stopped short of
+ * implementing proper IF handling for PUSHF/POPF for PVI, but STI/CLI are
+ * fine.  So we have to complete the PVI semantics using the following
+ * sequences.
+ */
+
+/* Shield your eyes.  These macros make it possible to use the same
+ * code for asm or C inline asm, with all the quoting, comma, and
+ * newline issues.  Q1 is for lines with no commas, and Q2 is for lines
+ * with one comma.
+ */
+#ifdef __ASSEMBLY__
+#define Q1(s0)     s0
+#define Q2(s0, s1) s0, s1
+#else
+#define Q1(s0)     #s0 "\n\t"
+#define Q2(s0, s1) #s0 "," #s1 "\n\t"
+#endif
+
+
+#ifndef __ASSEMBLY__
+__asm__ (
+#endif
+
+  Q1(.macro  cli)
+  Q1(.byte 0xfa)     /* cli (native instruction works fine in PVI) */
+  Q1(.endm)
+
+  Q1(.macro  sti)
+  Q1(.byte 0xfb)     /* sti (native instruction works fine in PVI) */
+  Q1(.endm)
+
+#ifndef __ASSEMBLY__
+  );
+#endif
+
+
+
+/* PUSHFL:
+ * When execution is monitored using PVI (Protected mode Virtual Interrupts),
+ * we have to complete the PVI semantics of the PUSHFL instruction, as
+ * per behaviour of VME.
+ *
+ *                        b19           b9
+ * CPU   image of eflags: VIF           IF
+ *                         |
+ *                         +------------+
+ *                                      |
+ *                                      v
+ * Stack image of eflags: VIF           IF
+ *
+ * Notes: VIF on the stack image could be cleared, if it matters.
+ */
+
+#ifndef __ASSEMBLY__
+__asm__ (
+#endif
+
+  Q1(  .macro pushfl)
+  Q1(  .byte 0x9c)                /* (pushfl) Final eflags stack image. */
+  Q1(  .byte 0x9c)                /* (pushfl) For restoring arith flags. */
+  Q2(  testl $(1<<19), 4(%esp))   /* Was VIF bit set? */
+  Q1(  jz 69001f)
+  Q2(  orl   $(1<<9), 4(%esp))    /* Yes: set stack image of IF. */
+  Q1(  jmp 69002f)
+  Q1(69001:)                      /* Use the zip of Lyon France :^} */
+  Q2(  andl  $~(1<<9), 4(%esp))   /* No: clear stack image of IF. */
+  Q1(69002:)
+  Q1(  .byte 0x9d)                /* (popfl) Restore arithmetic flags. */
+  Q1(  .endm)
+
+  /* The user may use pushf with an implicit size.  Just expand that
+   * to the previous macro.
+   */
+  Q1(.macro pushf)
+  Q1(pushfl)
+  Q1(.endm)
+
+#ifndef __ASSEMBLY__
+  );
+#endif
+
+
+
+/* POPFL:
+ * When execution is monitored using PVI (Protected mode Virtual Interrupts),
+ * we have to complete the PVI semantics of the POPFL instruction, as
+ * per behaviour of VME.
+ *
+ *                        b19           b9
+ * Stack image of eflags: VIF           IF
+ *                                      |
+ *                         +------------+
+ *                         |
+ *                         v
+ * CPU   image of eflags: VIF           IF
+ *
+ * Notes: IF of the eflags register retains its previous value, which
+ *   should be 1 (when monitored down to PL3, the processor ignores this
+ *   bit in a POPF).
+ */
+
+#ifndef __ASSEMBLY__
+__asm__ (
+#endif
+
+  Q1(  .macro popfl)
+  Q2(  testl $(1<<9), 0(%esp))    /* Is IF set on stack image? */
+  Q1(  jz 69003f)
+  Q1(  .byte 0x9d)                /* (popfl) Yes: restore from stack and */
+  Q1(  sti)                       /* force VIF=1. */
+  Q1(  jmp 69004f)
+  Q1(69003:)
+  Q1(  .byte 0x9d)                /* (popfl) No: restore from stack and */
+  Q1(  cli)                       /* force VIF=0. */
+  Q1(69004:)
+  Q1(  .endm)
+
+  /* The user may use popf with an implicit size.  Just expand that
+   * to the previous macro.
+   */
+  Q1(.macro popf)
+  Q1(popfl)
+  Q1(.endm)
+
+#ifndef __ASSEMBLY__
+  );
+#endif
+
+/* Get rid of quoting macros - good housekeeping. */
+#undef Q1
+#undef Q2
+
+
+#endif  /* _LINUX_ASM_IF_H_ */
+
+#endif  /*  NO_X86_HAL_INCLUDES */