From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Barry Silverman" Subject: Really really small xen0 Date: Mon, 8 Nov 2004 09:51:12 -0500 Message-ID: <001101c4c5a2$67272910$6400a8c0@gandalf> Mime-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_0012_01C4C578.7E512110" Return-path: Sender: xen-devel-admin@lists.sourceforge.net Errors-To: xen-devel-admin@lists.sourceforge.net List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , List-Archive: To: xen-devel@lists.sourceforge.net List-Id: xen-devel@lists.xenproject.org This is a multi-part message in MIME format. ------=_NextPart_000_0012_01C4C578.7E512110 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit I have been away from the xen list for quite a while so bear with my current level of ignorance. I was wondering if anyone has made a really minimal xen0 image. By this I mean an image that doesn't have much more than the kernel (f/e and b/e drivers linked in), and run from a crom or squashfs filesystem, and a minimal set of tools running in a busybox-like init process. If necessary, maybe even have a runtime xen0 that just does I/O, and a management privileged domain to run the mgmt tools. ------=_NextPart_000_0012_01C4C578.7E512110 Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

I have been away from the xen list for quite a while so bear with my current level of = ignorance.

I was = wondering if anyone has made a really minimal xen0 image. By this I mean an image = that doesn’t have much more than the kernel (f/e and b/e drivers linked in), and run from a crom or = squashfs filesystem, and a minimal set of tools running in = a busybox-like init process. =

If necessary, maybe even have a runtime xen0 that = just does I/O, and a management privileged domain to run the mgmt tools. =

------=_NextPart_000_0012_01C4C578.7E512110-- ------------------------------------------------------- This SF.Net email is sponsored by: Sybase ASE Linux Express Edition - download now for FREE LinuxWorld Reader's Choice Award Winner for best database on Linux. http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Mark A. Williamson" Subject: Re: Really really small xen0 Date: Mon, 8 Nov 2004 15:20:58 +0000 Message-ID: <200411081520.59245.mark.williamson@cl.cam.ac.uk> References: <001101c4c5a2$67272910$6400a8c0@gandalf> Reply-To: mark.williamson@cl.cam.ac.uk Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <001101c4c5a2$67272910$6400a8c0@gandalf> Content-Disposition: inline Sender: xen-devel-admin@lists.sourceforge.net Errors-To: xen-devel-admin@lists.sourceforge.net List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , List-Archive: To: xen-devel@lists.sourceforge.net Cc: Barry Silverman List-Id: xen-devel@lists.xenproject.org > I was wondering if anyone has made a really minimal xen0 image. By this > I mean an image that doesn't have much more than the kernel (f/e and b/e > drivers linked in), and run from a crom or squashfs filesystem, and a > minimal set of tools running in a busybox-like init process. I did something similar for the restartable device drivers work in the OASIS paper. I cut out everything I could from the kernel config and booted with a tiny ramdisk, containing only a very small init (which Keir wrote to be a few lines of assembler!). This was a bit more minimal than you describe - basically no userspace at all ;-) To run device drivers, the domain shouldn't need any user space at all. Its runtime footprint was about 3megabytes. There's currently no way to configure bridging / routing this way tho - we'd need to add some control interface messages. HTH, Mark ------------------------------------------------------- This SF.Net email is sponsored by: Sybase ASE Linux Express Edition - download now for FREE LinuxWorld Reader's Choice Award Winner for best database on Linux. http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Barry Silverman" Subject: RE: Really really small xen0 Date: Mon, 8 Nov 2004 10:39:20 -0500 Message-ID: <001e01c4c5a9$21004ff0$6400a8c0@gandalf> References: <200411081520.59245.mark.williamson@cl.cam.ac.uk> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <200411081520.59245.mark.williamson@cl.cam.ac.uk> Sender: xen-devel-admin@lists.sourceforge.net Errors-To: xen-devel-admin@lists.sourceforge.net List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , List-Archive: To: mark.williamson@cl.cam.ac.uk, xen-devel@lists.sourceforge.net List-Id: xen-devel@lists.xenproject.org Does it even need an init.exe? or a mounted file system? Couldn't the kernel just jump into a kernel thread xen idle loop? Would it then be possible to have this minimal xen0 linked in to xen? I guess there needs to be some way of communicating the parameters to the xen kernel. Is the ether bridging done in xen? or in domain 0? Maybe it can be driven by an initrd in xen0. Could you then still have another privileged domain do status monitoring, and user domain startup/shutdown? -----Original Message----- From: maw48@hermes.cam.ac.uk [mailto:maw48@hermes.cam.ac.uk] On Behalf Of Mark A. Williamson Sent: Monday, November 08, 2004 10:21 AM To: xen-devel@lists.sourceforge.net Cc: Barry Silverman Subject: Re: [Xen-devel] Really really small xen0 > I was wondering if anyone has made a really minimal xen0 image. By this > I mean an image that doesn't have much more than the kernel (f/e and b/e > drivers linked in), and run from a crom or squashfs filesystem, and a > minimal set of tools running in a busybox-like init process. I did something similar for the restartable device drivers work in the OASIS paper. I cut out everything I could from the kernel config and booted with a tiny ramdisk, containing only a very small init (which Keir wrote to be a few lines of assembler!). This was a bit more minimal than you describe - basically no userspace at all ;-) To run device drivers, the domain shouldn't need any user space at all. Its runtime footprint was about 3megabytes. There's currently no way to configure bridging / routing this way tho - we'd need to add some control interface messages. HTH, Mark ------------------------------------------------------- This SF.Net email is sponsored by: Sybase ASE Linux Express Edition - download now for FREE LinuxWorld Reader's Choice Award Winner for best database on Linux. http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Mark A. Williamson" Subject: Re: Really really small xen0 Date: Mon, 8 Nov 2004 16:08:34 +0000 Message-ID: <200411081608.35569.mark.williamson@cl.cam.ac.uk> References: <001e01c4c5a9$21004ff0$6400a8c0@gandalf> Reply-To: mark.williamson@cl.cam.ac.uk Mime-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <001e01c4c5a9$21004ff0$6400a8c0@gandalf> Content-Disposition: inline Sender: xen-devel-admin@lists.sourceforge.net Errors-To: xen-devel-admin@lists.sourceforge.net List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , List-Archive: To: Barry Silverman Cc: xen-devel@lists.sourceforge.net List-Id: xen-devel@lists.xenproject.org > Does it even need an init.exe? or a mounted file system? These are necessary if you're not going to try hacking in the kernel, purely to stop Linux getting upset. The init doesn't have to do anything and the filesystem can be empty (apart from the init). Using a suitable filesystem, you can make the initrd really small. > Couldn't the > kernel just jump into a kernel thread xen idle loop? Would it then be > possible to have this minimal xen0 linked in to xen? You might be able to hack the init kernel thread to just block but it's not clear there's much advantage over having a userspace init that just blocks. You still might find Linux wants to mount *something* at boot, though. I can check the mini init into the unstable tree as it might be useful to some people someday. > I guess there needs to be some way of communicating the parameters to > the xen kernel. Yeah. For the minimal driver kernels, we had the backend driver in the domain automatically set up bridging, etc. For flexibility and cleanness, we now set this up all from userspace. We already have a generic control message framework for passing messages to other domains from domain 0. We'd need to add some more message types in order to issue bridge setup commands, etc. > Is the ether bridging done in xen? or in domain 0? Xen itself doesn't know / care about devices, the bridging is done in domain 0. The "backend" network driver runs in dom0 and creates a virtual ethernet device to talk to each "frontend" network interface in an unprivileged domain. The standard Linux bridging / routing code is then used to bridge / route those virtual interfaces. So it all happens in dom0. > Maybe > it can be driven by an initrd in xen0. You could put some extra tools in the initrd in the driver domain but you'd still to figure out some way of telling them how to bridge / route new VIFs as domains are started. I suppose you could hack Xend to issue commands to the console interface, although that seems a bit skanky to me ;-) > Could you then still have another privileged domain do status > monitoring, and user domain startup/shutdown? In principal you could get Xen (with some hacking to add the required functionality to XenLinux / Xend) do something like this. This done, you could be able to run a driver for each block / net device in a separate domain and admin stuff in yet another. Many of the pieces required for this are in place now. There is a bit of a chicken and egg problem regarding how these domains get built in the first place (how does the driver domain get built without the admin domain / how does the admin domain load without the driver domain) but it's surmountable (e.g. use some large initrds!). This is quite interesting stuff, it's just not been as high priority as some of the other features in the release. HTH, Mark ------------------------------------------------------- This SF.Net email is sponsored by: Sybase ASE Linux Express Edition - download now for FREE LinuxWorld Reader's Choice Award Winner for best database on Linux. http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Barry Silverman" Subject: RE: Really really small xen0 Date: Mon, 8 Nov 2004 11:19:41 -0500 Message-ID: <002501c4c5ae$c428dc10$6400a8c0@gandalf> References: <200411081608.35569.mark.williamson@cl.cam.ac.uk> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <200411081608.35569.mark.williamson@cl.cam.ac.uk> Sender: xen-devel-admin@lists.sourceforge.net Errors-To: xen-devel-admin@lists.sourceforge.net List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , List-Archive: To: mark.williamson@cl.cam.ac.uk Cc: xen-devel@lists.sourceforge.net List-Id: xen-devel@lists.xenproject.org >>Yeah. For the minimal driver kernels, we had the backend driver in the domain >>automatically set up bridging, etc. For flexibility and cleanness, we now >>set this up all from userspace. >>We already have a generic control message framework for passing messages to >>other domains from domain 0. We'd need to add some more message types in >>order to issue bridge setup commands, etc. Rather than have to carry python and all the rest of the toolset, would it be possible to: 1) Save a pre-built configuration database to send to xen and/or 2) Save a recording of the control message traffic between dom0 and the other domains using a BIG xen0, store the recording in an initrd and then replay it back at startup time? (and log any errors somewhere!!) It would be nice not to have to carry any shared libraries... ------------------------------------------------------- This SF.Net email is sponsored by: Sybase ASE Linux Express Edition - download now for FREE LinuxWorld Reader's Choice Award Winner for best database on Linux. http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jacob Gorm Hansen Subject: Re: Really really small xen0 Date: Tue, 09 Nov 2004 00:21:49 +0100 Message-ID: <418FFF8D.6070103@melon.dk> References: <002501c4c5ae$c428dc10$6400a8c0@gandalf> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <002501c4c5ae$c428dc10$6400a8c0@gandalf> Sender: xen-devel-admin@lists.sourceforge.net Errors-To: xen-devel-admin@lists.sourceforge.net List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , List-Archive: To: Barry Silverman Cc: xen-devel@lists.sourceforge.net List-Id: xen-devel@lists.xenproject.org Barry Silverman wrote: > Rather than have to carry python and all the rest of the toolset, would > it be possible to: > 1) Save a pre-built configuration database to send to xen > and/or > 2) Save a recording of the control message traffic between dom0 and the > other domains using a BIG xen0, store the recording in an initrd and > then replay it back at startup time? (and log any errors somewhere!!) > > It would be nice not to have to carry any shared libraries... I am doing the following for my self-migration stuff: - Building my own domain creation etc tools using straight libxc. All statically linked. Now I can instantiate a new domain in 0.017s. - Sticking with Xen 1.3 which has all the drivers in ring 0, until xend is either massively simplified or rewritten in C. The old way of handling networking was much easier for what I am doing. - Writing my own unprived mini-os, running the tiny UIP TCP-stack, to receive incoming migrations, so that ultimately I can remove dom0 completely. Almost there now, at least I can ping the thing now. :-) Jacob ------------------------------------------------------- This SF.Net email is sponsored by: Sybase ASE Linux Express Edition - download now for FREE LinuxWorld Reader's Choice Award Winner for best database on Linux. http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click From mboxrd@z Thu Jan 1 00:00:00 1970 From: Nathan Lutchansky Subject: Re: Really really small xen0 Date: Mon, 8 Nov 2004 11:39:05 -0500 Message-ID: <20041108163905.GB11808@litech.org> References: <001101c4c5a2$67272910$6400a8c0@gandalf> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="DBIVS5p969aUjpLe" Return-path: Content-Disposition: inline In-Reply-To: <001101c4c5a2$67272910$6400a8c0@gandalf> Sender: xen-devel-admin@lists.sourceforge.net Errors-To: xen-devel-admin@lists.sourceforge.net List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , List-Archive: To: xen-devel@lists.sourceforge.net List-Id: xen-devel@lists.xenproject.org --DBIVS5p969aUjpLe Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Nov 08, 2004 at 09:51:12AM -0500, Barry Silverman wrote: > I was wondering if anyone has made a really minimal xen0 image. By this > I mean an image that doesn't have much more than the kernel (f/e and b/e > drivers linked in), and run from a crom or squashfs filesystem, and a > minimal set of tools running in a busybox-like init process.=20 I've had good luck with similar projects using the uClibc buildroot kit, which is intended for making tiny root filesystems for embedded systems but also works with x86 PC systems. There's not much documentation for it but see the CVSweb at to see what it comes with. It can make an initrd that will boot with exactly the software you want to run and a tmpfs for /tmp, /var and so on, but no changes to the filesystem can be saved. It comes with build scripts for Python and bridge-utils, but you'd have to add Twisted and the XEN tools. I secure my dom0 by only making it accessible over the console/serial port and not even giving it an IP address (except on the loopback IF). It acts as a layer-2 bridge only. This is still vulnerable to security bugs in the hypervisor and VBD/VIF data paths, of course, but it's much better than the typical config. -Nathan --DBIVS5p969aUjpLe Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (GNU/Linux) iD8DBQFBj6EpTviDkW8mhycRAta4AJ9nTmtUZyo4qd8R6HVk3WiKePMHWwCgqf1I 2kbtUpqsK8jE6zSpNvlFjLs= =8/PJ -----END PGP SIGNATURE----- --DBIVS5p969aUjpLe-- ------------------------------------------------------- This SF.Net email is sponsored by: Sybase ASE Linux Express Edition - download now for FREE LinuxWorld Reader's Choice Award Winner for best database on Linux. http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click