Diskless boot at cambridge

Diskless boot at cambridge

[Kip Macy]

All the power-cycling I've been doing appears have done bad things to
the local disk on one of my development machines. I'm hoping that you
might have short recipe on setting up diskless boot with xen.

I set up PXEboot long ago, but I recall the documentation as being rather poor.

Thanks.
    -Kip

RE: Diskless boot at cambridge

[James Bulpin]

Kip,

We use either grub with tftp (limited driver support here), or XenoBoot
(now an optional function of the new Xen demo CD). We have a fairly
elaborate system involving our lab-wide NFS tree and scripts and
symlinks for getting the right images for each machine. Root filesystems
for diskless systems are either iSCSI, as previously discussed or NFS.

James

-----Original Message-----
From: xen-devel-bounces@lists.xensource.com
[mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Kip Macy
Sent: Thursday, April 14, 2005 6:29 PM
To: xen-devel
Subject: [Xen-devel] Diskless boot at cambridge


All the power-cycling I've been doing appears have done bad things to
the local disk on one of my development machines. I'm hoping that you
might have short recipe on setting up diskless boot with xen.

I set up PXEboot long ago, but I recall the documentation as being
rather poor.

Thanks.
    -Kip

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel

Re: Diskless boot at cambridge

[Jacob Gorm Hansen]

James Bulpin wrote:
> Kip,
> 
> We use either grub with tftp (limited driver support here), or XenoBoot
> (now an optional function of the new Xen demo CD). We have a fairly
> elaborate system involving our lab-wide NFS tree and scripts and
> symlinks for getting the right images for each machine. Root filesystems
> for diskless systems are either iSCSI, as previously discussed or NFS.

I am using Grub 0.95 with an UNDI patch I found somewhere, that means I 
can PXE-boot from some (at least e1000) network adapters not directly 
supported by Grub.

I would recommend NFS for debug-rootfs, as with iSCSI you still have a 
chance of corrupting the fs image.

Jacob

Re: Diskless boot at cambridge

[Kip Macy]

> We use either grub with tftp (limited driver support here), 

I'm having hard time even getting to the GRUB prompt so I don't think
that is an option.

> or XenoBoot
> (now an optional function of the new Xen demo CD). 

Is that documented anywhere?

> We have a fairly
> elaborate system involving our lab-wide NFS tree and scripts and
> symlinks for getting the right images for each machine. 

We have innumerable mount points and so use the automounter (which you
obviously don't use as it is disabled in the default config).

> Root filesystems
> for diskless systems are either iSCSI, as previously discussed or NFS.

How would you give a machine a unique persistent initiator-name when
booting from a general cd?

 Thanks. 


               -Kip 


> James
> 
> -----Original Message-----
> From: xen-devel-bounces@lists.xensource.com
> [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Kip Macy
> Sent: Thursday, April 14, 2005 6:29 PM
> To: xen-devel
> Subject: [Xen-devel] Diskless boot at cambridge
> 
> All the power-cycling I've been doing appears have done bad things to
> the local disk on one of my development machines. I'm hoping that you
> might have short recipe on setting up diskless boot with xen.
> 
> I set up PXEboot long ago, but I recall the documentation as being
> rather poor.
> 
> Thanks.
>     -Kip
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
> 
>

Re: Diskless boot at cambridge

[Kip Macy]

> 
> I am using Grub 0.95 with an UNDI patch I found somewhere, that means I
> can PXE-boot from some (at least e1000) network adapters not directly
> supported by Grub.

Thanks. Do you know offhand if that supports the tg3 chip?

> I would recommend NFS for debug-rootfs, as with iSCSI you still have a
> chance of corrupting the fs image.
Do you export a separate mount point to each client, or do you have
something like cluster NFS?

Thanks.

          -Kip

Re: Diskless boot at cambridge

[Jacob Gorm Hansen]

Kip Macy wrote:
>>I am using Grub 0.95 with an UNDI patch I found somewhere, that means I
>>can PXE-boot from some (at least e1000) network adapters not directly
>>supported by Grub.
> 
> 
> Thanks. Do you know offhand if that supports the tg3 chip?

No, sorry I don't. If you can't get the tg3 working another option is to 
drop in an eepro100 or old 3com just for use with Grub.

>>I would recommend NFS for debug-rootfs, as with iSCSI you still have a
>>chance of corrupting the fs image.
> 
> Do you export a separate mount point to each client, or do you have
> something like cluster NFS?
> 

What I usually do for debugging is share the same mount point, and then 
mount /tmp and other similar stuff from a ramdisk formatted at boot, or 
from tmpfs.

Jacob

RE: Diskless boot at cambridge

[James Bulpin]

>> or XenoBoot
>> (now an optional function of the new Xen demo CD).
>
>Is that documented anywhere?

I'm sure Tim Deegan will reply to this. We'll try to get something in
the wiki.

>We have innumerable mount points and so use the automounter (which you
obviously don't use as it is disabled in >>the default config).

We use autofs within the lab cloud and use a userspace NFSd to reexport
this. Some folks dive straight in to machine:/mount/point

>> Root filesystems
>> for diskless systems are either iSCSI, as previously discussed or
NFS.
>
>How would you give a machine a unique persistent initiator-name when
booting from a general cd?

The XenoBoot CD pulls in a boot script which we generally populate with
a case statement based on DHCP host name. Only a couple of our boxes use
iSCSI at the moment so each has a separate hand-crafted boot command
line.

James

Re: Diskless boot at cambridge

[Jacob Gorm Hansen]

Kip Macy wrote:
>>We use either grub with tftp (limited driver support here), 
> 
> 
> I'm having hard time even getting to the GRUB prompt so I don't think
> that is an option.

Getting this stuff running is kind of hard, but it _will_ be worth it 
almost no matter how much time you spend :-) You can also put Grub on 
the disk and then use it's ifconfig or bootp commands to configure the 
network and do tftp from there, but I have never tried that myself.

If your network adapter does not support PXE you can stick etherboot on 
a floppy (if etherboot supports your adapter that is).

Jacob

Re: Diskless boot at cambridge

[Adam Heath]

On Thu, 14 Apr 2005, Jacob Gorm Hansen wrote:

> I am using Grub 0.95 with an UNDI patch I found somewhere, that means I
> can PXE-boot from some (at least e1000) network adapters not directly
> supported by Grub.

Couldn't you find that patch again?  I'd love to have it(we use grub and
nfsroot).

RE: Diskless boot at cambridge

[Ian Pratt]

 
> I am using Grub 0.95 with an UNDI patch I found somewhere, 
> that means I can PXE-boot from some (at least e1000) network 
> adapters not directly supported by Grub.

I've found that the Grub UNDI patch still only works on a subset of
machines. I've not had any luck with on board tg3 NICs.

Tim Deegan is looking into a multiboot patch to pxelinux, which should
be a good generic solution for booting in the local area. For co-lo
machines, you can't beat Xenoboot.

Ian

RE: Diskless boot at cambridge

[James Harper]

> All the power-cycling I've been doing appears have done bad things to
> the local disk on one of my development machines. I'm hoping that you
> might have short recipe on setting up diskless boot with xen.

I'm doing this on one machine with an eepro100 and iscsi, and another
with an e1000 card, but the e1000 just won't work with grub. It seems to
get as far as the grub menu (very slowly) and then just hangs when you
try and select one.

If you want more info I could put something together. What distro are
you using?

James

Re: Diskless boot at cambridge

[Jacob Gorm Hansen]

Adam Heath wrote:
> On Thu, 14 Apr 2005, Jacob Gorm Hansen wrote:
> 
> 
>>I am using Grub 0.95 with an UNDI patch I found somewhere, that means I
>>can PXE-boot from some (at least e1000) network adapters not directly
>>supported by Grub.
> 
> 
> Couldn't you find that patch again?  I'd love to have it(we use grub and
> nfsroot).

yes,

I have the made complete package with grub, the patches and a README 
available at:

http://www.diku.dk/~jacobg/grub_that_works_e1000.tar.gz

note that I compiled this package some six months ago, so perhaps never 
versions may be available.

/Jacob

Re: Diskless boot at cambridge

[William (Andy) Smith]

I posted over the Xen Users list about a 'Thin Dom0' client. I'm still in
the process of juggling flaming kittens, as it were. A recipe to duplicate
what I've seen in the replies would be ... useful in ways I cannot express.

Would there be room on the Xen site for a 'Useful Recipes' section, or
'Howto Accomplish "X"' for documentation?

--Romaq

RE: Diskless boot at cambridge

[Ian Pratt]

> I posted over the Xen Users list about a 'Thin Dom0' client. 
> I'm still in the process of juggling flaming kittens, as it 
> were. A recipe to duplicate what I've seen in the replies 
> would be ... useful in ways I cannot express.
> 
> Would there be room on the Xen site for a 'Useful Recipes' 
> section, or 'Howto Accomplish "X"' for documentation?

Sure -- add it yourself on "wiki.xensource.com"

We haven't announced it yet, but its there for people to add stuff.

Cheers,
Ian

Re: Diskless boot at cambridge

[Tim Deegan]

On Thu, Apr 14, 2005 at 10:51:14AM -0700, Kip Macy wrote:
> > or XenoBoot (now an optional function of the new Xen demo CD). 
> 
> Is that documented anywhere?

There's a README at
http://www.cl.cam.ac.uk/Research/SRG/netos/xeno/xenoboot/

Xenoboot boots to linux and then runs a script (that it downloads over
https and checks with GPG).  The usual function of the script is to call
'kexec' to load the kernel you're interested in.  The functionality is
being merged with the demo CD (partly so I only have to maintain one of
them and partly because the gentoo package management tools can't cope
any more with upgrading the xenoboot filesystem).  That version hasn't
got documentation yet, but maybe will do soon. :)

> How would you give a machine a unique persistent initiator-name when
> booting from a general cd?

We use its IP address (from statically-assigned DHCP).

Xenoboot is really designed for colo environments: it can be very useful
to have a working linux environment from which to debug a broken
system. :)  Its primary disadvantage in the development environment is
that it makes the reboot cycle longer, as it needs to do a whole extra
Linux boot.  (Its primary advantage is that it supports most NICs.)

If GrUB won't speak to your NIC, there are a couple of ways of using
PXELINUX to boot Xen -- either by wrapping up Xen, xenlinux and dom0's
initrd using mbootpack (in xen-unstable/tools/misc, or from
http://www.tjd.phlegethon.org/software/) and booting that using
PXELINUX, or by building a MEMDISK image that includes GrUB and the xen
images (http://syslinux.zytor.com/archives/2005-March/004941.html).

They both need some preprocessing beyond just sticking the right kernel
in the right place, though.  I'm working on adding support to PXELINUX
to boot Xen directly; hopefully that'll be ready in the next few weeks.

Tim.

-- 
Tim Deegan                           (My opinions, not the University's)
Systems Research Group
University of Cambridge Computer Laboratory

RE: Diskless boot at cambridge

[Adam Heath]

On Fri, 15 Apr 2005, James Harper wrote:

> > All the power-cycling I've been doing appears have done bad things to
> > the local disk on one of my development machines. I'm hoping that you
> > might have short recipe on setting up diskless boot with xen.
>
> I'm doing this on one machine with an eepro100 and iscsi, and another
> with an e1000 card, but the e1000 just won't work with grub. It seems to
> get as far as the grub menu (very slowly) and then just hangs when you
> try and select one.
>
> If you want more info I could put something together. What distro are
> you using?

There's a patch to upgrade grub's nic drivers in the grub bug system.  We had
this same problem, and the patch fixed it for us.  Not to mention the new pci
ids issue.

RE: Diskless boot at cambridge

[James Harper]

> >
> > I'm doing this on one machine with an eepro100 and iscsi, and
another
> > with an e1000 card, but the e1000 just won't work with grub. It
seems to
> > get as far as the grub menu (very slowly) and then just hangs when
you
> > try and select one.
>
> There's a patch to upgrade grub's nic drivers in the grub bug system.
We
> had
> this same problem, and the patch fixed it for us.  Not to mention the
new
> pci ids issue.

I'm using the latest patch I could find. It obviously works with other
e1000 cards, just not mine :(

James