RE: organizing virtual machines

RE: organizing virtual machines

[Tom Hibbert]

 

>just want to make sure I'm understanding things correctly.

>Every virtual machine must have effectively two partitions.  The first
being a root partition containing all of the system executables and
configuration files as well as the usual /var, /tmp, etc.  The second
being storage for your application/user.
>I imagine one could make a single partition to hold both of these sets
of information but that's not the wisest choice.  my preference would
also be to put /var on the data partition.

There is no requirement for seperating partitions on installation. This
is considered a best practice because in the event of a partition
faliure it increases the chance of recovering at least part of the
system. In the purists world, / is mounted read-only and the only parts
of the disk that can be written to without mounting readwrite are /var
and /home. 


>Assuming the use of a standard OS distribution like Gentoo, it is my
impression that each virtual machine would be updated independently just
like they would be on separate physical hardware platforms.

Absolutely true. However, it is possible to have a shared root/usr using
NFS, with independent /var, /home and /etc. This is a common
configuration for clusters and big iron.

>Obviously, this seems like a terrible waste of space but given the
current dogs breakfast known as /etc, I'm not sure that is another
solution.  I have a few ideas on how to fix this that may or may not pan
out but not the hands (rsi).

Dogs breakfast? Only in the event of badly constructed packages or an
inexperienced aministrator installing from source should there ever be
any configuration data stored outside of /etc. This is the sole reason
/etc exists. I don't consider it a dogs breakfast at all, even under
Gentoo.

Tom


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_ide95&alloc_id\x14396&op=click

Re: organizing virtual machines

[Eric S. Johansson]

Tom Hibbert wrote:
> There is no requirement for seperating partitions on installation. This
> is considered a best practice because in the event of a partition
> faliure it increases the chance of recovering at least part of the
> system. In the purists world, / is mounted read-only and the only parts
> of the disk that can be written to without mounting readwrite are /var
> and /home. 

(knock wood) I've never had a partition failure.  They have always been 
more on the lines of "*%#@%@ I lost another drive".

> Absolutely true. However, it is possible to have a shared root/usr using
> NFS, with independent /var, /home and /etc. This is a common
> configuration for clusters and big iron.

intriguing.  Makes sense and one could also do this with the dom0 
serving all the other virtual machines.  would have some problems with 
/etc however

> Dogs breakfast? 

yea.  a.k.a. "recycling" and in the wintertime "Poopsicles".

> Only in the event of badly constructed packages or an
> inexperienced aministrator installing from source should there ever be
> any configuration data stored outside of /etc. This is the sole reason
> /etc exists. I don't consider it a dogs breakfast at all, even under
> Gentoo.

I must politely disagree.  In the example you gave above where root 
end-user are shared, what happens when you change a program that also 
has a related change in /etc?  At best, nothing bad happens.  At worst, 
you start getting random failures that drive you mad until you finally 
get all of the images changed.  Sometimes it's a simple replication. 
More often than not it's customized changes on all machines.

Even updating a single machine it can be a "this sucks" moment.  I 
cannot tell you the number of times I have updated gentoo and found that 
I had to slog through 50 configuration file changes.

so, the reason I consider virtually all system configuration 
directories, registries etc. a dogs breakfast is that they don't handle 
change well and the changes are not replicated properly.

my fantasy world for proper system configuration management would record 
baseline and changes so that when baseline changes one can re-create the 
working configuration set (i.e. /etc).  For a virtual machine 
environment like xen, one could have virtual machine associated changes 
with a common baseline so that when you update your executables and 
configuration directory, all the changes replicate properly or can be 
flagged for human attention.

I'm planning a working on this when I have some spare neurons.

---eric

-- 
http://www.salon.com/books/review/2004/12/18/heloise/index.html

The basis of Abelard's philosophy, which he taught to Heloise, was
that logic had to be applied to religion in order to arrive at the
truth.


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click

Re: organizing virtual machines

[Tim Freeman]

On Mon, 21 Feb 2005 18:29:17 -0500
"Eric S. Johansson" <esj@harvee.org> wrote:
[...]
> my fantasy world for proper system configuration management would record 
> baseline and changes so that when baseline changes one can re-create the 
> working configuration set (i.e. /etc).  For a virtual machine 
> environment like xen, one could have virtual machine associated changes 
> with a common baseline so that when you update your executables and 
> configuration directory, all the changes replicate properly or can be 
> flagged for human attention.
> 
> I'm planning a working on this when I have some spare neurons.

Have some thoughts on all this, too, but quickly, there is a recent paper on
the subject: 
http://www.sc-conference.org/sc2004/schedule/pdfs/pap305.pdf

Besides debconf, here's some links to check out:

http://www.cfengine.org/
http://www.infrastructures.org/papers/bootstrap/bootstrap.html
http://csdl.computer.org/comp/proceedings/cluster/2003/2066/00/20660500abs.htm


Tim



-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click

Re: organizing virtual machines

[Eric S. Johansson]

Tom Hibbert wrote:
> Aha, I only have to read down to this level to see your issue. Gentoo is
> a moving target. That means there is no advanced warning for major
> updates such as the one you describe. Gentoo is best suited to
> developers or hardline BSD refugees who cannot bear to let go of their
> ports system.

unfortunately, I have scar tissue with this problem back to early system 
  V in the mid-1980s.

> Other distributions are different. Debian for instance is divided into
> 'stable' and 'unstable' distributions. 

I prefer to think of this as moldy and compost.  Far too often have 
found that needed versions of packages that debiam hasn't even put into 
unstable yet.  Also, I have horrible luck.  I was forever wiping out the 
distribution having to start over again in addition to the numerous and 
not well-documented Debbie and wrappers for different capabilities.

I personally do not like Red Hat anymore because of the "big bang" 
upgrade process.  I have two ancient Red Hat systems that really need to 
be decommissioned.  Using gentoo cautiously I've been able to keep 
client machines as well as machines here sufficiently current that they 
never get "too old"

.. even one running CP/M on a
> cluster of dead badgers.

I think I'll need to try the cluster of dead badgers.  I have a single 
dead badger running Windows for speech recognition... which is one of 
the reasons why I am so disappointed xen does not run Windows.  It would 
have been nice to use it as my speech recognition platform.  As a 
result, I'm looking at colinux but that has its problems with regards to 
windowing environments.

yes you're quite right that the testing cycle is important and xen is a 
wonderful way of testing.  I'm hoping to have two machines in my 
basement for these very purposes.  After all, one needs to test xen 
before applying live as well.  ;-)


>>my fantasy world for proper system configuration management would
> 
> record baseline and changes so that when baseline changes one can
> re-create the working configuration set (i.e. /etc).  For a virtual
> machine environment like xen, one could have virtual machine associated
> changes with a common baseline so that when you update your executables
> and configuration directory, all the changes replicate properly or can
> be flagged for human attention.
> 
> 
>>I'm planning a working on this when I have some spare neurons.
> 
> 
> Great idea. You might want to take a look at debconf and read the Debian
> packagers guide to give you some inspiration.

believe me, I will take inspiration from any source I can.  About three 
years ago now, took inspiration from Adam Back on hashcash as a form of 
anti-spam.  I figured out what was wrong with naive sender pays 
(hashcash stamp on everything) developed to different types of 
differential payment schemes which dropped the cost to most people 
extremely low and spammers very high.  One of these techniques even 
fixes blacklists so they're not censorship tools but merely very high 
cost barriers that only the truly dedicated can get through i.e. 
inappropriately blocked people.  in conjunction with a friend of mine, 
we've worked out a pricing model for sender pays systems etc. etc.

end result, a system that has very high usability, low visibility.

I believe me, when I'm inspired, I do very interesting things.  :-)

---eric

-- 
http://www.salon.com/books/review/2004/12/18/heloise/index.html

The basis of Abelard's philosophy, which he taught to Heloise, was
that logic had to be applied to religion in order to arrive at the
truth.


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click

Re: organizing virtual machines

[Eric S. Johansson]

dead badger bites can be pretty nasty...

I believe I was over thinking the problem.

Given three pieces of data (base line file, diffs, and result) you can 
derive any one file from the other two.  If you have a /etc baseline, 
/etc diffs you can derive your working /etc.  This also means that if 
you make changes to working /etc, you can regenerate your diffs.  If you 
upgrade and have changes to the baseline, you can regenerate your 
working /etc.

operationally, this means you really need only two commands. etc_working 
and etc_diffs to generate even the working file or its difference from 
the baseline.  they are effectively wrappers for patch and diff but 
hopefully somewhat more user-friendly and infinitely easier to get right 
much of the time.

If you create a user space file system to mediate all this information, 
a simple timestamp comparison of three elements would tell you whether 
or not you need to regenerate data or just hand back the cached copy. 
Performance shouldn't be too hideous.

in a virtual machine environment, one would obviously need to associate 
each diff directory with each machine but would only need a single 
baseline.  This would then allow for a single copy for your system 
binaries and only need to have /var as a start of per virtual machine 
storage

I don't have a good solution for cases where you really want to 
overwrite the baseline files instead of creating a diff.  The best I can 
think of it is if a regenerated working /etc file is the same as what is 
in the /etc directory then there is no baseline change.  If on the other 
hand there is a difference then that's an indication that the baseline 
changed.  It would be a relatively safe bet to pull the potential new 
baseline back into the baseline hierarchy and regenerate the deltas from 
there.  Obviously, if something breaks, it's time to call the human.

One way this can break is if somebody changes to file without pushing 
the difference back into the diff hierarchy.  But hopefully, people 
won't be too careless.

anyway, that's the simplest I can make it.  objections?


---eric

-- 
http://www.salon.com/books/review/2004/12/18/heloise/index.html

The basis of Abelard's philosophy, which he taught to Heloise, was
that logic had to be applied to religion in order to arrive at the
truth.


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click

organizing virtual machines

[Eric S. Johansson]

just want to make sure I'm understanding things correctly.

Every virtual machine must have effectively two partitions.  The first 
being a root partition containing all of the system executables and 
configuration files as well as the usual /var, /tmp, etc.  The second 
being storage for your application/user.

I imagine one could make a single partition to hold both of these sets 
of information but that's not the wisest choice.  my preference would 
also be to put /var on the data partition.

Assuming the use of a standard OS distribution like Gentoo, it is my 
impression that each virtual machine would be updated independently just 
like they would be on separate physical hardware platforms.

Obviously, this seems like a terrible waste of space but given the 
current dogs breakfast known as /etc, I'm not sure that is another 
solution.  I have a few ideas on how to fix this that may or may not pan 
out but not the hands (rsi).

anyway, just wanted to confirm the impressions from the documentation.

next task is mastering lvm2...

---eric


-- 
http://www.salon.com/books/review/2004/12/18/heloise/index.html

The basis of Abelard's philosophy, which he taught to Heloise, was
that logic had to be applied to religion in order to arrive at the
truth.


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click

Re: organizing virtual machines

[Andrew Theurer]

Eric S. Johansson wrote:

> just want to make sure I'm understanding things correctly.
>
> Every virtual machine must have effectively two partitions.  The first 
> being a root partition containing all of the system executables and 
> configuration files as well as the usual /var, /tmp, etc.  The second 
> being storage for your application/user.
>
> I imagine one could make a single partition to hold both of these sets 
> of information but that's not the wisest choice.  my preference would 
> also be to put /var on the data partition. 

The best setup for me has been a shared, read only /usr, and a seperate 
"/" for each of my domains.  I use ferdora3 as my base for these 
filesystems.   The shared /usr is about 2 GB and the / is about 100 MB 
each.  When creating domains for testing, I use lvm to carve up a second 
disk, 2GB logical volume for /usr, and the rest of the disk is divided 
into logical volumes equalling the number of domains I need.  This has 
worked pretty well for me so far, and I have been able to create over 50 
domains on one system.  Now, I haven't exaclty done much with 50 domains 
yet, but they all do at least boot.

-Andrew Theurer


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click

Re: organizing virtual machines

[Anthony Liguori]

Eric S. Johansson wrote:

> Every virtual machine must have effectively two partitions.  The first 
> being a root partition containing all of the system executables and 
> configuration files as well as the usual /var, /tmp, etc.  The second 
> being storage for your application/user.

This is one approach.  Not necessarily a perfect solution.

> Obviously, this seems like a terrible waste of space but given the 
> current dogs breakfast known as /etc, I'm not sure that is another 
> solution.  I have a few ideas on how to fix this that may or may not 
> pan out but not the hands (rsi).

There are really two other solutions that I know of.  Either some sort 
of content-addressable storage based file system (like Plan9's Venti) 
which would provide an optimum storage scenario (although at a 
performance/complexity cost) or some sort of Copy-On-Write filesystem or 
block device.

LVM snapshots has been suggested a COW mechanism.  The most appealing to 
me is something like UnionFS 
(http://www.fsl.cs.sunysb.edu/project-unionfs.html) however it's rather 
unstable and I don't think FiST is going in the kernel anytime soon.  
UnionFS does COW on a file-access level.  You have one read-only mount 
that's your root and if a file is changed, the read-only version is 
copied over to a read/write partition and that's used as the working copy.

A fantastic project for someone would be a from-scratch simplistic 
union-fs clone that could actually be integrated into the kernel.  Linux 
used to have such a filesystem (IFS) but it became unmaintained and 
eventually removed from the kernel.

Regards,

-- 
Anthony Liguori
anthony@codemonkey.ws



-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click

Re: organizing virtual machines

[Mark Williamson]

> There are really two other solutions that I know of.  Either some sort
> of content-addressable storage based file system (like Plan9's Venti)
> which would provide an optimum storage scenario (although at a
> performance/complexity cost) or some sort of Copy-On-Write filesystem or
> block device.

Someone in Cambridge was working on a CoW NFS server at one stage.  I'm not 
sure what happened to it (although in any case, you wouldn't want it for IO 
intensive filesystems).

> LVM snapshots has been suggested a COW mechanism.

I think LVM snapshots have proved not to be as well suited as people would 
like.

I'm currently working on a shared-memory based filesystem for Xen virtual 
machines.  It should (eventually) give very decent IO performance.  At some 
stage, it would be nice to add filesystem-level functionality too.

Cheers,
Mark


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click