mercurial now live

mercurial now live

[Ian Pratt]

We've now completed the switchover to mercurial. It took a little longer
than planned, but everything seems to be working OK.

The Xen mercurial repositories live at
http://xenbits.xensource.com/xen-unstable.hg (and similarly for
xen-2.0-testing.hg and xen-2.0.hg). You can also point a browser at this
location to view the mercurial revision history.

The mercurial repository retains pretty much all the revision history
and structure of the old bk repository. The only serious missing feature
in hg is support for file renames. This means that the revision history
for the various linux sparse tree versions has become segmented, which
makes follow changes awkward. We've retained all the rename information
in changeset comment metadata, and when mercurial finally gets rename
support, we'll be able to create a new repository that reconstructs this
information (unfortunately, the new repository will not have a common
ancestor with the old one, so this will require another 'flag day').

A mercurial patchbot has been created that will email out full patches
to the xen-changelog@lists.xensource.com mailing list within a few
minutes of changesets being committed. 

I've attempted to integrate mercurial into the nightly build, but this
is completely untested. It's almost certain to fail, but I'll sort it
out in the morning.

Appended is a mercurial 'cheat sheet' that should help get people up to
speed.

The switch-over to mercurial has required a lot of effort. Particular
thanks are due to Michael Fetterman and James Bulpin, but also Andrew
Warfield, Arun Sharma and others.

Bk is dead. Long live hg!

Best,
Ian



Mercurial(hg) Cheatsheet for Xen
================================

Written by Andrew Warfield, extended by Michael Fetterman and Ian Pratt
June 29, 2005

Overview
--------
The Xen project has moved from BitKeeper to Mercurial for source
control.  This note aims to provide a quick guide to getting up and
running with the new tools as quickly as possible, and is written from
the perspective of someone who has been using BK.

For a more detailed exposition, see the mecurial tutorial:
 http://www.serpentine.com/mercurial/index.cgi?Tutorial

The Hg manpage is available at:
 http://www.selenic.com/mercurial/hg.1.html

There's also a very useful FAQ that explains the terminology:
 http://www.selenic.com/mercurial/FAQ.html

There's also a good README:
 http://www.selenic.com/mercurial/README

Necessary software
------------------
Mercurial is available at:
  http://www.selenic.com/mercurial/ 

You will also need a Python version >= 2.3

How Mercurial is different from BK
----------------------------------
There are several pertinent differences between hg and bk.  This section
aims to give an overview of the conceptual differences between the two
SCMs -- if you just want examples to get going, skip ahead to "Getting
Xen". The key differences are:

  - No explicit per-file locking.  You do not need to explicitly 
    check a file out before editing it.
  - No notion (currently) of file renames.
  - A repository can have multiple active heads.
  - Automatic merge support is currently inferior to BK's.
  - No graphical tools.
  - No per-file revision history, only per-changeset (we never really
used this anyhow)
  - Hg repositories tend to be rather bigger than Bk ones, but Hg does
seem faster.

Mercurial is based on the notion of changesets as complete, immutable,
versions of the repository.  You make changes to a working version of
the repository that is based on a particular changeset.  When you
commit, you will generate a new child changeset with whatever changes
you choose to apply.

A major difference between Hg and BK is that you aren't forced to
resolve conflicts immediately: BK forced you to resolve conflicts
immediately on any merge, and it then immediately created a changeset
with those conflicts' resolutions.  Frequently, you then had to add yet
another changeset to fixup the things for which the automatic merge
yielded bad results. Hg puts the results of the merge into your work
directory, and remembers what you merged with (so that it can later
record both of the merge parents, if you decide to make a changeset),
but it doesn't immediately create a changeset.

A further feature of Hg is that it allows a repository to have multiple
heads. This means that you can have changesets with no common descendent
in one repository -- something BK won't allow. This is actually pretty
neat. For example, it would enable you to have both the 2.0-testing and
unstable trees in a single repository.

To get familiar with Hg, it's best to start playing with an existing
repository (see "Getting Xen", next), and understand the "history",
"co", and "manifest commands in Hg.  These are discussed below in the
context of working with a repository, but I'll mention them here in an
attempt to demonstrate how mercurial structures things:

Assuming you have an existing repository, you can do the following:

   hg history | head

Looking at the first few lines of output, you can see the changeset at
the head of the current branch, known as the 'tip' (and given a special
tag):

   | changeset:   5599:6cbf9ec05cd9e05c0c46a85df7fc00262633cd3d
   | tag:         tip
   | user:        kaf24@firebug.cl.cam.ac.uk
   | date:        Tue Jun 28 18:47:14 2005
   | summary:     bitkeeper revision 1.1768
(42c18d2259NPELcGV7ohyZNh72ufSw)

By default, Hg just shows the first line of the changset comments. You
can find further information with "hg -v history".

The changeset identifier has two parts, a _local_ monotonically
increasing changeset id, 4792 above, and a _global_ hash, which follows
the colon on the changeset line.  The hash uniquely identifies the
changeset and its lineage back to the root of the changeset tree
-- it is useful for distributed management and so on.  However, as it is
a bit unruly, the local id will allow you to work easily with the local
repo.  Additionally, a tags mechanism (described below) lets you give
common names to specific changesets.

You should always use the global hash when referring to versions of the
mainline Xen respoitory. With Bk you could often get away with using the
shortform version, but with Hg the local hashes are pretty much
guaranteed to be different.

The co command lets you change the working version of the repository to
a different changeset. You can use either the local or global hash name,
e.g.:

   hg co 4791

This would rewind my working repository to the previous changeset.
There are various options to this, mentioned below and in the
documentation ("hg help co").  In general, it does the right thing,
although it doesn't currently seem to clean up unused directories that
have been created by other checked-out versions. This can confuse the
Xen build system. Hg will probably get fixed soon, but in the meantime
you can cleanup with "find -depth -type d -print | xargs -r rmdir".

You can return to the tip with:
   
   hg co

The manifest command lets you see the contents of the repository for the
current changeset.

   hg manifest

This will print a bunch of records of the form: 

   | 98856c45c35a504bc6da06a62b7787ddfdfd1c8b 644 COPYING
   | f28971eedc5b54e7a9b26dd18d52992955354981 644 Config.mk
   | a3575cc4db59e50bbac8a039a0c74f081a8dfc4f 644 Makefile
   | 7fc869aae2945a9f4626fad96552db3103e61cb9 644 README
   | ...

This lists the hash of each file, its 1-bit 'executable' atribute
(either file permission mode 644 or 755), and the file name.  So, to
determine the files that change across two changesets, you would dump
the respective manifests to files, and use diff.

One slightly confusing aspect of Hg is that many of the commands have
aliases, and hence when looking things up in the man page its not always
obvious what the underlying command is. For example 'co' is actually an
alias for the 'update' command, but 'co' seems to make more sense, at
least to RCS refugees like me.

Getting Xen
-----------
To avoid taxing the Mercurial server with a complete pull of the Xen
repository, it is best to download a tarball of a seed repository
from:

   http://xenbits.xensource.com/xen-unstable.hg

Untar the repository on your disk, cd into it, and then pull the most
recent changes:

  hg pull -u

The repository parent is stored in a repository configuration file,
.hg/hgrc, from the repository root.  If you look at this file, you will
see:

  |  [paths]
  |  default = http://xenbits.xensource.com/xen-unstable.hg

"default" specifies the appropriate parent repository for hg to pull
from.  Hg allows you to pull additional repositories, for instance if
you want to work between unstable and testing concurrently.

The command "hg pull" simply adds changesets to your repository, without
any merging of any kind.  "hg pull -u" implies merging with the current
state of your working directory.  If you weren't already "updated" to
your local repository's tip, you might be surprised to find yourself
merging the results of the pull with a non-tip node in your local
repository.  

Creating a child repository from an existing repository
-------------------------------------------------------
If you wanted to create additional local child repositories,

   hg init [path or url]

is effectively equivalent to bk clone.  The major difference is that it
should be run from the root of your new repository.  So:

   bk clone /foo/bar

would be replaced with:

   mkdir bar
   cd bar
   hg init /foo/bar

NB: newer version of hg support a 'clone' command that works in the same
manner as bk.

Editing files
-------------

Normal edits may be made in place.  File creation needs explicit
marking, though deletes should be picked up automatically

creation:

   touch a (or otherwise created a file)
   hg add a

You can see what has changed using:

   hg status

   | C foo/foo.c
   | R foo/bar.c
   | ? a.txt

Shows that in the current repo, foo.c has been changed, bar.c has been
deleted, and a.txt is new, but has not been added.  '?' changes to 'A'
after "hg add a.txt".

Committing changes
-----------------

   hg commit

will pop up an editor with a list of files to be committed to the
repository.  It will look vaguely like this:

   | 
   | HG: manifest hash 6397b04bd5c2a992482d973b685a7e5e498788e7
   | HG: changed doc/thesis/new.tex
   | HG: removed doc/2005-hotdep-protection/paper.tex

Your comments can go anywhere in this file.  The first line is the most
important, as it will show as the changeset description in
non-verbose-mode history listings.

You can do commits without the editor and of partial sets of files using
command-line switches. See:

   hg help commit

You can use the -A (--addremove) flag to commit e.g. "hg -A commit" to
ash mercurial to scan the tree looking for newly created files to add in
to the changeset. This avoids having to explicitly use "hg add".


Generating a patch
------------------
Generating a patch is easy,

   hg export [changeset]

will generate a patch describing the diff between that changeset and its
parent.

Pushing changesets to a parent repository
-----------------------------------------

   hg push

Pushes changes up to a parent. You can't push if you pulled the
repository off the web interface. In fact, you can currently only push
to an ssh target -- filesystem drectory targets don't work, but this
will be fixed soon.


Repository history
------------------

Here are a collection of common commands to get you started:

   hg history | less

shows the history of changesets, starting from the most recent.  You
want to pipe it to some sort of pager.  For more complete details,
 
   hg -v history | less

will include files modified and full (not just first-line) comments.

Additionally, you can see just the tip (head of the current
branch) of the repository by typing:

   hg [-v] tip

Moving to a specific changeset
------------------------------

   hg co [changeset]

will update the working directory to reflect the specified changeset. By
default, it will refuse to overwrite any modified files.

You can use the -m option to cause the checked out version to be merged
into the current tree. The "-C" (--clean) option will additionally
overwrite any locally modified files.

Any commits that are made to non-head changesets will obviously fork the
tree, creating a new head. You can see all the heads in a tree with "hg
heads".

Managing changeset tags
-----------------------
To create a tag to the current changeset:

   hg tag tagname

This will _immediately_ generate a changeset with a change to the file
.hgtags in the repository root.   The new tag in this file will look
something like:

   | 35159ed4b30538e7a52c60ad0a63f7e9af156e4c tagname

and may be used to identify that changeset throughout the repo.
Storing tags in this file and generating changesets immediately forces
people to merge and keep tags up to date across the repository.

Note that tags are resolved by searching .hgtags in each of the
repository heads, sequentially, and using the first match.  "hg heads"
lists the current heads.

The "hg tags" command, will lists all the currently valid tags.


Hg server and source browser
----------------------------

   hg serve -p port

Launches a web server on the specified port, serving a source browser
for the repository.  This browser may be used to examine the changeset
history, view annotated source files, generate diffs.  
Additionally "hg pull" may be run against it.

Additional useful commands
(that probably only need one-line descriptions)
-----------------------------------------------

(Slightly) more detail on all of these is available with

  hg help [command]

View an annotated version of a source file:

   hg annotate

Get a historical version of a file:

   hg cat

 NB: Most commands accepting a version number want the changeset's
version number.  "hg cat" is different in that it wants the  *file*'s
version number.

Unadd a file to the current commit:

   hg forget

List all heads in the current repository:

   hg heads

Undo exactly one (and ONLY one) changeset:

   hg undo

Shows the differences between whatever changeset you most recently
checked out, and your current working directory:

   hg diff

Show the parents of a changeset:

   hg parents

 NB: Changesets have either one or two parent changesets. If your
working directory contains the uncommitted results of a merge, then  you
have two parents.  Otherwise, the single parent is the changeset  which
you most recently checked out.

Re: mercurial now live

[aq]

On 7/1/05, Ian Pratt <m+Ian.Pratt@cl.cam.ac.uk> wrote:
> 
> We've now completed the switchover to mercurial. It took a little longer
> than planned, but everything seems to be working OK.

never doubt this is a major milestone of Xen project. congrats ;-)

anyway, there is a problem with search box at 
http://xenbits.xensource.com/xen-unstable.hg. i typed a string into a
box ("console" in this case), enter, and got error message. please see
below.

----
A problem occurred in a Python script. Here is the sequence of
function calls leading up to the error, in the order they occurred.

 /var/www/html/xen-unstable.hg
    6 from mercurial import hgweb
    7 
    8 h = hgweb.hgweb("/home/xen/HG/xen-unstable.hg", "xen-unstable")
    9 h.run()
   10 
h = <mercurial.hgweb.hgweb instance>, h.run = <bound method hgweb.run
of <mercurial.hgweb.hgweb instance>>

 /usr/groups/netos/hg/lib/python/mercurial/hgweb.py in
run(self=<mercurial.hgweb.hgweb instance>)

 /usr/groups/netos/hg/lib/python/mercurial/hgweb.py in
write(*things=(<generator object>,))

 /usr/groups/netos/hg/lib/python/mercurial/hgweb.py in
changelog(self=<mercurial.hgweb.hgweb instance>, pos='console')

TypeError: unsupported operand type(s) for -: 'str' and 'int'
      args = ("unsupported operand type(s) for -: 'str' and 'int'",)

Re: mercurial now live

[Andrew Thompson]

[-- Attachment #1: Type: text/plain, Size: 403 bytes --]

aq wrote:
> anyway, there is a problem with search box at 
> http://xenbits.xensource.com/xen-unstable.hg. i typed a string into a
> box ("console" in this case), enter, and got error message. please see
> below.

Yeah, that part needs a little work. I'll try to look into that this 
weekend.

The search box currently accepts only a hash or a changeset number.

-- 
Andrew Thompson
http://aktzero.com/

[-- Attachment #2: andrewkt.vcf --]
[-- Type: text/x-vcard, Size: 129 bytes --]

begin:vcard
fn:Andrew Thompson
n:Thompson;Andrew
email;internet:andrewkt@aktzero.com
x-mozilla-html:FALSE
version:2.1
end:vcard


[-- Attachment #3: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: mercurial now live

[aq]

On 7/1/05, Ian Pratt <m+Ian.Pratt@cl.cam.ac.uk> wrote:
> 
> Getting Xen
> -----------
> To avoid taxing the Mercurial server with a complete pull of the Xen
> repository, it is best to download a tarball of a seed repository
> from:
> 
>    http://xenbits.xensource.com/xen-unstable.hg

I cannot find the "seed repository". Where to get it?

regards,
aq

Re: mercurial now live

[aq]

On 7/1/05, aq <aquynh@gmail.com> wrote:
> On 7/1/05, Ian Pratt <m+Ian.Pratt@cl.cam.ac.uk> wrote:
> >
> > Getting Xen
> > -----------
> > To avoid taxing the Mercurial server with a complete pull of the Xen
> > repository, it is best to download a tarball of a seed repository
> > from:
> >
> >    http://xenbits.xensource.com/xen-unstable.hg
> 
> I cannot find the "seed repository". Where to get it?
> 

oops, with the snapshot hg, we can easily clone xen with:

hg clone http://xenbits.xensource.com/xen-unstable.hg myxen

(mercurial 0.6 doesnt support clone command)

regards,
aq

RE: mercurial now live

[Ling, Xiaofeng]

I tried the following URL, but also failed.
http://www.cl.cam.ac.uk/Research/SRG/netos/xen/downloads/xen-unstable.hg.tar.gz"

aq <> wrote:
> On 7/1/05, Ian Pratt <m+Ian.Pratt@cl.cam.ac.uk> wrote:
>> 
>> Getting Xen
>> -----------
>> To avoid taxing the Mercurial server with a complete pull of the Xen
>> repository, it is best to download a tarball of a seed repository
>> from: 
>> 
>>    http://xenbits.xensource.com/xen-unstable.hg
> 
> I cannot find the "seed repository". Where to get it?
> 
> regards,
> aq
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

RE: mercurial now live

[Ian Pratt]

 


> > > To avoid taxing the Mercurial server with a complete pull 
> of the Xen 
> > > repository, it is best to download a tarball of a seed repository
> > > from:
> > >
> > >    http://xenbits.xensource.com/xen-unstable.hg
> > 
> > I cannot find the "seed repository". Where to get it?

Oops, my posting last night was far from clear about that. Fortunately,
the downloads web page has all the information you need. 

You can get the seed tar bar for the unstable tree from:

 
http://www.cl.cam.ac.uk/Research/SRG/netos/xen/downloads/xen-unstable.hg
.tar.gz
 
(similarly for xen-2.0 and xen-2.0-testing)

Just untar the file, 'cd' into it, and run:
 
 hg pull
 hg co

The .hg/hgrc file in the tar ball should automatically set as your
default pull parent to the folllwing URL, but you can do it manually
too:
 
 http://xenbits.xensource.com/xen-unstable.hg

> oops, with the snapshot hg, we can easily clone xen with:
> 
> hg clone http://xenbits.xensource.com/xen-unstable.hg myxen
> 
> (mercurial 0.6 doesnt support clone command)

You don't need the new clone command. You can have the same effect as
'bk clone' with:

  mkdir xen-unstable.hg
  hg init http://xenbits.xensource.com/xen-unstable.hg
  hg co
  cd ..
  
[the "hg co" is necessary because by default hg does not 'get' files
from the repository, which we had configured as the default for bk.]

I'll add the 'cheat sheet' to the downloads web page and keep it
updated.

Ian

RE: mercurial now live

[Ian Pratt]

I've updated the Mecurial for Xen cheatsheet and added it to the
documentation page and the wiki.

http://www.cl.cam.ac.uk/Research/SRG/netos/xen/readmes/hg-cheatsheet.txt

I've re-ordered things so that it follows more of a recommended workflow
for creating a respository, making some edits and creating a changeset.

Best,
Ian

Re: mercurial now live

[aq]

On 7/1/05, Ian Pratt <m+Ian.Pratt@cl.cam.ac.uk> wrote:
> 
> 
> 
> > > > To avoid taxing the Mercurial server with a complete pull
> > of the Xen
> > > > repository, it is best to download a tarball of a seed repository
> > > > from:
> > > >
> > > >    http://xenbits.xensource.com/xen-unstable.hg
> > >
> > > I cannot find the "seed repository". Where to get it?
> 
> Oops, my posting last night was far from clear about that. Fortunately,
> the downloads web page has all the information you need.
> 
> You can get the seed tar bar for the unstable tree from:
> 
> 
> http://www.cl.cam.ac.uk/Research/SRG/netos/xen/downloads/xen-unstable.hg
> .tar.gz
> 
> (similarly for xen-2.0 and xen-2.0-testing)
> 
> Just untar the file, 'cd' into it, and run:
> 
>  hg pull
>  hg co
> 
> The .hg/hgrc file in the tar ball should automatically set as your
> default pull parent to the folllwing URL, but you can do it manually
> too:
> 
>  http://xenbits.xensource.com/xen-unstable.hg
> 
> > oops, with the snapshot hg, we can easily clone xen with:
> >
> > hg clone http://xenbits.xensource.com/xen-unstable.hg myxen
> >
> > (mercurial 0.6 doesnt support clone command)
> 
> You don't need the new clone command. You can have the same effect as
> 'bk clone' with:
> 
>   mkdir xen-unstable.hg
>   hg init http://xenbits.xensource.com/xen-unstable.hg

anyway, looks like "hg init <source>" is deprecated in hg snapshot
version (hg would complains and exits). instead  "hg clone" is a
recommended method.

regards,
aq