sbuild (or whatever) integration

sbuild (or whatever) integration

[Florian Weimer]

I'd like to build something that integrates a Debian autobuilder
(sbuilder probably) with GIT.  The intent is to get rid of a
designated integrator (our team is of a size where this seems a
reasonable thing to do[1]) and ensure predictable builds, independent
of what developers happen to have installed on their local
workstations.

Basically, what I want to do is this:

  1. Perform a checkout from a specific branch.

  2. Synthesize a new version number that is bigger than the previous
     one.  (This needs to keep a bit of extra-repository state.  At
     this stage, it's probably a good idea to ensure that the previous
     commit built on this branch is an ancestor of the current HEAD.)

  3. Generate a fake Debian changelog entry giving the HEAD commit and
     the branch name, using the generated version number.

  4. Build the whole thing in a controlled environment.

  5. Store the build log for later review.

  6. If the build was successful, transfer the packages to some
     repository (based on the branch that was used).

The GIT repository in step one is a shared one, with an update hook
that that ensures fast-forward merges, as in:

<http://www.kernel.org/pub/software/scm/git/docs/howto/update-hook-example.txt>

The idea is that this means that features are monotonic along a
branch.  Ideally, I would like to have something which can assign
meaningful version numbers across two branches, but this is of course
impossible to automatically.  I hope to work with a single release
branch anyway, and use at most two (one for testing, one for
production).

Does this sound reasonable?  Has anybody built something similar?

[1] This is for internal development, and due to various constraints,
    we haven't got developer-specific integration testing
    environments.  That's why we need to linearize before actual
    deployment.

Re: sbuild (or whatever) integration

[Jan Hudec]

[-- Attachment #1: Type: text/plain, Size: 3338 bytes --]

On Sat, May 19, 2007 at 15:17:04 +0200, Florian Weimer wrote:
> I'd like to build something that integrates a Debian autobuilder
> (sbuilder probably) with GIT.  The intent is to get rid of a
> designated integrator (our team is of a size where this seems a
> reasonable thing to do[1]) and ensure predictable builds, independent
> of what developers happen to have installed on their local
> workstations.

That's definitely a useful and reasonable thing to do.

> Basically, what I want to do is this:
> 
>   1. Perform a checkout from a specific branch.
> 
>   2. Synthesize a new version number that is bigger than the previous
>      one.  (This needs to keep a bit of extra-repository state.  At
>      this stage, it's probably a good idea to ensure that the previous
>      commit built on this branch is an ancestor of the current HEAD.)
> 
>   3. Generate a fake Debian changelog entry giving the HEAD commit and
>      the branch name, using the generated version number.
> 
>   4. Build the whole thing in a controlled environment.
> 
>   5. Store the build log for later review.
> 
>   6. If the build was successful, transfer the packages to some
>      repository (based on the branch that was used).

I don't think there is actually too much code that would be common for
various git users. It has some code common for Debian developers, either
using Git or even other version control:

1. This is one or at most few commands in a script or makefile, so not much
   room for generic stuff there.
2. This is something big conitnuous integration suites have complex system to
   solve, just to hide the fact that it's worth few lines of perl. However:
    - You might want to make up something Debian specific. Something like
      looking at latest entry in debian/changelog (that would be the last
      release) and adding an extra number to that.
    - Alternatively you could use git describe to get the version number of
      last release. Since it's fast-forward only, next build is always from
      descendant of previous, so you might even use git-describe output
      directly.
3. This is mostly Debian-specific. If you remember commit id of previous
   build -- or leave the entries in debian/changelog around -- you can put
   the short log in the entry, but that's single git command once you have
   the respective version ids.
4. These are Debian-specific commands, but would be the same for every Debian
   package.
5. This is mainly issue of sbuild setup, no?
6. Again, sbuild -- and therefore Debian -- specific (but reusable in
   Debian).

IMHO generic tools won't help you much, because every other command is Debian
specific.

What *CAN* help you is the script in
    /usr/share/doc/git-core/contrib/continuous
that will make sure the package is built whenever there are some new changes
on the branch while not doing unnecessary work.

There are actually even big projects with web interface and stuff to do this
kind of thing, but they don't actually cover your steps above except 1 and 2.
1 is trivial and for 2 you have to tell it how to create those numbers
anyway, so it won't buy you much. IMHO the git contrib/continuous scripts can
do the same for you with much less overhead.

-- 
						 Jan 'Bulb' Hudec <bulb@ucw.cz>

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: sbuild (or whatever) integration

[Florian Weimer]

* Jan Hudec:

> IMHO generic tools won't help you much, because every other command is Debian
> specific.

Yeah, and I mainly wanted to reuse process, not code.  Thanks for your
comments.

Re: sbuild (or whatever) integration

[Jan Hudec]

[-- Attachment #1: Type: text/plain, Size: 462 bytes --]

On Tue, May 29, 2007 at 11:26:41 +0200, Florian Weimer wrote:
> * Jan Hudec:
> 
> > IMHO generic tools won't help you much, because every other command is Debian
> > specific.
> 
> Yeah, and I mainly wanted to reuse process, not code.  Thanks for your
> comments.

The process is definitely sensible. Also I mentioned the git
contrib/continuous scripts to help you schedule the builds.

Regards,

Jan

-- 
						 Jan 'Bulb' Hudec <bulb@ucw.cz>

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]