using stgit/guilt for public branches

using stgit/guilt for public branches

[Michael S. Tsirkin]

Hi!
On git.openfabrics.org we use git to manage all code for our OFED distribution.
For our kernel code we basically started with 2.6.20, and add some patches,
which we currently keep separate from upstream kernel source - this makes
it possible to update from upstream and extract the patches to post
them for upstream inclusion easily.

On the surface, it looks like using stg or guilt would be a good idea for us,
however multiple people need to collaborate on the patch series.

I am concerned that publishing a git branch managed by stg/guilt
would present problems: it seems that every time patches are re-ordered,
a patch is re-written or removed, or we update from upstream,
everyone who pulls the tree branch will have a hard-to-resolve conflict.

Is that really a problem? If so, would it be possible to work around this somehow?

Thanks,

-- 
MST

Re: using stgit/guilt for public branches

[Josef Sipek]

On Wed, Apr 25, 2007 at 03:20:49PM +0300, Michael S. Tsirkin wrote:
> Hi!
> On git.openfabrics.org we use git to manage all code for our OFED distribution.
> For our kernel code we basically started with 2.6.20, and add some patches,
> which we currently keep separate from upstream kernel source - this makes
> it possible to update from upstream and extract the patches to post
> them for upstream inclusion easily.
> 
> On the surface, it looks like using stg or guilt would be a good idea for us,
> however multiple people need to collaborate on the patch series.
> 
> I am concerned that publishing a git branch managed by stg/guilt
> would present problems: it seems that every time patches are re-ordered,
> a patch is re-written or removed, or we update from upstream,
> everyone who pulls the tree branch will have a hard-to-resolve conflict.
> 
> Is that really a problem? If so, would it be possible to work around this
> somehow?

I thought about this problem a while back when I was trying to decide how to
manage the Unionfs git repository. I came to the conclusion, that there was
no clean way of doing this (at least not using guilt - I can't really speak
for stgit, as I don't know how it does things exactly).

You could try to use git to version the patches directory
(.git/patches/$branch/) and publish that in addition to the actual kernel
repository.

Josef "Jeff" Sipek.

-- 
Keyboard not found!
Press F1 to enter Setup

Re: using stgit/guilt for public branches

[Michael S. Tsirkin]

> Quoting Josef Sipek <jsipek@fsl.cs.sunysb.edu>:
> Subject: Re: using stgit/guilt for public branches
> 
> You could try to use git to version the patches directory
> (.git/patches/$branch/) and publish that in addition to the actual kernel
> repository.

How does one do this, exactly?

-- 
MST

Re: using stgit/guilt for public branches

[Josef Sipek]

On Wed, Apr 25, 2007 at 10:52:48PM +0300, Michael S. Tsirkin wrote:
> > Quoting Josef Sipek <jsipek@fsl.cs.sunysb.edu>:
> > Subject: Re: using stgit/guilt for public branches
> > 
> > You could try to use git to version the patches directory
> > (.git/patches/$branch/) and publish that in addition to the actual kernel
> > repository.
> 
> How does one do this, exactly?

cd .git/patches/$branch/
git-init-db
git-commit -a -m "initial import"


Then set up this repository to push/pull.


No guarantees about it actually being helpful to your situation :)

Josef "Jeff" Sipek.

-- 
UNIX is user-friendly ... it's just selective about who it's friends are

Re: using stgit/guilt for public branches

[Robin Rosenberg]

onsdag 25 april 2007 skrev Josef Sipek:
> On Wed, Apr 25, 2007 at 03:20:49PM +0300, Michael S. Tsirkin wrote:
[...]
> > I am concerned that publishing a git branch managed by stg/guilt
> > would present problems: it seems that every time patches are re-ordered,
> > a patch is re-written or removed, or we update from upstream,
> > everyone who pulls the tree branch will have a hard-to-resolve conflict.
> > 
> > Is that really a problem? If so, would it be possible to work around this
> > somehow?
> 
> I thought about this problem a while back when I was trying to decide how to
> manage the Unionfs git repository. I came to the conclusion, that there was
> no clean way of doing this (at least not using guilt - I can't really speak
> for stgit, as I don't know how it does things exactly).

StGit has the same problem. Publishing such a branch is only for viewing if
you want to publish the tip, like the pu branch in the Git repo. You shouldn't
merge from pu either.

Patches your are done and not stored as patches anymore can be propagated 
to a "stable" branch using git branch -f stable HEAD~$(stg applied|wc -l) 
which is then pushed like  any other branch. Don't play the stg uncommit game 
too much to avoid a mess.

I'm  not sure the branch command is the best way, but you get the idea.

-- robin

Re: using stgit/guilt for public branches

[Yann Dirson]

On Wed, Apr 25, 2007 at 11:37:05PM +0200, Robin Rosenberg wrote:
> onsdag 25 april 2007 skrev Josef Sipek:
> > On Wed, Apr 25, 2007 at 03:20:49PM +0300, Michael S. Tsirkin wrote:
> [...]
> > > I am concerned that publishing a git branch managed by stg/guilt
> > > would present problems: it seems that every time patches are re-ordered,
> > > a patch is re-written or removed, or we update from upstream,
> > > everyone who pulls the tree branch will have a hard-to-resolve conflict.
> > > 
> > > Is that really a problem? If so, would it be possible to work around this
> > > somehow?
> > 
> > I thought about this problem a while back when I was trying to decide how to
> > manage the Unionfs git repository. I came to the conclusion, that there was
> > no clean way of doing this (at least not using guilt - I can't really speak
> > for stgit, as I don't know how it does things exactly).
> 
> StGit has the same problem. Publishing such a branch is only for viewing if
> you want to publish the tip, like the pu branch in the Git repo. You shouldn't
> merge from pu either.

You are right, in that what can be done with such branches is limited.
BUT you can safely "stg branch --create" off any remote stgit stack.
Then you can "stg rebase origin/master" to port your stack to the new
tip of the remote stack.

The next stgit release will allow you to declare the pull-policy for
your stack as "rebase", so when you "stg pull" it will indeed rebase
to the new tip of the parent branch.


As for publishing, I use the following config entries to publish my
own stack of patches to stgit.  You can see at
http://repo.or.cz/w/stgit/ydirson.git that gitweb shows pretty clearly
the structure of the stack (even though things could surely be made
better).

I use "git push -f" to publish - maybe the "+" refspec syntax would
work with push, I'll try it next time :)

[remote "orcz"]
        url = git+ssh://ydirson@repo.or.cz/srv/git/stgit/ydirson.git
        push = refs/heads/master:refs/heads/master
	push = refs/patches/master/*:refs/patches/master/*

Maybe we should provide some degree of automation in stgit itself
(eg. "stg branch --publish" or something).

Hope this helps,
-- 
Yann.

Re: using stgit/guilt for public branches

[Robin Rosenberg]

torsdag 03 maj 2007 skrev Yann Dirson:
[...]
> As for publishing, I use the following config entries to publish my
> own stack of patches to stgit.  You can see at
> http://repo.or.cz/w/stgit/ydirson.git that gitweb shows pretty clearly
> the structure of the stack (even though things could surely be made
> better).
> 
> I use "git push -f" to publish - maybe the "+" refspec syntax would
> work with push, I'll try it next time :)
> 
> [remote "orcz"]
>         url = git+ssh://ydirson@repo.or.cz/srv/git/stgit/ydirson.git
>         push = refs/heads/master:refs/heads/master
> 	push = refs/patches/master/*:refs/patches/master/*

Beautiful!!

I needed to change the update commit though for git to recognize the
refs/patches heads. Or is that only because I had initialized the repos
using v1.5.0 ?

-- robin

Re: using stgit/guilt for public branches

[Robin Rosenberg]

fredag 04 maj 2007 skrev Robin Rosenberg:
> torsdag 03 maj 2007 skrev Yann Dirson:
> [...]
> > As for publishing, I use the following config entries to publish my
> > own stack of patches to stgit.  You can see at
> > http://repo.or.cz/w/stgit/ydirson.git that gitweb shows pretty clearly
> > the structure of the stack (even though things could surely be made
> > better).
> > 
> > I use "git push -f" to publish - maybe the "+" refspec syntax would
> > work with push, I'll try it next time :)
> > 
> > [remote "orcz"]
> >         url = git+ssh://ydirson@repo.or.cz/srv/git/stgit/ydirson.git
> >         push = refs/heads/master:refs/heads/master
> > 	push = refs/patches/master/*:refs/patches/master/*
> 
> Beautiful!!
> 
Would it be possible to push only applied patches, and drop unapplied
ones? It would only matter when one wants to prune the remote repo
so it may not be terribly important, but it seems I pushed quite a 
lot of references from old patches that I haven't decided what to do
with yet and those will get new commit id's anyway.

-- robin

Re: using stgit/guilt for public branches

[Yann Dirson]

On Fri, May 04, 2007 at 01:31:17AM +0200, Robin Rosenberg wrote:
> fredag 04 maj 2007 skrev Robin Rosenberg:
> > torsdag 03 maj 2007 skrev Yann Dirson:
> > [...]
> > > As for publishing, I use the following config entries to publish my
> > > own stack of patches to stgit.  You can see at
> > > http://repo.or.cz/w/stgit/ydirson.git that gitweb shows pretty clearly
> > > the structure of the stack (even though things could surely be made
> > > better).
> > > 
> > > I use "git push -f" to publish - maybe the "+" refspec syntax would
> > > work with push, I'll try it next time :)
> > > 
> > > [remote "orcz"]
> > >         url = git+ssh://ydirson@repo.or.cz/srv/git/stgit/ydirson.git
> > >         push = refs/heads/master:refs/heads/master
> > > 	push = refs/patches/master/*:refs/patches/master/*
> > 
> > Beautiful!!
> > 
> Would it be possible to push only applied patches, and drop unapplied
> ones?

This would require stgit-level knowledge, so we would need an stgit
command.

Maybe "stg publish" or "stg branch --publish" with the current syntax,
since we already have "push".  I'd be in favor of "stg branch push"
for the next-gen syntax, though ;)

> It would only matter when one wants to prune the remote repo
> so it may not be terribly important, but it seems I pushed quite a 
> lot of references from old patches that I haven't decided what to do
> with yet and those will get new commit id's anyway.

For now you can delete them manually with "git push remote/ref:".

Best regards,
-- 
Yann.

Re: using stgit/guilt for public branches

[Michael S. Tsirkin]

> Quoting Yann Dirson <ydirson@altern.org>:
> Subject: Re: using stgit/guilt for public branches
> 
> On Wed, Apr 25, 2007 at 11:37:05PM +0200, Robin Rosenberg wrote:
> > onsdag 25 april 2007 skrev Josef Sipek:
> > > On Wed, Apr 25, 2007 at 03:20:49PM +0300, Michael S. Tsirkin wrote:
> > [...]
> > > > I am concerned that publishing a git branch managed by stg/guilt
> > > > would present problems: it seems that every time patches are re-ordered,
> > > > a patch is re-written or removed, or we update from upstream,
> > > > everyone who pulls the tree branch will have a hard-to-resolve conflict.
> > > > 
> > > > Is that really a problem? If so, would it be possible to work around this
> > > > somehow?
> > > 
> > > I thought about this problem a while back when I was trying to decide how to
> > > manage the Unionfs git repository. I came to the conclusion, that there was
> > > no clean way of doing this (at least not using guilt - I can't really speak
> > > for stgit, as I don't know how it does things exactly).
> > 
> > StGit has the same problem. Publishing such a branch is only for viewing if
> > you want to publish the tip, like the pu branch in the Git repo. You shouldn't
> > merge from pu either.
> 
> You are right, in that what can be done with such branches is limited.
> BUT you can safely "stg branch --create" off any remote stgit stack.
> Then you can "stg rebase origin/master" to port your stack to the new
> tip of the remote stack.

OK.
What happens if someone clones the repo, then reorders patches,
drops some of them, adds new patches in the middle of the stack?

-- 
MST

Re: using stgit/guilt for public branches

[Yann Dirson]

On Fri, May 04, 2007 at 08:20:59AM +0300, Michael S. Tsirkin wrote:
> > Quoting Yann Dirson <ydirson@altern.org>:
> > Subject: Re: using stgit/guilt for public branches
> > 
> > On Wed, Apr 25, 2007 at 11:37:05PM +0200, Robin Rosenberg wrote:
> > > onsdag 25 april 2007 skrev Josef Sipek:
> > > > On Wed, Apr 25, 2007 at 03:20:49PM +0300, Michael S. Tsirkin wrote:
> > > [...]
> > > > > I am concerned that publishing a git branch managed by stg/guilt
> > > > > would present problems: it seems that every time patches are re-ordered,
> > > > > a patch is re-written or removed, or we update from upstream,
> > > > > everyone who pulls the tree branch will have a hard-to-resolve conflict.
> > > > > 
> > > > > Is that really a problem? If so, would it be possible to work around this
> > > > > somehow?
> > > > 
> > > > I thought about this problem a while back when I was trying to decide how to
> > > > manage the Unionfs git repository. I came to the conclusion, that there was
> > > > no clean way of doing this (at least not using guilt - I can't really speak
> > > > for stgit, as I don't know how it does things exactly).
> > > 
> > > StGit has the same problem. Publishing such a branch is only for viewing if
> > > you want to publish the tip, like the pu branch in the Git repo. You shouldn't
> > > merge from pu either.
> > 
> > You are right, in that what can be done with such branches is limited.
> > BUT you can safely "stg branch --create" off any remote stgit stack.
> > Then you can "stg rebase origin/master" to port your stack to the new
> > tip of the remote stack.
> 
> OK.
> What happens if someone clones the repo, then reorders patches,
> drops some of them, adds new patches in the middle of the stack?

You can't do that out of the box, since you don't get a real stack
when you clone it, you only get the refs.  You would need to uncommit
patches manually, and there will not be much support to help you.

Now you're forcing me to unveil my secret plans :)

1. it would be quite easy to reconstruct a full-fledged stack from
those refs, and since you get the remote patchlogs, we could also
fetch any former version of the patch that would be still available
(more work for "stg clone")

2. if noone beats me to doing that, I'll enhance patchlogs some day to
record branching in patchlogs (eg. from "stg branch --clone" or "stg
pick"), as well as merges (eg. from "stg sync")

Note that proper merging from patchlog history will require working at
the meta-diff (ie. "diffs of diffs of trees") level, just like proper
merging at tree-level requires working at the diff level.  I don't
think we have the tools for this yet, so we still have a long way to
go :)


Best regards,
-- 
Yann.

Re: using stgit/guilt for public branches

[Yann Dirson]

On Thu, May 03, 2007 at 10:58:36PM +0200, Yann Dirson wrote:
> As for publishing, I use the following config entries to publish my
> own stack of patches to stgit.  You can see at
> http://repo.or.cz/w/stgit/ydirson.git that gitweb shows pretty clearly
> the structure of the stack (even though things could surely be made
> better).

Let's make things even more fun: I have now moved a couple of patches
(testsuite cases to demonstrate bugs still to be fixed) to a new stack
so they don't get pulled by error.  This stack itself is forked off my
master stack, and I just have to "stg pull" to get it rebased to the
new master head (requires stgit head, not in 0.12*):

[branch "bugs.stgit"]
        pull-policy = rebase
        parentbranch = master

> I use "git push -f" to publish - maybe the "+" refspec syntax would
> work with push, I'll try it next time :)

It indeed works as expected, so the final syntax is as follows, so
"git push orcz" is now sufficient to publish everything, and you can
now fork your own stack off my remote "bugs" stack, which itself
forked off my "master" stack - enjoy :)

[remote "orcz"]
        url = git+ssh://ydirson@repo.or.cz/srv/git/stgit/ydirson.git
        fetch = +refs/heads/*:refs/remotes/orcz/*

        push = +refs/heads/master:refs/heads/master
        push = +refs/patches/master/*:refs/patches/master/*

        push = +refs/heads/bugs:refs/heads/bugs
        push = +refs/patches/bugs/*:refs/patches/bugs/*

Best regards,
-- 
Yann.