Stacked GIT 0.3 (now more Quilt-like)

Stacked GIT 0.3 (now more Quilt-like)

[Catalin Marinas]

A new StGIT release is available from http://procode.org/stgit/

What's new in version 0.3:
      * closer to the Quilt functionality
      * there is only one commit object per patch which can be
        indefinitely modified using the 'refresh' command. The commit
        objects are stacked on top of the base and can also be accessed
        via standard GIT commands
      * no 'commit' command. Use 'refresh' instead

StGIT is a Python application providing similar functionality to Quilt
(i.e. pushing/popping patches to/from a stack) on top of GIT. These
operations are performed using GIT commands and the patches are stored
as GIT commit objects, allowing easy merging of the StGIT patches into
other repositories using standard GIT functionality.

Note that StGIT is not an SCM interface on top of GIT and it expects a
previously initialised GIT repository. For standard SCM operations,
either use plain GIT commands or the Cogito tool.

For more information, see the README file in the archive.

--
Catalin

Re: Stacked GIT 0.3 (now more Quilt-like)

[Peter Osterlund]

Catalin Marinas <catalin.marinas@gmail.com> writes:

> A new StGIT release is available from http://procode.org/stgit/
> 
> What's new in version 0.3:
>       * closer to the Quilt functionality
>       * there is only one commit object per patch which can be
>         indefinitely modified using the 'refresh' command. The commit
>         objects are stacked on top of the base and can also be accessed
>         via standard GIT commands
>       * no 'commit' command. Use 'refresh' instead
> 
> StGIT is a Python application providing similar functionality to Quilt
> (i.e. pushing/popping patches to/from a stack) on top of GIT. These
> operations are performed using GIT commands and the patches are stored
> as GIT commit objects, allowing easy merging of the StGIT patches into
> other repositories using standard GIT functionality.

This is good stuff and the 3-way merge really simplifies things.
However, if there is a merge conflict, you will basically be stuck
with a 2-way merge when resolving manually. It's usually much easier
if you can see all three version, so I think it's better to use -A
instead of -E in the diff3 command.

--- stgit-0.3/gitmergeonefile.py~	2005-06-28 14:15:22.000000000 +0200
+++ stgit-0.3/gitmergeonefile.py	2005-07-03 02:33:13.000000000 +0200
@@ -108,7 +108,7 @@
         # 3-way merge
         else:
             merge_ok = os.system('diff3 -L local -L older -L remote '
-                                 '-m -E %s %s %s > %s'
+                                 '-m -A %s %s %s > %s'
                                   % (src1, orig, src2, path)) == 0
 
             if merge_ok:

-- 
Peter Osterlund - petero2@telia.com
http://web.telia.com/~u89404340

Re: Stacked GIT 0.3 (now more Quilt-like)

[Catalin Marinas]

Hi Peter,

Thanks for trying this tool.

On Sun, 2005-07-03 at 10:38 +0200, Peter Osterlund wrote:
> This is good stuff and the 3-way merge really simplifies things.
> However, if there is a merge conflict, you will basically be stuck
> with a 2-way merge when resolving manually. It's usually much easier
> if you can see all three version, so I think it's better to use -A
> instead of -E in the diff3 command.

I know that using -A gives a more detailed output in case of a conflict.
The problem is that you will get a conflict even if the changes are
identical, making it impossible to detect when a patch was merged
upstream.

For example:

ancestor:	aaa
branch1:	aaa
		bbb
branch2:	aaa
		bbb

diff3 -E:	aaa
		bbb
diff3 -A:	aaa
		<<<<<<< ancestor
		=======
		bbb
		>>>>>>> branch2

In version 0.3, in case of a conflict, StGIT leaves 3 files in the tree:
file.older, file.local and file.remote so that one can use a graphical
tool to do the merge. I think these names are a bit misleading, .remote
extension means the file in the patch and .local is the one in the tree
before pushing the patch. Anyway, it might be invoking a graphical tool
directly from gitmergeonefile.py.

Speaking of detecting upstream merges, the latest StGIT snapshot shows a
'0' in front of a patch if it is empty, when 'stg series' is invoked.
When pushing, if all the changes are the same, it notifies you that the
patch is empty so that it can be safely removed.

--
Catalin

Re: Stacked GIT 0.3 (now more Quilt-like)

[Peter Osterlund]

Catalin Marinas <catalin.marinas@gmail.com> writes:

> Hi Peter,
> 
> Thanks for trying this tool.
> 
> On Sun, 2005-07-03 at 10:38 +0200, Peter Osterlund wrote:
> > This is good stuff and the 3-way merge really simplifies things.
> > However, if there is a merge conflict, you will basically be stuck
> > with a 2-way merge when resolving manually. It's usually much easier
> > if you can see all three version, so I think it's better to use -A
> > instead of -E in the diff3 command.
> 
> I know that using -A gives a more detailed output in case of a conflict.
> The problem is that you will get a conflict even if the changes are
> identical, making it impossible to detect when a patch was merged
> upstream.

OK, I see. How about using wiggle instead?

        http://cgi.cse.unsw.edu.au/~neilb/source/wiggle/

That's what patch-utils uses if you run "pushpatch -m". wiggle is also
a lot smarter than diff3, so there will be fewer cases that result in
a conflict. Maybe a parameter to "stg push" could enable wiggle mode.

Another nice thing from patch-utils is that if applying the patch
would have failed, nothing will be done by "pushpatch". You then have
the option to rerun it with -m (merge) or -f (force, create .rej
files), or decide that you don't want to push the patch at all. The
last part is quite useful if you try to reorder a patch series, find
out that you would get a thousand conflicts, and want to reconsider.

Is there a way in StGIT to undo a push that results in a large mess of
conflicts?

> Speaking of detecting upstream merges, the latest StGIT snapshot shows a
> '0' in front of a patch if it is empty, when 'stg series' is invoked.
> When pushing, if all the changes are the same, it notifies you that the
> patch is empty so that it can be safely removed.

That's a useful feature. With patch-utils, I used to drop patches
manually, but that could lose information if the patch applied
upstream is not exactly the same as the one I had locally.

-- 
Peter Osterlund - petero2@telia.com
http://web.telia.com/~u89404340

Re: Stacked GIT 0.3 (now more Quilt-like)

[Catalin Marinas]

On Sun, 2005-07-03 at 14:38 +0200, Peter Osterlund wrote:
> Catalin Marinas <catalin.marinas@gmail.com> writes:
> > I know that using -A gives a more detailed output in case of a conflict.
> > The problem is that you will get a conflict even if the changes are
> > identical, making it impossible to detect when a patch was merged
> > upstream.
> 
> OK, I see. How about using wiggle instead?
> 
>         http://cgi.cse.unsw.edu.au/~neilb/source/wiggle/
> 
> That's what patch-utils uses if you run "pushpatch -m". wiggle is also
> a lot smarter than diff3, so there will be fewer cases that result in
> a conflict. Maybe a parameter to "stg push" could enable wiggle mode.

I haven't used wiggle before but I will give it a try (though I prefer
such a tool not to be too smart since it might make mistakes). Anyway, I
will make this configurable, i.e. you could put something like below in
the .stgitrc file:

merger = 'diff3 -m -E %(branch1)s %(ancestor)s %(branch2)s'

or

merger = 'wiggle -m %(branch1)s %(ancestor)s %(branch2)s'

> Is there a way in StGIT to undo a push that results in a large mess of
> conflicts?

Good point. No, there isn't yet. I will think about an undo command. At
the moment, the old top and bottom ids of a patch are saved so that the
patch before the merge can be retrieved but there isn't any command to
make use of them.

--
Catalin

Re: Stacked GIT 0.3 (now more Quilt-like)

[Horst von Brand]

Catalin Marinas <catalin.marinas@gmail.com> wrote:
> On Sun, 2005-07-03 at 14:38 +0200, Peter Osterlund wrote:
> > Catalin Marinas <catalin.marinas@gmail.com> writes:
> > > I know that using -A gives a more detailed output in case of a conflict.
> > > The problem is that you will get a conflict even if the changes are
> > > identical, making it impossible to detect when a patch was merged
> > > upstream.

> > OK, I see. How about using wiggle instead?
> > 
> >         http://cgi.cse.unsw.edu.au/~neilb/source/wiggle/

[...]

> I haven't used wiggle before but I will give it a try (though I prefer
> such a tool not to be too smart since it might make mistakes).

Last version is 0.6, released 2 years back... I'm not sure it is wise to
rely on abandonware.
-- 
Dr. Horst H. von Brand                   User #22616 counter.li.org
Departamento de Informatica                     Fono: +56 32 654431
Universidad Tecnica Federico Santa Maria              +56 32 654239
Casilla 110-V, Valparaiso, Chile                Fax:  +56 32 797513

Re: Stacked GIT 0.3 (now more Quilt-like)

[Martin Langhoff]

> > OK, I see. How about using wiggle instead?
> >
> >         http://cgi.cse.unsw.edu.au/~neilb/source/wiggle/
> >
> > That's what patch-utils uses if you run "pushpatch -m". wiggle is also
> > a lot smarter than diff3, so there will be fewer cases that result in
> > a conflict. Maybe a parameter to "stg push" could enable wiggle mode.
> 
> I haven't used wiggle before but I will give it a try (though I prefer
> such a tool not to be too smart since it might make mistakes). Anyway, I
> will make this configurable, i.e. you could put something like below in
> the .stgitrc file:

wiggle is good but it's not safe. You should definitely review the
results of a wiggle merge with extra care. It sometimes does magic and
merges the unmergeable, and sometimes it does bad, bad things.

cheers,


martin

Re: Stacked GIT 0.3 (now more Quilt-like)

[Peter Osterlund]

Catalin Marinas <catalin.marinas@gmail.com> writes:

> On Sun, 2005-07-03 at 14:38 +0200, Peter Osterlund wrote:
> > Catalin Marinas <catalin.marinas@gmail.com> writes:
> > > I know that using -A gives a more detailed output in case of a conflict.
> > > The problem is that you will get a conflict even if the changes are
> > > identical, making it impossible to detect when a patch was merged
> > > upstream.
> > 
> > OK, I see. How about using wiggle instead?
> > 
> >         http://cgi.cse.unsw.edu.au/~neilb/source/wiggle/
> > 
> > That's what patch-utils uses if you run "pushpatch -m". wiggle is also
> > a lot smarter than diff3, so there will be fewer cases that result in
> > a conflict. Maybe a parameter to "stg push" could enable wiggle mode.
> 
> I haven't used wiggle before but I will give it a try (though I prefer
> such a tool not to be too smart since it might make mistakes). Anyway, I
> will make this configurable, i.e. you could put something like below in
> the .stgitrc file:
> 
> merger = 'diff3 -m -E %(branch1)s %(ancestor)s %(branch2)s'
> 
> or
> 
> merger = 'wiggle -m %(branch1)s %(ancestor)s %(branch2)s'
> 
> > Is there a way in StGIT to undo a push that results in a large mess of
> > conflicts?
> 
> Good point. No, there isn't yet. I will think about an undo command. At
> the moment, the old top and bottom ids of a patch are saved so that the
> patch before the merge can be retrieved but there isn't any command to
> make use of them.

I agree with the other comments, it's probably not wise to rely on
wiggle, and wiggle sometimes makes a mess. However, it often does the
right thing, and with a configurable merge program and an undo
function, this should not be a problem. Just undo and try again if you
don't like the result.

-- 
Peter Osterlund - petero2@telia.com
http://web.telia.com/~u89404340

Re: Stacked GIT 0.3 (now more Quilt-like)

[randy_dunlap]

On 04 Jul 2005 14:32:36 +0200 Peter Osterlund wrote:

| Catalin Marinas <catalin.marinas@gmail.com> writes:
| 
| > On Sun, 2005-07-03 at 14:38 +0200, Peter Osterlund wrote:
| > > Catalin Marinas <catalin.marinas@gmail.com> writes:
| > > > I know that using -A gives a more detailed output in case of a conflict.
| > > > The problem is that you will get a conflict even if the changes are
| > > > identical, making it impossible to detect when a patch was merged
| > > > upstream.
| > > 
| > > OK, I see. How about using wiggle instead?
| > > 
| > >         http://cgi.cse.unsw.edu.au/~neilb/source/wiggle/
| > > 
| > > That's what patch-utils uses if you run "pushpatch -m". wiggle is also
| > > a lot smarter than diff3, so there will be fewer cases that result in
| > > a conflict. Maybe a parameter to "stg push" could enable wiggle mode.
| > 
| > I haven't used wiggle before but I will give it a try (though I prefer
| > such a tool not to be too smart since it might make mistakes). Anyway, I
| > will make this configurable, i.e. you could put something like below in
| > the .stgitrc file:
| > 
| > merger = 'diff3 -m -E %(branch1)s %(ancestor)s %(branch2)s'
| > 
| > or
| > 
| > merger = 'wiggle -m %(branch1)s %(ancestor)s %(branch2)s'
| > 
| > > Is there a way in StGIT to undo a push that results in a large mess of
| > > conflicts?
| > 
| > Good point. No, there isn't yet. I will think about an undo command. At
| > the moment, the old top and bottom ids of a patch are saved so that the
| > patch before the merge can be retrieved but there isn't any command to
| > make use of them.
| 
| I agree with the other comments, it's probably not wise to rely on
| wiggle, and wiggle sometimes makes a mess. However, it often does the
| right thing, and with a configurable merge program and an undo
| function, this should not be a problem. Just undo and try again if you
| don't like the result.

You could try Chris Mason's 'rej' as well.
ftp://ftp.oregonstate.edu/pub/suse/people/mason/rej/rej-0.15.tar.gz
(from one mirror site)

---
~Randy

Re: Stacked GIT 0.3 (now more Quilt-like)

[Catalin Marinas]

On Mon, 2005-07-04 at 14:32 +0200, Peter Osterlund wrote:
> I agree with the other comments, it's probably not wise to rely on
> wiggle, and wiggle sometimes makes a mess. However, it often does the
> right thing, and with a configurable merge program and an undo
> function, this should not be a problem. Just undo and try again if you
> don't like the result.

I uploaded a new snapshot (stgit-20050704.tar.gz) which has support for
merge tool configuration. It installs a default /etc/stgitrc file but
you can define your own options in ~/.stgitrc or .git/stgitrc (the
latter overrides the options in the earlier files). You can even define
something like - 'diff3 ... || xxdiff ...' so that if one tool fails it
will try the next one (the whole command is executed with 'sh -c').

I will try to write an 'undo' command in the next days.

--
Catalin

Re: Stacked GIT 0.3 (now more Quilt-like)

[Catalin Marinas]

On Mon, 2005-07-04 at 14:32 +0200, Peter Osterlund wrote:
> I agree with the other comments, it's probably not wise to rely on
> wiggle, and wiggle sometimes makes a mess. However, it often does the
> right thing, and with a configurable merge program and an undo
> function, this should not be a problem. Just undo and try again if you
> don't like the result.

In the today's snapshot you can get the 'stg push --undo' command which
reverts the result of a push operation (either failed or not). The patch
is reverted to its previous state. It works even if you ran 'refresh'.

The current implementation does not remove the .older/.local/.remote
files from the tree when undoing a push. I think I will first implement
a 'resolve' command which takes care of these files.

Anyway, once I fully test the current state of stgit, I will make the
0.4 release (probably this weekend).

--
Catalin

Re: Stacked GIT 0.3 (now more Quilt-like)

[Peter Osterlund]

Catalin Marinas <catalin.marinas@gmail.com> writes:

> On Mon, 2005-07-04 at 14:32 +0200, Peter Osterlund wrote:
> > I agree with the other comments, it's probably not wise to rely on
> > wiggle, and wiggle sometimes makes a mess. However, it often does the
> > right thing, and with a configurable merge program and an undo
> > function, this should not be a problem. Just undo and try again if you
> > don't like the result.
> 
> In the today's snapshot you can get the 'stg push --undo' command which
> reverts the result of a push operation (either failed or not). The patch
> is reverted to its previous state. It works even if you ran 'refresh'.

Thanks, this seems to work as expected.

> The current implementation does not remove the .older/.local/.remote
> files from the tree when undoing a push. I think I will first implement
> a 'resolve' command which takes care of these files.
> 
> Anyway, once I fully test the current state of stgit, I will make the
> 0.4 release (probably this weekend).

I've found an unrelated problem. If I export patches with "stg export
dirname", there are no diffs included in the patches. The patch
description is all that is generated. If I omit the dirname parameter,
the export works correctly though.

-- 
Peter Osterlund - petero2@telia.com
http://web.telia.com/~u89404340

Re: Stacked GIT 0.3 (now more Quilt-like)

[Catalin Marinas]

On Thu, 2005-07-07 at 21:17 +0200, Peter Osterlund wrote:
> I've found an unrelated problem. If I export patches with "stg export
> dirname", there are no diffs included in the patches. The patch
> description is all that is generated. If I omit the dirname parameter,
> the export works correctly though.

OK, thanks. Fixed in the today's snapshot. I also added a 'resolved'
command to mark a solved conflict. 'stg status' also shows the conflicts
and 'refresh', 'push', 'pop' fail if there are unresolved conflicts. You
can override this by setting 'autoresolved: yes' in the stgitrc file and
a 'refresh' command marks all the conflicts as solved automatically (I
still need to find some time to write a tutorial for StGIT).

Catalin

Re: Stacked GIT 0.3 (now more Quilt-like)

[Peter Osterlund]

Catalin Marinas <catalin.marinas@gmail.com> writes:

> A new StGIT release is available from http://procode.org/stgit/

I think it would be good if it was possible to include diffstat output
in exported patches, something like this:


Added possibility to include diffstat output in exported patches.

Signed-off-by: Peter Osterlund <petero2@telia.com>
---

 b/stgit/git.py  |   22 ++++++++++++++++++++++
 b/stgit/main.py |    2 ++
 2 files changed, 24 insertions(+)

diff --git a/stgit/git.py b/stgit/git.py
--- a/stgit/git.py
+++ b/stgit/git.py
@@ -345,6 +345,28 @@ def diff(files = [], rev1 = 'HEAD', rev2
                      % (rev1, files_str, extra_args)) != 0:
             raise GitException, 'git-diff-cache failed'
 
+def diffstat(files = [], rev1 = 'HEAD', rev2 = None):
+    """Return the diffstat between rev1 and rev2
+    """
+    files_str = reduce(lambda x, y: x + ' ' + y, files, '')
+
+    os.system('git-update-cache --refresh > /dev/null')
+
+    if rev2:
+        f = os.popen('git-diff-tree -p %s %s %s | diffstat -p0'
+                     % (rev1, rev2, files_str), 'r')
+        str = f.read()
+        if f.close():
+            raise GitException, 'git-diff-tree failed'
+        return str
+    else:
+        f = os.popen('git-diff-cache -p %s %s | diffstat -p0'
+                     % (rev1, files_str), 'r')
+        str = f.read()
+        if f.close():
+            raise GitException, 'git-diff-cache failed'
+        return str
+
 def checkout(files = [], force = False):
     """Check out the given or all files
     """
diff --git a/stgit/main.py b/stgit/main.py
--- a/stgit/main.py
+++ b/stgit/main.py
@@ -659,6 +659,8 @@ def export(parser, options, args):
         patch = crt_series.get_patch(p)
 
         tmpl_dict = {'description': patch.get_description().rstrip(),
+                     'diffstat': git.diffstat(rev1 = __git_id('%s/bottom' % p),
+                                              rev2 = __git_id('%s/top' % p)),
                      'authname': patch.get_authname(),
                      'authemail': patch.get_authemail(),
                      'authdate': patch.get_authdate(),

-- 
Peter Osterlund - petero2@telia.com
http://web.telia.com/~u89404340

Re: Stacked GIT 0.3 (now more Quilt-like)

[Junio C Hamano]

>>>>> "PO" == Peter Osterlund <petero2@telia.com> writes:

PO> I think it would be good if it was possible to include diffstat output
PO> in exported patches, something like this:

PO> Added possibility to include diffstat output in exported patches.

And you wuold want to do things similarly to what others have
already done.  git-resolve-script uses "git-apply --stat" there,
instead of "diffstat", so people without diffstat installed can
still get the same information.

Even if you want to stick to "diffstat", at least I would
suggest giving -p1, not -p0, to it; otherwise you would see b/
like what you have there.

PO>  b/stgit/git.py  |   22 ++++++++++++++++++++++
PO>  b/stgit/main.py |    2 ++
PO>  2 files changed, 24 insertions(+)

Further, using "git-apply --stat --summary" would be nicer; this
is something you cannot do with plain "diffstat".

Re: Stacked GIT 0.3 (now more Quilt-like)

[Peter Osterlund]

Junio C Hamano <junkio@cox.net> writes:

> >>>>> "PO" == Peter Osterlund <petero2@telia.com> writes:
> 
> PO> I think it would be good if it was possible to include diffstat output
> PO> in exported patches, something like this:
> 
> PO> Added possibility to include diffstat output in exported patches.
> 
> And you wuold want to do things similarly to what others have
> already done.  git-resolve-script uses "git-apply --stat" there,
> instead of "diffstat", so people without diffstat installed can
> still get the same information.

Thanks for the hint. Here is a replacement patch that uses git-apply
instead.


Added possibility to include diffstat output in exported patches.

Signed-off-by: Peter Osterlund <petero2@telia.com>
---

 stgit/git.py  |   23 +++++++++++++++++++++++
 stgit/main.py |    2 ++
 2 files changed, 25 insertions(+), 0 deletions(-)

diff --git a/stgit/git.py b/stgit/git.py
--- a/stgit/git.py
+++ b/stgit/git.py
@@ -345,6 +345,29 @@ def diff(files = [], rev1 = 'HEAD', rev2
                      % (rev1, files_str, extra_args)) != 0:
             raise GitException, 'git-diff-cache failed'
 
+def diffstat(files = [], rev1 = 'HEAD', rev2 = None):
+    """Return the diffstat between rev1 and rev2
+    """
+    files_str = reduce(lambda x, y: x + ' ' + y, files, '')
+
+    os.system('git-update-cache --refresh > /dev/null')
+    ds_cmd = '| git-apply --stat --summary'
+
+    if rev2:
+        f = os.popen('git-diff-tree -p %s %s %s %s'
+                     % (rev1, rev2, files_str, ds_cmd), 'r')
+        str = f.read()
+        if f.close():
+            raise GitException, 'git-diff-tree failed'
+        return str
+    else:
+        f = os.popen('git-diff-cache -p %s %s %s'
+                     % (rev1, files_str, ds_cmd), 'r')
+        str = f.read()
+        if f.close():
+            raise GitException, 'git-diff-cache failed'
+        return str
+
 def checkout(files = [], force = False):
     """Check out the given or all files
     """
diff --git a/stgit/main.py b/stgit/main.py
--- a/stgit/main.py
+++ b/stgit/main.py
@@ -659,6 +659,8 @@ def export(parser, options, args):
         patch = crt_series.get_patch(p)
 
         tmpl_dict = {'description': patch.get_description().rstrip(),
+                     'diffstat': git.diffstat(rev1 = __git_id('%s/bottom' % p),
+                                              rev2 = __git_id('%s/top' % p)),
                      'authname': patch.get_authname(),
                      'authemail': patch.get_authemail(),
                      'authdate': patch.get_authdate(),

-- 
Peter Osterlund - petero2@telia.com
http://web.telia.com/~u89404340

Re: Stacked GIT 0.3 (now more Quilt-like)

[Catalin Marinas]

Peter Osterlund <petero2@telia.com> wrote:
> Added possibility to include diffstat output in exported patches.

Great. I had a plan to do this but I was busy with the push and
resolved commands. I will apply this patch.

Thanks.

-- 
Catalin