* Re: cvs import
From: Nathaniel Smith @ 2006-09-14 0:32 UTC (permalink / raw)
To: Keith Packard; +Cc: dev, monotone-devel, Git Mailing List
In-Reply-To: <1158190921.29313.175.camel@neko.keithp.com>
On Wed, Sep 13, 2006 at 04:42:01PM -0700, Keith Packard wrote:
> However, this means that parsecvs must hold the entire tree state in
> memory, which turned out to be its downfall with large repositories.
> Worked great for all of X.org, not so good with Mozilla.
Does anyone know how big Mozilla (or other humonguous repos, like KDE)
are, in terms of number of files?
A few numbers for repositories I had lying around:
Linux kernel -- ~21,000
gcc -- ~42,000
NetBSD "src" repo -- ~100,000
uClinux distro -- ~110,000
These don't seem very indimidating... even if it takes an entire
kilobyte per CVS revision to store the information about it that we
need to make decisions about how to move the frontier... that's only
110 megabytes for the largest of these repos. The frontier sweeping
algorithm only _needs_ to have available the current frontier, and the
current frontier+1. Storing information on every version of every
file in memory might be worse; but since the algorithm accesses this
data in a linear way, it'd be easy enough to stick those in a
lookaside table on disk if really necessary, like a bdb or sqlite file
or something.
(Again, in practice storing all the metadata for the entire 180k
revisions of the 100k files in the netbsd repo was possible on a
desktop. Monotone's cvs_import does try somewhat to be frugal about
memory, though, interning strings and suchlike.)
-- Nathaniel
--
When the flush of a new-born sun fell first on Eden's green and gold,
Our father Adam sat under the Tree and scratched with a stick in the mould;
And the first rude sketch that the world had seen was joy to his mighty heart,
Till the Devil whispered behind the leaves, "It's pretty, but is it Art?"
-- The Conundrum of the Workshops, Rudyard Kipling
^ permalink raw reply
* Re: [Monotone-devel] cvs import
From: Daniel Carosone @ 2006-09-13 23:52 UTC (permalink / raw)
To: Markus Schiltknecht, monotone-devel, dev, Git Mailing List
In-Reply-To: <20060913232139.GU29625@bcd.geek.com.au>
[-- Attachment #1: Type: text/plain, Size: 1478 bytes --]
On Thu, Sep 14, 2006 at 09:21:39AM +1000, Daniel Carosone wrote:
> > I have no particular idea on how to handle tags and branches here;
> > I've never actually wrapped my head around CVS's model for those :-).
> > I'm not seeing any obvious problem with handling them, though.
>
> Tags could be modelled as another 'event' in the file graph, like a
> commit. If your frontier advances through both revisions and a 'tag
> this revision' event, the same sequencing as above would work.
Likewise, if we had "file branched" events in the file lifeline (based
on the rcs id's), then we would be sure to always have a monotone
revision that corresponded to the branching event, where we could
attach the revisions in the branch.
Because we can't split tags, and can't split branch events, we will
end up splitting file commits (down to individual commits per file) in
order to arrive at the revisions we need for those.
Because tags and branches can be across subsets of the tree, we gain
some scheduling flexibility about where in the reconstructed sequence
they can come.
Many well-managed CVS repositories will use good practices, such as
having a branch base tag. If they do, then they will help this
algorithm produce correct results.
Once we have a branch with a base starting revision, we can pretty
much treat it independently from there: make a whole new set of file
lifelines along the RCS branches and a new frontier for it.
--
Dan.
[-- Attachment #2: Type: application/pgp-signature, Size: 186 bytes --]
^ permalink raw reply
* Re: [Monotone-devel] cvs import
From: Keith Packard @ 2006-09-13 23:42 UTC (permalink / raw)
To: Nathaniel Smith
Cc: keithp, Markus Schiltknecht, monotone-devel, dev,
Git Mailing List
In-Reply-To: <20060913225200.GA10186@frances.vorpus.org>
[-- Attachment #1: Type: text/plain, Size: 919 bytes --]
On Wed, 2006-09-13 at 15:52 -0700, Nathaniel Smith wrote:
> Regarding the basic dependency-based algorithm, the approach of
> throwing everything into blobs and then trying to tease them apart
> again seems backwards. What I'm thinking is, first we go through and
> build the history graph for each file. Now, advance a frontier across
> the all of these graphs simultaneously. Your frontier is basically a
> map <filename -> CVS revision>, that represents a tree snapshot.
Parsecvs does this, except backwards from now into the past; I found it
easier to identify merge points than branch points (Oh, look, these two
branches are the same now, they must have merged).
However, this means that parsecvs must hold the entire tree state in
memory, which turned out to be its downfall with large repositories.
Worked great for all of X.org, not so good with Mozilla.
--
keith.packard@intel.com
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]
^ permalink raw reply
* Re: cvs import
From: Daniel Carosone @ 2006-09-13 23:21 UTC (permalink / raw)
To: Markus Schiltknecht, monotone-devel, dev, Git Mailing List
In-Reply-To: <20060913225200.GA10186@frances.vorpus.org>
[-- Attachment #1.1: Type: text/plain, Size: 3325 bytes --]
On Wed, Sep 13, 2006 at 03:52:00PM -0700, Nathaniel Smith wrote:
> This isn't trivial problem. I think the main thing you want to avoid
> is:
> 1 2 3 4
> | | | |
> --o--o--o--o----- <-- current frontier
> | | | |
> A B A C
> |
> A
> There are a lot of approaches one could take here, on up to pulling
> out a full-on optimal constraint satisfaction system (if we can route
> chips, we should be able to pick a good ordering for accepting CVS
> edits, after all). A really simple heuristic, though, would be to
> just pick the file whose next commit has the earliest timestamp, then
> group in all the other "next commits" with the same commit message,
> and (maybe) a similar timestamp.
Pick the earliest first, or more generally: take all the file commits
immediately below the frontier. Find revs further below the frontier
(up to some small depth or time limit) on other files that might match
them, based on changelog etc (the same grouping you describe, and we
do now). Eliminate any of those that are not entirely on the frontier
(ie, have some other revision in the way, as with file 2). Commit the
remaining set in time order. [*]
If you wind up with an empty set, then you need to split revs, but at
this point you have only conflicting revs on the frontier i.e. you've
already committed all the other revs you can that might have avoided
this need, whereas we currently might be doing this too often).
For time order, you could look at each rev as having a time window,
from the first to last commit matching. If the revs windows are
non-overlapping, commit them in order. If the rev windows overlap, at
this point we already know the file changes don't overlap - we *could*
commit these as parallel heads and merge them, to better model the
original developer's overlapping commits.
> Handling file additions could potentially be slightly tricky in this
> model. I guess it is not so bad, if you model added files as being
> present all along (so you never have to add add whole new entries to
> the frontier), with each file starting out in a pre-birth state, and
> then addition of the file is the first edit performed on top of that,
> and you treat these edits like any other edits when considering how to
> advance the frontier.
CVS allows resurrections too..
> I have no particular idea on how to handle tags and branches here;
> I've never actually wrapped my head around CVS's model for those :-).
> I'm not seeing any obvious problem with handling them, though.
Tags could be modelled as another 'event' in the file graph, like a
commit. If your frontier advances through both revisions and a 'tag
this revision' event, the same sequencing as above would work. If tags
had been moved, this would wind up with a sequence whereby commits
interceded with tagging, and we'd need to split the commits such that
we could end up with a revision matching the tagged content.
> In this approach, incremental conversion is cheap, easy, and robust --
> simply remember what frontier corresponded to the final revision
> imported, and restart the process directly at that frontier.
Hm. Except for the tagging idea above, because tags can be applied
behind a live cvs frontier.
--
Dan.
[-- Attachment #1.2: Type: application/pgp-signature, Size: 186 bytes --]
[-- Attachment #2: Type: text/plain, Size: 158 bytes --]
_______________________________________________
Monotone-devel mailing list
Monotone-devel@nongnu.org
http://lists.nongnu.org/mailman/listinfo/monotone-devel
^ permalink raw reply
* Re: cvs import
From: Nathaniel Smith @ 2006-09-13 22:52 UTC (permalink / raw)
To: Markus Schiltknecht; +Cc: dev, monotone-devel, Git Mailing List
In-Reply-To: <45084400.1090906@bluegap.ch>
On Wed, Sep 13, 2006 at 07:46:40PM +0200, Markus Schiltknecht wrote:
> Hi,
>
> I've been trying to understand the cvsimport algorithm used by monotone
> and wanted to adjust that to be more like the one in cvs2svn.
>
> I've had some problems with cvs2svn itself and began to question the
> algorithm used there. It turned out that the cvs2svn people have
> discussed an improved algorithms and are about to write a cvs2svn 2.0.
> The main problem with the current algorithm is that it depends on the
> timestamp information stored in the CVS repository.
>
> Instead, it would be much better to just take the dependencies of the
> revisions into account. Considering the timestamp an irrelevant (for the
> import) attribute of the revision.
I just read over the thread on the cvs2svn list about this -- I have a
few random thoughts. Take them with a grain of salt, since I haven't
actually tried writing a CVS importer myself...
Regarding the basic dependency-based algorithm, the approach of
throwing everything into blobs and then trying to tease them apart
again seems backwards. What I'm thinking is, first we go through and
build the history graph for each file. Now, advance a frontier across
the all of these graphs simultaneously. Your frontier is basically a
map <filename -> CVS revision>, that represents a tree snapshot. The
basic loop is:
1) pick some subset of files to advance to their next revision
2) slide the frontier one CVS revision forward on each of those
files
3) snapshot the new frontier (write it to the target VCS as a new
tree commit)
4) go to step 1
Obviously, this will produce a target VCS history that respects the
CVS dependency graph, so that's good; it puts a strict limit on how
badly whatever heuristics we use can screw us over if they guess wrong
about things. Also, it makes the problem much simpler -- all the
heuristics are now in step 1, where we are given a bunch of possible
edits, and we have to pick some subset of them to accept next.
This isn't trivial problem. I think the main thing you want to avoid
is:
1 2 3 4
| | | |
--o--o--o--o----- <-- current frontier
| | | |
A B A C
|
A
say you have four files named "1", "2", "3", and "4". We want to
slide the frontier down, and the next edits were originally created by
one of three commits, A, B, or C. In this situation, we can take
commit B, or we can take commit C, but we don't want to take commit A
until _after_ we have taken commit B -- because otherwise we will end
up splitting A up into two different commits, A1, B, A2.
There are a lot of approaches one could take here, on up to pulling
out a full-on optimal constraint satisfaction system (if we can route
chips, we should be able to pick a good ordering for accepting CVS
edits, after all). A really simple heuristic, though, would be to
just pick the file whose next commit has the earliest timestamp, then
group in all the other "next commits" with the same commit message,
and (maybe) a similar timestamp. I have a suspicion that this
heuristic will work really, really, well in practice. Also, it's
cheap to apply, and worst case you accidentally split up a commit that
already had wacky timestamps, and we already know that we _have_ to do
that in some cases.
Handling file additions could potentially be slightly tricky in this
model. I guess it is not so bad, if you model added files as being
present all along (so you never have to add add whole new entries to
the frontier), with each file starting out in a pre-birth state, and
then addition of the file is the first edit performed on top of that,
and you treat these edits like any other edits when considering how to
advance the frontier.
I have no particular idea on how to handle tags and branches here;
I've never actually wrapped my head around CVS's model for those :-).
I'm not seeing any obvious problem with handling them, though.
In this approach, incremental conversion is cheap, easy, and robust --
simply remember what frontier corresponded to the final revision
imported, and restart the process directly at that frontier.
Regarding storing things on disk vs. in memory: we always used to
stress-test monotone's cvs importer with the gcc history; just a few
weeks ago someone did a test import of NetBSD's src repo (~180k
commits) on a desktop with 2 gigs of RAM. It takes a pretty big
history to really require disk (and for that matter, people with
histories that big likely have a big enough organization that they can
get access to some big iron to run the conversion on -- and probably
will want to anyway, to make it run in reasonable time).
> Now, that can be used to convert from CVS to about anything else.
> Obviously we were discussing about subversion, but then there was git,
> too. And monotone.
>
> I'm beginning to question if one could come up with a generally useful
> cleaned-and-sane-CVS-changeset-dump-format, which could then be used by
> importers to all sort of VCSes. This would make monotone's cvsimport
> function dependent on cvs2svn (and therefore python). But the general
> try-to-get-something-usefull-from-an-insane-CVS-repository-algorithm
> would only have to be written once.
>
> On the other hand, I see that lots of the cvsimport functionality for
> monotone has already been written (rcs file parsing, stuffing files,
> file deltas and complete revisions into the monotone database, etc..).
> Changing it to a better algorithm does not seem to be _that_ much work
> anymore. Plus the hard part seems to be to come up with a good
> algorithm, not implementing it. And we could still exchange our
> experience with the general algorithm with the cvs2svn people.
>
> Plus, the guy who mentioned git pointed out that git needs quite a
> different dump-format than subversion to do an efficient conversion. I
> think coming up with a generally-usable dump format would not be that easy.
Probably the biggest technical advantage of having the converter built
into monotone is that it makes it easy to import the file contents.
Since this data is huge (100x the repo size, maybe?), and the naive
algorithm for reconstructing takes time that is quadratic in the depth
of history, this is very valuable. I'm not sure what sort of dump
format one could come up with that would avoid making this step very
expensive.
I also suspect that SVN's dump format is suboptimal at the metadata
level -- we would essentially have to run a lot of branch/tag
inferencing logic _again_ to go from SVN-style "one giant tree with
branches described as copies, and multiple copies allowed for
branches/tags that are built up over time", to monotone-style
"DAG of tree snapshots". This would be substantially less annoying
inferencing logic than that needed to decipher CVS in the first place,
granted, and it's stuff we want to write at some point anyway to allow
SVN importing, but it adds another step where information could be
lost. I may be biased because I grok monotone better, but I suspect
it would be much easier to losslessly convert a monotone-style history
to an svn-style history than vice versa, possibly a generic dumping
tool would want to generate output that looks more like monotone's
model? The biggest stumbling block I see is if it is important to
build up branches and tags by multiple copies out of trunk -- there
isn't any way to represent that in monotone. A generic tool could
also use some sort of hybrid model (e.g., dag-of-snapshots plus
some extra annotations), if that worked better.
It's also very nice that users don't need any external software to
import CVS->monotone, just because it cuts down on hassle, but I would
rather have a more hasslesome tool that worked then a less hasslesome
tool that didn't, and I'm not the one volunteering to write the code,
so :-).
Even if we _do_ end up writing two implementations of the algorithm,
we should share a test suite. Testing cvs importers is way harder
than writing them, because there's no ground truth to compare your
program's output to... in fact, having two separate implementations
and testing them against each other would be useful to increase
confidence in each of them.
(I'm only on one of the CC'ed lists, so reply-to-all appreciated)
-- Nathaniel
--
"On arrival in my ward I was immediately served with lunch. `This is
what you ordered yesterday.' I pointed out that I had just arrived,
only to be told: `This is what your bed ordered.'"
-- Letter to the Editor, The Times, September 2000
^ permalink raw reply
* [PATCH] wt-status: remove extraneous newline from 'deleted:' output
From: Jeff King @ 2006-09-13 22:37 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
This was accidentally introduced during the fixes to avoid putting newlines
inside of colorized output.
Signed-off-by: Jeff King <peff@peff.net>
---
wt-status.c | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/wt-status.c b/wt-status.c
index ec2c728..c90aade 100644
--- a/wt-status.c
+++ b/wt-status.c
@@ -77,7 +77,7 @@ static void wt_status_print_filepair(int
p->one->path, p->two->path);
break;
case DIFF_STATUS_DELETED:
- color_printf_ln(c, "deleted: %s", p->one->path); break;
+ color_printf(c, "deleted: %s", p->one->path); break;
case DIFF_STATUS_MODIFIED:
color_printf(c, "modified: %s", p->one->path); break;
case DIFF_STATUS_RENAMED:
--
1.4.2.gd89c-dirty
^ permalink raw reply related
* Re: [RFC] Merge strategy 'applyreject'
From: Shawn Pearce @ 2006-09-13 22:26 UTC (permalink / raw)
To: Jakub Narebski; +Cc: git
In-Reply-To: <ee9ult$mtn$1@sea.gmane.org>
Jakub Narebski <jnareb@gmail.com> wrote:
> Shawn Pearce wrote:
>
> > But I don't really want this as a merge strategy in its own right.
> > I want it as part of merge-recur (...)
>
> Wouldn't it be better to have pluggable way to deal with conflicts,
> be it diff3/merge with conflict markers, .rej files, or invoking
> graphical merge tool (vimdiff, Emacs Emerge, xxdiff, Meld, KDiff3)?
Yes. :-)
Right now I think the user could just "plug in" their own "merge"
program earlier in PATH than the real one. :-)
It would be nice to get the speed of being able to just run the
xdiff code and the Git apply code directly on the file blobs in
memory, without forking, but I think pluggable file content merging
programs is a very good idea, for lots of different reasons.
--
Shawn.
^ permalink raw reply
* Re: [RFC] Merge strategy 'applyreject'
From: Jakub Narebski @ 2006-09-13 21:54 UTC (permalink / raw)
To: git
In-Reply-To: <20060913215043.GE30782@spearce.org>
Shawn Pearce wrote:
> But I don't really want this as a merge strategy in its own right.
> I want it as part of merge-recur (...)
Wouldn't it be better to have pluggable way to deal with conflicts,
be it diff3/merge with conflict markers, .rej files, or invoking
graphical merge tool (vimdiff, Emacs Emerge, xxdiff, Meld, KDiff3)?
--
Jakub Narebski
Warsaw, Poland
ShadeHawk on #git
^ permalink raw reply
* Re: [RFC] Merge strategy 'applyreject'
From: Petr Baudis @ 2006-09-13 21:50 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git, Shawn Pearce
In-Reply-To: <7v1wqf789j.fsf@assigned-by-dhcp.cox.net>
Dear diary, on Wed, Sep 13, 2006 at 11:43:04PM CEST, I got a letter
where Junio C Hamano <junkio@cox.net> said that...
> Petr Baudis <pasky@suse.cz> writes:
> > .rej files, what a nuisance to handle those... :)
>
> You were who asked for "apply --reject", weren't you?
Yes, but the point of that was obviously different. (It's a nuisance,
but only when compared to the conflict markers (yes, a matter of opinion
of course).)
Thanks for it, BTW!
--
Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
Snow falling on Perl. White noise covering line noise.
Hides all the bugs too. -- J. Putnam
^ permalink raw reply
* Re: [RFC] Merge strategy 'applyreject'
From: Shawn Pearce @ 2006-09-13 21:50 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Petr Baudis, git
In-Reply-To: <7v1wqf789j.fsf@assigned-by-dhcp.cox.net>
Junio C Hamano <junkio@cox.net> wrote:
> Petr Baudis <pasky@suse.cz> writes:
>
> > Dear diary, on Wed, Sep 13, 2006 at 11:08:17PM CEST, I got a letter
> > where Shawn Pearce <spearce@spearce.org> said that...
> >> Create merge strategy 'applyreject'.
> >>
> >> The applyreject merge strategy is a two head merge strategy which performs
> >> the merge by obtaining the diff between the common base and the branch
> >> being merged and applies it to the current branch using git-apply --reject.
> >> Consequently any failures are written to .rej files, rather than using
> >> the RCS <<<<<<< ======= >>>>>>> format.
> >
> > So, it's essentially the same as the classic resolve strategy, just
> > handling rejects differently? I think that should be more obvious from
> > its name, perhaps resolve-rej?
> >
> > .rej files, what a nuisance to handle those... :)
>
> You were who asked for "apply --reject", weren't you?
>
> I am not interested in this merge strategy myself. Having said
> that, if it is cleanly done, I do not have much objection adding
> it for other people's use, at least in principle.
I'll clean it up, document it and and resubmit the patch if others
want it as a top-level merge strategy.
But I don't really want this as a merge strategy in its own right.
I want it as part of merge-recur, so I can drop into my .git/config
file:
[merge-recursive]
conflictFormat = rejects
(for example) and not deal with the RCS merge program. This however
will require some hacking in merge-recursive.c and apply.c but I
think its workable.
--
Shawn.
^ permalink raw reply
* Re: [RFC] Merge strategy 'applyreject'
From: Shawn Pearce @ 2006-09-13 21:46 UTC (permalink / raw)
To: Petr Baudis; +Cc: git
In-Reply-To: <20060913211617.GJ23891@pasky.or.cz>
Petr Baudis <pasky@suse.cz> wrote:
> Dear diary, on Wed, Sep 13, 2006 at 11:08:17PM CEST, I got a letter
> where Shawn Pearce <spearce@spearce.org> said that...
> > Create merge strategy 'applyreject'.
> >
> > The applyreject merge strategy is a two head merge strategy which performs
> > the merge by obtaining the diff between the common base and the branch
> > being merged and applies it to the current branch using git-apply --reject.
> > Consequently any failures are written to .rej files, rather than using
> > the RCS <<<<<<< ======= >>>>>>> format.
>
> So, it's essentially the same as the classic resolve strategy, just
> handling rejects differently? I think that should be more obvious from
> its name, perhaps resolve-rej?
Unfortunately. More ideally it would be a configurable feature
of merge-recur that gets used rather than invoking the RCS merge.
This is just a quick and dirty 3 line shell script to propose
the concept.
> .rej files, what a nuisance to handle those... :)
A matter of opinion. :)
--
Shawn.
^ permalink raw reply
* Re: [RFC] Merge strategy 'applyreject'
From: Junio C Hamano @ 2006-09-13 21:43 UTC (permalink / raw)
To: Petr Baudis; +Cc: git, Shawn Pearce
In-Reply-To: <20060913211617.GJ23891@pasky.or.cz>
Petr Baudis <pasky@suse.cz> writes:
> Dear diary, on Wed, Sep 13, 2006 at 11:08:17PM CEST, I got a letter
> where Shawn Pearce <spearce@spearce.org> said that...
>> Create merge strategy 'applyreject'.
>>
>> The applyreject merge strategy is a two head merge strategy which performs
>> the merge by obtaining the diff between the common base and the branch
>> being merged and applies it to the current branch using git-apply --reject.
>> Consequently any failures are written to .rej files, rather than using
>> the RCS <<<<<<< ======= >>>>>>> format.
>
> So, it's essentially the same as the classic resolve strategy, just
> handling rejects differently? I think that should be more obvious from
> its name, perhaps resolve-rej?
>
> .rej files, what a nuisance to handle those... :)
You were who asked for "apply --reject", weren't you?
I am not interested in this merge strategy myself. Having said
that, if it is cleanly done, I do not have much objection adding
it for other people's use, at least in principle.
^ permalink raw reply
* Re: [RFC] Merge strategy 'applyreject'
From: Shawn Pearce @ 2006-09-13 21:38 UTC (permalink / raw)
To: Jakub Narebski; +Cc: git
In-Reply-To: <ee9spf$h98$1@sea.gmane.org>
Jakub Narebski <jnareb@gmail.com> wrote:
> > ---
> > .gitignore | 1 +
> > Makefile | 1 +
> > git-merge-applyreject.sh | 26 ++++++++++++++++++++++++++
> > git-merge.sh | 2 +-
> > 4 files changed, 29 insertions(+), 1 deletions(-)
>
> But where documentation (Documentation/merge-strategies.txt)?
If its interesting I'll document it. I just threw it together and
tossed it out there to see what others thought of the general idea.
> > +#!/bin/sh
> > +#
> > +# Simple merge strategy which produces reject files on failed merges.
> > +# Only handles two heads and one merge base, thus the command line
> > +# parameters must be: base -- head1 head2
>
> We can always get the base using git-merge-base, so the arguments could
> be either "base -- head1 head2", or just "head1 head2".
Actually the merge driver (git-merge.sh) feeds us the arguments like
that.
> Does "git pull -s applyreject . head2" works (when on head1) with
> this patch? Does the unified driver git-merge works correctly?
Yes. That's how I tested it.
> > +git-diff --binary -M $base $incoming \
> > + | git-apply --index --reject --verbose
>
> --index or --cached?
I believe that --index is correct. I want to patch the file in
the working directory, not the one that's currently in the index.
I also want to update the index if the patch applied cleanly.
--
Shawn.
^ permalink raw reply
* Re: cvs import
From: Jon Smirl @ 2006-09-13 21:38 UTC (permalink / raw)
To: Markus Schiltknecht
Cc: Martin Langhoff, Git Mailing List, monotone-devel, dev
In-Reply-To: <450872AE.5050409@bluegap.ch>
On 9/13/06, Markus Schiltknecht <markus@bluegap.ch> wrote:
> Martin Langhoff wrote:
> > On 9/14/06, Jon Smirl <jonsmirl@gmail.com> wrote:
> >> Let's copy the git list too and maybe we can come up with one importer
> >> for everyone.
> >
> > It's a really good idea. cvsps has been for a while a (limited, buggy)
> > attempt at that.
>
> BTW: good point, I always thought about cvsps. Does anybody know what
> 'dump' format that uses?
cvsps has potential but the multiple missing branch labels in the
Mozilla CVS confuse it and its throws away important data. It's
algorithm would need reworking too. cvs2svn is the only CVS converter
that imported Mozilla CVS on the first try and mostly got things
right.
Patchset format for cvsps
http://www.cobite.com/cvsps/README
AFAIK none of the CVS converters are using the dependency algorithm.
So the proposal on the table is to develop a new converter that uses
the dependency data from CVS to form the change sets and then outputs
this data in a form that all of the backends can consume. Of course
each of the backends is going to have to write some code in order to
consume this new import format.
>
> For sure it's algorithm isn't that strong. cvs2svn is better, IMHO. The
> proposed dependency resolving algorithm will be even better /me thinks.
>
> Regards
>
> Markus
>
--
Jon Smirl
jonsmirl@gmail.com
^ permalink raw reply
* Re: Marking abandoned branches
From: Jakub Narebski @ 2006-09-13 21:32 UTC (permalink / raw)
To: git
In-Reply-To: <7vhczb7ay9.fsf@assigned-by-dhcp.cox.net>
Junio C Hamano wrote:
> What this suggests is that the core support we
> will give needs a way to specify what subset of refs/* to
> include. peek-remote allows you to do this and it is fairly
> efficient for local case (although it could be made more
> efficient by not using the general git_connect() framework if we
> want a faster local-only command), but it gives back only the
> object names.
What is the difference between peek-remote and ls-remotes in the
local case, by the way?
--
Jakub Narebski
Warsaw, Poland
ShadeHawk on #git
^ permalink raw reply
* Re: [RFC] Merge strategy 'applyreject'
From: Jakub Narebski @ 2006-09-13 21:22 UTC (permalink / raw)
To: git
In-Reply-To: <20060913210817.GA30782@spearce.org>
Shawn Pearce wrote:
> -- >8 --
> Create merge strategy 'applyreject'.
>
> The applyreject merge strategy is a two head merge strategy which performs
> the merge by obtaining the diff between the common base and the branch
> being merged and applies it to the current branch using git-apply --reject.
> Consequently any failures are written to .rej files, rather than using
> the RCS <<<<<<< ======= >>>>>>> format.
Nice.
> ---
> .gitignore | 1 +
> Makefile | 1 +
> git-merge-applyreject.sh | 26 ++++++++++++++++++++++++++
> git-merge.sh | 2 +-
> 4 files changed, 29 insertions(+), 1 deletions(-)
But where documentation (Documentation/merge-strategies.txt)?
> +#!/bin/sh
> +#
> +# Simple merge strategy which produces reject files on failed merges.
> +# Only handles two heads and one merge base, thus the command line
> +# parameters must be: base -- head1 head2
We can always get the base using git-merge-base, so the arguments could
be either "base -- head1 head2", or just "head1 head2".
Does "git pull -s applyreject . head2" works (when on head1) with
this patch? Does the unified driver git-merge works correctly?
> +git-diff --binary -M $base $incoming \
> + | git-apply --index --reject --verbose
--index or --cached?
--
Jakub Narebski
Warsaw, Poland
ShadeHawk on #git
^ permalink raw reply
* Re: [RFC] Merge strategy 'applyreject'
From: Petr Baudis @ 2006-09-13 21:16 UTC (permalink / raw)
To: Shawn Pearce; +Cc: git
In-Reply-To: <20060913210817.GA30782@spearce.org>
Dear diary, on Wed, Sep 13, 2006 at 11:08:17PM CEST, I got a letter
where Shawn Pearce <spearce@spearce.org> said that...
> Create merge strategy 'applyreject'.
>
> The applyreject merge strategy is a two head merge strategy which performs
> the merge by obtaining the diff between the common base and the branch
> being merged and applies it to the current branch using git-apply --reject.
> Consequently any failures are written to .rej files, rather than using
> the RCS <<<<<<< ======= >>>>>>> format.
So, it's essentially the same as the classic resolve strategy, just
handling rejects differently? I think that should be more obvious from
its name, perhaps resolve-rej?
.rej files, what a nuisance to handle those... :)
--
Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
Snow falling on Perl. White noise covering line noise.
Hides all the bugs too. -- J. Putnam
^ permalink raw reply
* Re: cvs import
From: Martin Langhoff @ 2006-09-13 21:16 UTC (permalink / raw)
To: Markus Schiltknecht; +Cc: Jon Smirl, Git Mailing List, monotone-devel, dev
In-Reply-To: <4508724D.2050701@bluegap.ch>
On 9/14/06, Markus Schiltknecht <markus@bluegap.ch> wrote:
> Martin Langhoff wrote:
> > On 9/14/06, Jon Smirl <jonsmirl@gmail.com> wrote:
> >> Let's copy the git list too and maybe we can come up with one importer
> >> for everyone.
> >
> > It's a really good idea. cvsps has been for a while a (limited, buggy)
> > attempt at that. One thing that bothers me in the cvs2svn algorithm is
> > that is not stable in its decisions about where the branching point is
> > -- run the import twice at different times and it may tell you that
> > the branching point has moved.
>
> Huh? Really? Why is that? I don't see reasons for such a thing happening
> when studying the algorithm.
>
> For sure the proposed dependency-resolving algorithm which does not rely
> on timestamps does not have that problem.
IIRC, it places branch tags as late as possible. I haven't looked at
it in detail, but an import immediately after the first commit against
the branch may yield a different branchpoint from the same import done
a bit later.
cheers,
martin
^ permalink raw reply
* Re: cvs import
From: Oswald Buddenhagen @ 2006-09-13 21:15 UTC (permalink / raw)
To: Markus Schiltknecht
Cc: Martin Langhoff, Jon Smirl, Git Mailing List, monotone-devel, dev
In-Reply-To: <4508724D.2050701@bluegap.ch>
On Wed, Sep 13, 2006 at 11:04:13PM +0200, Markus Schiltknecht wrote:
> Martin Langhoff wrote:
> >One thing that bothers me in the cvs2svn algorithm is
> >that is not stable in its decisions about where the branching point is
> >-- run the import twice at different times and it may tell you that
> >the branching point has moved.
>
> Huh? Really? Why is that? I don't see reasons for such a thing happening
> when studying the algorithm.
>
that's certainly due to some hash being iterated. python intentionally
randomizes this to make wrong assumptions obvious.
there is actually a patch pending to improve the branch source selection
drastically. maybe this is affected as well.
> For sure the proposed dependency-resolving algorithm which does not rely
> on timestamps does not have that problem.
>
i think that's unrelated.
--
Hi! I'm a .signature virus! Copy me into your ~/.signature, please!
--
Chaos, panic, and disorder - my work here is done.
^ permalink raw reply
* Re: Marking abandoned branches
From: Jakub Narebski @ 2006-09-13 21:09 UTC (permalink / raw)
To: git
In-Reply-To: <7vhczb7ay9.fsf@assigned-by-dhcp.cox.net>
Junio C Hamano wrote:
> I'll try to code the one specified by the original strawman up
> sometime today; we can discuss enhancements after that.
I was thinking about implementing git-show-refs, "borrowing" format
parsing part from rpm (parseFormat function in rpmdb/header.c), but
what I had not found in core git was the function which parses object
into any_object like union; there is function which parses object,
but it extracts and returns only the common part as an object struct.
And any_object union is local to object.c
I also thought that if format doesn't require object parsing (sha1id,
type, name and size we can get without parsing) then do not do parsing,
but I wonder if this is really worth it.
--
Jakub Narebski
Warsaw, Poland
ShadeHawk on #git
^ permalink raw reply
* [RFC] Merge strategy 'applyreject'
From: Shawn Pearce @ 2006-09-13 21:08 UTC (permalink / raw)
To: git
Any thoughts on something like the following?
I was talking with Jakub Narebski on #git today about using a
diff/apply pipeline as a merge strategy.
I was talking about implementing this within the merge-recur driver
as an alternative to invoking the RCS 'merge' program. The idea
would be to take the current branch's file and try to apply the
diff of merge_base and the other branch to it. If that has any
rejects then try the other direction (apply diff of merge_base and
current to other) and leave the user with whichever one resulted
in the smallest number of rejected lines.
But that's a little bit more work, this is a quick hack. :-)
-- >8 --
Create merge strategy 'applyreject'.
The applyreject merge strategy is a two head merge strategy which performs
the merge by obtaining the diff between the common base and the branch
being merged and applies it to the current branch using git-apply --reject.
Consequently any failures are written to .rej files, rather than using
the RCS <<<<<<< ======= >>>>>>> format.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
---
.gitignore | 1 +
Makefile | 1 +
git-merge-applyreject.sh | 26 ++++++++++++++++++++++++++
git-merge.sh | 2 +-
4 files changed, 29 insertions(+), 1 deletions(-)
diff --git a/.gitignore b/.gitignore
index 0d608fe..0f43ece 100644
--- a/.gitignore
+++ b/.gitignore
@@ -56,6 +56,7 @@ git-ls-tree
git-mailinfo
git-mailsplit
git-merge
+git-merge-applyreject
git-merge-base
git-merge-index
git-merge-tree
diff --git a/Makefile b/Makefile
index 7b3114f..a57dab5 100644
--- a/Makefile
+++ b/Makefile
@@ -161,6 +161,7 @@ SCRIPT_SH = \
git-tag.sh git-verify-tag.sh \
git-applymbox.sh git-applypatch.sh git-am.sh \
git-merge.sh git-merge-stupid.sh git-merge-octopus.sh \
+ git-merge-applyreject.sh \
git-merge-resolve.sh git-merge-ours.sh \
git-lost-found.sh git-quiltimport.sh
diff --git a/git-merge-applyreject.sh b/git-merge-applyreject.sh
new file mode 100755
index 0000000..e4c8703
--- /dev/null
+++ b/git-merge-applyreject.sh
@@ -0,0 +1,26 @@
+#!/bin/sh
+#
+# Simple merge strategy which produces reject files on failed merges.
+# Only handles two heads and one merge base, thus the command line
+# parameters must be: base -- head1 head2
+
+base="$1"; shift; [ -z "$base" ] && exit 2
+while [ $# -gt 2 -a "X$1" != "X--" ]
+do
+ shift
+done
+if [ "X$1" = "X--" ]
+then
+ shift
+else
+ exit 2
+fi
+current="$1"; shift; [ -z "$current" ] && exit 2
+incoming="$1"; shift; [ -z "$incoming" ] && exit 2
+[ $# -gt 0 ] && exit 2
+
+git-read-tree --reset $current || exit 2
+git-update-index --refresh 2>/dev/null
+git-diff --binary -M $base $incoming \
+ | git-apply --index --reject --verbose
+[ $? = 0 ] || exit 1
diff --git a/git-merge.sh b/git-merge.sh
index d049e16..e39de0a 100755
--- a/git-merge.sh
+++ b/git-merge.sh
@@ -9,7 +9,7 @@ USAGE='[-n] [--no-commit] [--squash] [-s
LF='
'
-all_strategies='recursive recur octopus resolve stupid ours'
+all_strategies='recursive recur octopus resolve stupid ours applyreject'
case "${GIT_USE_RECUR_FOR_RECURSIVE}" in
'')
default_twohead_strategies=recursive ;;
--
1.4.2.gc52f
^ permalink raw reply related
* Re: cvs import
From: Markus Schiltknecht @ 2006-09-13 21:05 UTC (permalink / raw)
To: Martin Langhoff; +Cc: Jon Smirl, Git Mailing List, monotone-devel, dev
In-Reply-To: <46a038f90609131341se42b2dcne73c017cf757d13a@mail.gmail.com>
Martin Langhoff wrote:
> On 9/14/06, Jon Smirl <jonsmirl@gmail.com> wrote:
>> Let's copy the git list too and maybe we can come up with one importer
>> for everyone.
>
> It's a really good idea. cvsps has been for a while a (limited, buggy)
> attempt at that.
BTW: good point, I always thought about cvsps. Does anybody know what
'dump' format that uses?
For sure it's algorithm isn't that strong. cvs2svn is better, IMHO. The
proposed dependency resolving algorithm will be even better /me thinks.
Regards
Markus
^ permalink raw reply
* Re: cvs import
From: Markus Schiltknecht @ 2006-09-13 21:04 UTC (permalink / raw)
To: Martin Langhoff; +Cc: dev, monotone-devel, Jon Smirl, Git Mailing List
In-Reply-To: <46a038f90609131341se42b2dcne73c017cf757d13a@mail.gmail.com>
Martin Langhoff wrote:
> On 9/14/06, Jon Smirl <jonsmirl@gmail.com> wrote:
>> Let's copy the git list too and maybe we can come up with one importer
>> for everyone.
>
> It's a really good idea. cvsps has been for a while a (limited, buggy)
> attempt at that. One thing that bothers me in the cvs2svn algorithm is
> that is not stable in its decisions about where the branching point is
> -- run the import twice at different times and it may tell you that
> the branching point has moved.
Huh? Really? Why is that? I don't see reasons for such a thing happening
when studying the algorithm.
For sure the proposed dependency-resolving algorithm which does not rely
on timestamps does not have that problem.
Regards
Markus
^ permalink raw reply
* Re: Marking abandoned branches
From: Jakub Narebski @ 2006-09-13 21:02 UTC (permalink / raw)
To: git
In-Reply-To: <7vhczb7ay9.fsf@assigned-by-dhcp.cox.net>
Junio C Hamano wrote:
>> ... wouldn't refs cache (similar to current index for files) be
>> better idea?
>
> The ideal is to make a fast and easy way for Porcelains to
> access what they want to know about the refs without knowing
> their implementation. We already provide ways to do so except
> that they may not be fast nor easy. And the "may not be fast"
> part is what triggers 'cache would be better' reaction, but the
> right thing to do is not to work it around with a clutch, but to
> design what an appropriate core side support is and implement
> it.
The 'cache would be better' is because it is obviously backward
compatibile (which helps the porcelains and history viewers),
avoids some trouble with deleting (and renaming) branches, can
be fast (when used), and we can use symlinks/symrefs with it I guess.
--
Jakub Narebski
Warsaw, Poland
ShadeHawk on #git
^ permalink raw reply
* [PATCH] git-archive: inline default_parse_extra()
From: Rene Scharfe @ 2006-09-13 20:55 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Franck Bui-Huu, Git Mailing List
Signed-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx>
diff --git a/builtin-archive.c b/builtin-archive.c
index da3f714..6dabdee 100644
--- a/builtin-archive.c
+++ b/builtin-archive.c
@@ -145,17 +145,6 @@ void parse_treeish_arg(const char **argv
ar_args->time = archive_time;
}
-static const char *default_parse_extra(struct archiver *ar,
- const char **argv)
-{
- static char msg[64];
-
- snprintf(msg, sizeof(msg) - 4, "'%s' format does not handle %s",
- ar->name, *argv);
-
- return strcat(msg, "...");
-}
-
int parse_archive_args(int argc, const char **argv, struct archiver *ar)
{
const char *extra_argv[MAX_EXTRA_ARGS];
@@ -208,7 +197,8 @@ int parse_archive_args(int argc, const c
if (extra_argc) {
if (!ar->parse_extra)
- die("%s", default_parse_extra(ar, extra_argv));
+ die("'%s' format does not handle %s",
+ ar->name, extra_argv[0]);
ar->args.extra = ar->parse_extra(extra_argc, extra_argv);
}
ar->args.verbose = verbose;
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox