* Re: [PATCH] Prevent megablobs from gunking up git packs
From: Junio C Hamano @ 2007-05-23 0:28 UTC (permalink / raw)
To: Jakub Narebski; +Cc: Dana How, git
In-Reply-To: <200705230144.38290.jnareb@gmail.com>
Jakub Narebski <jnareb@gmail.com> writes:
> No, I was thinking about separate _kept_ pack (so it would be not
> repacked unless -f option is given) containing _only_ the large blobs.
> The only difference between this and your proposal is that megablobs
> would be in their mergablobs pack, but not loose.
I am not sure about the "unless -f option is given" part, but a
single .kept pack that contains only problematic blobs would be
an interesting experiment.
(0) prepare object names of problematic blobs, in huge.txt, one
object name per line;
(1) prepare a single pack that has them:
$ N=$(git-pack-object --depth=0 --window=0 pack <huge.txt)
$ echo 'Huge blobs -- do not repack' >pack-$N.keep
$ mv pack-$N.* .git/object/pack/.
(2) repack the remainder, with the default depth/window:
$ git repack -a -d
$ git prune
^ permalink raw reply
* Re: Git branching & pulling
From: Wink Saville @ 2007-05-23 0:22 UTC (permalink / raw)
To: Josef Weidendorfer; +Cc: Junio C Hamano, Steven Grimm, git
In-Reply-To: <200705202230.24362.Josef.Weidendorfer@gmx.de>
>
> Because git does not know what to merge; there is no config entry
> for "branch.test.merge", as the warning says. How would you reword
> this warning to make it easier to understand?
> But from your question, I assume that you expected git to have a
> "branch.test.merge" setting.
> What do you expect it to do? And why?
>
> Is my assumption correct that you want the last command to be equivalent
> to "git-pull . master"? And my further assumption, that you want this
> because you expect "git pull" to default to merging changes from its
> upstream (also when the upstream is local)?
Yes, as you said below that was the old behavior and besides when
the branch was created git was told what the upstream was it seems
reasonable that it remember that. When I clone a remote it does the
right thing it would seem when I make a branch it would behave
the same.
>
> Creation of a branch from another local one never has created
> "branch.x.remote" or "branch.x.merge" entries. I am not even sure
> that setting "branch.x.remote" to "." is working in the current version.
I tired to create the appropriate entries and it didn't work,
but maybe operator error.
> BTW: There was some old behavior of "git pull" to always pull the master
> branch from remote "origin" without any further parameters. I suppose that
> you did not want this to happen in your example above ?!
>
I expected it to pull from its upstream (i.e. the branches parent).
Wink
^ permalink raw reply
* Re: Commit ID in exported Tar Ball
From: René Scharfe @ 2007-05-22 23:44 UTC (permalink / raw)
To: Junio C Hamano
Cc: Shawn O. Pearce, git, Frank Lichtenheld, Johan Herland,
Thomas Glanzmann, Michael Gernoth, Linus Torvalds
In-Reply-To: <7vd50s79lg.fsf@assigned-by-dhcp.cox.net>
Junio C Hamano schrieb:
> René Scharfe <rene.scharfe@lsrfire.ath.cx> writes:
>
>> OK, so here's a first shot at the mentioned parser. It only understands
>> @@COMMITID@@ and @@@@, but it's easily extendible. The internals of
>> git-describe would need to be converted to library functions, preferably
>> offering every piece of version info separately (see thread "[PATCH]
>> Make sure an autogenerated version has at least four parts" for why).
>>
>> Before doing that, we should determine if this is the way to, though.
>>
>> René
>
> Hmmm. I am torn.
>
> It almost feels as if we'd better bite the bullet and do more
> insane things in ident substitution, instead of introducing this
> apparent syntax inconsistency between "$id$" and "@@COMMITID@@".
$Id$ (and $commit$) is reversible, @@COMMITID@@ is not. That means you
can create a synthetic file byte for byte with @@COMMITID@@ (and its not
yet implemented brethren), but you can't do that with $Id$ -- it's
impossible to get rid of the dollar signs.
I'm not attached to any particular syntax. It all started with
@@VERSION@@ from git.spec.in, which should not be implemented 1:1 anyway
(we'd need to be able to use arbitrary separators between version parts
to support different ways of ordering version numbers).
We could use $ to indicate reversible substitutions as before and @
(instead of @@) for one-way substitutions. I can't think of any other
use than in archives, though. It sure would be very confusing to have
such a conversion happen on checkout -- you'd need to use git-cat-file
to see the real file contents.
> That is, we could (I am not seriously proposing to do this, as I
> expect this will lead to a lot of insanity at the end):
>
> (1) introduce "const unsigned char commit_in_focus[20]",
> globally available to git suite, and clear it at the
> beginning of main();
Ugh. Requiring another global variable doesn't smell like good design.
By the way, we already have a similar, but very different syntax: the
one format_commit_message in commit.c. It's a one-way conversion, too.
Maybe we should copy the relevant pieces like %H and %h from there..
Do we want git-archive specific one-way conversions that are capable of
creating files like git.spec? Or is this just a shiny toy hypnotizing
me? 8-)
René
^ permalink raw reply
* Re: [PATCH] Prevent megablobs from gunking up git packs
From: Jakub Narebski @ 2007-05-22 23:44 UTC (permalink / raw)
To: Dana How; +Cc: git, Junio C Hamano
In-Reply-To: <56b7f5510705220959x1b37a4adk537cc0cba1a27530@mail.gmail.com>
Dana How wrote:
> On 5/22/07, Jakub Narebski <jnareb@gmail.com> wrote:
>> Dana How wrote:
>>> There's actually an even more extreme example from my day job.
>>> The software team has a project whose files/revisions would be
>>> similar to those in the linux kernel (larger commits, I'm sure).
>>> But they have *ONE* 500MB file they check in because it takes
>>> 2 or 3 days to generate and different people use different versions of it.
>>> I'm sure it has 50+ revisions now. If they converted to git and included
>>> these blobs in their packfile, that's a 25GB uncompressed increase!
>>> *Every* git operation must wade through 10X -- 100X more packfile.
>>> Or it could be kept in 50+ loose objects in objects/xx ,
>>> requiring a few extra syscalls by each user to get a new version.
>>
>> Or keeping those large objects in separate, _kept_ packfile, containing
>> only those objects (which can delta well, even if they are large).
>
> Yes, I experimented with various changes to git-repack and
> having it create .keep files just before coming up with the maxblobsize
> approach. The problem with a 12GB+ repo is not only the large
> repack time, but the fact that the repack time keeps growing with
> the repo size. So, with split packs, I had repack create .keep
> files for all new packs except the last (fragmentary) one. The next
> repack would then only repack new stuff plus the single fragmentary
> pack, keeping repack time from growing (until you deleted the .keep
> files [just the ones with "repack" in them] to start over from scratch).
> But this approach is not going to distribute commits and trees all that well.
No, I was thinking about separate _kept_ pack (so it would be not
repacked unless -f option is given) containing _only_ the large blobs.
The only difference between this and your proposal is that megablobs
would be in their mergablobs pack, but not loose.
--
Jakub Narebski
Poland
^ permalink raw reply
* Re: [PATCH] Add ability to specify environment extension to run_command
From: Alex Riesen @ 2007-05-22 23:14 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git, Sven Verdoolaege
In-Reply-To: <7v1wh88prw.fsf@assigned-by-dhcp.cox.net>
Junio C Hamano, Wed, May 23, 2007 00:19:47 +0200:
> > Others already discussed the issue. Just to be sure, I reimplemented
> > that comfortable putenv with unsetenv: if an environment entry ends
> > with a "=" it will be unset.
>
> Although combination of putenv and unsetenv gives a somewhat
> queasy feeling for obvious reasons, I'll let it pass. As we
> are coming up with an interface that uses only one string per
> environment element, that is probably a sensible thing to do,
> rather than trying to do the "historically correct" pairing of
> setenv/unsetenv.
>
> However, I do not think "VAR=" to unset it is a good interface.
> Having an environment variable whose value happens to be an
> empty string and not having the variable at all are two
> different things.
Right
> Because you _scan_ the whole string in your patch to see if it
> ends with = anyway, a trivial improvement would be to do:
>
> if (strchr(cmd->env, '='))
> putenv(cmd->env);
> else
> unsetenv(cmd->env);
I like this one. The env field in struct child_process and run_command
will have to mention it in comments (in run-command.h), it's kind of
special.
> If you do not mind such a special syntax (e.g. "VAR="), I would
> suggest doing that as a prefix (e.g. "!VAR") and do:
Nah, !VAR is a _working_ environment variable name.
int main(int argc, char *argv[], char *envp[])
{
const char *argv1[] = {"/usr/bin/perl", "-e", "print $ENV{'!VAR'}", NULL};
const char *envp1[] = {"!VAR=value", NULL};
execve(*argv, (char**)argv1, (char**)envp1);
return 0;
}
$ gcc ... && ./a.out
value
Someone could want it. We surely could use "=", though :)
^ permalink raw reply
* Re: Commit ID in exported Tar Ball
From: Junio C Hamano @ 2007-05-22 22:54 UTC (permalink / raw)
To: René Scharfe
Cc: Shawn O. Pearce, git, Frank Lichtenheld, Johan Herland,
Thomas Glanzmann, Michael Gernoth, Linus Torvalds
In-Reply-To: <46536E32.6000202@lsrfire.ath.cx>
René Scharfe <rene.scharfe@lsrfire.ath.cx> writes:
> OK, so here's a first shot at the mentioned parser. It only understands
> @@COMMITID@@ and @@@@, but it's easily extendible. The internals of
> git-describe would need to be converted to library functions, preferably
> offering every piece of version info separately (see thread "[PATCH]
> Make sure an autogenerated version has at least four parts" for why).
>
> Before doing that, we should determine if this is the way to, though.
>
> René
Hmmm. I am torn.
It almost feels as if we'd better bite the bullet and do more
insane things in ident substitution, instead of introducing this
apparent syntax inconsistency between "$id$" and "@@COMMITID@@".
That is, we could (I am not seriously proposing to do this, as I
expect this will lead to a lot of insanity at the end):
(1) introduce "const unsigned char commit_in_focus[20]",
globally available to git suite, and clear it at the
beginning of main();
(2) teach ident substitution to expand "$commit$" to
sprintf("$commit: %40s $", sha1_to_hex(commit_in_focus[])),
and unexpand "$commit: .* $".
(3) have git-archive set commit_in_focus[] before letting the
convert_to_working_tree do its work.
(4) later, we _might_ teach a single tree read-tree to also set
up commit_in_focus[], so that:
$ rm -f .git/index
$ git checkout -f HEAD
would expand "$commit$" in blobs.
This obviously have a lot of problems once we start adding the
commit_in_focus[] to more random programs. Even two-tree
read-tree case would behave in an unexpected way for an
uninitiated person, if you do something like:
$ git checkout master
$ git checkout next
I am CC'ing Linus because he would literally hate me suggesting
the above.
^ permalink raw reply
* Re: Commit ID in exported Tar Ball
From: René Scharfe @ 2007-05-22 22:26 UTC (permalink / raw)
To: Shawn O. Pearce
Cc: git, Junio C Hamano, Frank Lichtenheld, Johan Herland,
Thomas Glanzmann, Michael Gernoth
In-Reply-To: <4651F908.2000608@lsrfire.ath.cx>
[I'm quoting myself in full because I somehow sent my reply to everyone
but Shawn. A patch can be found at the end.]
René Scharfe schrieb:
> Shawn O. Pearce schrieb:
>> Ren?? Scharfe <rene.scharfe@lsrfire.ath.cx> wrote:
>>> Shawn O. Pearce schrieb:
>>>> git-describe is more human-friendly than a SHA-1...
>>> Yes, and the Makefile does even more than that: it adds a version
>>> file, a spec file and another version file for git-gui.
>>>
>>> The first two are probably useful for most projects that actually
>>> do versioned releases. We could have a simple parser that reads
>>> a template, replaces @@VERSION@@ with a git-describe output
>>> string and adds the result as a synthetic file to the archive.
>>> It's not exactly trivial -- e.g., how to specify git-describe
>>> options, template file and synthetic name, all in one command
>>> line parameter? -- but it's doable.
>> Maybe something just as simple as allowing the user to specify a
>> shell script in-tree that we unpack and run for them? That script
>> prints to stdout the content of the file to include.
>
> I doubt executing a shell script is simple. :-D You'd possibly get
> different results on different platforms (dare I mention Windows?).
>
> The template system I mentioned would be a kind of scripting language
> itself, but in this case we define its syntax and can guarantee
> consistency everywhere git runs. And since it would only have four
> types of tokens (@@VERSION@@, @@COMMITID@@, @@@@ and string literals)
> it could be fast and simple.
>
> We could implement it as a checkout converter, preferably one that is
> only applied by git-archive. Then we'd rename git.spec.in to
> git.spec, assign the "specfile" attribute to it and let git-archive
> replace the string @@VERSION@@ with git-describe's output.
> git-checkout would not expand the special strings, so you can simply
> edit and version the file as you can do with git.spec.in now.
> Michael would have a file containing only @@COMMITID@@ to solve his
> original problem. Make sense?
>
>> So now we're also really talking about, what should git-archive do
>> for a subproject? Sometimes you really do want to repackage and
>> redistribute the subproject as part of the superproject's tarball.
>> Sometimes you don't. I think in the case of git.git and
>> git-gui.git we want to include the subproject. ;-)
>
> Oh, yes, subprojects. git-archive currently exports them as empty
> directories. Using tar's append command you could simply build the
> project+subproject archive in the Makefile. That wouldn't work well
> with gitweb, though. Perhaps a --include-subproject=<path> option is
> needed?
OK, so here's a first shot at the mentioned parser. It only understands
@@COMMITID@@ and @@@@, but it's easily extendible. The internals of
git-describe would need to be converted to library functions, preferably
offering every piece of version info separately (see thread "[PATCH]
Make sure an autogenerated version has at least four parts" for why).
Before doing that, we should determine if this is the way to, though.
René
Documentation/gitattributes.txt | 19 ++++++-
archive-tar.c | 5 ++-
archive-zip.c | 5 ++-
cache.h | 1 +
convert.c | 112 +++++++++++++++++++++++++++++++++++++++
5 files changed, 139 insertions(+), 3 deletions(-)
diff --git a/Documentation/gitattributes.txt b/Documentation/gitattributes.txt
index d3ac9c7..84c414c 100644
--- a/Documentation/gitattributes.txt
+++ b/Documentation/gitattributes.txt
@@ -72,7 +72,7 @@ EFFECTS
-------
Certain operations by git can be influenced by assigning
-particular attributes to a path. Currently, three operations
+particular attributes to a path. Currently, four operations
are attributes-aware.
Checking-out and checking-in
@@ -374,6 +374,23 @@ frotz unspecified
----------------------------------------------------------------
+Creating an archive
+~~~~~~~~~~~~~~~~~~~
+
+
+`specfile`
+^^^^^^^^^^
+
+If the attribute `specfile` is set for a file then git will expand
+several placeholders when adding this file to an archive. The
+expansion depends on the availability of a commit ID, i.e. if
+`git-archive` has been given a tree instead of a commit or a tag
+then no replacement will be done.
+
+`@@COMMITID@@`:: is replaced by the commit hash.
+`@@@@`:: is replaced by `@@`.
+
+
GIT
---
Part of the gitlink:git[7] suite
diff --git a/archive-tar.c b/archive-tar.c
index 66fe3e3..eba24cb 100644
--- a/archive-tar.c
+++ b/archive-tar.c
@@ -17,6 +17,7 @@ static unsigned long offset;
static time_t archive_time;
static int tar_umask = 002;
static int verbose;
+static const unsigned char *commit_sha1;
/* writes out the whole block, but only if it is full */
static void write_if_needed(void)
@@ -285,7 +286,8 @@ static int write_tar_entry(const unsigned char *sha1,
buffer = NULL;
size = 0;
} else {
- buffer = convert_sha1_file(path.buf, sha1, mode, &type, &size);
+ buffer = sha1_file_to_archive(path.buf, sha1, mode, &type,
+ &size, commit_sha1);
if (!buffer)
die("cannot read %s", sha1_to_hex(sha1));
}
@@ -304,6 +306,7 @@ int write_tar_archive(struct archiver_args *args)
archive_time = args->time;
verbose = args->verbose;
+ commit_sha1 = args->commit_sha1;
if (args->commit_sha1)
write_global_extended_header(args->commit_sha1);
diff --git a/archive-zip.c b/archive-zip.c
index 444e162..93a5ab3 100644
--- a/archive-zip.c
+++ b/archive-zip.c
@@ -12,6 +12,7 @@
static int verbose;
static int zip_date;
static int zip_time;
+static const unsigned char *commit_sha1;
static unsigned char *zip_dir;
static unsigned int zip_dir_size;
@@ -195,7 +196,8 @@ static int write_zip_entry(const unsigned char *sha1,
if (S_ISREG(mode) && zlib_compression_level != 0)
method = 8;
result = 0;
- buffer = convert_sha1_file(path, sha1, mode, &type, &size);
+ buffer = sha1_file_to_archive(path, sha1, mode, &type, &size,
+ commit_sha1);
if (!buffer)
die("cannot read %s", sha1_to_hex(sha1));
crc = crc32(crc, buffer, size);
@@ -316,6 +318,7 @@ int write_zip_archive(struct archiver_args *args)
zip_dir = xmalloc(ZIP_DIRECTORY_MIN_SIZE);
zip_dir_size = ZIP_DIRECTORY_MIN_SIZE;
verbose = args->verbose;
+ commit_sha1 = args->commit_sha1;
if (args->base && plen > 0 && args->base[plen - 1] == '/') {
char *base = xstrdup(args->base);
diff --git a/cache.h b/cache.h
index cd875bc..0484904 100644
--- a/cache.h
+++ b/cache.h
@@ -550,6 +550,7 @@ extern void trace_argv_printf(const char **argv, int count, const char *format,
extern char *convert_to_git(const char *path, const char *src, unsigned long *sizep);
extern char *convert_to_working_tree(const char *path, const char *src, unsigned long *sizep);
extern void *convert_sha1_file(const char *path, const unsigned char *sha1, unsigned int mode, enum object_type *type, unsigned long *size);
+extern void *sha1_file_to_archive(const char *path, const unsigned char *sha1, unsigned int mode, enum object_type *type, unsigned long *size, const unsigned char *commit_sha1);
/* match-trees.c */
void shift_tree(const unsigned char *, const unsigned char *, unsigned char *, int);
diff --git a/convert.c b/convert.c
index 4b26b1a..1cdaec5 100644
--- a/convert.c
+++ b/convert.c
@@ -1,6 +1,7 @@
#include "cache.h"
#include "attr.h"
#include "run-command.h"
+#include "strbuf.h"
/*
* convert.c - convert a file when checking it out and checking it in.
@@ -667,3 +668,114 @@ void *convert_sha1_file(const char *path, const unsigned char *sha1,
}
return buffer;
}
+
+static void strbuf_append(struct strbuf *sb, const void *s, size_t len)
+{
+ if (sb->alloc < sb->len + len) {
+ sb->alloc = (sb->len + len) * 3 / 2 + 16;
+ sb->buf = xrealloc(sb->buf, sb->alloc);
+ }
+ memcpy(sb->buf + sb->len, s, len);
+ sb->len += len;
+}
+
+static unsigned int match_keyword(const char *data, unsigned int datalen,
+ const char *keyword)
+{
+ unsigned int keylen = strlen(keyword);
+ if (keylen > datalen)
+ return 0;
+ if (memcmp(data, keyword, keylen))
+ return 0;
+ return keylen;
+}
+
+static void *convert_to_archive(const char *path, const void *src,
+ unsigned long *sizep,
+ const unsigned char *commit_sha1)
+{
+ static struct git_attr *attr_specfile;
+ struct git_attr_check check[1];
+ const char *p = src;
+ unsigned long srcsize = *sizep;
+ int at_signs = 0;
+ struct strbuf dst;
+ unsigned int match;
+ int replaced_something = 0;
+
+ if (!commit_sha1)
+ return NULL;
+
+ if (!attr_specfile)
+ attr_specfile = git_attr("specfile", 8);
+
+ check[0].attr = attr_specfile;
+ if (git_checkattr(path, ARRAY_SIZE(check), check))
+ return NULL;
+ if (!ATTR_TRUE(check[0].value))
+ return NULL;
+
+ dst.alloc = srcsize + 128;
+ dst.buf = xmalloc(dst.alloc);
+ dst.len = dst.eof = 0;
+
+ while (srcsize > 0) {
+ if ((at_signs == 0 || at_signs == 1) && *p == '@') {
+ at_signs++;
+ p++;
+ srcsize--;
+ continue;
+ }
+ if (at_signs == 1) {
+ at_signs = 0;
+ strbuf_append(&dst, "@", 1);
+ }
+ if (at_signs == 0) {
+ strbuf_append(&dst, p, 1);
+ p++;
+ srcsize--;
+ continue;
+ }
+
+ if ((match = match_keyword(p, srcsize, "@@")))
+ strbuf_append(&dst, "@@", 2);
+ else if ((match = match_keyword(p, srcsize, "COMMITID@@")))
+ strbuf_append(&dst, sha1_to_hex(commit_sha1), 40);
+ else
+ strbuf_append(&dst, "@@", 2);
+ at_signs = 0;
+ p += match;
+ srcsize -= match;
+ replaced_something = 1;
+ }
+
+ if (!replaced_something) {
+ free(dst.buf);
+ return NULL;
+ }
+
+ *sizep = dst.len;
+ return dst.buf;
+}
+
+void *sha1_file_to_archive(const char *path, const unsigned char *sha1,
+ unsigned int mode, enum object_type *type,
+ unsigned long *size,
+ const unsigned char *commit_sha1)
+{
+ void *buffer = read_sha1_file(sha1, type, size);
+ if (S_ISREG(mode) && buffer) {
+ void *converted = convert_to_working_tree(path, buffer, size);
+ if (converted) {
+ free(buffer);
+ buffer = converted;
+ }
+
+ converted = convert_to_archive(path, buffer, size, commit_sha1);
+ if (converted) {
+ free(buffer);
+ buffer = converted;
+ }
+ }
+ return buffer;
+}
^ permalink raw reply related
* Re: [PATCH] revert/cherry-pick: allow the last parameter to be -h
From: Junio C Hamano @ 2007-05-22 22:23 UTC (permalink / raw)
To: Alex Riesen; +Cc: Jonas Fonseca, git
In-Reply-To: <20070522221156.GL30871@steel.home>
Alex Riesen <raa.lkml@gmail.com> writes:
> Jonas Fonseca, Tue, May 22, 2007 23:29:45 +0200:
>> + if (!strcmp(arg, "-h"))
>> + usage(usage_str);
>
> $ git rev-list --usage
> usage: git-rev-list [OPTION] <commit-id>... [ -- paths... ]
> limiting output:
> --max-count=nr
> --max-age=epoch
> ...
>
> Why should cherry-pick be different?
Good question. FYI
$ git rev-list --huh?
works equally well ;-)
^ permalink raw reply
* Re: [PATCH] Add ability to specify environment extension to run_command
From: Junio C Hamano @ 2007-05-22 22:19 UTC (permalink / raw)
To: Alex Riesen; +Cc: git, Sven Verdoolaege
In-Reply-To: <20070522214754.GD30871@steel.home>
Alex Riesen <raa.lkml@gmail.com> writes:
> Others already discussed the issue. Just to be sure, I reimplemented
> that comfortable putenv with unsetenv: if an environment entry ends
> with a "=" it will be unset.
Although combination of putenv and unsetenv gives a somewhat
queasy feeling for obvious reasons, I'll let it pass. As we
are coming up with an interface that uses only one string per
environment element, that is probably a sensible thing to do,
rather than trying to do the "historically correct" pairing of
setenv/unsetenv.
However, I do not think "VAR=" to unset it is a good interface.
Having an environment variable whose value happens to be an
empty string and not having the variable at all are two
different things.
Because you _scan_ the whole string in your patch to see if it
ends with = anyway, a trivial improvement would be to do:
if (strchr(cmd->env, '='))
putenv(cmd->env);
else
unsetenv(cmd->env);
If you do not mind such a special syntax (e.g. "VAR="), I would
suggest doing that as a prefix (e.g. "!VAR") and do:
if (cmd->env[0] != '!')
putenv(cmd->env);
else
unsetenv(cmd->env + 1);
The former look cleaner but less efficient; we are going to exec
so I do not think micro-optimization would matter at all, so my
suggestion would be to do the strchr().
^ permalink raw reply
* Re: [PATCH] revert/cherry-pick: allow the last parameter to be -h
From: Alex Riesen @ 2007-05-22 22:11 UTC (permalink / raw)
To: Jonas Fonseca; +Cc: git, Junio C Hamano
In-Reply-To: <20070522212945.GA8002@diku.dk>
Jonas Fonseca, Tue, May 22, 2007 23:29:45 +0200:
> + if (!strcmp(arg, "-h"))
> + usage(usage_str);
$ git rev-list --usage
usage: git-rev-list [OPTION] <commit-id>... [ -- paths... ]
limiting output:
--max-count=nr
--max-age=epoch
...
Why should cherry-pick be different?
^ permalink raw reply
* Re: [PATCH] Allow user to specify mailbox format for mailsplit
From: Alex Riesen @ 2007-05-22 22:04 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Fernando J. Pereda, Git Mailing List
In-Reply-To: <7vwsz1bw8y.fsf@assigned-by-dhcp.cox.net>
Junio C Hamano, Tue, May 22, 2007 01:19:25 +0200:
> Alex Riesen <raa.lkml@gmail.com> writes:
>
> > If the argument ends with a slash - assume it is a Maildir and try to
> > create it. Otherwise - it is an mbox.
>
> Sorry, I am lost here. git-mailsplit is to read an existing
> mbox and split the pieces of e-mail into the specified output
> directory. The recent Maildir support is to make it read pieces
> of e-mails from an existing Maildir (which presumably have bunch
> of files that store e-mail), isn't it?
Oh... Fernando, I'm very sorry. I never used mailsplit and didn't even
gave a second thought about its name. For reasons unknown, I assumed
the mbox/maildir argument is for the output.
Must be all the ozone depletion and the cosmic rays now causing
glitches in programmers brains...
> If the patch were about creating a new _output_ directory (that
> is, "dir" in cmd_mailsplit), I would understand what you are
> trying to do, but why would you even want to create the input
> Maildir in that loop (I take that your patch is on top of
> Fernando's patch)?
Yes, but don't apply it! :)
^ permalink raw reply
* Re: [RFC] Third round of support for cloning submodules
From: Alex Riesen @ 2007-05-22 21:56 UTC (permalink / raw)
To: Steven Grimm; +Cc: skimo, Junio C Hamano, git
In-Reply-To: <4650EA2E.7070204@midwinter.com>
Steven Grimm, Mon, May 21, 2007 02:39:10 +0200:
> Sven Verdoolaege wrote:
> >It's pretty easy to add a "submodule.*.skip" or "submodule.*.ignore".
> >Since the subcloning only happens at checkout, you could set these
> >before doing a checkout.
> >
>
> Can I take this to mean that you intend the default behavior to be to
> check out all subprojects, with individual ones suppressed via
> configuration as needed?
Neither fetch nor checkout none of them and leave the subproject
fetch+checkout as exercise to the user?
^ permalink raw reply
* Re: [RFC] Third round of support for cloning submodules
From: Alex Riesen @ 2007-05-22 21:54 UTC (permalink / raw)
To: Martin Waitz; +Cc: skimo, Junio C Hamano, git
In-Reply-To: <20070520231200.GR5412@admingilde.org>
Martin Waitz, Mon, May 21, 2007 01:12:01 +0200:
> On Mon, May 21, 2007 at 01:02:48AM +0200, Alex Riesen wrote:
> > > If the user did commit and then you do a supermodule checkout -m you
> > > will get a merge.
> >
> > Only if the user continue to use the last branch (or the detached
> > head) the subproject was on. He don't have to, he can even return to
> > the commit which does not conflict, unless he have to complicate
> > things.
>
> just curious:
> so you want to differenciate between a subproject HEAD which was
> set by the superproject and other ones?
No. Why do you think that I want to do that?
^ permalink raw reply
* Re: [PATCH] Add ability to specify environment extension to run_command
From: Alex Riesen @ 2007-05-22 21:51 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Martin Waitz, git, Sven Verdoolaege
In-Reply-To: <7v646l9xkn.fsf@assigned-by-dhcp.cox.net>
Junio C Hamano, Tue, May 22, 2007 08:33:44 +0200:
> struct child_process {
> ...
> const struct {
> const char *name;
> const char *value; /* NULL to unsetenv */
> } *env;
> ...
> };
I actually like how the environment is organized. And it is simple to
define in the source. And there are well-known routines for
environment-like array manipulation.
^ permalink raw reply
* [PATCH] Allow environment variables to be unset in the processes started by run_command
From: Alex Riesen @ 2007-05-22 21:49 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git, Sven Verdoolaege
In-Reply-To: <20070522214847.GF30871@steel.home>
Signed-off-by: Alex Riesen <raa.lkml@gmail.com>
---
run-command.c | 14 ++++++++++++--
1 files changed, 12 insertions(+), 2 deletions(-)
diff --git a/run-command.c b/run-command.c
index 605aa1e..3a5f737 100644
--- a/run-command.c
+++ b/run-command.c
@@ -77,8 +77,18 @@ int start_command(struct child_process *cmd)
die("exec %s: cd to %s failed (%s)", cmd->argv[0],
cmd->dir, strerror(errno));
if (cmd->env) {
- for (; *cmd->env; cmd->env++)
- putenv((char*)*cmd->env);
+ char *unsetbuf = NULL;
+ for (; *cmd->env; cmd->env++) {
+ size_t n = strlen(*cmd->env);
+ if (n && (*cmd->env)[n-1] == '=') {
+ unsetbuf = xrealloc(unsetbuf, n);
+ memcpy(unsetbuf, *cmd->env, --n);
+ unsetbuf[n] = '\0';
+ unsetenv(unsetbuf);
+ } else
+ putenv((char*)*cmd->env);
+ }
+ free(unsetbuf);
}
if (cmd->git_cmd) {
execv_git_cmd(cmd->argv);
--
1.5.2.51.g16099
^ permalink raw reply related
* [PATCH] Add ability to specify environment extension to run_command
From: Alex Riesen @ 2007-05-22 21:48 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git, Sven Verdoolaege
In-Reply-To: <20070522214823.GE30871@steel.home>
There is no way to specify and override for the environment:
there'd be no user for it yet.
Signed-off-by: Alex Riesen <raa.lkml@gmail.com>
---
run-command.c | 17 ++++++++++++++++-
run-command.h | 2 ++
2 files changed, 18 insertions(+), 1 deletions(-)
diff --git a/run-command.c b/run-command.c
index 043b570..605aa1e 100644
--- a/run-command.c
+++ b/run-command.c
@@ -76,6 +76,10 @@ int start_command(struct child_process *cmd)
if (cmd->dir && chdir(cmd->dir))
die("exec %s: cd to %s failed (%s)", cmd->argv[0],
cmd->dir, strerror(errno));
+ if (cmd->env) {
+ for (; *cmd->env; cmd->env++)
+ putenv((char*)*cmd->env);
+ }
if (cmd->git_cmd) {
execv_git_cmd(cmd->argv);
} else {
@@ -137,7 +141,8 @@ int run_command(struct child_process *cmd)
}
static void prepare_run_command_v_opt(struct child_process *cmd,
- const char **argv, int opt)
+ const char **argv,
+ int opt)
{
memset(cmd, 0, sizeof(*cmd));
cmd->argv = argv;
@@ -160,3 +165,13 @@ int run_command_v_opt_cd(const char **argv, int opt, const char *dir)
cmd.dir = dir;
return run_command(&cmd);
}
+
+int run_command_v_opt_cd_env(const char **argv, int opt, const char *dir, const char *const *env)
+{
+ struct child_process cmd;
+ prepare_run_command_v_opt(&cmd, argv, opt);
+ cmd.dir = dir;
+ cmd.env = env;
+ return run_command(&cmd);
+}
+
diff --git a/run-command.h b/run-command.h
index cbd7484..af1e0bf 100644
--- a/run-command.h
+++ b/run-command.h
@@ -17,6 +17,7 @@ struct child_process {
int in;
int out;
const char *dir;
+ const char *const *env;
unsigned close_in:1;
unsigned close_out:1;
unsigned no_stdin:1;
@@ -34,5 +35,6 @@ int run_command(struct child_process *);
#define RUN_COMMAND_STDOUT_TO_STDERR 4
int run_command_v_opt(const char **argv, int opt);
int run_command_v_opt_cd(const char **argv, int opt, const char *dir);
+int run_command_v_opt_cd_env(const char **argv, int opt, const char *dir, const char *const *env);
#endif
--
1.5.2.51.g16099
^ permalink raw reply related
* [PATCH] Add run_command_v_opt_cd: chdir into a directory before exec
From: Alex Riesen @ 2007-05-22 21:48 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git, Sven Verdoolaege
In-Reply-To: <20070522214754.GD30871@steel.home>
It can make code simplier (no need to preserve cwd) and safer
(no chance the cwd of the current process is accidentally forgotten).
Signed-off-by: Alex Riesen <raa.lkml@gmail.com>
---
run-command.c | 27 ++++++++++++++++++++++-----
run-command.h | 2 ++
2 files changed, 24 insertions(+), 5 deletions(-)
diff --git a/run-command.c b/run-command.c
index eff523e..043b570 100644
--- a/run-command.c
+++ b/run-command.c
@@ -73,6 +73,9 @@ int start_command(struct child_process *cmd)
close(cmd->out);
}
+ if (cmd->dir && chdir(cmd->dir))
+ die("exec %s: cd to %s failed (%s)", cmd->argv[0],
+ cmd->dir, strerror(errno));
if (cmd->git_cmd) {
execv_git_cmd(cmd->argv);
} else {
@@ -133,13 +136,27 @@ int run_command(struct child_process *cmd)
return finish_command(cmd);
}
+static void prepare_run_command_v_opt(struct child_process *cmd,
+ const char **argv, int opt)
+{
+ memset(cmd, 0, sizeof(*cmd));
+ cmd->argv = argv;
+ cmd->no_stdin = opt & RUN_COMMAND_NO_STDIN ? 1 : 0;
+ cmd->git_cmd = opt & RUN_GIT_CMD ? 1 : 0;
+ cmd->stdout_to_stderr = opt & RUN_COMMAND_STDOUT_TO_STDERR ? 1 : 0;
+}
+
int run_command_v_opt(const char **argv, int opt)
{
struct child_process cmd;
- memset(&cmd, 0, sizeof(cmd));
- cmd.argv = argv;
- cmd.no_stdin = opt & RUN_COMMAND_NO_STDIN ? 1 : 0;
- cmd.git_cmd = opt & RUN_GIT_CMD ? 1 : 0;
- cmd.stdout_to_stderr = opt & RUN_COMMAND_STDOUT_TO_STDERR ? 1 : 0;
+ prepare_run_command_v_opt(&cmd, argv, opt);
+ return run_command(&cmd);
+}
+
+int run_command_v_opt_cd(const char **argv, int opt, const char *dir)
+{
+ struct child_process cmd;
+ prepare_run_command_v_opt(&cmd, argv, opt);
+ cmd.dir = dir;
return run_command(&cmd);
}
diff --git a/run-command.h b/run-command.h
index 3680ef9..cbd7484 100644
--- a/run-command.h
+++ b/run-command.h
@@ -16,6 +16,7 @@ struct child_process {
pid_t pid;
int in;
int out;
+ const char *dir;
unsigned close_in:1;
unsigned close_out:1;
unsigned no_stdin:1;
@@ -32,5 +33,6 @@ int run_command(struct child_process *);
#define RUN_GIT_CMD 2 /*If this is to be git sub-command */
#define RUN_COMMAND_STDOUT_TO_STDERR 4
int run_command_v_opt(const char **argv, int opt);
+int run_command_v_opt_cd(const char **argv, int opt, const char *dir);
#endif
--
1.5.2.51.g16099
^ permalink raw reply related
* Re: [PATCH] Add ability to specify environment extension to run_command
From: Alex Riesen @ 2007-05-22 21:47 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git, Sven Verdoolaege
In-Reply-To: <7v7ir1dbl9.fsf@assigned-by-dhcp.cox.net>
Junio C Hamano, Tue, May 22, 2007 01:02:42 +0200:
> >
> > So here it is. On top of the previos patch regarding chdir before
> > exec. Junio, if needed, I can resend that first patch about chdir.
>
> Both of them in a row would be good, so yes, resend is
> appreciated.
Will be resent.
> > @@ -76,6 +76,10 @@ int start_command(struct child_process *cmd)
> > if (cmd->dir && chdir(cmd->dir))
> > die("exec %s: cd to %s failed (%s)", cmd->argv[0],
> > cmd->dir, strerror(errno));
> > + if (cmd->env) {
> > + for (; *cmd->env; cmd->env++)
> > + putenv((char*)*cmd->env);
> > + }
> > if (cmd->git_cmd) {
> > execv_git_cmd(cmd->argv);
> > } else {
>
> I had a feeling that some callers needed to be able to unsetenv
> some. How would this patch help them, or are they outside of
> the scope?
>
Others already discussed the issue. Just to be sure, I reimplemented
that comfortable putenv with unsetenv: if an environment entry ends
with a "=" it will be unset.
^ permalink raw reply
* [PATCH] revert/cherry-pick: allow the last parameter to be -h
From: Jonas Fonseca @ 2007-05-22 21:29 UTC (permalink / raw)
To: git; +Cc: Junio C Hamano
... to ask for the usage string.
Signed-off-by: Jonas Fonseca <fonseca@diku.dk>
---
builtin-revert.c | 2 ++
1 files changed, 2 insertions(+), 0 deletions(-)
diff --git a/builtin-revert.c b/builtin-revert.c
index ea2f15b..7984aeb 100644
--- a/builtin-revert.c
+++ b/builtin-revert.c
@@ -61,6 +61,8 @@ static void parse_options(int argc, const char **argv)
}
arg = argv[argc - 1];
+ if (!strcmp(arg, "-h"))
+ usage(usage_str);
if (get_sha1(arg, sha1))
die ("Cannot find '%s'", arg);
commit = (struct commit *)parse_object(sha1);
--
1.5.2.rc3.800.ga489e-dirty
--
Jonas Fonseca
^ permalink raw reply related
* Yet another Perforce importer
From: Alex Riesen @ 2007-05-22 21:13 UTC (permalink / raw)
To: git
In-Reply-To: <81b0412b0705100637g1072f5c7y54f781b67a569690@mail.gmail.com>
- primitive export support with --p4-edit-changed[=branch]
If no branch given the HEAD is assumed; this is useful when HEAD is
updated, but index and working directory are not. This is the case
after a remote repo does a push into a p4 mirroring repo. This has
its problems (only one commit will be described in p4 changelist),
but it has its uses too (default clone creates a push configuration
which modifies the master of the source repo), so it was left in.
The working directory of the p4 mirroring repo must be kept clean
(no changes not registered in the index. It's somewhat problematic
on cygwin - it keeps changing filemodes). Also, p4 cannot have
non-linear history (they wouldn't admit this, but they don't have
non-linear history, branching, merging and they are very slow), so
the branch to be exported is expected to fast-forward from the
mirroring branch (this probably can be changed, but it would be nice
to know in advance which files will be changed after the merge: to
checkout them from p4).
The workflow is like this:
$ git clone p4-mirror working
$ cd working; edit ...; git commit ...
$ git push origin +master:refs/remotes/working/master
$ cd ../p4-mirror
$ git-p4-import --p4-edit-changed=working/master
The state of working/master(123847298742...) is checked out
A p4 changelist 1234567 is prepared
$ p4 submit 1234567
Changelist 1234567 renamed 8375983 and submitted
$ git-p4-import -y -C 8375983
The --p4-edit-changed call will checkout the changed p4 files for
edit, addition and deletion, prepare submission by populating it
with commit descriptions, und update the files with the state from
the working branch. Addition of new files can be very problematic:
client view must be configured so that p4 knows where to places them
on server.
Submission left to the user: with that piece of sh%t the p4 is most
of us probably have to follow some special procedure for submission
anyway (I have to).
If someone else changes p4 repo and you import the changes, don't
forget to rebase your working branches: the current code blocks
export if there are changes between the working branch and mirroring
branch. This is the line where rev-list is called. Feel free to fix
that. I didn't because I just had enough of p4 stupidity for that day.
- Add --diffs to show changes between filesystem, index and HEAD
Useful after a "blind push" (the one which happens by default) on
the mirror branch: to see where was changed.
- Allow running the script without any description. Either a
description or --edit must be given: for mirroring branch commit.
A "-m" for message like in git commit should be a good idea,
perhaps: most descriptions of p4 submissions are useless and the
"spaghetti" of a p4 checkout seldom can be defined with one
changelist.
- Diverse small and large fixes, for instance a problem with creation
of empty commits which removed all files was fixed
It is still untested on Linux.
@rem = 'NT: CMD.EXE vim: syntax=perl noet sw=4
@perl -x -s %0 -- %*
@exit
@rem ';
#!perl -w
#line 7
local $VERBOSE = 0;
local $DRYRUN = 0;
local $SHOW_DIFFS = 0;
local $AUTO_COMMIT = 0;
local $JUST_COMMIT = 0;
local $P4CLIENT = undef;
local @EDIT_COMMIT = 0;
local @FULL_IMPORT = 0;
local @DESC = ();
local $SPEC = undef;
local @P4ARGS = ();
local $P4HAVE_FILE = undef;
local %P4USERS = ();
local $FULL_DESC = 1;
local $HEAD_FROM_P4 = 0;
local $P4_EDIT_CHANGED = 0;
local $P4_EDIT_HEAD = undef;
push(@P4ARGS, '-P', $ENV{P4PASSWD})
if defined($ENV{P4PASSWD}) and length($ENV{P4PASSWD});
use Cwd;
local $start_dir = cwd();
sub read_args {
my ($in_client, $in_cl, $in_fi, $in_p4, $no_opt) = (0,0,0,0,0);
foreach my $f ( @_ ) {
goto _files if $no_opt;
if ($in_client) { $in_client = 0; $P4CLIENT = $f; next }
if ($in_cl) { $in_cl=0; push(@DESC,"c$f"); next }
if ($in_fi) { $in_fi=0; push(@DESC,"f$f"); next }
if ($in_p4) { $in_p4=0; push(@DESC,"4$f"); next }
$no_opt=1, next if $f eq '--';
$DRYRUN=1, next if $f eq '-n' or $f eq '--dry-run';
$SHOW_DIFFS=$DRYRUN=1, next if $f eq '--diffs';
$AUTO_COMMIT=1, next if $f eq '-y' or $f eq '--yes';
$JUST_COMMIT=1, next if $f eq '--just-commit';
$EDIT_COMMIT=1, next if ($f eq '-e') or ($f eq '--edit');
$FULL_IMPORT=1, next if $f eq '--full';
$FULL_DESC++, next if $f eq '--p4-desc';
$P4_EDIT_CHANGED=1, next if $f eq '--p4-edit-changed';
if ($f =~ /^--p4-edit-changed=(.*)/) {
$P4_EDIT_CHANGED=1;
$P4_EDIT_HEAD=$1;
next
}
$VERBOSE++, next if $f eq '-v' or $f eq '--verbose';
$in_client = 1, next if $f eq '--client';
$P4CLIENT = $1, next if $f =~ /^--client=(.*)/;
$in_cl = 1, next if $f eq '-C';
push(@DESC,"c$1"), next if $f =~ /^--changelist=(.*)/;
$in_fi = 1, next if $f eq '-F';
push(@DESC,"f$1"), next if $f =~ /^--file=(.*)/;
$in_p4 = 1, next if ($f eq '--ptr') or ($f eq '--p4');
push(@DESC,"4$2"), next if $f =~ /^--(p4|ptr)=(.*)/;
if ($f eq '--help' or $f eq '-h') {
print <<EOF;
$0 [-n|--dry-run] [-y|--yes] [--client <client-name>] [--diffs] \
[-e|--edit] [--just-commit] [--full] [-v|--verbose] [-C <change-number>] \
[-F <filename>] [--ptr|--p4 <p4-path-and/or-revision>] [--p4-desc] \
[--] [<specification>]
Perforce client state importer. Creates a git commit on the current
branch from a state the given p4 client and working directory hold.
<specification> must be given and is expected to be a file which will be
stored on the side branch under the name "spec".
Remote-to-local mapping and the revisions of files are stored in "have",
and the client definition - in "client".
--client client Specify client name (saved in .git/p4/client for the next time)
--full Perform full import, don't even try to figure out what changed
-y|--yes Commit automatically (by default only index updated)
--just-commit To be used after you forgot to run with --yes first time
-n|--dry-run Do not update the index and do not commit
-e|--edit Edit commit description before committing
-v|--verbose Be more verbose. Can be given many times, increases verbosity
--file=file
-F file Take description for the commit from a file in the
next parameter
--changelist=change
-C change Take description for the commit from this p4 change
--p4|--ptr p4-path-and/or-revision
--p4|--ptr=p4-path-and/or-revision
Take description for the commit from the p4 change described
by this p4 path, possibly including revision specification
--p4-desc Increase amount of junk from p4 change description
--diffs Show files which are different between local filesystem, index,
and the current HEAD. Does not do anything else
--p4-edit-changed
--p4-edit-changed=sha1
Merge with the given sha1 and prepare a p4 submission.
Current HEAD must fast-forward to sha1. If sha1 is omited,
the current HEAD is assumed. The working directory must
have no local changes.
The descriptions taken from p4 changes given by -C and --p4 will
be concatenated if the options given multiple times.
"--" can be used to separate options from description files.
EOF
exit(0);
}
die "$0: unknown option $f\n" if $f =~ /^-/;
_files:
warn "$0: spec was already set, $SPEC ignored\n" if defined($SPEC);
$SPEC = $f;
}
}
read_args(@ARGV);
local ($GIT_DIR) = qx{git rev-parse --git-dir};
$GIT_DIR =~ s/\r?\n$//s if defined($GIT_DIR);
die "$0: git directory not found\n" if !defined($GIT_DIR) or !-d $GIT_DIR;
local $editor = $ENV{VISUAL};
$editor = $ENV{EDITOR} unless defined($editor);
$editor = 'd:/Programs/Vim/vim70/gvim.exe' unless defined($editor);
die "$0: no editor defined\n" unless defined($editor);
if ($SHOW_DIFFS) {
my $sep = $/;
$/="\0";
my ($show, $cnt) = (0, 0);
if (open(F, '-|', 'git diff-files -r --name-only -z')) {
while (<F>) {
my $c = chop;
$_ .= $c if $c ne "\0";
print "Changed files:\n" if !$show;
print " $_\n";
$show = 1;
$cnt++;
}
close(F);
}
if (open(F, '-|', 'git diff-index --cached -r -z HEAD')) {
$show = 0;
my ($diff, $info) = (0, 1);
while (<F>) {
my $c = chop;
$_ .= $c if $c ne "\0";
if ($info) {
next if !/^:(\d{6}) (\d{6}) ([0-9a-f]{40}) ([0-9a-f]{40}) ./o;
# show only content changes, p4 does not support exec-bit anyway
$diff = $3 ne $4;
} elsif ($diff) {
print "Changes between index and HEAD:\n" if !$show;
print " $_\n";
$show = 1;
$cnt++;
}
$info = !$info;
}
close(F);
}
$/ = $sep;
exit($cnt ? 1: 0);
}
# P4 client was given in command-line. Store it
if ( defined($P4CLIENT) ) {
mkdir "$GIT_DIR/p4", 0777;
if ( open(F, '>', "$GIT_DIR/p4/client") ) {
print F "$P4CLIENT\n";
close(F);
} else {
die "$0: cannot store client name: $!\n"
}
} else {
if ( open(F, '<', "$GIT_DIR/p4/client") ) {
($P4CLIENT) = <F>;
close(F);
$P4CLIENT =~ s/^\s*//,$P4CLIENT =~ s/\s*$// if defined($P4CLIENT);
}
}
die "P4 client not defined\n" if !defined($P4CLIENT) or !length($P4CLIENT);
print "reading P4 client $P4CLIENT\n" if $VERBOSE;
local ($P4ROOT, $p4clnt, $P4HOST);
open(my $fdo, '>', "$GIT_DIR/p4/client.def") or die "p4/client.def: $!\n";
binmode($fdo);
open(my $fdi, '-|', "p4 client -o $P4CLIENT") or die "p4 client: $!\n";
binmode($fdi);
my $last_line_len = 0;
while (<$fdi>) {
next if /^#/o;
if ( m/^\s*Root:\s*(\S+)[\\\/]*\s*$/so ) { $P4ROOT = $1 }
elsif ( m/^\s*Client:\s*(\S+)/o ) { $p4clnt = $1 }
elsif ( m/^\s*Host:\s*(\S+)/o ) { $P4HOST = $1 }
($VERBOSE and print), next if /^(Access|Update):/;
s/\r?\n$//so;
my $len = length($_);
print $fdo "$_\n" if $len or $len != $last_line_len;
$last_line_len = $len;
}
close($fdi);
close($fdo);
die "Client root not defined\n" unless defined($P4ROOT);
if ( $VERBOSE ) {
print "GIT_DIR: $GIT_DIR\n";
print "Root: $P4ROOT (cwd: $start_dir)\n";
print "Host: $P4HOST\n";
print "Client: $p4clnt\n" if $p4clnt ne $P4CLIENT;
}
if ($P4_EDIT_CHANGED) {
my $rc = system('git', 'diff-files', '--quiet');
exit(127) if $rc & 0xff; # error starting the program
die "$0: there are changes in $P4ROOT. Import them first.\n" if $rc;
$P4_EDIT_HEAD = 'HEAD' if !defined($P4_EDIT_HEAD);
my ($mergehead) = qx{git rev-parse $P4_EDIT_HEAD};
exit(127) if $?;
exit(1) if !defined($mergehead);
$mergehead =~ s/\r?\n//gs;
exit(1) if !length($mergehead);
print "Checking out $P4_EDIT_HEAD ($mergehead) for p4 edit\n" if $VERBOSE;
# Check if the give reference is a direct descendant of current branch
if ($P4_EDIT_HEAD ne 'HEAD') {
my ($sha1) = qx{git rev-list --max-count=1 $mergehead..HEAD};
exit(127) if $?;
die "$0: HEAD does not fast-forward to $P4_EDIT_HEAD\n"
if defined($sha1) and $sha1 =~ /^[0-9a-f]{40}\b/;
}
my $cnt;
my @files = ();
my $sep = $/;
$/="\0";
if (open(F, '-|', "git diff-index -R --cached -r -z $mergehead")) {
my ($diff, $info, $M) = (0, 1, '');
while (<F>) {
my $c = chop;
$_ .= $c if $c ne "\0";
if ($info) {
next if !/^:\d{6} \d{6} ([0-9a-f]{40}) ([0-9a-f]{40}) (\w+)/o;
# use only content changes, p4 does not support exec-bit
$diff = $1 ne $2;
$M = $3; # change type marker
} elsif ($diff) {
print "$M $_\n" if $VERBOSE;
die "File contains characters which p4 cannot support\n"
if /[\n@#%*]/s;
push @files, "$M$_";
$cnt++;
}
$info = !$info;
}
close(F);
}
$/ = $sep;
if (!$cnt) {
warn "$0: No content changes found between HEAD and $P4_EDIT_HEAD";
exit(0);
}
# Create a new changelist
my $p4;
open($p4, "p4 -c $P4CLIENT -H $P4HOST -d $P4ROOT change -o|") or
die "$0: failed to create changelist\n";
my @desc = map {s/\r?\n//so; $_} <$p4>;
close($p4);
open($p4, '>', "$GIT_DIR/p4/changelist") or
die "$GIT_DIR/p4/changelist: $!\n";
foreach (@desc) {
print $p4 "$_\n";
if (/^Description:/o) {
my $range = "..$mergehead";
$range = "${mergehead}^..$mergehead" if $P4_EDIT_HEAD eq 'HEAD';
if (open(my $fd, '-|', "git log $range")) {
while(<$fd>) {
# I believe it is not possible to save this information
# in Perforce.
next if /^(commit |Author:|Date:)/;
s/\r?\n$//so;
next if !length($_);
s/^\s+//o;
print $p4 " $_\n";
}
close($fd);
}
}
}
close($p4);
open(STDIN, '<', "$GIT_DIR/p4/changelist") or
die "$GIT_DIR/p4/changelist: $!\n";
open($p4, "p4 -c $P4CLIENT -H $P4HOST -d $P4ROOT change -i|") or
die "$0: failed to create changelist\n";
my ($newchange) = grep {s/^Change (\d+) created\b.*/$1/so} <$p4>;
close($p4);
print "Checking out P4 files in changelist $newchange\n" if $VERBOSE;
# open files for edit
$cnt = 0;
open($p4, '>', "$GIT_DIR/p4/files") or die "$GIT_DIR/p4/files: $!\n";
print $p4 "-c\n$newchange\n";
print $p4 (map {++$cnt; substr($_,1)."\n"} grep {/^M/} @files);
close($p4);
sub runp4 {
return system('p4','-c',$P4CLIENT,'-H',$P4HOST,'-d',$P4ROOT,@_);
}
runp4('-x',"$GIT_DIR/p4/files", 'edit') if $cnt;
$cnt = 0;
open($p4, '>', "$GIT_DIR/p4/files") or die "$GIT_DIR/p4/files: $!\n";
print $p4 "-c\n$newchange\n";
print $p4 (map {++$cnt; substr($_,1)."\n"} grep {/^A/} @files);
close($p4);
runp4('-x',"$GIT_DIR/p4/files", 'add') if $cnt;
$cnt = 0;
open($p4, '>', "$GIT_DIR/p4/files") or die "$GIT_DIR/p4/files: $!\n";
print $p4 "-c\n$newchange\n";
print $p4 (map {++$cnt; substr($_,1)."\n"} grep {/^D/} @files);
close($p4);
runp4('-x',"$GIT_DIR/p4/files", 'delete') if $cnt;
# p4 modifies working directory on checkout, stupid thing
system('git', 'update-index', '--refresh');
$rc = system('git', 'read-tree', '-m', '-u', $mergehead);
exit(127) if $rc & 0xff;
exit(1) if $rc;
print "The state of $P4_EDIT_HEAD($mergehead) is checked out.\n";
print "A p4 changelist $newchange is prepared.\n";
exit(0);
}
my ($git_head,$git_p4_head,$git_p4_have) = &git_p4_init;
if ($JUST_COMMIT) {
git_p4_commit($git_head, $git_p4_head);
exit 0;
}
local %gitignore_dirs = ();
$gitignore_dirs{'/'} = read_filter_file("$GIT_DIR/info/exclude");
push(@{$gitignore_dirs{'/'}}, @{read_filter_file('.gitignore')});
my %git_index = ();
$/ = "\0";
my @git_X = ();
print "Reading git file list(git ls-files @git_X --cached -z)...\n" if $VERBOSE;
foreach ( qx{git ls-files @git_X --cached -z} ) {
chop; # chop \0
next if m/^\.gitignore$/o;
next if m/\/\.gitignore$/o;
next if filtered($_);
$git_index{$_} = 1;
}
my @git_add = ();
my @git_addx = ();
my @git_del = ();
my @git_upd = ();
print "Reading P4 file list...\n" if $VERBOSE;
local ($Conflicts,$Ignored,$Added,$Deleted,$Updated) = (0,0,0,0,0);
$/ = "\n";
my $in_name = 0;
my @root = split(/[\/\\]+/, $P4ROOT);
my %p4_index = ();
my %p4_a_lc = ();
my %lnames = ();
my %lconflicts = ();
if (opendir(DIR, '.')) {
$lnames{'.'} = [grep {$_ ne '.' and $_ ne '..'} readdir(DIR)];
closedir(DIR);
#print "read $start_dir (",scalar(@{$lnames{'.'}}),")\n";
}
open(my $have, "p4 -G @P4ARGS -c $P4CLIENT -H $P4HOST -d $P4ROOT have |") or
die "$0: failed to start p4: $!\n";
binmode($have);
$P4HAVE_FILE = "$GIT_DIR/p4/have";
open(my $storedhave, '>', $P4HAVE_FILE) or die "$P4HAVE_FILE: $!\n";
binmode($storedhave);
my $ent;
while (defined($ent=read_pydict_entry($have))) {
next if !defined($ent->{depotFile}) or !defined($ent->{clientFile});
my $a = $ent->{depotFile};
$ent->{clientFile} =~ m!^//[^/]+/(.*)!o;
my $b = $1;
my @bb = split(/\/+/, $b);
print $storedhave "$a\0$ent->{clientFile}\0$ent->{haveRev}\0\n";
if ( $^O eq 'MSWin32' ) {
# stupid windows, daft activestate, dumb P4
# This piece below is checking for file name conflicts
# which happen on windows because of it mangling the names.
my $blc = lc $b;
if ( $#bb > 0 ) {
my $path = '.';
foreach my $n (@bb[0 .. $#bb -1]) {
my @conflicts =
grep {lc $_ eq lc $n and $_ ne $n} @{$lnames{$path}};
if (@conflicts and !exists($lconflicts{"$path/$n"})) {
warn "warning: $a -> $b\n".
"warning: conflict between path \"$path/$n\" and ".
"local filesystem in \"@conflicts\"\n";
$Conflicts++;
$lconflicts{"$path/$n"} = 1;
}
$path .= "/$n";
if (!exists($lnames{$path})) {
if (opendir(DIR, $path)) {
$lnames{$path} =
[grep {$_ ne '.' and $_ ne '..'} readdir(DIR)];
closedir(DIR);
#print "read $path (",scalar(@{$lnames{$path}}),")\n";
}
}
}
}
if (!exists($p4_a_lc{$blc})) {
$p4_a_lc{$blc} = [$a, $b];
} else {
warn("warning: $a -> $b\n".
"warning: conflicts with ".
$p4_a_lc{$blc}->[0]." -> ".
$p4_a_lc{$blc}->[1]."\n");
$Conflicts++;
next;
}
}
my $i;
for ($i = 0; $i < $#bb; ++$i) {
my $bdir = join('/',@bb[0 .. $i]) . '/';
if ( !exists($gitignore_dirs{$bdir}) ) {
$gitignore_dirs{$bdir} = read_filter_file("$bdir.gitignore");
}
}
if (filtered($b)) {
print " i $b\n" if $VERBOSE > 3;
$Ignored++;
next
}
$p4_index{$b} = $a;
if ( exists($git_index{$b}) ) {
my $needup = 1;
if (defined($git_p4_have)) {
$prev = $git_p4_have->{$a};
if (defined($prev)) {
$prev->[0] =~ m!^//[^/]+/(.*)!o;
$needup = 0 if ($b eq $1) and ($prev->[1] eq $ent->{haveRev});
if ($needup and $VERBOSE > 1) {
my $reason;
$reason = 'local file' if $b ne $1;
$reason = 'revision' if $prev->[1] ne $ent->{haveRev};
print "$a ($reason changed)\n";
}
}
}
if ($needup) {
$Updated++;
push(@git_upd, $b);
}
} else {
$Added++;
if ( $b =~ m/\.(bat|cmd|pl|sh|exe|dll)$/io )
{ push(@git_addx, $b) } else { push(@git_add, $b) }
}
}
close($storedhave);
close($have);
undef %p4_a_lc;
@git_del = grep { !exists($p4_index{$_}) } keys %git_index;
$Deleted = $#git_del + 1;
#foreach (keys %git_index)
#{ push(@git_del, $_) if !exists($p4_index{$_}) }
if ( $DRYRUN ) {
print($#git_add+$#git_addx+ 2," files to add\n") if $VERBOSE;
print map {" a $_\n"} @git_add if $VERBOSE > 2;
print map {" a $_\n"} @git_addx if $VERBOSE > 2;
print($#git_del+1," files to unreg\n") if $VERBOSE;
print map {" d $_\n"} @git_del if $VERBOSE > 2;
print($#git_upd+1," files to update\n") if $VERBOSE;
print map {" u $_\n"} @git_upd if $VERBOSE > 2;
print "added: $Added, unregd: $Deleted, updated: $Updated, ignored: $Ignored";
print ", conflicts: $Conflicts" if $Conflicts;
print "\n";
} else {
if (@git_add || @git_addx) {
print($#git_add+$#git_addx+ 2,
" files | git update-index --add -z --stdin\n")
if $VERBOSE;
if (@git_add) {
open(GIT, '| git update-index --add --chmod=-x -z --stdin') or
die "$0 git-update-index(add): $!\n";
print GIT map {print " a $_\n" if $VERBOSE > 1; "$_\0"} @git_add;
close(GIT);
}
if (@git_addx) {
open(GIT, '| git update-index --add --chmod=+x -z --stdin') or
die "$0 git-update-index(add): $!\n";
print GIT map {print " a $_\n" if $VERBOSE > 1; "$_\0"} @git_addx;
close(GIT);
}
}
if (@git_del) {
print($#git_del+1," files | git update-index --remove -z --stdin\n")
if $VERBOSE;
open(GIT, '| git update-index --force-remove -z --stdin') or
die "$0 git-update-index(del): $!\n";
print GIT map {print " d $_\n" if $VERBOSE > 1; "$_\0"} @git_del;
close(GIT);
}
if (@git_upd) {
print($#git_upd+1," files | git update-index -z --stdin\n")
if $VERBOSE;
open(GIT, '| git update-index -z --stdin') or
die "$0 git-update-index(upd): $!\n";
print GIT map {print " u $_\n" if $VERBOSE > 1; "$_\0"} @git_upd;
close(GIT);
}
print "added: $Added, unregd: $Deleted, updated: $Updated, ignored: $Ignored";
print ", conflicts: $Conflicts" if $Conflicts;
print "\n";
git_p4_commit($git_head, $git_p4_head) if $AUTO_COMMIT;
}
exit 0;
sub filtered {
my $name = shift;
study($name);
my @path = split(/\/+/o, $name);
my $dir = '';
$name = '';
foreach my $d (@path) {
$name .= $d;
# print STDERR "$dir: $name $d\n" if $v;
foreach my $re (@{$gitignore_dirs{'/'}}) {
return 1 if $name =~ m/$re/;
return 1 if $d =~ m/$re/;
}
if ( length($dir) and exists($gitignore_dirs{$dir}) ) {
foreach my $re (@{$gitignore_dirs{$dir}}) {
return 1 if $name =~ m/$re/;
return 1 if $d =~ m/$re/;
}
}
$name .= '/';
$dir = $name;
}
# print STDERR "$name not filtered\n" if $v;
return 0;
}
sub read_filter_file {
my @filts = ();
my $file = shift;
if ( open(my $if, '<', $file) ) {
print "added ignore file $file\n" if $VERBOSE;
$/ = "\n";
while (my $l = <$if>) {
next if $l =~ /^\s*#/o;
next if $l =~ /^\s*$/o;
$l =~ s/[\r\n]+$//so;
$l =~ s/\./\\./go;
$l =~ s/\*/.*/go;
if ( $l =~ m/\// ) {
$l = "^$l($|/)";
} else {
$l = "(^|/)$l\$";
}
print " filter $l\n" if $VERBOSE > 1;
push(@filts, qr/$l/);
}
close($if);
}
return \@filts;
}
sub r_pystr
{
my $fd = shift;
my ($len,$str)=('','');
my ($c,$rd,$b) = (4,0,'');
while ($c > 0) {
$rd = sysread($fd,$b,$c);
warn("failed to read len: $!"), return undef if !defined($rd);
warn("not enough data for len"), return undef if !$rd;
$len .= $b;
$c -= $rd;
}
$len = unpack('V',$len);
while ($len > 0) {
$rd = sysread($fd,$b,$len);
warn("failed to read data: $!"), return undef if !defined($rd);
warn("not enough data"), return undef if !$rd;
$str .= $b;
$len -= $rd;
}
return $str;
}
sub read_pydict_entry
{
my $f = shift;
my ($buf,$rd);
FIL: while (1) {
# object type identifier
$rd = sysread($f, $buf, 1);
last FIL if $rd == 0;
warn("object type: $!\n"),last if $rd != 1;
# '{' is a python marshalled dict
warn("object type: not {\n"),last if $buf ne '{';
my $ent = {};
PAIR: while (1) {
my ($b,$key);
# key type identifier
$rd = sysread($f, $b, 1);
warn("key type: $!\n"),last FIL if $rd != 1;
if ($b eq 's') { # length-prefixed string
$key = r_pystr($f);
warn("key: $!\n"),last FIL if !defined($b);
} elsif ($b eq '0') { # NULL-element, end of entry
last PAIR;
} else {
warn("key type: not s");
last FIL;
}
# value type identifier
$rd = sysread($f, $b, 1);
warn("$key value type: $!\n"),last FIL if $rd != 1;
if ($b eq 's') { # length-prefixed string
$b = r_pystr($f);
warn("$key value: $!"),last FIL if !defined($b);
$ent->{$key} = $b;
} else {
warn("$key value type: not s ($b)");
last FIL;
}
}
return $ent;
}
return undef;
}
sub cl2msg {
my $cl = shift;
my($o1,$o2,$i);
if(!open($o1, '>>', "$GIT_DIR/p4/msg")) {
warn "p4/msg: $!\n";
return;
}
binmode($o1);
if(!open($o2, '>>', "$GIT_DIR/p4/p4msg")) {
warn "p4/p4msg: $!\n";
close($o1);
return
}
binmode($o2);
if(!open($i, '-|', "p4 describe -s $cl")){
warn "p4 describe: $!\n";
close($o1);
close($o2);
return
}
binmode($i);
print $o1 "$cl: " if $FULL_DESC;
print $o2 "$cl: ";
my @a;
my $u = undef;
while (my $l = <$i>) {
if ($l =~ /^Change \d+ by (\S+)@[^ ]* on ([^\r\n]*)/so) {
$u = $1;
$ENV{GIT_AUTHOR_DATE} = $2 if length($2);
}
last if $FULL_DESC < 2 and $l =~ /^\s*Affected files \.{3}\s*$/so;
$l =~ s/\r?\n$//so;
push @a, $l;
}
close($i);
print $o2 substr($a[2],1),"\n"; # p4 side-branch commit description
close($o2);
# import branch commit description
if ($FULL_DESC > 1) {
# desc level 2+: keep the Change line
print $o1 map {"$_\n"} (substr($a[2],1),"\n",@a);
} else {
# levels 0 and 1: remove the Change line
print $o1 map { (length($_) ? substr($_,1):'')."\n" } @a[2..$#a];
}
close($o1);
if (defined($u)) {
if (!exists($P4USERS{$u})) {
my ($mail,$name) = grep {/^(Email|FullName):/} qx{p4 user -o $u};
if ($? == 0 and defined($mail) and defined($name)) {
s/^\S+: ([^\r\n]*)\r?\n$/$1/so for ($mail,$name);
if (length($name) and length($mail)) {
$P4USERS{$u} = {name=>$name, email=>$mail};
}
}
}
if ($P4USERS{$u}) {
$p4u = $P4USERS{$u};
$ENV{GIT_AUTHOR_NAME} = $p4u->{name};
$ENV{GIT_AUTHOR_EMAIL} = $p4u->{email};
}
}
}
sub git_p4_init {
my ($commit,$parent,$p4commit,$p4parent);
my ($HEAD) = qx{git rev-parse HEAD};
$HEAD = '' if $?;
my ($p4head) = qx{git rev-parse refs/p4import/$P4CLIENT};
$p4head = '' if $?;
s/\r?\n//gs for ($HEAD, $p4head);
die "No HEAD commit! Refusing to import.\n" if !length($HEAD);
if (length($p4head)) {
($commit,$p4parent) =
grep { s/^parent (.{40}).*/$1/s }
qx{git cat-file commit $p4head};
$commit = $p4parent = '' if $?;
$p4parent = '' if !defined($p4parent);
} else {
$commit = $p4parent = '';
}
while (($commit ne $HEAD) and length($p4parent)) {
$p4head = $p4parent;
($commit,$p4parent) =
grep { s/^parent (.{40}).*/$1/s }
qx{git cat-file commit $p4head};
$commit = $p4parent = '' if $?;
$p4parent = '' if !defined($p4parent);
if ($VERBOSE and ($HEAD eq $commit)) {
print "found p4 import commit ";
system('git','name-rev',$p4head);
}
}
if ($HEAD ne $commit) {
$HEAD_FROM_P4 = 0;
warn "Current HEAD is not from $P4CLIENT, doing full import\n";
} else {
$HEAD_FROM_P4 = 1;
}
my $p4have = undef;
if (!$FULL_IMPORT and ($HEAD eq $commit) and length($p4head)) {
if (open(my $f, '-|', "git cat-file blob $p4head:have")) {
my $old = $/;
$/ = "\0";
my $cnt = 0;
while(1) {
my $p4name = <$f>;
last if !defined($p4name);
$p4name =~ s/^.//so if $cnt; # remove \n
my $name = <$f>;
my $rev = <$f>;
last if !defined($name) or !defined($rev);
chop($p4name,$name,$rev);
++$cnt;
if (defined($p4have)) {
$p4have->{$p4name} = [$name,$rev];
} else {
$p4have = {$p4name=>[$name,$rev]};
}
}
$/ = $old;
close($f);
print "loaded $cnt revisions from $p4head\n" if $VERBOSE;
}
}
return ($HEAD, $p4head, $p4have);
}
sub git_p4_commit {
my ($HEAD, $p4head) = @_;
my ($commit,$parent,$p4commit,$p4parent);
my ($fdo,$fdi,$rc);
$rc = system('git','diff-index','--exit-code','--quiet','--cached','HEAD');
if ($rc == 0) {
warn("No changes\n");
return;
}
return if $DRYRUN;
if (!@DESC && !$EDIT_COMMIT) {
warn "$0: no commit description given\n";
return;
}
my $p4x = "$GIT_DIR/p4/idx.tmp";
unlink($p4x);
$ENV{PAGER} = 'cat';
if (!defined($SPEC) or !open(STDIN, '<', $SPEC)) {
if ( $^O eq 'MSWin32' ) {
open(STDIN, '<', 'NUL') or die "$SPEC: $!\n";
} else {
open(STDIN, '<', '/dev/null') or die "$SPEC: $!\n";
}
}
my ($p4spec) = qx{git hash-object -t blob -w --stdin};
die "Failed to store $SPEC in git repo\n" if $?;
open(STDIN, '<', "$GIT_DIR/p4/client.def") or die "cldef: $!\n";
my ($p4clnt) = qx{git hash-object -t blob -w --stdin};
die "Failed to save mappings of $P4CLIENT in git repo" if $?;
if (!defined($P4HAVE_FILE)) {
print "reading state of $P4CLIENT\n" if $VERBOSE;
$P4HAVE_FILE = "$GIT_DIR/p4/have";
open($fdo, '>', $P4HAVE_FILE) or die "p4/have: $!\n";
binmode($fdo);
open($fdi, "p4 -G @P4ARGS -c $P4CLIENT -H $P4HOST -d $P4ROOT have|") or
die "p4 have: $!\n";
binmode($fdi);
my $ent;
while (defined($ent=read_pydict_entry($fdi))) {
next if !defined($ent->{depotFile});
next if !defined($ent->{clientFile});
print $fdo "$ent->{depotFile}\0",
"$ent->{clientFile}\0",
"$ent->{haveRev}\0\n";
}
close($fdi);
close($fdo);
}
open(STDIN, '<', $P4HAVE_FILE) or die "$P4HAVE_FILE: $!\n";
my ($p4have) = qx{git hash-object -t blob -w --stdin};
die "Failed to save state of $P4CLIENT in git repo" if $?;
#
# Prepare commit messages
#
unlink("$GIT_DIR/p4/msg", "$GIT_DIR/p4/p4msg");
open($fdo, '>', "$GIT_DIR/p4/msg"); close($fdo);
open($fdo, '>', "$GIT_DIR/p4/p4msg"); close($fdo);
foreach my $i (@DESC) {
$i =~ s/^(.)//o;
if ('c' eq $1) {
print "reading changes for $i\n" if $VERBOSE;
cl2msg($i);
} elsif ('f' eq $1) {
my($o1,$o2,$i);
if (open($o1, '>>', "$GIT_DIR/p4/msg")) {
if (open($o2, '>>', "$GIT_DIR/p4/p4msg")) {
if (open($i, '<', $i)) {
my $n = 0;
while(<$i>) {
$n++;
print $o1 $_;
print $o2 $_ if $n == 1;
}
close($i);
}
close($o2);
}
close($o1);
}
} elsif ('4' eq $1) {
print "reading changes for $i\n" if $VERBOSE;
my ($change)=qx{p4 changes -m1 $i};
if (!defined($change) or $change !~ m/\s+(\d+)\s/) {
die "$i does not resolve into a change number\n";
}
cl2msg($1);
}
}
system("$editor $GIT_DIR/p4/msg") if $EDIT_COMMIT;
# copy mirror-branch commit message into side-branch
# commit message if no other description were given.
if (!-s "$GIT_DIR/p4/p4msg") {
open($fdi, '<', "$GIT_DIR/p4/msg") or die "$GIT_DIR/p4/msg: $!\n";
sysread($fdi,$buf,-s "$GIT_DIR/p4/msg");
close($fdi);
open($fdo, '>>', "$GIT_DIR/p4/p4msg") or die "$GIT_DIR/p4/p4msg: $!\n";
syswrite($fdo,$buf);
close($fdo);
}
#
# Store the imported file data
#
if ($VERBOSE < 2) {
if ( $^O eq 'MSWin32' ) { open(STDERR, "NUL") }
else { open(STDERR, "/dev/null") }
}
my ($tree) = qx{git write-tree};
die "Failed to write current tree\n" if $?;
$parent = length($HEAD) ? "-p $HEAD": '';
open(STDIN, '<', "$GIT_DIR/p4/msg") or die "p4/msg: $!\n";
$tree =~ s/\r?\n//gs;
($commit)=qx{git commit-tree $tree $parent};
die "failed to commit current tree\n" if $?;
s/\r?\n//gs for ($commit);
print "current tree stored in commit $commit\n" if $VERBOSE;
#
# Storing import control data
#
$ENV{GIT_INDEX_FILE} = $p4x;
open($fdo, '|-', 'git update-index --add --index-info') or
die "could not start git update-index\n";
binmode($fdo);
s/\r?\n//gs for ($p4spec,$p4clnt,$p4have);
print $fdo "100644 $p4spec\tspec\n";
print $fdo "100644 $p4clnt\tclient\n";
print $fdo "100644 $p4have\thave\n";
close($fdo);
if($?) {
die "Failed to store $SPEC in p4import index and git repo\n".
"Failed to save mappings of $P4CLIENT in p4import index and git repo\n".
"Failed to save state of $P4CLIENT in p4import index and git repo\n"
}
my ($p4tree)=qx{git write-tree};
die "Failed to store $SPEC (tree) in git repo\n" if $?;
# Bind import control data to the file data
$p4parent="-p $commit";
$p4parent="$p4parent -p $p4head" if length($p4head);
open(STDIN, '<', "$GIT_DIR/p4/p4msg") or die "p4/p4msg: $!\n";
$p4tree =~ s/\r?\n//gs;
($p4commit)=qx{git commit-tree $p4tree $p4parent};
die "Failed to store $SPEC (commit) in git repo\n" if $?;
$p4commit =~ s/\r?\n//gs;
# Finishing touches: update references
system('git','update-ref','-m','backup ref of current branch',
'p4/backup-HEAD','HEAD');
system('git','update-ref','-m','backup ref of p4import',
'p4/backup-p4import',"refs/p4import/$P4CLIENT");
$rc = system('git','update-ref','-m','data of p4import','HEAD',$commit);
die "Failed to update HEAD\n" if $rc;
$rc = system('git','update-ref','-m','p4import',"refs/p4import/$P4CLIENT",$p4commit);
die "Failed to store $SPEC (reference) in git repo\n" if $rc;
if ($VERBOSE) {
print STDOUT (grep {s/\r?\n//gs;s/.*?\s//} qx{git name-rev refs/p4import/$P4CLIENT}), ":\n";
system('git','log','--max-count=1','--pretty=format:%h %s%n',$p4commit);
}
print STDOUT (grep {s/\r?\n//gs;s/.*?\s//} qx{git name-rev HEAD}), ":\n";
system('git','log','--max-count=1','--pretty=format:%h %s%n',$commit);
}
^ permalink raw reply
* Re: [PATCH] Update bash completion to ignore some more plumbing commands
From: Jonas Fonseca @ 2007-05-22 21:06 UTC (permalink / raw)
To: Shawn O. Pearce; +Cc: git, Junio C Hamano
In-Reply-To: <20070521061228.GJ3141@spearce.org>
Shawn O. Pearce <spearce@spearce.org> wrote Mon, May 21, 2007:
> Jonas Fonseca <fonseca@diku.dk> wrote:
> > Signed-off-by: Jonas Fonseca <fonseca@diku.dk>
> > ---
> > contrib/completion/git-completion.bash | 4 ++++
> > 1 files changed, 4 insertions(+), 0 deletions(-)
>
> OK, sorry this missed 1.5.2. I've applied it into my fastimport
> master branch, I guess it will be in the next 1.5.2 based release.
> ;-)
Good! :) I didn't know, it was maintained in the fastimport branch. You
might consider mentioning it in the header of the file.
> > I am not sure if it is OK to exclude checkout-index as it is
> > the only method of exporting the source as a directory. Also,
>
> It might make sense to exclude checkout-index as plumbing, but I
> use it myself so often (and tab complete it too) that I want to
> keep it in. So I deleted that one line out of your patch when I
> applied it.
Very well.
> > I don't understand why git-diff-* is not excluded.
>
> Because I personally also use git diff-tree a lot. The others
> (diff-files, diff-index) I *never* use directly so I'm thinking
> maybe they should get flagged as plumbing and get removed from
> the completion.
OK.
--
Jonas Fonseca
^ permalink raw reply
* Re: [StGIT RFC] Changing patch@branch syntax
From: Yann Dirson @ 2007-05-22 21:00 UTC (permalink / raw)
To: Catalin Marinas; +Cc: GIT list
In-Reply-To: <b0943d9e0705220527x5d4c3d0fw2d0d66b37aab3f97@mail.gmail.com>
On Tue, May 22, 2007 at 01:27:08PM +0100, Catalin Marinas wrote:
> We had a discussion some time ago about using slashes for the //top or
> //bottom syntax and we ended up using two slashes. We could do the
> same to delimitate the branch from the patch - branch//patch//top. The
> branch or patch can have (single) slashes in their name.
>
> We couldn't use ":" at that time as it was used for the 'diff -r x:y'
> command. I later switched to the git ".." format (when git eventually
> defined it). As Karl said, branch:patch@top is another way, unless we
> later decide to add another level, the repository, and would like to
> have a uniform syntax (maybe always use ":" instead of "//"). We could
> run commands like:
>
> stg pick ../../path/to/linux-repo//branch//patch
Indeed I would possibly already have used this if it had been
available (working with 2 working copies for 2 cvs branches of a
single project, using stg-cvs in each).
> I don't think using "#" is feasible as bash ignores everything after
> it unless you use quotes or escape.
No, bash only considers # as a comment leader if there is space before
it, which would not be the case here (and is why the
http://repo#branch in cogito worked without problems). Not sure about
other shells, though.
I'm not very comfortable with // either, since having it special makes
it harder for script writers (need to canonicalize dirnames to ensure
a double slash is not inserted by error). Whereas it's not a big
problem to solve when the script writer is aware of the problem, it's
not something everyone would think about, and it may even break
existing scripts.
Following the "stg pick" example above, would we also want to allow
picking from a remote repo ? Then the URL fragment notation could be
suited, and we could have something like:
http://full/path/to/repo#my/branch:my/patch//top
That is, a formal syntax of:
[[[repo#]branch:]patch][//modifier]
Going further, since specifying a repo without a branch probably has
no meaning (unless we want to default to the HEAD branch), we could
simplify to the following:
[[[repo#]branch#]patch][//modifier]
I don't think we really want to allow "repo#patch", meaning that patch
on the current branch, as this gets easily confused by branch switching.
> Another nice thing to have is a way to get older versions of a patch
> via patchlogs. This should probably follow the current git notation,
> i.e. patch^^^ or patch~N.
Yes, that's something I've thought about several times, without coming
with a satisfying solution either: using the git notation to refer to
something different is not a good idea (we already have a "push"
command with a completely different meaning, let's keep these beasts
out if possible ;). There is already another syntax used by git for
reflogs (ref@{N}), which would have the same problem.
We should also keep in mind that someday we'll surely have merge
entries in patchlogs (one of the things near the top of my todo list),
so the same facility available in git for commit parents as ref^N will
be useful as well.
Another thing is, it would be great if the syntax we elect for this
would be "reserved" by git-core, so git-core does not start to use it
one day and cause a conflict. Maybe it could be "reserved" so other
porcelains can as well use it (yes, there is the problem of how those
other porcelains interact with stgit then - or rather, how will we
help the user not to get confused) ?
Best regards,
--
Yann.
^ permalink raw reply
* Re: [PATCH 07/16] git-read-tree: take --submodules option
From: Jan Hudec @ 2007-05-22 19:37 UTC (permalink / raw)
To: Martin Waitz; +Cc: Junio C Hamano, skimo, Alex Riesen, git
In-Reply-To: <20070521211133.GD5412@admingilde.org>
[-- Attachment #1: Type: text/plain, Size: 3604 bytes --]
On Mon, May 21, 2007 at 23:11:34 +0200, Martin Waitz wrote:
> On Mon, May 21, 2007 at 06:59:38PM +0200, Jan Hudec wrote:
> > There would be:
> > /
> > /.git
> > /subdir
> > /.git/submodules/submodule-name.git
> >
> > This would require changes to the logic how git finds GIT_DIR (which would be
> > really deep change), but it would provide place to store the submodule data
> > while the submodule is not being checked out.
>
> I agree that we need something like that.
>
> We don't have to move the entire subproject.git into the superproject,
> but we need to have all _referenced_ objects in the .git dir of the
> superproject.
>
> There are several possibilities to do so:
>
> * move the entire .git dir
> * move .git/objects
> * explicitly copy all referenced objects
I believe we really need entire .git dir. When the superporject checks out
revision which does not reference that subproject, we still need to preserve
not only the objects of subproject, but also the refs and config.
> I have some experimental code to configure a per-subproject directory
> in the superproject/.git as alternate object store for the submodule
> to make the last two solutions possible. Perhaps I should dig it out again
> and adapt it to current git.
>
> If there is a 1:1 relationship between subproject and object store then
> even efficient fsck and repack/prune are possible for the submodule without
> loosing objects.
> But such a 1:1 relationship is bad when you move subprojects to another
> location (or include the same subproject several times in different
> locations of the tree).
> Perhaps the user should be able to choose which one he wants.
That's why there should be the extra level of indirection using .gitmodules.
It should map the directory name to the object store name, so you can
relocate the subproject.
Including the same project several times is indeed interesting. Maybe the
subprojects should be "light checkouts" (I believe something like this was
already discussed on the list sometime). Those would be .git dirs, that would
only have HEAD and pointer to another .git dir with everything else.
> > > Not at all. There is no reason to believe that the case that
> > > superproject and subproject come from related URLs is more
> > > common. One of the reasons to do a separated project
> >
> > I definitely don't think it's more common. But it's the harder case and it
> > might happen. Generally it will happen if some people work on both the
> > superproject and the subproject. Of course the argument is that than it
> > should not be separate projects, but maybe the teams just partly overlap.
>
> I think it will be _very_ common to store super and subprojects in
> related locations. First to be independent from third-party servers
> while working on the superproject.
> Second (and I think more important) because many times there will
> be superproject related adaptations in the subproject. Yes they
> are independent, and exactly for that reason the subproject upstream
> maintainers may not take every change which is needed to satisfy the
> superproject. We _now_ see that in all Linux distributions already.
> So when you use superprojects to integrate several independent projects,
> then the superproject maintainer/administrator should really keep a
> clone of all subprojects handy on his site.
Yes, repositories with distribution-specific patches will add a large class
of cases requiring multiple sources support.
--
Jan 'Bulb' Hudec <bulb@ucw.cz>
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]
^ permalink raw reply
* Re: [PATCH] Prevent megablobs from gunking up git packs
From: Dana How @ 2007-05-22 18:07 UTC (permalink / raw)
To: Nicolas Pitre; +Cc: Junio C Hamano, Git Mailing List, danahow
In-Reply-To: <alpine.LFD.0.99.0705221329420.3366@xanadu.home>
On 5/22/07, Nicolas Pitre <nico@cam.org> wrote:
> On Mon, 21 May 2007, Dana How wrote:
> > This patch implements the following:
> > 1. git pack-objects takes a new --max-blob-size=N flag,
> > with the effect that only blobs less than N KB are written
> > to the packfiles(s). If a blob was in a pack but violates
> > this limit (perhaps the packs were created by fast-import
> > or max-blob-size was reduced), then a new loose object
> > is written out if needed so the data is not lost.
> > 2. git repack inspects repack.maxblobsize . If set, its
> > value is passed to git pack-objects on the command line.
> > The user should change repack.maxblobsize , NOT specify
> > --max-blob-size=N .
> > 3. No other caller of git pack-objects supplies this new flag,
> > so other callers see no change.
> >
> > This patch is on top of the earlier max-pack-size patch,
> > because I thought I needed some behavior it supplied,
> > but could be rebased on master if desired.
>
> I think what this patch is missing is a test after all options have been
> parsed to prevent --stdout and --max-blob-size to be used together.
Yes. I will also update the documentation.
Thanks,
--
Dana L. How danahow@gmail.com +1 650 804 5991 cell
^ permalink raw reply
* Re: [PATCH] Prevent megablobs from gunking up git packs
From: Nicolas Pitre @ 2007-05-22 17:38 UTC (permalink / raw)
To: Dana How; +Cc: Junio C Hamano, Git Mailing List
In-Reply-To: <46528A48.9050903@gmail.com>
On Mon, 21 May 2007, Dana How wrote:
>
> Using fast-import and repack with the max-pack-size patch,
> 3628 commits were imported from Perforce comprising
> 100.35GB (uncompressed) in 38829 blobs, and saved in
> 7 packfiles of 12.5GB total (--window=0 and --depth=0 were
> used due to runtime limits). When using these packfiles,
> several git commands showed very large process sizes,
> and some slowdowns (compared to comparable operations
> on the linux kernel repo) were also apparent.
>
> git stores data in loose blobs or in packfiles. The former
> has essentially now become an exception mechanism, to store
> exceptionally *young* blobs. Why not use this to store
> exceptionally *large* blobs as well? This allows us to
> re-use all the "exception" machinery with only a small change.
>
> Repacking the entire repository with a max-blob-size of 256KB
> resulted in a single 13.1MB packfile, as well as 2853 loose
> objects totaling 15.4GB compressed and 100.08GB uncompressed,
> 11 files per objects/xx directory on average. All was created
> in half the runtime of the previous yet with standard
> --window=10 and --depth=50 parameters. The data in the
> packfile was 270MB uncompressed in 35976 blobs. Operations
> such as "git-log --pretty=oneline" were about 30X faster
> on a cold cache and 2 to 3X faster otherwise. Process sizes
> remained reasonable.
>
> This patch implements the following:
> 1. git pack-objects takes a new --max-blob-size=N flag,
> with the effect that only blobs less than N KB are written
> to the packfiles(s). If a blob was in a pack but violates
> this limit (perhaps the packs were created by fast-import
> or max-blob-size was reduced), then a new loose object
> is written out if needed so the data is not lost.
> 2. git repack inspects repack.maxblobsize . If set, its
> value is passed to git pack-objects on the command line.
> The user should change repack.maxblobsize , NOT specify
> --max-blob-size=N .
> 3. No other caller of git pack-objects supplies this new flag,
> so other callers see no change.
>
> This patch is on top of the earlier max-pack-size patch,
> because I thought I needed some behavior it supplied,
> but could be rebased on master if desired.
I think what this patch is missing is a test after all options have been
parsed to prevent --stdout and --max-blob-size to be used together.
Nicolas
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox