* Re: [PATCH] tutorial.txt renamed
From: Junio C Hamano @ 2009-01-07 6:27 UTC (permalink / raw)
To: Brian Gernhardt; +Cc: Christian Couder, Joey Hess, git
In-Reply-To: <DA6E1A83-FFBA-46BC-9BCF-ED8A2D8F12E7@silverinsanity.com>
Brian Gernhardt <benji@silverinsanity.com> writes:
> This is the README file for the project, so it should advise looking
> at the Documentation directory as neither the man pages or git command
> are likely installed at this point.
I think that is a sane suggestion. It is better to keep the number of
prerequisites to the minimum for the user in order to follow README (and
INSTALL, of course).
^ permalink raw reply
* Re: [PATCH] tutorial.txt renamed
From: Christian Couder @ 2009-01-07 6:07 UTC (permalink / raw)
To: Brian Gernhardt; +Cc: Joey Hess, git
In-Reply-To: <DA6E1A83-FFBA-46BC-9BCF-ED8A2D8F12E7@silverinsanity.com>
Le mercredi 7 janvier 2009, Brian Gernhardt a écrit :
> On Jan 7, 2009, at 12:28 AM, Christian Couder wrote:
> > Le mercredi 7 janvier 2009, Joey Hess a écrit :
> >> diff --git a/README b/README
> >> index 548142c..5fa41b7 100644
> >> --- a/README
> >> +++ b/README
> >> @@ -24,7 +24,7 @@ It was originally written by Linus Torvalds with
> >> help
> >> of a group of hackers around the net. It is currently maintained by
> >> Junio
> >> C Hamano.
> >>
> >> Please read the file INSTALL for installation instructions.
> >> -See Documentation/tutorial.txt to get started, then see
> >> +See Documentation/gittutorial.txt to get started,
> >
> > "man gittutorial" and "git help tutorial" should work to display the
> > tutorial, so perhaps we should advise to use them instead of the
> > source,
> > since we are advising to use "man git-commandname" below to get help
> > on
> > each command.
>
> This is the README file for the project, so it should advise looking
> at the Documentation directory as neither the man pages or git command
> are likely installed at this point.
Well, this is debatable, because we first ask the user to read the INSTALL
file, and a tutorial for git may not be very usefull if you don't have it
installed to try out the tutorial commands.
Regards,
Christian.
^ permalink raw reply
* Re: Error: unable to unlink ... when using "git gc"
From: Jeff King @ 2009-01-07 5:59 UTC (permalink / raw)
To: Sitaram Chamarty; +Cc: git
In-Reply-To: <slrngm6uf5.vuo.sitaramc@sitaramc.homelinux.net>
On Tue, Jan 06, 2009 at 03:33:57PM +0000, Sitaram Chamarty wrote:
> > We also plan to do it in this way, just a small wondering that it
> > looks a kind of workaround instead of a more graceful solution.
>
> I wouldn't consider it a workaround. It uses normal Unix
> permissions the way they were designed to, including setgid
> for directories.
Yes, I think core.sharedrepository is the "official" way to do this, so
it is definitely not a workaround.
> Actually, I am yet to come up with a situation where I
> actually needed ACLs, though they are more generalised, and
> fine-grained.
I like ACLs mainly because you don't have to bug root to change
permissions (like you do to get them to create or modify a group).
-Peff
^ permalink raw reply
* Re: [PATCH] tutorial.txt renamed
From: Brian Gernhardt @ 2009-01-07 5:36 UTC (permalink / raw)
To: Christian Couder; +Cc: Joey Hess, git
In-Reply-To: <200901070628.38019.chriscool@tuxfamily.org>
On Jan 7, 2009, at 12:28 AM, Christian Couder wrote:
> Le mercredi 7 janvier 2009, Joey Hess a écrit :
>> diff --git a/README b/README
>> index 548142c..5fa41b7 100644
>> --- a/README
>> +++ b/README
>> @@ -24,7 +24,7 @@ It was originally written by Linus Torvalds with
>> help
>> of a group of hackers around the net. It is currently maintained by
>> Junio
>> C Hamano.
>>
>> Please read the file INSTALL for installation instructions.
>> -See Documentation/tutorial.txt to get started, then see
>> +See Documentation/gittutorial.txt to get started,
>
> "man gittutorial" and "git help tutorial" should work to display the
> tutorial, so perhaps we should advise to use them instead of the
> source,
> since we are advising to use "man git-commandname" below to get help
> on
> each command.
This is the README file for the project, so it should advise looking
at the Documentation directory as neither the man pages or git command
are likely installed at this point.
>> CVS users may also want to read Documentation/cvs-migration.txt.
>
> The "cvs-migration.txt" was also renamed "gitcvs-migration.txt". It
> should
> be available with "man gitcvs-migration" and "git help cvs-migration".
This however is a valid point.
I would also suggest that the patch have a different name, as I
expected it to be renaming tutorial.txt and I was going to ask why.
Perhaps "README: correct for renamed files"?
~~ Brian G.
^ permalink raw reply
* Re: [PATCH] tutorial.txt renamed
From: Christian Couder @ 2009-01-07 5:28 UTC (permalink / raw)
To: Joey Hess; +Cc: git
In-Reply-To: <20090107042337.GA24735@gnu.kitenet.net>
Le mercredi 7 janvier 2009, Joey Hess a écrit :
> The tutorial.txt file was renamed to gittutorial.txt some time ago,
> update README.
>
> Signed-off-by: Joey Hess <joey@gnu.kitenet.net>
> ---
> README | 2 +-
> 1 files changed, 1 insertions(+), 1 deletions(-)
>
> diff --git a/README b/README
> index 548142c..5fa41b7 100644
> --- a/README
> +++ b/README
> @@ -24,7 +24,7 @@ It was originally written by Linus Torvalds with help
> of a group of hackers around the net. It is currently maintained by Junio
> C Hamano.
>
> Please read the file INSTALL for installation instructions.
> -See Documentation/tutorial.txt to get started, then see
> +See Documentation/gittutorial.txt to get started,
"man gittutorial" and "git help tutorial" should work to display the
tutorial, so perhaps we should advise to use them instead of the source,
since we are advising to use "man git-commandname" below to get help on
each command.
> then see
> Documentation/everyday.txt for a useful minimum set of commands,
(But the everyday.txt file has not been converted to a man page, so we
cannot advise to use "man giteveryday".)
> and "man git-commandname" for documentation of each command.
Yeah "man git-commandname" and "git help commandname" should work.
> CVS users may also want to read Documentation/cvs-migration.txt.
The "cvs-migration.txt" was also renamed "gitcvs-migration.txt". It should
be available with "man gitcvs-migration" and "git help cvs-migration".
Thanks,
Christian.
> --
> 1.5.6.5
^ permalink raw reply
* Re: JGit vs. Git
From: Vagmi Mudumbai @ 2009-01-07 5:08 UTC (permalink / raw)
To: git
In-Reply-To: <alpine.DEB.1.00.0901062240240.30769@pacific.mpi-cbg.de>
Hi,
@Stephen
>> Or think about extending the Ruby gem grit to also use JGit. Which would certainly improve grit and probably help improve JGit also.
I just started working on that. It will be close to Grit let me see
how far I get with it. If you have an existing repo that you have
already worked on, please feel free to share it.
The JGit code both tests and the UI is quite readable.
Thanks a ton for all your help. :-)
Regards,
Vagmi
http://blog.vagmim.com
"Teaching children to use Windows is like teaching them to smoke
tobacco—in a world where only one company sells tobacco." - Richard
Stallman
^ permalink raw reply
* Re: git-rev-parse --symbolic-abbrev-name
From: Arnaud Lacombe @ 2009-01-07 4:58 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Karl Chen, Miklos Vajna, David Aguilar, Git mailing list
In-Reply-To: <7vocykkftg.fsf@gitster.siamese.dyndns.org>
On Tue, Jan 6, 2009 at 3:18 AM, Junio C Hamano <gitster@pobox.com> wrote:
>> diff --git a/builtin-rev-parse.c b/builtin-rev-parse.c
>> index 81d5a6f..70f4a33 100644
>> --- a/builtin-rev-parse.c
>> +++ b/builtin-rev-parse.c
>> @@ -24,6 +24,7 @@ static int show_type = NORMAL;
>>
>> #define SHOW_SYMBOLIC_ASIS 1
>> #define SHOW_SYMBOLIC_FULL 2
>> +#define SHOW_SYMBOLIC_SHORT 3
>> static int symbolic;
>> static int abbrev;
>> static int output_sq;
>
> I think --symbolic-short makes the most sense.
>
ok, thanks.
>> @@ -125,13 +129,20 @@ static void show_rev(int type, const unsigned char *sha1, const char *name)
>> */
>> break;
>> case 1: /* happy */
>> + if (symbolic == SHOW_SYMBOLIC_SHORT) {
>> + char *p;
>> + p = strrchr(full, (int)'/');
>> + if (p != NULL)
>> + full = p + 1;
>> + }
>
> However, this is not a good way to do it, I suspect. This patch most
> likely will be queued to the al/symbolic-short topic branch, but you are
> losing information here. You'd probably want to try substings from the
> tail of the full name (e.g. symbolic-short, al/symbolic-short,
> heads/al/symbolic-short, and finally refs/heads/al/symbolic-short) and
> feed them to dwim_ref() and pick the shortest one that yields the same ref
> unambiguously, or something like that.
>
ok, I see what you mean, I'll rework the patch to fix this. I was
about to do a proper patch submission when I saw you reply, so it will
be for next time!
> By the way, I do not see why you need to cast '/'.
>
overzealous type casting due to lack of cafeine in blood :-)
regards,
- Arnaud
^ permalink raw reply
* Re: [PATCH/RFC] Allow writing loose objects that are corrupted in a pack file
From: Linus Torvalds @ 2009-01-07 4:54 UTC (permalink / raw)
To: R. Tyler Ballance; +Cc: Nicolas Pitre, Jan Krüger, Git ML
In-Reply-To: <1231292360.8870.61.camel@starfruit>
On Tue, 6 Jan 2009, R. Tyler Ballance wrote:
>
> I'll back the patch out and redeploy, it's worth mentioning that a
> coworker of mine just got the issue as well (on 1.6.1). He was able to
> `git pull` and the error went away, but I doubt that it "magically fixed
> itself"
Quite frankly, that behaviour sounds like a disk _cache_ corruption issue.
The fact that some corruption "comes and goes" and sometimes magically
heals itself sounds very much like some disk cache problem, and then that
particular part of the cache gets replaced and then when re-populated it
is magically correct.
We had that in one case with a Linux NFS client, where a rename across
directories caused problems.
This was a networked filesystem on OS X, right? File caching is much more
"interesting" in networked filesystems than it is in normal private
on-disk ones.
> I've tarred one of the repositories that had it in a reproducible state
> so I can create a build and extract the tar and run against that to
> verify any patches anybody might have, but unfortunately at 7GB of
> company code and assets, I can't exactly share ;)
The thing to do is
- untar it on some trusted machine with a local disk and a known-good
filesystem.
IOW, not that networked samba share.
- verify that it really does happen on that machine, with that untarred
image. Because maybe it doesn't.
The hope is that you caught the corruption in the cache, and it
actually got written out to the tar-file. But if it _is_ a disk cache
(well, network cache) issue, maybe the IO required to tar everything up
was enough to flush it, and the tar-file actually _works_ because it
got repopulated correctly.
So that's why you should double-check that it really ends up being
corrupt after being untarred again.
- go back and test the original git repo on the network share, preferably
on another client. See if the error has gone away.
- If so, try to compare that known-corrupt filesystem with the original
one: and preferably do this on another machine over the network mount.
See if they differ. They obviously should *not* differ, since it's an
tar/untar of the same files, but ...
The fact that you seem to get a _lot_ of these errors really does make it
sound like something in your environment. It's actually really hard to get
git to corrupt anything. Especially objects that got packed. They've been
quiescent for a long time, they got repacked in a very simple way, they
are totally read-only.
But it is _not_ hard to corrupt network filesystems. It's downright
trivial with some of them, especially with some hardware (eg there's no
end-to-end checksumming except for the _extremely_ weak 16-bit IP csum,
and even that has been known to be disabled, or screwed up by ethernet
cards that do IP packet offloading and thus computing the csum not on the
data that tee user actually wrote, but the data that the card received,
which is not necessarily at all the same thing).
And while ethernet uses a stronger CRC, that one is not end-to-end, so
corruption on the card or in a switch in between easily defeats that too.
Just google for something like
"OS X" SMB "file corruption"
and you'll find quite a bit of hits. Not all that unusual.
Linus
^ permalink raw reply
* [PATCH] gitweb: support the rel=vcs microformat
From: Joey Hess @ 2009-01-07 4:25 UTC (permalink / raw)
To: git
The rel=vcs microformat allows a web page to indicate the locations of
repositories related to it in a machine-parseable manner.
(See http://kitenet.net/~joey/rfc/rel-vcs/)
Make gitweb use the microformat in the header of pages it generates,
if it has been configured with project url information in any of the usual
ways.
Since getting the urls can require hitting disk, I avoided putting the
microformat on *every* page gitweb generates. Just put it on the project
summary page, the project list page, and the forks list page.
The first of these already looks up the urls, so adding the microformat was
free. There is a small overhead in including the microformat on the
latter two pages, but getting the project descriptions for those pages
already incurs a similar overhead, and the ability to get every repo url
in one place seems worthwhile.
This changes git_get_project_description() to not check wantarray, and only
return in list context -- the only way it is used AFAICS.
Signed-off-by: Joey Hess <joey@gnu.kitenet.net>
---
gitweb/gitweb.perl | 38 ++++++++++++++++++++++++++------------
1 files changed, 26 insertions(+), 12 deletions(-)
diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl
index 99f71b4..3f8a228 100755
--- a/gitweb/gitweb.perl
+++ b/gitweb/gitweb.perl
@@ -789,6 +789,9 @@ $git_dir = "$projectroot/$project" if $project;
our @snapshot_fmts = gitweb_get_feature('snapshot');
@snapshot_fmts = filter_snapshot_fmts(@snapshot_fmts);
+# populated later with git urls for the project
+our @git_url_list;
+
# dispatch
if (!defined $action) {
if (defined $hash) {
@@ -2100,17 +2103,22 @@ sub git_show_project_tagcloud {
}
sub git_get_project_url_list {
+ # use per project git URL list in $projectroot/$path/cloneurl
+ # or make project git URL from git base URL and project name
my $path = shift;
+ my @ret;
+
$git_dir = "$projectroot/$path";
- open my $fd, "$git_dir/cloneurl"
- or return wantarray ?
- @{ config_to_multi(git_get_project_config('url')) } :
- config_to_multi(git_get_project_config('url'));
- my @git_project_url_list = map { chomp; $_ } <$fd>;
- close $fd;
+ if (open my $fd, "$git_dir/cloneurl") {
+ @ret = map { chomp; $_ } <$fd>;
+ close $fd;
+ }
+ else {
+ @ret = @{ config_to_multi(git_get_project_config('url')) };
+ }
- return wantarray ? @git_project_url_list : \@git_project_url_list;
+ return @ret ? @ret : map { "$_/$project" } @git_base_url_list;
}
sub git_get_projects_list {
@@ -2953,6 +2961,10 @@ EOF
print qq(<link rel="shortcut icon" href="$favicon" type="image/png" />\n);
}
+ foreach my $url (@git_url_list) {
+ print qq{<link rel="vcs" type="git" href="$url" />\n};
+ }
+
print "</head>\n" .
"<body>\n";
@@ -4380,6 +4392,8 @@ sub git_project_list {
die_error(404, "No projects found");
}
+ @git_url_list = map { git_get_project_url_list($_->{path}) } @list;
+
git_header_html();
if (-f $home_text) {
print "<div class=\"index_include\">\n";
@@ -4400,6 +4414,8 @@ sub git_forks {
if (defined $order && $order !~ m/none|project|descr|owner|age/) {
die_error(400, "Unknown order parameter");
}
+
+ @git_url_list = map { git_get_project_url_list($_->{path}) } @list;
my @list = git_get_projects_list($project);
if (!@list) {
@@ -4457,6 +4473,8 @@ sub git_summary {
@forklist = git_get_projects_list($project);
}
+ @git_url_list = git_get_project_url_list($project);
+
git_header_html();
git_print_page_nav('summary','', $head);
@@ -4468,12 +4486,8 @@ sub git_summary {
print "<tr id=\"metadata_lchange\"><td>last change</td><td>$cd{'rfc2822'}</td></tr>\n";
}
- # use per project git URL list in $projectroot/$project/cloneurl
- # or make project git URL from git base URL and project name
my $url_tag = "URL";
- my @url_list = git_get_project_url_list($project);
- @url_list = map { "$_/$project" } @git_base_url_list unless @url_list;
- foreach my $git_url (@url_list) {
+ foreach my $git_url (@git_url_list) {
next unless $git_url;
print "<tr class=\"metadata_url\"><td>$url_tag</td><td>$git_url</td></tr>\n";
$url_tag = "";
--
1.5.6.5
^ permalink raw reply related
* [PATCH] tutorial.txt renamed
From: Joey Hess @ 2009-01-07 4:23 UTC (permalink / raw)
To: git
The tutorial.txt file was renamed to gittutorial.txt some time ago, update
README.
Signed-off-by: Joey Hess <joey@gnu.kitenet.net>
---
README | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/README b/README
index 548142c..5fa41b7 100644
--- a/README
+++ b/README
@@ -24,7 +24,7 @@ It was originally written by Linus Torvalds with help of a group of
hackers around the net. It is currently maintained by Junio C Hamano.
Please read the file INSTALL for installation instructions.
-See Documentation/tutorial.txt to get started, then see
+See Documentation/gittutorial.txt to get started, then see
Documentation/everyday.txt for a useful minimum set of commands,
and "man git-commandname" for documentation of each command.
CVS users may also want to read Documentation/cvs-migration.txt.
--
1.5.6.5
--
see shy jo
^ permalink raw reply related
* Re: [JGIT RFC] How read versions of a specific object
From: Shawn O. Pearce @ 2009-01-07 4:04 UTC (permalink / raw)
To: Imran M Yousuf; +Cc: Git Mailing List
In-Reply-To: <7bfdc29a0901061944x454a9t1d01e6744f08cf78@mail.gmail.com>
Imran M Yousuf <imyousuf@gmail.com> wrote:
> I am trying to read all or n-th version of an object. Currently to do
> this I am using the following piece of code, which has to walk to
> every commit is present and from there prepare a set of its object id,
> it is definitely expensive if the commit history is huge, is there a
> faster/better way to achieve it?
Not really. You can more efficiently use JGit and reduce some of
the overheads, but that's about it.
> for (int i = 0; i < App.OBJECT_COUNT;
> ++i) {
> ObjectWalk objectWalk = new ObjectWalk(repo);
Don't use ObjectWalk, use a RevWalk. You don't need it to keep
track of tree or blob identities. The ObjectWalk code has more
overhead to do that bookkeeping.
> Commit revision = repo.mapCommit(revObject.getId());
> Tree versionTree = repo.mapTree(revision.getTreeId());
> if (versionTree.existsBlob(isbn)) {
> revisions.add(versionTree.findBlobMember(isbn).getId());
Use a TreeWalk to do this. Its quicker because it doesn't
have to parse as much data to come up with the same result.
More specifically there's a static factory method that sets up for
a path limited walk and returns the TreeWalk pointing at that entry.
You can use the fact that RevWalk.next() returns a RevCommit to get
you the RevTree, which is the tree you need to give to the TreeWalk
constructor (its the root level tree of the commit).
But if App.OBJECT_COUNT is quite large and covers most of your
objects, you are probably better off using a loop over the commits
and diff'ing against the ancestor:
final HashMap<String, Set<ObjectId>> versions = ...;
final RevWalk rw = new RevWalk(repo);
final TreeWalk tw = new TreeWalk(repo);
rw.markStart(rw.parseCommit(repo.parse(HEAD)));
tw.setFilter(TreeFilter.ANY_DIFF);
RevCommit c;
while ((c = rw.next()) != null) {
final ObjectId[] p = new ObjectId[c.getParentCount() + 1];
for (int i = 0; i < c.getParentCount(); i++) {
rw.parse(c.getParent(i));
p[i] = c.getParent(i).getTree();
}
final int me = p.length -1;
p[me] = c.getTree();
tw.reset(p);
while (tw.next()) {
if (tw.getFileMode(me).getObjectType() == Constants.OBJ_BLOB) {
// This path was modified relative to the ancestor(s).
//
String s = tw.getPathString();
Set<ObjectId> i = versions.get(s);
if (i == null)
versions.put(s, i = new HashSet<ObjectId>());
i.add(tw.getObjectId(me));
}
if (tw.isSubtree()) {
// make sure we recurse into modified directories
tw.enterSubtree();
}
}
}
--
Shawn.
^ permalink raw reply
* [JGIT RFC] How read versions of a specific object
From: Imran M Yousuf @ 2009-01-07 3:44 UTC (permalink / raw)
To: Git Mailing List
Hi,
I am trying to read all or n-th version of an object. Currently to do
this I am using the following piece of code, which has to walk to
every commit is present and from there prepare a set of its object id,
it is definitely expensive if the commit history is huge, is there a
faster/better way to achieve it?
for (int i = 0; i < App.OBJECT_COUNT;
++i) {
System.out.println("INDEX: " + i);
String isbn =
String.valueOf(Integer.parseInt(App.INIT_ID) + i);
System.out.println("ISBN: " + isbn);
ObjectWalk objectWalk = new ObjectWalk(repo);
/*
* Checks whether the Commit has the tree or not. It does not
* check whether it has changed or not.
*/
objectWalk.setTreeFilter(PathFilter.create(isbn));
RevObject revObject = null;
objectWalk.markStart(objectWalk.parseCommit(repo.resolve(
Constants.HEAD)));
Set<ObjectId> revisions =
new HashSet<ObjectId>();
do {
if (revObject != null) {
Commit revision = repo.mapCommit(revObject.getId());
Tree versionTree = repo.mapTree(revision.getTreeId());
if (versionTree.existsBlob(isbn)) {
revisions.add(versionTree.findBlobMember(isbn).getId());
}
}
revObject = objectWalk.next();
}
while (revObject != null);
System.out.println("Revisions: " + revisions);
}
The details source code of the project is available @
http://github.com/imyousuf/jgit-usage/tree/master
Thank you,
--
Imran M Yousuf
Entrepreneur & Software Engineer
Smart IT Engineering
Dhaka, Bangladesh
Email: imran@smartitengineering.com
Blog: http://imyousuf-tech.blogs.smartitengineering.com/
Mobile: +880-1711402557
^ permalink raw reply
* Re: [PATCH/RFC] Allow writing loose objects that are corrupted in a pack file
From: Nicolas Pitre @ 2009-01-07 3:21 UTC (permalink / raw)
To: R. Tyler Ballance; +Cc: Jan Krüger, Git ML
In-Reply-To: <1231296475.8870.89.camel@starfruit>
On Tue, 6 Jan 2009, R. Tyler Ballance wrote:
> On Tue, 2009-01-06 at 21:09 -0500, Nicolas Pitre wrote:
> > > I've tarred one of the repositories that had it in a reproducible
> > state
> >
> > That is wonderful.
> >
> > > so I can create a build and extract the tar and run against that to
> > > verify any patches anybody might have, but unfortunately at 7GB of
> > > company code and assets, I can't exactly share ;)
> >
> > First step is to understand what is going on. Only then could reliable
> > patches be made.
>
> If you want to point me in the right direction, I have a few hours to
> kill this evening and fscking around with gdb(1) and printf() just might
> be some of my favorite things</sarcasm> ;)
Heh. ;-)
To start with, a simple log of what you need to do to reproduce the
issue would be nice. Just do
script /tmp/foo
then reproduce the issue and exit, after which I'd be interrested in the
content of /tmp/foo.
Nicolas
^ permalink raw reply
* Re: [RFC PATCH] diff --no-index: test for pager after option parsing
From: Miklos Vajna @ 2009-01-07 3:20 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Thomas Rast, git
In-Reply-To: <7vfxjwf041.fsf@gitster.siamese.dyndns.org>
[-- Attachment #1: Type: text/plain, Size: 352 bytes --]
On Tue, Jan 06, 2009 at 04:09:18PM -0800, Junio C Hamano <gitster@pobox.com> wrote:
> But I wonder if it still makes a difference in real life.idn't we stop
> reporting the exit status from the pager some time ago?
I just wanted to write this, I think that code could be just removed
since ea27a18 (spawn pager via run_command interface, 2008-07-22).
[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]
^ permalink raw reply
* [PATCH v2] parse-opt: migrate builtin-ls-files.
From: Miklos Vajna @ 2009-01-07 3:11 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Pierre Habouzit, git
In-Reply-To: <20090106102202.GA30766@artemis.corp>
Signed-off-by: Miklos Vajna <vmiklos@frugalware.org>
---
On Tue, Jan 06, 2009 at 11:22:02AM +0100, Pierre Habouzit <madcoder@debian.org> wrote:
> > +static int option_parse_no_empty(const struct option *opt,
> > + const char *arg, int unset)
> > +{
> > + struct dir_struct *dir = opt->value;
> > +
> > + dir->hide_empty_directories = 1;
> > +
> > + return 0;
> > +}
>
> Should be option_parse_empty and deal with "unset" to know if `no-`
> was
> prefixed to it or not.
>
>
> > + { OPTION_CALLBACK, 0, "no-empty-directory", &dir, NULL,
> > + "do not list empty directories",
>
> This should be "empty-directory" and "list empty directories as well"
Ah, sure.
> I've not checked if you could also check more of the "unsets" things
> in
> your callbacks as well btw, but it looks like it could.
Right, added to option_parse_ignored() and option_parse_directory() as
well.
Interdiff: b3b6ad0..a44941c in git://repo.or.cz/git/vmiklos.git.
builtin-ls-files.c | 303 +++++++++++++++++++++++++++++-----------------------
1 files changed, 168 insertions(+), 135 deletions(-)
diff --git a/builtin-ls-files.c b/builtin-ls-files.c
index f72eb85..8a946ef 100644
--- a/builtin-ls-files.c
+++ b/builtin-ls-files.c
@@ -10,6 +10,7 @@
#include "dir.h"
#include "builtin.h"
#include "tree.h"
+#include "parse-options.h"
static int abbrev;
static int show_deleted;
@@ -28,6 +29,7 @@ static const char **pathspec;
static int error_unmatch;
static char *ps_matched;
static const char *with_tree;
+static int exc_given;
static const char *tag_cached = "";
static const char *tag_unmerged = "";
@@ -395,156 +397,187 @@ int report_path_error(const char *ps_matched, const char **pathspec, int prefix_
return errors;
}
-static const char ls_files_usage[] =
- "git ls-files [-z] [-t] [-v] (--[cached|deleted|others|stage|unmerged|killed|modified])* "
- "[ --ignored ] [--exclude=<pattern>] [--exclude-from=<file>] "
- "[ --exclude-per-directory=<filename> ] [--exclude-standard] "
- "[--full-name] [--abbrev] [--] [<file>]*";
+static const char * const ls_files_usage[] = {
+ "git ls-files [options] [<file>]*",
+ NULL
+};
+
+static int option_parse_z(const struct option *opt,
+ const char *arg, int unset)
+{
+ if (unset)
+ line_terminator = '\n';
+ else
+ line_terminator = 0;
+ return 0;
+}
+
+static int option_parse_exclude(const struct option *opt,
+ const char *arg, int unset)
+{
+ struct dir_struct *dir = opt->value;
+
+ exc_given = 1;
+ add_exclude(arg, "", 0, &dir->exclude_list[EXC_CMDL]);
+
+ return 0;
+}
+
+static int option_parse_exclude_from(const struct option *opt,
+ const char *arg, int unset)
+{
+ struct dir_struct *dir = opt->value;
+
+ exc_given = 1;
+ add_excludes_from_file(dir, arg);
+
+ return 0;
+}
+
+static int option_parse_exclude_standard(const struct option *opt,
+ const char *arg, int unset)
+{
+ struct dir_struct *dir = opt->value;
+
+ exc_given = 1;
+ setup_standard_excludes(dir);
+
+ return 0;
+}
+
+static int option_parse_ignored(const struct option *opt,
+ const char *arg, int unset)
+{
+ struct dir_struct *dir = opt->value;
+
+ if (unset)
+ dir->show_ignored = 0;
+ else
+ dir->show_ignored = 1;
+
+ return 0;
+}
+
+static int option_parse_directory(const struct option *opt,
+ const char *arg, int unset)
+{
+ struct dir_struct *dir = opt->value;
+
+ if (unset)
+ dir->show_other_directories = 0;
+ else
+ dir->show_other_directories = 1;
+
+ return 0;
+}
+
+static int option_parse_empty(const struct option *opt,
+ const char *arg, int unset)
+{
+ struct dir_struct *dir = opt->value;
+
+ if (unset)
+ dir->hide_empty_directories = 1;
+ else
+ dir->hide_empty_directories = 0;
+
+ return 0;
+}
+
+static int option_parse_full_name(const struct option *opt,
+ const char *arg, int unset)
+{
+ prefix_offset = 0;
+
+ return 0;
+}
int cmd_ls_files(int argc, const char **argv, const char *prefix)
{
- int i;
- int exc_given = 0, require_work_tree = 0;
+ int require_work_tree = 0, show_tag = 0;
struct dir_struct dir;
+ struct option builtin_ls_files_options[] = {
+ { OPTION_CALLBACK, 'z', NULL, NULL, NULL,
+ "paths are separated with NUL character",
+ PARSE_OPT_NOARG, option_parse_z },
+ OPT_BOOLEAN('t', NULL, &show_tag,
+ "identify the file status with tags"),
+ OPT_BOOLEAN('v', NULL, &show_valid_bit,
+ "use lowercase letters for 'assume unchanged' files"),
+ OPT_BOOLEAN('c', "cached", &show_cached,
+ "show cached files in the output (default)"),
+ OPT_BOOLEAN('d', "deleted", &show_deleted,
+ "show deleted files in the output"),
+ OPT_BOOLEAN('m', "modified", &show_modified,
+ "show modified files in the output"),
+ OPT_BOOLEAN('o', "others", &show_others,
+ "show other files in the output"),
+ { OPTION_CALLBACK, 'i', "ignored", &dir, NULL,
+ "show ignored files in the output",
+ PARSE_OPT_NOARG, option_parse_ignored },
+ OPT_BOOLEAN('s', "stage", &show_stage,
+ "show staged contents' object name in the output"),
+ OPT_BOOLEAN('k', "killed", &show_killed,
+ "show files on the filesystem that need to be removed"),
+ { OPTION_CALLBACK, 0, "directory", &dir, NULL,
+ "show 'other' directories' name only",
+ PARSE_OPT_NOARG, option_parse_directory },
+ { OPTION_CALLBACK, 0, "empty-directory", &dir, NULL,
+ "list empty directories",
+ PARSE_OPT_NOARG, option_parse_empty },
+ OPT_BOOLEAN('u', "unmerged", &show_unmerged,
+ "show unmerged files in the output"),
+ { OPTION_CALLBACK, 'x', "exclude", &dir, "pattern",
+ "skip files matching pattern",
+ 0, option_parse_exclude },
+ { OPTION_CALLBACK, 'X', "exclude-from", &dir, "file",
+ "exclude patterns are read from <file>",
+ 0, option_parse_exclude_from },
+ OPT_STRING(0, "exclude-per-directory", &dir.exclude_per_dir, "file",
+ "read additional per-directory exclude patterns in <file>"),
+ { OPTION_CALLBACK, 0, "exclude-standard", &dir, NULL,
+ "add the standard git exclusions",
+ PARSE_OPT_NOARG, option_parse_exclude_standard },
+ { OPTION_CALLBACK, 0, "full-name", NULL, NULL,
+ "make the output relative to the project top directory",
+ PARSE_OPT_NOARG, option_parse_full_name },
+ OPT_BOOLEAN(0, "error-unmatch", &error_unmatch,
+ "if any <file> is not in the index, treat this as an error"),
+ OPT_STRING(0, "with-tree", &with_tree, "tree-ish",
+ "pretend that paths removed since <tree-ish> are still present"),
+ OPT__ABBREV(&abbrev),
+ OPT_END()
+ };
memset(&dir, 0, sizeof(dir));
if (prefix)
prefix_offset = strlen(prefix);
git_config(git_default_config, NULL);
- for (i = 1; i < argc; i++) {
- const char *arg = argv[i];
-
- if (!strcmp(arg, "--")) {
- i++;
- break;
- }
- if (!strcmp(arg, "-z")) {
- line_terminator = 0;
- continue;
- }
- if (!strcmp(arg, "-t") || !strcmp(arg, "-v")) {
- tag_cached = "H ";
- tag_unmerged = "M ";
- tag_removed = "R ";
- tag_modified = "C ";
- tag_other = "? ";
- tag_killed = "K ";
- if (arg[1] == 'v')
- show_valid_bit = 1;
- continue;
- }
- if (!strcmp(arg, "-c") || !strcmp(arg, "--cached")) {
- show_cached = 1;
- continue;
- }
- if (!strcmp(arg, "-d") || !strcmp(arg, "--deleted")) {
- show_deleted = 1;
- continue;
- }
- if (!strcmp(arg, "-m") || !strcmp(arg, "--modified")) {
- show_modified = 1;
- require_work_tree = 1;
- continue;
- }
- if (!strcmp(arg, "-o") || !strcmp(arg, "--others")) {
- show_others = 1;
- require_work_tree = 1;
- continue;
- }
- if (!strcmp(arg, "-i") || !strcmp(arg, "--ignored")) {
- dir.show_ignored = 1;
- require_work_tree = 1;
- continue;
- }
- if (!strcmp(arg, "-s") || !strcmp(arg, "--stage")) {
- show_stage = 1;
- continue;
- }
- if (!strcmp(arg, "-k") || !strcmp(arg, "--killed")) {
- show_killed = 1;
- require_work_tree = 1;
- continue;
- }
- if (!strcmp(arg, "--directory")) {
- dir.show_other_directories = 1;
- continue;
- }
- if (!strcmp(arg, "--no-empty-directory")) {
- dir.hide_empty_directories = 1;
- continue;
- }
- if (!strcmp(arg, "-u") || !strcmp(arg, "--unmerged")) {
- /* There's no point in showing unmerged unless
- * you also show the stage information.
- */
- show_stage = 1;
- show_unmerged = 1;
- continue;
- }
- if (!strcmp(arg, "-x") && i+1 < argc) {
- exc_given = 1;
- add_exclude(argv[++i], "", 0, &dir.exclude_list[EXC_CMDL]);
- continue;
- }
- if (!prefixcmp(arg, "--exclude=")) {
- exc_given = 1;
- add_exclude(arg+10, "", 0, &dir.exclude_list[EXC_CMDL]);
- continue;
- }
- if (!strcmp(arg, "-X") && i+1 < argc) {
- exc_given = 1;
- add_excludes_from_file(&dir, argv[++i]);
- continue;
- }
- if (!prefixcmp(arg, "--exclude-from=")) {
- exc_given = 1;
- add_excludes_from_file(&dir, arg+15);
- continue;
- }
- if (!prefixcmp(arg, "--exclude-per-directory=")) {
- exc_given = 1;
- dir.exclude_per_dir = arg + 24;
- continue;
- }
- if (!strcmp(arg, "--exclude-standard")) {
- exc_given = 1;
- setup_standard_excludes(&dir);
- continue;
- }
- if (!strcmp(arg, "--full-name")) {
- prefix_offset = 0;
- continue;
- }
- if (!strcmp(arg, "--error-unmatch")) {
- error_unmatch = 1;
- continue;
- }
- if (!prefixcmp(arg, "--with-tree=")) {
- with_tree = arg + 12;
- continue;
- }
- if (!prefixcmp(arg, "--abbrev=")) {
- abbrev = strtoul(arg+9, NULL, 10);
- if (abbrev && abbrev < MINIMUM_ABBREV)
- abbrev = MINIMUM_ABBREV;
- else if (abbrev > 40)
- abbrev = 40;
- continue;
- }
- if (!strcmp(arg, "--abbrev")) {
- abbrev = DEFAULT_ABBREV;
- continue;
- }
- if (*arg == '-')
- usage(ls_files_usage);
- break;
+ argc = parse_options(argc, argv, builtin_ls_files_options,
+ ls_files_usage, 0);
+ if (show_tag || show_valid_bit) {
+ tag_cached = "H ";
+ tag_unmerged = "M ";
+ tag_removed = "R ";
+ tag_modified = "C ";
+ tag_other = "? ";
+ tag_killed = "K ";
}
+ if (show_modified || show_others || dir.show_ignored || show_killed)
+ require_work_tree = 1;
+ if (show_unmerged)
+ /* There's no point in showing unmerged unless
+ * you also show the stage information.
+ */
+ show_stage = 1;
+ if (dir.exclude_per_dir)
+ exc_given = 1;
if (require_work_tree && !is_inside_work_tree())
setup_work_tree();
- pathspec = get_pathspec(prefix, argv + i);
+ pathspec = get_pathspec(prefix, argv);
/* Verify that the pathspec matches the prefix */
if (pathspec)
--
1.6.1
^ permalink raw reply related
* Re: [PATCH/RFC] Allow writing loose objects that are corrupted in a pack file
From: R. Tyler Ballance @ 2009-01-07 2:47 UTC (permalink / raw)
To: Nicolas Pitre; +Cc: Jan Krüger, Git ML
In-Reply-To: <alpine.LFD.2.00.0901062059230.26118@xanadu.home>
[-- Attachment #1: Type: text/plain, Size: 751 bytes --]
On Tue, 2009-01-06 at 21:09 -0500, Nicolas Pitre wrote:
> > I've tarred one of the repositories that had it in a reproducible
> state
>
> That is wonderful.
>
> > so I can create a build and extract the tar and run against that to
> > verify any patches anybody might have, but unfortunately at 7GB of
> > company code and assets, I can't exactly share ;)
>
> First step is to understand what is going on. Only then could reliable
> patches be made.
If you want to point me in the right direction, I have a few hours to
kill this evening and fscking around with gdb(1) and printf() just might
be some of my favorite things</sarcasm> ;)
Looking forward to killing this issue
Cheers
--
-R. Tyler Ballance
Slide, Inc.
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 197 bytes --]
^ permalink raw reply
* Re: [PATCH/RFC] Allow writing loose objects that are corrupted in a pack file
From: Nicolas Pitre @ 2009-01-07 2:09 UTC (permalink / raw)
To: R. Tyler Ballance; +Cc: Jan Krüger, Git ML
In-Reply-To: <1231292360.8870.61.camel@starfruit>
[-- Attachment #1: Type: TEXT/PLAIN, Size: 2165 bytes --]
On Tue, 6 Jan 2009, R. Tyler Ballance wrote:
> On Tue, 2009-01-06 at 20:25 -0500, Nicolas Pitre wrote:
> > On Tue, 6 Jan 2009, R. Tyler Ballance wrote:
> >
> > > On Tue, 2008-12-09 at 09:36 +0100, Jan Krüger wrote:
> > > > For fixing a corrupted repository by using backup copies of individual
> > > > files, allow write_sha1_file() to write loose files even if the object
> > > > already exists in a pack file, but only if the existing entry is marked
> > > > as corrupted.
> > >
> > > I figured I'd reply to this again, since the issue cropped up again.
> > >
> > > We started experiencing *large* numbers of corruptions like the ones
> > > that started the thread (one developer was receiving them once or twice
> > > a day) with v1.6.0.4
> > >
> > > We went ahead and upgraded to a custom build of v1.6.1 with Jan's patch
> > > (below) and the issues /seem/ to have resolved themselves. I'm not
> > > certain whether Jan's patch was really responsible, or if there was
> > > another issue that caused this to correct itself in v1.6.1.
>
> I'll back the patch out and redeploy, it's worth mentioning that a
> coworker of mine just got the issue as well (on 1.6.1). He was able to
> `git pull` and the error went away, but I doubt that it "magically fixed
> itself"
Please describe the "issue", ideally with transcripts of error messages,
etc. Normally a simple pull operation should not provide any "fix" for
corruptions.
> I highly doubt this, I've got the issue appearing on at least 7
> different development boxes (not workstations, 2U quad-core ECC RAM, etc
> machines), while that doesn't mean that they all don't have issues, the
> probability of them *all* having disk issues, and it somehow only
> manifesting itself with Git usage, is low ;)
Agreed.
> I've tarred one of the repositories that had it in a reproducible state
That is wonderful.
> so I can create a build and extract the tar and run against that to
> verify any patches anybody might have, but unfortunately at 7GB of
> company code and assets, I can't exactly share ;)
First step is to understand what is going on. Only then could reliable
patches be made.
Nicolas
^ permalink raw reply
* Re: [PATCH v2] Add -ftabstop=WIDTH
From: Junio C Hamano @ 2009-01-07 1:52 UTC (permalink / raw)
To: Christopher Li; +Cc: Alexey Zaytsev, git
In-Reply-To: <70318cbf0901061637l29837d14nfaa8a3106652b7e5@mail.gmail.com>
"Christopher Li" <sparse@chrisli.org> writes:
> So here is my understanding of what you described. The 'pu' branch is
> for highly experiment changes. The 'pu' branch can rewind and rewrite
> the history. Once the patch merge to 'next', the history will not change
> any more. All update will stay as incremental changes.
>
> One question, does user suffer from conflict when then pull from the 'pu'
> branch?
[jc: I think this is going to the tangent for "sparse" list;
redirecting to git@vger.kernel.org]
I think they will, if they "pull", but:
(1) They are upfront strongly discouraged from doing so by the way 'pu'
is advertised. "It is a collection of not yet even testable series,
and any patch in it can be dropped and replaced".
(2) They can instead 'fetch + rebase' the changes they made on top of
previous round of 'pu', instead of 'pull' (= 'fetch + merge') to
mitigate the pain.
Suppose I have two un-ready topics A and B in pu, and you base your work
X, Y, and Z on what was done by A (in other words, you are not interested
in topic B at all). Then suppose one of A or B is replaced by wildly
different versions, and 'pu' is rebuilt:
X---Y---Z (private changes)
/
A----B pu (old)
/
/ A'---B' pu (new)
/ /
----o----o----o----o
Fig. 1
If you pull, even if A was not the one that was replaced, the merge will
have severe conflicts from the changes involved in the other series
(i.e. B).
But if A and A' did not change drastically in the meantime, rebasing X, Y,
Z on top of the updated pu (i.e. B') would not conflict:
X---Y---Z (private changes)
/
A----B pu (old) X'--Y'--Z' (private changes rebased)
/ /
/ A'---B' pu (new)
/ /
----o----o----o----o
Fig. 2
In either case, if A (i.e. the work X, Y, Z were made on top of) was
rewritten drastically to become A', neither rebase nor merge will be of
help anyway, and it would not help if the new A' were recorded as an
incremental change from A without rebasing/rewinding 'pu' itself, either.
But at least 'fetch + rebase' would avoid the issue when it is only the
other topics in 'pu' that you are not interested in that were replaced or
rewritten drastically.
By the way, I drew A and B as if they are single patches made _directly_
on pu, only for simplicity's sake. In reality, all topics fork from more
stable branches (maint or master), and the only commits you see on 'next'
or 'pu' are merges.
Which means, even if we assume that you never rewind 'pu':
> Here is an idea, I am just thinking it out loud.
>
> Given 'pu' branch like this, (each [ ] is a commit, A1 is a follow up
> change for A0).
>
> 'pu' branch: [A0] - [B0] - [A1] - [C0] -[B1] -[A2]
... the history of 'pu' won't look like this.
It would be more like this:
.-----[B0]----[B1] ... topic branch for B
/ \ \
/ ...--*---*---*---* ... pu
/ / /
/ [A0]---[A1] ... topic branch for A
/ /
o-------o----o master
Fig. 3
Side note: my 'next' never rewinds except for once every major
release, so the above "repeated merge from topics into the branch"
depicts how 'next' works pretty closely.
Or, if you rebuild 'pu' every day, it would be more like
this one day, and;
.-----[B0] ... topic branch for B
/ \
/ ...--*---* ... pu
/ /
/ [A0] ... topic branch for A
/ /
o-------o----o master
Fig. 4
the next day it would look like this:
.-----[B0]----[B1] ... topic branch for B
/ \
/ ...--*---* ... pu
/ /
/ [A0]---[A1] ... topic branch for A
/ /
o-------o----o master
Fig. 5
In either case, unless a topic began with too many early issues and
mistakes that requires a wholesale replacement, you can expect the
accumulation of A0,A1,...,An to end up in a good shape eventually and then
you have a good incremental history you would want to preserve.
At that point, you can merge the tip of the branch (i.e. An) to master and
declare victory. 'pu' or 'next' may have a messy history that would make
anybody who looks at gitk output barf, but that is Ok.
> We can have a temporary clean up branch fork from 'pu' looks like this:
>
> 'tmp_clean' branch: [A0 + A1 + A2] - [B0 + B1] - [C0]
>
> 'tmp_clean' and 'pu' will generate exactly the same tree. The
> only different is the history path it take to get there.
>
> Then we can have 'pu' merge from 'tmp_clean', with zero text
> changes. The only change is the change log and we tell git
> that the merge is for history clean up. So when we launch
> "git log", by default it will follow the "tmp_clean" path rather
> than the original "pu" path.
>
> So it just provide "alternative" view of the history without introduce
> real changes. When user pull from 'pu', it can automatically get the
> cleanup version of the history without introduce conflicts.
>
> It seems it can have the best of both worlds. I am not sure weather
> it is doable or worth while to do though.
I do not think it is worth it, for two reasons:
(1) That won't help the case where others based on their work on un-ready
changes in 'pu', as I described earlier, anyway.
(2) If you do not have any work on top of the un-ready 'pu', in other
words, if you are just following along, then "git checkout origin/pu"
won't care if yesterday's pu and today's pu are not fast-forward
anyway.
If you rebuild 'pu' from scratch every day, without keeping many repeated
merges so far, it will give a pleasant read in "gitk master..pu" than
'next' that never rewinds whose "gitk master..next" output is a disaster
;-).
There is one trick my experienced users use, knowing how 'pu' is managed.
If today's 'pu' looked like Fig. 4 above, and you are interested in the
topic A, you can find the tip of that topic by looking at:
git log --first-parent master..pu
It is what was merged to the merge that is at the second from the tip of
'pu' branch, i.e. "pu^^2 == A0".
And you fork your own enhancement to that topic by forking from A0,
creating "my-A" branch. Your own commits go to that branch.
Next day you will find a history that is depicted in Fig. 5 and find the
tip of topic A the same way. It is at A1.
Then you rebase "my-A" on top of A1 (or merge A1 to "my-A" branch). You
really do not care about other uncooked garbage in 'pu', and you can
ignore them this way.
If you are working on more than one such "topics started by others", you
will have many my-A, my-B, ... branches. You treat your 'master' branch
as if it is my 'next', i.e. fork from the last major release, merging all
of my-X branches, and employ the aggregated result for your own use.
^ permalink raw reply
* Re: [PATCH/RFC] Allow writing loose objects that are corrupted in a pack file
From: R. Tyler Ballance @ 2009-01-07 1:39 UTC (permalink / raw)
To: Nicolas Pitre; +Cc: Jan Krüger, Git ML
In-Reply-To: <alpine.LFD.2.00.0901062005290.26118@xanadu.home>
[-- Attachment #1: Type: text/plain, Size: 3010 bytes --]
On Tue, 2009-01-06 at 20:25 -0500, Nicolas Pitre wrote:
> On Tue, 6 Jan 2009, R. Tyler Ballance wrote:
>
> > On Tue, 2008-12-09 at 09:36 +0100, Jan Krüger wrote:
> > > For fixing a corrupted repository by using backup copies of individual
> > > files, allow write_sha1_file() to write loose files even if the object
> > > already exists in a pack file, but only if the existing entry is marked
> > > as corrupted.
> >
> > I figured I'd reply to this again, since the issue cropped up again.
> >
> > We started experiencing *large* numbers of corruptions like the ones
> > that started the thread (one developer was receiving them once or twice
> > a day) with v1.6.0.4
> >
> > We went ahead and upgraded to a custom build of v1.6.1 with Jan's patch
> > (below) and the issues /seem/ to have resolved themselves. I'm not
> > certain whether Jan's patch was really responsible, or if there was
> > another issue that caused this to correct itself in v1.6.1.
I'll back the patch out and redeploy, it's worth mentioning that a
coworker of mine just got the issue as well (on 1.6.1). He was able to
`git pull` and the error went away, but I doubt that it "magically fixed
itself"
> Please back it out. As it stands, that patch is a no op because of the
> way git is used, and even if the patch was to work as intended, its
> purpose is not to magically fix corruptions without special action from
> your part. If you have corruption problems coming back only because of
> the removal of this patch then something is really really fishy and I
> would really like to know about it.
>
> There were indeed many changes between v1.6.0.4 and v1.6.1: the exact
> number is 1029. A couple of them are especially addressing increased
> robustness against some kind of pack corruptions. But in any case you
> still should see error messages appearing about them.
>
> And don't underestimate the power of disk corruptions. I started to
> work on git corruption resilience simply because I ended up with a
> corrupted pack at some point. Then a while later I got another
> corrupted pack. Then another while later I lost my filesystem entirely
> and had to reinstall my system (after buying a new disk). Turns out
> that my old disk is silently corrupting data without signaling any
> errors to the host.
I highly doubt this, I've got the issue appearing on at least 7
different development boxes (not workstations, 2U quad-core ECC RAM, etc
machines), while that doesn't mean that they all don't have issues, the
probability of them *all* having disk issues, and it somehow only
manifesting itself with Git usage, is low ;)
I've tarred one of the repositories that had it in a reproducible state
so I can create a build and extract the tar and run against that to
verify any patches anybody might have, but unfortunately at 7GB of
company code and assets, I can't exactly share ;)
Cheers
--
-R. Tyler Ballance
Slide, Inc.
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 197 bytes --]
^ permalink raw reply
* Re: [PATCH/RFC] Allow writing loose objects that are corrupted in a pack file
From: Nicolas Pitre @ 2009-01-07 1:25 UTC (permalink / raw)
To: R. Tyler Ballance; +Cc: Jan Krüger, Git ML
In-Reply-To: <1231282320.8870.52.camel@starfruit>
[-- Attachment #1: Type: TEXT/PLAIN, Size: 2646 bytes --]
On Tue, 6 Jan 2009, R. Tyler Ballance wrote:
> On Tue, 2008-12-09 at 09:36 +0100, Jan Krüger wrote:
> > For fixing a corrupted repository by using backup copies of individual
> > files, allow write_sha1_file() to write loose files even if the object
> > already exists in a pack file, but only if the existing entry is marked
> > as corrupted.
>
> I figured I'd reply to this again, since the issue cropped up again.
>
> We started experiencing *large* numbers of corruptions like the ones
> that started the thread (one developer was receiving them once or twice
> a day) with v1.6.0.4
>
> We went ahead and upgraded to a custom build of v1.6.1 with Jan's patch
> (below) and the issues /seem/ to have resolved themselves. I'm not
> certain whether Jan's patch was really responsible, or if there was
> another issue that caused this to correct itself in v1.6.1.
>
> As it stands, I think it's safe to assume that given the frequency of
> the occurances that they were not tied to a memory or disk error (or
> other levels of the machine's stack would be suffering as well). The
> only thing I can think of is that /some/ developers who've experienced
> the issue are using Samba mount points and changing files in Mac OS X,
> but using Git on the mounted share (i.e. TextMate changes a file hosted
> on Samba, changes are committed in an SSH session on that machine), but
> that doesn't account for everything.
>
> If there was something else included in the v1.6.1 release please let me
> know so I can back Jan's patch out.
Please back it out. As it stands, that patch is a no op because of the
way git is used, and even if the patch was to work as intended, its
purpose is not to magically fix corruptions without special action from
your part. If you have corruption problems coming back only because of
the removal of this patch then something is really really fishy and I
would really like to know about it.
There were indeed many changes between v1.6.0.4 and v1.6.1: the exact
number is 1029. A couple of them are especially addressing increased
robustness against some kind of pack corruptions. But in any case you
still should see error messages appearing about them.
And don't underestimate the power of disk corruptions. I started to
work on git corruption resilience simply because I ended up with a
corrupted pack at some point. Then a while later I got another
corrupted pack. Then another while later I lost my filesystem entirely
and had to reinstall my system (after buying a new disk). Turns out
that my old disk is silently corrupting data without signaling any
errors to the host.
Nicolas
^ permalink raw reply
* Re: Problems getting rid of large files using git-filter-branch
From: Nicolas Pitre @ 2009-01-07 0:56 UTC (permalink / raw)
To: Stephen R. van den Berg; +Cc: ?yvind Harboe, git
In-Reply-To: <20090106231726.GB13379@cuci.nl>
On Wed, 7 Jan 2009, Stephen R. van den Berg wrote:
> Nicolas Pitre wrote:
> >On Tue, 6 Jan 2009, ?yvind Harboe wrote:
> >OK, try this:
>
> > git pull file://$(pwd)/../my_repo.orig
>
> Alternately, try:
>
> rm -rf .git/ORIG_HEAD .git/FETCH_HEAD .git/index .git/logs .git/info/refs \
> .git/objects/pack/pack-*.keep .git/refs/original .git/refs/patches \
> .git/patches .git/gitk.cache &&
> git prune --expire now &&
> git repack -a -d --window=200 &&
> git gc
This might not be sufficient. Or at least you better run 'git prune' at
the very end, and possibly add -f to 'git repack'. And if you somehow
delete something you shouldn't have deleted then you're really screwed,
whereas the pull method in another repository doesn't alter the original
repository in case you need to go back to it and try something
different.
Nicolas
^ permalink raw reply
* Re: [RFC PATCH] diff --no-index: test for pager after option parsing
From: Junio C Hamano @ 2009-01-07 0:09 UTC (permalink / raw)
To: Thomas Rast; +Cc: git
In-Reply-To: <1231286163-9422-1-git-send-email-trast@student.ethz.ch>
Thomas Rast <trast@student.ethz.ch> writes:
> I noticed this while working on the earlier patch for diff --no-index.
> It seems like the right thing to do (and passes tests), but I don't
> have a clue about git's normal setup sequences, so I'm flagging it
> RFC.
I think the patch itself makes sense from the logic flow point of view.
But I wonder if it still makes a difference in real life.idn't we stop
reporting the exit status from the pager some time ago?
^ permalink raw reply
* Re: [RFC PATCH] diff --no-index: test for pager after option parsing
From: Thomas Rast @ 2009-01-07 0:09 UTC (permalink / raw)
To: git; +Cc: Junio C Hamano
In-Reply-To: <1231286163-9422-1-git-send-email-trast@student.ethz.ch>
[-- Attachment #1: Type: text/plain, Size: 859 bytes --]
Speaking of diff --no-index code, there's also this bit:
/* diff-no-index.c:206 */
for (i = 1; i < argc - 2; ) {
int j;
if (!strcmp(argv[i], "--no-index"))
i++;
else if (!strcmp(argv[1], "-q"))
options |= DIFF_SILENT_ON_REMOVED;
Note the argv[i] vs. argv[1]. The entire block is from 0569e9b ("git
diff": do not ignore index without --no-index, 2008-05-23).
While it seems obvious that this should be argv[i], I'm rather
confused by the option itself. It is not documented in my version of
git-diff(1). Furthermore, I can't see what being silent about removed
paths (which relates to the index?) has to do with a diff --no-index
(which takes two paths that must exist).
Or perhaps I should take the "no mails/patches after midnight" rule a
tad bit more serious...
--
Thomas Rast
trast@{inf,student}.ethz.ch
[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 197 bytes --]
^ permalink raw reply
* [RFC PATCH] diff --no-index: test for pager after option parsing
From: Thomas Rast @ 2009-01-06 23:56 UTC (permalink / raw)
To: git; +Cc: Junio C Hamano
We need to parse options before we can see if --exit-code was
provided.
Signed-off-by: Thomas Rast <trast@student.ethz.ch>
---
I noticed this while working on the earlier patch for diff --no-index.
It seems like the right thing to do (and passes tests), but I don't
have a clue about git's normal setup sequences, so I'm flagging it
RFC.
diff-no-index.c | 14 +++++++-------
1 files changed, 7 insertions(+), 7 deletions(-)
diff --git a/diff-no-index.c b/diff-no-index.c
index b60d345..f655f64 100644
--- a/diff-no-index.c
+++ b/diff-no-index.c
@@ -198,13 +198,6 @@ void diff_no_index(struct rev_info *revs,
die("git diff %s takes two paths",
no_index ? "--no-index" : "[--no-index]");
- /*
- * If the user asked for our exit code then don't start a
- * pager or we would end up reporting its exit code instead.
- */
- if (!DIFF_OPT_TST(&revs->diffopt, EXIT_WITH_STATUS))
- setup_pager();
-
diff_setup(&revs->diffopt);
if (!revs->diffopt.output_format)
revs->diffopt.output_format = DIFF_FORMAT_PATCH;
@@ -222,6 +215,13 @@ void diff_no_index(struct rev_info *revs,
}
}
+ /*
+ * If the user asked for our exit code then don't start a
+ * pager or we would end up reporting its exit code instead.
+ */
+ if (!DIFF_OPT_TST(&revs->diffopt, EXIT_WITH_STATUS))
+ setup_pager();
+
if (prefix) {
int len = strlen(prefix);
--
tg: (e9b8523..) t/diff-no-index-status (depends on: origin/master)
^ permalink raw reply related
* Re: Problems getting rid of large files using git-filter-branch
From: Stephen R. van den Berg @ 2009-01-06 23:17 UTC (permalink / raw)
To: Nicolas Pitre; +Cc: ?yvind Harboe, git
In-Reply-To: <alpine.LFD.2.00.0901061709510.26118@xanadu.home>
Nicolas Pitre wrote:
>On Tue, 6 Jan 2009, ?yvind Harboe wrote:
>OK, try this:
> git pull file://$(pwd)/../my_repo.orig
Alternately, try:
rm -rf .git/ORIG_HEAD .git/FETCH_HEAD .git/index .git/logs .git/info/refs \
.git/objects/pack/pack-*.keep .git/refs/original .git/refs/patches \
.git/patches .git/gitk.cache &&
git prune --expire now &&
git repack -a -d --window=200 &&
git gc
--
Sincerely,
Stephen R. van den Berg.
"Very funny, Mr. Scott. Now beam down my clothes!"
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox