* Re: Tracking a repository for content instead of history
From: Andy Parkins @ 2006-12-12 13:26 UTC (permalink / raw)
To: git
In-Reply-To: <elm993$rf1$1@sea.gmane.org>
On Tuesday 2006 December 12 13:04, Jakub Narebski wrote:
> > Is it even sensible to want that? It strikes me that it's possible that
> > there isn't that much space/bandwidth saving to be made. Should I just
> > clone the repository and shut up? :-)
>
> I've had similar idea: search for "sparse clone" keyword. But no code.
While the functionality might not be built into git in terms of clone, would
there be a way to pull a particular commit from another repository?
The way I would do it given nothing else is to simply extract snapshots into a
working directory; and create a repository from scratch. I was just
wondering if a method existed that could reduce the size of the download.
I think the best way is going to be to use the patches published at kernel.org
and apply them one at a time with git-apply.
Andy
--
Dr Andy Parkins, M Eng (hons), MIEE
^ permalink raw reply
* Re: Using GIT to store /etc (Or: How to make GIT store all file permission bits)
From: Kyle Moffett @ 2006-12-12 13:49 UTC (permalink / raw)
To: Daniel Barkalow; +Cc: git
In-Reply-To: <Pine.LNX.4.64.0612111837210.20138@iabervon.org>
On Dec 11, 2006, at 22:45:25, Daniel Barkalow wrote:
> The first thing you'd want to do is correct the fact that the index
> doesn't keep full permissions. We decided long ago that we don't
> want to track more than 0100, but we're discarding the rest between
> the filesystem and the index, rather than between the index and the
> tree. (This is weird of us, since we keep gid and uid in the index,
> as changedness heuristics, but don't keep permissions; of course,
> we'd have to apply umask to the index when we check it out to sync
> what we expect to be there with what has actually been created.)
>
> I think that would be the only change needed to the index and index/
> working directory connection, although it might be necessary to
> support longer values for uid/gid/etc, since they'd be important
> data now.
Hmm, ok. It would seem to be a reasonable requirement that if you
want to change any of the "preserve_*_attributes" config options you
need to blow away and recreate your index, no? I would probably
change the underlying index format pretty completely and stick a new
version tag inside it.
> Note that git only stores content, not incidental information. But
> a lot of information which is incidental in a source tree is
> content in /etc. This implies that /etc and working/linux-2.6 are
> fundamentally different sorts of things, because different aspects
> of them are content.
Ahh, I hadn't thought of it that way before but that makes a lot of
sense. Thanks!
> I'd suggest a new object type for a directory with permissions,
> ACLs, and so forth. It should probably use symbolic owner and
> group, too. My guess is that you'll want to use "commit"s, the new
> object type, and "blob"s. Everything that uses trees would need to
> have a version that uses the new type. But I think that you
> generally want different behavior anyway, so that's not a major issue.
Ok, seems straightforward enough. One other thing that crossed my
mind was figuring out how to handle hardlinks. The simplest solution
would be to add an extra layer of indirection between the "file
inode" and the "file data". Instead of your directory pointing to a
"file-data" blob and "file-attributes" object, it would point to an
"file-inode" object with embedded attribute data and a pointer to the
file contents blob.
I remember reading some discussions from the early days of GIT about
how that was considered and discarded because the extra overhead
wouldn't give any real tangible benefit. On the other hand for
something like /etc the added benefits of tracking extended
attributes and hardlinks might outweigh the cost of a bunch of extra
objects in the database. A bit of care with the construction of the
index file should make it sufficiently efficient for day-to-day usage.
If you're interested in some random musings about using GIT concepts
to version whole filesystems (think checkpointing your disk drive and
instantly restoring when you screw up), read on below, otherwise
don't bother.
Cheers,
Kyle Moffett
<Random Tangential Off-the-Wall Thought Experiment>
NOTE: This probably belongs in it's own thread but it's such a
random, undeveloped, and off-the-wall concept that I threw it in here
just for kicks.
Combining extensions like those described above with something like
the Ext3 block-allocation, inode-management and journalling code to
produce a "versioned filesystem". With the exponential growth of
storage density over the last several years we've gotten to the point
where we can many many hours of extremely realistic video and audio
on your average small-computer drive. Versioning your home
directory, or even your entire computer, even with fairly steady
modifications to multimedia files, installation of software programs,
etc, doesn't seem like such an impossible undertaking anymore.
One predefined inode would contain a list of tags/heads and their
current hashes. Mount the filesystem with a "tag=$TAG" option to
specify the initial tree object used for the root directory (with
syscalls to navigate the history). Allocate an inode per-mount to
represent any changes from the last commit.
For efficiency purposes (no need to revision the entire system when I
commit a change in my home directory) add a "subtree" object type
which can specify either a particular hash or a symbolic tag/head
name as a pseudo sub-mountpoint. Trap traversal of the sub-
mountpoint node to mount the filesystem with "tag=$SUBTAG" on the sub-
mountpoint, expiring it some time after the last traversal.
The only remaining issue would be properly navigating through the
history, preserving or discarding changes. Since the kernel could
easily manage copy-on-write semantics for underlying disk blocks you
wouldn't need a separate "working copy" except where it's modified
from the original, and discarding changes is as simple as unlinking
any files referenced by the per-mount delta inode.
Committing changes would get tricky, you would need to hot-remap
memory-mapped pages read-only while you checksum and store them. The
next write attempt would then separate the page from the freshly-
committed on-disk version. Would need a mechanism for applications
to "trap" the commit so they could make databases consistent, with
the ability for root or the mountpoint owner to commit without
waiting for synchronization. Only needs to synchronize files
belonging to the new commit. Merges would be managed from userspace,
as long as there is a way to browse through objects by hash given
sufficient permissions.
Make sure it's really easy to make a new atomic commit and/or reset
to a known state every time the computer is rebooted (whether soft-
rebooted or via crash/powerkill). With journalling and the write-
once nature of GIT it would be trivial to never require an fsck run.
Also needs a way to move data between filesystems. Makes LVM largely
irrelevant; it doesn't matter how many disks you have if they're all
treated as a shared storage pool for your GITfs data. Make sure it's
possible to archive data onto slower disks/media and purge older
commits from the archive (missing parent commit references are
tolerable in many situations). Needs a way to notice hash collisions
and take action to avoid them.
</Random Tangential Off-the-Wall Thought Experiment>
Cheers,
Kyle Moffett
^ permalink raw reply
* Re: [PATCH] Uninstall rule for top level Makefile
From: Johannes Schindelin @ 2006-12-12 14:15 UTC (permalink / raw)
To: Andy Parkins; +Cc: git
In-Reply-To: <200612121251.33051.andyparkins@gmail.com>
Hi,
On Tue, 12 Dec 2006, Andy Parkins wrote:
> I assume I've missed something in the linked thread - what was it you
> wanted me to pick up from it?
The uninstall target _only_ solves the case you mentioned: when you
installed Git in the wrong place. But then it is fragile: if you put the
new path into config.mak, or you provided the PREFIX on the command line,
and possibly do not remember what it was, the uninstall goes wrong.
But then, I do not care at all about that target being in the Makefile or
not. I'll just ignore it, just as almost everybody else.
Ciao,
Dscho
^ permalink raw reply
* Re: Tracking a repository for content instead of history
From: Johannes Schindelin @ 2006-12-12 14:28 UTC (permalink / raw)
To: Andy Parkins; +Cc: git
In-Reply-To: <200612121326.24508.andyparkins@gmail.com>
Hi,
On Tue, 12 Dec 2006, Andy Parkins wrote:
> The way I would do it given nothing else is to simply extract snapshots
> into a working directory; and create a repository from scratch. I was
> just wondering if a method existed that could reduce the size of the
> download.
You are not by any chance talking about the --remote option to
git-archive?
If you want to reduce the number of objects to be downloaded, by telling
the other side what you have, you literally end up with something like
shallow clone: the other side _has_ to support it.
Ciao,
Dscho
^ permalink raw reply
* Re: Tracking a repository for content instead of history
From: Andy Parkins @ 2006-12-12 15:38 UTC (permalink / raw)
To: git
In-Reply-To: <Pine.LNX.4.63.0612121527070.2807@wbgn013.biozentrum.uni-wuerzburg.de>
On Tuesday 2006 December 12 14:28, Johannes Schindelin wrote:
> You are not by any chance talking about the --remote option to
> git-archive?
I wasn't; but that's certainly a helpful switch. It's certainly a huge help.
> If you want to reduce the number of objects to be downloaded, by telling
> the other side what you have, you literally end up with something like
> shallow clone: the other side _has_ to support it.
I suppose so; but I was thinking more an automated way of getting the data
that is supplied for the kernel anyway. So:
base-v1.0.0.tar.gz
patch-v1.0.1.gz
patch-v1.0.2.gz
etc
Each patch is obviously smaller than "base". Git could easily make the
patches, and each of those patches could be fed by hand into a repository
with git-apply. It doesn't seem like something that would require support on
the other side, because it isn't so much a shallow clone (which /would/
preserve history, making it available if wanted); it is pulling just, say,
tagged commits out of an existing repository.
Given a list of tags it is almost:
git-archive <get me base>
ssh remote git-diff v1.0.0..v1.0.1 | git-apply; git commit
ssh remote git-diff v1.0.1..v1.0.2 | git-apply; git commit
If that makes sense? Obviously though it would be possible to use git rather
than ssh to do this.
However... please don't waste any more time thinking about this; it's not a
problem I have that needs a solution - it was more a "because I'm curious"
sort of question.
Andy
--
Dr Andy Parkins, M Eng (hons), MIEE
^ permalink raw reply
* git-send-email and msmtp
From: Aneesh Kumar @ 2006-12-12 15:37 UTC (permalink / raw)
To: Git Mailing List
Hi,
I am using msmtp to send email using git-send-email. The problem i am
facing now is vger.kernel.org is dropping the mails generated by
git-send-email. When i am sending the same patch as an attachment in
thunderbird everything works fine. So i guess it is not the patch
content that is getting filtered. Any idea how to find out what is
causing the patch to get dropped and how to fix this.
The header of the patch that was dropped is
Return-Path: <aneesh.kumar@gmail.com>
Received: from localhost ( [156.153.255.234])
by mx.google.com with ESMTP id q13sm7841591qbq.2006.12.12.00.56.10;
Tue, 12 Dec 2006 00:56:12 -0800 (PST)
From: "Aneesh Kumar K.V" <aneesh.kumar@gmail.com>
To: linux-kernel@vger.kernel.org
Cc: trs80@ucc.gu.uwa.edu.au, Aneesh Kumar K.V <aneesh.kumar@gmail.com>
Subject: [PATCH] Alpha increase PERCPU_ENOUGH_ROOM
Date: Tue, 12 Dec 2006 14:26:06 +0530
Message-Id: <11659137661938-git-send-email-aneesh.kumar@gmail.com>
X-Mailer: git-send-email 1.4.4.2.gdb98-dirty
Header of the patch that went through fine is below
Return-Path: <aneesh.kumar@gmail.com>
Received: from ?217.236.218.183? ( [156.153.255.234])
by mx.google.com with ESMTP id e18sm7940070qba.2006.12.12.01.29.53;
Tue, 12 Dec 2006 01:29:55 -0800 (PST)
Message-ID: <457E768F.6050606@gmail.com>
Date: Tue, 12 Dec 2006 14:59:51 +0530
From: "Aneesh Kumar K.V" <aneesh.kumar@gmail.com>
User-Agent: Thunderbird 1.5.0.8 (X11/20061115)
MIME-Version: 1.0
To: linux-kernel@vger.kernel.org
Subject: [PATCH] Alpha increase PERCPU_ENOUGH_ROOM
Content-Type: multipart/mixed;
boundary="------------040307060102020004050000"
This is a multi-part message in MIME format.
--------------040307060102020004050000
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
--------------040307060102020004050000
Content-Type: text/plain;
name="0002-Alpha-increase-PERCPU_ENOUGH_ROOM.txt"
Content-Transfer-Encoding: 7bit
Content-Disposition: inline;
filename="0002-Alpha-increase-PERCPU_ENOUGH_ROOM.txt"
^ permalink raw reply
* Re: [PATCH] Uninstall rule for top level Makefile
From: Andy Parkins @ 2006-12-12 15:42 UTC (permalink / raw)
To: git
In-Reply-To: <Pine.LNX.4.63.0612121512060.2807@wbgn013.biozentrum.uni-wuerzburg.de>
On Tuesday 2006 December 12 14:15, Johannes Schindelin wrote:
> The uninstall target _only_ solves the case you mentioned: when you
> installed Git in the wrong place. But then it is fragile: if you put the
> new path into config.mak, or you provided the PREFIX on the command line,
> and possibly do not remember what it was, the uninstall goes wrong.
Absolutely. I agree entirely. The only reason I raised it was because I had
cause to need it. The quickest way I found was just to write the missing
recipe. As I'd written it, I thought I'd share. As you say, it has no other
use than unpicking a stupid "install", done by a butterbrain like myself :-)
> But then, I do not care at all about that target being in the Makefile or
> not. I'll just ignore it, just as almost everybody else.
Until I needed it; I didn't care that it wasn't there. Even then "need" is
perhaps too strong a word. It was handy. That's it. If there are strong
moral objections to it; it doesn't bother me.
Andy
--
Dr Andy Parkins, M Eng (hons), MIEE
^ permalink raw reply
* Re: Using GIT to store /etc (Or: How to make GIT store all file permission bits)
From: Andy Parkins @ 2006-12-12 15:53 UTC (permalink / raw)
To: git
In-Reply-To: <8900B938-1360-4A67-AB15-C9E84255107B@mac.com>
On Tuesday 2006 December 12 13:49, Kyle Moffett wrote:
> Hmm, ok. It would seem to be a reasonable requirement that if you
> want to change any of the "preserve_*_attributes" config options you
> need to blow away and recreate your index, no? I would probably
> change the underlying index format pretty completely and stick a new
> version tag inside it.
I wonder if git's skill at managing content is the answer? Rather than mess
around with git's internals, the index, or the object database; how about
simply having a pre-commit script that writes out a file that looks like:
-rw-r--r-- andyp andyp CHANGES
-rw-r--r-- andyp andyp COPYING
-rw-rw-r-- andyp andyp CREDITS
-rw-r--r-- andyp andyp Configure
-rw-rw-r-- andyp andyp Makefile
-rw-r--r-- andyp andyp README
If /that/ file were stored in the repository and you had a script that could
read that file and apply the permissions after a checkout you'd have what you
want.
If the permissions of a file changed but the content didn't, then
this ".gitpermissions" file would have changed content but the file itself
would remain the same. If the content changed but not the permissions
then ".gitpermissions" would be untouched.
Assuming that you're allowed to mess with the index in pre-commit (I haven't
checked), one half of it can be automatic. I suppose you could also plead
for a post-checkout hook to apply those permissions and the whole lot would
be transparent.
Andy
--
Dr Andy Parkins, M Eng (hons), MIEE
^ permalink raw reply
* Re: git-send-email and msmtp
From: Jakub Narebski @ 2006-12-12 16:11 UTC (permalink / raw)
To: git
In-Reply-To: <cc723f590612120737t7d7e8dabs4e82ae2ac705e4e1@mail.gmail.com>
Aneesh Kumar wrote:
> I am using msmtp to send email using git-send-email. The problem i am
> facing now is vger.kernel.org is dropping the mails generated by
> git-send-email. When i am sending the same patch as an attachment in
> thunderbird everything works fine. So i guess it is not the patch
> content that is getting filtered. Any idea how to find out what is
> causing the patch to get dropped and how to fix this.
>
> The header of the patch that was dropped is
Probably problem with vger.kernel.org checking your server if it
is permitted to send mails. I use sendmail, and have solved this
using my gmail mail account as SMTP tunnel/forwarding using
define(`SMART_HOST',`[smtp.gmail.com]')
msmtp can do something like this. Ask chexum on IRC.
--
Jakub Narebski
Warsaw, Poland
ShadeHawk on #git
^ permalink raw reply
* Re: Tracking a repository for content instead of history
From: Johannes Schindelin @ 2006-12-12 16:24 UTC (permalink / raw)
To: Andy Parkins; +Cc: git
In-Reply-To: <200612121538.41197.andyparkins@gmail.com>
Hi,
On Tue, 12 Dec 2006, Andy Parkins wrote:
> On Tuesday 2006 December 12 14:28, Johannes Schindelin wrote:
>
> > You are not by any chance talking about the --remote option to
> > git-archive?
>
> I wasn't; but that's certainly a helpful switch. It's certainly a huge
> help.
>
> > If you want to reduce the number of objects to be downloaded, by telling
> > the other side what you have, you literally end up with something like
> > shallow clone: the other side _has_ to support it.
>
> I suppose so; but I was thinking more an automated way of getting the data
> that is supplied for the kernel anyway. So:
>
> base-v1.0.0.tar.gz
> patch-v1.0.1.gz
> patch-v1.0.2.gz
> etc
>
> Each patch is obviously smaller than "base". Git could easily make the
> patches, and each of those patches could be fed by hand into a repository
> with git-apply.
If it weren't for the recent discussion of kernel.org being overloaded
with gitweb processes, I'd just write down a hint like
http://repo.or.cz/w/git/jnareb-git.git?a=commitdiff_plain;h=next;hp=master
But since kernel.org is overloaded, I will not do that.
Ciao,
^ permalink raw reply
* Re: Tracking a repository for content instead of history
From: Johannes Schindelin @ 2006-12-12 16:35 UTC (permalink / raw)
To: Andy Parkins; +Cc: git
In-Reply-To: <Pine.LNX.4.63.0612121715230.2807@wbgn013.biozentrum.uni-wuerzburg.de>
Hi,
On Tue, 12 Dec 2006, Johannes Schindelin wrote:
> If it weren't for the recent discussion of kernel.org being overloaded
> with gitweb processes, I'd just write down a hint like
> [URL edited out]
>
> But since kernel.org is overloaded, I will not do that.
Side note: it would probably not help you. The diff is uncompressed, and
thus likely _substantially larger_ than getting the snapshot via gitweb,
which _is_ compressed.
Ciao,
Dscho
^ permalink raw reply
* git-pull from git.git - no remote ref for pu or next?
From: Randal L. Schwartz @ 2006-12-12 16:44 UTC (permalink / raw)
To: git
I just got this on this morning's git-fetch:
error: no such remote ref refs/heads/pu
error: no such remote ref refs/heads/next
Fetch failure: git://git.kernel.org/pub/scm/git/git.git
Here's my remotes/origin:
URL: git://git.kernel.org/pub/scm/git/git.git
Pull: master:origin
Pull: man:man
Pull: html:html
Pull: +pu:pu
Pull: +next:next
--
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
<merlyn@stonehenge.com> <URL:http://www.stonehenge.com/merlyn/>
Perl/Unix/security consulting, Technical writing, Comedy, etc. etc.
^ permalink raw reply
* Re: git-pull from git.git - no remote ref for pu or next?
From: Randal L. Schwartz @ 2006-12-12 16:47 UTC (permalink / raw)
To: git
In-Reply-To: <863b7l83o9.fsf@blue.stonehenge.com>
>>>>> "Randal" == Randal L Schwartz <merlyn@stonehenge.com> writes:
Randal> I just got this on this morning's git-fetch:
Randal> error: no such remote ref refs/heads/pu
Randal> error: no such remote ref refs/heads/next
Randal> Fetch failure: git://git.kernel.org/pub/scm/git/git.git
Randal> Here's my remotes/origin:
Randal> URL: git://git.kernel.org/pub/scm/git/git.git
Randal> Pull: master:origin
Randal> Pull: man:man
Randal> Pull: html:html
Randal> Pull: +pu:pu
Randal> Pull: +next:next
And then it mysteriously fixed itself a few minutes later.
Is there some sort of publishing failure, or intermittent race condition?
Or is this something unique to git.git?
Or just bad electron spin or something?
--
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
<merlyn@stonehenge.com> <URL:http://www.stonehenge.com/merlyn/>
Perl/Unix/security consulting, Technical writing, Comedy, etc. etc.
^ permalink raw reply
* [PATCH] gitweb: Sprinkle some mod_perl goodies
From: Jakub Narebski @ 2006-12-12 16:55 UTC (permalink / raw)
To: git; +Cc: Jakub Narebski
Add $r variable which holds Apache::RequestRec if script is run under
mod_perl (if $ENV{MOD_PERL} is defined). It is used as argument to
constructor of CGI object (needs CGI module version at least 2.93).
Following advice from CGI(3pm) man page, precompile all CGI routines
for mod_perl.
Use $r->path_info() instead of $ENV{"PATH_INFO"}.
All this makes gitweb slightly faster under mod_perl.
Signed-off-by: Jakub Narebski <jnareb@gmail.com>
---
gitweb/gitweb.perl | 11 +++++++++--
1 files changed, 9 insertions(+), 2 deletions(-)
diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl
index 040ee71..ebe59b8 100755
--- a/gitweb/gitweb.perl
+++ b/gitweb/gitweb.perl
@@ -18,11 +18,18 @@ use File::Find qw();
use File::Basename qw(basename);
binmode STDOUT, ':utf8';
-our $cgi = new CGI;
+# mod_perl request
+my $r;
+$r = shift @_ if $ENV{MOD_PERL};
+
+our $cgi = new CGI($r);
our $version = "++GIT_VERSION++";
our $my_url = $cgi->url();
our $my_uri = $cgi->url(-absolute => 1);
+# speeding up mod_perl and FastCGI (later)
+$cgi->compile() if $r;
+
# core git executable to use
# this can just be "git" if your webserver has a sensible PATH
our $GIT = "++GIT_BINDIR++/git";
@@ -364,7 +371,7 @@ if (defined $searchtype) {
# now read PATH_INFO and use it as alternative to parameters
sub evaluate_path_info {
return if defined $project;
- my $path_info = $ENV{"PATH_INFO"};
+ my $path_info = $r ? $r->path_info() : $ENV{"PATH_INFO"};
return if !$path_info;
$path_info =~ s,^/+,,;
return if !$path_info;
--
1.4.4.1
^ permalink raw reply related
* [PATCH] Move Fink and Ports check to after config file
From: Brian Gernhardt @ 2006-12-12 17:01 UTC (permalink / raw)
To: git
Putting NO_FINK or NO_DARWIN_PORTS in config.mak is ignored because the
checks are done before the config is included.
Signed-off-by: Brian Gernhardt <benji@silverinsanity.com>
---
Makefile | 27 +++++++++++++++------------
1 files changed, 15 insertions(+), 12 deletions(-)
diff --git a/Makefile b/Makefile
index a1861de..c1f1135 100644
--- a/Makefile
+++ b/Makefile
@@ -326,18 +326,6 @@ ifeq ($(uname_S),Darwin)
NEEDS_SSL_WITH_CRYPTO = YesPlease
NEEDS_LIBICONV = YesPlease
NO_STRLCPY = YesPlease
- ifndef NO_FINK
- ifeq ($(shell test -d /sw/lib && echo y),y)
- BASIC_CFLAGS += -I/sw/include
- BASIC_LDFLAGS += -L/sw/lib
- endif
- endif
- ifndef NO_DARWIN_PORTS
- ifeq ($(shell test -d /opt/local/lib && echo y),y)
- BASIC_CFLAGS += -I/opt/local/include
- BASIC_LDFLAGS += -L/opt/local/lib
- endif
- endif
endif
ifeq ($(uname_S),SunOS)
NEEDS_SOCKET = YesPlease
@@ -415,6 +403,21 @@ endif
-include config.mak.autogen
-include config.mak
+ifeq ($(uname_S),Darwin)
+ ifndef NO_FINK
+ ifeq ($(shell test -d /sw/lib && echo y),y)
+ ALL_CFLAGS += -I/sw/include
+ ALL_LDFLAGS += -L/sw/lib
+ endif
+ endif
+ ifndef NO_DARWIN_PORTS
+ ifeq ($(shell test -d /opt/local/lib && echo y),y)
+ ALL_CFLAGS += -I/opt/local/include
+ ALL_LDFLAGS += -L/opt/local/lib
+ endif
+ endif
+endif
+
ifndef NO_CURL
ifdef CURLDIR
# This is still problematic -- gcc does not always want -R.
--
1.4.4.1.GIT
^ permalink raw reply related
* Re: git-pull from git.git - no remote ref for pu or next?
From: Johannes Schindelin @ 2006-12-12 17:40 UTC (permalink / raw)
To: Randal L. Schwartz; +Cc: git
In-Reply-To: <86y7pd6oz7.fsf@blue.stonehenge.com>
Hi,
On Tue, 12 Dec 2006, Randal L. Schwartz wrote:
> >>>>> "Randal" == Randal L Schwartz <merlyn@stonehenge.com> writes:
>
> Randal> I just got this on this morning's git-fetch:
>
> Randal> error: no such remote ref refs/heads/pu
> Randal> error: no such remote ref refs/heads/next
> Randal> Fetch failure: git://git.kernel.org/pub/scm/git/git.git
Congratulations!
You are experiencing the Thundering Herd Phenomenon we talked a lot about
lately (the kernel.org mirroring thread).
Ciao,
Dscho
^ permalink raw reply
* Re: git-pull from git.git - no remote ref for pu or next?
From: Randal L. Schwartz @ 2006-12-12 17:42 UTC (permalink / raw)
To: Johannes Schindelin; +Cc: git
In-Reply-To: <Pine.LNX.4.63.0612121839200.2807@wbgn013.biozentrum.uni-wuerzburg.de>
>>>>> "Johannes" == Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
Johannes> Congratulations!
Johannes> You are experiencing the Thundering Herd Phenomenon we talked a lot about
Johannes> lately (the kernel.org mirroring thread).
As long as others are sharing my pain, I'll be OK. :)
--
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
<merlyn@stonehenge.com> <URL:http://www.stonehenge.com/merlyn/>
Perl/Unix/security consulting, Technical writing, Comedy, etc. etc.
^ permalink raw reply
* git-pull: "Cannot obtain needed none"?
From: Alex Riesen @ 2006-12-12 17:43 UTC (permalink / raw)
To: git
[-- Attachment #1: Type: text/plain, Size: 2986 bytes --]
Sometimes I wonder what language do we use in user-visible messages :)
What none was it, which could not be obtained? Why could none be needed?
Why does it happen processing commit 00000...?
error: Unable to find 9a6e87b60dbd2305c95cecce7d9d60f849a0658d under
http://www.kernel.org/pub/scm/git/git.git/
Cannot obtain needed none 9a6e87b60dbd2305c95cecce7d9d60f849a0658d
while processing commit 0000000000000000000000000000000000000000.
Complete trace:
GIT_TRACE=1 git pull
trace: exec: '/d/scripts/git/git-pull'
trace: built-in: git 'rev-parse' '--git-dir'
trace: built-in: git 'rev-parse' '--git-dir'
trace: built-in: git 'symbolic-ref' 'HEAD'
trace: built-in: git 'repo-config' '--get' 'branch.raa/2.remote'
trace: built-in: git 'repo-config' '--get' 'remote.origin.url'
trace: built-in: git 'repo-config' '--get' 'remote.origin.url'
trace: exec: '/d/scripts/git/git-ls-remote'
'http://www.kernel.org/pub/scm/git/git.git'
trace: built-in: git 'repo-config' '--get' 'remote.http:.url'
trace: built-in: git 'repo-config' '--bool' 'http.noEPSV'
trace: built-in: git 'repo-config' '--get' 'remote.origin.url'
trace: built-in: git 'symbolic-ref' 'HEAD'
trace: built-in: git 'repo-config' '--get' 'branch.raa/2.remote'
trace: built-in: git 'symbolic-ref' 'HEAD'
trace: built-in: git 'repo-config' '--get-all' 'branch.raa/2.merge'
trace: built-in: git 'check-ref-format' 'heads/git'
trace: built-in: git 'check-ref-format' 'heads/pu'
trace: built-in: git 'check-ref-format' 'heads/next'
trace: built-in: git 'repo-config' '--bool' 'http.noEPSV'
Fetching refs/heads/master from
http://www.kernel.org/pub/scm/git/git.git using http
trace: built-in: git 'rev-parse' '--verify'
'6f9872582246b9b8ee4bdc9f6a563b409aab1ecb^0'
trace: built-in: git 'cat-file' '-t' '6f9872582246b9b8ee4bdc9f6a563b409aab1ecb'
trace: built-in: git 'rev-parse' '--short'
'6f9872582246b9b8ee4bdc9f6a563b409aab1ecb'
trace: built-in: git 'repo-config' '--bool' 'http.noEPSV'
Fetching refs/heads/pu from http://www.kernel.org/pub/scm/git/git.git using http
Getting alternates list for http://www.kernel.org/pub/scm/git/git.git/
Getting pack list for http://www.kernel.org/pub/scm/git/git.git/
error: Unable to find 9a6e87b60dbd2305c95cecce7d9d60f849a0658d under
http://www.kernel.org/pub/scm/git/git.git/
Cannot obtain needed none 9a6e87b60dbd2305c95cecce7d9d60f849a0658d
while processing commit 0000000000000000000000000000000000000000.
FWIW, a patch changing it to:
git fetch
Fetching refs/heads/master from
http://www.kernel.org/pub/scm/git/git.git using http
Fetching refs/heads/pu from http://www.kernel.org/pub/scm/git/git.git using http
Getting alternates list for http://www.kernel.org/pub/scm/git/git.git/
Getting pack list for http://www.kernel.org/pub/scm/git/git.git/
error: Unable to find 9a6e87b60dbd2305c95cecce7d9d60f849a0658d under
http://www.kernel.org/pub/scm/git/git.git/
Cannot obtain needed object 9a6e87b60dbd2305c95cecce7d9d60f849a0658d
attached.
Signed-off-by: Alex Riesen <raa.lkml@gmail.com>
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: needed-none.patch --]
[-- Type: text/x-diff; name="needed-none.patch", Size: 1533 bytes --]
From 38e4d243fd163ca77233f73007b6b623eec81c68 Mon Sep 17 00:00:00 2001
From: Alex Riesen <raa.lkml@gmail.com>
Date: Tue, 12 Dec 2006 18:34:02 +0100
Subject: [PATCH] Clarify fetch error for missing objects.
Otherwise there're such things like:
Cannot obtain needed none 9a6e87b60dbd2305c95cecce7d9d60f849a0658d
while processing commit 0000000000000000000000000000000000000000.
which while looks weird. What is the none needed for?
diff --git a/fetch.c b/fetch.c
index c426c04..663b4b2 100644
--- a/fetch.c
+++ b/fetch.c
@@ -22,14 +22,15 @@ void pull_say(const char *fmt, const char *hex)
fprintf(stderr, fmt, hex);
}
-static void report_missing(const char *what, const unsigned char *missing)
+static void report_missing(const struct object *obj)
{
char missing_hex[41];
-
- strcpy(missing_hex, sha1_to_hex(missing));;
- fprintf(stderr,
- "Cannot obtain needed %s %s\nwhile processing commit %s.\n",
- what, missing_hex, sha1_to_hex(current_commit_sha1));
+ strcpy(missing_hex, sha1_to_hex(obj->sha1));;
+ fprintf(stderr, "Cannot obtain needed %s %s\n",
+ obj->type ? typename(obj->type): "object", missing_hex);
+ if (!is_null_sha1(current_commit_sha1))
+ fprintf(stderr, "while processing commit %s.\n",
+ sha1_to_hex(current_commit_sha1));
}
static int process(struct object *obj);
@@ -177,7 +178,7 @@ static int loop(void)
*/
if (! (obj->flags & TO_SCAN)) {
if (fetch(obj->sha1)) {
- report_missing(typename(obj->type), obj->sha1);
+ report_missing(obj);
return -1;
}
}
^ permalink raw reply related
* Re: git-pull from git.git - no remote ref for pu or next?
From: Linus Torvalds @ 2006-12-12 17:56 UTC (permalink / raw)
To: Randal L. Schwartz; +Cc: git
In-Reply-To: <86y7pd6oz7.fsf@blue.stonehenge.com>
On Tue, 12 Dec 2006, Randal L. Schwartz wrote:
> >>>>> "Randal" == Randal L Schwartz <merlyn@stonehenge.com> writes:
>
> Randal> I just got this on this morning's git-fetch:
>
> Randal> error: no such remote ref refs/heads/pu
> Randal> error: no such remote ref refs/heads/next
> Randal> Fetch failure: git://git.kernel.org/pub/scm/git/git.git
>
> Randal> Here's my remotes/origin:
>
> Randal> URL: git://git.kernel.org/pub/scm/git/git.git
> Randal> Pull: master:origin
> Randal> Pull: man:man
> Randal> Pull: html:html
> Randal> Pull: +pu:pu
> Randal> Pull: +next:next
>
> And then it mysteriously fixed itself a few minutes later.
> Is there some sort of publishing failure, or intermittent race condition?
It's mirroring.
The way that kernel.org works is that there is one master site, which is
not actually running any public services at all. That's the one that
people who have write access can ssh into, and rather than run any public
services, it runs the security-conscious things, like the secure logins
and the automated signing scripts.
The actual _public_ sites are just mirrors, with just rsync between the
things. All to keep the services on the master site minimal.
But because the public sites just mirror using rsync, and aren't really
aware of git repositories etc at that stage, what can happen is that a
mirroring is on-going when Junio does a push, and then the changes to the
"refs/" directory might get rsync'ed before the "object/" directory does,
and you end up with the public sites having references to objects that
don't even _exist_ on those public sites any more.
When they then run git-daemon, git-deamon will basically see a corrupt git
archive, and not expose those "broken" refs at all. Which explains what
you see.
And once the mirroring completes, the issue just goes away, which explains
why it just magically works five minutes later.
If the public sites used git itself to synchronize git repositories,
they'd never see anything like this (because git itself will only write
the new refs after it has actually updated the data). But since the thing
needs mirroring for non-git uses too, and since rsync generally _works_
apart from the slight race-condition issue, that's what it just uses.
^ permalink raw reply
* Re: git-pull from git.git - no remote ref for pu or next?
From: Johannes Schindelin @ 2006-12-12 18:09 UTC (permalink / raw)
To: Linus Torvalds; +Cc: Randal L. Schwartz, git
In-Reply-To: <Pine.LNX.4.64.0612120949230.3535@woody.osdl.org>
Hi,
On Tue, 12 Dec 2006, Linus Torvalds wrote:
> But since the thing needs mirroring for non-git uses too, and since
> rsync generally _works_ apart from the slight race-condition issue,
... and git would probably change the pack structure (i.e. which objects
are in which packs, or even loose) which would be too bad for all those
HTTP leechers ...
> that's what it just uses.
Ciao,
Dscho
^ permalink raw reply
* Re: index manipulation quickref
From: Junio C Hamano @ 2006-12-12 18:18 UTC (permalink / raw)
To: Nguyen Thai Ngoc Duy; +Cc: git
In-Reply-To: <fcaeb9bf0612120257p35dc9483ob65eea9ae21b5f7b@mail.gmail.com>
"Nguyen Thai Ngoc Duy" <pclouds@gmail.com> writes:
> I'm trying to collect all operations related to index from user
> perspective and corresponding commands. The list may be put to git
> wiki if people think it can help newbies:
I think this goes in the wrong direction. For "newbies" the
Porcelain-ish set is supposed to be enough and if there is
something missing that they need to do update-index command
itself or a pipeline that involves update-index to achieve
common tasks, we should enhance Porcelain-ish that captures the
pattern.
I think quick-ref for Porcelain writers would not hurt, but they
have manuals.
^ permalink raw reply
* Re: git-pull from git.git - no remote ref for pu or next?
From: Linus Torvalds @ 2006-12-12 18:23 UTC (permalink / raw)
To: Johannes Schindelin; +Cc: Randal L. Schwartz, git
In-Reply-To: <Pine.LNX.4.63.0612121908100.2807@wbgn013.biozentrum.uni-wuerzburg.de>
On Tue, 12 Dec 2006, Johannes Schindelin wrote:
> > rsync generally _works_ apart from the slight race-condition issue,
>
> ... and git would probably change the pack structure (i.e. which objects
> are in which packs, or even loose) which would be too bad for all those
> HTTP leechers ...
Well, as it is, I end up repacking my git archives on kernel.org every two
weeks or so anyway, so anybody who uses stupid protocols (rsync or http)
will end up downloading everything anew anyway.
And kernel.org will probably start doing automatic repacking, since the
current situation just means that some people don't repack on their own,
and have tens of thousands of loose objects.
You really don't want to use the non-native protocols unless you have to,
or for projects that don't change.
^ permalink raw reply
* Re: Adding a new file as if it had existed
From: Junio C Hamano @ 2006-12-12 18:31 UTC (permalink / raw)
To: Bahadir Balban; +Cc: git, Johannes Schindelin, Andy Parkins, Andreas Ericsson
In-Reply-To: <7ac1e90c0612120332o20d6778bsa16a788fdc04a3a1@mail.gmail.com>
"Bahadir Balban" <bahadir.balban@gmail.com> writes:
> ... I said "performance reasons" assuming all the
> file hashes need checked for every commit -a to see if they're
> changed, but I just tried on a PIII and it seems not so slow.
Ok.
Other people have already cleared the fear for 'commit' case, so
I hope you are happier.
There is one thing we could further optimize, though.
Switching branches with 100k blobs in a commit even when there
are a handful paths different between the branches would still
need to populate the index by reading two trees and collapsing
them into a single stage. In theory, we should be able to do a
lot better if two-tree case of read-tree took advanrage of
cache-tree information. If ce_match_stat() says Ok for all
paths in a subdirectory and the cached tree object name for that
subdirectory in the index match what we are reading from the new
tree, we should be able to skip reading that subdirectory (and
its subdirectories) from the new tree object at all.
Anybody interested to give it a try?
^ permalink raw reply
* Re: [RFC] E-mail aliases
From: Junio C Hamano @ 2006-12-12 18:37 UTC (permalink / raw)
To: Catalin Marinas; +Cc: git
In-Reply-To: <b0943d9e0612120449k4a13ac85t313df1e460ed46e4@mail.gmail.com>
"Catalin Marinas" <catalin.marinas@gmail.com> writes:
> sets the "To:" header to "linux-kernel@vger.kernel.org".
>
> The aliases are read from the [aliases] section in the gitconfig files
> ("git repo-config aliases.lkml" in this case). I just want to make
> sure than there isn't any other intended use for the [aliases] section
> in the config files.
[alias] is used to hold command-line alias, and it seems
slightly confusing to have two sections that mean something
quite different.
Perhaps
[mail "alias"]
lkml = linux-kernel@vger.kernel.org
if we are going to have other mail-related configurations that
might turn out be handy. send-email would want smtp-server in
there, for example.
^ permalink raw reply
* Re: [PATCH] Colourise git-branch output
From: Junio C Hamano @ 2006-12-12 18:43 UTC (permalink / raw)
To: Andy Parkins; +Cc: git
In-Reply-To: <200612120641.52556.andyparkins@gmail.com>
Andy Parkins <andyparkins@gmail.com> writes:
> The colour parameter is "color.branch" rather than "branch.color" to
> avoid clashing with the default namespace for default branch merge
> definitions.
Very nice.
> c = ' ';
> if (ref_list.list[i].kind == REF_LOCAL_BRANCH &&
> - !strcmp(ref_list.list[i].name, head))
> + !strcmp(ref_list.list[i].name, head)) {
> c = '*';
> + color = COLOR_BRANCH_CURRENT;
> + }
>
> if (verbose) {
> - printf("%c %-*s", c, ref_list.maxwidth,
> - ref_list.list[i].name);
> + printf("%c %s%-*s%s", c,
> + branch_get_color(color),
> + ref_list.maxwidth,
> + ref_list.list[i].name,
> + branch_get_color(COLOR_BRANCH_RESET));
> print_ref_info(ref_list.list[i].sha1, abbrev);
> }
> else
> - printf("%c %s\n", c, ref_list.list[i].name);
> + printf("%c %s%s%s\n", c,
> + branch_get_color(color),
> + ref_list.list[i].name,
> + branch_get_color(COLOR_BRANCH_RESET));
> }
Now this makes me wonder if under output coloring we would still
want the two-space indent and '*' prefix.
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox