* Rebasing published branches?
From: Josef Wolf @ 2012-11-06 20:18 UTC (permalink / raw)
To: git
Hello,
I know, I should never rebase published branches. But...
I frequently work on different computers and would like to share my private
branches across them. When done and the feature is in a good shape, I'd like
to rebase to clean up history before I make it available to other people.
I guess rebasing such branches would be OK as long as I can reliably remember
to delete those branches on _all_ the clones I ever created.
But waht if I ever make a mistake? How would one recover from such rebase
disasters? Anybody knows a good description how such a recover would be done?
^ permalink raw reply
* RE: Workflow for templates?
From: Pyeron, Jason J CTR (US) @ 2012-11-06 20:21 UTC (permalink / raw)
To: git@vger.kernel.org
In-Reply-To: <20121106195045.GD28437@raven.wolf.lan>
[-- Attachment #1: Type: text/plain, Size: 3325 bytes --]
Maybe I lost sight of your problem. Can you give a specific example of where "it" does not work?
> -----Original Message-----
> From: git-owner@vger.kernel.org [mailto:git-owner@vger.kernel.org] On
> Behalf Of Josef Wolf
> Sent: Tuesday, November 06, 2012 2:51 PM
> To: git@vger.kernel.org
> Subject: Re: Workflow for templates?
>
> No suggestions on this one?
>
> On Wed, Oct 31, 2012 at 11:44:04AM +0100, Josef Wolf wrote:
> > I am somewhat unsure whether it would work this way. After all, there
> seems to
> > be an unbreakable rule with git: never rebase published branches.
> >
> > Thus, once I have published my work to other people who also need to
> work on
> > the same localizations as I do, I have no longer the option of
> rebasing to get
> > rid of the localizations and put the generic template stuff for
> upstream.
> >
> > I guess, my concern is because I have not yet fully understood the
> problems of
> > rebasing, and how to recover from them.
> >
> > Maybe I should try to explain the problem in terms of repository
> > hierarchy. Let's assume, there is this hierarchy of repositories:
> >
> > upstream: central repository, containing the generic template
> >
> > foo-site: repository for site foo. Here we have localizations for a
> specific
> > administrative entity named foo (say, google).
> > This is where clones for production are made from, and
> production
> > boxes pull from here to be kept up-to-date.
> >
> > foo-prodA: A clone of foo-site, put in production and pulling from a
> specific
> > branch on foo-site to receive released, blessed updates.
> > foo-prodB: Similar to foo-prodA, but on another box.
> >
> > foo-devA: A clone of foo-site to make development, releases, and
> whatever for
> > foo.
> > foo-devB: One more clone of foo-site, Developer B is working here.
> >
> > Then, we might have more administrative entities: bar-site, bar-
> prodA,
> > bar-prodB, bar-devA, bar-devB, for example. This might be Microsoft,
> for
> > example.
> >
> > Further, foo-devA might be the same person as bar-devA.
> >
> > So when foo-devA pulls from foo-devB, then foo-devB will create
> problems when
> > he rebases after that pull.
> >
> > I think I have some kind of misunderstanding here, but I just can't
> figure
> > what it is.
> >
> >
> > Maybe I should try to explain the problem in yet other words:
> >
> > What I am trying to achieve, is to extend the workflow from
> development to
> > deployment across multiple administrative entities. As a picture:
> >
> > upstream (templates only).
> > ^
> > |
> > v
> > development (configured, might contain experimental changes)
> > ^
> > |
> > v
> > deployment (configured)
> >
> > This workflow should not stop at administrative borders. Just replace
> foo by
> > google and bar by Microsoft to get an idea of what I am trying to
> achieve.
> > --
> > To unsubscribe from this list: send the line "unsubscribe git" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at http://vger.kernel.org/majordomo-info.html
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
[-- Attachment #2: smime.p7s --]
[-- Type: application/x-pkcs7-signature, Size: 5615 bytes --]
^ permalink raw reply
* What about SHA-1 collisions?
From: Josef Wolf @ 2012-11-06 20:26 UTC (permalink / raw)
To: git
Hello,
we all know, the probability for SHA-1 collisions is very, very low, almost
non-existant. But we also know that they are not impossible.
Just for curiosity: what would happen if such a collision would occur within
one repository?
^ permalink raw reply
* [git-svn] [bug report] Index in strange state after git svn rebase
From: Samuel Rossille @ 2012-11-06 20:37 UTC (permalink / raw)
To: git; +Cc: Samuel
Hi Everybody,
Please Cc: me, I am not subscribed
I'm encountering an issue with "git svn rebase". Included in this
email is a summary of the issue, and it is fully detailed in this
stackoverflow question:
http://stackoverflow.com/questions/13183534/why-does-git-rebase-leave-opposite-sets-of-modifications-in-the-stage-and-the
I and several other people think it's a bug, but I could not find any
reference to it in the mailing list archive.
Some people have the same issue with the same svn repository, which is
quite large.
Unfortunately I have not been able to reproduce it completely from
scratch with a new svn repository. I seems to happen only on large
repository with a lot of history, branches, and files.
Here is the summary:
*What I wanted to do:*
In a branch tracking an svn remote, fetch team's content and rebase my
commit after them.
*What I did:*
Git version: "git version 1.7.10.msysgit.1"
With a clean working copy and empty index, I typed: "git svn rebase"
*What I expected:*
Fetch a couple of commits, then a successful rebase, with at the end
an empty index, and a clean working copy.
*What actually happened:*
Fetch a couple of commits, then a successful rebase, with at the end
an index containing modification that revert my commit, and a working
copy containing the expected content ("the revert of the revert")
Please feel free to contact me directly or via the SO question for any
useful additional information.
Thanks,
Samuel Rossille
^ permalink raw reply
* Re: Workflow for templates?
From: Josef Wolf @ 2012-11-06 21:07 UTC (permalink / raw)
To: git@vger.kernel.org
In-Reply-To: <871B6C10EBEFE342A772D1159D13208537AA184A@umechphj.easf.csd.disa.mil>
On Tue, Nov 06, 2012 at 08:21:25PM +0000, Pyeron, Jason J CTR (US) wrote:
> Maybe I lost sight of your problem. Can you give a specific example of where "it" does not work?
I guess it's _me_ who's lost. I can't figure how this is supposed to
work. Maybe you have an example?
^ permalink raw reply
* git ls-files --ignored and ignored directory
From: Damien Robert @ 2012-11-06 21:19 UTC (permalink / raw)
To: git
I would like to use git ls-files to show all the ignored files, including
directory.
As an example of setup:
mkdir /tmp/git && cd /tmp/git
git init
mkdir a b
touch a/a
touch b/b
cat >.gitignore << EOF
a/
b/*
EOF
Then if I do:
$ git ls-files --exclude-standard --ignored --others
b/b
$ git ls-files --exclude-standard --ignored --others --directory
a/
In the first case I was expecting
a/a
b/b
and in the second case
a/
b/
Is there a way to get the behavior I'd like? Thanks!
^ permalink raw reply
* Re: git ls-files --ignored and ignored directory
From: Damien Robert @ 2012-11-06 21:24 UTC (permalink / raw)
To: git
In-Reply-To: <k7bus9$o9t$1@ger.gmane.org>
Damien Robert wrote in message <k7bus9$o9t$1@ger.gmane.org>:
> I would like to use git ls-files to show all the ignored files, including
> directory.
> Is there a way to get the behavior I'd like? Thanks!
I forgot to add that this is with version 1.8.0, I haven't tried with
current master.
$ git --version
git version 1.8.0
^ permalink raw reply
* Re: What about SHA-1 collisions?
From: John McKown @ 2012-11-06 21:41 UTC (permalink / raw)
To: git
In-Reply-To: <20121106202643.GF28437@raven.wolf.lan>
Josef Wolf <jw <at> raven.inka.de> writes:
>
> Hello,
>
> we all know, the probability for SHA-1 collisions is very, very low, almost
> non-existant. But we also know that they are not impossible.
>
> Just for curiosity: what would happen if such a collision would occur within
> one repository?
>
In a sense, this cannot happen. Suppose you have a new working directory.
You do a "git init" to initialize it for use by git. You then copy in a
bunch of data from elsewhere. By chance, files "a" and "b" have different
content, but the same sha1 (they collide). The "git add ." command is
basically a short cut for doing something like:
for i in *;do git add $i;done
That is, it seems to add each file, one at a time in some order. Suppose
it creates the sha1 for "a" first. It then creates the appropriate
"stuff" for file "a" in the .git subdirectory, based on the sha1 value. Now,
it gets around to processing "b". It gets the sha1 value of b and finds
that it already has an entry for that value. At that point, the "git add" thinks
"Oh, I've already processed this file. No need to do anything!" So the contents
of file "b" are not saved anywhere in git and, bottom line, that version of "b"
will not be in the git repository. Ever. Because "a" already has that SHA1 "tied
up" and it is (theoretically) never released.
I think of the SHA1 value being a unique key into a "write once" database. Once
you've added some content (a file) into the database, then the SHA1 value of
that content (file) is unmodifiable. Attempts to write another record into the
database is rejected (in a read DB, you'd get some sort of DUPLICATE KEY
response). Git considers the "duplicate key" to be just fine because it ASSUMES
that the SHA1 is unique to the first file (content) which generates it.
Hope I made sense.
John
^ permalink raw reply
* Re: What about SHA-1 collisions?
From: Josef Wolf @ 2012-11-06 22:09 UTC (permalink / raw)
To: git
In-Reply-To: <loom.20121106T223000-502@post.gmane.org>
On Tue, Nov 06, 2012 at 09:41:29PM +0000, John McKown wrote:
> Josef Wolf <jw <at> raven.inka.de> writes:
> > Just for curiosity: what would happen if such a collision would occur within
> > one repository?
> In a sense, this cannot happen.
In the scenario you described, contents of this version of file "b" are lost
and replaced by the contents of file "a". So file "b" is broken.
What happens when files "a" and "b" are added into different repositories?
File "a" is added to repos "A", and file "b" is added to repos "B". Now it
depends from which repository you fetch the collided blob first. If you fetch
it from "A", file "b" will be broken. If you fetch first from "B", your "a"
will be broken.
It becomes even more interesting, if some commit or tree object would have
the same SHA1 as some other object. I guess, in such a case the repository
would be completely hosed?
^ permalink raw reply
* Re: Rebasing published branches?
From: Andrew Ardill @ 2012-11-06 22:53 UTC (permalink / raw)
To: Josef Wolf, git@vger.kernel.org
In-Reply-To: <20121106201822.GE28437@raven.wolf.lan>
Hi Josef,
On 7 November 2012 07:18, Josef Wolf <jw@raven.inka.de> wrote:
>
> Hello,
>
> I know, I should never rebase published branches. But...
>
> I frequently work on different computers and would like to share my private
> branches across them. When done and the feature is in a good shape, I'd like
> to rebase to clean up history before I make it available to other people.
>
> I guess rebasing such branches would be OK as long as I can reliably remember
> to delete those branches on _all_ the clones I ever created.
>
> But waht if I ever make a mistake? How would one recover from such rebase
> disasters? Anybody knows a good description how such a recover would be done?
The only real problem you should encounter is not knowing which
repository holds the 'true' feature branch, that is the one which you
want to publish.
The reason why rebasing public branches (where 'public' means somebody
_else_ is tracking your branch) is bad is that you are rewriting
somebody else's history. This can cause headache and pain for them if
they have based work off what you have already published.
In your situation, you are the only one working on these feature
branches, and you know that you plan to rebase them. There is little
risk of you rewriting someone else's history, and even if you did it
is clear that these branches were always meant to be rebased so you
should surprise anyone.
As a practical note, you'll probably find it easier to keep track of
the current state of your feature if you use one repository as your
own 'blessed' repository. After working on a clone somewhere, always
push to the blessed repository, and sync from it before you start
work. This way you will always have the correct version of your
feature branch.
If worse comes to worse (somehow) remember that rebasing does not
delete the old commits, just recreates them and points the branch at
the recreated versions. The old versions of these commits should be
available in the reflog, at least for a few weeks after the rebase.
Regards,
Andrew Ardill
^ permalink raw reply
* Re: Rebasing published branches?
From: Antony Male @ 2012-11-06 23:14 UTC (permalink / raw)
To: Josef Wolf, git
In-Reply-To: <20121106201822.GE28437@raven.wolf.lan>
On 06/11/2012 8:18 pm, Josef Wolf wrote:
> Hello,
>
> I know, I should never rebase published branches. But...
The major trouble with making rewritten branches public is one of merges.
Assume I have two local repos, A and B, sharing a single remote. I
create a branch in A, push it to the remote, then fetch it into B. I
then re-write the branch in A and force-push it, and fetch from B.
As far as B is now concerned, its local history diverges from the
remote's -- a scenario which must be resolved, usually through a merge,
before any work can be pushed. Unfortunately, this merge merges together
the two versions of history -- the old one from B's local history, and
the new one from the remote -- leading to a mess. If B then pushes, this
mess is published.
So "published", in the "don't rewrite published branches" sense, means
"a branch which someone else might regularly pull from, and in doing so
merge together two versions of history".
In general, remembering that you've pushed rewritten history, and to
makes sure that you haven't merged two versions of history after a
merge/pull, is sufficient. After rewriting history on a remote, rebase /
pull --rebase on a local, un-rewritten branch is sufficient to avoid the
merging-two-versions-of-history nightmare.
See "RECOVERING FROM UPSTREAM REBASE" in man git-rebase for a more
in-depth explanation and more discussion of solutions.
> I frequently work on different computers and would like to share my private
> branches across them. When done and the feature is in a good shape, I'd like
> to rebase to clean up history before I make it available to other people.
Rebasing a branch which is about to be deleted (after merging,
presumably) is generally regarded as fine, provided you're not expecting
people to base work on the branch before it's rewritten.
Antony
^ permalink raw reply
* Re: Support for a series of patches, i.e. patchset or changeset?
From: Eric Miao @ 2012-11-07 1:50 UTC (permalink / raw)
To: Johannes Sixt; +Cc: Michael J Gruber, git
In-Reply-To: <5098BFC0.6040709@viscovery.net>
On Tue, Nov 6, 2012 at 3:44 PM, Johannes Sixt <j.sixt@viscovery.net> wrote:
> Am 11/6/2012 7:56, schrieb Eric Miao:
>> On Tue, Nov 6, 2012 at 2:39 PM, Johannes Sixt <j.sixt@viscovery.net> wrote:
>>> Am 11/6/2012 1:58, schrieb Eric Miao:
>>>> E.g. when we merged a series of patches:
>>>>
>>>> [PATCH 00/08]
>>>> [PATCH 01/08]
>>>> ...
>>>> [PATCH 08/08]
>>>>
>>>> How do we know this whole series after merged when only one of these
>>>> commits are known?
>>>
>>> You can use git name-rev. For example:
>>>
>>> $ git name-rev 9284bdae3
>>> 9284bdae3 remotes/origin/pu~2^2~7
>>>
>>> This tell you that the series was merged two commits before origin/pu, and
>>> then it is the 7th from the tip of the series. Now you can
>>>
>>> $ git log origin/pu~2^..origin/pu~2^2
>>>
>>> to see the whole series.
>>
>> I'm just curious how this is implemented in git, are we keeping the info
>> of the series that's applied in a whole?
>
> If the maintainer did his job well, then everything that you had in [PATCH
> 01/08] ... [PATCH 08/08] is in the commits of the series, and [PATCH
> 00/08] (the cover letter) is in the commit that merged the series.
>
> Anything else that I didn't mention but you consider as "the info of the
> series"?
>
>> But this still looks like be inferred basing on a branch head, and I'm
>> afraid this may not be applicable in every case.
>
> What's the problem? That it's inferred? Or that it needs a branch head?
Take kernel development for example, sub-maintainers not always keep
a patchset in a single branch, instead, there could be a mix of patchset
and single fixing patches on a same branch:
---A1---A2---A3---B---C---D---E1---E2---E3---E4---F---G---H---> branch
When we identify a specific patch, e.g. E3, is it possible to figure out
the whole patchset of E<n>?
>
> -- Hannes
^ permalink raw reply
* Bug? Subtree merge seems to choke on trailing slashes.
From: Jack O'Connor @ 2012-11-07 1:58 UTC (permalink / raw)
To: git
I'm summarizing from here:
http://stackoverflow.com/questions/5904256/git-subtree-merge-into-a-deeply-nested-subdirectory
Quick repro:
1) I do an initial subtree merge in what I think is the standard way
(http://www.kernel.org/pub/software/scm/git/docs/howto/using-merge-subtree.html).
My prefix is simply "test/".
2) I try to merge more upstream changes on top of that with the
following command:
git merge --strategy-option=subtree='test/' $upstream_stuff
3) Git fails with an obscure error:
fatal: entry not found in tree daf4d0f0a20b8b6ec007be9fcafeac84a6eba4f0
If I remove the trailing slash from the command in step 2, it works just fine:
git merge --strategy-option=subtree='test' $upstream_stuff
Note in the error message above, there's a double space after "entry".
Is it looking for a tree with an empty name? Did my trailing slash
imply a directory named empty-string?
Thanks for your help.
-- Jack O'Connor
^ permalink raw reply
* Re: why '--init' in git-submodule update
From: Francis Moreau @ 2012-11-07 7:10 UTC (permalink / raw)
To: Heiko Voigt; +Cc: git
In-Reply-To: <20121105213004.GA2797@book.hvoigt.net>
Hi,
On Mon, Nov 5, 2012 at 10:30 PM, Heiko Voigt <hvoigt@hvoigt.net> wrote:
> Hi,
>
> On Mon, Nov 05, 2012 at 05:30:51PM +0100, Francis Moreau wrote:
>> I'm wondering why the --init option from git-submodule-update is not
>> at least the defaut. Or even wilder, why this option exists at all and
>> git-submodule-update always behave like --init was always passed.
>
> That was a design decision. People may not be interested in all
> submodules but just a subset they are working with. E.g. think of a
> large media directory as a submodule the developers might not be
> interested in having it around all the time because only the designers
> are heavily working with it.
>
Thanks for explaining.
Another possible solution was to check if one or several submodules
are already checkout. If so then update only them. If none of them is
checkout then assume --init is passed and init all of them. Same if a
path is passed and this path corresponds to a submodule not checkout
yet.
--
Francis
^ permalink raw reply
* Re: [PATCH] push/pull: adjust missing upstream help text to changed interface
From: Michael J Gruber @ 2012-11-07 8:22 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git, Jeroen van der Ham, Jeff King
In-Reply-To: <03ff9252-71dc-4c28-a542-d65d16ba6549@email.android.com>
Junio C Hamano venit, vidit, dixit 06.11.2012 15:45:
>
>
> Michael J Gruber <git@drmicha.warpmail.net> wrote:
>
>> Yes, I'm in low hanging fruits mood.
>>
>> Signed-off-by: Michael J Gruber <git@drmicha.warpmail.net>
>
> It is called tying loose ends, and is very important. Very much appreciated.
;)
By the way: The gotcha with the signed-off-by after the note is due to
my old format-patch-notes being tripped up by the recent restructuring;
your version in next get's it right. I've dumped my old branch, and I
hope the maintainer in chief will shuffle around the s-o-b where it belongs.
Michael
^ permalink raw reply
* Re: [PATCH] push/pull: adjust missing upstream help text to changed interface
From: Jeff King @ 2012-11-07 10:44 UTC (permalink / raw)
To: Michael J Gruber; +Cc: Junio C Hamano, git, Jeroen van der Ham
In-Reply-To: <509A1A4E.5060805@drmicha.warpmail.net>
On Wed, Nov 07, 2012 at 09:22:38AM +0100, Michael J Gruber wrote:
> Junio C Hamano venit, vidit, dixit 06.11.2012 15:45:
> >
> >
> > Michael J Gruber <git@drmicha.warpmail.net> wrote:
> >
> >> Yes, I'm in low hanging fruits mood.
> >>
> >> Signed-off-by: Michael J Gruber <git@drmicha.warpmail.net>
> >
> > It is called tying loose ends, and is very important. Very much appreciated.
>
> ;)
>
> By the way: The gotcha with the signed-off-by after the note is due to
> my old format-patch-notes being tripped up by the recent restructuring;
> your version in next get's it right. I've dumped my old branch, and I
> hope the maintainer in chief will shuffle around the s-o-b where it belongs.
Yes, will do. Thanks.
-Peff
^ permalink raw reply
* git-clone and unreliable links?
From: Josef Wolf @ 2012-11-07 15:35 UTC (permalink / raw)
To: git
Hello,
When using git-clone over an unreliable link (say, UMTS) and the network goes
down, git-clone deletes everything what was downloaded. When the network goes
up again and you restart git-clone, it has to start over from the
beginning. Then, eventually, the network goes down again, and everything is
deleted again.
Is there a way to omit the deleting step, so the second invocation would start
where the first invocation was interrupted?
^ permalink raw reply
* Re: What about SHA-1 collisions?
From: Shawn Pearce @ 2012-11-07 15:42 UTC (permalink / raw)
To: Josef Wolf, git
In-Reply-To: <20121106220938.GH28437@raven.wolf.lan>
On Tue, Nov 6, 2012 at 2:09 PM, Josef Wolf <jw@raven.inka.de> wrote:
>
> On Tue, Nov 06, 2012 at 09:41:29PM +0000, John McKown wrote:
> > Josef Wolf <jw <at> raven.inka.de> writes:
> > > Just for curiosity: what would happen if such a collision would occur within
> > > one repository?
>
> > In a sense, this cannot happen.
>
> In the scenario you described, contents of this version of file "b" are lost
> and replaced by the contents of file "a". So file "b" is broken.
>
> What happens when files "a" and "b" are added into different repositories?
> File "a" is added to repos "A", and file "b" is added to repos "B". Now it
> depends from which repository you fetch the collided blob first. If you fetch
> it from "A", file "b" will be broken. If you fetch first from "B", your "a"
> will be broken.
>
> It becomes even more interesting, if some commit or tree object would have
> the same SHA1 as some other object. I guess, in such a case the repository
> would be completely hosed?
When exchanging objects over the network, Git compares byte-for-byte
any object that one side sent that the other side already has, and
complains loudly when there is a collision detected. This only works
if the sender includes the "wrong" content for the named object. Git
also does assume the SHA-1 is unique and that it is not always
necessary to transmit the object. In these cases you would not be able
to detect the collision, because there isn't one. Your repository
would simply be using the wrong content for a file. Presumably one
would notice your build doesn't work anymore and investigate why.
^ permalink raw reply
* Re: git-clone and unreliable links?
From: Shawn Pearce @ 2012-11-07 15:54 UTC (permalink / raw)
To: Josef Wolf, git
In-Reply-To: <20121107153519.GI28437@raven.wolf.lan>
On Wed, Nov 7, 2012 at 7:35 AM, Josef Wolf <jw@raven.inka.de> wrote:
> When using git-clone over an unreliable link (say, UMTS) and the network goes
> down, git-clone deletes everything what was downloaded. When the network goes
> up again and you restart git-clone, it has to start over from the
> beginning. Then, eventually, the network goes down again, and everything is
> deleted again.
>
> Is there a way to omit the deleting step, so the second invocation would start
> where the first invocation was interrupted?
No, because a clone is not resumable.
The best way to obtain a repository over an unstable link is to ask
the repository owner to make a bundle file with `git bundle create
--heads --tags` and serve the file using standard HTTP or rsync, which
are resumable protocols. After you download the file, you can clone or
fetch from the bundle to initialize your local repository, and then
run git fetch to incrementally update to anything that is more recent
than the bundle's creation.
^ permalink raw reply
* Re: Workflow for templates?
From: Holger Hellmuth (IKS) @ 2012-11-07 16:03 UTC (permalink / raw)
To: Josef Wolf, git@vger.kernel.org
In-Reply-To: <20121106210719.GG28437@raven.wolf.lan>
Am 06.11.2012 22:07, schrieb Josef Wolf:
> On Tue, Nov 06, 2012 at 08:21:25PM +0000, Pyeron, Jason J CTR (US) wrote:
>> Maybe I lost sight of your problem. Can you give a specific example of where "it" does not work?
>
> I guess it's _me_ who's lost. I can't figure how this is supposed to
> work. Maybe you have an example?
Let me ask a different question: What is wrong with cherry-picking
downstream changes to your upstream branch? Without rebasing it to
downstream.
That might mean there is a rather useless merge downstream later on, but
that's the price you pay for not doing the change in a development branch.
^ permalink raw reply
* Re: Long clone time after "done."
From: Uri Moszkowicz @ 2012-11-07 17:32 UTC (permalink / raw)
To: git
In-Reply-To: <CAMJd5AQBbnFqT5xrFuPOEsJevwDE=jUgBVFZ5KqTZk5zv5+NOw@mail.gmail.com>
To elaborate from before, my repository is about 2GB and has about 7k
branches and tags with 15 years of history imported from CVS. I
created a toy repository attempting to replicate the problem but
haven't had any luck so instead I launched Git in a debugger to see
what operation is taking so long in "git clone". I stopped randomly in
the middle and stepped to the line which seems to be contributing the
most:
#0 read_packed_sha1 (sha1=0xb0ee98
"\017C\205Wj\001`\254\356\307Z\332\367\353\233.\375P}D",
type=0x7fbfffe7a4, size=0x7fbfffe798) at sha1_file.c:2237
#1 0x00000000004d366e in read_object (sha1=0xb0ee98
"\017C\205Wj\001`\254\356\307Z\332\367\353\233.\375P}D",
type=0x7fbfffe7a4, size=0x7fbfffe798) at sha1_file.c:2277
#2 0x00000000004d3839 in read_sha1_file_extended (sha1=0xb0ee98
"\017C\205Wj\001`\254\356\307Z\332\367\353\233.\375P}D",
type=0x7fbfffe7a4, size=0x7fbfffe798, flag=<optimized out>) at
sha1_file.c:2307
#3 0x00000000004ae000 in read_sha1_file (sha1=0x7fbfffc5e0 "",
type=0x6be000, size=0x2b039b9010) at cache.h:730
#4 parse_object (sha1=0xb0ee98
"\017C\205Wj\001`\254\356\307Z\332\367\353\233.\375P}D") at
object.c:212
#5 0x00000000004ae9ec in handle_one_ref (path=0xb0eec0
"refs/tags/<removed>", sha1=0xb0ee98
"\017C\205Wj\001`\254\356\307Z\332\367\353\233.\375P}D", flags=2,
cb_data=<optimized out>) at pack-refs.
#6 0x00000000004bc402 in do_one_ref (base=<optimized out>,
fn=0x4ae8f0 <handle_one_ref>, trim=<optimized out>, flags=0,
cb_data=0x7fbfffe9c0, entry=0xb0ee90) at refs.c:527
#7 0x00000000004bd38b in do_for_each_ref_in_dir (dir=0xb09be8,
offset=<optimized out>, base=0x4fd9b0 "", fn=0x4ae8f0
<handle_one_ref>, trim=0, flags=0, cb_data=0x7fbfffe9c0) at refs.c:553
#8 0x00000000004bd6cb in do_for_each_ref_in_dirs (dir1=0x6a8588,
dir2=0xb0ab18, base=0x4fd9b0 "", fn=0x4ae8f0 <handle_one_ref>, trim=0,
flags=0, cb_data=0x7fbfffe9c0) at refs.c:599
#9 0x00000000004bd6cb in do_for_each_ref_in_dirs (dir1=0x6a3118,
dir2=0xb038b8, base=0x4fd9b0 "", fn=0x4ae8f0 <handle_one_ref>, trim=0,
flags=0, cb_data=0x7fbfffe9c0) at refs.c:599
#10 0x00000000004bdb3f in do_for_each_ref (submodule=<optimized
out>, base=0x4fd9b0 "", fn=0x4ae8f0 <handle_one_ref>, trim=0, flags=0,
cb_data=0x7fbfffe9c0) at refs.c:1288
#11 0x00000000004aeaba in pack_refs (flags=<optimized out>) at pack-refs.c:133
#12 0x000000000041da30 in write_remote_refs (local_refs=<optimized
out>) at builtin/clone.c:470
#13 update_remote_refs (refs=<optimized out>, mapped_refs=<optimized
out>, remote_head_points_at=<optimized out>, branch_top=0x6a4210
"refs/remotes/origin/", msg=0x6a1160 "clone: from
/z/umoszkow/git/clone/../merged/merged.git") at bu
#14 cmd_clone (argc=<optimized out>, argv=0x7fbffff1b0,
prefix=<optimized out>) at builtin/clone.c:898
#15 0x0000000000404eef in handle_internal_command (argc=3,
argv=0x7fbffff1b0) at git.c:306
#16 0x0000000000405347 in run_argv (argcp=<optimized out>,
argv=<optimized out>) at git.c:513
#17 main (argc=3, argv=0x7fbffff1b0) at git.c:588
It looks like handle_one_ref() is called for each ref and most result
in a call to read_sha1_file(). It only takes a second or so for each
call but when you have thousands of them (one for each ref) it adds
up. Adding --single-branch --branch <branch> doesn't appear to help as
it is implemented afterwards. I would like to debug this problem
further but am not familiar enough with the implementation to know
what the next step is. Can anyone offer some suggestions? I don't see
why a clone should be dependent on an O(#refs) operations.
On Tue, Oct 23, 2012 at 1:30 PM, Uri Moszkowicz <uri@4refs.com> wrote:
> I have a large repository which I ran "git gc --aggressive" on that
> I'm trying to clone on a local file system. I would expect it to
> complete very quickly with hard links but it's taking about 6min to
> complete with no checkout (git clone -n). I see the message "Clining
> into 'repos'... done." appear after a few seconds but then Git just
> hangs there for another 6min. Any idea what it's doing at this point
> and how I can speed it up?
^ permalink raw reply
* [PATCH/RFC] launch_editor: ignore SIGINT while the editor has control
From: Paul Fox @ 2012-11-07 19:16 UTC (permalink / raw)
To: git; +Cc: gitster
the user's editor likely catches SIGINT (ctrl-C). but if the user
spawns a command from the editor and uses ctrl-C to kill that command,
the SIGINT will likely also kill git itself. (depending on the
editor, this can leave the terminal in an unusable state.)
Signed-off-by: Paul Fox <pgf@foxharp.boston.ma.us>
---
i often shell out of my editor while composing a git commit message,
in order to recheck the diffs or the log, do a final test build, etc.
when i interrupt one of these operations, the spawned program gets
killed. in addition git itself gets killed, which in turn kills my
editor. this is never what i intended. :-)
the problem is easy to demonstrate with vim, vile, or em. in a vi-like
editor:
git commit foo
:!sleep 10
^C
both CVS and my usual mailer (MH) protect against this behavior when
spawning editors by using code similar to the patch below, which
causes the spawning process to ignore SIGINT while the editor is
running.
i looked at the other invocations of run_command_v_opt_xxx() in git,
but couldn't convince myself that any of the others needed similar
protection. i also couldn't convince myself that i wouldn't cause
collateral damage if i tried moving the sigchain_push/pop into
run-command.c. (but perhaps it's simple -- maybe the RUN_USING_SHELL
flag should always imply this behavior.)
the patch is against current master.
paul
editor.c | 6 +++++-
1 files changed, 5 insertions(+), 1 deletions(-)
diff --git a/editor.c b/editor.c
index d834003..775f22d 100644
--- a/editor.c
+++ b/editor.c
@@ -37,8 +37,12 @@ int launch_editor(const char *path, struct strbuf *buffer, const char *const *en
if (strcmp(editor, ":")) {
const char *args[] = { editor, path, NULL };
+ int ret;
- if (run_command_v_opt_cd_env(args, RUN_USING_SHELL, NULL, env))
+ sigchain_push(SIGINT, SIG_IGN);
+ ret = run_command_v_opt_cd_env(args, RUN_USING_SHELL, NULL, env);
+ sigchain_pop(SIGINT);
+ if (ret)
return error("There was a problem with the editor '%s'.",
editor);
}
--
1.7.5.4
=---------------------
paul fox, pgf@foxharp.boston.ma.us (arlington, ma, where it's 31.8 degrees)
^ permalink raw reply related
* Re: [PATCH 14/13] wildmatch: fix tests that fail on Windows due to path mangling
From: Johannes Sixt @ 2012-11-07 19:32 UTC (permalink / raw)
To: Nguyen Thai Ngoc Duy; +Cc: git, Junio C Hamano
In-Reply-To: <CACsJy8DEH53mA4fDyZwbVsbyGcNY8b54Sn1p5-rto6r_h48dkA@mail.gmail.com>
Am 06.11.2012 13:47, schrieb Nguyen Thai Ngoc Duy:
> On Mon, Nov 5, 2012 at 4:00 AM, Johannes Sixt <j6t@kdbg.org> wrote:
>> Patterns beginning with a slash are converted to Windows paths before
>> test-wildmatch gets to see them. Use a different first character.
>
> Or we could prepend the paths with something, which is then cut out by
> test-wildmatch. Not sure if it's intuitive to look at the tests
> though.
That would be a possibility, too.
>> After this change, there are still 3 failing tests that are in connection
>> with [[:xdigit:]]. Don't know, yet, what's going on there.
>
> the wildmatch tests or fnmatch ones?
fnmatch ones:
ok 147 - wildmatch: match '5' '[[:xdigit:]]'
not ok 148 - fnmatch: match '5' '[[:xdigit:]]'
#
# test-wildmatch fnmatch '5' '[[:xdigit:]]'
#
ok 149 - wildmatch: match 'f' '[[:xdigit:]]'
not ok 150 - fnmatch: match 'f' '[[:xdigit:]]'
#
# test-wildmatch fnmatch 'f' '[[:xdigit:]]'
#
ok 151 - wildmatch: match 'D' '[[:xdigit:]]'
not ok 152 - fnmatch: match 'D' '[[:xdigit:]]'
#
# test-wildmatch fnmatch 'D' '[[:xdigit:]]'
#
She same tests fail on Linux, BTW, when built with NO_FNMATCH=1 and the
"#if defined _LIBC || !defined __GNU_LIBRARY__" brackets removed.
-- Hannes
^ permalink raw reply
* Re: [PATCHv2 2/2] git p4: add support for 'p4 move' in P4Submit
From: Vitor Antunes @ 2012-11-07 21:06 UTC (permalink / raw)
To: Pete Wyckoff, ggibons; +Cc: Git Mailing List
In-Reply-To: <20121105175724.GB889@padd.com>
On Mon, Nov 5, 2012 at 5:57 PM, Pete Wyckoff <pw@padd.com> wrote:
> vitor.hda@gmail.com wrote on Mon, 05 Nov 2012 17:37 +0000:
>> Hi Pete,
>>
>> I've just been hit by a situation where this command is available but is
>> disabled in the server. I don't know what is the best approach to avoid
>> this issue.
>
> Really? The command exists in the server because it returns the
> text output for "p4 help move". But "p4 move" itself fails
> because it is somehow disabled in the server?
>
> I didn't even know it was possible to administratively disable
> commands.
>
> What's the actual error message? And versions of your client and
> server (p4 -V, p4d -V, p4 info).
>
> Any ideas Gary?
I don't feel comfortable in testing this again because I'm working in a
production server. Can Gary provide any details on this type of
configuration on the server side?
Vitor
^ permalink raw reply
* Re: crash on git diff-tree -Ganything <tree> for new files with textconv filter
From: Peter Oberndorfer @ 2012-11-07 21:10 UTC (permalink / raw)
To: Jeff King; +Cc: git, Junio C Hamano
In-Reply-To: <20121029223521.GJ20513@sigill.intra.peff.net>
On 2012-10-29 23:35, Jeff King wrote:
> On Mon, Oct 29, 2012 at 09:19:48PM +0100, Peter Oberndorfer wrote:
>
>> I could reproduce with my 0x3000 bytes file on linux. The buffer is not
>> read with a trailing null byte it is mapped by mmap in
>> diff_populate_filespec...
>> So i think we will not get away with expecting a trailing null :-/
> Thanks for the reproduction recipe. I was testing with "git log", which
> does not use the mmap optimization.
>
>> For me the key to reproduce the problem was to have 2 commits.
>> Adding the file in the root commit it did not work. [1]
> You probably would need to pass "--root" for it to do the diff of the
> initial commit.
>
> The patch below fixes it, but it's terribly inefficient (it just detects
> the situation and reallocates). It would be much better to disable the
> reuse_worktree_file mmap when we populate the filespec, but it is too
> late to pass an option; we may have already populated from an earlier
> diffcore stage.
Hi,
I tested your patch, and i can confirm it fixes the problem for me.
(also on my real world test in msysgit)
Again, thanks a lot!
Greetings Peter
> I guess if we teach the whole diff code that "-G" (and --pickaxe-regex)
> is brittle, we can disable the optimization from the beginning based on
> the diff options. I'll take a look.
>
> <snip patch>
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox