* RE: Git vs Monotone
From: Linus Torvalds @ 2008-07-31 19:52 UTC (permalink / raw)
To: Craig L. Ching; +Cc: sverre, Git Mailinglist
In-Reply-To: <63BEA5E623E09F4D92233FB12A9F79430238A5EC@emailmn.mqsoftware.com>
On Thu, 31 Jul 2008, Craig L. Ching wrote:
>
> It's possible he's doing that, but it's also possible he just isn't that
> familiar with git.
Possible. But it really sounded like he didn't even try. Because quite
frankly, if he had even bothered to _try_, he wouldn't have gotten the
numbers he got.
The fact is, even without "-s", a local clone will do hardlinks for the
database. And since the original pack-file is marked as a 'keep' file,
that original pack-file won't even be broken apart.
So literally, if he had just bothered to even _try_ the git setup, he'd
have noticed that git actually uses less disk than monotone would do. But
it sounds like he didn't even try it.
So completely ignoring the fact that you could do a single database with
git, and completely ignoring the fact that with git you'd probably use
branches for at least some of those 11 repos anyway, he'd _still_ have had
less disk space used by git unless he would do something intentionally odd
(like clone all the repositories over the network separately).
Linus
^ permalink raw reply
* Re: Monotone workflow compared to Git workflow ( was RE: Git vs Monotone)
From: Linus Torvalds @ 2008-07-31 20:09 UTC (permalink / raw)
To: Craig L. Ching; +Cc: sverre, Git Mailinglist
In-Reply-To: <63BEA5E623E09F4D92233FB12A9F79430238A5EE@emailmn.mqsoftware.com>
On Thu, 31 Jul 2008, Craig L. Ching wrote:
>
> We find ourselves constantly having to shift gears and work on other
> things in the middle of whatever it is we're currently working on. For
> instance, in the scenario above, A might be branch that contains a
> feature going into our next release. B might be a bugfix and takes
> priority over A, so you have to leave A as-is and start work on B. When
> I come back to work on A, I have to rebuild A to continue working, and
> that's just too expensive for us. So we use the monotone-like
> new-workdir which allows us to save those build artifacts.
>
> So, that said, I ask again, am I missing something? Is there a better
> way to do this? How do the kernel developers do this, surely they're
> switching branches back and forth having to build in-between?
Sure, if you want to keep the build tree around, you would probably not
use branches.
But yes, then you'd likely do "git clone -s" with some single "common
point" or use "git worktree". And even if you don't use "-s", you should
_still_ effectively share at least all the old history (which tends to be
the bulk) thanks to even a default "git clone" will just hardlink the
pack-files.
So literally, if you do
git clone <cntral-repo-over-network> <local>
and then do
git clone <local> <otherlocal>
git clone <local> <thirdlocal>
then all of those will all share the initial pack-file on-disk. Try it.
(You may then want to edit the "origin" branch info in the .git/config to
point to the network one etc, of course).
Oh, and to make sure I'm not lying I actually did test this, but I also
noticed that "git clone" no longer marks the initial pack-file with
"keep", so it looks like "git gc" will then break the link. That's sad. I
wonder when that changed, or maybe I'm just confused and it never did.
Junio?
Linus
^ permalink raw reply
* Re: Monotone workflow compared to Git workflow ( was RE: Git vs Monotone)
From: Shawn O. Pearce @ 2008-07-31 20:18 UTC (permalink / raw)
To: Linus Torvalds; +Cc: Craig L. Ching, sverre, Git Mailinglist
In-Reply-To: <alpine.LFD.1.10.0807311253140.3277@nehalem.linux-foundation.org>
Linus Torvalds <torvalds@linux-foundation.org> wrote:
>
> Oh, and to make sure I'm not lying I actually did test this, but I also
> noticed that "git clone" no longer marks the initial pack-file with
> "keep", so it looks like "git gc" will then break the link. That's sad. I
> wonder when that changed, or maybe I'm just confused and it never did.
It was a bug in git-clone that we were recording the .keep file on
initial clone. We left the lock file in place after the fetch pack
call was done, but didn't remove it after the refs were updated.
If we want to go back to .keep'ing the original pack creating
during clone it probably should be threshold based. For many
smaller projects with only a 25M pack (or less) there is no point
in .keep'ing that first pack. For larger projects where the pack
is over a few hundred megabytes, then yea, maybe there is value
in .keep'ing it during clone.
--
Shawn.
^ permalink raw reply
* Re: git-scm.com
From: Kevin Ballard @ 2008-07-31 20:19 UTC (permalink / raw)
To: Jon Loeliger; +Cc: Tom Werner, git
In-Reply-To: <489206E4.2030901@freescale.com>
On Jul 31, 2008, at 11:39 AM, Jon Loeliger wrote:
> Tom Werner wrote:
>>
>> The problem is that I'm only a casual C coder. It takes me a while to
>> figure out what's going on in the git source. We needed a way to
>> serve
>> public git repositories from a hashed directory structure (e.g.
>> /a/b/c/user/repo.git) and we needed it fast.
>
> I'm not exactly sure what you mean by "hashed directory structure",
> but I suspect that your goal is some form of virtualized hosting
> that allows for directory names to be dynamically constructed with
> a component that appears to be the user name.
>
> Wouldn't the --interpolated-path ability of git-daemon either
> directly or with minor modifications directly support that?
Tom, correct me if I'm wrong, but my understanding of this is that,
with GFS, they were running into the problem of too many dirents in
one directory, thus causing lots of stability problems (GFS has a far
lower limit than other filesystems in this regard). So the GitHub guys
had to switch to a directory sharding structure (similar to how the
git objects db uses the first 2 characters of the hash as the dir
name) to split this up and keep the numbers manageable. However, they
still had to support the old git://github.com/user/project.git paths.
-Kevin Ballard
--
Kevin Ballard
http://kevin.sb.org
kevin@sb.org
http://www.tildesoft.com
^ permalink raw reply
* Re: Git vs Monotone
From: Junio C Hamano @ 2008-07-31 20:24 UTC (permalink / raw)
To: Linus Torvalds; +Cc: Craig L. Ching, sverre, Git Mailinglist
In-Reply-To: <alpine.LFD.1.10.0807311244240.3277@nehalem.linux-foundation.org>
Linus Torvalds <torvalds@linux-foundation.org> writes:
> ... And since the original pack-file is marked as a 'keep' file,
> that original pack-file won't even be broken apart.
Oops, isn't that something we fixed recently as a "bug"?
> So completely ignoring the fact that you could do a single database with
> git, and completely ignoring the fact that with git you'd probably use
> branches for at least some of those 11 repos anyway, he'd _still_ have had
> less disk space used by git unless he would do something intentionally odd
> (like clone all the repositories over the network separately).
Well, people are not perfect and they are free to express their opinions
based on faulty understanding of reality on their blogs. The right things
to do are (1) ignore them on the list and not waste many people's time,
and/or (2) educate them, but in private or in a circle where many other
similar ignorants benefit from such education. That is not here but
perhaps on #monotone channel?
^ permalink raw reply
* Re: Git vs Monotone
From: Jeff King @ 2008-07-31 20:32 UTC (permalink / raw)
To: sverre
Cc: Craig L. Ching, Petr Baudis, Stephen R. van den Berg,
Git Mailinglist
In-Reply-To: <bd6139dc0807311219h670f782cm8bed74bed2b4558@mail.gmail.com>
On Thu, Jul 31, 2008 at 09:19:41PM +0200, Sverre Rabbelier wrote:
> I repacked with --depth=100 and --window=100, I tried out 500 at first
> but it was just insanely slow (on a VM with one 2.4Ghz Core
> available). This resulted in a .git dir of 76MB. With that dir I did
> the following:
I tried 200/200 and got a 74M packfile. So I think we're getting into
diminishing returns.
> $ du -sh .
> 742M .
>
> So... monotone, eat your heart out ;).
:)
-Peff
^ permalink raw reply
* Re: [PATCH 0/2] gitweb use sections
From: Petr Baudis @ 2008-07-31 20:32 UTC (permalink / raw)
To: Gustavo Sverzut Barbieri; +Cc: git, Jakub Narebski
In-Reply-To: <b1f2b8c40807311243i5689683avcc3c3c91e4e6a900@mail.gmail.com>
Hi,
On Thu, Jul 31, 2008 at 04:43:35PM -0300, Gustavo Sverzut Barbieri wrote:
> Since nobody replied and I missed some gitweb guys in CC, I'm adding
> Petr and Jakub, as some guys said on IRC.
>
> Have anyone tried this patch, any problems?
sorry, I have it in my review queue. At first pass it was looking
good, but I wanted to look at it better before commenting.
One thing I'm wondering about is how to make this stuff configurable,
since I'm not very comfortable with adding more "unbound" configuration
variables and would rather prefer stuff to be added to the $features
array... I'm not at all sure about my own sentiment here, however.
--
Petr "Pasky" Baudis
As in certain cults it is possible to kill a process if you know
its true name. -- Ken Thompson and Dennis M. Ritchie
^ permalink raw reply
* Re: Git vs Monotone
From: Linus Torvalds @ 2008-07-31 20:30 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Craig L. Ching, sverre, Git Mailinglist
In-Reply-To: <7vmyjxyf7a.fsf@gitster.siamese.dyndns.org>
On Thu, 31 Jul 2008, Junio C Hamano wrote:
> Linus Torvalds <torvalds@linux-foundation.org> writes:
>
> > ... And since the original pack-file is marked as a 'keep' file,
> > that original pack-file won't even be broken apart.
>
> Oops, isn't that something we fixed recently as a "bug"?
Ehh, apparently. I had thought it was a feature (not that it was me who
implemented it), and didn't realize that others thought it was a bug.
Oops.
The default *.keep file was _wonderful_ for cloning a large tree onto a
small machine. It did exactly the right thing (never mind any shared
repositories - it just made repacking much more reasonable).
So maybe it was unintentional (a "bug"), but I had always seen it as being
something good.
Linus
^ permalink raw reply
* RE: Monotone workflow compared to Git workflow ( was RE: Git vs Monotone)
From: Craig L. Ching @ 2008-07-31 20:37 UTC (permalink / raw)
To: Linus Torvalds; +Cc: Git Mailinglist
In-Reply-To: <alpine.LFD.1.10.0807311253140.3277@nehalem.linux-foundation.org>
> From: Linus Torvalds [mailto:torvalds@linux-foundation.org]
> Sent: Thursday, July 31, 2008 3:09 PM
>
> Sure, if you want to keep the build tree around, you would
> probably not use branches.
>
I think we'd still use branches, but we just need to isolate their
workdirs from each other.
> But yes, then you'd likely do "git clone -s" with some single
> "common point" or use "git worktree". And even if you don't
> use "-s", you should _still_ effectively share at least all
> the old history (which tends to be the bulk) thanks to even a
> default "git clone" will just hardlink the pack-files.
>
> So literally, if you do
>
> git clone <cntral-repo-over-network> <local>
>
> and then do
>
> git clone <local> <otherlocal>
> git clone <local> <thirdlocal>
>
> then all of those will all share the initial pack-file
> on-disk. Try it.
>
> (You may then want to edit the "origin" branch info in the
> .git/config to point to the network one etc, of course).
>
Yes, thank you for the explanation. Having used git a fair amount now,
that makes perfect sense to me, in fact, it sounds a lot like
git-new-workdir, but I think I'll change our use of git-new-workdir to
something more "core" git. It seems to me that maybe this is something
that could be documented more prominently? Or maybe it is and I've just
missed it. This would have saved me a lot of time originally to be
sure.
> Oh, and to make sure I'm not lying I actually did test this,
> but I also noticed that "git clone" no longer marks the
> initial pack-file with "keep", so it looks like "git gc" will
> then break the link. That's sad. I wonder when that changed,
> or maybe I'm just confused and it never did.
>
What's the consequence of that then? Because of that, would you say
"don't gc your master local repo until all derived repos are merged?"
If that link is broken is it just a loss of space? Or is it more?
> Linus
>
Thanks again!
Cheers,
Craig
^ permalink raw reply
* RE: Git vs Monotone
From: Blum, Robert @ 2008-07-31 20:42 UTC (permalink / raw)
To: Linus Torvalds; +Cc: git@vger.kernel.org
In-Reply-To: <alpine.LFD.1.10.0807311244240.3277@nehalem.linux-foundation.org>
>The fact is, even without "-s", a local clone will do hardlinks for the
>database. And since the original pack-file is marked as a 'keep' file,
>that original pack-file won't even be broken apart.
Then again, Pidgin is, among other things, a Windows project. I.e. hardlinks are not exactly trivial. There's a good chance nobody jumped through the hoops of junction points for git on win32... (Somebody correct me if I'm wrong)
>So literally, if he had just bothered to even _try_ the git setup, he'd
>have noticed that git actually uses less disk than monotone would do. But
>it sounds like he didn't even try it.
Well, he *did* try it, for *one* repository. He just didn't know that there's a better way than having 11 clones. And I lay the blame for that squarely at the git documentation ;)
Yes, I know, why don't I make it better...?
Because I'm fairly new to git and would feel like an idiot 'documenting' something that I feel I've only scratched the surface of. I do expect to write a few uninformed rants on my blog, though. And maybe at some point, I can contribute to actual docs :)
Either way, it's another interesting data point for all of us still comparing DVCSs. I just wish he had comments on his blog so somebody could inform him that he's mistaken...
- Robert
^ permalink raw reply
* Re: Merging submodules
From: H.Merijn Brand @ 2008-07-31 20:44 UTC (permalink / raw)
To: Santi Béjar; +Cc: Petr Baudis, Brian Gernhardt, Git List, Lars Noschinski
In-Reply-To: <8aa486160807311203o3fb4deb8u4a5ae57818c76fab@mail.gmail.com>
On Thu, 31 Jul 2008 21:03:26 +0200, "Santi Béjar" <sbejar@gmail.com>
wrote:
> On Thu, Jul 31, 2008 at 20:15, H.Merijn Brand <h.m.brand@xs4all.nl> wrote:
> > On Thu, 31 Jul 2008 17:24:40 +0200, "Santi Béjar" <sbejar@gmail.com>
> > wrote:
> >
> >> I see all OK. Can you provide a self consistent simple testcase that
> >> shows what is wrong?
> >
> > Yes. was rather easy.
> > http://www.xs4all.nl/~hmbrand/testcase.tgz
> >
>
> It is because you cannot merge a branch with an empty branch.
Super. So I start with a .gitignore and continue this process. It
worked!
> So, or you create an initial commit in the "superproject" or you
> create a commit just moving the files of the first module as in:
>
> http://article.gmane.org/gmane.comp.version-control.git/79887
Thanks again!
--
H.Merijn Brand Amsterdam Perl Mongers http://amsterdam.pm.org/
using & porting perl 5.6.2, 5.8.x, 5.10.x, 5.11.x on HP-UX 10.20, 11.00,
11.11, 11.23, and 11.31, SuSE 10.1, 10.2, and 10.3, AIX 5.2, and Cygwin.
http://mirrors.develooper.com/hpux/ http://www.test-smoke.org/
http://qa.perl.org http://www.goldmark.org/jeff/stupid-disclaimers/
^ permalink raw reply
* Re: Monotone workflow compared to Git workflow ( was RE: Git vs Monotone)
From: Björn Steinbrink @ 2008-07-31 20:54 UTC (permalink / raw)
To: Linus Torvalds; +Cc: Craig L. Ching, sverre, Git Mailinglist
In-Reply-To: <alpine.LFD.1.10.0807311253140.3277@nehalem.linux-foundation.org>
On 2008.07.31 13:09:09 -0700, Linus Torvalds wrote:
>
>
> On Thu, 31 Jul 2008, Craig L. Ching wrote:
> >
> > We find ourselves constantly having to shift gears and work on other
> > things in the middle of whatever it is we're currently working on. For
> > instance, in the scenario above, A might be branch that contains a
> > feature going into our next release. B might be a bugfix and takes
> > priority over A, so you have to leave A as-is and start work on B. When
> > I come back to work on A, I have to rebuild A to continue working, and
> > that's just too expensive for us. So we use the monotone-like
> > new-workdir which allows us to save those build artifacts.
> >
> > So, that said, I ask again, am I missing something? Is there a better
> > way to do this? How do the kernel developers do this, surely they're
> > switching branches back and forth having to build in-between?
>
> Sure, if you want to keep the build tree around, you would probably not
> use branches.
>
> But yes, then you'd likely do "git clone -s" with some single "common
> point" or use "git worktree". And even if you don't use "-s", you should
> _still_ effectively share at least all the old history (which tends to be
> the bulk) thanks to even a default "git clone" will just hardlink the
> pack-files.
>
> So literally, if you do
>
> git clone <cntral-repo-over-network> <local>
Hum, I guess I'm just missing something and prepare to get flamed, but
wouldn't you want that one to be bare? Otherwise, the other clones won't
see all of the original repo's branches, right?
Maybe even better:
mkdir local-mirror
cd local-mirror
git --bare init
git remote add -f --mirror origin <central-repo-over-network>
A cronjob (or whatever) could keep the local mirror up-to-date and the
other repos can fetch from there. Pushing would need to go to a
different remote then though.. Humm... Maybe not worth the trouble for a
bit of additional object sharing.
Björn
^ permalink raw reply
* Re: [PATCH 0/2] gitweb use sections
From: Gustavo Sverzut Barbieri @ 2008-07-31 20:58 UTC (permalink / raw)
To: Petr Baudis; +Cc: git, Jakub Narebski
In-Reply-To: <20080731203250.GO10151@machine.or.cz>
On Thu, Jul 31, 2008 at 5:32 PM, Petr Baudis <pasky@suse.cz> wrote:
> Hi,
>
> On Thu, Jul 31, 2008 at 04:43:35PM -0300, Gustavo Sverzut Barbieri wrote:
>> Since nobody replied and I missed some gitweb guys in CC, I'm adding
>> Petr and Jakub, as some guys said on IRC.
>>
>> Have anyone tried this patch, any problems?
>
> sorry, I have it in my review queue. At first pass it was looking
> good, but I wanted to look at it better before commenting.
no problem, just to see it was noticed or not :-)
> One thing I'm wondering about is how to make this stuff configurable,
> since I'm not very comfortable with adding more "unbound" configuration
> variables and would rather prefer stuff to be added to the $features
> array... I'm not at all sure about my own sentiment here, however.
Path comparison (first patch), Sections (second), both?
I know path comparison can be a performance hit on large listings on
sites with heavy traffic. However, I don't see many people accessing
the projects page at the same time for long periods, it's not like
slashdot... people mostly use it to know about repositories and then
use git to track it.
The slowness is due O(n^2) worst case of sort and each step is not
a bit heavier since it need to split path into components and walk
these. Maybe cache the split?
--
Gustavo Sverzut Barbieri
http://profusion.mobi embedded systems
--------------------------------------
MSN: barbieri@gmail.com
Skype: gsbarbieri
Mobile: +55 (19) 9225-2202
^ permalink raw reply
* Re: Monotone workflow compared to Git workflow ( was RE: Git vs Monotone)
From: Sean Estabrooks @ 2008-07-31 20:57 UTC (permalink / raw)
To: Craig L. Ching; +Cc: Linus Torvalds, sverre, Git Mailinglist
In-Reply-To: <63BEA5E623E09F4D92233FB12A9F79430238A5EE@emailmn.mqsoftware.com>
On Thu, 31 Jul 2008 14:48:21 -0500
"Craig L. Ching" <cching@mqsoftware.com> wrote:
> I have a question about this. I asked this awhile back and didn't
> really get any satisfactory answers except to use git-new-workdir, which
> makes git behave a lot like monotone. In our workflow, we do create
> branches for nearly everything, but we do find that we have a need to
> keep the build artifacts of those branches isolated from each other
> because rebuilding is expensive. IOW, we have this sort of workflow:
Is there a problem using git-new-workdir? It sounds like it does
exactly what you want.
> We find ourselves constantly having to shift gears and work on other
> things in the middle of whatever it is we're currently working on. For
> instance, in the scenario above, A might be branch that contains a
> feature going into our next release. B might be a bugfix and takes
> priority over A, so you have to leave A as-is and start work on B. When
> I come back to work on A, I have to rebuild A to continue working, and
> that's just too expensive for us. So we use the monotone-like
> new-workdir which allows us to save those build artifacts.
>
> So, that said, I ask again, am I missing something? Is there a better
> way to do this? How do the kernel developers do this, surely they're
> switching branches back and forth having to build in-between?
A decent build system will only compile the source files that actually
changed when switching branches. Couple that with a compiler cache
(such as ccache) and switching between branches in the kernel or git
project usually isn't prohibitively time consuming.
Sean
^ permalink raw reply
* Re: markdown 2 man, was Re: Git Community Book
From: Jan Krüger @ 2008-07-31 20:57 UTC (permalink / raw)
To: Abdelrazak Younes
Cc: git, Johannes Schindelin, Julian Phillips, Scott Chacon,
Petr Baudis
In-Reply-To: <4891A0D0.6060503@lyx.org>
Hi,
> Disclaimer: I am involved in LyX development, so anything I said will
> be biased :-)
I think that's fine since I consider LaTeX (and therefore LyX as the
best graphical editor for it that I know) a choice always worth
considering when it comes to projects that have the size of a book.
> Now, about my shameless plug: LyX is ideally suited for structured
> documentation writing :-)
That may well be, but it gets really complicated once you want to
get your document into other markup-based formats while preserving all
the important aspects of formatting. I know this because I started
using LaTeX for a project that was supposed to be available in HTML
form along with, say, PDF. I've found that the only converter that
comes close to being useful for somewhat more ambitious sources
(including, perhaps, custom environments and stuff like that) without
spending a ridiculous amount of time trying to understand it is hevea.
Of course, hevea only translates to HTML, so, for example, generating
manpages or plain text is an entirely different matter of considerable
difficulty.
In addition to that, I suspect that LyX files might be difficult to
deal with in forky Git situations. For example, what if two
separately contributed patches need merging into a LyX source file?
This will only work automatically if the LyX source, treated as plain
text, has a really low chance of randomly changing in other places than
what the patch is supposed to touch. Also, if a merge does cause a
conflict, I imagine it would be difficult to resolve that.
Finally, it's pretty much a given that Git's manpages continue to use
AsciiDoc because there are few other things that can generate actual
manpages. I'm not sure it would be a good idea to keep half of Git's
documentation in one format and the rest in another. And AsciiDoc is --
by far! -- not the worst choice. I'm tempted to say it's the best that
I know.
-Jan
^ permalink raw reply
* Re: q: git-fetch a tad slow?
From: Ingo Molnar @ 2008-07-31 21:03 UTC (permalink / raw)
To: Shawn O. Pearce; +Cc: git
In-Reply-To: <20080731044531.GB1860@spearce.org>
* Shawn O. Pearce <spearce@spearce.org> wrote:
> Ingo Molnar <mingo@elte.hu> wrote:
> > alas, fetching still seems to be slow:
> >
> > titan:~/tip> time git-fetch origin
> >
> > real 0m5.112s
> > user 0m0.972s
> > sys 0m3.380s
>
> What version of git are dealing with on the client side?
the client side on titan has:
titan:~> git version
git version 1.5.2.2
oldish but not outrageously old, right?
server side has:
earth4:~> git version
git version 1.5.6.1.108.g660379
>
> fetch times of ~472 ms over git:// to your -tip.git tree and ~128 ms
> for strictly local fetch. If your SSH overhead is ~300 ms this is
> only a ~700 ms real time for `git fetch origin`, not 5100 ms.
>
> Is your git-fetch a shell script? Or a compiled binary? The port
> into C made it go _much_ faster, even though it is still a naive
> O(N^2) matching algorithm. Yea, we still should fix that, but I think
> an upgrade to 1.5.4 or later would make the client side improve
> consideribly.
ah, it is a shell script indeed! I'll upgrade to latest.
Ingo
^ permalink raw reply
* Re: Monotone workflow compared to Git workflow ( was RE: Git vs Monotone)
From: Avery Pennarun @ 2008-07-31 21:10 UTC (permalink / raw)
To: Björn Steinbrink
Cc: Linus Torvalds, Craig L. Ching, sverre, Git Mailinglist
In-Reply-To: <20080731205400.GA7911@atjola.homenet>
On 7/31/08, Björn Steinbrink <B.Steinbrink@gmx.de> wrote:
> Maybe even better:
>
> mkdir local-mirror
> cd local-mirror
> git --bare init
> git remote add -f --mirror origin <central-repo-over-network>
>
> A cronjob (or whatever) could keep the local mirror up-to-date and the
> other repos can fetch from there. Pushing would need to go to a
> different remote then though.. Humm... Maybe not worth the trouble for a
> bit of additional object sharing.
What would be *really* great is if we could find a way for multiple
local clones to share the same objects, refs, and configuration - ie.
without pushing and pulling between them at all. Then they could
*all* point at the remote upstream repo through "origin", and
pushing/pulling with that repo would update the objects and refs for
all the local repos.
I'm not sure of the best way to do this, though. In particular, it
seems like having multiple work trees checked out on the same ref
could be problematic.
Is that just what git-new-workdir is for? (It seems to be
undocumented so it's hard to tell.) And what about this
.gitlink/.gitfile stuff I've heard about? Could I use that to have
multiple work trees share the same .git folder?
Thanks,
Avery
^ permalink raw reply
* Re: q: git-fetch a tad slow?
From: Ingo Molnar @ 2008-07-31 21:11 UTC (permalink / raw)
To: Shawn O. Pearce; +Cc: git
In-Reply-To: <20080731210307.GF25138@elte.hu>
* Ingo Molnar <mingo@elte.hu> wrote:
> > for strictly local fetch. If your SSH overhead is ~300 ms this is
> > only a ~700 ms real time for `git fetch origin`, not 5100 ms.
> >
> > Is your git-fetch a shell script? Or a compiled binary? The port
> > into C made it go _much_ faster, even though it is still a naive
> > O(N^2) matching algorithm. Yea, we still should fix that, but I
> > think an upgrade to 1.5.4 or later would make the client side
> > improve consideribly.
>
> ah, it is a shell script indeed! I'll upgrade to latest.
on another box, with 1.5.4, i have:
dione:~/tip> time git fetch origin
real 0m0.481s
user 0m0.136s
sys 0m0.060s
dione:~/tip> time ./tip-fetch
b714d1a257cca93ba6422ca3276ac80a2cde2b59
b714d1a257cca93ba6422ca3276ac80a2cde2b59
real 0m0.273s
user 0m0.012s
sys 0m0.020s
that's a 2.66 GHz core2 quad, i.e. a pretty fast box too. As you can see
most time spent in the tip-fetch case was waiting for the network. So
there's about 200 msecs of extra CPU cost on the local side. On a CPU
1-2 generations older that could be up to 1000 msecs or more.
In any case, performance has improved significantly with the C version!
(I'll still use tip-fetch to squeeze out the last bit of performance,
but it's quite comparable now.)
Sorry that i didnt notice that titan had 1.5.2 - i almost never notice
it when i switch between stable git versions. (you guys are doing a
really good job on compatibility)
Ingo
^ permalink raw reply
* Re: Monotone workflow compared to Git workflow ( was RE: Git vs Monotone)
From: Linus Torvalds @ 2008-07-31 21:13 UTC (permalink / raw)
To: Björn Steinbrink; +Cc: Craig L. Ching, sverre, Git Mailinglist
In-Reply-To: <20080731205400.GA7911@atjola.homenet>
On Thu, 31 Jul 2008, Björn Steinbrink wrote:
> >
> > So literally, if you do
> >
> > git clone <cntral-repo-over-network> <local>
>
> Hum, I guess I'm just missing something and prepare to get flamed, but
> wouldn't you want that one to be bare? Otherwise, the other clones won't
> see all of the original repo's branches, right?
Making it bare might be a good idea for other reasons too (it makes it
much more obvious that it's a "local clone" and is somehow special). But
it's really a matter of taste - and the project - exactly how you do it.
For example, the kernel only has a single master branch in the top repo,
so there it really doesn't matter, and yes, I'm more kernel-oriented than
anything else, of course.
But I don't think it's exactly wrong to have the initial clone be a real
repository that you do work in. Quite often the history really is the
_bulk_ of the database by far (at least with projects that have big enough
repositories for this to even matter in the first place!), and as long as
you just download that once and share that thing, you're already ahead of
the game and the rest is really just details.
> Maybe even better:
>
> mkdir local-mirror
> cd local-mirror
> git --bare init
> git remote add -f --mirror origin <central-repo-over-network>
>
> A cronjob (or whatever) could keep the local mirror up-to-date and the
> other repos can fetch from there.
Heh. You can certainly do it many ways. I suspect the _easiest_ model is
actually to do one single local repo that is special (and perhaps bare),
and then you can clone all the other ones with
git clone --reference <local-reference> <remote> <new-local>
because that will automatically set up the new local repo to have the
local reference as an alternates thing, and will avoid downloading
unnecessary stuff.
So my point about the eleven repos was not that it's the best way to do
one remote clone and then eleven local ones - my point was that even if
you do that _stupid_ thing, you'd have seen sharing without even knowing
what you really did.
If you want to explicitly share, I think the local bare reference and
using "git clone --reference" is the best way. It sets up a special
link-file (it's just a text-file that git knows about, so it should work
fine under Windows too - no need for filesystem support) in
.git/objects/info/alternates.
IOW, git-clone --reference works like "git clone -s", but does so with one
special local database, while allowing you to clone from anywhere. Very
convenient.
And no, I don't think we document all these "tricks" very well. Partly
because people are _already_ complaining about how git can do so many
things ;) But partly because if you don't know what you're doing, the
"tricks" are often things you really need to understand, and can be a bit
dangerous otherwise.
For example, the "git clone -s" (or --reference) thing is *very* useful,
but one result of other repositories then sharing a database with the
reference one is that suddenly the reference repo is very special. You
must not remove it (obviously!), but you also must not rebase it and prune
it etc.
So all the normal git workflows are at least designed to be _safe_ even in
the absense of people not knowing what they are doing. The duplication may
be using harddisk space, but
- quite often the checkout is actually an even bigger issue, and the git
repo is small enough that lots of people don't really worry.
- duplicating the repo also means that you cannot _possibly_ screw up
other people/repos and does give you a kind of backup (even if
same-disk backups are obviously of dubious use: they shouldn't be your
_primary_ backup, but having multiple copies on a single disk still
protects against a _lot_ of problems)
so... It's a trade-off.
Linus
^ permalink raw reply
* Re: q: git-fetch a tad slow?
From: Shawn O. Pearce @ 2008-07-31 21:19 UTC (permalink / raw)
To: Ingo Molnar; +Cc: git
In-Reply-To: <20080731211141.GA1159@elte.hu>
Ingo Molnar <mingo@elte.hu> wrote:
>
> on another box, with 1.5.4, i have:
>
> dione:~/tip> time git fetch origin
>
> real 0m0.481s
> user 0m0.136s
> sys 0m0.060s
>
> dione:~/tip> time ./tip-fetch
> b714d1a257cca93ba6422ca3276ac80a2cde2b59
> b714d1a257cca93ba6422ca3276ac80a2cde2b59
>
> real 0m0.273s
> user 0m0.012s
> sys 0m0.020s
>
> that's a 2.66 GHz core2 quad, i.e. a pretty fast box too. As you can see
> most time spent in the tip-fetch case was waiting for the network. So
> there's about 200 msecs of extra CPU cost on the local side.
Yea. My testing last night was suggesting about 1/2 of that 200
ms is on the client, and the other 200 ms is on the server side
of the connection. That matches up somewhat with your test above,
where git-fetch used about 100 ms more user time on the client side
than your tip-fetch shell script.
I have no clue where the bottleneck is, I didn't get that far before
I realized you must have been running a shell script based git-fetch
to be seeing the performance you were.
Maybe 1.6.1 or .2 we can try to squeeze fetch to run faster.
Its far too late for 1.6.0.
> Sorry that i didnt notice that titan had 1.5.2 - i almost never notice
> it when i switch between stable git versions. (you guys are doing a
> really good job on compatibility)
Yea, its easy to not realize your git isn't giving you the latest
and greatest toys. ;-)
--
Shawn.
^ permalink raw reply
* Re: Monotone workflow compared to Git workflow ( was RE: Git vs Monotone)
From: Theodore Tso @ 2008-07-31 21:22 UTC (permalink / raw)
To: Sean Estabrooks; +Cc: Craig L. Ching, Linus Torvalds, sverre, Git Mailinglist
In-Reply-To: <BLU0-SMTP273E4683B41DB7E44122F0AE7C0@phx.gbl>
On Thu, Jul 31, 2008 at 04:57:24PM -0400, Sean Estabrooks wrote:
> > We find ourselves constantly having to shift gears and work on other
> > things in the middle of whatever it is we're currently working on. For
> > instance, in the scenario above, A might be branch that contains a
> > feature going into our next release. B might be a bugfix and takes
> > priority over A, so you have to leave A as-is and start work on B. When
> > I come back to work on A, I have to rebuild A to continue working, and
> > that's just too expensive for us. So we use the monotone-like
> > new-workdir which allows us to save those build artifacts.
>
> A decent build system will only compile the source files that actually
> changed when switching branches. Couple that with a compiler cache
> (such as ccache) and switching between branches in the kernel or git
> project usually isn't prohibitively time consuming.
That being said, if the bugfix is on a "maint" branch, and one of the
things that has changed is a header file that forces most of the
project to be recompiled, a separate work directory may be more
convenient. Of course, a separate work directory (whether created
using "git clone -s" or "git-new-workdir" means more disk space and it
means greater use of the page cache or a slowdown while the different
sets of sources get paged in and out. Of course, you could hack
git-work-dir to use cp -rl to initially copy the working directory
using hard links, and then when the new branch is checked out, if most
of the files haven't changed, the files in the working directory could
be shared too. A lot depends on how much you want to squeeze the last
bit of hard drive and speed optimization, and how big your project is.
- Ted
^ permalink raw reply
* Re: linking libgit.a in C++ projects
From: cte @ 2008-07-31 21:31 UTC (permalink / raw)
To: Boaz Harrosh; +Cc: git
In-Reply-To: <4891B872.3040707@panasas.com>
Excellent idea; thank you!
> The practice of avoiding C++ keywords from public C headers is
> very welcome. You should send a patch and try to push it.
>
> That said the problem can be easily avoided.
>
> Produce a C file and header that defines some stable API to your
> GUI application, that does not expose any git internal headers.
> Then compile that, say git_api.c, with C compiler in Makefile
> and extern "C" link that file to your C++ application. This will
> completely insulate you from any git code.
>
> This could also solve the other problem of API changing, only
> the git_api.c need change, your outer GUI code stays the same.
>
> And if you do all that maybe you can submit it for inclusion
> as a: somewhat stable high-level library, for developers.
> Ala git-dev
>
> Cheers
> Boaz
>
>
^ permalink raw reply
* Re: Monotone workflow compared to Git workflow ( was RE: Git vs Monotone)
From: Martin Langhoff @ 2008-07-31 21:43 UTC (permalink / raw)
To: Björn Steinbrink
Cc: Linus Torvalds, Craig L. Ching, sverre, Git Mailinglist
In-Reply-To: <20080731205400.GA7911@atjola.homenet>
On Fri, Aug 1, 2008 at 8:54 AM, Björn Steinbrink <B.Steinbrink@gmx.de> wrote:
>> So literally, if you do
>>
>> git clone <cntral-repo-over-network> <local>
>
> Hum, I guess I'm just missing something and prepare to get flamed, but
> wouldn't you want that one to be bare? Otherwise, the other clones won't
> see all of the original repo's branches, right?
Yes, that's why
git clone --reference /path/to/fat/checkout/.git/ <central-repo>
is far better. Each "thin" checkout sees the central repo normally,
but they borrow the object store from the referenced local "fat"
checkout.
cheers,
m
--
martin.langhoff@gmail.com
martin@laptop.org -- School Server Architect
- ask interesting questions
- don't get distracted with shiny stuff - working code first
- http://wiki.laptop.org/go/User:Martinlanghoff
^ permalink raw reply
* Re: linking libgit.a in C++ projects
From: cte @ 2008-07-31 21:44 UTC (permalink / raw)
To: sverre; +Cc: Dmitry Potapov, git
In-Reply-To: <bd6139dc0807311127j57d9ab5ckd6acf16d17621614@mail.gmail.com>
On Thu, Jul 31, 2008 at 11:27 AM, Sverre Rabbelier <alturin@gmail.com> wrote:
> On Thu, Jul 31, 2008 at 13:10, cte <cestreich@gmail.com> wrote:
>> I'm not worried about the interfaces changing; the gui is tied to a
>> particular version of git, and I will update the code that calls into
>> libgit I pull new changes from the mainline into my local clone.
>
> You should be ;). Unless you are planning to learn a lot of C very
> fast, you should be worried about the interfaces changing. That is, if
> you want your GUI to be able to stay up to date with the current git
> version.
That is the plan.
>> who's to say that the output of the various commands won't change
>> formats with future releases of git?
>
> Junio is to say. Plumbing output format is git's API.
Using output from the command line utilities as an API has its own set
of problems. For instance, check out some of the difficulties that
gitk and qgit have had to deal with:
http://kerneltrap.org/mailarchive/git/2007/11/2/379067. Digging into
the git internals and reusing its core functions will always be more
powerful and flexible than parsing command line output. Of course, it
is not always easy; git wasn't written to be easily compiled into a
library and reused (graceful error handling and memory management are
problematic). But I think the right thing to do is to work towards
making the awesome git internals easier to use for other developers so
great tools can continue to be built on top of git.
>> There is no correct solution if
>> you are worried about forward compatibility, unless a well defined API
>> is created (which would be sweet btw, but is probably not a priority).
>
> There is, use the plumbing, forward compatibility is 95% assured. With
> the exception of major releases, for which any plumbing
> output/behavior changes will be announced in the changelog, usually
> including an explanation on how to change your code to match.
95% assured != correct, IMO :)
> In short, use the forc-... errr, plumbing ;).
>
> --
> Cheers,
>
> Sverre Rabbelier
>
^ permalink raw reply
* "remote add -f --mirror" fails when the stash is non-empty
From: Björn Steinbrink @ 2008-07-31 21:50 UTC (permalink / raw)
To: git
Hi,
when there are stashed changes in a repo, creating a mirror with "remote
add -f --mirror" fails, because it is missing the objects that are on
the stash.
$ git remote add -f --mirror or ~/src/git
Updating or
remote: Counting objects: 80150, done.
remote: Compressing objects: 100% (21254/21254), done.
remote: Total 80150 (delta 57481), reused 80146 (delta 57477)
Receiving objects: 100% (80150/80150), 21.21 MiB | 16028 KiB/s, done.
Resolving deltas: 100% (57481/57481), done.
From /home/doener/src/git
* [new branch] master -> master
* [new branch] rebase -> rebase
* [new branch] origin/HEAD -> origin/HEAD
* [new branch] origin/html -> origin/html
* [new branch] origin/maint -> origin/maint
* [new branch] origin/man -> origin/man
* [new branch] origin/master -> origin/master
* [new branch] origin/next -> origin/next
* [new branch] origin/pu -> origin/pu
* [new branch] origin/todo -> origin/todo
error: unable to find 9e5c2066d7d2bdca937a11b45a7ed8354c637bc9
fatal: object 9e5c2066d7d2bdca937a11b45a7ed8354c637bc9 not found
error: Could not fetch or
In ~/src/git:
$ git rev-parse stash
9e5c2066d7d2bdca937a11b45a7ed8354c637bc9
No idea if that is a recent breakage, as I learned about the --mirror
option recently and most of the time my stash is empty anyway.
Björn
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox