* Re: [PATCH] graph API: fix bug in graph_is_interesting()
From: Adam Simpkins @ 2009-08-19 22:55 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Santi Béjar, Git Mailing List
In-Reply-To: <7v4os41frm.fsf@alter.siamese.dyndns.org>
On Tue, Aug 18, 2009 at 11:25:49PM -0700, Junio C Hamano wrote:
> Junio C Hamano <gitster@pobox.com> writes:
> >
> > When simplify_commit() logic (now called get_comit_action()) decides to
> > show this commit because revs->show_all was specified, we did not rewrite
> > its parents, but now we will?
>
> That is, here is what I meant...
>
> - if (action == commit_show && revs->prune && revs->dense && want_ancestry(revs)) {
> + if (action == commit_show &&
> + !revs->show_all &&
> + revs->prune && revs->dense && want_ancestry(revs)) {
>
> We may want to add some tests to demonstrate the breakage this fix
> addresses.
Yes, you're right. Thanks for catching that. I'll submit a test case
that checks this scenario.
--
Adam Simpkins
simpkins@facebook.com
^ permalink raw reply
* Re: Continue git clone after interruption
From: René Scharfe @ 2009-08-19 22:23 UTC (permalink / raw)
To: Nicolas Pitre; +Cc: Johannes Schindelin, Jakub Narebski, Tomasz Kontusz, git
In-Reply-To: <alpine.LFD.2.00.0908191122020.6044@xanadu.home>
Nicolas Pitre schrieb:
> 3) "git archive --remote=git://foo.bar/baz CLONE_HEAD" and store the
> result locally. Keep track of how many files are received, and how
> many bytes for the currently received file.
>
> 4) if network connection is broken, loop back to (3) adding
> --skip=${nr_files_received},${nr_bytes_in_curr_file_received} to
> the git-archive argument list. REmote server simply skips over
> specified number of files and bytes into the next file.
>
> 5) Get content from remote commit object for CLONE_HEAD somehow. (?)
[...]
> - That probably would be a good idea to have a tgz format to 'git
> archive' which might be simpler to deal with than the zip format.
Adding support for the tgz format would be useful anyway, I guess, and
is easy to implement.
And adding support for cpio (and cpio.gz) and writing an extractor for
it should be simpler than writing a tar extractor alone.
One needs to take a closer look at the limits of the chosen archive
format (file name length, supported file types and attributes, etc.) to
make sure any archive can be turned back into the same git tree.
The commit object could be sent as the first (fake) file of the archive.
You'd need a way to turn off the effect of the attributes export-subst
and export-ignore.
Currently, convert_to_working_tree() is used on the contents of all
files in an archive. You'd need a way to turn that off, too.
Adding a new format type is probably the easiest way to bundle the
special requirements of the previous three paragraphs.
René
^ permalink raw reply
* Re: question concerning branches
From: Linus Torvalds @ 2009-08-19 21:51 UTC (permalink / raw)
To: Ingo Brueckl; +Cc: Jakub Narebski, git
In-Reply-To: <4a8c51f5@wupperonline.de>
On Wed, 19 Aug 2009, Ingo Brueckl wrote:
> Jakub Narebski <jnareb@gmail.com> writes:
>
> > You finish old work (or stash it away), _then_ you begin new work.
>
> Ok, this helps me a little bit to understand.
>
> The branches aren't designed to split my work, but rather something to
> collect the different parts of my work.
Hmm. Yes. That's one way of looking at it.
At the same time, thinking about it another way may explain the git
choices in this area. There's two issues:
- if we _don't_ carry the edits around across branch switches, then what
would we do?
Basically, since you haven't committed things, it's kind of floating
around. You switch to another branch, what should we do? There are
really only two choices: either we'd need to 'stash' the state with the
branch we switch away from (which is apparently what you expected), or
we need to just move the changes to the new branch (which is what git
does, or complains if it cannot).
Now, 'stashing' the changes is actually very much against the whole git
philosophy. Git was built up around the index and the database, and
branches have always been pointers to the top-of-commit, so there
literally isn't any way to stash things that makes sense. Sure, later
on we ended up having the 'stash' command, but that's totally separate
from branches, and is an independently useful thing.
- One of the big reasons to act like git does is that the way at least
_I_ work is to actually create a new branch with the explicit intention
of committing work I have already done!
IOW, your example was
git branch test
git checkout test
# edit foo.bar
git checkout master
and you were surprised that the edit followed you back to the "master"
branch, but what is actualyl a much more natural way of working is
# edit foo.bar
# realize that this was actually the start of a new feature
git branch new-feature
git checkout new-feature
# maybe continue to edit foo.bar until it's all good
git commit -a
ie the git behavior explicitly _encourages_ you to not have to decide
before-the-fact to create a branch - it may be that only after you've
done the changes do you realize that "oops, these changes were _way_
more intrusive than I originally anticipated, and I don't want to
commit them on the master branch, I want to commit them on an
experimental topic branch instead"
So there are two different reasons why git works the way it does: a pure
implementation reason ("working any other way would not fit the git
model") and a practical workflow reason ("you are _expected_ to move dirty
state around with your branches, because one common case is to create a
branch _for_ that dirty state").
Linus
^ permalink raw reply
* Re: Continue git clone after interruption
From: Nicolas Pitre @ 2009-08-19 21:13 UTC (permalink / raw)
To: Jakub Narebski; +Cc: Tomasz Kontusz, git, Johannes Schindelin
In-Reply-To: <200908192142.51384.jnareb@gmail.com>
On Wed, 19 Aug 2009, Jakub Narebski wrote:
> Cc-ed Dscho, so he can easier participate in this subthread.
>
> On Wed, 19 Aug 2009, Nicolas Pitre wrote:
> > On Wed, 19 Aug 2009, Jakub Narebski wrote:
>
> > > P.S. What do you think about 'bundle' capability extension mentioned
> > > in a side sub-thread?
> >
> > I don't like it. Reason is that it forces the server to be (somewhat)
> > stateful by having to keep track of those bundles and cycle them, and it
> > doubles the disk usage by having one copy of the repository in the form
> > of the original pack(s) and another copy as a bundle.
>
> I agree about problems with disk usage, but I disagree about server
> having to be stateful; server can just simply scan for bundles, and
> offer links to them if client requests 'bundles' capability, somewhere
> around initial git-ls-remote list of refs.
But that's the client that has to deal with what the server wants to
offer, instead of the server actually serving data as the client wants.
> Well, offering daily bundle in addition to daily snapshot could be
> a good practice, at least until git acquires resumable fetch (resumable
> clone).
Outside of Git: maybe. Through the git protocol: no. And what would
that bundle contain over the daily snapshot? The whole history? If so
that goes against the idea that people concerned by all this have slow
links and probably aren't interested in the time to download it all. If
the bundle contains only the top revision then it has no advantage over
the snapshot. Somewhere in the middle? Sure, but then where to draw
the line? That's for the client to decide, not the server
administrator.
And what if you start your slow transfer which breaks in the middle.
The next morning you want to restart it in the hope that you might
resume the transfer of the bundle that is incomplete. But crap, the
server has updated its bundle and your half-bundle is now useless.
You've wasted your bandwidth for nothing.
> > If you think about git.kernel.org which has maybe hundreds of
> > repositories where the big majority of them are actually forks of Linus'
> > own repository, then having all those forks reference Linus' repository
> > is a big disk space saver (and IO too as the referenced repository is
> > likely to remain cached in memory). Having a bundle ready for each of
> > them will simply kill that space advantage, unless they all share the
> > same bundle.
>
> I am thinking about sharing the same bundle for related projects.
... meaning more administrative burden.
> > Now sharing that common bundle could be done of course, but that makes
> > things yet more complex while still wasting IO because some requests
> > will hit the common pack and some others will hit the bundle, making
> > less efficient usage of the disk cache on the server.
>
> Hmmm... true (unless bundles are on separate server).
... meaning additional but avoidable costs.
> > Yet, that bundle would probably not contain the latest revision if it is
> > only periodically updated, even less so if it is shared between multiple
> > repositories as outlined above. And what people with slow/unreliable
> > network links are probably most interested in is the latest revision and
> > maybe a few older revisions, but probably not the whole repository as
> > that is simply too long to wait for. Hence having a big bundle is not
> > flexible either with regards to the actual data transfer size.
>
> I agree that bundle would be useful for restartable clone, and not
> useful for restartable fetch. Well, unless you count (non-existing)
> GitTorrent / git-mirror-sync as this solution... ;-)
I don't think fetches after a clone are such an issue. They are
typically transfers being orders of magnitude smaller than the initial
clone. Same goes for fetches to deepen a shallow clone which are in
fact fetches going back in history instead of forward. I still stands
by my assertion that bundles are suboptimal for a restartable clone.
As for GitTorrent / git-mirror-sync... those are still vaporwares to me
and I therefore have doubts about their actual feasability. So no, I
don't count on them.
> > Hence having a restartable git-archive service to create the top
> > revision with the ability to cheaply (in terms of network bandwidth)
> > deepen the history afterwards is probably the most straight forward way
> > to achieve that. The server needs no be aware of separate bundles, etc.
> > And the shared object store still works as usual with the same cached IO
> > whether the data is needed for a traditional fetch or a "git archive"
> > operation.
>
> It's the "cheaply deepen history" that I doubt would be easy. This is
> the most difficult part, I think (see also below).
Don't think so. Try this:
mkdir test
cd test
git init
git fetch --depth=1 git://git.kernel.org/pub/scm/git/git.git
REsult:
remote: Counting objects: 1824, done.
remote: Compressing objects: 100% (1575/1575), done.
Receiving objects: 100% (1824/1824), 3.01 MiB | 975 KiB/s, done.
remote: Total 1824 (delta 299), reused 1165 (delta 180)
Resolving deltas: 100% (299/299), done.
From git://git.kernel.org/pub/scm/git/git
* branch HEAD -> FETCH_HEAD
You'll get the very latest revision for HEAD, and only that. The size
of the transfer will be roughly the size of a daily snapshot, except it
is fully up to date. It is however non resumable in the event of a
network outage. My proposal is to replace this with a "git archive"
call. It won't get all branches, but for the purpose of initialising
one's repository that should be good enough. And the "git archive" can
be fully resumable as I explained.
Now to deepen that history. Let's say you want 10 more revisions going
back then you simply perform the fetch again with a --depth=10. Right
now it doesn't seem to work optimally, but the pack that is then being
sent could be made of deltas against objects found in the commits we
already have. Currently it seems that a pack that also includes those
objects we already have in addition to those we want is created, which
is IMHO a flaw in the shallow support that shouldn't be too hard to fix.
Each level of deepening should then be as small as standard fetches
going forward when updating the repository with new revisions.
> > Why "git archive"? Because its content is well defined. So if you give
> > it a commit SHA1 you will always get the same stream of bytes (after
> > decompression) since the way git sort files is strictly defined. It is
> > therefore easy to tell a remote "git archive" instance that we want the
> > content for commit xyz but that we already got n files already, and that
> > the last file we've got has m bytes. There is simply no confusion about
> > what we've got already, unlike with a partial pack which might need
> > yet-to-be-received objects in order to make sense of what has been
> > already received. The server simply has to skip that many files and
> > resume the transfer at that point, independently of the compression or
> > even the archive format.
>
> Let's reiterate it to check if I understand it correctly:
>
> Any "restartable clone" / "resumable fetch" solution must begin with
> a file which is rock-solid stable wrt. reproductability given the same
> parameters. git-archive has this feature, packfile doesn't (so I guess
> that bundle also doesn't, unless it was cached / saved on disk).
Right.
> It would be useful if it was possible to generate part of this rock-solid
> file for partial (range, resume) request, without need to generate
> (calculate) parts that client already downloaded. Otherwise server has
> to either waste disk space and IO for caching, or waste CPU (and IO)
> on generating part which is not needed and dropping it to /dev/null.
> git-archive you say has this feature.
"Could easily have" is more appropriate.
> Next you need to tell server that you have those objects got using
> resumable download part ("git archive HEAD" in your proposal), and
> that it can use them and do not include them in prepared file/pack.
> "have" is limited to commits, and "have <sha1>" tells server that
> you have <sha1> and all its prerequisites (dependences). You can't
> use "have <sha1>" with git-archive solution. I don't know enough
> about 'shallow' capability (and what it enables) to know whether
> it can be used for that. Can you elaborate?
See above, or Documentation/technical/shallow.txt.
> Then you have to finish clone / fetch. All solutions so far include
> some kind of incremental improvements. My first proposal of bisect
> fetching 1/nth or predefined size pack is buttom-up solution, where
> we build full clone from root commits up. You propose, from what
> I understand build full clone from top commit down, using deepening
> from shallow clone. In this step you either get full incremental
> or not; downloading incremental (from what I understand) is not
> resumable / they do not support partial fetch.
Right. However, like I said, the incremental part should be much
smaller and therefore less susceptible to network troubles.
Nicolas
^ permalink raw reply
* Re: question concerning branches
From: Jakub Narebski @ 2009-08-19 20:57 UTC (permalink / raw)
To: Theodore Tso; +Cc: Ingo Brueckl, git
In-Reply-To: <20090819203917.GH27206@mit.edu>
Theodore Tso wrote:
> Personally, in the cases where I can't finish a commit before I need
> to switch away to another branch, my preference is to not use "git
> stash", but instead to create a topic branch, and then check in a
> partially completed change on the topic branch, which I can later
> ammend using "git commit --amend" (or if I have multiple commits on
> the topic branch, "git rebase --interactive"). This is because I can
> use the commit description to leave myself some notes about what still
> needs to be done before the commit can be finalized.
Errr... you are aware that you can use "git stash save <message>" (i.e.
specify commit message for stash; well, the subject), don't you?
--
Jakub Narebski
Poland
^ permalink raw reply
* Re: question concerning branches
From: Theodore Tso @ 2009-08-19 20:39 UTC (permalink / raw)
To: Ingo Brueckl; +Cc: Jakub Narebski, git
In-Reply-To: <4a8c51f5@wupperonline.de>
On Wed, Aug 19, 2009 at 09:45:00PM +0200, Ingo Brueckl wrote:
> Jakub Narebski <jnareb@gmail.com> writes:
>
> > You finish old work (or stash it away), _then_ you begin new work.
>
> Ok, this helps me a little bit to understand.
>
> The branches aren't designed to split my work, but rather something to
> collect the different parts of my work.
>
> But as software development often is something where you are coding on
> several issues at the same time which can't be committed immediately, it
> sounds that 'stash' is the developer's best friend.
Context switching has overhead; so it's usually better to try to
complete one task before switching to another. Granted, sometimes it
can't be done, but it's something you should really try to do.
Also, commits are easier to review if they are kept small; if you
localize changes into separate commits, it's often easier to detet
problems when doing "git bisect", for example. So if you are often
needing to switch while leaving something that isn't ready to be
committed, you might want to ask yourself if you are putting too many
changes into a single ocmmit.
Personally, in the cases where I can't finish a commit before I need
to switch away to another branch, my preference is to not use "git
stash", but instead to create a topic branch, and then check in a
partially completed change on the topic branch, which I can later
ammend using "git commit --amend" (or if I have multiple commits on
the topic branch, "git rebase --interactive"). This is because I can
use the commit description to leave myself some notes about what still
needs to be done before the commit can be finalized.
- Ted
^ permalink raw reply
* Re: question concerning branches
From: Jakub Narebski @ 2009-08-19 20:01 UTC (permalink / raw)
To: Ingo Brueckl; +Cc: git, Avery Pennarun
In-Reply-To: <4a8c51f5@wupperonline.de>
On Wed, 19 Aug 2009, Ingo Brueckl wrote:
> Jakub Narebski <jnareb@gmail.com> writes:
>
> > You finish old work (or stash it away), _then_ you begin new work.
>
> Ok, this helps me a little bit to understand.
>
> The branches aren't designed to split my work, but rather something to
> collect the different parts of my work.
Well, git is flexible enough that it can support also the workflow you
tried to use.
Namely you can have many working directories tied to single repository
(each of those checkouts should be of different branch). You can use
git-new-workdir script from contrib/worktree for that. Then to switch
branches you would just cd to appropriate directory (and keep unsaved
changes and untracked files). That said it is [much] less used
workflow.
> But as software development often is something where you are coding on
> several issues at the same time which can't be committed immediately,
> it sounds that 'stash' is the developer's best friend.
Well, you can also commit and then clean up history with interactive
rebase (or patch management interface such as StGit or Guilt). In
distributed version control systems like Git the act of publishing
changes is separate from the act of committing them (you should not
rewrite published history, though).
--
Jakub Narebski
Poland
^ permalink raw reply
* Re: question concerning branches
From: Jacob Helwig @ 2009-08-19 19:53 UTC (permalink / raw)
To: Ingo Brueckl; +Cc: Jakub Narebski, git
In-Reply-To: <4a8c51f5@wupperonline.de>
On Wed, Aug 19, 2009 at 12:45, Ingo Brueckl<ib@wupperonline.de> wrote:
>
> But as software development often is something where you are coding on
> several issues at the same time which can't be committed immediately, it
> sounds that 'stash' is the developer's best friend.
>
> Ingo
>
There is no problem with having temporary commits on local branches,
however.
Quite frequently, I'll "git commit -a -m 'Temp commit'; git checkout
other-branch". As long as you don't make these temporary commits
public, it's very easy to munge them (See: "git rebase --interactive
<commitish>", and "git reset --soft <commitish>").
-Jacob
^ permalink raw reply
* Re: question concerning branches
From: Avery Pennarun @ 2009-08-19 19:50 UTC (permalink / raw)
To: Ingo Brueckl; +Cc: Jakub Narebski, git
In-Reply-To: <4a8c51f5@wupperonline.de>
On Wed, Aug 19, 2009 at 7:45 PM, Ingo Brueckl<ib@wupperonline.de> wrote:
> Jakub Narebski <jnareb@gmail.com> writes:
>
>> You finish old work (or stash it away), _then_ you begin new work.
>
> Ok, this helps me a little bit to understand.
>
> The branches aren't designed to split my work, but rather something to
> collect the different parts of my work.
>
> But as software development often is something where you are coding on
> several issues at the same time which can't be committed immediately, it
> sounds that 'stash' is the developer's best friend.
Or you could just 'commit' more frequently, but don't 'push' so you're
not disturbing anyone else until you're done.
This is a big difference from how centralized VCSs work: there, a
commit is a major operation that you're afraid to do in case you make
someone else mad. In git, commits are cheap, you just need to be
careful about pushing.
(You can also clean up your series of commits before pushing by using
'git rebase')
Have fun,
Avery
^ permalink raw reply
* question concerning branches
From: Ingo Brueckl @ 2009-08-19 19:45 UTC (permalink / raw)
To: Jakub Narebski; +Cc: git
In-Reply-To: <m33a7noc3u.fsf@localhost.localdomain>
Jakub Narebski <jnareb@gmail.com> writes:
> You finish old work (or stash it away), _then_ you begin new work.
Ok, this helps me a little bit to understand.
The branches aren't designed to split my work, but rather something to
collect the different parts of my work.
But as software development often is something where you are coding on
several issues at the same time which can't be committed immediately, it
sounds that 'stash' is the developer's best friend.
Ingo
^ permalink raw reply
* Re: Continue git clone after interruption
From: Jakub Narebski @ 2009-08-19 19:42 UTC (permalink / raw)
To: Nicolas Pitre; +Cc: Tomasz Kontusz, git, Johannes Schindelin
In-Reply-To: <alpine.LFD.2.00.0908191326360.6044@xanadu.home>
Cc-ed Dscho, so he can easier participate in this subthread.
On Wed, 19 Aug 2009, Nicolas Pitre wrote:
> On Wed, 19 Aug 2009, Jakub Narebski wrote:
> > P.S. What do you think about 'bundle' capability extension mentioned
> > in a side sub-thread?
>
> I don't like it. Reason is that it forces the server to be (somewhat)
> stateful by having to keep track of those bundles and cycle them, and it
> doubles the disk usage by having one copy of the repository in the form
> of the original pack(s) and another copy as a bundle.
I agree about problems with disk usage, but I disagree about server
having to be stateful; server can just simply scan for bundles, and
offer links to them if client requests 'bundles' capability, somewhere
around initial git-ls-remote list of refs.
> Of course, the idea of having a cron job generating a bundle and
> offering it for download through HTTP or the like is fine if people are
> OK with that, and that requires zero modifications to git. But I don't
> think that is a solution that scales.
Well, offering daily bundle in addition to daily snapshot could be
a good practice, at least until git acquires resumable fetch (resumable
clone).
>
> If you think about git.kernel.org which has maybe hundreds of
> repositories where the big majority of them are actually forks of Linus'
> own repository, then having all those forks reference Linus' repository
> is a big disk space saver (and IO too as the referenced repository is
> likely to remain cached in memory). Having a bundle ready for each of
> them will simply kill that space advantage, unless they all share the
> same bundle.
I am thinking about sharing the same bundle for related projects.
>
> Now sharing that common bundle could be done of course, but that makes
> things yet more complex while still wasting IO because some requests
> will hit the common pack and some others will hit the bundle, making
> less efficient usage of the disk cache on the server.
Hmmm... true (unless bundles are on separate server).
>
> Yet, that bundle would probably not contain the latest revision if it is
> only periodically updated, even less so if it is shared between multiple
> repositories as outlined above. And what people with slow/unreliable
> network links are probably most interested in is the latest revision and
> maybe a few older revisions, but probably not the whole repository as
> that is simply too long to wait for. Hence having a big bundle is not
> flexible either with regards to the actual data transfer size.
I agree that bundle would be useful for restartable clone, and not
useful for restartable fetch. Well, unless you count (non-existing)
GitTorrent / git-mirror-sync as this solution... ;-)
>
> Hence having a restartable git-archive service to create the top
> revision with the ability to cheaply (in terms of network bandwidth)
> deepen the history afterwards is probably the most straight forward way
> to achieve that. The server needs no be aware of separate bundles, etc.
> And the shared object store still works as usual with the same cached IO
> whether the data is needed for a traditional fetch or a "git archive"
> operation.
It's the "cheaply deepen history" that I doubt would be easy. This is
the most difficult part, I think (see also below).
>
> Why "git archive"? Because its content is well defined. So if you give
> it a commit SHA1 you will always get the same stream of bytes (after
> decompression) since the way git sort files is strictly defined. It is
> therefore easy to tell a remote "git archive" instance that we want the
> content for commit xyz but that we already got n files already, and that
> the last file we've got has m bytes. There is simply no confusion about
> what we've got already, unlike with a partial pack which might need
> yet-to-be-received objects in order to make sense of what has been
> already received. The server simply has to skip that many files and
> resume the transfer at that point, independently of the compression or
> even the archive format.
Let's reiterate it to check if I understand it correctly:
Any "restartable clone" / "resumable fetch" solution must begin with
a file which is rock-solid stable wrt. reproductability given the same
parameters. git-archive has this feature, packfile doesn't (so I guess
that bundle also doesn't, unless it was cached / saved on disk).
It would be useful if it was possible to generate part of this rock-solid
file for partial (range, resume) request, without need to generate
(calculate) parts that client already downloaded. Otherwise server has
to either waste disk space and IO for caching, or waste CPU (and IO)
on generating part which is not needed and dropping it to /dev/null.
git-archive you say has this feature.
Next you need to tell server that you have those objects got using
resumable download part ("git archive HEAD" in your proposal), and
that it can use them and do not include them in prepared file/pack.
"have" is limited to commits, and "have <sha1>" tells server that
you have <sha1> and all its prerequisites (dependences). You can't
use "have <sha1>" with git-archive solution. I don't know enough
about 'shallow' capability (and what it enables) to know whether
it can be used for that. Can you elaborate?
Then you have to finish clone / fetch. All solutions so far include
some kind of incremental improvements. My first proposal of bisect
fetching 1/nth or predefined size pack is buttom-up solution, where
we build full clone from root commits up. You propose, from what
I understand build full clone from top commit down, using deepening
from shallow clone. In this step you either get full incremental
or not; downloading incremental (from what I understand) is not
resumable / they do not support partial fetch.
Do I understand this correctly?
--
Jakub Narebski
Poland
^ permalink raw reply
* question concerning branches
From: Ingo Brueckl @ 2009-08-19 19:21 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
In-Reply-To: <7vr5v7vehj.fsf@alter.siamese.dyndns.org>
Junio C Hamano <gitster@pobox.com> writes:
> This is one of the most useful features.
Wow. I'm sursprised to hear that, because I consider it at the moment as a
very strange one.
> For example, it is an essential
> part of supporting the workflow described here:
> http://gitster.livejournal.com/25892.html
Here is what I'd expect to do with git (described with my own words, not in
git commands):
1. commit the quick fix to the release branch
2. push this single commit to origin and master
Now that all branches have the commit a later push and pull should notice
this and "skip" it.
This leads to a second question I have. Assuming I have three patches in my
repo (#1, #2 and #3), is it possible to push only #2 (because it is a
quick fix) and later, maybe after I committed #4, the rest, i.e. #1, #2 and
#4?
Ingo
^ permalink raw reply
* Re: question concerning branches
From: Jakub Narebski @ 2009-08-19 19:08 UTC (permalink / raw)
To: Ingo Bruecki; +Cc: Avery Pennarun, git
In-Reply-To: <4a8c4425@wupperonline.de>
ib@wupperonline.de (Ingo Brueckl) writes:
> Avery Pennarun <apenwarr@gmail.com> writes:
>
> > You seem to have forgotten the "git commit" step before switching back
> > to master.
>
> No, I passed over the commit in my example. I know that after the commit the
> things are as they ought to be, but what if I can't do a commit because I am
> in the middle of coding and have to have a break?
Then you use git-stash. It was invented for that.
> > You have a modified file in your repository; what did you *want* to happen
> > when you switched branches?
>
> I want an unchanged file in master if I switch there (because I worked in a
> different branch) and a changed version in the test branch.
>
> Why is the *master* different depending on whether my work in test in still
> going on or committed?!
Branches are about commits. State of a working directory doesn't
belong to a branch (in Git). Learning concepts behind Git would help
you in understanding it (Git is very consistent), which in turn would
help in using it.
What about untracked files? Do you want to lose them when you switch
branches?
>
> Actually, I cannot image how branches are practicable if I always have to
> have in mind possibly still uncommitted work. Shouldn't it be git's work
> to ensure that master will remain it was when branching?
>
> Without git I'd make a copy for testing new features. With git, it seems that
> I have to do the same (a clone). This is what I don't understand.
You finish old work (or stash it away), _then_ you begin new work.
>
> > (Many people find the current behaviour very convenient.)
Take the following example. You started coding some feature on
'master' branch, then you realized that this feature is more
complicated than you thought at first, so it should be developed in
separate topic branch. You do "git checkout -b featureA", and voila
you are now coding on feature branch 'featureA'.
> > You might also want to look at the "git stash" command.
>
> Yes, but isn't it annoying to leave the test branch always either with stash
> or commit in order to have an unchanged master?!
No, it isn't.
--
Jakub Narebski
Git User's Survey 2009: http://tinyurl.com/GitSurvey2009
^ permalink raw reply
* Re: Continue git clone after interruption
From: Nicolas Pitre @ 2009-08-19 19:04 UTC (permalink / raw)
To: Jakub Narebski; +Cc: Tomasz Kontusz, git
In-Reply-To: <200908191719.52974.jnareb@gmail.com>
On Wed, 19 Aug 2009, Jakub Narebski wrote:
> There are 114937 objects in this packfile, including 56249 objects
> used as base (can be deltified or not). git-verify-pack -v shows
> that all objects have total size-in-packfile of 33 MB (which agrees
> with packfile size of 33 MB), with 17 MB size-in-packfile taken by
> deltaified objects, and 16 MB taken by base objects.
>
> git verify-pack -v |
> grep -v "^chain" |
> grep -v "objects/pack/pack-" > verify-pack.out
>
> sum=0; bsum=0; dsum=0;
> while read sha1 type size packsize off depth base; do
> echo "$sha1" >> verify-pack.sha1.out
> sum=$(( $sum + $packsize ))
> if [ -n "$base" ]; then
> echo "$sha1" >> verify-pack.delta.out
> dsum=$(( $dsum + $packsize ))
> else
> echo "$sha1" >> verify-pack.base.out
> bsum=$(( $bsum + $packsize ))
> fi
> done < verify-pack.out
> echo "sum=$sum; bsum=$bsum; dsum=$dsum"
Your object classification is misleading. Because an object has no
base, that doesn't mean it is necessarily a base itself. You'd have to
store $base into a separate file and then sort it and remove duplicates
to know the actual number of base objects. What you have right now is
strictly delta objects and non-delta objects. And base objects can
themselves be delta objects already of course.
Also... my git repo after 'git gc --aggressive' contains a pack which
size is 22 MB. Your script tells me:
sum=22930254; bsum=14142012; dsum=8788242
and:
29558 verify-pack.base.out
82043 verify-pack.delta.out
111601 verify-pack.out
111601 verify-pack.sha1.out
meaning that I have 111601 total objects, of which 29558 are non-deltas
occupying 14 MB and 82043 are deltas occupying 8 MB. That certainly
shows how deltas are space efficient. And with a minor modification to
your script, I know that 44985 objects are actually used as a delta
base. So, on average, each base is responsible for nearly 2 deltas.
> >>>> (BTW what happens if this pack is larger than file size limit for
> >>>> given filesystem?).
> [...]
>
> >> If I remember correctly FAT28^W FAT32 has maximum file size of 2 GB.
> >> FAT is often used on SSD, on USB drive. Although if you have 2 GB
> >> packfile, you are doing something wrong, or UGFWIINI (Using Git For
> >> What It Is Not Intended).
> >
> > Hopefully you're not performing a 'git clone' off of a FAT filesystem.
> > For physical transport you may repack with the appropriate switches.
>
> Not off a FAT filesystem, but into a FAT filesystem.
That's what I meant, sorry. My point still stands.
> > The front of the pack is the critical point. If you get enough to
> > create the top commit then further transfers can be done incrementally
> > with only the deltas between each commits.
>
> How? You have some objects that can be used as base; how to tell
> git-daemon that we have them (but not theirs prerequisites), and how
> to generate incrementals?
Just the same as when you perform a fetch to update your local copy of a
remote branch: you tell the remote about the commit you have and the one
you want, and git-repack will create delta objects for the commit you
want against similar objects from the commit you already have, and skip
those objects from the commit you want that are already included in the
commit you have.
> >> A question about pack protocol negotiation. If clients presents some
> >> objects as "have", server can and does assume that client has all
> >> prerequisites for such objects, e.g. for tree objects that it has
> >> all objects for files and directories inside tree; for commit it means
> >> all ancestors and all objects in snapshot (have top tree, and its
> >> prerequisites). Do I understand this correctly?
> >
> > That works only for commits.
>
> Hmmmm... how do you intent for "prefetch top objects restartable-y first"
> to work, then?
See my latest reply to dscho (you were in CC already).
> >> BTW. because of compression it might be more difficult to resume
> >> archive creation in the middle, I think...
> >
> > Why so? the tar+gzip format is streamable.
>
> gzip format uses sliding window in compression. "cat a b | gzip"
> is different from "cat <(gzip a) <(gzip b)".
>
> But that doesn't matter. If we are interrupted in the middle, we can
> uncompress what we have to check how far did we get, and tell server
> to send the rest; this way server wouldn't have to even generate
> (but not send) what we get as partial transfer.
You got it.
> P.S. What do you think about 'bundle' capability extension mentioned
> in a side sub-thread?
I don't like it. Reason is that it forces the server to be (somewhat)
stateful by having to keep track of those bundles and cycle them, and it
doubles the disk usage by having one copy of the repository in the form
of the original pack(s) and another copy as a bundle.
Of course, the idea of having a cron job generating a bundle and
offering it for download through HTTP or the like is fine if people are
OK with that, and that requires zero modifications to git. But I don't
think that is a solution that scales.
If you think about git.kernel.org which has maybe hundreds of
repositories where the big majority of them are actually forks of Linus'
own repository, then having all those forks reference Linus' repository
is a big disk space saver (and IO too as the referenced repository is
likely to remain cached in memory). Having a bundle ready for each of
them will simply kill that space advantage, unless they all share the
same bundle.
Now sharing that common bundle could be done of course, but that makes
things yet more complex while still wasting IO because some requests
will hit the common pack and some others will hit the bundle, making
less efficient usage of the disk cache on the server.
Yet, that bundle would probably not contain the latest revision if it is
only periodically updated, even less so if it is shared between multiple
repositories as outlined above. And what people with slow/unreliable
network links are probably most interested in is the latest revision and
maybe a few older revisions, but probably not the whole repository as
that is simply too long to wait for. Hence having a big bundle is not
flexible either with regards to the actual data transfer size.
Hence having a restartable git-archive service to create the top
revision with the ability to cheaply (in terms of network bandwidth)
deepen the history afterwards is probably the most straight forward way
to achieve that. The server needs no be aware of separate bundles, etc.
And the shared object store still works as usual with the same cached IO
whether the data is needed for a traditional fetch or a "git archive"
operation.
Why "git archive"? Because its content is well defined. So if you give
it a commit SHA1 you will always get the same stream of bytes (after
decompression) since the way git sort files is strictly defined. It is
therefore easy to tell a remote "git archive" instance that we want the
content for commit xyz but that we already got n files already, and that
the last file we've got has m bytes. There is simply no confusion about
what we've got already, unlike with a partial pack which might need
yet-to-be-received objects in order to make sense of what has been
already received. The server simply has to skip that many files and
resume the transfer at that point, independently of the compression or
even the archive format.
Nicolas
^ permalink raw reply
* question concerning branches
From: Ingo Brueckl @ 2009-08-19 18:31 UTC (permalink / raw)
To: Avery Pennarun; +Cc: git
In-Reply-To: <32541b130908191107v2ab6752awb43f521f805b5f1a@mail.gmail.com>
Avery Pennarun <apenwarr@gmail.com> writes:
> You seem to have forgotten the "git commit" step before switching back
> to master.
No, I passed over the commit in my example. I know that after the commit the
things are as they ought to be, but what if I can't do a commit because I am
in the middle of coding and have to have a break?
> You have a modified file in your repository; what did you *want* to happen
> when you switched branches?
I want an unchanged file in master if I switch there (because I worked in a
different branch) and a changed version in the test branch.
Why is the *master* different depending on whether my work in test in still
going on or committed?!
Actually, I cannot image how branches are practicable if I always have to
have in mind possibly still uncommitted work. Shouldn't it be git's work
to ensure that master will remain it was when branching?
Without git I'd make a copy for testing new features. With git, it seems that
I have to do the same (a clone). This is what I don't understand.
> (Many people find the current behaviour very convenient.)
I find it highly confusing. I understood a branch as something I can do in
whatever I want without affecting master. But now a learn that everything I
do in the branch will happen in master, too, until I commit. Strange. Very
strange.
> You might also want to look at the "git stash" command.
Yes, but isn't it annoying to leave the test branch always either with stash
or commit in order to have an unchanged master?!
Ingo
^ permalink raw reply
* Re: question concerning branches
From: Junio C Hamano @ 2009-08-19 18:35 UTC (permalink / raw)
To: Ingo Brueckl; +Cc: git
In-Reply-To: <4a8c373f@wupperonline.de>
ib@wupperonline.de (Ingo Brueckl) writes:
> I'm a git novice and have a comprehension question concerning branches.
>
> Within a git repository, I do:
>
> git branch test
> git checkout test
> # edit foo.bar
> git checkout master
>
> I'd expect that master is in the exactly same unchanged state it was at
> branching time, but what a surprise, foo.bar is modified here, too!
No. Local modification that are not committed do not belong to any
branch. Rather it belongs to your work tree and the index, and follow you
when you switch branches.
This is one of the most useful features. For example, it is an essential
part of supporting the workflow described here:
http://gitster.livejournal.com/25892.html
On the other hand, there are cases that you do not want to see your local
changes to foo.bar appear when you switch back to master, and the most
obvious case is "I started working on something, and I used a separate
branch 'test' because the change will be involved. I am in the middle of
the work, the changes so far I made to foo.bar is far from ready, but I
have to handle emergency fix on the master branch". For that kind of
situation, you can use "git stash", like:
http://gitster.livejournal.com/29577.html
^ permalink raw reply
* Re: question concerning branches
From: Avery Pennarun @ 2009-08-19 18:07 UTC (permalink / raw)
To: Ingo Brueckl; +Cc: git
In-Reply-To: <4a8c373f@wupperonline.de>
On Wed, Aug 19, 2009 at 5:33 PM, Ingo Brueckl<ib@wupperonline.de> wrote:
> Within a git repository, I do:
>
> git branch test
> git checkout test
> # edit foo.bar
> git checkout master
>
> I'd expect that master is in the exactly same unchanged state it was at
> branching time, but what a surprise, foo.bar is modified here, too!
You seem to have forgotten the "git commit" step before switching back
to master. You have a modified file in your repository; what did you
*want* to happen when you switched branches? (Many people find the
current behaviour very convenient.)
You might also want to look at the "git stash" command.
Avery
^ permalink raw reply
* Re: question concerning branches
From: Bruce Stephens @ 2009-08-19 18:07 UTC (permalink / raw)
To: Ingo Brueckl; +Cc: git
In-Reply-To: <4a8c373f@wupperonline.de>
ib@wupperonline.de (Ingo Brueckl) writes:
> I'm a git novice and have a comprehension question concerning branches.
>
> Within a git repository, I do:
>
> git branch test
> git checkout test
> # edit foo.bar
> git checkout master
>
> I'd expect that master is in the exactly same unchanged state it was at
> branching time, but what a surprise, foo.bar is modified here, too!
You didn't commit your change to foo.bar.
[...]
> What must I do to get a test branch I can't work without affecting master?
commit changes.
^ permalink raw reply
* question concerning branches
From: Ingo Brueckl @ 2009-08-19 17:33 UTC (permalink / raw)
To: git
I'm a git novice and have a comprehension question concerning branches.
Within a git repository, I do:
git branch test
git checkout test
# edit foo.bar
git checkout master
I'd expect that master is in the exactly same unchanged state it was at
branching time, but what a surprise, foo.bar is modified here, too!
If I continue now working in the master branch (applying patches and such) I
will use a changed foo.bar with testing branch content. I can't even apply
patches to foo.bar without conflict.
Of what use are branches if the files aren't totally separated from each
other?
What must I do to get a test branch I can't work without affecting master?
Ingo
^ permalink raw reply
* Re: ls-files -t broken? Or do I just not understand it?
From: Matthieu Moy @ 2009-08-19 17:53 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Nguyen Thai Ngoc Duy, Björn Steinbrink, git
In-Reply-To: <7v1vn7wvce.fsf@alter.siamese.dyndns.org>
Junio C Hamano <gitster@pobox.com> writes:
> So I wounld't be surprised if there were bugs lurking around "-t" family.
> Patches to fix them are welcome; people's scripts depend on them.
In the particular case of -t, as far as my grep can tell, it's
untested, not used in Git itself, and rather unlikely to be used
elsewhere given it's strange behavior.
I'd say the best thing to to is to keep it, but mark it as deprecated
in the doc, and point to 'diff --name-status' and 'status' as superior
alternatives in the doc.
--
Matthieu
^ permalink raw reply
* Re: ls-files -t broken? Or do I just not understand it?
From: Junio C Hamano @ 2009-08-19 17:45 UTC (permalink / raw)
To: Nguyen Thai Ngoc Duy; +Cc: Björn Steinbrink, Matthieu Moy, git
In-Reply-To: <fcaeb9bf0908190218x487dd7f4y355e411a8cb1f254@mail.gmail.com>
Nguyen Thai Ngoc Duy <pclouds@gmail.com> writes:
> It shows you whether it's a normal entry (marked as "H") or unmerged
> entry ("M") as far as I can tell. Junio may give more detail
> explanation about this command.
Sorry, I won't. I refuse to take responsibility for some "features" in
this command ;-).
My recollection is that the "-t for quick status" were primarily added to
support a scripted Porcelain that is now defunct, while I was looking the
other way, eh, rather, back when I was not yet in control of the project.
Some of the options of this command are not useful anymore, not in the
sense that they no longer work as advertised, but more in the sense that
there are better ways to do what they were originally invented for.
For example, the options "-[mdt]" under discussion in this thread were
primarily invented as poor-mans' substitute for diff-files when the diff
infrastructure was not mature enough. It could have been that people who
added these flags did not understand diff---I do not remember. You would
notice the apparent inconsistency of its choice of the "status" letters,
compared with the rest of the system---it is a sign that the "-t" feature
was never taken seriously by core git developers.
So I wounld't be surprised if there were bugs lurking around "-t" family.
Patches to fix them are welcome; people's scripts depend on them.
I have to say that ls-files currently is in an unfortunate position. It is
a building block for scripts and it will be kept maintained for that
purpose. But people still use it even from the command line.
Some options are still the only way to get certain information out of the
system, e.g. "-v" to see the assume-unchanged bit. "-u" unfortunately is
still the quickest way to list the conflicted paths, even though the new
"git status -s" may change this situation. "-o/-i" is the kosher way for
scripts to get the list of untracked and/or ignored paths, but some people
seem to have learned to use that from the command line. While there is
nothing _wrong_ in doing so per-se, these days from the command line use
for human consumption I tend to think it is much handier to view output
from "git clean -n".
^ permalink raw reply
* Re: Continue git clone after interruption
From: Nicolas Pitre @ 2009-08-19 17:21 UTC (permalink / raw)
To: Johannes Schindelin; +Cc: Jakub Narebski, Tomasz Kontusz, git
In-Reply-To: <alpine.DEB.1.00.0908190921360.5594@intel-tinevez-2-302>
[-- Attachment #1: Type: TEXT/PLAIN, Size: 5306 bytes --]
On Wed, 19 Aug 2009, Johannes Schindelin wrote:
> Hi,
>
> On Tue, 18 Aug 2009, Nicolas Pitre wrote:
>
> > On Wed, 19 Aug 2009, Johannes Schindelin wrote:
> >
> > > But seriously, I miss a very important idea in this discussion: we
> > > control the Git source code. So we _can_ add a upload_pack feature
> > > that a client can ask for after the first failed attempt.
> >
> > Indeed. So what do you think about my proposal? It was included in my
> > first reply to this thread.
>
> Did you not talk about an extension of the archive protocol? That's not
> what I meant. The archive protocol can be disabled for completely
> different reasons than to prevent restartable clones.
And those reasons are?
> But you brought up an important point: shallow repositories.
>
> Now, the problem, of course, is that if you cannot even get a single ref
> (shallow'ed to depth 0 -- which reminds me: I think I promised to fix
> that, but I did not do that yet) due to intermittent network failures, you
> are borked, as you said.
Exact.
> But here comes an idea: together with Nguyễn's sparse series, it is
> conceivable that we support a shallow & narrow clone via the upload-pack
> protocol (also making mithro happy). The problem with narrow clones was
> not the pack generation side, that is done by a rev-list that can be
> limited to certain paths. The problem was that we end up with missing
> tree objects. However, if we can make a sparse checkout, we can avoid
> the problem.
Sure, if you can salvage as much as you can from a partial pack and
create a shallow and narrow clone out of it then it should be possible
to do some restartable clone. I still think this might be much less
complex to achieve through git-archive, especially if some files i.e.
objects are large enough to expose themselves to network outage. It is
like the same issue as being able to fetch at least one revision but to
a lesser degree. You might be able to get that first revision through
multiple attempts by gathering missing objects on each attempt. But if
you encounter an object large enough you then might be unlucky enough
not to be able to transfer it all before the next network failure.
With a simple extension to git-archive, any object content could be
resumed many times from any offset. Then, deepening the history should
make use of deltas through the pack protocol which should hopefully
consist of much smaller transfers and therefore less prone to network
outage.
That could be sketched like this, supposing user runs
"git clone git://foo.bar/baz":
1) "git ini baz" etc. as usual.
2) "git ls-remote git://foo.bar/baz HEAD" and store the result in
.git/CLONE_HEAD so not to be confused by the remote HEAD possibly
changing before we're done.
3) "git archive --remote=git://foo.bar/baz CLONE_HEAD" and store the
result locally. Keep track of how many files are received, and how
many bytes for the currently received file.
4) if network connection is broken, loop back to (3) adding
--skip=${nr_files_received},${nr_bytes_in_curr_file_received} to
the git-archive argument list. REmote server simply skips over
specified number of files and bytes into the next file.
5) Get content from remote commit object for CLONE_HEAD somehow. (?)
6) "git add . && git write-tree" and make sure the top tree SHA1 matches
the one in the commit from (5).
7) "git hash-object -w -t commit" with data obtained in (5), and make
sure it matches SHA1 from CLONE_HEAD.
8) Update local HEAD with CLONE_HEAD and set it up as a shallow clone.
Delete .git/CLONE_HEAD.
9) Run "git fetch" with the --depth parameter to get more revisions.
Notes:
- This mode of operation should probably be optional, like by using
--safe or --restartable with 'git clone'. And since this mode of
operation is really meant for people with slow and unreliable network
connections, they're unlikely to wish for the whole history to be
fetched. Hence this mode could simply be triggered by the --depth
parameter to 'git clone' which would provide a clear depth value to
use in (9).
- If the transfer is interrupted locally with ^C then it should be
possible to resume it by noticing the presence of .git/CLONE_HEAD
up front. DEtermining how many files to skip when resuming with
git-archive can be done with $((`git ls-files -o | wc -l` - 1)) and
$(git ls-files -o | tail -1 | wc -c).
- That probably would be a good idea to have a tgz format to 'git
archive' which might be simpler to deal with than the zip format.
- Step (3) could be optimized in many ways, like by directly using
hash-object and update-index, or by using a filter to pipe the result
directly into fast-import.
- So to say that the above should be pretty easy to implement even
with a shell script. A builtin version could then be made if this
proves to actually be useful. And the server remains stateless with
no additional caching needed which would go against any attempt
at making a busy server like git.kernel.org share as much of the
object store between plenty of mostly identical repositoryes.
> Note: this is not well thought-through, but just a brainstorm-like answer
> to your ideas.
And so is the above.
Nicolas
^ permalink raw reply
* Re: [PATCH 12/12] Add gitignore and ReadMe file for MSVC Build
From: Erik Faye-Lund @ 2009-08-19 16:00 UTC (permalink / raw)
To: Frank Li; +Cc: git, msysgit
In-Reply-To: <1250697475-4340-1-git-send-email-lznuaa@gmail.com>
On Wed, Aug 19, 2009 at 5:57 PM, Frank Li<lznuaa@gmail.com> wrote:
> +3. Open gitbuild\gitbuild.sln with VS2008. Then press F7.
> \ No newline at end of file
Again, please add a newline.
--
Erik "kusma" Faye-Lund
kusmabite@gmail.com
(+47) 986 59 656
^ permalink raw reply
* Re: [PATCH 08/12] Add MSVC porting files.
From: Erik Faye-Lund @ 2009-08-19 15:59 UTC (permalink / raw)
To: Frank Li; +Cc: git, msysgit
In-Reply-To: <1250697167-5536-8-git-send-email-lznuaa@gmail.com>
On Wed, Aug 19, 2009 at 5:52 PM, Frank Li<lznuaa@gmail.com> wrote:
> +#endif
> \ No newline at end of file
Please add a newline after #endif.
--
Erik "kusma" Faye-Lund
kusmabite@gmail.com
(+47) 986 59 656
^ permalink raw reply
* [PATCH 12/12] Add gitignore and ReadMe file for MSVC Build
From: Frank Li @ 2009-08-19 15:57 UTC (permalink / raw)
To: git, msysgit; +Cc: Frank Li
Add ReadMe to user how to build git with MSVC.
Add gitignore to filter out files created file by MSVC.
Signed-off-by: Frank Li <lznuaa@gmail.com>
---
compat/vcbuild/.gitignore | 3 +++
compat/vcbuild/ReadMe | 13 +++++++++++++
2 files changed, 16 insertions(+), 0 deletions(-)
create mode 100644 compat/vcbuild/.gitignore
create mode 100644 compat/vcbuild/ReadMe
diff --git a/compat/vcbuild/.gitignore b/compat/vcbuild/.gitignore
new file mode 100644
index 0000000..7796990
--- /dev/null
+++ b/compat/vcbuild/.gitignore
@@ -0,0 +1,3 @@
+Debug
+Release
+*.user
\ No newline at end of file
diff --git a/compat/vcbuild/ReadMe b/compat/vcbuild/ReadMe
new file mode 100644
index 0000000..f4e5efa
--- /dev/null
+++ b/compat/vcbuild/ReadMe
@@ -0,0 +1,13 @@
+The Steps of Build Git with VS2008
+
+1. Create VC Build Environment.
+
+ git clone git://repo.or.cz/gitbuild.git
+
+ gitbuild include VS solution file and library such as zlib.
+
+2. Get Submodule
+
+ git submodule update --init
+
+3. Open gitbuild\gitbuild.sln with VS2008. Then press F7.
\ No newline at end of file
--
1.6.4.msysgit.0
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox