* Re: Change set based shallow clone
From: Jakub Narebski @ 2006-09-07 20:21 UTC (permalink / raw)
To: git
In-Reply-To: <9e4733910609071252ree73effwb06358e9a22ba965@mail.gmail.com>
Jon Smirl wrote:
> If you haven't updated for six months when the server walks backwards
> for 10 change sets it's not going to find anything you have locally.
> When this situation is encountered the server needs to generate a
> delta just for you between one of the change sets it knows you have
> and one of the 10 change sets you want. By generating this one-off
> delta it lets you avoid the need to fetch all of the objects back to a
> common branch ancestor. The delta functions as a jump over the
> intervening space.
I don't understand. Git is _not_ patchset based (like GNU Arch, or
Mercurial, or CVS). It is snapshot based. So if you want to download
"skip", you need only for the local part of doenloader to make appropriate
grafts, like below
*--*--*--*--*--*--*--*--*--*--*--HEAD (server)
*--*--*...........*--*--*--*--*--HEAD (shallow/sparse clone)
But the part you were talking about is _easy_ part; the hard part is
merges including merging branch which was split off the trunk before
cutoff-point, history rewriting (c.f. 'pu' branch, and rebases), etc.
--
Jakub Narebski
Warsaw, Poland
ShadeHawk on #git
^ permalink raw reply
* Change set based shallow clone
From: Jon Smirl @ 2006-09-07 19:52 UTC (permalink / raw)
To: git, linux@horizon.com
Here's a change set based shallow clone scheme I've been thinking
about, does it have potential?
When the client wants a shallow clone it starts by telling the server
all of the HEADs and how many change sets down each of those HEADs it
has locally. That's a small amout of data to transmit and it can be
easily tracked. Let's ignore merged branches for the moment.
The client then says I want at least 10 (or N) change sets for all of
the HEADs present at the server. The server starts from each HEAD and
works backwards until it encounters a change set present on the
client. At that point it will be able to compute efficient deltas to
send.
If you haven't updated for six months when the server walks backwards
for 10 change sets it's not going to find anything you have locally.
When this situation is encountered the server needs to generate a
delta just for you between one of the change sets it knows you have
and one of the 10 change sets you want. By generating this one-off
delta it lets you avoid the need to fetch all of the objects back to a
common branch ancestor. The delta functions as a jump over the
intervening space.
In the case of an initial shallow clone the client won't have anything
to delta against. The server will be forced to send a full version
for one of the 10 change sets requested and deltas for the rest.
Getting an initial shallow clone should take about as long as a CVS
check out.
This scheme does require the server to sometimes generate custom diffs
for the client, but in all the cases I have been working with
everything is always IO bound so it is better to spend some CPU to
reduce the IO needed.
--
Jon Smirl
jonsmirl@gmail.com
^ permalink raw reply
* Re: Dropping Git.pm (at least Git.xs)?
From: Petr Baudis @ 2006-09-07 19:51 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
In-Reply-To: <7vodtxuqt4.fsf@assigned-by-dhcp.cox.net>
Hi,
Dear diary, on Sun, Sep 03, 2006 at 01:34:31PM CEST, I got a letter
where Junio C Hamano <junkio@cox.net> said that...
> - I think the clean-up promise of Git.pm is great (e.g.
> safe_qx should be part of it not in git-svn alone).
>
> - I think Git.xs was a bit premature and raised the hurdle of
> cleaning up and consolidating various core-wrappers from
> existing Perl scripts into Git.pm and have them use Git.pm.
> It would be nice if we can drop this part for now, and do a
> bit more Perl-level clean-up first.
>
> - I think "repository" abstraction, if we are going to have
> one, should be designed from the core level if we are going
> to have it accessible from Git.xs. Unfortunately I am not
> ready to invest great time and effort for core level
> libification at this moment.
I basically agree on all three points - I will try to submit a patch
to implement get_object() purely in Git.pm reasonably soon, and will see
about introducing some actually designed repository division abstraction
when I get some time. I hope to see Git.xs come back again in few
months, but for now I have to concur that we should go back to the
drawing board.
--
Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
Snow falling on Perl. White noise covering line noise.
Hides all the bugs too. -- J. Putnam
^ permalink raw reply
* Re: A look at some alternative PACK file encodings
From: linux @ 2006-09-07 19:22 UTC (permalink / raw)
To: linux, nico; +Cc: git, gitzilla
In-Reply-To: <Pine.LNX.4.64.0609071322010.18635@xanadu.home>
> Well... I'm using object type 6 for deltas with offset reference
> (instead of object type 7 that is delta with sha1 reference). So
> there is no extra bit needed and no backward compatibility breakage.
Ah, my apologies! I didn't realize that you weren't constructing a
whole new pack format, or using a global flag. In this case,
the overhead is already bought and paid for, so there's no point
to not taking advantage of it.
And thin packs should Just Work, once the builder is taught to
use type 7 objects when necessary.
> There could be of course a new object type which payload is only a sha1
> that deltas type 6 reference. But that can be introduced at a later date
> if it turns out to be worthwhile (i.e. the actual saving in real use
> scenarios is worth the (small but still) added complexity).
In this case, it's only of potential benefit if there is more than
one reference to an object, and figuring it out is almost certainly
more trouble than it's worth. Deltas usually form long skinny
chains, not high branching factor bushes.
^ permalink raw reply
* Re: A look at some alternative PACK file encodings
From: linux @ 2006-09-07 19:16 UTC (permalink / raw)
To: linux, nico; +Cc: git, gitzilla
In-Reply-To: <Pine.LNX.4.64.0609071228560.18635@xanadu.home>
> So I don't think we would gain that much using that encoding
> unless/until the pack format is made completely incompatible due to
> other changes, and that's something we should try to avoid as much as
> possible anyway.
Ah, I'm sorry; I didn't realize that it was already used elsewhere in
the pack format and this was just code re-use. That is indeed a good
reason to stick with what's already there.
I agree the savings are minimal, but if you're starting from scratch,
they're free, so why not use them? I tried to show that the code
complexity is truly negligible.
I'm sorry I didn't realize the details of pack structure before.
^ permalink raw reply
* Re: A look at some alternative PACK file encodings
From: Junio C Hamano @ 2006-09-07 18:50 UTC (permalink / raw)
To: Shawn Pearce; +Cc: git
In-Reply-To: <20060907054651.GD31580@spearce.org>
Shawn Pearce <spearce@spearce.org> writes:
> Junio C Hamano <junkio@cox.net> wrote:
>> Shawn Pearce <spearce@spearce.org> writes:
>>
>> > And I'm half-way done with the 64 bit mmap sliding window. You,
>> > Junio and I are all hacking on the same tiny bit of core code for
>> > different reasons. :-)
>>
>> Which makes me quite nervous, actually.
>
> What order do you want the patches in then?
>
> I'm willing to go before or after Nico's offset changes, and before
> or after your 64 bit index changes.
>
> Either way two of us are going to have to redo our work on top of
> the others. I'm finding that I'm basically rewriting the sliding
> window code onto your 64 bit version - there's no easy merge here.
> And Nico's got the same problem, he's in unpack_delta_entry too.
While 64-bit offset exercise was a good way for me to see where
the <pack_base+offset> assumption lies in the current code, and
while it would have been the right way to do so in the ideal
world, I realize that it may not be the best way to do things in
the real world for a few reasons:
- Always doing u64 arithmetic would be expensive, especially on
32-bit architectures, and doing so conditionally is quite an
extra complexity.
- Not necessarily everybody has large file offsets, so asking
to mmap 8MB at file offset at 6GB may not be possible. Even
on systems with large file offsets, truly huge single file is
cumbersome to manage than a handful files under 32-bit offset
(e.g. using a DVD-RW to back up your packs).
- Mozilla may not be the largest project in the world, but it
certainly is on the larger side than majority of our target
audience, and even its entire history, 450MB, comfortably
fits in 32-bit space.
So while I merged the 64-bit offset change in "next" branch, I
am quite doubtful that it was a good change. As long as we
devise an easy way to keep each packfile under manageable size,
the current 32-bit arrangement would be more beneficial and
practical than doing everything internally in 64-bit. Larger
projects, if needed, can use multiple packs, and even a project
that _could_ pack everything under 450MB would definitely want
to use multiple packs anyway (i.e. one or more huge historical
packs and one active pack) to avoid repacking and incremental
update cost. Your "mmap parts into windows" update is of a far
more practical value than the idx with 64-bit offsets. So I am
very inclined to revert my 64-bit offset change from "next".
Before declaring it a failure, in order to keep the window open
for the future, we would need to make sure that we can change
things as needed to do everything we want with 32-bit offset
packs. The changes we would need I can think of are in the
following areas:
- "git repack" needs to be taught to limit the resulting packs
under 32-bit offset. I think this is a given, and we would
need some updates to the command because it needs to be
taught not to touch "huge historical packs" anyway.
- "git send-pack" and "git fetch-pack" are Ok -- the receiving
end explodes the pack that comes over the wire using
unpack-objects into loose objects, and there is no offset
involved during the process. Note that pack-objects may need
to keep the offset in the stream in u64 if we were to do the
offset encoding of the base object, but that is pretty much
an independent issue.
- "git fetch-pack -k" has a problem. The daemon or sending
side runs rev-list piped to pack-objects and there is no
limit in the size of the pack generated in the pipeline.
Worse yet, the protocol does not allow more than one packs to
be sent, so we would need a protocol update to say "the data
you asked would result in a pack that is more than 4GB, so
I'll send more than one packs -- here is one, here is
another, that's all folks". The receiving end needs to be
taught to handle this.
Note: I am not proposing to do any of the above right now, until
there is no need for this by any real project. We just need to
be sure that when need arises there is a way out.
The only case that can be problematic is when a single object is
larger than 4GB deflated, and at that point we would hit a very
hard wall of 32-bit offset. But handling such a huge object
would have other problems [*1*] anyway, so I think it is Ok to
declare that there is a hard limit on deflated object size, at
least for now.
[*1*]
For example, we tend to do everything in core, but doing diffs
or anything on such a huge blob should probably be done by
swapping in only parts of it at a time).
Also we give the whole deflated size to zlib as avail_in, which
is "uInt", so on architectures where int is shorter than 64-bit
we lose (my 64-bit index patch does not deal with this). This
however is something your "mmap parts into windows" update would
solve if it were updated to deal with the internal 64-bit
offsets.
^ permalink raw reply
* Re: A look at some alternative PACK file encodings
From: Junio C Hamano @ 2006-09-07 17:40 UTC (permalink / raw)
To: Richard Curnow; +Cc: git, linux
In-Reply-To: <loom.20060907T163036-849@post.gmane.org>
Richard Curnow <rc@rc0.org.uk> writes:
> <linux <at> horizon.com> writes:
>
>> For regular packs, such objects wouldn't even be present, because
>> all base objects are in the pack itself.
>
> It would actually be useful if this restriction were lifted.
The primary reason why we require the base object to be present
in the same pack is because our assumption has always been we
mmap a pack as a whole, and we assumed that we can make sure
that the address space of a process to be enough to mmap at
least one pack (otherwise the user will be given a way to keep
size of each pack down to a size that is a mmapable piece) but
not necessarily more than one. In other words, while unpacking
one object that is deltified, we wanted to make sure everything
that is necessary is availble in memory. Otherwise, when
multiple packs are needed to reconstitute a single object
because the object is a delta in one pack and its base is in the
other pack, we might not be able to mmap both of them into the
address space at the same time.
The "mmap only parts of a huge packfile into map windows" update
Shawn Pearce is working on would lift that worry, so it would be
less of a reason.
Having said that, people must realize that keeping thin packs on
disk, indexed, is a major undertaking. It can be made to work,
but it will be painful.
People can grab a thin pack via rsync or http without realizing
it is thin, try to use it and then later (possibly much later,
when a delta that is against a missing object happens to be
needed) discover some of the pack data is unusable. To avoid
this, the dumb transports need to walk the ancestry chains of
commits and their associated objects while fetching as they do
right now, but also need to walk the delta chains to make sure
their base objects are available.
I like the proposal by Linus to have stub objects that records
what the missing base object is in the thin pack stream. It
simplifies things for the offset representation of bases. But
if we were to do so, I suspect indexing such a thing would need
more surgery to the .idx format. The question is if we should
record the stub object in the .idx file.
Right now we record the beginning of data for each object
(either base or delta) without storing anything to say about
where it ends -- the pack is assumed to be dense, so the point
where next object begins should be where the current one ends.
This means that we need to build a reverse index to list the
objects in the pack in offset order, when we want to find the
end of each object efficiently.
We work this around when using a pack by not doing the full
verification. We start decoding the header from the beginning
and in either base or delta the deflated stream sits at the end.
We inflate that deflated stream until zlib says it got enough,
and we do not check if that "enough" point stepped into the
location that is supposed to be inside the "next" object (if it
did so then it is an error in the packfile we missed to detect).
If things inflate well and delta applies cleanly, we are happy.
But pack-objects, when reusing the data from an existing pack,
wants to be careful, so it _does_ build a reverse index itself
and makes sure that inflate() ends at the location we expect it
to. For that process to remain the correct way to determine
where each object ends, the .idx file must record the presense
of stub objects.
However, if we write the names of stub objects in the .idx file,
we have another problem. Currently, the existence of an object
name in the .idx means that such an object exists in the
corresponding .pack; this will not hold true anymore.
So if we were to do this, we need to give one extra bit to each
of the .idx entries to mark which ones are stub and which ones
are not. This would allow the reverse index to properly notice
where each object ends, and allow the dumb transports to walk
the .idx for missing base objects efficiently. The code that
looks at .idx at runtime to determine which pack has needed
object needs to notice this bit.
Again, all of this can be made to work, but it will be painful.
^ permalink raw reply
* Re: A look at some alternative PACK file encodings
From: Nicolas Pitre @ 2006-09-07 17:32 UTC (permalink / raw)
To: linux; +Cc: gitzilla, git
In-Reply-To: <20060907090756.30111.qmail@science.horizon.com>
On Thu, 7 Sep 2006, linux@horizon.com wrote:
> > Support for 'thin' packs would pretty much require mixing IDs and
> > (relative) offsets in the same pack file.
>
> An alternative would be to create a small "placeholder" object that
> just gives an ID, then refer to it by offset.
>
> That would avoid the need for an id/offset bit with every offset,
> and possibly save more space if the same object was referenced
> multiple times.
Well... I'm using object type 6 for deltas with offset reference
(instead of object type 7 that is delta with sha1 reference). So
there is no extra bit needed and no backward compatibility breakage.
There could be of course a new object type which payload is only a sha1
that deltas type 6 reference. But that can be introduced at a later date
if it turns out to be worthwhile (i.e. the actual saving in real use
scenarios is worth the (small but still) added complexity).
> And it just seems simpler.
Well I would not say so. Simply prefixing a delta with the sha1 of its
base object cannot be simpler when streaming out a pack.
Nicolas
^ permalink raw reply
* Re: Add git-archive [take #2]
From: Franck Bui-Huu @ 2006-09-07 17:26 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Franck, git, Rene Scharfe, Jakub Narebski
In-Reply-To: <450019C3.4030001@innova-card.com>
2006/9/7, Franck Bui-Huu <vagabon.xyz@gmail.com>:
>
> 4/ Progress indicator support. Junio wants to mimic upload-pack for
> that. But it will lead in a lot of duplicated code if we don't
> try to share code. Can we copy that code anyways and clean up
> later ?
>
BTW, could we move the side-band code into git_connect() function and
adding to it a new parameter like:
int git_connect(int fd[2], char *url, const char *prog, int side_band)
or
int git_connect(int fd[2], char *url, const char *prog, flags)
Hence it automatically spawns a side-band process in the client side
and it also sends an extended option for git-daemon to ask for using
side band for server services.
--
Franck
^ permalink raw reply
* Re: A look at some alternative PACK file encodings
From: A Large Angry SCM @ 2006-09-07 17:22 UTC (permalink / raw)
To: Jon Smirl; +Cc: linux@horizon.com, git
In-Reply-To: <9e4733910609070557jd8cfc57nd4f7a8973b69f6ed@mail.gmail.com>
Jon Smirl wrote:
> On 7 Sep 2006 05:07:56 -0400, linux@horizon.com <linux@horizon.com> wrote:
>> > Support for 'thin' packs would pretty much require mixing IDs and
>> > (relative) offsets in the same pack file.
>>
>> An alternative would be to create a small "placeholder" object that
>> just gives an ID, then refer to it by offset.
>>
>> That would avoid the need for an id/offset bit with every offset,
>> and possibly save more space if the same object was referenced
>> multiple times.
>>
>> And it just seems simpler.
>
> There are 2 million objects in the Mozilla pack. This table would take:
> 2M * (20b (sha) + 10b(object index/overhead) = 60MB
> This 60MB is pretty much incompressible and increases download time.
>
> Much better if storage of the sha1s can be totally eliminated and
> replaced by something smaller. Alternatively this map could be
> stripped for transmission and rebuilt locally.
>
You've lost me. What are you attempting to do again?
^ permalink raw reply
* Re: A look at some alternative PACK file encodings
From: Nicolas Pitre @ 2006-09-07 17:20 UTC (permalink / raw)
To: linux; +Cc: gitzilla, git
In-Reply-To: <20060907084158.25725.qmail@science.horizon.com>
On Thu, 7 Sep 2006, linux@horizon.com wrote:
> A few notes:
>
>
> Re: base-128 encodings, it's a pet peeve of mine that meny people, even
> while trying to save space, waste it by allowing redundant encodings.
> The optimal way, assming msbit=1 means "more", is
>
> 0x00 -> 0 0x01 -> 1
> 0x7f -> 127 0x80 0x00 -> 128
> 0x80 0x7f -> 255 0x81 0x00 -> 256
> 0xfe 0x7f -> 16383 0xff 0x00 -> 16384
> 0xff 0x7f -> 16511 0x80 0x00 0x00 -> 16512
Indeed. But...
Since we already use 3 bit of object type and that most objects are
larger than 15 bytes this means with 2 bytes we have 11 bits or up to
2047. With your encoding that would mean 2175. So a byte would be
saved only for objects whose size is between 2048 and 2175. I don't
know what is the proportion of objects that fall into that range in the
average pack, but even for those objects that means a reduction of less
than 0.1% with an average deflate rate of 50%.
And we can forget about cases where the size would require a fourth byte
or more since saving a byte in those cases is even less significant.
So I don't think we would gain that much using that encoding
unless/until the pack format is made completely incompatible due to
other changes, and that's something we should try to avoid as much as
possible anyway.
Nicolas
^ permalink raw reply
* Re: A look at some alternative PACK file encodings
From: linux @ 2006-09-07 15:01 UTC (permalink / raw)
To: jonsmirl, linux; +Cc: git, gitzilla
In-Reply-To: <9e4733910609070719g42ff7d9buf4b798a9c6b3a7ee@mail.gmail.com>
> Yes I missed the thin pack context. Has anyone tried building a thin
> pack? Having thin packs would address major concerns with the download
> time of the initial Mozilla checkout. Right now you have to download
> 450MB of data that 99% of the people are never going to look at.
Unfortunately, no, it wouldn't. There's a second misunderstanding here:
thin packs are used by the git on-the-wire protocol to send a delta
relative to an object *that you know the recipient already has*.
This can greatly speed up incremental updates. It does NOT have
any effect on the initial checkout, because you know the recipient
has nothing, so you have to send everything.
What you seem to be talking about is what people call a "shallow clone".
Unfortunately, making a shallow clone is difficult for reasons that
are deeply embedded in git's incremental update protocol.
When you fetch from a remote repository, it needs to figure out what
you need and what you already have. The fundamental assumption which
the protocol uses to send this efficiently is that if you have a commit
object, you have all its ancestors, and their trees and blobs.
So you can say "I have 60d4684068ff1eec78f55b5888d0bd2d4cca1520"
(which is the commit behind tags/v2.6.18-rc5 from the Linux kernel)
and the server you're pulling from knows a lot about what you have.
The first hard part about a shallow repository is describing *efficiently*
to git-upload-pack what you don't have.
The second is figuring out what you should get.
Suppose that you clone from a repository that has a linear chain
of development, and you cut off everything before version X.
Simple enough so far.
Then you try to pull from someone whose work branched off at version
X-1. Do you want to fetch everything back to the common ancestor?
If you want to cut off their branch, where do we cut it off? A simple
timestamp limit? What if you get a bogus timestamp on a commit?
This has been considered by git developers quite carefully and is NOT
an easy problem. A solution is welcomed, but if you come up with an
easy one, please spend a few minutes considering the possibility that
the reason it's so easy is that your solution is flawed.
There are also problems giving sensible error messages if you try to
refer to a missing object and making your clone deeper if desired,
but those are secondary to the big problems.
> I'm not sure I would mix these external references in with other
> objects. Instead I would make a pair of packs, one only contains
> external reference stubs and is tiny. The second is the full version
> of the same data. That makes getting the old data easy, it just
> replaces the stub pack. The stub pack doesn't need to contain stubs
> for all the objects, only the ones referenced upstream.
Huh?? This is flatly impossible, and completely defeats the purpose.
This is one of those "Pray, Mr. Babbage, if you put into the machine
wrong figures, will the right answers come out?" questions.
I think there is a *serious* miscommunication here.
I have a delta-compressed object. It is stored in the pack as a series
of changes to some base object. To be of any use, it needs to identify
that base object. If the base object identification takes the form of
an intra-pack offset (the new proposal) as opposed to a global 20-byte
object ID, then the base object needs to be stored *intra-pack*.
Within the same pack.
The whole idea is "hey, we don't allow delta-ing relative to objects
not in the pack anyway, so why waste space storing such large pointers?"
This would make the currently forbidden case of a delta relative to
an object not in the same pack completely impossible. It's like
asking for the square root of 2 in roman numerals.
> I haven't tried doing this, does the current git code support having
> multiple pack files covering difference pieces of the project? Can we
> have an archive pack and a local current pack that get merged
> together?
Yes, it supports it perfectly. A pack is just a bag of objects. There is
zero required higher-level structure. In particular, you can have a
"100% blobs" pack with no tree or commit objects at all.
Git uses the object relationships in its compression heuristics, but
it has no effect on correctness.
The only thing is that the original pack was a *self-contained* bag of
objects; every object in the pack could be reconstructed given only the
pack itself. In particular, cross-pack deltas were forbidden.
However, then someone realized that making an exception for git protocol
updates would be easy and a Really Good Thing. A pack that is missing
one or more base objects is called a "thin pack".
So the rule I mentioned above has already been broken. Supporting
that is what I'm talking about.
^ permalink raw reply
* Re: A look at some alternative PACK file encodings
From: Richard Curnow @ 2006-09-07 14:39 UTC (permalink / raw)
To: git
In-Reply-To: <20060907133456.24226.qmail@science.horizon.com>
<linux <at> horizon.com> writes:
>
> For regular packs, such objects wouldn't even be present, because
> all base objects are in the pack itself.
>
It would actually be useful if this restriction were lifted.
Granted, for working repositories there's not much point, because 'git repack -a
-d' can be run regularly these days.
But for repositories served by the dumb http:// transport it makes some sense.
You don't want to run 'repack -a -d' on those, because anybody tracking them
ends up having to download the entire history again every time they pull. But
if you pack too often, you incur the cost of the packs containing lots of base
objects - it's as though the delta chain lengths can only ever be about 1 or 2.
If the packs were thin, regular pullers would already have the base objects, and
occasional pullers would just have to download more of the intervening (thin)
packs to get the missing deltas in the middle of the chains. (I think everyone
has to download all the .idx files in any case.)
^ permalink raw reply
* Re: Cygwin playbook?
From: Christopher Faylor @ 2006-09-07 14:23 UTC (permalink / raw)
To: Shawn Pearce, git, Martin Langhoff
In-Reply-To: <20060907072741.GA10652@spearce.org>
On Thu, Sep 07, 2006 at 03:27:41AM -0400, Shawn Pearce wrote:
>Git pretty much works as you would expect; its just somewhat slower
>than on a good UNIX system. Maybe its Cygwin, maybe its Windows,
>maybe its the 4+ year old system its running on. :-)
It's likely to be the fact that Windows doesn't support fork or exec
so Cygwin has to emulate both - slowly.
cgf
^ permalink raw reply
* Re: Cygwin playbook?
From: Christopher Faylor @ 2006-09-07 14:22 UTC (permalink / raw)
To: git, Martin Langhoff
In-Reply-To: <46a038f90609070018k319b748cmae7a9b30da45b3b6@mail.gmail.com>
On Thu, Sep 07, 2006 at 07:18:10PM +1200, Martin Langhoff wrote:
>Thanks! that's a starting point, though I'm intrigued about the
>packages required. Are there ways to query what packages you have
>installed (a la dpkg -l) and to install a list of packages from
>commandline?
>
>Oh, what an ignorant fop I am.
The program which shows what packages are installed is "cygcheck" (i.e.,
"man cygcheck").
The mailing list for Cygwin is mentioned at the Cygwin web site. You
might have more luck with questions like this on the Cygwin mailing
list.
cgf
^ permalink raw reply
* Re: A look at some alternative PACK file encodings
From: Jon Smirl @ 2006-09-07 14:19 UTC (permalink / raw)
To: linux@horizon.com; +Cc: git, gitzilla
In-Reply-To: <20060907133456.24226.qmail@science.horizon.com>
On 7 Sep 2006 09:34:56 -0400, linux@horizon.com <linux@horizon.com> wrote:
> >> An alternative would be to create a small "placeholder" object that
> >> just gives an ID, then refer to it by offset.
> >>
> >> That would avoid the need for an id/offset bit with every offset,
> >> and possibly save more space if the same object was referenced
> >> multiple times.
> >>
> >> And it just seems simpler.
>
> > There are 2 million objects in the Mozilla pack. This table would take:
> > 2M * (20b (sha) + 10b(object index/overhead) = 60MB
> > This 60MB is pretty much incompressible and increases download time.
> >
> > Much better if storage of the sha1s can be totally eliminated and
> > replaced by something smaller. Alternatively this map could be
> > stripped for transmission and rebuilt locally.
>
> Um, I think I wasn't clear. Objects in a "thin" pack (for network
> updating of a different pack) that are referred to but not included
> would have stand-ins containing just the object ID. Objects that *are*
> present would simply be present and referred to by offset as usual.
Yes I missed the thin pack context. Has anyone tried building a thin
pack? Having thin packs would address major concerns with the download
time of the initial Mozilla checkout. Right now you have to download
450MB of data that 99% of the people are never going to look at.
Thin packs would also set an 'archival' bit as discussed earlier. The
'archival' bit tells the local tools to leave these packs alone. You
don't want to accidentally trigger a 450MB down just because you did a
git-repack. Of course git would need to developer error messages
indicating that you asked for something in the archive that isn't
present.
I'm not sure I would mix these external references in with other
objects. Instead I would make a pair of packs, one only contains
external reference stubs and is tiny. The second is the full version
of the same data. That makes getting the old data easy, it just
replaces the stub pack. The stub pack doesn't need to contain stubs
for all the objects, only the ones referenced upstream.
I haven't tried doing this, does the current git code support having
multiple pack files covering difference pieces of the project? Can we
have an archive pack and a local current pack that get merged
together?
> Imagine you have a "thin" pack containing a delta to an object that the
> recipient has, so isn't in the pack. The delta has to specify the
> base object somehow. If the base object is in the pack, you can
> specify it by offset. If it's not, you can either:
>
> - Generalize the base object pointer to allow an object ID option, or
> - Provide a pointer to a magic kind of "external reference" pointer
> object.
>
> I was proposing the latter.
>
> For regular packs, such objects wouldn't even be present, because
> all base objects are in the pack itself.
>
> And, of course, you'd only create such objects if you needed to,
> if there was at least one pointer to them.
>
> Compared to putting the object ID directly in the pointer, it has
> Cost: An extra offset pointer and object header.
> Extra time follwoing the indirection resolving the pointer.
> Benefit: Non-indirect object pointers are a bit smaller.
> The code is simpler.
> Second and later references to the same external object are
> another offset, not another 20 bytes.
>
--
Jon Smirl
jonsmirl@gmail.com
^ permalink raw reply
* Re: A look at some alternative PACK file encodings
From: linux @ 2006-09-07 13:34 UTC (permalink / raw)
To: jonsmirl, linux; +Cc: git, gitzilla
In-Reply-To: <9e4733910609070557jd8cfc57nd4f7a8973b69f6ed@mail.gmail.com>
>> An alternative would be to create a small "placeholder" object that
>> just gives an ID, then refer to it by offset.
>>
>> That would avoid the need for an id/offset bit with every offset,
>> and possibly save more space if the same object was referenced
>> multiple times.
>>
>> And it just seems simpler.
> There are 2 million objects in the Mozilla pack. This table would take:
> 2M * (20b (sha) + 10b(object index/overhead) = 60MB
> This 60MB is pretty much incompressible and increases download time.
>
> Much better if storage of the sha1s can be totally eliminated and
> replaced by something smaller. Alternatively this map could be
> stripped for transmission and rebuilt locally.
Um, I think I wasn't clear. Objects in a "thin" pack (for network
updating of a different pack) that are referred to but not included
would have stand-ins containing just the object ID. Objects that *are*
present would simply be present and referred to by offset as usual.
Imagine you have a "thin" pack containing a delta to an object that the
recipient has, so isn't in the pack. The delta has to specify the
base object somehow. If the base object is in the pack, you can
specify it by offset. If it's not, you can either:
- Generalize the base object pointer to allow an object ID option, or
- Provide a pointer to a magic kind of "external reference" pointer
object.
I was proposing the latter.
For regular packs, such objects wouldn't even be present, because
all base objects are in the pack itself.
And, of course, you'd only create such objects if you needed to,
if there was at least one pointer to them.
Compared to putting the object ID directly in the pointer, it has
Cost: An extra offset pointer and object header.
Extra time follwoing the indirection resolving the pointer.
Benefit: Non-indirect object pointers are a bit smaller.
The code is simpler.
Second and later references to the same external object are
another offset, not another 20 bytes.
^ permalink raw reply
* [PATCH 3/4] git-archive: wire up ZIP format.
From: Franck Bui-Huu @ 2006-09-07 13:12 UTC (permalink / raw)
To: junkio; +Cc: rene.scharfe, git
In-Reply-To: <450019C3.4030001@innova-card.com>
From: Rene Scharfe <rene.scharfe@lsrfire.ath.cx>
Signed-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx>
Signed-off-by: Franck Bui-Huu <vagabon.xyz@gmail.com>
---
archive.h | 1 +
builtin-archive.c | 1 +
builtin-zip-tree.c | 28 ++++++++++++++++++++++++++++
3 files changed, 30 insertions(+), 0 deletions(-)
diff --git a/archive.h b/archive.h
index 3690c53..760776d 100644
--- a/archive.h
+++ b/archive.h
@@ -41,5 +41,6 @@ extern void parse_pathspec_arg(const cha
*
*/
extern int write_tar_archive(struct archiver_args *);
+extern int write_zip_archive(struct archiver_args *);
#endif /* ARCHIVE_H */
diff --git a/builtin-archive.c b/builtin-archive.c
index 214ec5d..fd53e9a 100644
--- a/builtin-archive.c
+++ b/builtin-archive.c
@@ -17,6 +17,7 @@ static const char archive_usage[] = \
struct archiver archivers[] = {
{ .name = "tar", .write_archive = write_tar_archive },
+ { .name = "zip", .write_archive = write_zip_archive },
};
diff --git a/builtin-zip-tree.c b/builtin-zip-tree.c
index a5b834d..788317c 100644
--- a/builtin-zip-tree.c
+++ b/builtin-zip-tree.c
@@ -8,6 +8,7 @@ #include "blob.h"
#include "tree.h"
#include "quote.h"
#include "builtin.h"
+#include "archive.h"
static const char zip_tree_usage[] =
"git-zip-tree [-0|...|-9] <tree-ish> [ <base> ]";
@@ -351,3 +352,30 @@ int cmd_zip_tree(int argc, const char **
return 0;
}
+
+int write_zip_archive(struct archiver_args *args)
+{
+ int plen = strlen(args->base);
+
+ dos_time(&args->time, &zip_date, &zip_time);
+
+ zip_dir = xmalloc(ZIP_DIRECTORY_MIN_SIZE);
+ zip_dir_size = ZIP_DIRECTORY_MIN_SIZE;
+
+ if (args->base && plen > 0 && args->base[plen - 1] == '/') {
+ char *base = strdup(args->base);
+ int baselen = strlen(base);
+
+ while (baselen > 0 && base[baselen - 1] == '/')
+ base[--baselen] = '\0';
+ write_zip_entry(args->tree->object.sha1, "", 0, base, 040777, 0);
+ free(base);
+ }
+ read_tree_recursive(args->tree, args->base, plen, 0,
+ args->pathspec, write_zip_entry);
+ write_zip_trailer(args->commit_sha1);
+
+ free(zip_dir);
+
+ return 0;
+}
--
1.4.2
^ permalink raw reply related
* [PATCH 1/4] Add git-archive
From: Franck Bui-Huu @ 2006-09-07 13:12 UTC (permalink / raw)
To: junkio; +Cc: rene.scharfe, git
In-Reply-To: <450019C3.4030001@innova-card.com>
git-archive is a command to make TAR and ZIP archives of a git tree.
It helps prevent a proliferation of git-{format}-tree commands.
Instead of directly calling git-{tar,zip}-tree command, it defines
a very simple API, that archiver should implement and register in
"git-archive.c". This API is made up by 2 functions whose prototype
is defined in "archive.h" file.
- The first one is used to parse 'extra' parameters which have
signification only for the specific archiver. That would allow
different archive backends to have different kind of options.
- The second one is used to ask to an archive backend to build
the archive given some already resolved parameters.
The main reason for making this API is to avoid using
git-{tar,zip}-tree commands, hence making them useless. Maybe it's
time for them to die ?
It also implements remote operations by defining a very simple
protocol: it first sends the name of the specific uploader followed
the repository name (git-upload-tar git://example.org/repo.git).
Then it sends options. It's done by sending a sequence of one
argument per packet, with prefix "argument ", followed by a flush.
The remote protocol is implemented in "git-archive.c" for client
side and is triggered by "--remote=<repo>" option. For example,
to fetch a TAR archive in a remote repo, you can issue:
$ git archive --format=tar --remote=git://xxx/yyy/zzz.git HEAD
We choose to not make a new command "git-fetch-archive" for example,
avoind one more GIT command which should be nice for users (less
commands to remember, keeps existing --remote option).
Signed-off-by: Franck Bui-Huu <vagabon.xyz@gmail.com>
---
.gitignore | 1
Documentation/git-archive.txt | 100 ++++++++++++++++++
Makefile | 3 -
archive.h | 41 +++++++
builtin-archive.c | 227 +++++++++++++++++++++++++++++++++++++++++
builtin.h | 1
generate-cmdlist.sh | 1
git.c | 1
8 files changed, 374 insertions(+), 1 deletions(-)
diff --git a/.gitignore b/.gitignore
index 78cb671..a3e7ca1 100644
--- a/.gitignore
+++ b/.gitignore
@@ -8,6 +8,7 @@ git-apply
git-applymbox
git-applypatch
git-archimport
+git-archive
git-bisect
git-branch
git-cat-file
diff --git a/Documentation/git-archive.txt b/Documentation/git-archive.txt
new file mode 100644
index 0000000..913528d
--- /dev/null
+++ b/Documentation/git-archive.txt
@@ -0,0 +1,100 @@
+git-archive(1)
+==============
+
+NAME
+----
+git-archive - Creates a archive of the files in the named tree
+
+
+SYNOPSIS
+--------
+'git-archive' --format=<fmt> [--list] [--prefix=<prefix>/] [<extra>]
+ [--remote=<repo>] <tree-ish> [path...]
+
+DESCRIPTION
+-----------
+Creates an archive of the specified format containing the tree
+structure for the named tree. If <prefix> is specified it is
+prepended to the filenames in the archive.
+
+'git-archive' behaves differently when given a tree ID versus when
+given a commit ID or tag ID. In the first case the current time is
+used as modification time of each file in the archive. In the latter
+case the commit time as recorded in the referenced commit object is
+used instead. Additionally the commit ID is stored in a global
+extended pax header if the tar format is used; it can be extracted
+using 'git-get-tar-commit-id'. In ZIP files it is stored as a file
+comment.
+
+OPTIONS
+-------
+
+--format=<fmt>::
+ Format of the resulting archive: 'tar', 'zip'...
+
+--list::
+ Show all available formats.
+
+--prefix=<prefix>/::
+ Prepend <prefix>/ to each filename in the archive.
+
+<extra>::
+ This can be any options that the archiver backend understand.
+
+--remote=<repo>::
+ Instead of making a tar archive from local repository,
+ retrieve a tar archive from a remote repository.
+
+<tree-ish>::
+ The tree or commit to produce an archive for.
+
+path::
+ If one or more paths are specified, include only these in the
+ archive, otherwise include all files and subdirectories.
+
+CONFIGURATION
+-------------
+By default, file and directories modes are set to 0666 or 0777 in tar
+archives. It is possible to change this by setting the "umask" variable
+in the repository configuration as follows :
+
+[tar]
+ umask = 002 ;# group friendly
+
+The special umask value "user" indicates that the user's current umask
+will be used instead. The default value remains 0, which means world
+readable/writable files and directories.
+
+EXAMPLES
+--------
+git archive --format=tar --prefix=junk/ HEAD | (cd /var/tmp/ && tar xf -)::
+
+ Create a tar archive that contains the contents of the
+ latest commit on the current branch, and extracts it in
+ `/var/tmp/junk` directory.
+
+git archive --format=tar --prefix=git-1.4.0/ v1.4.0 | gzip >git-1.4.0.tar.gz::
+
+ Create a compressed tarball for v1.4.0 release.
+
+git archive --format=tar --prefix=git-1.4.0/ v1.4.0{caret}\{tree\} | gzip >git-1.4.0.tar.gz::
+
+ Create a compressed tarball for v1.4.0 release, but without a
+ global extended pax header.
+
+git archive --format=zip --prefix=git-docs/ HEAD:Documentation/ > git-1.4.0-docs.zip::
+
+ Put everything in the current head's Documentation/ directory
+ into 'git-1.4.0-docs.zip', with the prefix 'git-docs/'.
+
+Author
+------
+Written by Franck Bui-Huu and Rene Scharfe.
+
+Documentation
+--------------
+Documentation by David Greaves, Junio C Hamano and the git-list <git@vger.kernel.org>.
+
+GIT
+---
+Part of the gitlink:git[7] suite
diff --git a/Makefile b/Makefile
index 389daf7..51ed4dd 100644
--- a/Makefile
+++ b/Makefile
@@ -242,7 +242,7 @@ LIB_FILE=libgit.a
XDIFF_LIB=xdiff/lib.a
LIB_H = \
- blob.h cache.h commit.h csum-file.h delta.h \
+ archive.h blob.h cache.h commit.h csum-file.h delta.h \
diff.h object.h pack.h para-walk.h pkt-line.h quote.h refs.h \
run-command.h strbuf.h tag.h tree.h git-compat-util.h revision.h \
tree-walk.h log-tree.h dir.h path-list.h unpack-trees.h builtin.h
@@ -267,6 +267,7 @@ LIB_OBJS = \
BUILTIN_OBJS = \
builtin-add.o \
builtin-apply.o \
+ builtin-archive.o \
builtin-cat-file.o \
builtin-checkout-index.o \
builtin-check-ref-format.o \
diff --git a/archive.h b/archive.h
new file mode 100644
index 0000000..f33398e
--- /dev/null
+++ b/archive.h
@@ -0,0 +1,41 @@
+#ifndef ARCHIVE_H
+#define ARCHIVE_H
+
+#define MAX_EXTRA_ARGS 32
+#define MAX_ARGS (MAX_EXTRA_ARGS + 32)
+
+struct archiver_args {
+ const char *base;
+ struct tree *tree;
+ const unsigned char *commit_sha1;
+ time_t time;
+ const char **pathspec;
+ void *extra;
+};
+
+typedef int (*write_archive_fn_t)(struct archiver_args *);
+
+typedef void *(*parse_extra_args_fn_t)(int argc, const char **argv);
+
+struct archiver {
+ const char *name;
+ const char *remote;
+ struct archiver_args args;
+ write_archive_fn_t write_archive;
+ parse_extra_args_fn_t parse_extra;
+};
+
+extern struct archiver archivers[];
+
+extern int parse_archive_args(int argc,
+ const char **argv,
+ struct archiver **ar);
+
+extern void parse_treeish_arg(const char **treeish,
+ struct archiver_args *ar_args,
+ const char *prefix);
+
+extern void parse_pathspec_arg(const char **pathspec,
+ struct archiver_args *args);
+
+#endif /* ARCHIVE_H */
diff --git a/builtin-archive.c b/builtin-archive.c
new file mode 100644
index 0000000..6064358
--- /dev/null
+++ b/builtin-archive.c
@@ -0,0 +1,227 @@
+/*
+ * Copyright (c) 2006 Franck Bui-Huu
+ * Copyright (c) 2006 Rene Scharfe
+ */
+#include <time.h>
+#include "cache.h"
+#include "builtin.h"
+#include "archive.h"
+#include "commit.h"
+#include "tree-walk.h"
+#include "exec_cmd.h"
+#include "pkt-line.h"
+
+static const char archive_usage[] = \
+"git-archive --format=<fmt> [--prefix=<prefix>/] [<extra>] <tree-ish> [path...]";
+
+
+struct archiver archivers[] = { };
+
+
+static int run_remote_archiver(struct archiver *ar, int argc,
+ const char **argv)
+{
+ char *url, buf[1024];
+ int fd[2], i, len, rv;
+ pid_t pid;
+
+ sprintf(buf, "git-upload-archive");
+
+ url = strdup(ar->remote);
+ pid = git_connect(fd, url, buf);
+ if (pid < 0)
+ return pid;
+
+ for (i = 1; i < argc; i++) {
+ if (!strncmp(argv[i], "--remote=", 9))
+ continue;
+ packet_write(fd[1], "argument %s\n", argv[i]);
+ }
+ packet_flush(fd[1]);
+
+ len = packet_read_line(fd[0], buf, sizeof(buf));
+ if (!len)
+ die("git-archive: expected ACK/NAK, got EOF");
+ if (buf[len-1] == '\n')
+ buf[--len] = 0;
+ if (strcmp(buf, "ACK")) {
+ if (len > 5 && !strncmp(buf, "NACK ", 5))
+ die("git-archive: NACK %s", buf + 5);
+ die("git-archive: protocol error");
+ }
+
+ len = packet_read_line(fd[0], buf, sizeof(buf));
+ if (len)
+ die("git-archive: expected a flush");
+
+ /* Now, start reading from fd[0] and spit it out to stdout */
+ rv = copy_fd(fd[0], 1);
+
+ close(fd[0]);
+ rv |= finish_connect(pid);
+
+ return !!rv;
+}
+
+static struct archiver *get_archiver(const char *name)
+{
+ struct archiver *ar = NULL;
+ int i;
+
+ for (i = 0; i < ARRAY_SIZE(archivers); i++) {
+ if (!strcmp(name, archivers[i].name)) {
+ ar = &archivers[i];
+ break;
+ }
+ }
+ return ar;
+}
+
+void parse_pathspec_arg(const char **pathspec, struct archiver_args *ar_args)
+{
+ ar_args->pathspec = get_pathspec(ar_args->base, pathspec);
+}
+
+void parse_treeish_arg(const char **argv, struct archiver_args *ar_args,
+ const char *prefix)
+{
+ const char *name = argv[0];
+ const unsigned char *commit_sha1;
+ time_t archive_time;
+ struct tree *tree;
+ struct commit *commit;
+ unsigned char sha1[20];
+
+ if (get_sha1(name, sha1))
+ die("Not a valid object name");
+
+ commit = lookup_commit_reference_gently(sha1, 1);
+ if (commit) {
+ commit_sha1 = commit->object.sha1;
+ archive_time = commit->date;
+ } else {
+ archive_time = time(NULL);
+ }
+
+ tree = parse_tree_indirect(sha1);
+ if (tree == NULL)
+ die("not a tree object");
+
+ if (prefix) {
+ unsigned char tree_sha1[20];
+ unsigned int mode;
+ int err;
+
+ err = get_tree_entry(tree->object.sha1, prefix,
+ tree_sha1, &mode);
+ if (err || !S_ISDIR(mode))
+ die("current working directory is untracked");
+
+ free(tree);
+ tree = parse_tree_indirect(tree_sha1);
+ }
+ //free(tree);
+ ar_args->tree = tree;
+ ar_args->commit_sha1 = commit_sha1;
+ ar_args->time = archive_time;
+}
+
+static const char *default_parse_extra(struct archiver *ar,
+ const char **argv)
+{
+ static char msg[64];
+
+ snprintf(msg, sizeof(msg) - 4, "'%s' format does not handle %s",
+ ar->name, *argv);
+
+ return strcat(msg, "...");
+}
+
+int parse_archive_args(int argc, const char **argv, struct archiver **ar)
+{
+ const char *extra_argv[MAX_EXTRA_ARGS];
+ int extra_argc = 0;
+ const char *format = NULL; /* some default values */
+ const char *remote = NULL;
+ const char *base = "";
+ int list = 0;
+ int i;
+
+ for (i = 1; i < argc; i++) {
+ const char *arg = argv[i];
+
+ if (!strcmp(arg, "--list") || !strcmp(arg, "-l")) {
+ list = 1;
+ continue;
+ }
+ if (!strncmp(arg, "--format=", 9)) {
+ format = arg + 9;
+ continue;
+ }
+ if (!strncmp(arg, "--prefix=", 9)) {
+ base = arg + 9;
+ continue;
+ }
+ if (!strncmp(arg, "--remote=", 9)) {
+ remote = arg + 9;
+ continue;
+ }
+ if (!strcmp(arg, "--")) {
+ i++;
+ break;
+ }
+ if (arg[0] == '-') {
+ extra_argv[extra_argc++] = arg;
+ continue;
+ }
+ break;
+ }
+ if (list) {
+ if (!remote) {
+ for (i = 0; i < ARRAY_SIZE(archivers); i++)
+ printf("%s\n", archivers[i].name);
+ exit(0);
+ }
+ die("--list and --remote are mutually exclusive");
+ }
+ if (argc - i < 1) {
+ die("%s", archive_usage);
+ }
+ if (!format){
+ die("You must specify an archive format");
+ }
+ *ar = get_archiver(format);
+ if (*ar == NULL) {
+ die("Unknown archive format '%s'", format);
+ }
+ if (extra_argc) {
+ if (!(*ar)->parse_extra) {
+ die("%s", default_parse_extra(*ar, extra_argv));
+ }
+ (*ar)->args.extra = (*ar)->parse_extra(extra_argc, extra_argv);
+ }
+ (*ar)->remote = remote;
+ (*ar)->args.base = base;
+
+ return i;
+}
+
+int cmd_archive(int argc, const char **argv, const char *prefix)
+{
+ struct archiver *ar;
+ int tree_idx;
+
+ tree_idx = parse_archive_args(argc, argv, &ar);
+
+ if (ar->remote)
+ return run_remote_archiver(ar, argc, argv);
+
+ if (prefix == NULL)
+ prefix = setup_git_directory();
+
+ argv += tree_idx;
+ parse_treeish_arg(argv, &ar->args, prefix);
+ parse_pathspec_arg(argv + 1, &ar->args);
+
+ return ar->write_archive(&ar->args);
+}
diff --git a/builtin.h b/builtin.h
index 8472c79..2391afb 100644
--- a/builtin.h
+++ b/builtin.h
@@ -15,6 +15,7 @@ extern int write_tree(unsigned char *sha
extern int cmd_add(int argc, const char **argv, const char *prefix);
extern int cmd_apply(int argc, const char **argv, const char *prefix);
+extern int cmd_archive(int argc, const char **argv, const char *prefix);
extern int cmd_cat_file(int argc, const char **argv, const char *prefix);
extern int cmd_checkout_index(int argc, const char **argv, const char *prefix);
extern int cmd_check_ref_format(int argc, const char **argv, const char *prefix);
diff --git a/generate-cmdlist.sh b/generate-cmdlist.sh
index ec1eda2..5450918 100755
--- a/generate-cmdlist.sh
+++ b/generate-cmdlist.sh
@@ -12,6 +12,7 @@ struct cmdname_help common_cmds[] = {"
sort <<\EOF |
add
apply
+archive
bisect
branch
checkout
diff --git a/git.c b/git.c
index 82c8fee..c62c5cf 100644
--- a/git.c
+++ b/git.c
@@ -218,6 +218,7 @@ static void handle_internal_command(int
} commands[] = {
{ "add", cmd_add, RUN_SETUP },
{ "apply", cmd_apply },
+ { "archive", cmd_archive },
{ "cat-file", cmd_cat_file, RUN_SETUP },
{ "checkout-index", cmd_checkout_index, RUN_SETUP },
{ "check-ref-format", cmd_check_ref_format },
--
1.4.2
^ permalink raw reply related
* [PATCH 4/4] Add git-upload-archive
From: Franck Bui-Huu @ 2006-09-07 13:12 UTC (permalink / raw)
To: junkio; +Cc: rene.scharfe, git
In-Reply-To: <450019C3.4030001@innova-card.com>
This command implements the git archive protocol on the server
side. This command is not intended to be used by the end user.
Underlying git-archive command line options are sent over the
protocol from "git-archive --remote=...", just like upload-tar
currently does with "git-tar-tree=...".
As for "git-archive" command implementation, this new command
does not execute any existing "git-{tar,zip}-tree" but rely
on the archive API defined by "git-archive" patch. Hence we
get 2 good points:
- "git-archive" and "git-upload-archive" share all option
parsing code.
- All kind of git-upload-{tar,zip} could be removed.
Signed-off-by: Franck Bui-Huu <vagabon.xyz@gmail.com>
---
.gitignore | 1
Documentation/git-upload-archive.txt | 37 +++++++++++++++++
Makefile | 1
builtin-upload-archive.c | 72 ++++++++++++++++++++++++++++++++++
builtin.h | 1
daemon.c | 7 +++
git.c | 1
7 files changed, 120 insertions(+), 0 deletions(-)
diff --git a/.gitignore b/.gitignore
index a3e7ca1..fda9bf2 100644
--- a/.gitignore
+++ b/.gitignore
@@ -119,6 +119,7 @@ git-unpack-objects
git-update-index
git-update-ref
git-update-server-info
+git-upload-archive
git-upload-pack
git-upload-tar
git-var
diff --git a/Documentation/git-upload-archive.txt b/Documentation/git-upload-archive.txt
new file mode 100644
index 0000000..388bb53
--- /dev/null
+++ b/Documentation/git-upload-archive.txt
@@ -0,0 +1,37 @@
+git-upload-archive(1)
+====================
+
+NAME
+----
+git-upload-archive - Send archive
+
+
+SYNOPSIS
+--------
+'git-upload-archive' <directory>
+
+DESCRIPTION
+-----------
+Invoked by 'git-archive --remote' and sends a generated archive to the
+other end over the git protocol.
+
+This command is usually not invoked directly by the end user. The UI
+for the protocol is on the 'git-archive' side, and the program pair
+is meant to be used to get an archive from a remote repository.
+
+OPTIONS
+-------
+<directory>::
+ The repository to get a tar archive from.
+
+Author
+------
+Written by Franck Bui-Huu.
+
+Documentation
+--------------
+Documentation by Junio C Hamano and the git-list <git@vger.kernel.org>.
+
+GIT
+---
+Part of the gitlink:git[7] suite
diff --git a/Makefile b/Makefile
index 51ed4dd..461e1d6 100644
--- a/Makefile
+++ b/Makefile
@@ -305,6 +305,7 @@ BUILTIN_OBJS = \
builtin-unpack-objects.o \
builtin-update-index.o \
builtin-update-ref.o \
+ builtin-upload-archive.o \
builtin-upload-tar.o \
builtin-verify-pack.o \
builtin-write-tree.o \
diff --git a/builtin-upload-archive.c b/builtin-upload-archive.c
new file mode 100644
index 0000000..4fad377
--- /dev/null
+++ b/builtin-upload-archive.c
@@ -0,0 +1,72 @@
+/*
+ * Copyright (c) 2006 Franck Bui-Huu
+ */
+#include <time.h>
+#include "cache.h"
+#include "builtin.h"
+#include "archive.h"
+#include "pkt-line.h"
+
+static const char upload_archive_usage[] =
+ "git-upload-archive <repo>";
+
+
+int cmd_upload_archive(int argc, const char **argv, const char *prefix)
+{
+ struct archiver *ar;
+ const char *sent_argv[MAX_ARGS];
+ const char *arg_cmd = "argument ";
+ char *p, buf[4096];
+ int treeish_idx;
+ int sent_argc;
+ int len;
+
+ if (argc != 2)
+ usage(upload_archive_usage);
+
+ if (strlen(argv[1]) > sizeof(buf))
+ die("insanely long repository name");
+
+ strcpy(buf, argv[1]); /* enter-repo smudges its argument */
+
+ if (!enter_repo(buf, 0))
+ die("not a git archive");
+
+ /* put received options in sent_argv[] */
+ sent_argc = 1;
+ sent_argv[0] = "git-upload-archive";
+ for (p = buf;;) {
+ /* This will die if not enough free space in buf */
+ len = packet_read_line(0, p, (buf + sizeof buf) - p);
+ if (len == 0)
+ break; /* got a flush */
+ if (sent_argc > MAX_ARGS - 2)
+ die("Too many options (>29)");
+
+ if (p[len-1] == '\n') {
+ p[--len] = 0;
+ }
+ if (len < strlen(arg_cmd) ||
+ strncmp(arg_cmd, p, strlen(arg_cmd)))
+ die("'argument' token or flush expected");
+
+ len -= strlen(arg_cmd);
+ memmove(p, p + strlen(arg_cmd), len);
+ sent_argv[sent_argc++] = p;
+ p += len;
+ *p++ = 0;
+ }
+ sent_argv[sent_argc] = NULL;
+
+ /* parse all options sent by the client */
+ treeish_idx = parse_archive_args(sent_argc, sent_argv, &ar);
+
+ parse_treeish_arg(sent_argv + treeish_idx, &ar->args, prefix);
+ parse_pathspec_arg(sent_argv + treeish_idx + 1, &ar->args);
+
+ packet_write(1, "ACK\n");
+ packet_flush(1);
+
+ return ar->write_archive(&ar->args);
+}
+
diff --git a/builtin.h b/builtin.h
index 2391afb..f3efb58 100644
--- a/builtin.h
+++ b/builtin.h
@@ -58,6 +58,7 @@ extern int cmd_zip_tree(int argc, const
extern int cmd_unpack_objects(int argc, const char **argv, const char *prefix);
extern int cmd_update_index(int argc, const char **argv, const char *prefix);
extern int cmd_update_ref(int argc, const char **argv, const char *prefix);
+extern int cmd_upload_archive(int argc, const char **argv, const char *prefix);
extern int cmd_upload_tar(int argc, const char **argv, const char *prefix);
extern int cmd_version(int argc, const char **argv, const char *prefix);
extern int cmd_whatchanged(int argc, const char **argv, const char *prefix);
diff --git a/daemon.c b/daemon.c
index a4a08f3..5570878 100644
--- a/daemon.c
+++ b/daemon.c
@@ -324,6 +324,12 @@ static int upload_pack(void)
return -1;
}
+static int upload_archive(void)
+{
+ execl_git_cmd("upload-archive", ".", NULL);
+ return -1;
+}
+
static int upload_tar(void)
{
execl_git_cmd("upload-tar", ".", NULL);
@@ -331,6 +337,7 @@ static int upload_tar(void)
}
static struct daemon_service daemon_service[] = {
+ { "upload-archive", "uploadarch", upload_archive, 0, 1 },
{ "upload-pack", "uploadpack", upload_pack, 1, 1 },
{ "upload-tar", "uploadtar", upload_tar, 0, 1 },
};
diff --git a/git.c b/git.c
index c62c5cf..315dc0b 100644
--- a/git.c
+++ b/git.c
@@ -261,6 +261,7 @@ static void handle_internal_command(int
{ "unpack-objects", cmd_unpack_objects, RUN_SETUP },
{ "update-index", cmd_update_index, RUN_SETUP },
{ "update-ref", cmd_update_ref, RUN_SETUP },
+ { "upload-archive", cmd_upload_archive },
{ "upload-tar", cmd_upload_tar },
{ "version", cmd_version },
{ "whatchanged", cmd_whatchanged, RUN_SETUP | USE_PAGER },
--
1.4.2
^ permalink raw reply related
* [PATCH 2/4] git-archive: wire up TAR format.
From: Franck Bui-Huu @ 2006-09-07 13:12 UTC (permalink / raw)
To: junkio; +Cc: rene.scharfe, git
In-Reply-To: <450019C3.4030001@innova-card.com>
From: Rene Scharfe <rene.scharfe@lsrfire.ath.cx>
Signed-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx>
Signed-off-by: Franck Bui-Huu <vagabon.xyz@gmail.com>
---
archive.h | 4 +++
builtin-archive.c | 4 ++-
builtin-tar-tree.c | 67 ++++++++++++++++++++++++++++++++++++++++++++++++++++
3 files changed, 74 insertions(+), 1 deletions(-)
diff --git a/archive.h b/archive.h
index f33398e..3690c53 100644
--- a/archive.h
+++ b/archive.h
@@ -37,5 +37,9 @@ extern void parse_treeish_arg(const char
extern void parse_pathspec_arg(const char **pathspec,
struct archiver_args *args);
+/*
+ *
+ */
+extern int write_tar_archive(struct archiver_args *);
#endif /* ARCHIVE_H */
diff --git a/builtin-archive.c b/builtin-archive.c
index 6064358..214ec5d 100644
--- a/builtin-archive.c
+++ b/builtin-archive.c
@@ -15,7 +15,9 @@ static const char archive_usage[] = \
"git-archive --format=<fmt> [--prefix=<prefix>/] [<extra>] <tree-ish> [path...]";
-struct archiver archivers[] = { };
+struct archiver archivers[] = {
+ { .name = "tar", .write_archive = write_tar_archive },
+};
static int run_remote_archiver(struct archiver *ar, int argc,
diff --git a/builtin-tar-tree.c b/builtin-tar-tree.c
index 61a4135..1134730 100644
--- a/builtin-tar-tree.c
+++ b/builtin-tar-tree.c
@@ -9,6 +9,7 @@ #include "strbuf.h"
#include "tar.h"
#include "builtin.h"
#include "pkt-line.h"
+#include "archive.h"
#define RECORDSIZE (512)
#define BLOCKSIZE (RECORDSIZE * 20)
@@ -338,6 +339,72 @@ static int generate_tar(int argc, const
return 0;
}
+static int write_tar_entry(const unsigned char *sha1,
+ const char *base, int baselen,
+ const char *filename, unsigned mode, int stage)
+{
+ static struct strbuf path;
+ int filenamelen = strlen(filename);
+ void *buffer;
+ char type[20];
+ unsigned long size;
+
+ if (!path.alloc) {
+ path.buf = xmalloc(PATH_MAX);
+ path.alloc = PATH_MAX;
+ path.len = path.eof = 0;
+ }
+ if (path.alloc < baselen + filenamelen) {
+ free(path.buf);
+ path.buf = xmalloc(baselen + filenamelen);
+ path.alloc = baselen + filenamelen;
+ }
+ memcpy(path.buf, base, baselen);
+ memcpy(path.buf + baselen, filename, filenamelen);
+ path.len = baselen + filenamelen;
+ if (S_ISDIR(mode)) {
+ strbuf_append_string(&path, "/");
+ buffer = NULL;
+ size = 0;
+ } else {
+ buffer = read_sha1_file(sha1, type, &size);
+ if (!buffer)
+ die("cannot read %s", sha1_to_hex(sha1));
+ }
+
+ write_entry(sha1, &path, mode, buffer, size);
+ free(buffer);
+
+ return READ_TREE_RECURSIVE;
+}
+
+int write_tar_archive(struct archiver_args *args)
+{
+ int plen = strlen(args->base);
+
+ git_config(git_tar_config);
+
+ archive_time = args->time;
+
+ if (args->commit_sha1)
+ write_global_extended_header(args->commit_sha1);
+
+ if (args->base && plen > 0 && args->base[plen - 1] == '/') {
+ char *base = strdup(args->base);
+ int baselen = strlen(base);
+
+ while (baselen > 0 && base[baselen - 1] == '/')
+ base[--baselen] = '\0';
+ write_tar_entry(args->tree->object.sha1, "", 0, base, 040777, 0);
+ free(base);
+ }
+ read_tree_recursive(args->tree, args->base, plen, 0,
+ args->pathspec, write_tar_entry);
+ write_trailer();
+
+ return 0;
+}
+
static const char *exec = "git-upload-tar";
static int remote_tar(int argc, const char **argv)
--
1.4.2
^ permalink raw reply related
* Add git-archive [take #2]
From: Franck Bui-Huu @ 2006-09-07 13:08 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Franck, git, Rene Scharfe, Jakub Narebski
In-Reply-To: <7vr6yo2isu.fsf@assigned-by-dhcp.cox.net>
I'm sending a new version of the patchset which allows 'git-archive'
and 'git-upload-archive' command. I tried to take into account all
feedbacks made by Junio and Rene, but there are still some open points.
1/ Allow 'git-upload-archive' command to enable/disable some
formats. This should be done by 'git-upload-archive'.
2/ Can I remove 'git-upload-tar' command ? If so, the current
implementation of 'git-upload-archive' won't work with the old
'git-tar-tree --remote' command because the protocol needs a
"argument --format" to be passed to the server. Therefore we can
either modify 'git-tar-tree' or simply remove it.
3/ Should I kill 'git-zip-tree' command ? If so should I rename
builtin-zip-tree.c file into zip-tree.c or something else ?
4/ Progress indicator support. Junio wants to mimic upload-pack for
that. But it will lead in a lot of duplicated code if we don't
try to share code. Can we copy that code anyways and clean up
later ?
5/ Should we use "struct tree_desc" based interface for tree parsing
? According to Rene it doesn't worth it as soon as you actually
start to do something to the trees
6/ Simple subtree matching would be enough, at least for now.
Did I forgot something else ?
Point 1 seems to be important. As soons as we plug git-upload-archive,
we need to control what kind of formats the server deal with.
Point 2 and 3 are easy to achieve, just need a "go wild" authorization.
Point 4, seems to be high in priority since we don't want to deal with
clients and servers interoperate issues.
Points 5 and 6 can be done later. These improvements won't break if
done after releasing.
Franck
^ permalink raw reply
* Re: A look at some alternative PACK file encodings
From: Jon Smirl @ 2006-09-07 12:57 UTC (permalink / raw)
To: linux@horizon.com; +Cc: gitzilla, git
In-Reply-To: <20060907090756.30111.qmail@science.horizon.com>
On 7 Sep 2006 05:07:56 -0400, linux@horizon.com <linux@horizon.com> wrote:
> > Support for 'thin' packs would pretty much require mixing IDs and
> > (relative) offsets in the same pack file.
>
> An alternative would be to create a small "placeholder" object that
> just gives an ID, then refer to it by offset.
>
> That would avoid the need for an id/offset bit with every offset,
> and possibly save more space if the same object was referenced
> multiple times.
>
> And it just seems simpler.
There are 2 million objects in the Mozilla pack. This table would take:
2M * (20b (sha) + 10b(object index/overhead) = 60MB
This 60MB is pretty much incompressible and increases download time.
Much better if storage of the sha1s can be totally eliminated and
replaced by something smaller. Alternatively this map could be
stripped for transmission and rebuilt locally.
--
Jon Smirl
jonsmirl@gmail.com
^ permalink raw reply
* [PATCH] autoconf: Add checking for BSD-compatibile 'install' program
From: Jakub Narebski @ 2006-09-07 12:31 UTC (permalink / raw)
To: git
Uncomment `AC_PROG_INSTALL' macro in configure.ac, and setting
`INSTALL' variable in config.mak.in
Add `install-sh' shell script (required when using `AC_PROG_INSTALL'
macro) scriptversion=2005-11-07.23, taken from KDE SDK 3.5.3.
Signed-off-by: Jakub Narebski <jnareb@gmail.com>
---
Perhaps `install-sh' script should be replaced by something simplier,
for example by script which was sent to git mailing list, but AFAIK
not accepted.
config.mak.in | 2
configure.ac | 3
install-sh | 401 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++
3 files changed, 404 insertions(+), 2 deletions(-)
diff --git a/config.mak.in b/config.mak.in
index fccde61..d5b6451 100644
--- a/config.mak.in
+++ b/config.mak.in
@@ -4,7 +4,7 @@ # @configure_input@
CC = @CC@
AR = @AR@
TAR = @TAR@
-#INSTALL = @INSTALL@ # needs install-sh or install.sh in sources
+INSTALL = @INSTALL@
prefix = @prefix@
exec_prefix = @exec_prefix@
diff --git a/configure.ac b/configure.ac
index 6a980a5..1a501f5 100644
--- a/configure.ac
+++ b/configure.ac
@@ -92,11 +92,12 @@ AC_SUBST(NO_PYTHON)
AC_SUBST(PYTHON_PATH)
+## ----------------------------------------------------------------------
## Checks for programs.
AC_MSG_NOTICE([CHECKS for programs])
#
AC_PROG_CC
-#AC_PROG_INSTALL # needs install-sh or install.sh in sources
+AC_PROG_INSTALL
AC_CHECK_TOOL(AR, ar, :)
AC_CHECK_PROGS(TAR, [gtar tar])
#
diff --git a/install-sh b/install-sh
new file mode 100644
index 0000000..0ae12c0
--- /dev/null
+++ b/install-sh
@@ -0,0 +1,401 @@
+#!/bin/sh
+# install - install a program, script, or datafile
+
+scriptversion=2005-11-07.23
+
+# This originates from X11R5 (mit/util/scripts/install.sh), which was
+# later released in X11R6 (xc/config/util/install.sh) with the
+# following copyright and license.
+#
+# Copyright (C) 1994 X Consortium
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to
+# deal in the Software without restriction, including without limitation the
+# rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
+# sell copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in
+# all copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# X CONSORTIUM BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN
+# AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNEC-
+# TION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+#
+# Except as contained in this notice, the name of the X Consortium shall not
+# be used in advertising or otherwise to promote the sale, use or other deal-
+# ings in this Software without prior written authorization from the X Consor-
+# tium.
+#
+#
+# FSF changes to this file are in the public domain.
+#
+# Calling this script install-sh is preferred over install.sh, to prevent
+# `make' implicit rules from creating a file called install from it
+# when there is no Makefile.
+#
+# This script is compatible with the BSD install script, but was written
+# from scratch. It can only install one file at a time, a restriction
+# shared with many OS's install programs.
+
+# set DOITPROG to echo to test this script
+
+# Don't use :- since 4.3BSD and earlier shells don't like it.
+doit="${DOITPROG-}"
+
+# put in absolute paths if you don't have them in your path; or use env. vars.
+
+mvprog="${MVPROG-mv}"
+cpprog="${CPPROG-cp}"
+chmodprog="${CHMODPROG-chmod}"
+chownprog="${CHOWNPROG-chown}"
+chgrpprog="${CHGRPPROG-chgrp}"
+stripprog="${STRIPPROG-strip}"
+rmprog="${RMPROG-rm}"
+mkdirprog="${MKDIRPROG-mkdir}"
+
+posix_glob=
+posix_mkdir=
+
+# Symbolic mode for testing mkdir with directories.
+# It is the same as 755, but also tests that "u+" works.
+test_mode=u=rwx,g=rx,o=rx,u+wx
+
+# Desired mode of installed file.
+mode=0755
+
+# Desired mode of newly created intermediate directories.
+# It is empty if not known yet.
+intermediate_mode=
+
+chmodcmd=$chmodprog
+chowncmd=
+chgrpcmd=
+stripcmd=
+rmcmd="$rmprog -f"
+mvcmd="$mvprog"
+src=
+dst=
+dir_arg=
+dstarg=
+no_target_directory=
+
+usage="Usage: $0 [OPTION]... [-T] SRCFILE DSTFILE
+ or: $0 [OPTION]... SRCFILES... DIRECTORY
+ or: $0 [OPTION]... -t DIRECTORY SRCFILES...
+ or: $0 [OPTION]... -d DIRECTORIES...
+
+In the 1st form, copy SRCFILE to DSTFILE.
+In the 2nd and 3rd, copy all SRCFILES to DIRECTORY.
+In the 4th, create DIRECTORIES.
+
+Options:
+-c (ignored)
+-d create directories instead of installing files.
+-g GROUP $chgrpprog installed files to GROUP.
+-m MODE $chmodprog installed files to MODE.
+-o USER $chownprog installed files to USER.
+-s $stripprog installed files.
+-t DIRECTORY install into DIRECTORY.
+-T report an error if DSTFILE is a directory.
+--help display this help and exit.
+--version display version info and exit.
+
+Environment variables override the default commands:
+ CHGRPPROG CHMODPROG CHOWNPROG CPPROG MKDIRPROG MVPROG RMPROG STRIPPROG
+"
+
+while test -n "$1"; do
+ case $1 in
+ -c) shift
+ continue;;
+
+ -d) dir_arg=true
+ shift
+ continue;;
+
+ -g) chgrpcmd="$chgrpprog $2"
+ shift
+ shift
+ continue;;
+
+ --help) echo "$usage"; exit $?;;
+
+ -m) mode=$2
+ shift
+ shift
+ continue;;
+
+ -o) chowncmd="$chownprog $2"
+ shift
+ shift
+ continue;;
+
+ -s) stripcmd=$stripprog
+ shift
+ continue;;
+
+ -t) dstarg=$2
+ shift
+ shift
+ continue;;
+
+ -T) no_target_directory=true
+ shift
+ continue;;
+
+ --version) echo "$0 $scriptversion"; exit $?;;
+
+ *) # When -d is used, all remaining arguments are directories to create.
+ # When -t is used, the destination is already specified.
+ test -n "$dir_arg$dstarg" && break
+ # Otherwise, the last argument is the destination. Remove it from $@.
+ for arg
+ do
+ if test -n "$dstarg"; then
+ # $@ is not empty: it contains at least $arg.
+ set fnord "$@" "$dstarg"
+ shift # fnord
+ fi
+ shift # arg
+ dstarg=$arg
+ done
+ break;;
+ esac
+done
+
+if test -z "$1"; then
+ if test -z "$dir_arg"; then
+ echo "$0: no input file specified." >&2
+ exit 1
+ fi
+ # It's OK to call `install-sh -d' without argument.
+ # This can happen when creating conditional directories.
+ exit 0
+fi
+
+test -n "$dir_arg" || trap '(exit $?); exit' 1 2 13 15
+
+for src
+do
+ # Protect names starting with `-'.
+ case $src in
+ -*) src=./$src ;;
+ esac
+
+ if test -n "$dir_arg"; then
+ dst=$src
+ dstdir=$dst
+ test -d "$dstdir"
+ dstdir_status=$?
+ else
+
+ # Waiting for this to be detected by the "$cpprog $src $dsttmp" command
+ # might cause directories to be created, which would be especially bad
+ # if $src (and thus $dsttmp) contains '*'.
+ if test ! -f "$src" && test ! -d "$src"; then
+ echo "$0: $src does not exist." >&2
+ exit 1
+ fi
+
+ if test -z "$dstarg"; then
+ echo "$0: no destination specified." >&2
+ exit 1
+ fi
+
+ dst=$dstarg
+ # Protect names starting with `-'.
+ case $dst in
+ -*) dst=./$dst ;;
+ esac
+
+ # If destination is a directory, append the input filename; won't work
+ # if double slashes aren't ignored.
+ if test -d "$dst"; then
+ if test -n "$no_target_directory"; then
+ echo "$0: $dstarg: Is a directory" >&2
+ exit 1
+ fi
+ dstdir=$dst
+ dst=$dstdir/`basename "$src"`
+ dstdir_status=0
+ else
+ # Prefer dirname, but fall back on a substitute if dirname fails.
+ dstdir=`
+ (dirname "$dst") 2>/dev/null ||
+ expr X"$dst" : 'X\(.*[^/]\)//*[^/][^/]*/*$' \| \
+ X"$dst" : 'X\(//\)[^/]' \| \
+ X"$dst" : 'X\(//\)$' \| \
+ X"$dst" : 'X\(/\)' \| \
+ . : '\(.\)' 2>/dev/null ||
+ echo X"$dst" |
+ sed '/^X\(.*[^/]\)\/\/*[^/][^/]*\/*$/{ s//\1/; q; }
+ /^X\(\/\/\)[^/].*/{ s//\1/; q; }
+ /^X\(\/\/\)$/{ s//\1/; q; }
+ /^X\(\/\).*/{ s//\1/; q; }
+ s/.*/./; q'
+ `
+
+ test -d "$dstdir"
+ dstdir_status=$?
+ fi
+ fi
+
+ obsolete_mkdir_used=false
+
+ if test $dstdir_status != 0; then
+ case $posix_mkdir in
+ '')
+ posix_mkdir=false
+ if $mkdirprog -m $test_mode -p -- / >/dev/null 2>&1; then
+ posix_mkdir=true
+ else
+ # Remove any dirs left behind by ancient mkdir implementations.
+ rmdir ./-m "$test_mode" ./-p ./-- 2>/dev/null
+ fi ;;
+ esac
+
+ if
+ $posix_mkdir && {
+
+ # With -d, create the new directory with the user-specified mode.
+ # Otherwise, create it using the same intermediate mode that
+ # mkdir -p would use when creating intermediate directories.
+ # POSIX says that this mode is "$(umask -S),u+wx", so use that
+ # if umask -S works.
+
+ if test -n "$dir_arg"; then
+ mkdir_mode=$mode
+ else
+ case $intermediate_mode in
+ '')
+ if umask_S=`(umask -S) 2>/dev/null`; then
+ intermediate_mode=$umask_S,u+wx
+ else
+ intermediate_mode=$test_mode
+ fi ;;
+ esac
+ mkdir_mode=$intermediate_mode
+ fi
+
+ $mkdirprog -m "$mkdir_mode" -p -- "$dstdir"
+ }
+ then :
+ else
+
+ # mkdir does not conform to POSIX, or it failed possibly due to
+ # a race condition. Create the directory the slow way, step by
+ # step, checking for races as we go.
+
+ case $dstdir in
+ /*) pathcomp=/ ;;
+ -*) pathcomp=./ ;;
+ *) pathcomp= ;;
+ esac
+
+ case $posix_glob in
+ '')
+ if (set -f) 2>/dev/null; then
+ posix_glob=true
+ else
+ posix_glob=false
+ fi ;;
+ esac
+
+ oIFS=$IFS
+ IFS=/
+ $posix_glob && set -f
+ set fnord $dstdir
+ shift
+ $posix_glob && set +f
+ IFS=$oIFS
+
+ for d
+ do
+ test "x$d" = x && continue
+
+ pathcomp=$pathcomp$d
+ if test ! -d "$pathcomp"; then
+ $mkdirprog "$pathcomp"
+ # Don't fail if two instances are running concurrently.
+ test -d "$pathcomp" || exit 1
+ fi
+ pathcomp=$pathcomp/
+ done
+ obsolete_mkdir_used=true
+ fi
+ fi
+
+ if test -n "$dir_arg"; then
+ { test -z "$chowncmd" || $doit $chowncmd "$dst"; } &&
+ { test -z "$chgrpcmd" || $doit $chgrpcmd "$dst"; } &&
+ { test "$obsolete_mkdir_used$chowncmd$chgrpcmd" = false ||
+ test -z "$chmodcmd" || $doit $chmodcmd "$mode" "$dst"; } || exit 1
+ else
+
+ # Make a couple of temp file names in the proper directory.
+ dsttmp=$dstdir/_inst.$$_
+ rmtmp=$dstdir/_rm.$$_
+
+ # Trap to clean up those temp files at exit.
+ trap 'ret=$?; rm -f "$dsttmp" "$rmtmp" && exit $ret' 0
+
+ # Copy the file name to the temp name.
+ $doit $cpprog "$src" "$dsttmp" &&
+
+ # and set any options; do chmod last to preserve setuid bits.
+ #
+ # If any of these fail, we abort the whole thing. If we want to
+ # ignore errors from any of these, just make sure not to ignore
+ # errors from the above "$doit $cpprog $src $dsttmp" command.
+ #
+ { test -z "$chowncmd" || $doit $chowncmd "$dsttmp"; } \
+ && { test -z "$chgrpcmd" || $doit $chgrpcmd "$dsttmp"; } \
+ && { test -z "$stripcmd" || $doit $stripcmd "$dsttmp"; } \
+ && { test -z "$chmodcmd" || $doit $chmodcmd "$mode" "$dsttmp"; } &&
+
+ # Now rename the file to the real destination.
+ { $doit $mvcmd -f "$dsttmp" "$dst" 2>/dev/null \
+ || {
+ # The rename failed, perhaps because mv can't rename something else
+ # to itself, or perhaps because mv is so ancient that it does not
+ # support -f.
+
+ # Now remove or move aside any old file at destination location.
+ # We try this two ways since rm can't unlink itself on some
+ # systems and the destination file might be busy for other
+ # reasons. In this case, the final cleanup might fail but the new
+ # file should still install successfully.
+ {
+ if test -f "$dst"; then
+ $doit $rmcmd -f "$dst" 2>/dev/null \
+ || { $doit $mvcmd -f "$dst" "$rmtmp" 2>/dev/null \
+ && { $doit $rmcmd -f "$rmtmp" 2>/dev/null; :; }; }\
+ || {
+ echo "$0: cannot unlink or rename $dst" >&2
+ (exit 1); exit 1
+ }
+ else
+ :
+ fi
+ } &&
+
+ # Now rename the file to the real destination.
+ $doit $mvcmd "$dsttmp" "$dst"
+ }
+ } || exit 1
+
+ trap '' 0
+ fi
+done
+
+# Local variables:
+# eval: (add-hook 'write-file-hooks 'time-stamp)
+# time-stamp-start: "scriptversion="
+# time-stamp-format: "%:y-%02m-%02d.%02H"
+# time-stamp-end: "$"
+# End:
--
1.4.2
^ permalink raw reply related
* [PATCH] autoconf: Add config.cache to .gitignore
From: Jakub Narebski @ 2006-09-07 12:30 UTC (permalink / raw)
To: git
Add generated file config.cache (default cache file, when running
./configure with -C, --config-cache option) to the list of ignored
files.
Signed-off-by: Jakub Narebski <jnareb@gmail.com>
---
'make clean' does remove config.cache already.
This file is not generated by default.
.gitignore | 1 +
1 files changed, 1 insertions(+), 0 deletions(-)
diff --git a/.gitignore b/.gitignore
index 78cb671..0d608fe 100644
--- a/.gitignore
+++ b/.gitignore
@@ -141,6 +141,7 @@ git-core.spec
*.py[co]
config.mak
autom4te.cache
+config.cache
config.log
config.status
config.mak.autogen
--
1.4.2
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox