* Re: Git and GCC
From: Jon Smirl @ 2007-12-06 22:11 UTC (permalink / raw)
To: Nicolas Pitre
Cc: Linus Torvalds, Jeff King, Daniel Berlin, Harvey Harrison,
David Miller, ismail, gcc, git
In-Reply-To: <alpine.LFD.0.99999.0712061645120.555@xanadu.home>
On 12/6/07, Nicolas Pitre <nico@cam.org> wrote:
> On Thu, 6 Dec 2007, Jon Smirl wrote:
>
> > On 12/6/07, Nicolas Pitre <nico@cam.org> wrote:
> > > > When I lasted looked at the code, the problem was in evenly dividing
> > > > the work. I was using a four core machine and most of the time one
> > > > core would end up with 3-5x the work of the lightest loaded core.
> > > > Setting pack.threads up to 20 fixed the problem. With a high number of
> > > > threads I was able to get a 4hr pack to finished in something like
> > > > 1:15.
> > >
> > > But as far as I know you didn't try my latest incarnation which has been
> > > available in Git's master branch for a few months already.
> >
> > I've deleted all my giant packs. Using the kernel pack:
> > 4GB Q6600
> >
> > Using the current thread pack code I get these results.
> >
> > The interesting case is the last one. I set it to 15 threads and
> > monitored with 'top'.
> > For 0-60% compression I was at 300% CPU, 60-74% was 200% CPU and
> > 74-100% was 100% CPU. It never used all for cores. The only other
> > things running were top and my desktop. This is the same load
> > balancing problem I observed earlier.
>
> Well, that's possible with a window 25 times larger than the default.
>
> The load balancing is solved with a master thread serving relatively
> small object list segments to any work thread that finished with its
> previous segment. But the size for those segments is currently fixed to
> window * 1000 which is way too large when window == 250.
>
> I have to find a way to auto-tune that segment size somehow.
That would be nice. Threading is most important on the giant
pack/window combinations. The normal case is fast enough that I don't
real notice it. These giant pack/window combos can run 8-10 hours.
>
> But with the default window size there should not be any such noticeable
> load balancing problem.
I only spend 30 seconds in the compression phase without making the
window larger. It's not long enough to really see what is going on.
>
> Note that threading only happens in the compression phase. The count
> and write phase are hardly paralleled.
>
>
> Nicolas
>
--
Jon Smirl
jonsmirl@gmail.com
^ permalink raw reply
* Re: Git and GCC
From: Jon Smirl @ 2007-12-06 22:22 UTC (permalink / raw)
To: Nicolas Pitre
Cc: Linus Torvalds, Jeff King, Daniel Berlin, Harvey Harrison,
David Miller, ismail, gcc, git
In-Reply-To: <alpine.LFD.0.99999.0712061645120.555@xanadu.home>
On 12/6/07, Nicolas Pitre <nico@cam.org> wrote:
> On Thu, 6 Dec 2007, Jon Smirl wrote:
>
> > On 12/6/07, Nicolas Pitre <nico@cam.org> wrote:
> > > > When I lasted looked at the code, the problem was in evenly dividing
> > > > the work. I was using a four core machine and most of the time one
> > > > core would end up with 3-5x the work of the lightest loaded core.
> > > > Setting pack.threads up to 20 fixed the problem. With a high number of
> > > > threads I was able to get a 4hr pack to finished in something like
> > > > 1:15.
> > >
> > > But as far as I know you didn't try my latest incarnation which has been
> > > available in Git's master branch for a few months already.
> >
> > I've deleted all my giant packs. Using the kernel pack:
> > 4GB Q6600
> >
> > Using the current thread pack code I get these results.
> >
> > The interesting case is the last one. I set it to 15 threads and
> > monitored with 'top'.
> > For 0-60% compression I was at 300% CPU, 60-74% was 200% CPU and
> > 74-100% was 100% CPU. It never used all for cores. The only other
> > things running were top and my desktop. This is the same load
> > balancing problem I observed earlier.
>
> Well, that's possible with a window 25 times larger than the default.
Why did it never use more than three cores?
>
> The load balancing is solved with a master thread serving relatively
> small object list segments to any work thread that finished with its
> previous segment. But the size for those segments is currently fixed to
> window * 1000 which is way too large when window == 250.
>
> I have to find a way to auto-tune that segment size somehow.
>
> But with the default window size there should not be any such noticeable
> load balancing problem.
>
> Note that threading only happens in the compression phase. The count
> and write phase are hardly paralleled.
>
>
> Nicolas
>
--
Jon Smirl
jonsmirl@gmail.com
^ permalink raw reply
* Re: Git and GCC
From: David Kastrup @ 2007-12-06 22:26 UTC (permalink / raw)
To: Junio C Hamano
Cc: Jon Loeliger, Linus Torvalds, Daniel Berlin, David Miller, ismail,
gcc, Git List
In-Reply-To: <7vabonczad.fsf@gitster.siamese.dyndns.org>
Junio C Hamano <gitster@pobox.com> writes:
> Junio C Hamano <gitster@pobox.com> writes:
>
>> Jon Loeliger <jdl@freescale.com> writes:
>>
>>> I'd like to learn more about that. Can someone point me to
>>> either more documentation on it? In the absence of that,
>>> perhaps a pointer to the source code that implements it?
>>
>> See Documentation/technical/pack-heuristics.txt,
>
> A somewhat funny thing about this is ...
>
> $ git show --stat --summary b116b297
> commit b116b297a80b54632256eb89dd22ea2b140de622
> Author: Jon Loeliger <jdl@jdl.com>
> Date: Thu Mar 2 19:19:29 2006 -0600
>
> Added Packing Heursitics IRC writeup.
Ah, fishing for compliments. The cookie baking season...
--
David Kastrup, Kriemhildstr. 15, 44793 Bochum
^ permalink raw reply
* Re: Git and GCC
From: Nicolas Pitre @ 2007-12-06 22:30 UTC (permalink / raw)
To: Jon Smirl
Cc: Linus Torvalds, Jeff King, Daniel Berlin, Harvey Harrison,
David Miller, ismail, gcc, git
In-Reply-To: <9e4733910712061422w139273c0gf3cfb04c6ba8c509@mail.gmail.com>
On Thu, 6 Dec 2007, Jon Smirl wrote:
> On 12/6/07, Nicolas Pitre <nico@cam.org> wrote:
> > On Thu, 6 Dec 2007, Jon Smirl wrote:
> >
> > > On 12/6/07, Nicolas Pitre <nico@cam.org> wrote:
> > > > > When I lasted looked at the code, the problem was in evenly dividing
> > > > > the work. I was using a four core machine and most of the time one
> > > > > core would end up with 3-5x the work of the lightest loaded core.
> > > > > Setting pack.threads up to 20 fixed the problem. With a high number of
> > > > > threads I was able to get a 4hr pack to finished in something like
> > > > > 1:15.
> > > >
> > > > But as far as I know you didn't try my latest incarnation which has been
> > > > available in Git's master branch for a few months already.
> > >
> > > I've deleted all my giant packs. Using the kernel pack:
> > > 4GB Q6600
> > >
> > > Using the current thread pack code I get these results.
> > >
> > > The interesting case is the last one. I set it to 15 threads and
> > > monitored with 'top'.
> > > For 0-60% compression I was at 300% CPU, 60-74% was 200% CPU and
> > > 74-100% was 100% CPU. It never used all for cores. The only other
> > > things running were top and my desktop. This is the same load
> > > balancing problem I observed earlier.
> >
> > Well, that's possible with a window 25 times larger than the default.
>
> Why did it never use more than three cores?
You have 648366 objects total, and only 647457 of them are subject to
delta compression.
With a window size of 250 and a default thread segment of window * 1000
that means only 3 segments will be distributed to threads, hence only 3
threads with work to do.
Nicolas
^ permalink raw reply
* [OT] Re: Git and GCC
From: Randy Dunlap @ 2007-12-06 22:38 UTC (permalink / raw)
To: David Kastrup
Cc: Junio C Hamano, Jon Loeliger, Linus Torvalds, Daniel Berlin,
David Miller, ismail, gcc, Git List
In-Reply-To: <85r6hzo3y8.fsf@lola.goethe.zz>
On Thu, 06 Dec 2007 23:26:07 +0100 David Kastrup wrote:
> Junio C Hamano <gitster@pobox.com> writes:
>
> > Junio C Hamano <gitster@pobox.com> writes:
> >
> >> Jon Loeliger <jdl@freescale.com> writes:
> >>
> >>> I'd like to learn more about that. Can someone point me to
> >>> either more documentation on it? In the absence of that,
> >>> perhaps a pointer to the source code that implements it?
> >>
> >> See Documentation/technical/pack-heuristics.txt,
> >
> > A somewhat funny thing about this is ...
> >
> > $ git show --stat --summary b116b297
> > commit b116b297a80b54632256eb89dd22ea2b140de622
> > Author: Jon Loeliger <jdl@jdl.com>
> > Date: Thu Mar 2 19:19:29 2006 -0600
> >
> > Added Packing Heursitics IRC writeup.
>
> Ah, fishing for compliments. The cookie baking season...
Indeed. Here are some really good & sweet recipes (IMHO).
http://www.xenotime.net/linux/recipes/
---
~Randy
Features and documentation: http://lwn.net/Articles/260136/
^ permalink raw reply
* Re: Git and GCC
From: Jon Smirl @ 2007-12-06 22:44 UTC (permalink / raw)
To: Nicolas Pitre
Cc: Linus Torvalds, Jeff King, Daniel Berlin, Harvey Harrison,
David Miller, ismail, gcc, git
In-Reply-To: <alpine.LFD.0.99999.0712061726240.555@xanadu.home>
On 12/6/07, Nicolas Pitre <nico@cam.org> wrote:
> > > Well, that's possible with a window 25 times larger than the default.
> >
> > Why did it never use more than three cores?
>
> You have 648366 objects total, and only 647457 of them are subject to
> delta compression.
>
> With a window size of 250 and a default thread segment of window * 1000
> that means only 3 segments will be distributed to threads, hence only 3
> threads with work to do.
One little tweak and the clock time drops from 9.5 to 6 minutes. The
tweak makes all four cores work.
jonsmirl@terra:/home/apps/git$ git diff
diff --git a/builtin-pack-objects.c b/builtin-pack-objects.c
index 4f44658..e0dd12e 100644
--- a/builtin-pack-objects.c
+++ b/builtin-pack-objects.c
@@ -1645,7 +1645,7 @@ static void ll_find_deltas(struct object_entry
**list, unsigned list_size,
}
/* this should be auto-tuned somehow */
- chunk_size = window * 1000;
+ chunk_size = window * 50;
do {
unsigned sublist_size = chunk_size;
jonsmirl@terra:/home/linux/.git$ time git repack -a -d -f --depth=250
--window=250
Counting objects: 648366, done.
Compressing objects: 100% (647457/647457), done.
Writing objects: 100% (648366/648366), done.
Total 648366 (delta 539043), reused 0 (delta 0)
real 6m2.109s
user 20m0.491s
sys 0m4.608s
jonsmirl@terra:/home/linux/.git$
>
>
> Nicolas
>
--
Jon Smirl
jonsmirl@gmail.com
^ permalink raw reply related
* Gaahh!!! We've been throough this before
From: A Large Angry SCM @ 2007-12-06 22:47 UTC (permalink / raw)
To: git; +Cc: gitster
Git does not NEED a name or email address to checkout something. So how
are "Big Scary Warnings"(tm) about things that are not required
considered anything but user hostile?
Oh, and "email.com" actually belongs to someone so should not be used as
an example domain name.
"Big Scary Warning"(tm) follows:
git> git-checkout master
*** Your name cannot be determined from your system services (gecos).
Run
git config --global user.email "you@email.com"
git config --global user.name "Your Name"
to set your account's default identity.
Omit --global to set the identity only in this repository.
fatal: empty ident <unknown@localhost> not allowed
git>
^ permalink raw reply
* Re: [PATCH] Silence iconv warnings on Leopard
From: Blake Ramsdell @ 2007-12-06 23:04 UTC (permalink / raw)
To: Wincent Colaiuta; +Cc: git, jnareb
In-Reply-To: <1196968023-45284-1-git-send-email-win@wincent.com>
On Dec 6, 2007 11:07 AM, Wincent Colaiuta <win@wincent.com> wrote:
> Apple ships a newer version of iconv with Leopard (Mac OS X 10.5/Darwin
> 9). Ensure that OLD_ICONV is not set on any version of Darwin in the
> 9.x series; this should be good for at least a couple of years, when
> Darwin 10 comes out and we can invert the sense of the test to
> specifically check for Darwin 7 or 8.
This approach seems fine to me, there was some concern about matching
the OS type / version in the past, but I haven't really seen a better
answer.
> A more sophisticated and robust check is possible for those who use
> autoconf, but not everybody does that.
I did make a patch for configure.ac that does this. If it's
interesting, I'll send it along.
Blake
--
Blake Ramsdell | http://www.blakeramsdell.com
^ permalink raw reply
* [PATCH] Change from using email.com to example.com as example domain, as per RFC 2606.
From: David Symonds @ 2007-12-06 23:36 UTC (permalink / raw)
To: Junio Hamano; +Cc: git, David Symonds
Signed-off-by: David Symonds <dsymonds@gmail.com>
---
Documentation/SubmittingPatches | 2 +-
ident.c | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/Documentation/SubmittingPatches b/Documentation/SubmittingPatches
index 83bf54c..d234c8d 100644
--- a/Documentation/SubmittingPatches
+++ b/Documentation/SubmittingPatches
@@ -10,7 +10,7 @@ Checklist (and a short version for the impatient):
- the first line of the commit message should be a short
description and should skip the full stop
- if you want your work included in git.git, add a
- "Signed-off-by: Your Name <your@email.com>" line to the
+ "Signed-off-by: Your Name <your@example.com>" line to the
commit message (or just use the option "-s" when
committing) to confirm that you agree to the Developer's
Certificate of Origin
diff --git a/ident.c b/ident.c
index 07b4998..7631698 100644
--- a/ident.c
+++ b/ident.c
@@ -175,7 +175,7 @@ static const char *env_hint =
"\n"
"Run\n"
"\n"
-" git config --global user.email \"you@email.com\"\n"
+" git config --global user.email \"you@example.com\"\n"
" git config --global user.name \"Your Name\"\n"
"\n"
"to set your account\'s default identity.\n"
--
1.5.3.1
^ permalink raw reply related
* Re: [RFC] Introduce .git/BRANCH to point to the current branch
From: Robin Rosenberg @ 2007-12-06 23:39 UTC (permalink / raw)
To: Junio C Hamano
Cc: Jakub Narebski, Matthieu Moy, Salikh Zakirov, Git Mailing List
In-Reply-To: <7vir3e428i.fsf@gitster.siamese.dyndns.org>
tisdag 04 december 2007 skrev Junio C Hamano:
> Jakub Narebski <jnareb@gmail.com> writes:
>
> >> Currently, I can do:
> >>
> >> # Oh, what did this look like two commits ago?
> >> $ git checkout HEAD^^
> >> # Ah, OK, let's go back to the tip
> >> $ git checkout branch-name
> >> ^^^^^^^^^^^
> >> But I have to remember and re-type the branch name.
> >
> > No, you don't have. You can use
> > $ git checkout ORIG_HEAD
> > or
> > $ git checkout HEAD@{1}
>
> But the point is he wants to go back to the branch he came from. He
> does not want to detach HEAD at the original commit.
>
> Having said that, I am not sympathetic to "I have to remember".
I abuse git bisect for this temporary switcing. It only gives me a one
level memory, but otoh the git prompt tells me I'm on a discourse.
[me@lathund GIT (rr/abspath|BISECTING)]$ git checkout master
Switched to branch "master"
[me@lathund GIT (master|BISECTING)]$ git checkout HEAD~2
Note: moving to "HEAD~2" which isn't a local branch
If you want to create a new branch from this checkout, you may do so
(now or later) by using -b with the checkout command again. Example:
git checkout -b <new_branch_name>
HEAD is now at afcc4f7... Merge branch 'js/prune-expire'
[me@lathund GIT (afcc4f7...|BISECTING)]$ git bisect reset
Previous HEAD position was afcc4f7... Merge branch 'js/prune-expire'
Switched to branch "rr/abspath"
[me@lathund GIT (rr/abspath)]$
-- robin
^ permalink raw reply
* Better value for chunk_size when threaded
From: Jon Smirl @ 2007-12-06 23:58 UTC (permalink / raw)
To: Git Mailing List, Nicolas Pitre
I tried some various ideas out for chunk_size and the best strategy I
found was to simply set it to a constant. How does 20,000 work on
other CPUs?
I'd turn on default threaded support with this change. With threads=1
versus non-threaded there is no appreciable difference in the time.
Is there an API to ask how many CPUs are in the system? It would be
nice to default the number of threads equal to the number of CPUs and
only use pack.threads=X to override.
Making all of this work by default should help when outside people
decide to do a massive import.
diff --git a/builtin-pack-objects.c b/builtin-pack-objects.c
index 4f44658..4d73be8 100644
--- a/builtin-pack-objects.c
+++ b/builtin-pack-objects.c
@@ -1645,7 +1645,7 @@ static void ll_find_deltas(struct object_entry
**list, unsigned list_size,
}
/* this should be auto-tuned somehow */
- chunk_size = window * 1000;
+ chunk_size = 20000;
do {
unsigned sublist_size = chunk_size;
with chunk_size = 20000, everything is on a q6600 4GB
threads = 5
time git repack -a -d -f --depth=250 --window=250
real 6m20.123s
user 20m25.841s
sys 0m5.520s
threads = 4
time git repack -a -d -f --depth=250 --window=250
real 6m15.525s
user 20m20.852s
sys 0m5.356s
threads = 4
time git repack -a -d -f
real 1m31.537s
user 3m2.063s
sys 0m3.064s
threads = 1
time git repack -a -d -f --depth=250 --window=250
real 18m46.005s
user 18m43.122s
sys 0m1.228s
threads = 1
time git repack -a -d -f
real 2m57.774s
user 2m54.211s
sys 0m1.228s
Non-threaded
time git repack -a -d -f --depth=250 --window=250
real 18m51.183s
user 18m46.538s
sys 0m1.604s
Non-threaded
time git repack -a -d -f
real 2m54.849s
user 2m51.267s
sys 0m1.412s
--
Jon Smirl
jonsmirl@gmail.com
^ permalink raw reply related
* Re: [PATCH] Silence iconv warnings on Leopard
From: Jakub Narebski @ 2007-12-07 0:11 UTC (permalink / raw)
To: Blake Ramsdell; +Cc: Wincent Colaiuta, git
In-Reply-To: <985966520712061504s686395d6jf680363c7b3b9de7@mail.gmail.com>
On Fri, 7 Dec 2007, Blake Ramsdell wrote:
> On Dec 6, 2007 11:07 AM, Wincent Colaiuta <win@wincent.com> wrote:
>> Apple ships a newer version of iconv with Leopard (Mac OS X 10.5/Darwin
>> 9). Ensure that OLD_ICONV is not set on any version of Darwin in the
>> 9.x series; this should be good for at least a couple of years, when
>> Darwin 10 comes out and we can invert the sense of the test to
>> specifically check for Darwin 7 or 8.
>> A more sophisticated and robust check is possible for those who use
>> autoconf, but not everybody does that.
>
> I did make a patch for configure.ac that does this. If it's
> interesting, I'll send it along.
I would be interested, as I tried to make a patch to configure.ac which
does that. The problem is that it should be check that tests for compile
time _warnings_; my solution was to use '-Werror' flag to make warning
into error, and AC_COMPILE_IFELSE, but this might be gcc only solution.
Message-ID: <1196895948-25115-1-git-send-email-jnareb@gmail.com>
http://permalink.gmane.org/gmane.comp.version-control.git/67209
So please send it.
Thanks in advance
--
Jakub Narebski
Poland
^ permalink raw reply
* Re: [PATCH] Silence iconv warnings on Leopard
From: Blake Ramsdell @ 2007-12-07 0:12 UTC (permalink / raw)
To: Jakub Narebski; +Cc: Wincent Colaiuta, git
In-Reply-To: <200712070111.23283.jnareb@gmail.com>
On Dec 6, 2007 4:11 PM, Jakub Narebski <jnareb@gmail.com> wrote:
> I would be interested, as I tried to make a patch to configure.ac which
> does that. The problem is that it should be check that tests for compile
> time _warnings_; my solution was to use '-Werror' flag to make warning
> into error, and AC_COMPILE_IFELSE, but this might be gcc only solution.
That's precisely what I did. If this isn't suitable, then bummer --
impossible ;).
Blake
--
Blake Ramsdell | http://www.blakeramsdell.com
^ permalink raw reply
* Re: Git and GCC
From: Jakub Narebski @ 2007-12-07 0:29 UTC (permalink / raw)
To: Linus Torvalds
Cc: Jon Loeliger, Daniel Berlin, David Miller, Ismail Donmez, gcc,
git
In-Reply-To: <alpine.LFD.0.9999.0712061118050.13796@woody.linux-foundation.org>
Linus Torvalds <torvalds@linux-foundation.org> writes:
> On Thu, 6 Dec 2007, Jon Loeliger wrote:
>> I guess one question I posit is, would it be more accurate
>> to think of this as a "delta net" in a weighted graph rather
>> than a "delta chain"?
>
> It's certainly not a simple chain, it's more of a set of acyclic directed
> graphs in the object list. And yes, it's weigted by the size of the delta
> between objects, and the optimization problem is kind of akin to finding
> the smallest spanning tree (well, forest - since you do *not* want to
> create one large graph, you also want to make the individual trees shallow
> enough that you don't have excessive delta depth).
>
> There are good algorithms for finding minimum spanning trees, but this one
> is complicated by the fact that the biggest cost (by far!) is the
> calculation of the weights itself. So rather than really worry about
> finding the minimal tree/forest, the code needs to worry about not having
> to even calculate all the weights!
>
> (That, btw, is a common theme. A lot of git is about traversing graphs,
> like the revision graph. And most of the trivial graph problems all assume
> that you have the whole graph, but since the "whole graph" is the whole
> history of the repository, those algorithms are totally worthless, since
> they are fundamentally much too expensive - if we have to generate the
> whole history, we're already screwed for a big project. So things like
> revision graph calculation, the main performance issue is to avoid having
> to even *look* at parts of the graph that we don't need to see!)
Hmmm...
I think that these two problems (find minimal spanning forest with
limited depth and traverse graph) with the additional constraint to
avoid calculating weights / avoid calculating whole graph would be
a good problem to present at CompSci course.
Just a thought...
--
Jakub Narebski
Poland
ShadeHawk on #git
^ permalink raw reply
* Re: [PATCH] Silence iconv warnings on Leopard
From: Linus Torvalds @ 2007-12-07 0:30 UTC (permalink / raw)
To: Jakub Narebski; +Cc: Blake Ramsdell, Wincent Colaiuta, git
In-Reply-To: <200712070111.23283.jnareb@gmail.com>
On Fri, 7 Dec 2007, Jakub Narebski wrote:
>
> The problem is that it should be check that tests for compile time
> _warnings_; my solution was to use '-Werror' flag to make warning into
> error, and AC_COMPILE_IFELSE, but this might be gcc only solution.
>
> Message-ID: <1196895948-25115-1-git-send-email-jnareb@gmail.com>
> http://permalink.gmane.org/gmane.comp.version-control.git/67209
>
> So please send it.
Umm. Why not just make the test be whether the following compiles cleanly?
#include <iconv.h>
extern size_t iconv(iconv_t cd,
char **inbuf, size_t *inbytesleft,
char **outbuf, size_t *outbytesleft);
because if the compiler has seen a "const char **inbuf", then it should
error out with a "conflicting types for ‘iconv’" style message..
Just do
$CC -c test-iconv.c
or something.
Totally untested. I don't do autoconf.
Linus
^ permalink raw reply
* Re: [PATCH] Silence iconv warnings on Leopard
From: Blake Ramsdell @ 2007-12-07 0:41 UTC (permalink / raw)
To: Linus Torvalds; +Cc: Jakub Narebski, Wincent Colaiuta, git
In-Reply-To: <alpine.LFD.0.9999.0712061628070.13796@woody.linux-foundation.org>
On Dec 6, 2007 4:30 PM, Linus Torvalds <torvalds@linux-foundation.org> wrote:
> Umm. Why not just make the test be whether the following compiles cleanly?
>
> #include <iconv.h>
>
> extern size_t iconv(iconv_t cd,
> char **inbuf, size_t *inbytesleft,
> char **outbuf, size_t *outbytesleft);
>
> because if the compiler has seen a "const char **inbuf", then it should
> error out with a "conflicting types for 'iconv'" style message..
Yeah, this is what I did:
diff --git a/configure.ac b/configure.ac
index 5f8a15b..675d3e0 100644
--- a/configure.ac
+++ b/configure.ac
@@ -182,6 +182,29 @@ AC_SUBST(NEEDS_LIBICONV)
AC_SUBST(NO_ICONV)
test -n "$NEEDS_LIBICONV" && LIBS="$LIBS -liconv"
#
+# Define OLD_ICONV if the iconv function prototype uses const** (Darwin and
+# some FreeBSD installations).
+AC_DEFUN([OLDICONVTEST_SRC], [
+#include <iconv.h>
+
+int main(void)
+{
+ char* value = "test";
+
+ (void) iconv (NULL, &value, NULL, NULL, NULL);
+}
+])
+AC_MSG_CHECKING([for old iconv])
+old_CFLAGS="$CFLAGS"
+CFLAGS="$CFLAGS -Werror"
+AC_COMPILE_IFELSE(OLDICONVTEST_SRC,
+ [AC_MSG_RESULT([no])
+ OLD_ICONV=],
+ [AC_MSG_RESULT([yes])
+ OLD_ICONV=UnfortunatelyYes])
+CFLAGS="$old_CFLAGS"
+AC_SUBST(OLD_ICONV)
+#
# Define NO_DEFLATE_BOUND if deflateBound is missing from zlib.
AC_DEFUN([ZLIBTEST_SRC], [
#include <zlib.h>
The problem is that AC_COMPILE_IFELSE doesn't barf on warnings, so I
had to put in the CFLAGS hack to do -Werror (this is what Jakub did
also, I think).
So if this isn't rude to use -Werror (which is probably gcc-specific
in one or more ways), then fine. If it is rude to use -Werror, then
yeah, there needs to be some check for the warning, which I confess in
my five minutes of learning autoconf I don't understand well enough to
say if it's possible.
Blake
--
Blake Ramsdell | http://www.blakeramsdell.com
^ permalink raw reply related
* Re: [PATCH] Silence iconv warnings on Leopard
From: Blake Ramsdell @ 2007-12-07 0:44 UTC (permalink / raw)
To: Linus Torvalds; +Cc: Jakub Narebski, Wincent Colaiuta, git
In-Reply-To: <985966520712061641y4eba054dj5dea1d0308d50b12@mail.gmail.com>
On Dec 6, 2007 4:41 PM, Blake Ramsdell <blaker@gmail.com> wrote:
> On Dec 6, 2007 4:30 PM, Linus Torvalds <torvalds@linux-foundation.org> wrote:
> > Umm. Why not just make the test be whether the following compiles cleanly?
> >
> > #include <iconv.h>
> >
> > extern size_t iconv(iconv_t cd,
> > char **inbuf, size_t *inbytesleft,
> > char **outbuf, size_t *outbytesleft);
> >
> > because if the compiler has seen a "const char **inbuf", then it should
> > error out with a "conflicting types for 'iconv'" style message..
>
> Yeah, this is what I did:
My apologies. Your suggestion is completely different, and should work
without -Werror. Let me try that.
Blake
--
Blake Ramsdell | http://www.blakeramsdell.com
^ permalink raw reply
* [PATCH/RFC (take 3)] autoconf: Add test for OLD_ICONV (squelching compiler warning)
From: Jakub Narebski @ 2007-12-07 1:27 UTC (permalink / raw)
To: git
Cc: Linus Torvalds, Blake Ramsdell, Wincent Colaiuta, Junio C Hamano,
Pascal Obry, Ramsay Jones, Arjen Laarhoven, Brian Gernhardt,
Jakub Narebski
In-Reply-To: <alpine.LFD.0.9999.0712061628070.13796@woody.linux-foundation.org>
Update configure.ac (and config.mak.in) to keep up with git
development by adding [compile] test whether your library has an old
iconv(), where the second (input buffer pointer) parameter is declared
with type (const char **) (OLD_ICONV).
Test-proposed-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Jakub Narebski <jnareb@gmail.com>
---
CC-ed to all parties (in this and previous thread).
I should probably used AC_LANG_PROGRAM like for NO_C99_FORMAT instead
of generating whole programlet^W test program by hand (I hope that for
example I haven't missed some header file which needs to be included);
I have followed example for NO_ICONV / NEEDS_LIBICONV and
NO_DEFLATE_BOUND test. I'm also not sure if I have put this test in
the correct autoconf section, but that is probably matter of taste.
On Fri, 7 Dec 2007, Linus Torvalds wrote:
>
> On Fri, 7 Dec 2007, Jakub Narebski wrote:
>>
>> The problem is that it should be check that tests for compile time
>> _warnings_; my solution was to use '-Werror' flag to make warning into
>> error, and AC_COMPILE_IFELSE, but this might be gcc only solution.
>>
>> Message-ID: <1196895948-25115-1-git-send-email-jnareb@gmail.com>
>> http://permalink.gmane.org/gmane.comp.version-control.git/67209
>>
>> So please send it.
>
> Umm. Why not just make the test be whether the following compiles cleanly?
>
> #include <iconv.h>
>
> extern size_t iconv(iconv_t cd,
> char **inbuf, size_t *inbytesleft,
> char **outbuf, size_t *outbytesleft);
>
> because if the compiler has seen a "const char **inbuf", then it should
> error out with a "conflicting types for ?iconv?" style message..
[...]
> Totally untested. I don't do autoconf.
Thanks for a suggestion. Implemented in the patch below. It should
work correctly, but could those who have old iconv(), or used to have
old iconv() please check if works correctly for them?
By the way, perhaps the previous idea would work using AC_LANG_WERROR
instead of passing adding [temporarily] GCC `-Werror' option to
`CFLAGS', or use parts of its implementation.
- Macro: AC_LANG_WERROR
Normally Autoconf ignores warnings generated by the compiler,
linker, and preprocessor. If this macro is used, warnings will be
treated as fatal errors instead for the current language. This
macro is useful when the results of configuration will be used
where warnings are unacceptable; for instance, if parts of a
program are built with the GCC `-Werror' option. If the whole
program will be built using `-Werror' it is often simpler to put
`-Werror' in the compiler flags (`CFLAGS' etc.).
On Fri, 7 Dec 2007, Blake Ramsdell wrote:
> On Dec 6, 2007 4:41 PM, Blake Ramsdell <blaker@gmail.com> wrote:
>> On Dec 6, 2007 4:30 PM, Linus Torvalds <torvalds@linux-foundation.org> wrote:
>>> Umm. Why not just make the test be whether the following compiles cleanly?
>>>
>>> #include <iconv.h>
>>>
>>> extern size_t iconv(iconv_t cd,
>>> char **inbuf, size_t *inbytesleft,
>>> char **outbuf, size_t *outbytesleft);
>>>
>>> because if the compiler has seen a "const char **inbuf", then it should
>>> error out with a "conflicting types for 'iconv'" style message..
>>
>> Yeah, this is what I did:
>
> My apologies. Your suggestion is completely different, and should work
> without -Werror. Let me try that.
Is something like the patch below what you wanted to try?
config.mak.in | 1 +
configure.ac | 22 ++++++++++++++++++++++
2 files changed, 23 insertions(+), 0 deletions(-)
diff --git a/config.mak.in b/config.mak.in
index 11d256e..7d5df9b 100644
--- a/config.mak.in
+++ b/config.mak.in
@@ -41,4 +41,5 @@ NO_STRTOUMAX=@NO_STRTOUMAX@
NO_SETENV=@NO_SETENV@
NO_MKDTEMP=@NO_MKDTEMP@
NO_ICONV=@NO_ICONV@
+OLD_ICONV=@OLD_ICONV@
NO_DEFLATE_BOUND=@NO_DEFLATE_BOUND@
diff --git a/configure.ac b/configure.ac
index 5f8a15b..86be19a 100644
--- a/configure.ac
+++ b/configure.ac
@@ -212,6 +212,28 @@ test -n "$NEEDS_SOCKET" && LIBS="$LIBS -lsocket"
## Checks for header files.
+AC_MSG_NOTICE([CHECKS for header files])
+#
+# Define OLD_ICONV if your library has an old iconv(), where the second
+# (input buffer pointer) parameter is declared with type (const char **).
+AC_DEFUN([OLDICONVTEST_SRC], [[
+#include <iconv.h>
+
+extern size_t iconv(iconv_t cd,
+ char **inbuf, size_t *inbytesleft,
+ char **outbuf, size_t *outbytesleft);
+
+int main(void)
+{
+ return 0;
+}
+]])
+AC_MSG_CHECKING([for old iconv()])
+AC_COMPILE_IFELSE(OLDICONVTEST_SRC,
+ [AC_MSG_RESULT([no])],
+ [AC_MSG_RESULT([yes])
+ OLD_ICONV=UnfortunatelyYes])
+AC_SUBST(OLD_ICONV)
## Checks for typedefs, structures, and compiler characteristics.
--
1.5.3.7
^ permalink raw reply related
* Re: Better value for chunk_size when threaded
From: Nicolas Pitre @ 2007-12-07 1:27 UTC (permalink / raw)
To: Jon Smirl; +Cc: Git Mailing List
In-Reply-To: <9e4733910712061558k19fbc864ia1fb7a3431fd2603@mail.gmail.com>
On Thu, 6 Dec 2007, Jon Smirl wrote:
> I tried some various ideas out for chunk_size and the best strategy I
> found was to simply set it to a constant. How does 20,000 work on
> other CPUs?
That depends on the object size. If you have a repo with big objects
but only 1000 of them for example, then the constant doesn't work.
Ideally I'd opt for a value that tend towards around 5 seconds worth of
work per segment, or something like that. Maybe using the actual
objects size could be another way.
> I'd turn on default threaded support with this change. With threads=1
> versus non-threaded there is no appreciable difference in the time.
Would need a way to determine pthreads availability from Makefile.
> Is there an API to ask how many CPUs are in the system? It would be
> nice to default the number of threads equal to the number of CPUs and
> only use pack.threads=X to override.
If there is one besides futzing with /proc/cpuinfo I'd like to know
about it. Bonus points if it is portable.
Nicolas
^ permalink raw reply
* Re: Better value for chunk_size when threaded
From: Jon Smirl @ 2007-12-07 1:37 UTC (permalink / raw)
To: Nicolas Pitre; +Cc: Git Mailing List
In-Reply-To: <alpine.LFD.0.99999.0712062014060.555@xanadu.home>
On 12/6/07, Nicolas Pitre <nico@cam.org> wrote:
> On Thu, 6 Dec 2007, Jon Smirl wrote:
>
> > I tried some various ideas out for chunk_size and the best strategy I
> > found was to simply set it to a constant. How does 20,000 work on
> > other CPUs?
>
> That depends on the object size. If you have a repo with big objects
> but only 1000 of them for example, then the constant doesn't work.
How about defaulting it to 20,000 and allowing an override? It's not
fatal if we guess wrong, we just want to most common cases to work out
of the box. 20,000 is definitely better than the current window *
1000.
> Ideally I'd opt for a value that tend towards around 5 seconds worth of
> work per segment, or something like that. Maybe using the actual
> objects size could be another way.
>
> > I'd turn on default threaded support with this change. With threads=1
> > versus non-threaded there is no appreciable difference in the time.
>
> Would need a way to determine pthreads availability from Makefile.
configure knows if pthreads is there.
> > Is there an API to ask how many CPUs are in the system? It would be
> > nice to default the number of threads equal to the number of CPUs and
> > only use pack.threads=X to override.
>
> If there is one besides futzing with /proc/cpuinfo I'd like to know
> about it. Bonus points if it is portable.
>
>
> Nicolas
>
--
Jon Smirl
jonsmirl@gmail.com
^ permalink raw reply
* In future, to replace autotools by cmake like KDE4 did?
From: J.C. Pizarro @ 2007-12-07 2:10 UTC (permalink / raw)
To: gcc, git, David Miller, Daniel Berlin, Ismail Donmez
The autotools ( automake + libtool + autoconf + ... ) generate many big
files that they have been slowing the building's computation and growing
enormously their cvs/svn/git/hg repositories because of generated files.
To see below interesting links:
1. http://dot.kde.org/1172083974/
2. http://sam.zoy.org/lectures/20050910-debian/
3. https://lwn.net/Articles/188693/
4. http://en.wikipedia.org/wiki/GNU_Build_Tools
5. http://en.wikipedia.org/wiki/GNU_Automake
The benefits could be:
* +40% faster in the KDE4 building vs KDE 3.5.6.
* elimination of redundant and unnecesary generated files as those
from autotools.
* smaller cvs/svn/git/hg repositories.
* less errors/crashes when it's configuring.
* can be improved the cmake's sources for better performance's gain.
* good and long maintainance life.
I hope if the files for cmake+make can be well integrated in GCC 4.4
J.C.Pizarro
^ permalink raw reply
* Re: git-am: catch missing author date early.
From: Len Brown @ 2007-12-07 2:34 UTC (permalink / raw)
To: git
This change made git-am more difficult to use.
I tend to check-in a fair number of patches that didn't come from an e-mail message.
In the past, I slap a Subject: and a From: on the top of the commit message
and off I go.
With this change git-am fails b/c I haven't also conjured up the time,
when in the past it figured out what time it was for me.
thanks,
-Len
commit 6e9e0327b7d7f384d8a223b4bc40330ef3e7fb61
Author: Junio C Hamano <gitster@pobox.com>
Date: Sun Dec 2 14:38:40 2007 -0800
git-am: catch missing author date early.
Even though commit-tree would default to the current time if the incoming
e-mail message somehow did not record the timestamp, it is safer to catch
the breakage sooner.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
diff --git a/git-am.sh b/git-am.sh
index 76c1c84..2e40708 100755
--- a/git-am.sh
+++ b/git-am.sh
@@ -307,9 +307,9 @@ do
GIT_AUTHOR_EMAIL="$(sed -n '/^Email/ s/Email: //p' "$dotest/info")"
GIT_AUTHOR_DATE="$(sed -n '/^Date/ s/Date: //p' "$dotest/info")"
- if test -z "$GIT_AUTHOR_EMAIL"
+ if test -z "$GIT_AUTHOR_EMAIL" || test -z "$GIT_AUTHOR_DATE"
then
- echo "Patch does not have a valid e-mail address."
+ echo "Patch does not have valid authorship information."
stop_here $this
fi
^ permalink raw reply related
* Re: Git and GCC
From: Harvey Harrison @ 2007-12-07 2:42 UTC (permalink / raw)
To: Daniel Berlin; +Cc: Linus Torvalds, David Miller, ismail, gcc, git
In-Reply-To: <4aca3dc20712061004g43f5902cw79bf633917d3ade9@mail.gmail.com>
On Thu, 2007-12-06 at 13:04 -0500, Daniel Berlin wrote:
> On 12/6/07, Linus Torvalds <torvalds@linux-foundation.org> wrote:
>
> > So the equivalent of "git gc --aggressive" - but done *properly* - is to
> > do (overnight) something like
> >
> > git repack -a -d --depth=250 --window=250
> >
> I gave this a try overnight, and it definitely helps a lot.
> Thanks!
I've updated the public mirror repo with the very-packed version.
People cloning it now should get the just over 300MB repo now.
git.infradead.org/gcc.git
Cheers,
Harvey
^ permalink raw reply
* After-the-fact submodule detection or creation
From: Michael Poole @ 2007-12-07 3:01 UTC (permalink / raw)
To: git
I have a project where most branches have one subdirectory for the
core program and other subdirectories for support scripts, programs
and libraries. The supporting subdirectories vary from branch to
branch. However, some branches contain just the core program, with no
subdirectories. Starting from scratch, I would say the core program
should be a submodule, but historically it was not populated as one.
Is there a way to easily create such a pseudo-submodule branch in git,
or are real submodules strongly preferred for this kind of scenario?
(The added twist: I ran into this while importing a Subversion
repository. git-svn identified two trees of history in the repository
-- one history containing the core plus support trees, one history
containing just the core program. Since the trees do have the same
contents, only the commits are duplicated, but that is not quite
ideal.
It seems like using the current submodule code would mean that this
kind of import would need two passes over the foreign repository,
rather than one if the branch could be created after the parent tree
is initially imported. I can live with that -- it is a rather unusual
case -- but maybe there is a better way.)
Michael Poole
^ permalink raw reply
* Re: Git and GCC
From: Linus Torvalds @ 2007-12-07 3:01 UTC (permalink / raw)
To: Harvey Harrison; +Cc: Daniel Berlin, David Miller, ismail, gcc, git
In-Reply-To: <1196995353.22471.20.camel@brick>
On Thu, 6 Dec 2007, Harvey Harrison wrote:
>
> I've updated the public mirror repo with the very-packed version.
Side note: it might be interesting to compare timings for
history-intensive stuff with and without this kind of very-packed
situation.
The very density of a smaller pack-file might be enough to overcome the
downsides (more CPU time to apply longer delta-chains), but regardless,
real numbers talks, bullshit walks. So wouldn't it be nice to have real
numbers?
One easy way to get real numbers for history would be to just time some
reasonably costly operation that uses lots of history. Ie just do a
time git blame -C gcc/regclass.c > /dev/null
and see if the deeper delta chains are very expensive.
(Yeah, the above is pretty much designed to be the worst possible case for
this kind of aggressive history packing, but I don't know if that choice
of file to try to annotate is a good choice or not. I suspect that "git
blame -C" with a CVS import is just horrid, because CVS commits tend to be
pretty big and nasty and not as localized as we've tried to make things in
the kernel, so doing the code copy detection is probably horrendously
expensive)
Linus
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox