* Re: Using GIT to store /etc (Or: How to make GIT store all file permission bits)
From: Johannes Schindelin @ 2006-12-11 13:33 UTC (permalink / raw)
To: Josef Weidendorfer; +Cc: Andreas Ericsson, Kyle Moffett, Jakub Narebski, git
In-Reply-To: <200612111313.34292.Josef.Weidendorfer@gmx.de>
Hi,
On Mon, 11 Dec 2006, Josef Weidendorfer wrote:
> On Monday 11 December 2006 11:39, Andreas Ericsson wrote:
> > > Import/export scripts literally require wrapping every single GIT
> > > command with a script that changes directory a few times, reads from a
> > > different checked-out tree, and permutes some extended-attribute data
> > > slightly before storing it in the underlying GIT tree. Even without
> > > adding any new functionality whatsoever that doubles the amount of code
> > > just for finding your repository and checking command-line arguments,
> > > and that's a crazy trade-off to make in any situation.
> > >
> >
> > GIT_DIR=/some/where/else/.git git log -p
>
> Doing this everytime you want to run a git command *is* a lot of time
> wasted for typing.
>
> The .gitlink proposal would come in handy here: you have a simple
> file instead of .git/, which links to the real repository.
I beg your pardon; I'm just joining in. Why is a symbolic link for .git
inacceptable?
Ciao,
Dscho
^ permalink raw reply
* Re: [ANNOUNCE] CGit v0.1-pre
From: Michael @ 2006-12-11 13:37 UTC (permalink / raw)
To: git
In-Reply-To: <eli7e3$srp$1@sea.gmane.org>
Jakub Narebski wrote:
> wit
> in Python (is it actively developed? where it can be downloaded from?),
here?
^ permalink raw reply
* Re: [RFC \ WISH] Add -o option to git-rev-list
From: Andreas Ericsson @ 2006-12-11 13:40 UTC (permalink / raw)
To: Marco Costalba
Cc: Linus Torvalds, Git Mailing List, Junio C Hamano, Alex Riesen,
Shawn Pearce
In-Reply-To: <e5bfff550612110459w205cb9b3lf735359012f84f7c@mail.gmail.com>
Marco Costalba wrote:
> On 12/11/06, Andreas Ericsson <ae@op5.se> wrote:
>> Marco Costalba wrote:
>> > On 12/10/06, Linus Torvalds <torvalds@osdl.org> wrote:
>> >>
>> >> Why don't you use the pipe and standard read()?
>> >>
>> >> Even if you use "popen()" and get a "FILE *" back, you can still do
>> >>
>> >> int fd = fileno(file);
>> >>
>> >> and use the raw IO capabilities.
>> >>
>> >> The thing is, temporary files can actually be faster under Linux just
>> >> because the Linux page-cache simply kicks ass. But it's not going
>> to be
>> >> _that_ big of a difference, and you need all that crazy "wait for
>> >> rev-list
>> >> to finish" and the "clean up temp-file on errors" etc crap, so
>> there's no
>> >> way it's a better solution.
>> >>
>> >
>> > Two things.
>> >
>> > - memory use: the next natural step with files is, instead of loading
>> > the file content in memory and *keep it there*, we could load one
>> > chunk at a time, index the chunk and discard. At the end we keep in
>> > memory only indexing info to quickly get to the data when needed, but
>> > the big part of data stay on the file.
>> >
>>
>> memory usage vs speed tradeoff. Since qgit is a pure user-app, I think
>> it's safe to opt for the memory hungry option. If people run it on too
>> lowbie hardware they'll just have to make do with other ways of viewing
>> the DAG or shutting down some other programs.
>>
>> > - This is probably my ignorance, but experimenting with popen() I
>> > found I could not know *when* git-rev-list ends because both feof()
>> > and ferror() give 0 after a fread() with git-rev-list already defunct.
>> > Not having a reference to the process (it is hidden behind popen() ),
>> > I had to check for 0 bytes read after a successful read (to avoid
>> > racing in case I ask the pipe before the first data it's ready) to
>> > know that job is finished and call pclose().
>> >
>>
>> (coding in MUA, so highly untested)
>>
>
> Thanks Andreas, I will do some tests with your code. But at first
> sight I fail to see (I'm not an expert on this tough ;-) ) where is
> the difference from using popen() and fileno() to get the file
> descriptors.
>
read() vs fread(), so no libc buffers. When I did comparisons with this
(a long time ago, I don't have the test-program around) in style of
read(out[0], buf, sizeof(buf));
write(fileno(stdout), buf, sizeof(buf));
with a command line like this;
cat any-file | test-program > /dev/null
I saw a static ~10ms increase in execution time compared to
cat any-file > /dev/null
regardless of the size of "any-file", so I assume this overhead comes
from the extra fork(), which you'll never get rid of unless you use
libgit.a.
--
Andreas Ericsson andreas.ericsson@op5.se
OP5 AB www.op5.se
^ permalink raw reply
* doc bug?
From: Han-Wen Nienhuys @ 2006-12-11 13:54 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
In-Reply-To: <7vslfodi7b.fsf@assigned-by-dhcp.cox.net>
Junio C Hamano escreveu:
>> asciidoc -b docbook -d manpage -f ../Documentation/asciidoc.conf ../Documentation/git-tools.txt
>> ERROR: git-tools.txt: line 2: malformed manpage title
>> ERROR: git-tools.txt: line 6: first section must be named NAME
>> ERROR: git-tools.txt: line 9: malformed NAME section body
>> FAILED: /home/lilydev/vc/git/Documentation/git-tools.txt: line 9: unexpected error:
> So that means you may have found a bug in our Makefile, or your
> make (by the way I think we pretty much requires GNU make to
> build git), or perhaps both.
Hi,
it was my bad, caused by me tweaking the Makefiles. I will be posting
the last version of my separate builddir patch separately, with
support for the documentation as well. Can you have a look so I know
whether it's worth investing any more time in?
Unfortunately, asciidoc doesn't seem to support a --include-path
option or similar, so I can't get the documentation support to work
completely.
--
^ permalink raw reply
* [PATCH] Allow building GIT in a different directory from the source directory
From: Han-Wen Nienhuys @ 2006-12-11 13:56 UTC (permalink / raw)
To: git
GIT can now be built in a separate builddirectory. This is done as
follows:
mkdir build
cd build
$my_git_dir/configure
make
In this case, configure creates an empty directory tree based on the
source directory, and wraps Makefiles from source directory in the
build directory. The rest of the functionality is delivered with the
VPATH feature of Make.
To make this work the Makefile should not mention ./ explicitly in
rules, but rather use $< and $^ to automatically look in the source
dir too.
perl/Makefile and perl/Makefile.PL need special massaging because perl
is not VPATH aware.
The documentation subdirectory has limited support for separate building,
as asciidoc doesn't have a --include-path option.
Documentation/Makefile | 36 ++++++++++++++++++++++--------------
INSTALL | 7 +++++++
Makefile | 23 ++++++++++++++---------
config.mak.in | 9 ++++-----
configure.ac | 34 ++++++++++++++++++++++++++++++++++
generate-cmdlist.sh | 2 +-
perl/Makefile | 4 ++++
perl/Makefile.PL | 16 +++++++++++++---
8 files changed, 99 insertions(+), 32 deletions(-)
diff --git a/Documentation/Makefile b/Documentation/Makefile
index c00f5f6..39bcd45 100644
--- a/Documentation/Makefile
+++ b/Documentation/Makefile
@@ -1,6 +1,6 @@
MAN1_TXT= \
- $(filter-out $(addsuffix .txt, $(ARTICLES) $(SP_ARTICLES)), \
- $(wildcard git-*.txt)) \
+ $(filter-out $(foreach i, $(ARTICLES) $(SP_ARTICLES), $(here-srcdir)/$(i).txt), \
+ $(wildcard $(here-srcdir)/git-*.txt)) \
gitk.txt
MAN7_TXT=git.txt
@@ -33,8 +33,16 @@ # DESTDIR=
INSTALL?=install
--include ../config.mak.autogen
+ifndef srcdir
+srcdir=..
+endif
+
+
+here-srcdir=$(srcdir)/Documentation
+ASCIIDOC_CONF=$(here-srcdir)/asciidoc.conf
+
+-include ../config.mak.autogen
#
# Please note that there is a minor bug in asciidoc.
# The version after 6.0.3 _will_ include the patch found here:
@@ -63,9 +71,9 @@ install: man
#
# Determine "include::" file references in asciidoc files.
#
-doc.dep : $(wildcard *.txt) build-docdep.perl
+doc.dep : $(wildcard $(here-srcdir)/*.txt) build-docdep.perl
rm -f $@+ $@
- perl ./build-docdep.perl >$@+
+ perl $(lastword $^) >$@+
mv $@+ $@
-include doc.dep
@@ -80,35 +88,35 @@ clean:
rm -f *.xml *.html *.1 *.7 howto-index.txt howto/*.html doc.dep README
%.html : %.txt
- asciidoc -b xhtml11 -d manpage -f asciidoc.conf $<
+ asciidoc -b xhtml11 -d manpage -f $(ASCIIDOC_CONF) $<
%.1 %.7 : %.xml
- xmlto -m callouts.xsl man $<
+ xmlto --searchpath $(here-srcdir)/ -m callouts.xsl man $<
%.xml : %.txt
- asciidoc -b docbook -d manpage -f asciidoc.conf $<
+ asciidoc -b docbook -d manpage -f $(ASCIIDOC_CONF) $<
git.html: git.txt README
glossary.html : glossary.txt sort_glossary.pl
cat $< | \
- perl sort_glossary.pl | \
+ perl $(lastword $^) | \
asciidoc -b xhtml11 - > glossary.html
-howto-index.txt: howto-index.sh $(wildcard howto/*.txt)
+howto-index.txt: howto-index.sh $(wildcard $(here-srcdir)/howto/*.txt)
rm -f $@+ $@
- sh ./howto-index.sh $(wildcard howto/*.txt) >$@+
+ sh $^ >$@+
mv $@+ $@
$(patsubst %,%.html,$(ARTICLES)) : %.html : %.txt
- asciidoc -b xhtml11 $*.txt
+ asciidoc -b xhtml11 $<
WEBDOC_DEST = /pub/software/scm/git/docs
-$(patsubst %.txt,%.html,$(wildcard howto/*.txt)): %.html : %.txt
+$(patsubst %.txt,%.html,$(wildcard $(here-srcdir)/howto/*.txt)): %.html : %.txt
rm -f $@+ $@
sed -e '1,/^$$/d' $< | asciidoc -b xhtml11 - >$@+
mv $@+ $@
install-webdoc : html
- sh ./install-webdoc.sh $(WEBDOC_DEST)
+ sh $(here-srcdir)/install-webdoc.sh $(WEBDOC_DEST)
diff --git a/INSTALL b/INSTALL
index 8f69039..b1355fa 100644
--- a/INSTALL
+++ b/INSTALL
@@ -21,6 +21,13 @@ set up install paths (via config.mak.aut
$ make all doc ;# as yourself
# make install install-doc ;# as root
+When using configure, git may be built in a separate directory. Use
+
+ $ mkdir builddir
+ $ cd builddir
+ $ GIT-SOURCE-DIR/configure [options]
+ $ make
+ # make install
Issues of note:
diff --git a/Makefile b/Makefile
index a1861de..8e6ec62 100644
--- a/Makefile
+++ b/Makefile
@@ -97,7 +97,7 @@ # MakeMaker (e.g. using ActiveState unde
#
GIT-VERSION-FILE: .FORCE-GIT-VERSION-FILE
- @$(SHELL_PATH) ./GIT-VERSION-GEN
+ @$(SHELL_PATH) $(srcdir)/GIT-VERSION-GEN
-include GIT-VERSION-FILE
uname_S := $(shell sh -c 'uname -s 2>/dev/null || echo not')
@@ -110,15 +110,20 @@ # CFLAGS and LDFLAGS are for the users t
CFLAGS = -g -O2 -Wall
LDFLAGS =
-ALL_CFLAGS = $(CFLAGS)
+ALL_CFLAGS = $(CFLAGS) -I.
ALL_LDFLAGS = $(LDFLAGS)
STRIP ?= strip
prefix = $(HOME)
bindir = $(prefix)/bin
gitexecdir = $(bindir)
-template_dir = $(prefix)/share/git-core/templates/
-# DESTDIR=
+datadir = $(prefix)/share
+GIT_datadir = $(datadir)/git-core
+template_dir = $(GIT_datadir)/templates/
+srcdir = .
+
+# this is usually set on the make command line.
+DESTDIR=
# default configuration for gitweb
GITWEB_CONFIG = gitweb_config.perl
@@ -595,8 +600,8 @@ git-merge-recur$X: git-merge-recursive$X
$(BUILT_INS): git$X
rm -f $@ && ln git$X $@
-common-cmds.h: Documentation/git-*.txt
- ./generate-cmdlist.sh > $@+
+common-cmds.h: $(wildcard $(srcdir)/Documentation/git-*.txt)
+ $(srcdir)/generate-cmdlist.sh $(srcdir)/Documentation/ > $@+
mv $@+ $@
$(patsubst %.sh,%,$(SCRIPT_SH)) : % : %.sh
@@ -605,7 +610,7 @@ common-cmds.h: Documentation/git-*.txt
-e 's|@@PERL@@|$(PERL_PATH_SQ)|g' \
-e 's/@@GIT_VERSION@@/$(GIT_VERSION)/g' \
-e 's/@@NO_CURL@@/$(NO_CURL)/g' \
- $@.sh >$@+
+ $^ >$@+
chmod +x $@+
mv $@+ $@
@@ -626,7 +631,7 @@ perl/perl.mak: GIT-CFLAGS
-e '}' \
-e 's|@@INSTLIBDIR@@|'"$$INSTLIBDIR"'|g' \
-e 's/@@GIT_VERSION@@/$(GIT_VERSION)/g' \
- $@.perl >$@+
+ $^ >$@+
chmod +x $@+
mv $@+ $@
@@ -670,7 +675,7 @@ git-instaweb: git-instaweb.sh gitweb/git
-e '/@@GITWEB_CGI@@/d' \
-e '/@@GITWEB_CSS@@/r gitweb/gitweb.css' \
-e '/@@GITWEB_CSS@@/d' \
- $@.sh > $@+
+ $< > $@+
chmod +x $@+
mv $@+ $@
diff --git a/config.mak.in b/config.mak.in
index 9a57840..bfbbf46 100644
--- a/config.mak.in
+++ b/config.mak.in
@@ -10,17 +10,16 @@ #INSTALL = @INSTALL@ # needs install-sh
prefix = @prefix@
exec_prefix = @exec_prefix@
bindir = @bindir@
-#gitexecdir = @libexecdir@/git-core/
+mandir=@mandir@
+
+## unused, but necessary for some autoconf versions (?)
datarootdir = @datarootdir@
-template_dir = @datadir@/git-core/templates/
-mandir=@mandir@
srcdir = @srcdir@
-VPATH = @srcdir@
export exec_prefix mandir
-export srcdir VPATH
+export srcdir
NEEDS_SSL_WITH_CRYPTO=@NEEDS_SSL_WITH_CRYPTO@
NO_OPENSSL=@NO_OPENSSL@
diff --git a/configure.ac b/configure.ac
index 34e3478..403c410 100644
--- a/configure.ac
+++ b/configure.ac
@@ -5,6 +5,7 @@ AC_PREREQ(2.59)
AC_INIT([git], [@@GIT_VERSION@@], [git@vger.kernel.org])
AC_CONFIG_SRCDIR([git.c])
+srcdir=`cd $srcdir && pwd`
config_file=config.mak.autogen
config_append=config.mak.append
@@ -334,6 +335,39 @@ ## Output files
AC_CONFIG_FILES(["${config_file}":"${config_in}":"${config_append}"])
AC_OUTPUT
+if test "$srcdir" != "."; then
+
+ ## if we're building in another directory
+ ## we need to set it up like the sourcedir
+ for d in `cd $srcdir && find . -type d -print | grep -v '\.git'` ;
+ do
+ if test ! -d $d ; then
+ echo creating $d
+ mkdir $d
+ fi
+
+ if test -f $srcdir/$d/Makefile ; then
+
+ dnl [[]] is to keep m4 happy
+ depth=`echo $d/ | sed -e 's!^\./!!g' -e 's![[^/]]*/!../!g'`
+ echo creating $d/Makefile
+
+ if test "$depth" = ""; then
+ INCLUDE_AUTOGEN="include config.mak.autogen" ;
+ else
+ INCLUDE_AUTOGEN="srcdir=$srcdir"
+ fi
+
+ cat << EOF > $d/Makefile
+$INCLUDE_AUTOGEN
+VPATH = $srcdir/$d
+export VPATH
+include $srcdir/$d/Makefile
+EOF
+
+ fi
+ done
+fi
## Cleanup
rm -f "${config_append}"
diff --git a/generate-cmdlist.sh b/generate-cmdlist.sh
index 5450918..e744fbb 100755
--- a/generate-cmdlist.sh
+++ b/generate-cmdlist.sh
@@ -47,6 +47,6 @@ do
x
s/.*git-'"$cmd"' - \(.*\)/ {"'"$cmd"'", "\1"},/
p
- }' "Documentation/git-$cmd.txt"
+ }' "$1/git-$cmd.txt"
done
echo "};"
diff --git a/perl/Makefile b/perl/Makefile
index 099beda..ae61cf6 100644
--- a/perl/Makefile
+++ b/perl/Makefile
@@ -28,6 +28,10 @@ instdir_SQ = $(subst ','\'',$(prefix)/li
echo instlibdir: >> $@
echo ' echo $(instdir_SQ)' >> $@
else
+
+PERL_SRCDIR=$(srcdir)/perl
+export PERL_SRCDIR
+
$(makfile): Makefile.PL ../GIT-CFLAGS
'$(PERL_PATH_SQ)' $< PREFIX='$(prefix_SQ)'
endif
diff --git a/perl/Makefile.PL b/perl/Makefile.PL
index 4168775..c136231 100644
--- a/perl/Makefile.PL
+++ b/perl/Makefile.PL
@@ -8,21 +8,31 @@ instlibdir:
MAKE_FRAG
}
-my %pm = ('Git.pm' => '$(INST_LIBDIR)/Git.pm');
+
+$src_prefix = "";
+if (defined ($ENV{PERL_SRCDIR})) {
+ $src_prefix = "$ENV{PERL_SRCDIR}/"
+}
+
+
+my %pm = ("$src_prefix/Git.pm" => '$(INST_LIBDIR)/Git.pm');
+
+
# We come with our own bundled Error.pm. It's not in the set of default
# Perl modules so install it if it's not available on the system yet.
eval { require Error };
if ($@) {
- $pm{'private-Error.pm'} = '$(INST_LIBDIR)/Error.pm';
+ $pm{"$src_prefix/private-Error.pm"} = '$(INST_LIBDIR)/Error.pm';
}
my %extra;
$extra{DESTDIR} = $ENV{DESTDIR} if $ENV{DESTDIR};
+
WriteMakefile(
NAME => 'Git',
- VERSION_FROM => 'Git.pm',
+ VERSION_FROM => "$src_prefix/Git.pm",
PM => \%pm,
MAKEFILE => 'perl.mak',
%extra
--
Han-Wen Nienhuys - hanwen@xs4all.nl - http://www.xs4all.nl/~hanwen
^ permalink raw reply related
* Re: git-svnimport breakage as of git-1.4.4
From: Daniel Drake @ 2006-12-11 14:27 UTC (permalink / raw)
To: Sasha Khapyorsky; +Cc: git
In-Reply-To: <20061208203230.GA9193@sashak.voltaire.com>
On Fri, 2006-12-08 at 22:32 +0200, Sasha Khapyorsky wrote:
> > # git-svnimport -v -i -C repo -r https://server repo
>
> Is this 'server' public? Can I rerun this git-svnimport?
Sorry, it is not.
> @@ -906,7 +912,7 @@ sub commit_all {
> my ($changed_paths, $revision, $author, $date, $message, $pool) = @_;
> my %p;
> while(my($path,$action) = each %$changed_paths) {
> - $p{$path} = [ $action->action,$action->copyfrom_path, $action->copyfrom_rev, $path ];
> + $p{$path} = [ $action->action,$svn_dir$action->copyfrom_path, $action->copyfrom_rev, $svn_dir$path ];
This is not valid perl - I think you wanted $svn_dir . $path
After making that modification it's not fixed though:
Fetching from 1 to 10742 ...
Tree ID 4b825dc642cb6eb9a060e54bf8d69288fbee4904
Committed change 1:/ 2004-12-22 22:53:27)
Committing initial tree 4b825dc642cb6eb9a060e54bf8d69288fbee4904
Commit ID 2614c05ac4c5f24eb89cea056a7d46c909084d8c
Writing to refs/heads/origin
DONE: 1 origin 2614c05ac4c5f24eb89cea056a7d46c909084d8c
perl: subversion/libsvn_subr/path.c:377: svn_path_basename: Assertion
`is_canonical (path, len)' failed.
Aborted
--
Daniel Drake
Brontes Technologies, A 3M Company
^ permalink raw reply
* stgit: No patches to pop
From: Bahadir Balban @ 2006-12-11 14:32 UTC (permalink / raw)
To: git
Hi,
I've just recently started using stgit (v0.11, git v1.4.3.4). On a
test repository, I do:
% stg applied
first
second
third
but when I pop I get:
% stg pop
stg pop: No patches to pop
I also get things like:
% stg pop second
popping patch "third"... done
Now at patch "second"
Is this normal?
Thanks,
^ permalink raw reply
* Re: Documentation/git-commit.txt
From: Andreas Ericsson @ 2006-12-11 14:58 UTC (permalink / raw)
To: Alan Chandler; +Cc: git, Horst H. von Brand
In-Reply-To: <200612100923.56707.alan@chandlerfamily.org.uk>
Alan Chandler wrote:
> On Sunday 10 December 2006 00:11, Horst H. von Brand wrote:
>> Alan Chandler <alan@chandlerfamily.org.uk> wrote:
>>
>> [...]
>>
>>> How about the following wording here
>>>
>>> Instead of staging the content of each file immediately after
>>> changing it, you can wait until you have completed all the changes
>>> you want to make and then use the `-a` option to tell `git commit`
>>> to look for all changes to the content it is tracking and commit it
>>> automatically. That
>> ^^^^^^^
>> files
>> (or "files whose contents")
>>
>>> is, this example ...
>> [Yes, git tracks the contents of files, but it also has a list of
>> files whose contents it is tracking]
>
> regardless, I think "it" should become "them"
>
> ... it is tracking and commit them automatically.
>
Works wonderfully, as the syntactically anal will notice that "them"
refers to "the changes" while newbies glossing over the docs will think
of "them" as "the files" (which is technically incorrect but amounts to
the same thing from that same newbies perspective).
--
Andreas Ericsson andreas.ericsson@op5.se
OP5 AB www.op5.se
^ permalink raw reply
* Re: Using GIT to store /etc (Or: How to make GIT store all file permission bits)
From: Josef Weidendorfer @ 2006-12-11 15:07 UTC (permalink / raw)
To: Johannes Schindelin; +Cc: Andreas Ericsson, Kyle Moffett, Jakub Narebski, git
In-Reply-To: <Pine.LNX.4.63.0612111432520.2807@wbgn013.biozentrum.uni-wuerzburg.de>
On Monday 11 December 2006 14:33, Johannes Schindelin wrote:
> On Mon, 11 Dec 2006, Josef Weidendorfer wrote:
>
> > On Monday 11 December 2006 11:39, Andreas Ericsson wrote:
> > > > Import/export scripts literally require wrapping every single GIT
> > > > command with a script that changes directory a few times, reads from a
> > > > different checked-out tree, and permutes some extended-attribute data
> > > > slightly before storing it in the underlying GIT tree. Even without
> > > > adding any new functionality whatsoever that doubles the amount of code
> > > > just for finding your repository and checking command-line arguments,
> > > > and that's a crazy trade-off to make in any situation.
> > > >
> > >
> > > GIT_DIR=/some/where/else/.git git log -p
> >
> > Doing this everytime you want to run a git command *is* a lot of time
> > wasted for typing.
> >
> > The .gitlink proposal would come in handy here: you have a simple
> > file instead of .git/, which links to the real repository.
>
> I beg your pardon; I'm just joining in. Why is a symbolic link for .git
> inacceptable?
You are totally right.
The .gitlink thing is tailored to allow submodule support later. It includes
some smart searching for the git repository to allow moving the checkout in
some limits without breaking the link to the repository.
Aside from this, the proposal is more flexible in that you can specify not
only GIT_DIR (or the GIT_DIR_HINT to trigger smart search), but also
GIT_INDEX_FILE and GIT_HEAD_FILE, which allows different checkouts
(with different index state and HEAD) for the same repo easily.
Which is not needed in this case.
So, sorry for the noise ;-)
^ permalink raw reply
* Re: stgit: No patches to pop
From: Catalin Marinas @ 2006-12-11 16:11 UTC (permalink / raw)
To: Bahadir Balban; +Cc: git
In-Reply-To: <7ac1e90c0612110632x56a2917cu4db33e47923f34c2@mail.gmail.com>
"Bahadir Balban" <bahadir.balban@gmail.com> wrote:
> I've just recently started using stgit (v0.11, git v1.4.3.4). On a
> test repository, I do:
>
> % stg applied
> first
> second
> third
>
> but when I pop I get:
>
> % stg pop
> stg pop: No patches to pop
Can you write a list of commands or script so that I can easily
reproduce this? It works OK for me.
> I also get things like:
>
> % stg pop second
> popping patch "third"... done
> Now at patch "second"
In version 0.11, the "pop" command is equivalent to "goto". I changed
this in the latest version (in the StGIT repository) so that "pop
<patch>" tries to only extract that patch from the stack by popping
all the patches to the given one and pushing them back without the one
you specified. It also supports patch ranges (i.e. patch1..patch4).
--
^ permalink raw reply
* Re: stgit: No patches to pop
From: Jakub Narebski @ 2006-12-11 16:20 UTC (permalink / raw)
To: git
In-Reply-To: <tnxpsaqwgxl.fsf@arm.com>
Catalin Marinas wrote:
>> I also get things like:
>>
>> % stg pop second
>> popping patch "third"... done
>> Now at patch "second"
>
> In version 0.11, the "pop" command is equivalent to "goto". I changed
> this in the latest version (in the StGIT repository) so that "pop
> <patch>" tries to only extract that patch from the stack by popping
> all the patches to the given one and pushing them back without the one
> you specified. It also supports patch ranges (i.e. patch1..patch4).
That is kind of strange. Pop should work like pop does, for example
the one in Perl or Python, removing n elements from the stack of applied
patches. Not work as "float <patch>"...
--
Jakub Narebski
Warsaw, Poland
ShadeHawk on #git
^ permalink raw reply
* Re: error: Can't lock ref
From: Jon Masters @ 2006-12-11 16:35 UTC (permalink / raw)
To: Andreas Ericsson; +Cc: git
In-Reply-To: <457D2013.4060908@op5.se>
On Mon, 2006-12-11 at 10:08 +0100, Andreas Ericsson wrote:
> Jon Masters wrote:
> > Hi folks,
> >
> > I've just pushed up a packed git repo:
> >
> > http://www.kerneltools.org/pub/downloads/module-init-tools/devel/module-init-tools.git
> >
> >
> > Attempting to clone this repo using http generates the error in
> > $subject. Brief searching via Google suggests this is an appache
> > permissions problem - anyone got a trivially obvious apache config fix
> > before I go trawling through to figure out what's up?
> What does your apache error log say?
That's just the thing - nothing. I get what looks like a good access in
the access log, but then that error - I can play around, but I know
others have discovered this to be some "known" apache problem. So, I'm
hoping someone knows what that is, before I start debugging it.
Jon.
^ permalink raw reply
* Re: [RFC \ WISH] Add -o option to git-rev-list
From: Linus Torvalds @ 2006-12-11 16:59 UTC (permalink / raw)
To: Marco Costalba
Cc: Git Mailing List, Junio C Hamano, Alex Riesen, Shawn Pearce
In-Reply-To: <e5bfff550612102317w331f3b0fyc3c6b095ff21191a@mail.gmail.com>
On Mon, 11 Dec 2006, Marco Costalba wrote:
>
> These are tipical values (warm cache):
>
> $ time git rev-list --header --boundary --parents --topo-order HEAD | cat > /dev/null
> 3.67user 0.36system 0:04.29elapsed 93%CPU (0avgtext+0avgdata 0maxresident)k
> 0inputs+0outputs (0major+18033minor)pagefaults 0swaps
That's not timing what I asked. That's just timing the "git-rev-list". You
need to time the "cat" part too. Either use a script, or do something like
time sh -c "git-rev-list ... | cat > /dev/null".
> $ time git rev-list --header --boundary --parents --topo-order HEAD >
> /tmp/tmp.txt; cat /tmp/tmp.txt > /dev/null
> 3.44user 0.28system 0:03.74elapsed 99%CPU (0avgtext+0avgdata 0maxresident)k
> 0inputs+0outputs (0major+18033minor)pagefaults 0swaps
Again - you're only timing the _writer_, not the "cat" at all.
> For some reason the CPU *never* goes up 93% with pipe (while it's easy
> in the other two cases)
That's because you're only timing 93% of the work (ignoring the "cat"
part), and in the other cases you're ignoring the "cat" (that happens
_afterwards_, not concurrently) entirely.
> OK. I just don't understand how after waiting 100ms I get only 60KB
> and stay in the loop only one cycle instead of reading,
That's because the _writer_ will block. It notices that the reader cannot
read ass fast as it can write, so it blocks when its buffers fill up.
If you want to be fast, you need to be fast at reading. It's that simple.
^ permalink raw reply
* Re: [ANNOUNCE] CGit v0.1-pre
From: Linus Torvalds @ 2006-12-11 17:01 UTC (permalink / raw)
To: Lars Hjemli; +Cc: git
In-Reply-To: <8c5c35580612110033y4f6260c9xffeaf4457f4ead21@mail.gmail.com>
On Mon, 11 Dec 2006, Lars Hjemli wrote:
>
> But this also applies to the case where the cachefile has expired,
> right? In that case, after getting the lock, I have to recheck that
> the cachefile is _still_ expired.
Yes.
> Anyway, I must say I find it rather unlikely for these cases to occur
> (frequently) in real life. That would seem to imply that the caching
> isn't really needed at all.
The point is, if you have races, you will hit them _occasionally_. It may
not be a performance problem in real life, BUT:
- quite often, you can take advantage of the serialization guarantees
that a front-side cache offers you, and do the uncached accesses (or
writing the final cache-file) knowing that there's only one thing that
does that.
- If so, then a race condition in the cache goes from a "unlikely
performance problem" to a BUG.
> > As a side note: how do you release your caches?
>
> Simple timeouts (time()-stat.st_mtime), depending on what kind of page
> was requested. If anyone cares about invalid cache content (branch
> head moving), relevant cachefiles can be deleted with an update-hook.
I was more thinking about the locking part. Again, to be safe, you should
probably take the lock before releasing any cache.
^ permalink raw reply
* Re: stgit: No patches to pop
From: Bahadir Balban @ 2006-12-11 17:05 UTC (permalink / raw)
To: Jakub Narebski; +Cc: git
In-Reply-To: <elk0c3$ovu$1@sea.gmane.org>
On 12/11/06, Jakub Narebski <jnareb@gmail.com> wrote:
> That is kind of strange. Pop should work like pop does, for example
> the one in Perl or Python, removing n elements from the stack of applied
> patches. Not work as "float <patch>"...
> --
> Jakub Narebski
> Warsaw, Poland
> ShadeHawk on #git
As a new user my first expectation was as Catalin has revised. If it's
going to stay as in 0.11 at least it would be nice to have it
described in the wiki.
Thanks,
^ permalink raw reply
* Re: [RFC \ WISH] Add -o option to git-rev-list
From: Linus Torvalds @ 2006-12-11 17:07 UTC (permalink / raw)
To: Marco Costalba
Cc: Git Mailing List, Junio C Hamano, Alex Riesen, Shawn Pearce
In-Reply-To: <Pine.LNX.4.64.0612110855550.12500@woody.osdl.org>
On Mon, 11 Dec 2006, Linus Torvalds wrote:
>
> That's not timing what I asked. That's just timing the "git-rev-list". You
> need to time the "cat" part too. Either use a script, or do something like
>
> time sh -c "git-rev-list ... | cat > /dev/null".
Btw, if you do this, you'll actually see one of the _real_ advantages of
pipes.
On an SMP system, you'll easily get output like "110%CPU", since now the
git-rev-list and the user can run in parallel. Of course, they can do so
with temp-files too, but then you have all the "is it a true EOF, or is it
just the fact that the writer hasn't written everything yet" problem (and
the _normal_ solution for that is to simply not allow any overlapping
work).
^ permalink raw reply
* [PATCH] gitweb: Don't use Content-Encoding: header in git_snapshot
From: Jakub Narebski @ 2006-12-11 17:09 UTC (permalink / raw)
To: git; +Cc: Jakub Narebski
Do not use Content-Encoding: HTTP header in git_snapshot, using
instead type according to the snapshot type (compression type).
Some of web browser take Content-Encoding: to be _transparent_
also for downloading, and store decompressed file (with incorrect
compression suffix) on download.
Signed-off-by: Jakub Narebski <jnareb@gmail.com>
---
gitweb/gitweb.perl | 3 +--
1 files changed, 1 insertions(+), 2 deletions(-)
diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl
index 7d24c10..040ee71 100755
--- a/gitweb/gitweb.perl
+++ b/gitweb/gitweb.perl
@@ -3489,8 +3489,7 @@ sub git_snapshot {
my $filename = basename($project) . "-$hash.tar.$suffix";
print $cgi->header(
- -type => 'application/x-tar',
- -content_encoding => $ctype,
+ -type => "application/$ctype",
-content_disposition => 'inline; filename="' . "$filename" . '"',
-status => '200 OK');
--
1.4.4.1
^ permalink raw reply related
* Re: [RFC \ WISH] Add -o option to git-rev-list
From: Marco Costalba @ 2006-12-11 17:28 UTC (permalink / raw)
To: Josef Weidendorfer
Cc: Linus Torvalds, Git Mailing List, Junio C Hamano, Alex Riesen,
Shawn Pearce
In-Reply-To: <200612111428.29390.Josef.Weidendorfer@gmx.de>
On 12/11/06, Josef Weidendorfer <Josef.Weidendorfer@gmx.de> wrote:
> On Monday 11 December 2006 13:52, Marco Costalba wrote:
> > On 12/11/06, Josef Weidendorfer <Josef.Weidendorfer@gmx.de> wrote:
> > > On Monday 11 December 2006 01:15, Marco Costalba wrote:
> > > > guiUpdateTimer.start(100, true);
> > >
> > > What is the result with "guiUpdateTimer.start(0, true);" ?
> > ...
> > IOW you suggest to use a brute force polling of the pipe.
>
> Ah, yes. That is probably not what you want.
>
> Why did you introduce the timer at all? What was the problem
> with QProcess and handling its signal "data available" ?
>
I did. Actually the released 1.5.3 version and all the verions before
that do use QProcess, but I experimented an increase in performance
reading the data on a timer instead of on "data available" signal.
BTW that observation led to all this stuff ;-)
The problem is that the frequency of the signal is very very high
because the producer can write a big amount of data with great speed.
We are talking of a signal "data available" each few ms.
By the way of experimenting with QProcess I have found it's better a
timer when the stream is very very fast, _then_ I have update the
reading code to take _more_ adavntage of the big chunks we were
reading with a low frequency timer and avoid a double copy, but the
increase was _already_ present also with the double copy needed by the
original 'small chunks handling' code activated by "data available"
signal.
If you take a look at released qgit you will see, in dataloader.cpp
the reading slot called on the signal.
^ permalink raw reply
* Re: [ANNOUNCE] CGit v0.1-pre
From: Linus Torvalds @ 2006-12-11 17:33 UTC (permalink / raw)
To: Lars Hjemli; +Cc: git
In-Reply-To: <Pine.LNX.4.64.0612110859200.12500@woody.osdl.org>
On Mon, 11 Dec 2006, Linus Torvalds wrote:
>
> The point is, if you have races, you will hit them _occasionally_. It may
> not be a performance problem in real life, BUT:
Side note: another problem with races is that
- exactly because they are rare, any potential problems they cause are
really hard to debug - you may have a hard time reproducing things.
- some loads may be able to trigger them thanks to very unlucky timing,
so even if the problem is "just a theoretical performance issue",
sometimes that theoretical performance problem that only happens once
in a blue moon ends up happening a _lot_ for a particular user.
I'm probably biased, simply because I've done system programming for so
long (and race conditions etc are one of the most common source of subtle
bugs), but I've long since come to the very strong opinion that locking is
simply too important to not do right.
Even if you can argue that "it doesn't matter" (and you may even be
right), I personally tend to just consider dodgy locking to be a serious
bug _whether_ it really matters or not.
Just a hangup of mine.
^ permalink raw reply
* Re: [RFC \ WISH] Add -o option to git-rev-list
From: Marco Costalba @ 2006-12-11 17:39 UTC (permalink / raw)
To: Linus Torvalds
Cc: Git Mailing List, Junio C Hamano, Alex Riesen, Shawn Pearce
In-Reply-To: <Pine.LNX.4.64.0612110905180.12500@woody.osdl.org>
On 12/11/06, Linus Torvalds <torvalds@osdl.org> wrote:
>
>
> On Mon, 11 Dec 2006, Linus Torvalds wrote:
> >
> > That's not timing what I asked. That's just timing the "git-rev-list". You
> > need to time the "cat" part too. Either use a script, or do something like
> >
> > time sh -c "git-rev-list ... | cat > /dev/null".
>
> Btw, if you do this, you'll actually see one of the _real_ advantages of
> pipes.
>
> On an SMP system, you'll easily get output like "110%CPU", since now the
> git-rev-list and the user can run in parallel. Of course, they can do so
> with temp-files too, but then you have all the "is it a true EOF, or is it
> just the fact that the writer hasn't written everything yet" problem (and
> the _normal_ solution for that is to simply not allow any overlapping
> work).
>
Sorry for wrong data, I will repost as soon as I can.
Regarding the _normal_ solution we have one more hipotesys to take
advantage of: git-rev-list when has nothing more to read..exits.
This is what lead to the exit loop condition based on git-rev-list is
still alive, instead of messing with an EOF.
eof = (no more data && !rev_list_is_running);
With this we could enter the loop one more time then necessary, but
nothing else bad should happen.
^ permalink raw reply
* Re: [ANNOUNCE] CGit v0.1-pre
From: Lars Hjemli @ 2006-12-11 17:40 UTC (permalink / raw)
To: Linus Torvalds; +Cc: git
In-Reply-To: <Pine.LNX.4.64.0612110859200.12500@woody.osdl.org>
On 12/11/06, Linus Torvalds <torvalds@osdl.org> wrote:
>
> On Mon, 11 Dec 2006, Lars Hjemli wrote:
> > Anyway, I must say I find it rather unlikely for these cases to occur
> > (frequently) in real life. That would seem to imply that the caching
> > isn't really needed at all.
>
> The point is, if you have races, you will hit them _occasionally_. It may
> not be a performance problem in real life, BUT:
>
> - quite often, you can take advantage of the serialization guarantees
> that a front-side cache offers you, and do the uncached accesses (or
> writing the final cache-file) knowing that there's only one thing that
> does that.
>
> - If so, then a race condition in the cache goes from a "unlikely
> performance problem" to a BUG.
Yes, I finally understood (see my other reply)
> > > As a side note: how do you release your caches?
> >
> > Simple timeouts (time()-stat.st_mtime), depending on what kind of page
> > was requested. If anyone cares about invalid cache content (branch
> > head moving), relevant cachefiles can be deleted with an update-hook.
>
> I was more thinking about the locking part. Again, to be safe, you should
> probably take the lock before releasing any cache.
Ok. Code speeks louder than words, so I'll blatantly paste the key functions:
--->8----
const int NOLOCK = -1;
int cache_lock(struct cacheitem *item)
{
int i = 0;
char *lockfile = fmt("%s.lock", item->name);
top:
if (++i > cgit_max_lock_attempts)
die("cache_lock: unable to lock %s: %s",
item->name, strerror(errno));
item->fd = open(lockfile, O_WRONLY|O_CREAT|O_EXCL, S_IRUSR|S_IWUSR);
if (item->fd == NOLOCK && errno == ENOENT && cache_create_dirs())
goto top;
if (item->fd == NOLOCK && errno == EEXIST &&
cache_refill_overdue(lockfile) && !unlink(lockfile))
goto top;
return (item->fd > 0);
}
int cache_unlock(struct cacheitem *item)
{
close(item->fd);
return (rename(fmt("%s.lock", item->name), item->name) == 0);
}
static void cgit_check_cache(struct cacheitem *item)
{
int i = 0;
cache_prepare(item);
top:
if (++i > cgit_max_lock_attempts) {
die("cgit_refresh_cache: unable to lock %s: %s",
item->name, strerror(errno));
}
if (!cache_exist(item)) {
if (!cache_lock(item)) {
sleep(1);
goto top;
}
if (!cache_exist(item))
cgit_fill_cache(item);
cache_unlock(item);
} else if (cache_expired(item) && cache_lock(item)) {
if (cache_expired(item))
cgit_fill_cache(item);
cache_unlock(item);
}
}
--->8----
I am a bit conserned about the effect of cache_unlink() if another
concurrent process gets "lucky" with the test
"cache_refill_overdue(lockfile) && !unlink(lockfile)".
This is supposed to be a safety valve against a stale lock file (lock
file not modified in n secs), but it doesn't feel quite right.
Probably, if cache_unlink() fails in cgit_check_cache(), I should "goto top".
Opinions?
--
^ permalink raw reply
* [PATCH] Don't use memcpy when source and dest. buffers may overlap
From: Jim Meyering @ 2006-12-11 18:06 UTC (permalink / raw)
To: git
git-index-pack can call memcpy with overlapping source and destination
buffers. The patch below makes it use memmove instead.
If you want to demonstrate a failure, add the following two lines
+ if (input_offset < input_len)
+ abort ();
before the existing memcpy call (shown in the patch below),
and then run this:
(cd t; sh ./t5500-fetch-pack.sh)
Signed-off-by: Jim Meyering <jim@meyering.net>
---
index-pack.c | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/index-pack.c b/index-pack.c
index 8331d99..6d6c92b 100644
--- a/index-pack.c
+++ b/index-pack.c
@@ -96,7 +96,7 @@ static void flush(void)
if (output_fd >= 0)
write_or_die(output_fd, input_buffer, input_offset);
SHA1_Update(&input_ctx, input_buffer, input_offset);
- memcpy(input_buffer, input_buffer + input_offset, input_len);
+ memmove(input_buffer, input_buffer + input_offset, input_len);
input_offset = 0;
}
}
--
^ permalink raw reply related
* Re: [RFC \ WISH] Add -o option to git-rev-list
From: Linus Torvalds @ 2006-12-11 18:15 UTC (permalink / raw)
To: Marco Costalba
Cc: Git Mailing List, Junio C Hamano, Alex Riesen, Shawn Pearce
In-Reply-To: <e5bfff550612110939r58df1a73ud28983633c12e487@mail.gmail.com>
On Mon, 11 Dec 2006, Marco Costalba wrote:
>
> Regarding the _normal_ solution we have one more hipotesys to take
> advantage of: git-rev-list when has nothing more to read..exits.
Yes. You can just wait for the child exit signal.
However, you seem to continually ignore the thing I've asked you to do
several times: try with a cold-cache situation.
The thing is, using pipes and poll()/select()/epoll()/whatever will not
only be efficient, but it will also WORK CORRECTLY in the presense of a
writer that is _slower_ than the reader.
Right now, you're testing the exact opposite. You're testing the case
where the reader is slower than the writer, which is actually not that
interesting - because if "git-rev-list" is instantaneous, then the fastest
thing to do is to simply _always_ just read the result buffer directly
into memory without any select() loop etc what-so-ever.
So just by testing that case (and you've slowed down the reader
artificially even _apart_ from the fact that qgit will probably always be
slower than git-rev-list off a packed and hot-cache environment), you're
always going to skew your results into the "do everything in one go"
direction.
But the point of the pipe and the poll() is that it works nicely even when
the data trickles in slowly.
[ Of course, as long as you ask for "--topo-order", you'll never see a lot
of trickling, and you'll never be able to do really well for the
cold-cache case. To see the _real_ advantage of pipes, you should void
"--topo-order" entirely, and do it dynamically within qgit, repainting
the graph as needed.
At that point, you'd actually do something that gitk can't do at all,
namely work well for the cold-cache not-very-packed large-repository
case. ]
To see this in practice (even with hot-caches), do something like the
following on the full historic Linux archive:
time sh -c "git rev-list HEAD | head"
time sh -c "git rev-list --topo-order HEAD | head"
where for me, the firstone takes 0.002s, and the second one takes 0.878s.
Now THAT is an optimization. Not just "10%" or even "ten times", but "four
HUNDRED times" faster.
And why is that? Simply because it only needed to look at _part_ of the
data. For the exact same reason, if you were to do the topological sort
only on the part that you had _looked_ at first, you'd be able to do these
kinds of several-orders-of-magnitude improvements.
^ permalink raw reply
* Re: [ANNOUNCE] CGit v0.1-pre
From: Linus Torvalds @ 2006-12-11 18:18 UTC (permalink / raw)
To: Lars Hjemli; +Cc: git
In-Reply-To: <8c5c35580612110940p767fd360p52762a8818fbc8c1@mail.gmail.com>
On Mon, 11 Dec 2006, Lars Hjemli wrote:
>
> Ok. Code speeks louder than words, so I'll blatantly paste the key functions:
Yeah, they're still buggy:
> int cache_unlock(struct cacheitem *item)
> {
> close(item->fd);
> return (rename(fmt("%s.lock", item->name), item->name) == 0);
> }
...
> if (!cache_exist(item)) {
> if (!cache_lock(item)) {
> sleep(1);
> goto top;
> }
> if (!cache_exist(item))
> cgit_fill_cache(item);
> cache_unlock(item);
What do you think happens if that last "cache_exist()" returned true?
That's right: the "cache_unlock()" will now OVERWRITE the valid cache with
the (empty) lock-file that you didn't fill in.
Oops.
So you really have two different cases:
- the "I created the file" case: rename lockfile to final name
- the "somebody else created the file": remove the lockfile
and you can't use the same "cache_unlock()" for both of them.
^ permalink raw reply
* Re: [ANNOUNCE] CGit v0.1-pre
From: Lars Hjemli @ 2006-12-11 18:38 UTC (permalink / raw)
To: Linus Torvalds; +Cc: git
In-Reply-To: <Pine.LNX.4.64.0612111016220.12500@woody.osdl.org>
On 12/11/06, Linus Torvalds <torvalds@osdl.org> wrote:
>
>
> On Mon, 11 Dec 2006, Lars Hjemli wrote:
> >
> > Ok. Code speeks louder than words, so I'll blatantly paste the key functions:
>
> Yeah, they're still buggy:
>
> > int cache_unlock(struct cacheitem *item)
> > {
> > close(item->fd);
> > return (rename(fmt("%s.lock", item->name), item->name) == 0);
> > }
> ...
> > if (!cache_exist(item)) {
> > if (!cache_lock(item)) {
> > sleep(1);
> > goto top;
> > }
> > if (!cache_exist(item))
> > cgit_fill_cache(item);
> > cache_unlock(item);
>
> What do you think happens if that last "cache_exist()" returned true?
>
> That's right: the "cache_unlock()" will now OVERWRITE the valid cache with
> the (empty) lock-file that you didn't fill in.
>
> Oops.
>
Wow, that's embarrassing.
Thanks for the debugging, I'll try to actually fix it this time :-)
--
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox