[PATCH] auto-detect changed $prefix in Makefile and properly rebuild to avoid broken install

[PATCH] auto-detect changed $prefix in Makefile and properly rebuild to avoid broken install

[Yakov Lerner]

Many times, I mistakenly used 'make prefix=... install' where prefix value
was different from prefix value during build. This resulted in broken
install. This patch adds auto-detection of $prefix change to the Makefile.
This results in correct install whenever prefix is changed.

Signed-off-by: Yakov Lerner <iler.ml@gmail.com>
---
 Makefile |   29 ++++++++++++++++++++++-------
 1 files changed, 22 insertions(+), 7 deletions(-)

diff --git a/Makefile b/Makefile
index 2a1e639..015c9b2 100644
--- a/Makefile
+++ b/Makefile
@@ -464,6 +464,7 @@ DESTDIR_SQ = $(subst ','\'',$(DESTDIR))
 bindir_SQ = $(subst ','\'',$(bindir))
 gitexecdir_SQ = $(subst ','\'',$(gitexecdir))
 template_dir_SQ = $(subst ','\'',$(template_dir))
+prefix_SQ = $(subst ','\'',$(prefix))
 
 SHELL_PATH_SQ = $(subst ','\'',$(SHELL_PATH))
 PERL_PATH_SQ = $(subst ','\'',$(PERL_PATH))
@@ -484,7 +485,7 @@ all:
 strip: $(PROGRAMS) git$X
 	$(STRIP) $(STRIP_OPTS) $(PROGRAMS) git$X
 
-git$X: git.c common-cmds.h $(BUILTIN_OBJS) $(GITLIBS)
+git$X: git.c common-cmds.h $(BUILTIN_OBJS) $(GITLIBS) .git.prefix
 	$(CC) -DGIT_VERSION='"$(GIT_VERSION)"' \
 		$(ALL_CFLAGS) -o $@ $(filter %.c,$^) \
 		$(BUILTIN_OBJS) $(ALL_LDFLAGS) $(LIBS)
@@ -516,7 +517,7 @@ common-cmds.h: Documentation/git-*.txt
 	chmod +x $@+
 	mv $@+ $@
 
-$(patsubst %.py,%,$(SCRIPT_PYTHON)) : % : %.py
+$(patsubst %.py,%,$(SCRIPT_PYTHON)) : % : %.py .git.prefix
 	rm -f $@ $@+
 	sed -e '1s|#!.*python|#!$(PYTHON_PATH_SQ)|' \
 	    -e 's|@@GIT_PYTHON_PATH@@|$(GIT_PYTHON_DIR_SQ)|g' \
@@ -540,19 +541,19 @@ git$X git.spec \
 	$(patsubst %.py,%,$(SCRIPT_PYTHON)) \
 	: GIT-VERSION-FILE
 
-%.o: %.c
+%.o: %.c .git.prefix
 	$(CC) -o $*.o -c $(ALL_CFLAGS) $<
 %.o: %.S
 	$(CC) -o $*.o -c $(ALL_CFLAGS) $<
 
-exec_cmd.o: exec_cmd.c
+exec_cmd.o: exec_cmd.c .git.prefix
 	$(CC) -o $*.o -c $(ALL_CFLAGS) '-DGIT_EXEC_PATH="$(gitexecdir_SQ)"' $<
 
-http.o: http.c
+http.o: http.c .git.prefix
 	$(CC) -o $*.o -c $(ALL_CFLAGS) -DGIT_USER_AGENT='"git/$(GIT_VERSION)"' $<
 
 ifdef NO_EXPAT
-http-fetch.o: http-fetch.c http.h
+http-fetch.o: http-fetch.c http.h .git.prefix
 	$(CC) -o $*.o -c $(ALL_CFLAGS) -DNO_EXPAT $<
 endif
 
@@ -609,6 +610,14 @@ tags:
 	rm -f tags
 	find . -name '*.[hcS]' -print | xargs ctags -a
 
+### Detect prefix changes
+.git.prefix: .FORCE-git.prefix
+	@PREFIXES='$(bindir_SQ):$(gitexecdir_SQ):$(template_dir_SQ):$(prefix_SQ)';\
+	    if test x"$$PREFIXES" != x"`cat .git.prefix 2>/dev/null`" ; then \
+		echo 1>&2 "    * prefix changed"; \
+		echo "$$PREFIXES" >.git.prefix; \
+            fi
+
 ### Testing rules
 
 # GNU make supports exporting all variables by "export" without parameters.
@@ -632,6 +641,12 @@ test-dump-cache-tree$X: dump-cache-tree.
 check:
 	for i in *.c; do sparse $(ALL_CFLAGS) $(SPARSE_FLAGS) $$i || exit; done
 
+test-prefix-change:
+	mkdir -p "`pwd`/tmp1" "`pwd`/tmp2"
+	$(MAKE) clean install prefix="`pwd`/tmp1"
+	$(MAKE) install prefix="`pwd`/tmp2"
+	@grep -r "`pwd`/tmp1" "`pwd`/tmp2" >/dev/null; if test $$? = 0 ; then\
+	    echo Error, test failed; exit 1; else echo Ok, test passed; fi
 
 
 ### Installation rules
@@ -714,7 +729,7 @@ clean:
 	rm -f GIT-VERSION-FILE
 
 .PHONY: all install clean strip
-.PHONY: .FORCE-GIT-VERSION-FILE TAGS tags
+.PHONY: .FORCE-GIT-VERSION-FILE TAGS tags .FORCE-git.prefix
 
 ### Check documentation
 #
-- 
1.4.0

Re: [PATCH] auto-detect changed $prefix in Makefile and properly rebuild to avoid broken install

[Junio C Hamano]

Yakov Lerner <iler.ml@gmail.com> writes:

> Many times, I mistakenly used 'make prefix=... install' where prefix value
> was different from prefix value during build. This resulted in broken
> install. This patch adds auto-detection of $prefix change to the Makefile.
> This results in correct install whenever prefix is changed.
>
> Signed-off-by: Yakov Lerner <iler.ml@gmail.com>

I do not mind this per se, and probably even agree that this is
an improvement compared to the current state of affairs, but a few
points:

 - please make sure you clean that state file in "make clean";

 - we may want to make the state file a bit more visible (IOW, I
   somewhat do mind the name being dot-git-dot-prefix).

 - we might want to later (or at the same time as this patch)
   do "consistent set of compilation flags" (e.g. run early
   part of compilation with openssl SHA-1 implementation,
   interrupt it and build and link the rest with mozilla SHA-1
   implementation -- then you will get a nonsense binary without
   linker errors).  It might make sense to prepare this
   mechanism so we could reuse it for that purpose.

Re: [PATCH] auto-detect changed $prefix in Makefile and properly rebuild to avoid broken install

[Yakov Lerner]

On 6/14/06, Junio C Hamano <junkio@cox.net> wrote:
> Yakov Lerner <iler.ml@gmail.com> writes:
>
> > Many times, I mistakenly used 'make prefix=... install' where prefix value
> > was different from prefix value during build. This resulted in broken
> > install. This patch adds auto-detection of $prefix change to the Makefile.
> > This results in correct install whenever prefix is changed.
> >
> > Signed-off-by: Yakov Lerner <iler.ml@gmail.com>
>
> I do not mind this per se, and probably even agree that this is
> an improvement compared to the current state of affairs, but a few
> points:
>
>  - please make sure you clean that state file in "make clean";
done

>  - we may want to make the state file a bit more visible (IOW, I
>    somewhat do mind the name being dot-git-dot-prefix).
I renamed .git.prefix to GIT-PREFIX. Is this ok.

>  - we might want to later (or at the same time as this patch)
>    do "consistent set of compilation flags" (e.g. run early
>    part of compilation with openssl SHA-1 implementation,
>    interrupt it and build and link the rest with mozilla SHA-1
>    implementation -- then you will get a nonsense binary without
>    linker errors).  It might make sense to prepare this
>    mechanism so we could reuse it for that purpose.

Do you think two separate GIT-PREFIX and GIT-BUILD-FLAGS are needed,
or just once GIT-BUILD-FLAGS will do, which will include
prefixes (as passed with -D... to cc) ?

I think single GIT-BUILD-FLAGS
is enough, which will cover prefixes, too. Is this OK ?

BTW, I think it's useful to add Makefile itself as prerequisite for all *.o,
so change in Makefile will cause recompilations. Shall I include this
into this patch, too ?

Yakov

Re: [PATCH] auto-detect changed $prefix in Makefile and properly rebuild to avoid broken install

[Junio C Hamano]

"Yakov Lerner" <iler.ml@gmail.com> writes:

> I think single GIT-BUILD-FLAGS
> is enough, which will cover prefixes, too. Is this OK ?

Yes, it was what I was getting at.  I think a single
GIT-BUILD-FLAGS (or whatever name the list can fight over while
I am away) is preferred.

> BTW, I think it's useful to add Makefile itself as prerequisite for all *.o,
> so change in Makefile will cause recompilations. Shall I include this
> into this patch, too ?

I've thought about it but in practice this would make things
more inconvenient for developers without much gain, so I'd leave
it out.

Re: [PATCH] auto-detect changed $prefix in Makefile and properly rebuild to avoid broken install

[Yakov Lerner]

On 6/15/06, Junio C Hamano <junkio@cox.net> wrote:
> "Yakov Lerner" <iler.ml@gmail.com> writes:
>
> > I think single GIT-BUILD-FLAGS
> > is enough, which will cover prefixes, too. Is this OK ?
>
> Yes, it was what I was getting at.  I think a single
> GIT-BUILD-FLAGS (or whatever name the list can fight over while
> I am away) is preferred.

Either GIT-CFLAGS or GIT-BUILD-FLAGS,
whichever is shorter :-)

Yakov

Re: [PATCH] auto-detect changed $prefix in Makefile and properly rebuild to avoid broken install

[Matthias Lederhofer]

> Either GIT-CFLAGS or GIT-BUILD-FLAGS,
> whichever is shorter :-)

I would not take GIT-CFLAGS because there is an environment variable
used by make named CFLAGS and means something else.

What is the target test-prefix-change good for? Should it really be
included?

Re: [PATCH] auto-detect changed $prefix in Makefile and properly rebuild to avoid broken install

[Yakov Lerner]

On 6/15/06, Matthias Lederhofer <matled@gmx.net> wrote:
> > Either GIT-CFLAGS or GIT-BUILD-FLAGS,
> > whichever is shorter :-)
>
> I would not take GIT-CFLAGS because there is an environment variable
> used by make named CFLAGS and means something else.

What is your preference for this filename ? Is GIT-BUILD-FLAGS
better ? GIT-FLAGS ? Other ?

GIT-CFLAGS is filename here, whereas  $(CFLAGS) is variable.
Maybe writing it the filename ./GIT-CFLAGS would
emphasize betterthat it's not make variable, but a filename ?

> What is the target test-prefix-change good for? Should it really be
> included?

The purpose of 'test-prefix-change' is to check that there are no
missing GIT-FLAGS as prerequisite which would result in broken install
if prefix changed between 'make' and 'make install'. The
'make test-prefix-change' would catch it if GIT-FLAGS happened
to be missing in some places that if substantially dependent of $prefix.

Yakov

Re: [PATCH] auto-detect changed $prefix in Makefile and properly rebuild to avoid broken install

[Santi]

Wed, 14 Jun 2006 22:26 +0300, Yakov Lerner <iler.ml@gmail.com>:
> Many times, I mistakenly used 'make prefix=... install' where prefix value
> was different from prefix value during build. This resulted in broken
> install. This patch adds auto-detection of $prefix change to the Makefile.
> This results in correct install whenever prefix is changed.

I do this each time I install packages from source. I keep them with
"stow" and the usual sequence is:

make prefix=/home/santi/usr
make install prefix=/home/santi/usr/stow/git
cd /home/santi/usr/stow/
stow -v git

so with this auto-detection I'll compile some programs twice.

Santi

Re: [PATCH] auto-detect changed $prefix in Makefile and properly rebuild to avoid broken install

[Yakov Lerner]

On 6/15/06, Santi <sbejar@gmail.com> wrote:
> Wed, 14 Jun 2006 22:26 +0300, Yakov Lerner <iler.ml@gmail.com>:
> > Many times, I mistakenly used 'make prefix=... install' where prefix value
> > was different from prefix value during build. This resulted in broken
> > install. This patch adds auto-detection of $prefix change to the Makefile.
> > This results in correct install whenever prefix is changed.
>
> I do this each time I install packages from source. I keep them with
> "stow" and the usual sequence is:
>
> make prefix=/home/santi/usr
> make install prefix=/home/santi/usr/stow/git
> cd /home/santi/usr/stow/
> stow -v git
>
> so with this auto-detection I'll compile some programs twice.

I'm not familiar with stow. Does stow create some kind of symlinks from
/home/santi/usr/stow/git to home/santi/usr ? If so, why can't you
use prefix=/home/santi/usr/stow/git both in 'make' and in 'make install' ?
Would this work ?

BTW, is it possible to have git use argv[0] to automatically determine
the executable_dir without compiled-in paths ?

Yakov

Re: [PATCH] auto-detect changed $prefix in Makefile and properly rebuild to avoid broken install

[Santi]

2006/6/15, Yakov Lerner <iler.ml@gmail.com>:
> On 6/15/06, Santi <sbejar@gmail.com> wrote:
> > Wed, 14 Jun 2006 22:26 +0300, Yakov Lerner <iler.ml@gmail.com>:
> > > Many times, I mistakenly used 'make prefix=... install' where prefix value
> > > was different from prefix value during build. This resulted in broken
> > > install. This patch adds auto-detection of $prefix change to the Makefile.
> > > This results in correct install whenever prefix is changed.
> >
> > I do this each time I install packages from source. I keep them with
> > "stow" and the usual sequence is:
> >
> > make prefix=/home/santi/usr
> > make install prefix=/home/santi/usr/stow/git
> > cd /home/santi/usr/stow/
> > stow -v git
> >
> > so with this auto-detection I'll compile some programs twice.
>
> I'm not familiar with stow. Does stow create some kind of symlinks from
> /home/santi/usr/stow/git to home/santi/usr ?
It makes links from .../usr/ to .../usr/stow/git/, for expample:

$ls -l ~/usr/bin/git
lrwxrwxrwx 1 santi santi 19 2006-06-01 09:42 /home/santi/usr/bin/git
-> ../stow/git/bin/git*


> If so, why can't you
> use prefix=/home/santi/usr/stow/git both in 'make' and in 'make install' ?
> Would this work ?

For the few tests I've made it does works, but it is not the
recommended method from the stow developers, and I suppose they know
better than me.

>
> BTW, is it possible to have git use argv[0] to automatically determine
> the executable_dir without compiled-in paths ?
>
> Yakov
>

Re: [PATCH] auto-detect changed $prefix in Makefile and properly rebuild to avoid broken install

[Alexander Litvinov]

> > If so, why can't you
> > use prefix=/home/santi/usr/stow/git both in 'make' and in 'make install'
> > ? Would this work ?
>
> For the few tests I've made it does works, but it is not the
> recommended method from the stow developers, and I suppose they know
> better than me.

I always use make prefix=/usr/local/stow/git-bla-bla and it works. The only 
thing that make break is the case when you install library into such prefix 
and it store it in the pkgconfig file. And when you will compile application 
it will be build with library paths set to /usr/local/stow/lib/libxyz and not 
to /usr/local/lib/libxyz.

But I dont make such big dependences from sources so it works.

Re: [PATCH] auto-detect changed $prefix in Makefile and properly rebuild to avoid broken install

[Junio C Hamano]

Santi <sbejar@gmail.com> writes:

> Wed, 14 Jun 2006 22:26 +0300, Yakov Lerner <iler.ml@gmail.com>:
>> Many times, I mistakenly used 'make prefix=... install' where prefix value
>> was different from prefix value during build. This resulted in broken
>> install. This patch adds auto-detection of $prefix change to the Makefile.
>> This results in correct install whenever prefix is changed.
>
> I do this each time I install packages from source. I keep them with
> "stow" and the usual sequence is:
>
> make prefix=/home/santi/usr
> make install prefix=/home/santi/usr/stow/git
> cd /home/santi/usr/stow/
> stow -v git
>
> so with this auto-detection I'll compile some programs twice.

I do not know what "stow" is about, but if it is to allow you to
run make-install to install things in somewhere else, examine
the result, and then move the result to the real location
(implying that you should be able to nuke the "somewhere else"
after you have done so), with the patch, the above sequence
would install the binaries pointing at a wrong directory,
because the second compilation would make them point at the
temporary installation directory ~/usr/stow/git, not the final
location ~/usr/.

Re: [PATCH] auto-detect changed $prefix in Makefile and properly rebuild to avoid broken install

[Junio C Hamano]

"Yakov Lerner" <iler.ml@gmail.com> writes:

> BTW, is it possible to have git use argv[0] to automatically determine
> the executable_dir without compiled-in paths ?

It is possible and I think we did that some time ago.

We fixed it since then, so that you can migrate away from
assuming bindir==gitexecdir.

Re: [PATCH] auto-detect changed $prefix in Makefile and properly rebuild to avoid broken install

[Karl Hasselström]

On 2006-06-16 22:26:38 -0700, Junio C Hamano wrote:

> I do not know what "stow" is about, but if it is to allow you to run
> make-install to install things in somewhere else, examine the
> result, and then move the result to the real location (implying that
> you should be able to nuke the "somewhere else" after you have done
> so), with the patch, the above sequence would install the binaries
> pointing at a wrong directory, because the second compilation would
> make them point at the temporary installation directory
> ~/usr/stow/git, not the final location ~/usr/.

GNU stow doesn't move installed programs, it just maintains symlinks
to them. You install programs under /usr/local/stow/foo-4.7.11, and
stow sets up symlinks to them under /usr/local. (So for example,
/usr/local/bin/foo would be a symlink to
/usr/local/stow/foo-4.7.11/bin/foo.) This gives you the ability to
nuke an installed program cleanly. And it just works, pathwise, since
the program remains in its original location.

-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle

Re: [PATCH] auto-detect changed $prefix in Makefile and properly rebuild to avoid broken install

[Junio C Hamano]

Karl Hasselström <kha@treskal.com> writes:

> GNU stow doesn't move installed programs, it just maintains symlinks
> to them. You install programs under /usr/local/stow/foo-4.7.11, and
> stow sets up symlinks to them under /usr/local. (So for example,
> /usr/local/bin/foo would be a symlink to
> /usr/local/stow/foo-4.7.11/bin/foo.) This gives you the ability to
> nuke an installed program cleanly. And it just works, pathwise, since
> the program remains in its original location.

Thanks for the explanation.

If that's the case, I think it makes the original problem Santi
brought up a non-issue.  In this sequence:

        make prefix=/home/santi/usr
        make install prefix=/home/santi/usr/stow/git
        cd /home/santi/usr/stow/
        stow -v git

the building phase could have used the same prefix as the
install phase uses, and git can find its subprograms in
gitexecdir (= ~/usr/stow/git/bin) just fine.  It probably is
even slightly more efficient since it does not have to go
through the symlink stow installs.

Re: [PATCH] auto-detect changed $prefix in Makefile and properly rebuild to avoid broken install

[Petr Baudis]

Dear diary, on Wed, Jun 14, 2006 at 10:04:43PM CEST, I got a letter
where Junio C Hamano <junkio@cox.net> said that...
>  - we may want to make the state file a bit more visible (IOW, I
>    somewhat do mind the name being dot-git-dot-prefix).

What is the point? It is just a bit of internal build system state made
persistent and shouldn't be interesting for the user, so why give it
extra publicity in the tree?

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
A person is just about as big as the things that make them angry.

Re: [PATCH] auto-detect changed $prefix in Makefile and properly rebuild to avoid broken install

[Karl Hasselström]

On 2006-06-18 04:47:19 -0700, Junio C Hamano wrote:

> Thanks for the explanation.
>
> If that's the case, I think it makes the original problem Santi
> brought up a non-issue. In this sequence:
>
>         make prefix=/home/santi/usr
>         make install prefix=/home/santi/usr/stow/git
>         cd /home/santi/usr/stow/
>         stow -v git
>
> the building phase could have used the same prefix as the install
> phase uses, and git can find its subprograms in gitexecdir (=
> ~/usr/stow/git/bin) just fine. It probably is even slightly more
> efficient since it does not have to go through the symlink stow
> installs.

Yes, exactly. I've always built git like this:

  $ make prefix=/usr/local/stow/git
  $ sudo make prefix=/usr/local/stow/git install
  $ cd /usr/local/stow
  $ sudo stow git

It works for all other programs I've tried too (most of which only
require me to specify the prefix once, with ./configure --prefix=...).
The programs never need to know about the symlinks; they're only there
for when other programs need to access them (via PATH, etc.).

-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle