Removing .tmp_versions considered harmful

Removing .tmp_versions considered harmful

[Pavel Roskin]

Hello!

A patch applied shortly after Linux 2.6.16
(fb3cbd2e575f9ac0700bfa1e7cb9f4119fbd0abd in git) causes
the .tmp_versions directory to be removed every time make is run to
build external modules.

This is bad for two reasons.

1) When "make install" is run as root, .tmp_versions is re-created and
becomes owned by root.  Subsequent "make" by user fails
because .tmp_versions cannot be removed.

2) The projects where modules are build in more than one directory (such
as MadWifi) are now compiled with spurious warnings about unresolved
symbols.  This happens because every module is compiled individually,
and the *.mod files for one module are removed before the other is
compiled.

In both cases, I cannot think of any sane workarounds.  For the first
problem, MODVERDIR could be set to different values for different
usernames.  For the second problem, I considered clever ways to sabotage
some targets based on the target name, something like:

divert_crmodverdir = some_other_dir
MODVERDIR=$(SYMBOLSDIR)'$$(divert_$$@)'

When I looked on this the next day I realized that it's ugly as hell.
Why should I invent workarounds for what should be a standard situation?

Since I don't see any better approach, I'm asking the commit
fb3cbd2e575f9ac0700bfa1e7cb9f4119fbd0abd to be reverted for Linux
2.6.17.  The problem it solves (stale *.mod files) is not nearly as big
as the problems it creates.

Note that the removal on *.mod files only would not address the MadWifi
problem.

-- 
Regards,
Pavel Roskin

Re: Removing .tmp_versions considered harmful

[Sam Ravnborg]

On Fri, Apr 21, 2006 at 12:22:22AM -0400, Pavel Roskin wrote:
> Hello!
> 
> A patch applied shortly after Linux 2.6.16
> (fb3cbd2e575f9ac0700bfa1e7cb9f4119fbd0abd in git) causes
> the .tmp_versions directory to be removed every time make is run to
> build external modules.
> 
> 
> 2) The projects where modules are build in more than one directory (such
> as MadWifi) are now compiled with spurious warnings about unresolved
> symbols.  This happens because every module is compiled individually,
> and the *.mod files for one module are removed before the other is
> compiled.

Then fix madwifi so it builds modules as documented in
Documentation/kbuild/modules.txt
See: --- 5.3 External modules using several directories

As with many other external modules madwifi contains a lot of ugly
makefile hackery - and if done as documentated it gets so much simpler.
I'm aware that 2-4 support complicates things a little but if people
made it be slimm and nice for 2.6 and _then_ added 2.4 supporrrrrrrrrrit
woulllllld be much simpler.
It seems that people keep all the hackery for 2.4 and does a bad job
adapting to 2.6.

All the bad FAQ's out there does a good job confusing people.
Almost no-one mention SUBDIRS= as is preferred with 2.4 but seldom used
:-(

	Sam

Re: Removing .tmp_versions considered harmful

[Pavel Roskin]

Hello, Sam!

On Fri, 2006-04-21 at 09:32 +0200, Sam Ravnborg wrote:
> On Fri, Apr 21, 2006 at 12:22:22AM -0400, Pavel Roskin wrote:
> > Hello!
> > 
> > A patch applied shortly after Linux 2.6.16
> > (fb3cbd2e575f9ac0700bfa1e7cb9f4119fbd0abd in git) causes
> > the .tmp_versions directory to be removed every time make is run to
> > build external modules.
> > 
> > 
> > 2) The projects where modules are build in more than one directory (such
> > as MadWifi) are now compiled with spurious warnings about unresolved
> > symbols.  This happens because every module is compiled individually,
> > and the *.mod files for one module are removed before the other is
> > compiled.
> 
> Then fix madwifi so it builds modules as documented in
> Documentation/kbuild/modules.txt
> See: --- 5.3 External modules using several directories

Thanks for showing me this section!  I wish I saw it before I submitted
patches to MadWifi, HostAP and linux-wlan-ng makefiles.  Now I know how
to fix all that mess.

> As with many other external modules madwifi contains a lot of ugly
> makefile hackery - and if done as documentated it gets so much simpler.
> I'm aware that 2-4 support complicates things a little but if people
> made it be slimm and nice for 2.6 and _then_ added 2.4 supporrrrrrrrrrit
> woulllllld be much simpler.
> It seems that people keep all the hackery for 2.4 and does a bad job
> adapting to 2.6.

I agree with your sentiment, and I'm trying to fix it.

However, the problem with "make install" creating .tmp_versions owned by
root still exists.  Can we settle on removing only *.mod files
under .tmp_versions without removing the directory?
Removing .tmp_versions does nothing if the goal is to remove stale *.mod
files.

-- 
Regards,
Pavel Roskin

Re: Removing .tmp_versions considered harmful

[Pavel Roskin]

Hello, Sam!

How about following patch?  Something needs to be done before 2.6.17.
Complaints about .tmp_versions are almost in every list about wireless
drivers I'm subscribed to.

I'm not asking to keep *.mod files, just please keep the .tmp_versions
directory.

-------------------------------
Remove *.mod files but not .tmp_versions for external builds

From: Pavel Roskin <proski@gnu.org>

When "make install" is run as root, .tmp_versions is re-created and
becomes owned by root.  Subsequent "make" run by user fails because
.tmp_versions cannot be removed.

Signed-off-by: Pavel Roskin <proski@gnu.org>
---

 Makefile |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/Makefile b/Makefile
index a940eae..6d7db09 100644
--- a/Makefile
+++ b/Makefile
@@ -1086,7 +1086,7 @@ # We are always building modules
 KBUILD_MODULES := 1
 PHONY += crmodverdir
 crmodverdir:
-	$(Q)rm -rf $(MODVERDIR)
+	$(Q)rm -f $(MODVERDIR)/*.mod
 	$(Q)mkdir -p $(MODVERDIR)
 
 PHONY += $(objtree)/Module.symvers



-- 
Regards,
Pavel Roskin

Re: Removing .tmp_versions considered harmful

[Arjan van de Ven]

On Mon, 2006-04-24 at 15:55 -0400, Pavel Roskin wrote:
> Hello, Sam!
> 
> How about following patch?  Something needs to be done before 2.6.17.
> Complaints about .tmp_versions are almost in every list about wireless
> drivers I'm subscribed to.

seems all wireless drivers stole eachothers broken makefiles then ;)
Makes it also easy to fix I suppose

Re: Removing .tmp_versions considered harmful

[Pavel Roskin]

Hello!

On Mon, 2006-04-24 at 21:58 +0200, Arjan van de Ven wrote:
> On Mon, 2006-04-24 at 15:55 -0400, Pavel Roskin wrote:
> > Hello, Sam!
> > 
> > How about following patch?  Something needs to be done before 2.6.17.
> > Complaints about .tmp_versions are almost in every list about wireless
> > drivers I'm subscribed to.
> 
> seems all wireless drivers stole eachothers broken makefiles then ;)

There is is some similarity, and it's a good thing.

> Makes it also easy to fix I suppose

Could you please elaborate?  How can it be fixed?  Do you know any
external module that doesn't have this problem?

If you have a could of minutes, please look at Orinoco, which probably
has the simplest build system, yet it suffers from the .tmp_versions
problem.  The Subversion repository is here
http://sourceforge.net/svn/?group_id=44338

To check the main development branch, please use
svn co https://svn.sourceforge.net/svnroot/orinoco/trunk orinoco

This is the Makefile:
http://svn.sourceforge.net/viewcvs.cgi/*checkout*/orinoco/trunk/Makefile

What would you fix?

-- 
Regards,
Pavel Roskin

Re: Removing .tmp_versions considered harmful

[Sam Ravnborg]

On Mon, Apr 24, 2006 at 03:55:27PM -0400, Pavel Roskin wrote:
> Hello, Sam!
> 
> How about following patch?  Something needs to be done before 2.6.17.
> Complaints about .tmp_versions are almost in every list about wireless
> drivers I'm subscribed to.
> 
> I'm not asking to keep *.mod files, just please keep the .tmp_versions
> directory.
> 
> -------------------------------
> Remove *.mod files but not .tmp_versions for external builds
> 
> From: Pavel Roskin <proski@gnu.org>
> 
> When "make install" is run as root, .tmp_versions is re-created and
> becomes owned by root.  Subsequent "make" run by user fails because
> .tmp_versions cannot be removed.

What architecture?
For i386 and x86_64 make install no longer try to compile the kernel.

I have anyway added the following patch:



------------------

Remove *.mod files but not .tmp_versions for external builds

When "make install" is run as root, .tmp_versions is re-created and
becomes owned by root.  Subsequent "make" run by user fails because
.tmp_versions cannot be removed.

Signed-off-by: Pavel Roskin <proski@gnu.org>
Signed-off-by: Sam Ravnborg <sam@ravnborg.org>
---

diff --git a/Makefile b/Makefile
index 6bf9962..8517f7b 100644
--- a/Makefile
+++ b/Makefile
@@ -796,8 +796,8 @@ prepare2: prepare3 outputmakefile
 prepare1: prepare2 include/linux/version.h include/asm \
                    include/config/MARKER
 ifneq ($(KBUILD_MODULES),)
-	$(Q)rm -rf $(MODVERDIR)
 	$(Q)mkdir -p $(MODVERDIR)
+	$(Q)rm -f $(MODVERDIR)/*
 endif
 
 archprepare: prepare1 scripts_basic
@@ -1086,8 +1086,8 @@ # We are always building modules
 KBUILD_MODULES := 1
 PHONY += crmodverdir
 crmodverdir:
-	$(Q)rm -rf $(MODVERDIR)
 	$(Q)mkdir -p $(MODVERDIR)
+	$(Q)rm -f $(MODVERDIR)/*
 
 PHONY += $(objtree)/Module.symvers
 $(objtree)/Module.symvers:

Re: Removing .tmp_versions considered harmful

[Pavel Roskin]

Hello, Sam!

On Sun, 2006-04-30 at 23:47 +0200, Sam Ravnborg wrote:
> On Mon, Apr 24, 2006 at 03:55:27PM -0400, Pavel Roskin wrote:
> > Remove *.mod files but not .tmp_versions for external builds
> > 
> > From: Pavel Roskin <proski@gnu.org>
> > 
> > When "make install" is run as root, .tmp_versions is re-created and
> > becomes owned by root.  Subsequent "make" run by user fails because
> > .tmp_versions cannot be removed.
> 
> What architecture?
> For i386 and x86_64 make install no longer try to compile the kernel.

That's x86_64.  It turns out the dependency of "install" on "all" in the
project Makefile was causing .tmp_versions to be re-created.  Removing
the dependency on "all" fixes the problem.  This is the original rule:

install: all
        $(MAKE) $(KBUILD_FLAGS) modules_install \
                INSTALL_MOD_DIR=kernel/drivers/net/wireless
        $(DEPMOD) -ae

However, I'd rather keep the dependency of "install" on "all".  It's
better to compile a few files as root than to install out-of-date
modules.  I'm glad that your patch gives me a choice how to write the
project's Makefile.

-- 
Regards,
Pavel Roskin