[Buildroot] autotools, autorefconf, libtoolize and non-existing m4 directory

[Buildroot] autotools, autorefconf, libtoolize and non-existing m4 directory

[Michael Walle]

Hi Thomas, Hi Peter,

we have a package [1] where:
  - we have AC_CONFIG_MACRO_DIR([m4]) in configure.ac
  - we have ACLOCAL_AMFLAGS = -I m4 in Makefile.am
  - we do *not* have an m4/ subdirectory (there is actually a commit
    that introduces an empty m4/ directory, but that should not be
    necessary)

We now do trigger a bug where autoreconf from the host works perfectly
and the autoreconf of busybox do not. Here is our first analysis:
  - originating from [2], there was an upstream patch which degrades the
    absence of a local m4 directory from fatal to just a warning in the
    aclocal step.
  - the current aclocal script treats just the _first_ include directory
    that way. Ie. it assumes, that the local m4/ is the first one in
    @user_includes, see [3]
  - buildroot sets the first one by its own in the automake package,
    see [4], because it supplies the first include already in the
    "$(ACLOCAL)"

Is this behaviour intended? There is also an install option to aclocal
which uses first directory in the list.

Unfortunately, I cannot come up with a solution other than:
   ACLOCAL_AMFLAGS = -I m4 $(SOME_OTHER_FLAGS)
and buildroot would set its own include path in SOME_OTHER_FLAGS. But
I'd guess that will break many packages. So maybe the current behaviour
is the lesser evil. But it does not really work like on the host system.

-michael


[1] https://github.com/kontron/keapi
[2] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=565663#50
[3] 
http://git.savannah.gnu.org/cgit/automake.git/tree/bin/aclocal.in?id=5ae02cc829c8efe8afcf72467d0147444feab651#n439
[4] 
https://git.buildroot.net/buildroot/tree/package/automake/automake.mk?id=c699f5ab5a9d86d4416647a3ba9405476bf6bb9b#n35

[Buildroot] autotools, autorefconf, libtoolize and non-existing m4 directory

[Thomas Petazzoni]

Hello,

On Tue, 10 Dec 2019 11:34:51 +0100
Michael Walle <michael@walle.cc> wrote:

> Hi Thomas, Hi Peter,
> 
> we have a package [1] where:
>   - we have AC_CONFIG_MACRO_DIR([m4]) in configure.ac
>   - we have ACLOCAL_AMFLAGS = -I m4 in Makefile.am
>   - we do *not* have an m4/ subdirectory (there is actually a commit
>     that introduces an empty m4/ directory, but that should not be
>     necessary)
> 
> We now do trigger a bug where autoreconf from the host works perfectly
> and the autoreconf of busybox do not. Here is our first analysis:
>   - originating from [2], there was an upstream patch which degrades the
>     absence of a local m4 directory from fatal to just a warning in the
>     aclocal step.
>   - the current aclocal script treats just the _first_ include directory
>     that way. Ie. it assumes, that the local m4/ is the first one in
>     @user_includes, see [3]
>   - buildroot sets the first one by its own in the automake package,
>     see [4], because it supplies the first include already in the
>     "$(ACLOCAL)"
> 
> Is this behaviour intended? There is also an install option to aclocal
> which uses first directory in the list.
> 
> Unfortunately, I cannot come up with a solution other than:
>    ACLOCAL_AMFLAGS = -I m4 $(SOME_OTHER_FLAGS)
> and buildroot would set its own include path in SOME_OTHER_FLAGS. But
> I'd guess that will break many packages. So maybe the current behaviour
> is the lesser evil. But it does not really work like on the host system.

Maybe I misunderstood, but why don't you solve this like we're doing in
many other Buildroot packages already:

define LIBISCSI_CREATE_M4_DIR
        mkdir -p $(@D)/m4
endef
LIBISCSI_POST_PATCH_HOOKS += LIBISCSI_CREATE_M4_DIR

This is simple, and works just fine.

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

[Buildroot] autotools, autorefconf, libtoolize and non-existing m4 directory

[Michael Walle]

Am 2019-12-10 14:04, schrieb Thomas Petazzoni:
> Hello,
> 
> On Tue, 10 Dec 2019 11:34:51 +0100
> Michael Walle <michael@walle.cc> wrote:
> 
>> Hi Thomas, Hi Peter,
>> 
>> we have a package [1] where:
>>   - we have AC_CONFIG_MACRO_DIR([m4]) in configure.ac
>>   - we have ACLOCAL_AMFLAGS = -I m4 in Makefile.am
>>   - we do *not* have an m4/ subdirectory (there is actually a commit
>>     that introduces an empty m4/ directory, but that should not be
>>     necessary)
>> 
>> We now do trigger a bug where autoreconf from the host works perfectly
>> and the autoreconf of busybox do not. Here is our first analysis:
>>   - originating from [2], there was an upstream patch which degrades 
>> the
>>     absence of a local m4 directory from fatal to just a warning in 
>> the
>>     aclocal step.
>>   - the current aclocal script treats just the _first_ include 
>> directory
>>     that way. Ie. it assumes, that the local m4/ is the first one in
>>     @user_includes, see [3]
>>   - buildroot sets the first one by its own in the automake package,
>>     see [4], because it supplies the first include already in the
>>     "$(ACLOCAL)"
>> 
>> Is this behaviour intended? There is also an install option to aclocal
>> which uses first directory in the list.
>> 
>> Unfortunately, I cannot come up with a solution other than:
>>    ACLOCAL_AMFLAGS = -I m4 $(SOME_OTHER_FLAGS)
>> and buildroot would set its own include path in SOME_OTHER_FLAGS. But
>> I'd guess that will break many packages. So maybe the current 
>> behaviour
>> is the lesser evil. But it does not really work like on the host 
>> system.
> 
> Maybe I misunderstood, but why don't you solve this like we're doing in
> many other Buildroot packages already:
> 
> define LIBISCSI_CREATE_M4_DIR
>         mkdir -p $(@D)/m4
> endef
> LIBISCSI_POST_PATCH_HOOKS += LIBISCSI_CREATE_M4_DIR
> 
> This is simple, and works just fine.

Correct, that would work, actually, we already had that construct in the
recipe. But as I've already said, this behaviour is very different than
on the host. Esp. if you'd use "aclocal -i"; which I don't know if it is
used at all. Currently there are many workarounds, all are fine. My
intention was to make you aware of this bug; or if you already knew it,
your opinion on it. Eg. aclocal is handling the first include path in a
special way, but buildroot already occupies that slot.

-michael

[Buildroot] autotools, autorefconf, libtoolize and non-existing m4 directory

[Thomas Petazzoni]

On Tue, 10 Dec 2019 14:21:06 +0100
Michael Walle <michael@walle.cc> wrote:

> > This is simple, and works just fine.  
> 
> Correct, that would work, actually, we already had that construct in the
> recipe. But as I've already said, this behaviour is very different than
> on the host. Esp. if you'd use "aclocal -i"; which I don't know if it is
> used at all. Currently there are many workarounds, all are fine. My
> intention was to make you aware of this bug; or if you already knew it,
> your opinion on it. Eg. aclocal is handling the first include path in a
> special way, but buildroot already occupies that slot.

No, I was not aware that the first include path is handled in a special
way, and that because Buildroot uses that slot, the m4/ directory no
longer gets created automatically.

In fact, aclocal's man page says:

       --install
              copy third-party files to the first -I directory

So indeed that's I guess how the m4/ directory gets created.

Perhaps we should use:

       --system-acdir=DIR
              directory holding third-party system-wide files

instead? Could you try that in the definition of $(AUTORECONF), instead
of the -I options?

In fact, it is quite worrying that aclocal -i -I/foo/bar copies stuff
to /foo/bar. Indeed, we clearly don't want aclocal to clutter
$(ACLOCAL_DIR) or $(ACLOCAL_HOST_DIR) with additional files by effect
of running autoreconf.

Best regards,

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com