subdir-objects

subdir-objects

[Roald van Loon]

Hi Ceph devs,

I'm working on doing some structuring/designing plugins for RGW. In a
recent discussion with Yehuda, I'm looking into how we can statically
link 'plugins' into the RGW core (so, not really plugins, but more of
a code separation).

However, the ceph automake structure doesn't use AM's subdir-objects
settings, which is causing me headaches. The reason: two 'plugins'
might share similar filenames in different subdirectories;

./src/rgw/plugins/example/plugin.cc
./src/rgw/plugins/hello_world/plugin.cc

Both of which, of course, gets compiled into radosgw-plugin.o because
of the absence of the subdir-objects setting. I can not circumvent
this by making it a library, because partially linking prevents me
from auto-registering the plugins in the RGW core. The only fix for
this is making the RGW core aware of the subdirs/plugins, but I think
that's nasty design. I'd like to have it in my make conf.

So, the question is; is there a reason why we don't use subdir objects?

Roald

Re: subdir-objects

[Noah Watkins]

On Wed, Aug 21, 2013 at 12:45 PM, Roald van Loon <roaldvanloon@gmail.com> wrote:
>
> from auto-registering the plugins in the RGW core. The only fix for
> this is making the RGW core aware of the subdirs/plugins, but I think
> that's nasty design. I'd like to have it in my make conf.

This patch will turn on the option (which should also fix your problem
if I understand correctly?), and should probably be committed anyway
as newer versions of autotools will complain loudly about our current
Makefile structure.

diff --git a/src/Makefile.am b/src/Makefile.am
index 93f3331..fb7c9dd 100644
--- a/src/Makefile.am
+++ b/src/Makefile.am
@@ -1,4 +1,4 @@
-AUTOMAKE_OPTIONS = gnu
+AUTOMAKE_OPTIONS = gnu subdir-objects
 SUBDIRS = ocf java
 DIST_SUBDIRS = gtest ocf libs3 java

> So, the question is; is there a reason why we don't use subdir objects?

I believe it is just historical, and unfortunately has just been
repeated over and over. Ideally I think that there should be a
restructuring to place a Makefile.am in every subdirectory. This would
address your issue and make it significantly easier to deal with
situations where we want to build a subset of Ceph, such as just FUSE
and librados, for example.

Re: subdir-objects

[Roald van Loon]

Hi,

On Wed, Aug 21, 2013 at 10:01 PM, Noah Watkins <noah.watkins@inktank.com> wrote:
> This patch will turn on the option (which should also fix your problem
> if I understand correctly?), and should probably be committed anyway
> as newer versions of autotools will complain loudly about our current
> Makefile structure.

The patch is indeed quite trivial, and I already enabled it for my own
dev branches, but I like it to be upstream or else my code could never
be pulled :-)

> I believe it is just historical, and unfortunately has just been
> repeated over and over. Ideally I think that there should be a
> restructuring to place a Makefile.am in every subdirectory. This would
> address your issue and make it significantly easier to deal with
> situations where we want to build a subset of Ceph, such as just FUSE
> and librados, for example.

I agree. Is there already a project running for this?

(I feel like I'm hitting depedency after dependency... I started out
trying to implement RGW S3 Keystone support, for that I noticed that
plugins would be really nice, and now I'm hitting automake issues
:-)... Mayby someday I'll finish something useful hehe)

Roald

Re: subdir-objects

[Sage Weil]

On Wed, 21 Aug 2013, Noah Watkins wrote:
> On Wed, Aug 21, 2013 at 12:45 PM, Roald van Loon <roaldvanloon@gmail.com> wrote:
> >
> > from auto-registering the plugins in the RGW core. The only fix for
> > this is making the RGW core aware of the subdirs/plugins, but I think
> > that's nasty design. I'd like to have it in my make conf.
> 
> This patch will turn on the option (which should also fix your problem
> if I understand correctly?), and should probably be committed anyway
> as newer versions of autotools will complain loudly about our current
> Makefile structure.
> 
> diff --git a/src/Makefile.am b/src/Makefile.am
> index 93f3331..fb7c9dd 100644
> --- a/src/Makefile.am
> +++ b/src/Makefile.am
> @@ -1,4 +1,4 @@
> -AUTOMAKE_OPTIONS = gnu
> +AUTOMAKE_OPTIONS = gnu subdir-objects
>  SUBDIRS = ocf java
>  DIST_SUBDIRS = gtest ocf libs3 java
> 
> > So, the question is; is there a reason why we don't use subdir objects?
> 
> I believe it is just historical, and unfortunately has just been
> repeated over and over. Ideally I think that there should be a
> restructuring to place a Makefile.am in every subdirectory. This would
> address your issue and make it significantly easier to deal with
> situations where we want to build a subset of Ceph, such as just FUSE
> and librados, for example.

Yes, the Makefile.am is in dire need of from TLC from someone who knows a 
bit of autotools-fu.  It is only this way because in the beginning I 
didn't know any better.

sage

Re: subdir-objects

[Roald van Loon]

On Wed, Aug 21, 2013 at 10:41 PM, Sage Weil <sage@inktank.com> wrote:
> Yes, the Makefile.am is in dire need of from TLC from someone who knows a
> bit of autotools-fu.  It is only this way because in the beginning I
> didn't know any better.

Well, my average knowledge of autotools could at least fix this
particular issue and clean up a bit more. It's a start I guess and
helps me to continue my RGW things.

I'll send out a pull request when I've found some time to implement
and test this.

Roald

Re: subdir-objects

[Noah Watkins]

Hi Roald,

Sage just pointed me at your wip-automake branch. I also just pushed
up a branch, make-refactor, that I was hacking on a bit. Not sure how
much overlap there is, or if my approach is bogus, but I thought I'd
point it out to see if there is anything that can be combined :)

-Noah

On Wed, Aug 21, 2013 at 2:01 PM, Roald van Loon <roaldvanloon@gmail.com> wrote:
> On Wed, Aug 21, 2013 at 10:41 PM, Sage Weil <sage@inktank.com> wrote:
>> Yes, the Makefile.am is in dire need of from TLC from someone who knows a
>> bit of autotools-fu.  It is only this way because in the beginning I
>> didn't know any better.
>
> Well, my average knowledge of autotools could at least fix this
> particular issue and clean up a bit more. It's a start I guess and
> helps me to continue my RGW things.
>
> I'll send out a pull request when I've found some time to implement
> and test this.
>
> Roald

Re: subdir-objects

[Roald van Loon]

Hi Noah,

I just had a quick look at your build-refactor branch, and I think the
greatest difference is that you use recursive builds and I don't. I'm
more in favor of non-recursive builds using includes for a number of
reasons. I think the most important reasons for me are;

1) recursive make leads to repetitive AM code
2) recursive make takes much more time to compile (as each directory
needs to run configure and probably most important: you loose optimal
-jX usage due to serialization)
3) non-recursive make knows all deps so rebuildings is much quicker
(it only compiles/links what is required instead of entering all
subdirs)

There is ÏMHO one good reason to use recursive build, and that is
separation of AM code. However, that can be easily achieved with
includes and subdir-objects.

I think this is the most important difference between your and my
approach, and I like to hear your arguments for recursive builds so we
can agree on recursive vs non-recursive make. Then I think it would be
great to combine work!

Roald

On Fri, Sep 6, 2013 at 7:27 PM, Noah Watkins <noah.watkins@inktank.com> wrote:
> Hi Roald,
>
> Sage just pointed me at your wip-automake branch. I also just pushed
> up a branch, make-refactor, that I was hacking on a bit. Not sure how
> much overlap there is, or if my approach is bogus, but I thought I'd
> point it out to see if there is anything that can be combined :)
>
> -Noah
>
> On Wed, Aug 21, 2013 at 2:01 PM, Roald van Loon <roaldvanloon@gmail.com> wrote:
>> On Wed, Aug 21, 2013 at 10:41 PM, Sage Weil <sage@inktank.com> wrote:
>>> Yes, the Makefile.am is in dire need of from TLC from someone who knows a
>>> bit of autotools-fu.  It is only this way because in the beginning I
>>> didn't know any better.
>>
>> Well, my average knowledge of autotools could at least fix this
>> particular issue and clean up a bit more. It's a start I guess and
>> helps me to continue my RGW things.
>>
>> I'll send out a pull request when I've found some time to implement
>> and test this.
>>
>> Roald
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: subdir-objects

[Noah Watkins]

The non-recursive approach is interesting. I just had a quick look in
the tree I despise building the most, openmpi. It has 414 Makefile.am,
and uses recursive builds. The rebuild definitely takes a while to
visit all the sub-dirs, and is pretty annoying when my patience is low
:)

And there is definitely a big +1 for avoiding the SUBDIRS
synchronization that slows down parallel make.

I think the benefits of using recursive builds are that it may be
familiar to the most people, it reflects the methods/suggestions in
the automake manual, and, most importantly, it would seem that its use
forces good decomposition where as a non-recursive approach relies on
guidelines that are easily broken.

Given that the Ceph tree is relatively small (certinaly in comparison
to the 414 directory openmpi monster), are there benefits to the
non-recursive approach that are not performance related?

- Noah

On Sat, Sep 7, 2013 at 1:52 AM, Roald van Loon <roaldvanloon@gmail.com> wrote:
> Hi Noah,
>
> I just had a quick look at your build-refactor branch, and I think the
> greatest difference is that you use recursive builds and I don't. I'm
> more in favor of non-recursive builds using includes for a number of
> reasons. I think the most important reasons for me are;
>
> 1) recursive make leads to repetitive AM code
> 2) recursive make takes much more time to compile (as each directory
> needs to run configure and probably most important: you loose optimal
> -jX usage due to serialization)
> 3) non-recursive make knows all deps so rebuildings is much quicker
> (it only compiles/links what is required instead of entering all
> subdirs)
>
> There is ÏMHO one good reason to use recursive build, and that is
> separation of AM code. However, that can be easily achieved with
> includes and subdir-objects.
>
> I think this is the most important difference between your and my
> approach, and I like to hear your arguments for recursive builds so we
> can agree on recursive vs non-recursive make. Then I think it would be
> great to combine work!
>
> Roald
>
> On Fri, Sep 6, 2013 at 7:27 PM, Noah Watkins <noah.watkins@inktank.com> wrote:
>> Hi Roald,
>>
>> Sage just pointed me at your wip-automake branch. I also just pushed
>> up a branch, make-refactor, that I was hacking on a bit. Not sure how
>> much overlap there is, or if my approach is bogus, but I thought I'd
>> point it out to see if there is anything that can be combined :)
>>
>> -Noah
>>
>> On Wed, Aug 21, 2013 at 2:01 PM, Roald van Loon <roaldvanloon@gmail.com> wrote:
>>> On Wed, Aug 21, 2013 at 10:41 PM, Sage Weil <sage@inktank.com> wrote:
>>>> Yes, the Makefile.am is in dire need of from TLC from someone who knows a
>>>> bit of autotools-fu.  It is only this way because in the beginning I
>>>> didn't know any better.
>>>
>>> Well, my average knowledge of autotools could at least fix this
>>> particular issue and clean up a bit more. It's a start I guess and
>>> helps me to continue my RGW things.
>>>
>>> I'll send out a pull request when I've found some time to implement
>>> and test this.
>>>
>>> Roald
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: subdir-objects

[Noah Watkins]

Oh, and one question about the non-recursive approach. If I stick a
Makefile.am in the test/ directory I can do things like:

  LDADD = all-the-test-dependencies

and then avoid redundant per-target primaries like test_LDADD = (deps)
$(LDADD), because it applies to everything in the Makefile.

Is that possible with the include approach, or would a naked LDADD in
an included Makefile fragment affect all the targets in the file
including it?

-Noah

On Sat, Sep 7, 2013 at 10:38 AM, Noah Watkins <noah.watkins@inktank.com> wrote:
> The non-recursive approach is interesting. I just had a quick look in
> the tree I despise building the most, openmpi. It has 414 Makefile.am,
> and uses recursive builds. The rebuild definitely takes a while to
> visit all the sub-dirs, and is pretty annoying when my patience is low
> :)
>
> And there is definitely a big +1 for avoiding the SUBDIRS
> synchronization that slows down parallel make.
>
> I think the benefits of using recursive builds are that it may be
> familiar to the most people, it reflects the methods/suggestions in
> the automake manual, and, most importantly, it would seem that its use
> forces good decomposition where as a non-recursive approach relies on
> guidelines that are easily broken.
>
> Given that the Ceph tree is relatively small (certinaly in comparison
> to the 414 directory openmpi monster), are there benefits to the
> non-recursive approach that are not performance related?
>
> - Noah
>
> On Sat, Sep 7, 2013 at 1:52 AM, Roald van Loon <roaldvanloon@gmail.com> wrote:
>> Hi Noah,
>>
>> I just had a quick look at your build-refactor branch, and I think the
>> greatest difference is that you use recursive builds and I don't. I'm
>> more in favor of non-recursive builds using includes for a number of
>> reasons. I think the most important reasons for me are;
>>
>> 1) recursive make leads to repetitive AM code
>> 2) recursive make takes much more time to compile (as each directory
>> needs to run configure and probably most important: you loose optimal
>> -jX usage due to serialization)
>> 3) non-recursive make knows all deps so rebuildings is much quicker
>> (it only compiles/links what is required instead of entering all
>> subdirs)
>>
>> There is ÏMHO one good reason to use recursive build, and that is
>> separation of AM code. However, that can be easily achieved with
>> includes and subdir-objects.
>>
>> I think this is the most important difference between your and my
>> approach, and I like to hear your arguments for recursive builds so we
>> can agree on recursive vs non-recursive make. Then I think it would be
>> great to combine work!
>>
>> Roald
>>
>> On Fri, Sep 6, 2013 at 7:27 PM, Noah Watkins <noah.watkins@inktank.com> wrote:
>>> Hi Roald,
>>>
>>> Sage just pointed me at your wip-automake branch. I also just pushed
>>> up a branch, make-refactor, that I was hacking on a bit. Not sure how
>>> much overlap there is, or if my approach is bogus, but I thought I'd
>>> point it out to see if there is anything that can be combined :)
>>>
>>> -Noah
>>>
>>> On Wed, Aug 21, 2013 at 2:01 PM, Roald van Loon <roaldvanloon@gmail.com> wrote:
>>>> On Wed, Aug 21, 2013 at 10:41 PM, Sage Weil <sage@inktank.com> wrote:
>>>>> Yes, the Makefile.am is in dire need of from TLC from someone who knows a
>>>>> bit of autotools-fu.  It is only this way because in the beginning I
>>>>> didn't know any better.
>>>>
>>>> Well, my average knowledge of autotools could at least fix this
>>>> particular issue and clean up a bit more. It's a start I guess and
>>>> helps me to continue my RGW things.
>>>>
>>>> I'll send out a pull request when I've found some time to implement
>>>> and test this.
>>>>
>>>> Roald
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: subdir-objects

[Roald van Loon]

On Sat, Sep 7, 2013 at 7:47 PM, Noah Watkins <noah.watkins@inktank.com> wrote:
> Oh, and one question about the non-recursive approach. If I stick a
> Makefile.am in the test/ directory I can do things like:
>
>   LDADD = all-the-test-dependencies
>
> and then avoid redundant per-target primaries like test_LDADD = (deps)
> $(LDADD), because it applies to everything in the Makefile.
>
> Is that possible with the include approach, or would a naked LDADD in
> an included Makefile fragment affect all the targets in the file
> including it?

LDADD = xyz would indeed affect all targets. However, it's not
something you want to do anyway; using an _LDADD at a target is less
confusing and less prone to errors because you know exactly what
libraries a target needs.

For instance, in test/Makefile.am you can have a debug target
depending on libglobal, which has dependencies set by libglobal
itself;

CEPH_GLOBAL = $(LIBGLOBAL) $(PTHREAD_LIBS) -lm $(CRYPTO_LIBS) $(EXTRALIBS)

And then in test/Makefile.am;

ceph_test_crypto_SOURCES = test/testcrypto.cc
ceph_test_crypto_LDADD = $(CEPH_GLOBAL)
bin_DEBUGPROGRAMS += ceph_test_crypto

And a unittest also depending on libosd;

unittest_pglog_SOURCES = test/osd/TestPGLog.cc
unittest_pglog_LDADD = $(LIBOSD) $(CEPH_GLOBAL)
check_PROGRAMS += unittest_pglog

However, libosd requires libos and libosdc, but that dependency is set
by libosd;

LIBOSD += $(LIBOSDC) $(LIBOS)

This way, you have the dependencies in the right place. With recusive
builds you'll need an "LDADD = libosd.la libosdc.la libos.la
libglobal.la $(PTHREAD_LIBS) -lm $(CRYPTO_LIBS) $(EXTRALIBS)", so
basically you're setting the dependencies of the required libraries in
the makefile requiring those libraries, which is IMHO way to complex.

>> I think the benefits of using recursive builds are that it may be
>> familiar to the most people, it reflects the methods/suggestions in
>> the automake manual, and, most importantly, it would seem that its use
>> forces good decomposition where as a non-recursive approach relies on
>> guidelines that are easily broken.

I don't know how which method is more familiar, but I personally think
that anyone understanding recursive automake is capable of
understanding a simple include :-)

The decomposition is a valid argument. I think that there are some
libraries which might benefit from complete separation, like librados
and a "libceph_client" or something like it. Those can be separated,
but most other can't. The mon, mds, os, and osd subdirs have
inter-dependencies for instance.

We might need to restructure the source tree anyway because at some
points it has grown messy (for instance, libcommon including stuff
from mds, mon but also from include). However, I think implementing a
recursive automake right now forces us to do two things at once;
cleanup the makefiles and do some restructuring in the subdirs. I
personally think it's best to start with cleaning up makefiles and use
an include per subdir, so we can restructure the subdirs into
segregated libraries later on.

So I all boils down to; what to do first :-) Because I agree some
things are better of with recursive builds, but it might be wise not
to do that before we have revisited the source tree layout.

Roald

Re: subdir-objects

[Sage Weil]

I'm particularly keen on doing what we can to reduce build time.  I also 
agree there is a fair bit of tree restructuring that needs to be done.  
(For example, libcommon.la is huge and includes lots of stuff that most 
targets don't actually need; breaking this into smaller pieces would 
probably help.)

Anyway, my gut says that starting with the includes seems like an easy 
incremental path to get us there.  Then we can break out any fully modular 
components where it makes sense...

sage


On Sat, 7 Sep 2013, Roald van Loon wrote:

> On Sat, Sep 7, 2013 at 7:47 PM, Noah Watkins <noah.watkins@inktank.com> wrote:
> > Oh, and one question about the non-recursive approach. If I stick a
> > Makefile.am in the test/ directory I can do things like:
> >
> >   LDADD = all-the-test-dependencies
> >
> > and then avoid redundant per-target primaries like test_LDADD = (deps)
> > $(LDADD), because it applies to everything in the Makefile.
> >
> > Is that possible with the include approach, or would a naked LDADD in
> > an included Makefile fragment affect all the targets in the file
> > including it?
> 
> LDADD = xyz would indeed affect all targets. However, it's not
> something you want to do anyway; using an _LDADD at a target is less
> confusing and less prone to errors because you know exactly what
> libraries a target needs.
> 
> For instance, in test/Makefile.am you can have a debug target
> depending on libglobal, which has dependencies set by libglobal
> itself;
> 
> CEPH_GLOBAL = $(LIBGLOBAL) $(PTHREAD_LIBS) -lm $(CRYPTO_LIBS) $(EXTRALIBS)
> 
> And then in test/Makefile.am;
> 
> ceph_test_crypto_SOURCES = test/testcrypto.cc
> ceph_test_crypto_LDADD = $(CEPH_GLOBAL)
> bin_DEBUGPROGRAMS += ceph_test_crypto
> 
> And a unittest also depending on libosd;
> 
> unittest_pglog_SOURCES = test/osd/TestPGLog.cc
> unittest_pglog_LDADD = $(LIBOSD) $(CEPH_GLOBAL)
> check_PROGRAMS += unittest_pglog
> 
> However, libosd requires libos and libosdc, but that dependency is set
> by libosd;
> 
> LIBOSD += $(LIBOSDC) $(LIBOS)
> 
> This way, you have the dependencies in the right place. With recusive
> builds you'll need an "LDADD = libosd.la libosdc.la libos.la
> libglobal.la $(PTHREAD_LIBS) -lm $(CRYPTO_LIBS) $(EXTRALIBS)", so
> basically you're setting the dependencies of the required libraries in
> the makefile requiring those libraries, which is IMHO way to complex.
> 
> >> I think the benefits of using recursive builds are that it may be
> >> familiar to the most people, it reflects the methods/suggestions in
> >> the automake manual, and, most importantly, it would seem that its use
> >> forces good decomposition where as a non-recursive approach relies on
> >> guidelines that are easily broken.
> 
> I don't know how which method is more familiar, but I personally think
> that anyone understanding recursive automake is capable of
> understanding a simple include :-)
> 
> The decomposition is a valid argument. I think that there are some
> libraries which might benefit from complete separation, like librados
> and a "libceph_client" or something like it. Those can be separated,
> but most other can't. The mon, mds, os, and osd subdirs have
> inter-dependencies for instance.
> 
> We might need to restructure the source tree anyway because at some
> points it has grown messy (for instance, libcommon including stuff
> from mds, mon but also from include). However, I think implementing a
> recursive automake right now forces us to do two things at once;
> cleanup the makefiles and do some restructuring in the subdirs. I
> personally think it's best to start with cleaning up makefiles and use
> an include per subdir, so we can restructure the subdirs into
> segregated libraries later on.
> 
> So I all boils down to; what to do first :-) Because I agree some
> things are better of with recursive builds, but it might be wise not
> to do that before we have revisited the source tree layout.
> 
> Roald
> 
> 

Re: subdir-objects

[Noah Watkins]

I'm so excited to have a refactored automake setup :)

I was just looking through build-refactor, and it doesn't really look
like there is much that could be reused for the non-recrusive
approach. I'll leave it up for a few days, just in case.

On Sat, Sep 7, 2013 at 1:11 PM, Roald van Loon <roaldvanloon@gmail.com> wrote:
> On Sat, Sep 7, 2013 at 7:47 PM, Noah Watkins <noah.watkins@inktank.com> wrote:
>> Oh, and one question about the non-recursive approach. If I stick a
>> Makefile.am in the test/ directory I can do things like:
>>
>>   LDADD = all-the-test-dependencies
>>
>> and then avoid redundant per-target primaries like test_LDADD = (deps)
>> $(LDADD), because it applies to everything in the Makefile.
>>
>> Is that possible with the include approach, or would a naked LDADD in
>> an included Makefile fragment affect all the targets in the file
>> including it?
>
> LDADD = xyz would indeed affect all targets. However, it's not
> something you want to do anyway; using an _LDADD at a target is less
> confusing and less prone to errors because you know exactly what
> libraries a target needs.
>
> For instance, in test/Makefile.am you can have a debug target
> depending on libglobal, which has dependencies set by libglobal
> itself;
>
> CEPH_GLOBAL = $(LIBGLOBAL) $(PTHREAD_LIBS) -lm $(CRYPTO_LIBS) $(EXTRALIBS)
>
> And then in test/Makefile.am;
>
> ceph_test_crypto_SOURCES = test/testcrypto.cc
> ceph_test_crypto_LDADD = $(CEPH_GLOBAL)
> bin_DEBUGPROGRAMS += ceph_test_crypto
>
> And a unittest also depending on libosd;
>
> unittest_pglog_SOURCES = test/osd/TestPGLog.cc
> unittest_pglog_LDADD = $(LIBOSD) $(CEPH_GLOBAL)
> check_PROGRAMS += unittest_pglog
>
> However, libosd requires libos and libosdc, but that dependency is set
> by libosd;
>
> LIBOSD += $(LIBOSDC) $(LIBOS)
>
> This way, you have the dependencies in the right place. With recusive
> builds you'll need an "LDADD = libosd.la libosdc.la libos.la
> libglobal.la $(PTHREAD_LIBS) -lm $(CRYPTO_LIBS) $(EXTRALIBS)", so
> basically you're setting the dependencies of the required libraries in
> the makefile requiring those libraries, which is IMHO way to complex.
>
>>> I think the benefits of using recursive builds are that it may be
>>> familiar to the most people, it reflects the methods/suggestions in
>>> the automake manual, and, most importantly, it would seem that its use
>>> forces good decomposition where as a non-recursive approach relies on
>>> guidelines that are easily broken.
>
> I don't know how which method is more familiar, but I personally think
> that anyone understanding recursive automake is capable of
> understanding a simple include :-)
>
> The decomposition is a valid argument. I think that there are some
> libraries which might benefit from complete separation, like librados
> and a "libceph_client" or something like it. Those can be separated,
> but most other can't. The mon, mds, os, and osd subdirs have
> inter-dependencies for instance.
>
> We might need to restructure the source tree anyway because at some
> points it has grown messy (for instance, libcommon including stuff
> from mds, mon but also from include). However, I think implementing a
> recursive automake right now forces us to do two things at once;
> cleanup the makefiles and do some restructuring in the subdirs. I
> personally think it's best to start with cleaning up makefiles and use
> an include per subdir, so we can restructure the subdirs into
> segregated libraries later on.
>
> So I all boils down to; what to do first :-) Because I agree some
> things are better of with recursive builds, but it might be wise not
> to do that before we have revisited the source tree layout.
>
> Roald

Re: subdir-objects

[Roald van Loon]

Hi Noah, Sage,

The build time has improved in wip-automake somewhat (mainly because
almost all files are now only compiled to objects once instead of
per-target), but much of the time goes into linking.

> I was just looking through build-refactor, and it doesn't really look
> like there is much that could be reused for the non-recrusive
> approach. I'll leave it up for a few days, just in case.

It would be great if you could look at wip-automake and see if you
have ideas/improvements or stuff you implemented that I didn't. All
non-automake changes that I needed to make this branch work just got
merged to master, so this branch is now only about changes .am files.
Of course every src/Makefile.am adjustment in master will require a
rebase, so this branch is a real work-in-progress and gets
force-pushed regularly :-)

Roald

Re: subdir-objects

[Matt W. Benjamin]

Thanks for doing this.  Also in the same interest of building Ceph faster, we've
been experimenting with a CMake translation of the current AM build.  I have no
idea yet if it will be any help ;).

Matt

----- "Noah Watkins" <noah.watkins@inktank.com> wrote:

> I'm so excited to have a refactored automake setup :)
> 
> I was just looking through build-refactor, and it doesn't really look
> like there is much that could be reused for the non-recrusive
> approach. I'll leave it up for a few days, just in case.
> 
> On Sat, Sep 7, 2013 at 1:11 PM, Roald van Loon
> <roaldvanloon@gmail.com> wrote:
> > On Sat, Sep 7, 2013 at 7:47 PM, Noah Watkins
> <noah.watkins@inktank.com> wrote:
> >> Oh, and one question about the non-recursive approach. If I stick
> a
> >> Makefile.am in the test/ directory I can do things like:
> >>
> >>   LDADD = all-the-test-dependencies
> >>
> >> and then avoid redundant per-target primaries like test_LDADD =
> (deps)
> >> $(LDADD), because it applies to everything in the Makefile.
> >>
> >> Is that possible with the include approach, or would a naked LDADD
> in
> >> an included Makefile fragment affect all the targets in the file
> >> including it?
> >
> > LDADD = xyz would indeed affect all targets. However, it's not
> > something you want to do anyway; using an _LDADD at a target is
> less
> > confusing and less prone to errors because you know exactly what
> > libraries a target needs.
> >
> > For instance, in test/Makefile.am you can have a debug target
> > depending on libglobal, which has dependencies set by libglobal
> > itself;
> >
> > CEPH_GLOBAL = $(LIBGLOBAL) $(PTHREAD_LIBS) -lm $(CRYPTO_LIBS)
> $(EXTRALIBS)
> >
> > And then in test/Makefile.am;
> >
> > ceph_test_crypto_SOURCES = test/testcrypto.cc
> > ceph_test_crypto_LDADD = $(CEPH_GLOBAL)
> > bin_DEBUGPROGRAMS += ceph_test_crypto
> >
> > And a unittest also depending on libosd;
> >
> > unittest_pglog_SOURCES = test/osd/TestPGLog.cc
> > unittest_pglog_LDADD = $(LIBOSD) $(CEPH_GLOBAL)
> > check_PROGRAMS += unittest_pglog
> >
> > However, libosd requires libos and libosdc, but that dependency is
> set
> > by libosd;
> >
> > LIBOSD += $(LIBOSDC) $(LIBOS)
> >
> > This way, you have the dependencies in the right place. With
> recusive
> > builds you'll need an "LDADD = libosd.la libosdc.la libos.la
> > libglobal.la $(PTHREAD_LIBS) -lm $(CRYPTO_LIBS) $(EXTRALIBS)", so
> > basically you're setting the dependencies of the required libraries
> in
> > the makefile requiring those libraries, which is IMHO way to
> complex.
> >
> >>> I think the benefits of using recursive builds are that it may be
> >>> familiar to the most people, it reflects the methods/suggestions
> in
> >>> the automake manual, and, most importantly, it would seem that its
> use
> >>> forces good decomposition where as a non-recursive approach relies
> on
> >>> guidelines that are easily broken.
> >
> > I don't know how which method is more familiar, but I personally
> think
> > that anyone understanding recursive automake is capable of
> > understanding a simple include :-)
> >
> > The decomposition is a valid argument. I think that there are some
> > libraries which might benefit from complete separation, like
> librados
> > and a "libceph_client" or something like it. Those can be
> separated,
> > but most other can't. The mon, mds, os, and osd subdirs have
> > inter-dependencies for instance.
> >
> > We might need to restructure the source tree anyway because at some
> > points it has grown messy (for instance, libcommon including stuff
> > from mds, mon but also from include). However, I think implementing
> a
> > recursive automake right now forces us to do two things at once;
> > cleanup the makefiles and do some restructuring in the subdirs. I
> > personally think it's best to start with cleaning up makefiles and
> use
> > an include per subdir, so we can restructure the subdirs into
> > segregated libraries later on.
> >
> > So I all boils down to; what to do first :-) Because I agree some
> > things are better of with recursive builds, but it might be wise
> not
> > to do that before we have revisited the source tree layout.
> >
> > Roald
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel"
> in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

-- 
Matt Benjamin
The Linux Box
206 South Fifth Ave. Suite 150
Ann Arbor, MI  48104

http://linuxbox.com

tel.  734-761-4689 
fax.  734-769-8938 
cel.  734-216-5309 

Re: subdir-objects

[Sage Weil]

On Sun, 8 Sep 2013, Roald van Loon wrote:
> Hi Noah, Sage,
> 
> The build time has improved in wip-automake somewhat (mainly because
> almost all files are now only compiled to objects once instead of
> per-target), but much of the time goes into linking.
> 
> > I was just looking through build-refactor, and it doesn't really look
> > like there is much that could be reused for the non-recrusive
> > approach. I'll leave it up for a few days, just in case.
> 
> It would be great if you could look at wip-automake and see if you
> have ideas/improvements or stuff you implemented that I didn't. All
> non-automake changes that I needed to make this branch work just got
> merged to master, so this branch is now only about changes .am files.
> Of course every src/Makefile.am adjustment in master will require a
> rebase, so this branch is a real work-in-progress and gets
> force-pushed regularly :-)

I took a quick look and it looks pretty great.  The builds are coming back 
okay, too.. any compelling reason not to merge it now?

BTW, Roald, I think the obj_bencher cct change did something strange... 
see http://tracker.ceph.com/issues/6256.  Trying to reproduce it now.

sage

Re: subdir-objects

[Milosz Tanski]

> The build time has improved in wip-automake somewhat (mainly because
> almost all files are now only compiled to objects once instead of
> per-target), but much of the time goes into linking.

Roald,

Are you using gold as a linker already instead of the traditional
binutils bdf based ld. It does make a difference esp on large C++ code
bases (where linking takes a lot of time).

- M


>> I was just looking through build-refactor, and it doesn't really look
>> like there is much that could be reused for the non-recrusive
>> approach. I'll leave it up for a few days, just in case.
>
> It would be great if you could look at wip-automake and see if you
> have ideas/improvements or stuff you implemented that I didn't. All
> non-automake changes that I needed to make this branch work just got
> merged to master, so this branch is now only about changes .am files.
> Of course every src/Makefile.am adjustment in master will require a
> rebase, so this branch is a real work-in-progress and gets
> force-pushed regularly :-)
>
> Roald
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: subdir-objects

[Roald van Loon]

On Mon, Sep 9, 2013 at 1:16 AM, Sage Weil <sage@inktank.com> wrote:
> I took a quick look and it looks pretty great.  The builds are coming back
> okay, too.. any compelling reason not to merge it now?

Nope, not really. I'll do some last build checks at the end of today
(monday over here) and then send you a pull request.

> BTW, Roald, I think the obj_bencher cct change did something strange...
> see http://tracker.ceph.com/issues/6256.  Trying to reproduce it now.

I was wondering when the first bug would show, but I see you've
already fixed it...

Roald

Re: subdir-objects

[Roald van Loon]

On Mon, Sep 9, 2013 at 1:27 AM, Milosz Tanski <milosz@adfin.com> wrote:
> Are you using gold as a linker already instead of the traditional
> binutils bdf based ld. It does make a difference esp on large C++ code
> bases (where linking takes a lot of time).

Actually I'm using by default on code bases the size of Ceph. For me,
the time of a "make check" goes down from ~7m to ~6m, so that's a nice
improvement. Might be a good idea to set up a gitbuilder instance
using gold and let it running on the side for a while?

Roald

Re: subdir-objects

[Roald van Loon]

On Sat, Sep 7, 2013 at 10:39 PM, Sage Weil <sage@inktank.com> wrote:
> I'm particularly keen on doing what we can to reduce build time.  I also
> agree there is a fair bit of tree restructuring that needs to be done.
> (For example, libcommon.la is huge and includes lots of stuff that most
> targets don't actually need; breaking this into smaller pieces would
> probably help.)

I've made some notes while cleaning up the automake stuff. It's on the
wiki as a blueprint now so people can contribute and hopefully it will
start a discussion about this;

http://wiki.ceph.com/01Planning/02Blueprints/Emperor/Source_tree_restructuring

Feel free to add/comment!

Roald