* building boost statically
@ 2016-04-28 22:47 Sage Weil
2016-04-28 22:56 ` Robin H. Johnson
` (2 more replies)
0 siblings, 3 replies; 21+ messages in thread
From: Sage Weil @ 2016-04-28 22:47 UTC (permalink / raw)
To: ceph-devel; +Cc: kdreyer
Hi all,
We'd really like to use the latest boost, mainly so that we can use
small_vector in a zillion places and avoid extra memory allocations. The
distros, as always, are behind.
The thoguht is to bring it in as yet another submodule, build it
statically (it's pretty fast), and link statically. There are a handful
of places where the distro boost links dynamically. For example, on
xenial, it's
libboost-iostreams1.58.0
libboost-program-options1.58.0
libboost-random1.58.0
libboost-system1.58.0
libboost-thread1.58.0
I'm not sure how we should deal with that. We could simply link it
statically. Alternatively, we could ship packages.. but if we do that we
might as well just conditionally build against the new features and then
build a full set of backported boost packages for the distros we care
about.
I'm not sure how to best proceed. But it would be really great to get
access to new boost features like small_vector...
sage
^ permalink raw reply [flat|nested] 21+ messages in thread* Re: building boost statically 2016-04-28 22:47 building boost statically Sage Weil @ 2016-04-28 22:56 ` Robin H. Johnson 2016-04-28 23:14 ` Sage Weil 2016-04-28 23:05 ` Yehuda Sadeh-Weinraub 2016-04-29 2:31 ` Haomai Wang 2 siblings, 1 reply; 21+ messages in thread From: Robin H. Johnson @ 2016-04-28 22:56 UTC (permalink / raw) To: ceph-devel [-- Attachment #1: Type: text/plain, Size: 1151 bytes --] On Thu, Apr 28, 2016 at 06:47:55PM -0400, Sage Weil wrote: > Hi all, > > We'd really like to use the latest boost, mainly so that we can use > small_vector in a zillion places and avoid extra memory allocations. The > distros, as always, are behind. From a distribution perspective, LESS static building is better, so this proposal is a concern, but I do realize that the Gentoo/ArchLinux/CoreOS perspective is very different than the Ubuntu LTS perspective. What's the state of having a more-up-to-date Boost in things like Ubuntu backports, so that you don't have the overhead of having to maintain your own boost packages or statically link? If support for statically linking a newer Boost is brought it, please, please keep dynamic boost builds as fully supported, for distributions that can keep up to date. As a bonus, at some point in the future, when the slower distributions catch up, you might be able to escape the static again. -- Robin Hugh Johnson Gentoo Linux: Developer, Infrastructure Lead, Foundation Trustee E-Mail : robbat2@gentoo.org GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85 [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 445 bytes --] ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: building boost statically 2016-04-28 22:56 ` Robin H. Johnson @ 2016-04-28 23:14 ` Sage Weil 2016-04-28 23:45 ` Matt Benjamin 2016-04-29 7:01 ` Piotr Dałek 0 siblings, 2 replies; 21+ messages in thread From: Sage Weil @ 2016-04-28 23:14 UTC (permalink / raw) To: ceph-devel On Thu, 28 Apr 2016, Robin H. Johnson wrote: > On Thu, Apr 28, 2016 at 06:47:55PM -0400, Sage Weil wrote: > > Hi all, > > > > We'd really like to use the latest boost, mainly so that we can use > > small_vector in a zillion places and avoid extra memory allocations. The > > distros, as always, are behind. > > From a distribution perspective, LESS static building is better, so this > proposal is a concern, but I do realize that the Gentoo/ArchLinux/CoreOS > perspective is very different than the Ubuntu LTS perspective. > > What's the state of having a more-up-to-date Boost in things like Ubuntu > backports, so that you don't have the overhead of having to maintain > your own boost packages or statically link? > > If support for statically linking a newer Boost is brought it, please, > please keep dynamic boost builds as fully supported, for distributions > that can keep up to date. As a bonus, at some point in the future, when > the slower distributions catch up, you might be able to escape the > static again. Yeah, having the option to either build statically or dynamically against an up-to-date distro is probably the right carrot/stick combination to incentivize the distros to move to a newer boost. Being able to conditionally not use the new stuff (e.g., typedef small_vector<> back to vector<>) may or may not work well, depending on which new thing we're trying to use. FWIW, we made the static -> dynamic transition with leveldb because the distros complained and it was nothing but sadness--so so many hours wasted chasing bugs in ancient distro versions of leveldb, and a huge matrix of version possibilities that made it difficult to reproduce user issues. I can't tell if there was ever an instance where it actually gained us anything (e.g., security update in leveldb), but I kind of doubt it. sage ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: building boost statically 2016-04-28 23:14 ` Sage Weil @ 2016-04-28 23:45 ` Matt Benjamin 2016-04-29 7:01 ` Piotr Dałek 1 sibling, 0 replies; 21+ messages in thread From: Matt Benjamin @ 2016-04-28 23:45 UTC (permalink / raw) To: Sage Weil; +Cc: ceph-devel Hi, ----- Original Message ----- > From: "Sage Weil" <sage@newdream.net> > To: ceph-devel@vger.kernel.org > Sent: Thursday, April 28, 2016 7:14:15 PM > Subject: Re: building boost statically > > On Thu, 28 Apr 2016, Robin H. Johnson wrote: > > On Thu, Apr 28, 2016 at 06:47:55PM -0400, Sage Weil wrote: > > > Hi all, > > > > > > We'd really like to use the latest boost, mainly so that we can use > > > small_vector in a zillion places and avoid extra memory allocations. The > > > distros, as always, are behind. > > > > From a distribution perspective, LESS static building is better, so this > > proposal is a concern, but I do realize that the Gentoo/ArchLinux/CoreOS > > perspective is very different than the Ubuntu LTS perspective. > > > > What's the state of having a more-up-to-date Boost in things like Ubuntu > > backports, so that you don't have the overhead of having to maintain > > your own boost packages or statically link? > > > > If support for statically linking a newer Boost is brought it, please, > > please keep dynamic boost builds as fully supported, for distributions > > that can keep up to date. As a bonus, at some point in the future, when > > the slower distributions catch up, you might be able to escape the > > static again. > > Yeah, having the option to either build statically or dynamically against > an up-to-date distro is probably the right carrot/stick combination to > incentivize the distros to move to a newer boost. ++ > > Being able to conditionally not use the new stuff (e.g., typedef > small_vector<> back to vector<>) may or may not work well, depending on > which new thing we're trying to use. I wouldn't want to do that. > > FWIW, we made the static -> dynamic transition with leveldb because the > distros complained and it was nothing but sadness--so so many hours wasted > chasing bugs in ancient distro versions of leveldb, and a huge matrix of > version possibilities that made it difficult to reproduce user issues. I > can't tell if there was ever an instance where it actually gained us > anything (e.g., security update in leveldb), but I kind of doubt it. This reminds me--we seem to still have issues with distro leveldbs, rocksdbs and clang++. Matt > > sage > -- -- Matt Benjamin Red Hat, Inc. 315 West Huron Street, Suite 140A Ann Arbor, Michigan 48103 http://www.redhat.com/en/technologies/storage tel. 734-707-0660 fax. 734-769-8938 cel. 734-216-5309 ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: building boost statically 2016-04-28 23:14 ` Sage Weil 2016-04-28 23:45 ` Matt Benjamin @ 2016-04-29 7:01 ` Piotr Dałek 1 sibling, 0 replies; 21+ messages in thread From: Piotr Dałek @ 2016-04-29 7:01 UTC (permalink / raw) To: ceph-devel On Thu, Apr 28, 2016 at 07:14:15PM -0400, Sage Weil wrote: > On Thu, 28 Apr 2016, Robin H. Johnson wrote: > > If support for statically linking a newer Boost is brought it, please, > > please keep dynamic boost builds as fully supported, for distributions > > that can keep up to date. As a bonus, at some point in the future, when > > the slower distributions catch up, you might be able to escape the > > static again. > > Yeah, having the option to either build statically or dynamically against > an up-to-date distro is probably the right carrot/stick combination to > incentivize the distros to move to a newer boost. Plus, final binary sizes would be smaller if dynamic linking would succeed. > Being able to conditionally not use the new stuff (e.g., typedef > small_vector<> back to vector<>) may or may not work well, depending on > which new thing we're trying to use. That's bad idea, IMHO. Why not check for features we need on configure state and use static or dynamic accordingly? -- Piotr Dałek branch@predictor.org.pl http://blog.predictor.org.pl -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: building boost statically 2016-04-28 22:47 building boost statically Sage Weil 2016-04-28 22:56 ` Robin H. Johnson @ 2016-04-28 23:05 ` Yehuda Sadeh-Weinraub 2016-04-29 2:31 ` Haomai Wang 2 siblings, 0 replies; 21+ messages in thread From: Yehuda Sadeh-Weinraub @ 2016-04-28 23:05 UTC (permalink / raw) To: Sage Weil; +Cc: ceph-devel, Dreyer, Ken On Thu, Apr 28, 2016 at 3:47 PM, Sage Weil <sweil@redhat.com> wrote: > Hi all, > > We'd really like to use the latest boost, mainly so that we can use > small_vector in a zillion places and avoid extra memory allocations. The > distros, as always, are behind. > > The thoguht is to bring it in as yet another submodule, build it > statically (it's pretty fast), and link statically. There are a handful > of places where the distro boost links dynamically. For example, on > xenial, it's > > libboost-iostreams1.58.0 > libboost-program-options1.58.0 > libboost-random1.58.0 > libboost-system1.58.0 > libboost-thread1.58.0 > > I'm not sure how we should deal with that. We could simply link it > statically. Alternatively, we could ship packages.. but if we do that we > might as well just conditionally build against the new features and then > build a full set of backported boost packages for the distros we care > about. > > I'm not sure how to best proceed. But it would be really great to get > access to new boost features like small_vector... > If we need to have it as a submodule then build it conditionally. Then make a decision per distro, whether we want to not use new functionality, or backport packages, or have it linked statically. Yehuda ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: building boost statically 2016-04-28 22:47 building boost statically Sage Weil 2016-04-28 22:56 ` Robin H. Johnson 2016-04-28 23:05 ` Yehuda Sadeh-Weinraub @ 2016-04-29 2:31 ` Haomai Wang 2016-04-29 22:55 ` Ken Dreyer 2 siblings, 1 reply; 21+ messages in thread From: Haomai Wang @ 2016-04-29 2:31 UTC (permalink / raw) To: Sage Weil; +Cc: ceph-devel@vger.kernel.org, Ken Dreyer On Thu, Apr 28, 2016 at 5:47 PM, Sage Weil <sweil@redhat.com> wrote: > Hi all, > > We'd really like to use the latest boost, mainly so that we can use > small_vector in a zillion places and avoid extra memory allocations. The > distros, as always, are behind. > > The thoguht is to bring it in as yet another submodule, build it > statically (it's pretty fast), and link statically. There are a handful > of places where the distro boost links dynamically. For example, on > xenial, it's > > libboost-iostreams1.58.0 > libboost-program-options1.58.0 > libboost-random1.58.0 > libboost-system1.58.0 > libboost-thread1.58.0 Yes, I'm always glad to see more deps removed! And I think we can remove part of boost library deps totally to reduce adding static lib size? like libboost-random and libboost-thread or anything else. We may replace with c++11 lib(I'm not a boost fan....). > > I'm not sure how we should deal with that. We could simply link it > statically. Alternatively, we could ship packages.. but if we do that we > might as well just conditionally build against the new features and then > build a full set of backported boost packages for the distros we care > about. > > I'm not sure how to best proceed. But it would be really great to get > access to new boost features like small_vector... > > sage > > -- > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: building boost statically 2016-04-29 2:31 ` Haomai Wang @ 2016-04-29 22:55 ` Ken Dreyer 2016-04-30 1:43 ` Sage Weil 0 siblings, 1 reply; 21+ messages in thread From: Ken Dreyer @ 2016-04-29 22:55 UTC (permalink / raw) To: Haomai Wang; +Cc: Sage Weil, ceph-devel@vger.kernel.org On Thu, Apr 28, 2016 at 8:31 PM, Haomai Wang <haomai@xsky.com> wrote: > Yes, I'm always glad to see more deps removed! It's not removed, it's just moved. The Ceph project will then be on the hook for all boost security updates. I think we should bundle less not more. Build times are already long enough... - Ken ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: building boost statically 2016-04-29 22:55 ` Ken Dreyer @ 2016-04-30 1:43 ` Sage Weil 2016-04-30 3:24 ` Allen Samuels 0 siblings, 1 reply; 21+ messages in thread From: Sage Weil @ 2016-04-30 1:43 UTC (permalink / raw) To: Ken Dreyer; +Cc: Haomai Wang, ceph-devel@vger.kernel.org On Fri, 29 Apr 2016, Ken Dreyer wrote: > On Thu, Apr 28, 2016 at 8:31 PM, Haomai Wang <haomai@xsky.com> wrote: > > Yes, I'm always glad to see more deps removed! > > It's not removed, it's just moved. The Ceph project will then be on > the hook for all boost security updates. > > I think we should bundle less not more. Build times are already long enough... If building separate packages and including them in all the repos is in fact easier, that seems like the right choice. Static linkage makes development easier, but we can make do. Whether it's static or dynamic, though, we're on the hook for updating the packages for bugs or security update either way. Unless we just give up an wait for the distros to catch up, but I don't think that's a viable option... sage ^ permalink raw reply [flat|nested] 21+ messages in thread
* RE: building boost statically 2016-04-30 1:43 ` Sage Weil @ 2016-04-30 3:24 ` Allen Samuels 2016-05-01 15:08 ` Jesse Williamson 0 siblings, 1 reply; 21+ messages in thread From: Allen Samuels @ 2016-04-30 3:24 UTC (permalink / raw) To: Sage Weil, Ken Dreyer; +Cc: Haomai Wang, ceph-devel@vger.kernel.org Let's not wait. As another option. If all we're looking for is "small_vector", then I looked at taking the equivalent out of the Folly library a while ago. It looked like it would be relatively easy to hack up the one header file to make it standalone. Then we could just include it in the Ceph source tree as a common file (it's a header-only thing). Once we're confident that all of the relevant distro's are updated we can easily dispose of the file and do a top level rename. Allen Samuels Software Architect, Fellow, Systems and Software Solutions 2880 Junction Avenue, San Jose, CA 95134 T: +1 408 801 7030| M: +1 408 780 6416 allen.samuels@SanDisk.com > -----Original Message----- > From: ceph-devel-owner@vger.kernel.org [mailto:ceph-devel- > owner@vger.kernel.org] On Behalf Of Sage Weil > Sent: Friday, April 29, 2016 6:44 PM > To: Ken Dreyer <kdreyer@redhat.com> > Cc: Haomai Wang <haomai@xsky.com>; ceph-devel@vger.kernel.org > Subject: Re: building boost statically > > On Fri, 29 Apr 2016, Ken Dreyer wrote: > > On Thu, Apr 28, 2016 at 8:31 PM, Haomai Wang <haomai@xsky.com> > wrote: > > > Yes, I'm always glad to see more deps removed! > > > > It's not removed, it's just moved. The Ceph project will then be on > > the hook for all boost security updates. > > > > I think we should bundle less not more. Build times are already long > enough... > > If building separate packages and including them in all the repos is in fact > easier, that seems like the right choice. Static linkage makes development > easier, but we can make do. > > Whether it's static or dynamic, though, we're on the hook for updating the > packages for bugs or security update either way. Unless we just give up an > wait for the distros to catch up, but I don't think that's a viable option... > > sage > > -- > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the > body of a message to majordomo@vger.kernel.org More majordomo info at > http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 21+ messages in thread
* RE: building boost statically 2016-04-30 3:24 ` Allen Samuels @ 2016-05-01 15:08 ` Jesse Williamson 2016-05-01 15:14 ` Sage Weil 0 siblings, 1 reply; 21+ messages in thread From: Jesse Williamson @ 2016-05-01 15:08 UTC (permalink / raw) To: Allen Samuels Cc: Sage Weil, Ken Dreyer, Haomai Wang, ceph-devel@vger.kernel.org On Sat, 30 Apr 2016, Allen Samuels wrote: > Let's not wait. > > As another option. If all we're looking for is "small_vector", then I > looked at taking the equivalent out of the Folly library a while ago. It > looked like it would be relatively easy to hack up the one header file > to make it standalone. There's a utility in boost, "bcp", that might be useful: http://www.boost.org/doc/libs/1_60_0/tools/bcp/doc/html/index.html -Jesse ^ permalink raw reply [flat|nested] 21+ messages in thread
* RE: building boost statically 2016-05-01 15:08 ` Jesse Williamson @ 2016-05-01 15:14 ` Sage Weil 2016-05-02 3:28 ` Adam C. Emerson 2016-05-03 16:28 ` Robert LeBlanc 0 siblings, 2 replies; 21+ messages in thread From: Sage Weil @ 2016-05-01 15:14 UTC (permalink / raw) To: Jesse Williamson Cc: Allen Samuels, Ken Dreyer, Haomai Wang, ceph-devel@vger.kernel.org On Sun, 1 May 2016, Jesse Williamson wrote: > On Sat, 30 Apr 2016, Allen Samuels wrote: > > > Let's not wait. > > > > As another option. If all we're looking for is "small_vector", then I looked > > at taking the equivalent out of the Folly library a while ago. It looked > > like it would be relatively easy to hack up the one header file to make it > > standalone. > > There's a utility in boost, "bcp", that might be useful: > > http://www.boost.org/doc/libs/1_60_0/tools/bcp/doc/html/index.html Given that this is the only piece we want immediately (that i know of), that there would be significant work to either pull boost inline as a submodule *or* to build and ship a proper package, let's go the bcp route for now. Jesse, is this something you can tackle? Thanks! sage ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: building boost statically 2016-05-01 15:14 ` Sage Weil @ 2016-05-02 3:28 ` Adam C. Emerson 2016-05-02 16:09 ` Jesse Williamson 2016-05-03 16:28 ` Robert LeBlanc 1 sibling, 1 reply; 21+ messages in thread From: Adam C. Emerson @ 2016-05-02 3:28 UTC (permalink / raw) To: Sage Weil Cc: Jesse Williamson, Allen Samuels, Ken Dreyer, Haomai Wang, ceph-devel@vger.kernel.org On 01/05/2016, Sage Weil wrote: > Given that this is the only piece we want immediately (that i know of), > that there would be significant work to either pull boost inline as a > submodule *or* to build and ship a proper package, let's go the bcp route > for now. > > Jesse, is this something you can tackle? > I'm not sure if I like this idea. There are other places where I've wanted to use something newer than the boost on Trusty Tahr/RHEL7, like the newer intrusive_ptr. Some other people have mentioned wishing for newer boost functionality, too. If we're going to bundle anything at all, I'd rather we bundle the whole thing so we get the full advantage of trying to swallow the extra capacity. BCP sounds like we're signing up for just as much work and maintenance burden and doesn't give us the full benefit. -- Senior Software Engineer Red Hat Storage, Ann Arbor, MI, US IRC: Aemerson@{RedHat, OFTC, Freenode} 0x80F7544B90EDBFB9 E707 86BA 0C1B 62CC 152C 7C12 80F7 544B 90ED BFB9 ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: building boost statically 2016-05-02 3:28 ` Adam C. Emerson @ 2016-05-02 16:09 ` Jesse Williamson 2016-05-02 16:14 ` Adam C. Emerson 0 siblings, 1 reply; 21+ messages in thread From: Jesse Williamson @ 2016-05-02 16:09 UTC (permalink / raw) To: Adam C. Emerson Cc: Sage Weil, Allen Samuels, Ken Dreyer, Haomai Wang, ceph-devel@vger.kernel.org On Sun, 1 May 2016, Adam C. Emerson wrote: >> Jesse, is this something you can tackle? I'll have a look at it this morning and see how it goes. > I'm not sure if I like this idea. There are other places where I've > wanted to use something newer than the boost on Trusty Tahr/RHEL7, like > the newer intrusive_ptr. Some other people have mentioned wishing for > newer boost functionality, too. I think that in the long run, rather than bundling or using bcp, it would be nice to have cmake/autoconf let you point to the boost version you'd like to use. -Jesse ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: building boost statically 2016-05-02 16:09 ` Jesse Williamson @ 2016-05-02 16:14 ` Adam C. Emerson 2016-05-02 16:18 ` Jesse Williamson 2016-05-02 23:04 ` Matt Benjamin 0 siblings, 2 replies; 21+ messages in thread From: Adam C. Emerson @ 2016-05-02 16:14 UTC (permalink / raw) To: Jesse Williamson Cc: Sage Weil, Allen Samuels, Ken Dreyer, Haomai Wang, ceph-devel@vger.kernel.org On 02/05/2016, Jesse Williamson wrote: > I think that in the long run, rather than bundling or using bcp, it would be > nice to have cmake/autoconf let you point to the boost version you'd like to > use. If we could manage it, that would be my ideal world. Rather than making it A Submodule, say, if we could have cmake examine the system boost to see what version it is and, if the version is greater than or equal to the required version, just use it. Otherwise just fetch the required version and build against it statically. -- Senior Software Engineer Red Hat Storage, Ann Arbor, MI, US IRC: Aemerson@{RedHat, OFTC, Freenode} 0x80F7544B90EDBFB9 E707 86BA 0C1B 62CC 152C 7C12 80F7 544B 90ED BFB9 ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: building boost statically 2016-05-02 16:14 ` Adam C. Emerson @ 2016-05-02 16:18 ` Jesse Williamson 2016-05-02 16:31 ` Adam C. Emerson 2016-05-02 23:04 ` Matt Benjamin 1 sibling, 1 reply; 21+ messages in thread From: Jesse Williamson @ 2016-05-02 16:18 UTC (permalink / raw) To: Adam C. Emerson Cc: Sage Weil, Allen Samuels, Ken Dreyer, Haomai Wang, ceph-devel@vger.kernel.org On Mon, 2 May 2016, Adam C. Emerson wrote: > If we could manage it, that would be my ideal world. Rather than > making it A Submodule, say, if we could have cmake examine the system > boost to see what version it is and, if the version is greater than or > equal to the required version, just use it. Otherwise just fetch the > required version and build against it statically. Auto-fetching would be nice. One caution against just going with greater-than is that some libraries in the Boost project introduce breaking changes now and again, so being able to go with a specific version can be important. -Jesse ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: building boost statically 2016-05-02 16:18 ` Jesse Williamson @ 2016-05-02 16:31 ` Adam C. Emerson 0 siblings, 0 replies; 21+ messages in thread From: Adam C. Emerson @ 2016-05-02 16:31 UTC (permalink / raw) To: Jesse Williamson Cc: Sage Weil, Allen Samuels, Ken Dreyer, Haomai Wang, ceph-devel@vger.kernel.org On 02/05/2016, Jesse Williamson wrote: > Auto-fetching would be nice. One caution against just going with > greater-than is that some libraries in the Boost project introduce breaking > changes now and again, so being able to go with a specific version can be > important. Specific range maybe? Or, if we required an exact match, had CMake print out that it is going to fetch that version rather than using the system Boost, and we had an option to override and tell it to use the system Boost anyway that would be good. That might be a good compromise between a distribution's desire to use system packages and our desire not to be on the hook for versions we've never tested against. -- Senior Software Engineer Red Hat Storage, Ann Arbor, MI, US IRC: Aemerson@{RedHat, OFTC, Freenode} 0x80F7544B90EDBFB9 E707 86BA 0C1B 62CC 152C 7C12 80F7 544B 90ED BFB9 ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: building boost statically 2016-05-02 16:14 ` Adam C. Emerson 2016-05-02 16:18 ` Jesse Williamson @ 2016-05-02 23:04 ` Matt Benjamin 2016-05-03 1:01 ` Jesse Williamson 1 sibling, 1 reply; 21+ messages in thread From: Matt Benjamin @ 2016-05-02 23:04 UTC (permalink / raw) To: Adam C. Emerson Cc: Jesse Williamson, Sage Weil, Allen Samuels, Ken Dreyer, Haomai Wang, ceph-devel Just wanted to ++ this. Matt ----- Original Message ----- > From: "Adam C. Emerson" <aemerson@redhat.com> > To: "Jesse Williamson" <jwilliamson@suse.com> > Cc: "Sage Weil" <sage@newdream.net>, "Allen Samuels" <Allen.Samuels@sandisk.com>, "Ken Dreyer" <kdreyer@redhat.com>, > "Haomai Wang" <haomai@xsky.com>, ceph-devel@vger.kernel.org > Sent: Monday, May 2, 2016 12:14:50 PM > Subject: Re: building boost statically > > On 02/05/2016, Jesse Williamson wrote: > > I think that in the long run, rather than bundling or using bcp, it would > > be > > nice to have cmake/autoconf let you point to the boost version you'd like > > to > > use. > > If we could manage it, that would be my ideal world. Rather than > making it A Submodule, say, if we could have cmake examine the system > boost to see what version it is and, if the version is greater than or > equal to the required version, just use it. Otherwise just fetch the > required version and build against it statically. > > -- > Senior Software Engineer Red Hat Storage, Ann Arbor, MI, US > IRC: Aemerson@{RedHat, OFTC, Freenode} > 0x80F7544B90EDBFB9 E707 86BA 0C1B 62CC 152C 7C12 80F7 544B 90ED BFB9 > -- > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > -- Matt Benjamin Red Hat, Inc. 315 West Huron Street, Suite 140A Ann Arbor, Michigan 48103 http://www.redhat.com/en/technologies/storage tel. 734-707-0660 fax. 734-769-8938 cel. 734-216-5309 ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: building boost statically 2016-05-02 23:04 ` Matt Benjamin @ 2016-05-03 1:01 ` Jesse Williamson 2016-05-03 1:14 ` Matt Benjamin 0 siblings, 1 reply; 21+ messages in thread From: Jesse Williamson @ 2016-05-03 1:01 UTC (permalink / raw) To: Matt Benjamin Cc: Adam C. Emerson, Sage Weil, Allen Samuels, Ken Dreyer, Haomai Wang, ceph-devel On Mon, 2 May 2016, Matt Benjamin wrote: So, just a quick update here: I used bcp to extract boost containers. This works nicely, but you end up with 1628 files. It's doing what it's /supposed/ to do, since part of the purpose is to provide all the dependencies (for example, Boost::MPL is used), but that seems like maybe not what we want to do. Just using the boost/container headers is also possible, at a bit of risk that if we try to build on a system where one of the dependencies is incompatible in some way, Ceph won't build (perhaps a fairly low risk, but I'm throwing it out there). If that risk is acceptable, then we wouldn't have a large set of headers, but in effect would still be shipping our own little mini-fork. CMake looks like it has a module that can help us, and the general direction during an IRC chat was that we should do this as part of the move to cmake, rather than fidding with autoconf. -Jesse > Just wanted to ++ this. > > Matt > > ----- Original Message ----- >> From: "Adam C. Emerson" <aemerson@redhat.com> >> To: "Jesse Williamson" <jwilliamson@suse.com> >> Cc: "Sage Weil" <sage@newdream.net>, "Allen Samuels" <Allen.Samuels@sandisk.com>, "Ken Dreyer" <kdreyer@redhat.com>, >> "Haomai Wang" <haomai@xsky.com>, ceph-devel@vger.kernel.org >> Sent: Monday, May 2, 2016 12:14:50 PM >> Subject: Re: building boost statically >> >> On 02/05/2016, Jesse Williamson wrote: >>> I think that in the long run, rather than bundling or using bcp, it would >>> be >>> nice to have cmake/autoconf let you point to the boost version you'd like >>> to >>> use. >> >> If we could manage it, that would be my ideal world. Rather than >> making it A Submodule, say, if we could have cmake examine the system >> boost to see what version it is and, if the version is greater than or >> equal to the required version, just use it. Otherwise just fetch the >> required version and build against it statically. >> >> -- >> Senior Software Engineer Red Hat Storage, Ann Arbor, MI, US >> IRC: Aemerson@{RedHat, OFTC, Freenode} >> 0x80F7544B90EDBFB9 E707 86BA 0C1B 62CC 152C 7C12 80F7 544B 90ED BFB9 >> -- >> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >> the body of a message to majordomo@vger.kernel.org >> More majordomo info at http://vger.kernel.org/majordomo-info.html >> > > -- > Matt Benjamin > Red Hat, Inc. > 315 West Huron Street, Suite 140A > Ann Arbor, Michigan 48103 > > http://www.redhat.com/en/technologies/storage > > tel. 734-707-0660 > fax. 734-769-8938 > cel. 734-216-5309 > -- > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > > ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: building boost statically 2016-05-03 1:01 ` Jesse Williamson @ 2016-05-03 1:14 ` Matt Benjamin 0 siblings, 0 replies; 21+ messages in thread From: Matt Benjamin @ 2016-05-03 1:14 UTC (permalink / raw) To: Jesse Williamson Cc: Adam C. Emerson, Sage Weil, Allen Samuels, Ken Dreyer, Haomai Wang, ceph-devel Hi Jesse, ----- Original Message ----- > From: "Jesse Williamson" <jwilliamson@suse.com> > To: "Matt Benjamin" <mbenjamin@redhat.com> > Cc: "Adam C. Emerson" <aemerson@redhat.com>, "Sage Weil" <sage@newdream.net>, "Allen Samuels" > <Allen.Samuels@sandisk.com>, "Ken Dreyer" <kdreyer@redhat.com>, "Haomai Wang" <haomai@xsky.com>, > ceph-devel@vger.kernel.org > Sent: Monday, May 2, 2016 9:01:44 PM > Subject: Re: building boost statically > > On Mon, 2 May 2016, Matt Benjamin wrote: > > So, just a quick update here: I used bcp to extract boost containers. This > works nicely, but you end up with 1628 files. It's doing what it's > /supposed/ to do, since part of the purpose is to provide all the > dependencies (for example, Boost::MPL is used), but that seems like maybe > not what we want to do. Yes. I experimented with this, but it didn't seem at all compact enough to be preferable to just building boost. The latter, building just what Ceph uses from boost now, took less than a minute on a single core (i7). > > Just using the boost/container headers is also possible, at a bit of risk > that if we try to build on a system where one of the dependencies is > incompatible in some way, Ceph won't build (perhaps a fairly low risk, but > I'm throwing it out there). If that risk is acceptable, then we wouldn't > have a large set of headers, but in effect would still be shipping our own > little mini-fork. I experimented with this also. What I found was that the number of boost internal (including unexposed) dependencies is much larger than container. This is a marvel of efficiency for the boost library internally, but isn't helpful for extracting a single like small_vector. In particular, conflicts with an installed boost are many. > > CMake looks like it has a module that can help us, and the general > direction during an IRC chat was that we should do this as part of the > move to cmake, rather than fidding with autoconf. That seems like a good idea. Matt > > -Jesse > > > Just wanted to ++ this. > > > > Matt > > > > ----- Original Message ----- > >> From: "Adam C. Emerson" <aemerson@redhat.com> > >> To: "Jesse Williamson" <jwilliamson@suse.com> > >> Cc: "Sage Weil" <sage@newdream.net>, "Allen Samuels" > >> <Allen.Samuels@sandisk.com>, "Ken Dreyer" <kdreyer@redhat.com>, > >> "Haomai Wang" <haomai@xsky.com>, ceph-devel@vger.kernel.org > >> Sent: Monday, May 2, 2016 12:14:50 PM > >> Subject: Re: building boost statically > >> > >> On 02/05/2016, Jesse Williamson wrote: > >>> I think that in the long run, rather than bundling or using bcp, it would > >>> be > >>> nice to have cmake/autoconf let you point to the boost version you'd like > >>> to > >>> use. > >> > >> If we could manage it, that would be my ideal world. Rather than > >> making it A Submodule, say, if we could have cmake examine the system > >> boost to see what version it is and, if the version is greater than or > >> equal to the required version, just use it. Otherwise just fetch the > >> required version and build against it statically. > >> > >> -- > >> Senior Software Engineer Red Hat Storage, Ann Arbor, MI, US > >> IRC: Aemerson@{RedHat, OFTC, Freenode} > >> 0x80F7544B90EDBFB9 E707 86BA 0C1B 62CC 152C 7C12 80F7 544B 90ED BFB9 > >> -- > >> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > >> the body of a message to majordomo@vger.kernel.org > >> More majordomo info at http://vger.kernel.org/majordomo-info.html > >> > > > > -- > > Matt Benjamin > > Red Hat, Inc. > > 315 West Huron Street, Suite 140A > > Ann Arbor, Michigan 48103 > > > > http://www.redhat.com/en/technologies/storage > > > > tel. 734-707-0660 > > fax. 734-769-8938 > > cel. 734-216-5309 > > -- > > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > > the body of a message to majordomo@vger.kernel.org > > More majordomo info at http://vger.kernel.org/majordomo-info.html > > > > > -- Matt Benjamin Red Hat, Inc. 315 West Huron Street, Suite 140A Ann Arbor, Michigan 48103 http://www.redhat.com/en/technologies/storage tel. 734-707-0660 fax. 734-769-8938 cel. 734-216-5309 ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: building boost statically 2016-05-01 15:14 ` Sage Weil 2016-05-02 3:28 ` Adam C. Emerson @ 2016-05-03 16:28 ` Robert LeBlanc 1 sibling, 0 replies; 21+ messages in thread From: Robert LeBlanc @ 2016-05-03 16:28 UTC (permalink / raw) To: Sage Weil Cc: Jesse Williamson, Allen Samuels, Ken Dreyer, Haomai Wang, ceph-devel@vger.kernel.org -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 I really wanted to use intrusive maps that was in boost 1.59 for the WPQ. So I think there are multiple places that could leverage the new boost features if it were available. -----BEGIN PGP SIGNATURE----- Version: Mailvelope v1.3.6 Comment: https://www.mailvelope.com wsFcBAEBCAAQBQJXKNGPCRDmVDuy+mK58QAAr3cP+wbXVtaBFTkvJQSAH+TQ 5C8NOQcmph0Nmf/tL9byox8USML8OS4RiKh2EI6MnbRLa6ZrE65irnkBEXX3 b+OM1JysNbLnyo+myTn7MEtGfjQCG4rsm0JtHX69cs35kLRtLdvRj7YTiWkf MWjt9nIuiLDWJH9TyhO9aBQp+t52m3u1Lxrl7XL9ie7vFq8eQp4yXvESvj9t 5xawxNp3xaDLZHt48qA2kmuA1S9oqymqNWJU4yqeKqZY1OtGE1FN5bg1cn0d ApvuI250jyWEV0pjyp0T8Auq+E0x7+Fn52FRfyZzMNO3FeRzAMn+y2z8djT9 XHf6j3Y/PAF9udASVguaUHPg4OnUihM+ffGmMCkSrpoNcc/cWRpB8bRyG/12 UKbwwREYQ+UhENohoblkl3zTqn3E7891/d34mObcmZUQKy5i5i1l+wl33cZO NtXY/xC567kXAUv1zKEx+2bSUhQTtA4rJh7Kqrgk/lyJrNozRsK3lhQbucx7 a45VA2GW9RqI9zX9GSiDDZiZinOMExcN+E+5DadsdeOPGqswE7lgmDF7BH0L TIJjnr+4r4vSQ+4/8OEHP05Gh5OZv5p/penzF+LCvkboYDDZGfwwbfbNE04P 96An6l8Yf+L2XbhJfptR0155zjhyBRCvm8FK8M6O/3BckSDEC7IvYVMVgp1F RB99 =SK2q -----END PGP SIGNATURE----- ---------------- Robert LeBlanc PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1 On Sun, May 1, 2016 at 9:14 AM, Sage Weil <sage@newdream.net> wrote: > On Sun, 1 May 2016, Jesse Williamson wrote: >> On Sat, 30 Apr 2016, Allen Samuels wrote: >> >> > Let's not wait. >> > >> > As another option. If all we're looking for is "small_vector", then I looked >> > at taking the equivalent out of the Folly library a while ago. It looked >> > like it would be relatively easy to hack up the one header file to make it >> > standalone. >> >> There's a utility in boost, "bcp", that might be useful: >> >> http://www.boost.org/doc/libs/1_60_0/tools/bcp/doc/html/index.html > > Given that this is the only piece we want immediately (that i know of), > that there would be significant work to either pull boost inline as a > submodule *or* to build and ship a proper package, let's go the bcp route > for now. > > Jesse, is this something you can tackle? > > Thanks! > sage > -- > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 21+ messages in thread
end of thread, other threads:[~2016-05-03 16:28 UTC | newest] Thread overview: 21+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2016-04-28 22:47 building boost statically Sage Weil 2016-04-28 22:56 ` Robin H. Johnson 2016-04-28 23:14 ` Sage Weil 2016-04-28 23:45 ` Matt Benjamin 2016-04-29 7:01 ` Piotr Dałek 2016-04-28 23:05 ` Yehuda Sadeh-Weinraub 2016-04-29 2:31 ` Haomai Wang 2016-04-29 22:55 ` Ken Dreyer 2016-04-30 1:43 ` Sage Weil 2016-04-30 3:24 ` Allen Samuels 2016-05-01 15:08 ` Jesse Williamson 2016-05-01 15:14 ` Sage Weil 2016-05-02 3:28 ` Adam C. Emerson 2016-05-02 16:09 ` Jesse Williamson 2016-05-02 16:14 ` Adam C. Emerson 2016-05-02 16:18 ` Jesse Williamson 2016-05-02 16:31 ` Adam C. Emerson 2016-05-02 23:04 ` Matt Benjamin 2016-05-03 1:01 ` Jesse Williamson 2016-05-03 1:14 ` Matt Benjamin 2016-05-03 16:28 ` Robert LeBlanc
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.