From mboxrd@z Thu Jan 1 00:00:00 1970 From: Willem Jan Withagen Subject: Re: inline vector container Date: Thu, 4 Feb 2016 18:01:31 +0100 Message-ID: <56B383EB.6020207@digiware.nl> References: <1382260306.31547015.1454533040094.JavaMail.zimbra@redhat.com> <59448274.31581132.1454534771209.JavaMail.zimbra@redhat.com> <1674294591.31606738.1454537442587.JavaMail.zimbra@redhat.com> <56B33EB6.5030004@digiware.nl> <1606323224.32079941.1454596978667.JavaMail.zimbra@redhat.com> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: QUOTED-PRINTABLE Return-path: Received: from smtp.digiware.nl ([31.223.170.169]:31424 "EHLO smtp.digiware.nl" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1161049AbcBDRBo (ORCPT ); Thu, 4 Feb 2016 12:01:44 -0500 In-Reply-To: <1606323224.32079941.1454596978667.JavaMail.zimbra@redhat.com> Sender: ceph-devel-owner@vger.kernel.org List-ID: To: Matt Benjamin Cc: Allen Samuels , Casey Bodley , Sage Weil , ceph-devel@vger.kernel.org On 4-2-2016 15:42, Matt Benjamin wrote: > Hi Willem, >=20 > Milosz pasted a file inline. Facebook's upstream repo is > https://github.com/facebook/folly (Mr. Google), but what I was > agreeing is, for the present, if we can't realize boost small_vector, > we might adapt just that type from Folly for the short term (that was > Milosz meaning, in this thread). >=20 > (So don't panic ;) It'll take a bit more to panic, but it would be a shame if we have to port something for the intermediate 2-times. And then to only throw it out a bit later. Where I still have to look at "trivia" like AIO, and RBD... Although my main interest is to get RBD in the freebsd guest DOM's running, especially bhyve. So I hope that that is going to be a bit les= s convoluted than writting a RBD device in the kernel. Especially since only recently I saw patches to add VirtFS to the bhyve virt-io infra. --WjW >=20 > Matt >=20 > ----- Original Message ----- >> From: "Willem Jan Withagen" To: "Matt Benjamin" >> , "Allen Samuels" >> Cc: "Casey Bodley" >> , "Sage Weil" , >> ceph-devel@vger.kernel.org Sent: Thursday, February 4, 2016 7:06:14 >> AM Subject: Re: inline vector container >>=20 >> On 3-2-2016 23:10, Matt Benjamin wrote: >>> I'll prioritize this. Using more of folly may be a worthy >>> project as well, but that can wait for later... :) >>=20 >> Hi Matt, >>=20 >> Can you give me a URL for the folly stuff. Since I cannot find any >> reference to it in the FreeBSD ports. >>=20 >> I think I'd otherwise prefer Boost, since the ports already >> contains a fair amount of Boost stuff. >>=20 >> Currently I have installed: boost-all-1.55.0 The "meta-port" >> for boost libraries boost-docs-1.55.0 Documentation for >> libraries from boost.org boost-jam-1.55.0 Build tool from the >> boost.org boost-libs-1.55.0_9 Free portable C++ libraries >> (without Boost.Python) >>=20 >> But I see that the new ports have migrated to:=20 >> /usr/ports/devel/boost_build/Makefile:PORTVERSION=3D 2.0.m12 >>=20 >> So that would move it beyond 1.6, I presume. >>=20 >> --WjW >>=20 >>=20 >>> tx! >>>=20 >>> Matt >>>=20 >>> ----- Original Message ----- >>>> From: "Allen Samuels" To: "Casey >>>> Bodley" , "Matt Benjamin"=20 >>>> Cc: "Sage Weil" , >>>> ceph-devel@vger.kernel.org Sent: Wednesday, February 3, 2016 >>>> 5:06:57 PM Subject: RE: inline vector container >>>>=20 >>>> I would say that just pulling the latest boost into the tree >>>> and using it for builds is the best path forward. But I'm in no >>>> way competent to make that kind of change in the build system >>>> -- especially during the automake cmake changeover period. >>>>=20 >>>>=20 >>>> Allen Samuels Software Architect, Fellow, Systems and Software >>>> Solutions >>>>=20 >>>> 2880 Junction Avenue, San Jose, CA 95134 T: +1 408 801 7030| M: >>>> +1 408 780 6416 allen.samuels@SanDisk.com >>>>=20 >>>>=20 >>>> -----Original Message----- From: Casey Bodley >>>> [mailto:cbodley@redhat.com] Sent: Wednesday, February 03, 2016 >>>> 1:26 PM To: Matt Benjamin Cc: Sage Weil >>>> ; Allen Samuels ;=20 >>>> ceph-devel@vger.kernel.org Subject: Re: inline vector >>>> container >>>>=20 >>>> Hi Allen, >>>>=20 >>>> I did spend some time a couple months ago experimenting with a >>>> custom allocator with inline storage for standard containers. >>>> You can find my branch on github at=20 >>>> https://github.com/cbodley/ceph/commits/wip-preallocator. >>>>=20 >>>> I was aiming for a general solution that worked with >>>> non-contiguous containers, but couldn't find a good way to >>>> handle deallocations. So the approach was really only a good >>>> fit for std::vector. >>>>=20 >>>> I also had trouble getting it to do the right thing with the >>>> copy and move operators. I found that it was trying to use the >>>> new allocator to release memory that came from the old >>>> allocator. Presumably there's a way to make this work using >>>> std::allocator_traits; have you had better luck here? I'd love >>>> to see your prototype. >>>>=20 >>>> I tend to agree that we should go with the boost solution if we >>>> can. But if a) we can't use boost::small_vector in the >>>> near-term due to versioning, and b) we can get a custom version >>>> tested and working, it might be worth pursuing. >>>>=20 >>>> Casey >>>>=20 >>>> ----- Original Message ----- >>>>> Hi, >>>>>=20 >>>>> I think we should go with small_vector for this role, yes. I >>>>> found two issues. First, there are critical defects in >>>>> small_vector prior to boost 1.6.0. Second, though it is >>>>> header-only, the coupling with related version-specific boost >>>>> types (container, allocator, respective detail types, and >>>>> platform-specific selectors...) seems to make pulling it in >>>>> separately pretty unrealistic as an option. I looked at the >>>>> cost of pulling in and selectively compiling (libs) an >>>>> entire boost 1.6, and that went a lot easier (added less than >>>>> a minute to my build time). >>>>>=20 >>>>> Matt >>>>>=20 >>>>> ----- Original Message ----- >>>>>> From: "Sage Weil" To: "Allen Samuels" >>>>>> Cc: ceph-devel@vger.kernel.org, >>>>>> mbenjami@redhat.com, cbodley@redhat.com Sent: Wednesday, >>>>>> February 3, 2016 3:04:57 PM Subject: Re: inline vector >>>>>> container >>>>>>=20 >>>>>> On Wed, 3 Feb 2016, Allen Samuels wrote: >>>>>>> One thing that the code base needs is a simple vector >>>>>>> container that is optimized for a small number of >>>>>>> elements, i.e., for small element counts we avoid the >>>>>>> malloc/free overhead. But fully supports being resized >>>>>>> into larger containers that are unbounded. Of course it >>>>>>> will need to follow all of the proper object copy/move >>>>>>> semantics so that things like unique_ptr, etc work=20 >>>>>>> correctly. >>>>>>>=20 >>>>>>> I heard rumors that there was one available in the boost >>>>>>> world, but I can=E2=80=99t seem to find it. Do you know about o= ne >>>>>>> being available? >>>>>>=20 >>>>>> http://www.boost.org/doc/libs/1_60_0/doc/html/boost/container/sm= all_ >>>>>> >>>>>>=20 vector.html >>>>>>=20 >>>>>>> I=E2=80=99ve been prototyping one myself, just to get re-famili= ar >>>>>>> with all of the C++11 move sementics, and trivial >>>>>>> constructors, etc. >>>>>>>=20 >>>>>>> The code is pretty complete, but the testing gets rather >>>>>>> involved. There=E2=80=99s no reason to contain this if it=E2=80= =99s >>>>>>> available elsewhere. But if it isn=E2=80=99t I plan on continui= ng >>>>>>> this=E2=80=A6 >>>>>>=20 >>>>>> I think Casey had prototyped something similar using a >>>>>> custom allocator, too, but I didn't actually look at the >>>>>> result. And Matt talked about just pulling the boost one >>>>>> in-tree until we get updated boost in the supported >>>>>> distros, but I forget if he ran into problems there or >>>>>> not... >>>>>>=20 >>>>>> sage >>>>>=20 >>>>> -- -- Matt Benjamin Red Hat, Inc. 315 West Huron Street, >>>>> Suite 140A Ann Arbor, Michigan 48103 >>>>>=20 >>>>> http://www.redhat.com/en/technologies/storage >>>>>=20 >>>>> tel. 734-707-0660 fax. 734-769-8938 cel. 734-216-5309 --=20 >>>>> To unsubscribe from this list: send the line "unsubscribe >>>>> ceph-devel" in the body of a message to >>>>> majordomo@vger.kernel.org More majordomo info at >>>>> http://vger.kernel.org/majordomo-info.html >>>>>=20 >>>> PLEASE NOTE: The information contained in this electronic mail >>>> message is intended only for the use of the designated >>>> recipient(s) named above. If the reader of this message is not >>>> the intended recipient, you are hereby notified that you have >>>> received this message in error and that any review,=20 >>>> dissemination, distribution, or copying of this message is >>>> strictly prohibited. If you have received this communication in >>>> error, please notify the sender by telephone or e-mail (as >>>> shown above) immediately and destroy any and all copies of this >>>> message in your possession (whether hard copies or >>>> electronically stored copies). >>>>=20 >>>=20 >>=20 >> -- To unsubscribe from this list: send the line "unsubscribe >> ceph-devel" in the body of a message to majordomo@vger.kernel.org=20 >> More majordomo info at http://vger.kernel.org/majordomo-info.html >>=20 >=20 -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" i= n the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html