ceph / rocksdb

ceph / rocksdb

[Marcus Watts]

So, about that rocksdb thing.

Rocksdb ships with 2 build systems:
	cmake		- windows only
	make based	- everything else

The makefile is very "retro".  Um.  Let's just leave it there.

The cmake part was more interesting; the main problem it had
was it was *way* too windows specific.  Which is actually kinda
hard to do, since that's just what cmake wasn't supposed to be.

Building rocksdb (-g) takes about 1g of build tree space,
and running "make check" on it takes about half an hour.
I really don't want to slow down my ceph builds this way,
so rather than make rocksdb with the rest of ceph, I would
much rather it be packaged / installed separately.

So I put together:
a set of changes to build rocksdb with cmake on linux
an rpm .spec file to build it for fedora.

Rpms (source & amd64) can be found here:
	http://people.redhat.com/mwatts/rocksdb/
and a git repo with the cmake changes here,
	https://github.com/mdw-at-linuxbox/rocksdb

The cmake parts could certainly still use more work; I haven't
tested this and it will probably need changes on anything other
than x86_64, such as certainly any non-gcc/non-linux platform.

					-Marcus Watts

Re: ceph / rocksdb

[Ken Dreyer]

I'm really interested in getting our various bundled libraries into
separate packages.

Does ceph's rocksdb have a lot of changes from rocksdb upstream? If
so, I'm leaning towards packaging this as "ceph-rocksdb" until those
changes are present in an upstream rocksdb release.

- Ken

On Tue, Feb 23, 2016 at 11:05 PM, Marcus Watts <mwatts@redhat.com> wrote:
> So, about that rocksdb thing.
>
> Rocksdb ships with 2 build systems:
>         cmake           - windows only
>         make based      - everything else
>
> The makefile is very "retro".  Um.  Let's just leave it there.
>
> The cmake part was more interesting; the main problem it had
> was it was *way* too windows specific.  Which is actually kinda
> hard to do, since that's just what cmake wasn't supposed to be.
>
> Building rocksdb (-g) takes about 1g of build tree space,
> and running "make check" on it takes about half an hour.
> I really don't want to slow down my ceph builds this way,
> so rather than make rocksdb with the rest of ceph, I would
> much rather it be packaged / installed separately.
>
> So I put together:
> a set of changes to build rocksdb with cmake on linux
> an rpm .spec file to build it for fedora.
>
> Rpms (source & amd64) can be found here:
>         http://people.redhat.com/mwatts/rocksdb/
> and a git repo with the cmake changes here,
>         https://github.com/mdw-at-linuxbox/rocksdb
>
> The cmake parts could certainly still use more work; I haven't
> tested this and it will probably need changes on anything other
> than x86_64, such as certainly any non-gcc/non-linux platform.
>
>                                         -Marcus Watts
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: ceph / rocksdb

[Allen Samuels]

I believe that Sage has made a lot of changes in the way that journaling is done. I would also vote to change the name to avoid any chance of accidentally reusing a local version.

Allen Samuels
Software Architect, Emerging Storage Solutions

2880 Junction Avenue, Milpitas, CA 95134
T: +1 408 801 7030| M: +1 408 780 6416
allen.samuels@SanDisk.com


-----Original Message-----
From: ceph-devel-owner@vger.kernel.org [mailto:ceph-devel-owner@vger.kernel.org] On Behalf Of Ken Dreyer
Sent: Wednesday, February 24, 2016 9:59 AM
To: Marcus Watts <mwatts@redhat.com>
Cc: ceph-devel <ceph-devel@vger.kernel.org>
Subject: Re: ceph / rocksdb

I'm really interested in getting our various bundled libraries into separate packages.

Does ceph's rocksdb have a lot of changes from rocksdb upstream? If so, I'm leaning towards packaging this as "ceph-rocksdb" until those changes are present in an upstream rocksdb release.

- Ken

On Tue, Feb 23, 2016 at 11:05 PM, Marcus Watts <mwatts@redhat.com> wrote:
> So, about that rocksdb thing.
>
> Rocksdb ships with 2 build systems:
>         cmake           - windows only
>         make based      - everything else
>
> The makefile is very "retro".  Um.  Let's just leave it there.
>
> The cmake part was more interesting; the main problem it had was it
> was *way* too windows specific.  Which is actually kinda hard to do,
> since that's just what cmake wasn't supposed to be.
>
> Building rocksdb (-g) takes about 1g of build tree space, and running
> "make check" on it takes about half an hour.
> I really don't want to slow down my ceph builds this way, so rather
> than make rocksdb with the rest of ceph, I would much rather it be
> packaged / installed separately.
>
> So I put together:
> a set of changes to build rocksdb with cmake on linux an rpm .spec
> file to build it for fedora.
>
> Rpms (source & amd64) can be found here:
>         http://people.redhat.com/mwatts/rocksdb/
> and a git repo with the cmake changes here,
>         https://github.com/mdw-at-linuxbox/rocksdb
>
> The cmake parts could certainly still use more work; I haven't tested
> this and it will probably need changes on anything other than x86_64,
> such as certainly any non-gcc/non-linux platform.
>
>                                         -Marcus Watts
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel"
> in the body of a message to majordomo@vger.kernel.org More majordomo
> info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@vger.kernel.org More majordomo info at  http://vger.kernel.org/majordomo-info.html
PLEASE NOTE: The information contained in this electronic mail message is intended only for the use of the designated recipient(s) named above. If the reader of this message is not the intended recipient, you are hereby notified that you have received this message in error and that any review, dissemination, distribution, or copying of this message is strictly prohibited. If you have received this communication in error, please notify the sender by telephone or e-mail (as shown above) immediately and destroy any and all copies of this message in your possession (whether hard copies or electronically stored copies).

Re: ceph / rocksdb

[Sage Weil]

On Wed, 24 Feb 2016, Ken Dreyer wrote:
> I'm really interested in getting our various bundled libraries into
> separate packages.
> 
> Does ceph's rocksdb have a lot of changes from rocksdb upstream? If
> so, I'm leaning towards packaging this as "ceph-rocksdb" until those
> changes are present in an upstream rocksdb release.

There are a few that aren't upstream yet, but they'll merge eventually. 
That said, I fully expect that we'll make other changes in the future that 
we'll want to build/test/ship before they go upstream... so a special 
package name probably makes sense.

sage


> 
> - Ken
> 
> On Tue, Feb 23, 2016 at 11:05 PM, Marcus Watts <mwatts@redhat.com> wrote:
> > So, about that rocksdb thing.
> >
> > Rocksdb ships with 2 build systems:
> >         cmake           - windows only
> >         make based      - everything else
> >
> > The makefile is very "retro".  Um.  Let's just leave it there.
> >
> > The cmake part was more interesting; the main problem it had
> > was it was *way* too windows specific.  Which is actually kinda
> > hard to do, since that's just what cmake wasn't supposed to be.
> >
> > Building rocksdb (-g) takes about 1g of build tree space,
> > and running "make check" on it takes about half an hour.
> > I really don't want to slow down my ceph builds this way,
> > so rather than make rocksdb with the rest of ceph, I would
> > much rather it be packaged / installed separately.
> >
> > So I put together:
> > a set of changes to build rocksdb with cmake on linux
> > an rpm .spec file to build it for fedora.
> >
> > Rpms (source & amd64) can be found here:
> >         http://people.redhat.com/mwatts/rocksdb/
> > and a git repo with the cmake changes here,
> >         https://github.com/mdw-at-linuxbox/rocksdb
> >
> > The cmake parts could certainly still use more work; I haven't
> > tested this and it will probably need changes on anything other
> > than x86_64, such as certainly any non-gcc/non-linux platform.
> >
> >                                         -Marcus Watts
> > --
> > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
> 

Re: ceph / rocksdb

[Marcus Watts]

On Wed, Feb 24, 2016 at 01:06:48PM -0500, Sage Weil wrote:
> On Wed, 24 Feb 2016, Ken Dreyer wrote:
> > I'm really interested in getting our various bundled libraries into
> > separate packages.
> > 
> > Does ceph's rocksdb have a lot of changes from rocksdb upstream? If
> > so, I'm leaning towards packaging this as "ceph-rocksdb" until those
> > changes are present in an upstream rocksdb release.
> 
> There are a few that aren't upstream yet, but they'll merge eventually. 
> That said, I fully expect that we'll make other changes in the future that 
> we'll want to build/test/ship before they go upstream... so a special 
> package name probably makes sense.
> 
> sage
> 
> 
> > 
> > - Ken
> > 
... and I'm eliding what I posted originally to conserve electrons.

When I looked at upstream rocksdb I couldn't find anything we had that
they didn't have.  I believe it should not be hard to work towards
a target of using the unmodified distribution version in production
environments.

Looks like:
* debian - has rocksdb (=4.1.0); appears to be very recent.  Has no
  current packages that use rocksdb.  There appear to be plans to use
  rocksdb with osquery.
* ubuntu.  Looks to be the identical state to debian, same maintainer.
* fedora.  No rocksdb currently.  I think we could be the maintainer here,
		if we want.

All this is going to be very distribution specific, involve maintainers,
upstream, &etc.  Still, this looks like it could be very tractable.

What we do locally with "build/test/ship" is another matter entirely.
I can see a world where the gitbuilders just install "our" rocksdb as
another build dependency for ceph.  Conceivably if we want to get fancy
those gitbuilders (or others?) could also be building rocksdb (presumably
from our repo), producing packages that after additional vetting (teuthology
etc.) can be made available to the ceph git builders.  The only real
complexity I can see is it might be useful sometimes to steer different
rocksdb packages towards different ceph branches.

					-Marcus Watts

Re: ceph / rocksdb

[Nathan Cutler]

On 02/24/2016 06:58 PM, Ken Dreyer wrote:
> I'm really interested in getting our various bundled libraries into
> separate packages.

+1 !

> Does ceph's rocksdb have a lot of changes from rocksdb upstream? If
> so, I'm leaning towards packaging this as "ceph-rocksdb" until those
> changes are present in an upstream rocksdb release.

How about "rocksdb-ceph" for the name? To me the first component 
(rocksdb) expresses what the package *is* (i.e. what you get when you 
install it) and the second component (ceph) expresses the "flavor".

And does this mean I now have a green light for "civetweb-ceph"? ;-)

All the submodules should be separate packages IMO.

Also the tool "ceph-detect-init" seems like it deserves an independent 
existence.

-- 
Nathan Cutler
Software Engineer Distributed Storage
SUSE LINUX, s.r.o.
Tel.: +420 284 084 037

Re: ceph / rocksdb

[Willem Jan Withagen]

On 25-2-2016 21:21, Nathan Cutler wrote:
> On 02/24/2016 06:58 PM, Ken Dreyer wrote:
>> I'm really interested in getting our various bundled libraries into
>> separate packages.
> 
> +1 !
> 
>> Does ceph's rocksdb have a lot of changes from rocksdb upstream? If
>> so, I'm leaning towards packaging this as "ceph-rocksdb" until those
>> changes are present in an upstream rocksdb release.
> 
> How about "rocksdb-ceph" for the name? To me the first component
> (rocksdb) expresses what the package *is* (i.e. what you get when you
> install it) and the second component (ceph) expresses the "flavor".
> 
> And does this mean I now have a green light for "civetweb-ceph"? ;-)
> 
> All the submodules should be separate packages IMO.
> 
> Also the tool "ceph-detect-init" seems like it deserves an independent
> existence.
> 

I know that this discussion has run for some time.
And when I build a fresh clone, I ended up with a more recent rocksdb
than that I'd liked....  (or actual Clang)

But 3 trivial fixes further I could again compile, so I've committed
those to the Rocksdb github, and they got accepted this morning.
Now the trick question here is:
	How do they end up in the src/rocksdb tree?

for convenience sake the hashes:
	131397f
	69baec6
	deba7f6

thanx,
--WjW
	

Re: ceph / rocksdb

[Nathan Cutler]

> I know that this discussion has run for some time.
> And when I build a fresh clone, I ended up with a more recent rocksdb
> than that I'd liked....  (or actual Clang)
>
> But 3 trivial fixes further I could again compile, so I've committed
> those to the Rocksdb github, and they got accepted this morning.
> Now the trick question here is:
> 	How do they end up in the src/rocksdb tree?
>

Since rocksdb is built from a git submodule, the answer lies in

https://github.com/ceph/ceph/blob/master/.gitmodules#L15-L18

Which to me looks like it just takes the latest upstream rocksdb (branch 
defaults to "master").

Nathan

Re: ceph / rocksdb

[Sage Weil]

On Mon, 6 Jun 2016, Willem Jan Withagen wrote:
> On 25-2-2016 21:21, Nathan Cutler wrote:
> > On 02/24/2016 06:58 PM, Ken Dreyer wrote:
> >> I'm really interested in getting our various bundled libraries into
> >> separate packages.
> > 
> > +1 !
> > 
> >> Does ceph's rocksdb have a lot of changes from rocksdb upstream? If
> >> so, I'm leaning towards packaging this as "ceph-rocksdb" until those
> >> changes are present in an upstream rocksdb release.
> > 
> > How about "rocksdb-ceph" for the name? To me the first component
> > (rocksdb) expresses what the package *is* (i.e. what you get when you
> > install it) and the second component (ceph) expresses the "flavor".
> > 
> > And does this mean I now have a green light for "civetweb-ceph"? ;-)
> > 
> > All the submodules should be separate packages IMO.
> > 
> > Also the tool "ceph-detect-init" seems like it deserves an independent
> > existence.
> > 
> 
> I know that this discussion has run for some time.
> And when I build a fresh clone, I ended up with a more recent rocksdb
> than that I'd liked....  (or actual Clang)
> 
> But 3 trivial fixes further I could again compile, so I've committed
> those to the Rocksdb github, and they got accepted this morning.
> Now the trick question here is:
> 	How do they end up in the src/rocksdb tree?
> 
> for convenience sake the hashes:
> 	131397f
> 	69baec6
> 	deba7f6

I've pull up rocksdb to include those commits (though it looks like their 
sha1s changed because they've been rebased).

sage

Re: ceph / rocksdb

[Willem Jan Withagen]

On 6-6-2016 14:26, Sage Weil wrote:
> On Mon, 6 Jun 2016, Willem Jan Withagen wrote:
>> On 25-2-2016 21:21, Nathan Cutler wrote:
>>> On 02/24/2016 06:58 PM, Ken Dreyer wrote:
>>>> I'm really interested in getting our various bundled libraries into
>>>> separate packages.
>>>
>>> +1 !
>>>
>>>> Does ceph's rocksdb have a lot of changes from rocksdb upstream? If
>>>> so, I'm leaning towards packaging this as "ceph-rocksdb" until those
>>>> changes are present in an upstream rocksdb release.
>>>
>>> How about "rocksdb-ceph" for the name? To me the first component
>>> (rocksdb) expresses what the package *is* (i.e. what you get when you
>>> install it) and the second component (ceph) expresses the "flavor".
>>>
>>> And does this mean I now have a green light for "civetweb-ceph"? ;-)
>>>
>>> All the submodules should be separate packages IMO.
>>>
>>> Also the tool "ceph-detect-init" seems like it deserves an independent
>>> existence.
>>>
>>
>> I know that this discussion has run for some time.
>> And when I build a fresh clone, I ended up with a more recent rocksdb
>> than that I'd liked....  (or actual Clang)
>>
>> But 3 trivial fixes further I could again compile, so I've committed
>> those to the Rocksdb github, and they got accepted this morning.
>> Now the trick question here is:
>> 	How do they end up in the src/rocksdb tree?
>>
>> for convenience sake the hashes:
>> 	131397f
>> 	69baec6
>> 	deba7f6
> 
> I've pull up rocksdb to include those commits (though it looks like their 
> sha1s changed because they've been rebased).

Thanx,

The inards of git are still less than transparent. So I'm glad you know
what to do.

Thanx,
--WjW