Re: [ceph-commit] [ceph/ceph] e6a154: osx: compile on OSX

Re: [ceph-commit] [ceph/ceph] e6a154: osx: compile on OSX

[Gregory Farnum]

Oooh, very nice! Do you have a list of the dependencies that you actually needed to install?

Apart from breaking this up into smaller patches, we'll also want to reformat some of it. Rather than sticking an #if APPLE on top of every spin lock, we should have utility functions that do this for us. ;)

Also, we should be able to find libatomic_ops for OS X (its parent project works under OS X), and we can use that to construct a spin lock if we think it'll be useful. I'm not too sure how effective its muteness are at spinlock-y workloads.
-Greg


On Sunday, December 9, 2012 at 9:41 AM, GitHub wrote:

> Branch: refs/heads/wip-osx
> Home: https://github.com/ceph/ceph
> Commit: e6a1544d42737b1aacf12210c0818200bb6d29aa
> https://github.com/ceph/ceph/commit/e6a1544d42737b1aacf12210c0818200bb6d29aa
> Author: Noah Watkins <nwatkins@martini.local>
> Date: 2012-12-09 (Sun, 09 Dec 2012)
> 
> Changed paths:
> M autogen.sh (http://autogen.sh)
> M configure.ac (http://configure.ac)
> A m4/ax_c_var_func.m4
> A m4/ax_cxx_static_cast.m4
> M src/Makefile.am (http://Makefile.am)
> M src/client/Client.cc (http://Client.cc)
> M src/client/fuse_ll.cc (http://fuse_ll.cc)
> M src/client/ioctl.h
> M src/common/OutputDataSocket.cc (http://OutputDataSocket.cc)
> M src/common/admin_socket.cc (http://admin_socket.cc)
> M src/common/blkdev.cc (http://blkdev.cc)
> M src/common/ceph_context.cc (http://ceph_context.cc)
> M src/common/ceph_context.h
> M src/common/code_environment.cc (http://code_environment.cc)
> A src/common/cpipe.c
> A src/common/cpipe.h
> M src/common/ipaddr.cc (http://ipaddr.cc)
> M src/common/lockdep.cc (http://lockdep.cc)
> R src/common/pipe.c
> R src/common/pipe.h
> M src/common/sctp_crc32.c
> M src/common/sync_filesystem.h
> M src/common/xattr.c
> M src/crush/crush.h
> M src/crush/hash.h
> M src/include/assert.h
> M src/include/atomic.h
> M src/include/buffer.h
> M src/include/byteorder.h
> M src/include/cephfs/libcephfs.h
> M src/include/compat.h
> M src/include/inttypes.h
> M src/include/linux_fiemap.h
> M src/include/msgr.h
> M src/include/types.h
> M src/leveldb
> M src/log/Log.cc (http://Log.cc)
> M src/log/Log.h
> M src/mon/LogMonitor.cc (http://LogMonitor.cc)
> M src/msg/Pipe.cc (http://Pipe.cc)
> M src/msg/SimpleMessenger.cc (http://SimpleMessenger.cc)
> M src/msg/SimpleMessenger.h
> M src/os/FileJournal.cc (http://FileJournal.cc)
> M src/os/FileStore.cc (http://FileStore.cc)
> M src/osd/OSD.cc (http://OSD.cc)
> M src/rados.cc (http://rados.cc)
> M src/test/system/cross_process_sem.cc (http://cross_process_sem.cc)
> M src/test/system/systest_runnable.cc (http://systest_runnable.cc)
> M src/tools/ceph.cc (http://ceph.cc)
> M src/tools/common.cc (http://common.cc)
> 
> Log Message:
> -----------
> osx: compile on OSX
> 
> This patch allows the full tree to build on OSX, but currently there are
> a lot of segfaults, inconsistent uses of DARWIN/__APPLE__ defines, and
> some of the semantic changes are likely wrong. We'll need to split this
> up into a much longer patch series, and can probably start with the
> minimal change set needed to make fuse work.
> 
> Use homebrew installed in its default location (/usr/local/) to install
> dependencies, and configure with --without-libatomic-ops --with-libaio.
> 
> _______________________________________________
> Ceph-commit mailing list
> Ceph-commit@lists.ceph.newdream.net (mailto:Ceph-commit@lists.ceph.newdream.net)
> http://lists.ceph.newdream.net/listinfo.cgi/ceph-commit-ceph.newdream.net

Re: [ceph-commit] [ceph/ceph] e6a154: osx: compile on OSX

[Noah Watkins]

On Sun, Dec 9, 2012 at 10:05 AM, Gregory Farnum <greg@inktank.com> wrote:
> Oooh, very nice! Do you have a list of the dependencies that you actually needed to install?

I can put that together. They were boost, gperf, fuse4x, cryptopp. I
think that might have been it.

> Apart from breaking this up into smaller patches, we'll also want to reformat some of it. Rather than sticking an #if APPLE on top of every spin lock, we should have utility functions that do this for us. ;)

Definitely. OSX has spinlock implementations for user space, but it's
going to take some reading. For example, spinlocks in Ceph are
initialized for shared memory, rather than the default private. It
isn't clear from documentation what the semantics are of OSX
spinlocks, nor is it clear if the shared memory attribute is needed.

> Also, we should be able to find libatomic_ops for OS X (its parent project works under OS X), and we can use that to construct a spin lock if we think it'll be useful. I'm not too sure how effective its muteness are at spinlock-y workloads.

This patch set uses the OSX atomic inc/dec ops, rather than spinlocks.

Another fun fact:

msg/Pipe.cc and common/pipe.c are compiled into libcommon_la-Pipe.o
and libcommon_la-pipe.o, but HFS+ is case-insensitive by default.
Result is duplicate symbols. That took a while to figure out :P

-Noah

Re: [ceph-commit] [ceph/ceph] e6a154: osx: compile on OSX

[Jimmy Tang]

On 9 Dec 2012, at 18:22, Noah Watkins wrote:

> On Sun, Dec 9, 2012 at 10:05 AM, Gregory Farnum <greg@inktank.com> wrote:
>> Oooh, very nice! Do you have a list of the dependencies that you actually needed to install?
> 
> I can put that together. They were boost, gperf, fuse4x, cryptopp. I
> think that might have been it.
> 

Is libaio really needed to build ceph-fuse? I use macports on my system and the last time I tried to make a change set to let ceph/ceph-fuse build on my laptop failed as I didn't have libaio, though I could just write a port for it.

>> Apart from breaking this up into smaller patches, we'll also want to reformat some of it. Rather than sticking an #if APPLE on top of every spin lock, we should have utility functions that do this for us. ;)
> 
> Definitely. OSX has spinlock implementations for user space, but it's
> going to take some reading. For example, spinlocks in Ceph are
> initialized for shared memory, rather than the default private. It
> isn't clear from documentation what the semantics are of OSX
> spinlocks, nor is it clear if the shared memory attribute is needed.
> 
>> Also, we should be able to find libatomic_ops for OS X (its parent project works under OS X), and we can use that to construct a spin lock if we think it'll be useful. I'm not too sure how effective its muteness are at spinlock-y workloads.
> 
> This patch set uses the OSX atomic inc/dec ops, rather than spinlocks.
> 
> Another fun fact:
> 
> msg/Pipe.cc and common/pipe.c are compiled into libcommon_la-Pipe.o
> and libcommon_la-pipe.o, but HFS+ is case-insensitive by default.
> Result is duplicate symbols. That took a while to figure out :P
> 

good catch, that might explain why my last look at ceph on osx failed so miserably.


Jimmy.

--
Senior Software Engineer, Digital Repository of Ireland (DRI)
Trinity Centre for High Performance Computing,
Lloyd Building, Trinity College Dublin, Dublin 2, Ireland.
http://www.tchpc.tcd.ie/ | jtang@tchpc.tcd.ie
Tel: +353-1-896-3847

Re: [ceph-commit] [ceph/ceph] e6a154: osx: compile on OSX

[Sage Weil]

On Mon, 10 Dec 2012, Jimmy Tang wrote:
> 
> On 9 Dec 2012, at 18:22, Noah Watkins wrote:
> 
> > On Sun, Dec 9, 2012 at 10:05 AM, Gregory Farnum <greg@inktank.com> wrote:
> >> Oooh, very nice! Do you have a list of the dependencies that you actually needed to install?
> > 
> > I can put that together. They were boost, gperf, fuse4x, cryptopp. I
> > think that might have been it.
> > 
> 
> Is libaio really needed to build ceph-fuse? I use macports on my system 
> and the last time I tried to make a change set to let ceph/ceph-fuse 
> build on my laptop failed as I didn't have libaio, though I could just 
> write a port for it.

libaio is only used by ceph-osd.  Not needed by fuse.

sage


> 
> >> Apart from breaking this up into smaller patches, we'll also want to reformat some of it. Rather than sticking an #if APPLE on top of every spin lock, we should have utility functions that do this for us. ;)
> > 
> > Definitely. OSX has spinlock implementations for user space, but it's
> > going to take some reading. For example, spinlocks in Ceph are
> > initialized for shared memory, rather than the default private. It
> > isn't clear from documentation what the semantics are of OSX
> > spinlocks, nor is it clear if the shared memory attribute is needed.
> > 
> >> Also, we should be able to find libatomic_ops for OS X (its parent project works under OS X), and we can use that to construct a spin lock if we think it'll be useful. I'm not too sure how effective its muteness are at spinlock-y workloads.
> > 
> > This patch set uses the OSX atomic inc/dec ops, rather than spinlocks.
> > 
> > Another fun fact:
> > 
> > msg/Pipe.cc and common/pipe.c are compiled into libcommon_la-Pipe.o
> > and libcommon_la-pipe.o, but HFS+ is case-insensitive by default.
> > Result is duplicate symbols. That took a while to figure out :P
> > 
> 
> good catch, that might explain why my last look at ceph on osx failed so miserably.
> 
> 
> Jimmy.
> 
> --
> Senior Software Engineer, Digital Repository of Ireland (DRI)
> Trinity Centre for High Performance Computing,
> Lloyd Building, Trinity College Dublin, Dublin 2, Ireland.
> http://www.tchpc.tcd.ie/ | jtang@tchpc.tcd.ie
> Tel: +353-1-896-3847
> 
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
> 

Re: [ceph-commit] [ceph/ceph] e6a154: osx: compile on OSX

[Sam Lang]

On 12/10/2012 07:01 AM, Sage Weil wrote:
> On Mon, 10 Dec 2012, Jimmy Tang wrote:
>>
>> On 9 Dec 2012, at 18:22, Noah Watkins wrote:
>>
>>> On Sun, Dec 9, 2012 at 10:05 AM, Gregory Farnum <greg@inktank.com> wrote:
>>>> Oooh, very nice! Do you have a list of the dependencies that you actually needed to install?
>>>
>>> I can put that together. They were boost, gperf, fuse4x, cryptopp. I
>>> think that might have been it.
>>>
>>
>> Is libaio really needed to build ceph-fuse? I use macports on my system
>> and the last time I tried to make a change set to let ceph/ceph-fuse
>> build on my laptop failed as I didn't have libaio, though I could just
>> write a port for it.
>
> libaio is only used by ceph-osd.  Not needed by fuse.

An alternative on OSX could be aio-lite: 
https://trac.mcs.anl.gov/projects/aio-lite

It might perform better on linux as well because of the request 
serialization there, although that library was implemented a few years 
ago, and the linux implementation may have improved significantly since 
then.  It also wouldn't be hard to do something similar with ceph thread 
structures instead of depending on an external library like this one.

-sam

>
> sage
>
>
>>
>>>> Apart from breaking this up into smaller patches, we'll also want to reformat some of it. Rather than sticking an #if APPLE on top of every spin lock, we should have utility functions that do this for us. ;)
>>>
>>> Definitely. OSX has spinlock implementations for user space, but it's
>>> going to take some reading. For example, spinlocks in Ceph are
>>> initialized for shared memory, rather than the default private. It
>>> isn't clear from documentation what the semantics are of OSX
>>> spinlocks, nor is it clear if the shared memory attribute is needed.
>>>
>>>> Also, we should be able to find libatomic_ops for OS X (its parent project works under OS X), and we can use that to construct a spin lock if we think it'll be useful. I'm not too sure how effective its muteness are at spinlock-y workloads.
>>>
>>> This patch set uses the OSX atomic inc/dec ops, rather than spinlocks.
>>>
>>> Another fun fact:
>>>
>>> msg/Pipe.cc and common/pipe.c are compiled into libcommon_la-Pipe.o
>>> and libcommon_la-pipe.o, but HFS+ is case-insensitive by default.
>>> Result is duplicate symbols. That took a while to figure out :P
>>>
>>
>> good catch, that might explain why my last look at ceph on osx failed so miserably.
>>
>>
>> Jimmy.
>>
>> --
>> Senior Software Engineer, Digital Repository of Ireland (DRI)
>> Trinity Centre for High Performance Computing,
>> Lloyd Building, Trinity College Dublin, Dublin 2, Ireland.
>> http://www.tchpc.tcd.ie/ | jtang@tchpc.tcd.ie
>> Tel: +353-1-896-3847
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
>>
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Re: [ceph-commit] [ceph/ceph] e6a154: osx: compile on OSX

[Christoph Hellwig]

On Mon, Dec 10, 2012 at 07:11:44AM -1000, Sam Lang wrote:
> >>Is libaio really needed to build ceph-fuse? I use macports on my system
> >>and the last time I tried to make a change set to let ceph/ceph-fuse
> >>build on my laptop failed as I didn't have libaio, though I could just
> >>write a port for it.
> >
> >libaio is only used by ceph-osd.  Not needed by fuse.
> 
> An alternative on OSX could be aio-lite:
> https://trac.mcs.anl.gov/projects/aio-lite
> 
> It might perform better on linux as well because of the request
> serialization there, although that library was implemented a few
> years ago, and the linux implementation may have improved
> significantly since then.  It also wouldn't be hard to do something
> similar with ceph thread structures instead of depending on an
> external library like this one.

libaio is the library the provides the kernel AIO API, which is very
different from Posix AIO.