building Ceph on various architectures

building Ceph on various architectures

[Laszlo Boszormenyi]

Hi List,

Being the Debian packager of Ceph, I ask for help about building 0.24.3
on armel. It fails with:
g++ -DHAVE_CONFIG_H -I.    -Wall -D__CEPH__ -D_FILE_OFFSET_BITS=64
-D_REENTRANT -D_THREAD_SAFE -rdynamic -g -O2 -MT AuthAuthorizeHandler.o
-MD -MP -MF .deps/AuthAuthorizeHandler.Tpo -c -o AuthAuthorizeHandler.o
`test -f 'auth/AuthAuthorizeHandler.cc' || echo
'./'`auth/AuthAuthorizeHandler.cc
In file included from ./include/buffer.h:55,
                 from ./include/encoding.h:20,
                 from ./include/object.h:29,
                 from ./include/types.h:64,
                 from auth/Crypto.h:17,
                 from auth/Auth.h:18,
                 from auth/AuthAuthorizeHandler.cc:1:
./include/atomic.h: In member function 'void ceph::atomic_t::inc()':
./include/atomic.h:33: error: 'AO_fetch_and_add1' was not declared in this scope
./include/atomic.h: In member function 'size_t ceph::atomic_t::dec()':
./include/atomic.h:36: error: 'AO_fetch_and_sub1_write' was not declared in this scope
./include/atomic.h: In member function 'void ceph::atomic_t::add(size_t)':
./include/atomic.h:39: error: 'AO_fetch_and_add' was not declared in this scope
./include/atomic.h: In member function 'void ceph::atomic_t::sub(int)':
./include/atomic.h:43: error: 'AO_fetch_and_add_write' was not declared in this scope
make[3]: *** [AuthAuthorizeHandler.o] Error 1

You can read the full build log on the web[1].

It also fails on s390, on the same file but with a different error
message:
g++ -DHAVE_CONFIG_H -I.    -Wall -D__CEPH__ -D_FILE_OFFSET_BITS=64 -D_REENTRANT -D_THREAD_SAFE -rdynamic -g -O2 -MT AuthAuthorizeHandler.o -MD -MP -MF .deps/AuthAuthorizeHandler.Tpo -c -o AuthAuthorizeHandler.o `test -f 'auth/AuthAuthorizeHandler.cc' || echo './'`auth/AuthAuthorizeHandler.cc
In file included from ./msg/msg_types.h:19,
                 from auth/Auth.h:19,
                 from auth/AuthAuthorizeHandler.cc:1:
./include/blobhash.h: In member function 'size_t blobhash::operator()(const char*, unsigned int)':
./include/blobhash.h:42: error: no match for call to '(rjhash<long unsigned int>) (size_t&)'
make[3]: *** [AuthAuthorizeHandler.o] Error 1

The full building log can be found on the same location[2]. I would be
grateful if someone with more knowledge of Ceph internals can look into
them.

Regards,
Laszlo/GCS
[1] https://buildd.debian.org/fetch.cgi?pkg=ceph&arch=armel&ver=0.24.3-2&stamp=1298712439&file=log&as=raw
[2] https://buildd.debian.org/fetch.cgi?pkg=ceph&arch=s390&ver=0.24.3-2&stamp=1298712185&file=log&as=raw

Re: building Ceph on various architectures

[Gregory Farnum]

On Saturday, February 26, 2011 at 3:41 AM, Laszlo Boszormenyi wrote:
Hi List,
> 
> Being the Debian packager of Ceph, I ask for help about building 0.24.3
> on armel. It fails with:
> g++ -DHAVE_CONFIG_H -I. -Wall -D__CEPH__ -D_FILE_OFFSET_BITS=64
> -D_REENTRANT -D_THREAD_SAFE -rdynamic -g -O2 -MT AuthAuthorizeHandler.o
> -MD -MP -MF .deps/AuthAuthorizeHandler.Tpo -c -o AuthAuthorizeHandler.o
> `test -f 'auth/AuthAuthorizeHandler.cc' || echo
> './'`auth/AuthAuthorizeHandler.cc
> In file included from ./include/buffer.h:55,
>  from ./include/encoding.h:20,
>  from ./include/object.h:29,
>  from ./include/types.h:64,
>  from auth/Crypto.h:17,
>  from auth/Auth.h:18,
>  from auth/AuthAuthorizeHandler.cc:1:
> ./include/atomic.h: In member function 'void ceph::atomic_t::inc()':
> ./include/atomic.h:33: error: 'AO_fetch_and_add1' was not declared in this scope
> ./include/atomic.h: In member function 'size_t ceph::atomic_t::dec()':
> ./include/atomic.h:36: error: 'AO_fetch_and_sub1_write' was not declared in this scope
> ./include/atomic.h: In member function 'void ceph::atomic_t::add(size_t)':
> ./include/atomic.h:39: error: 'AO_fetch_and_add' was not declared in this scope
> ./include/atomic.h: In member function 'void ceph::atomic_t::sub(int)':
> ./include/atomic.h:43: error: 'AO_fetch_and_add_write' was not declared in this scope
> make[3]: *** [AuthAuthorizeHandler.o] Error 1

This looks like an error that came up (and got fixed) on Ubuntu several months ago:
https://bugs.launchpad.net/ubuntu/+source/ceph/+bug/635388

> You can read the full build log on the web[1].
> 
> It also fails on s390, on the same file but with a different error
> message:
> g++ -DHAVE_CONFIG_H -I. -Wall -D__CEPH__ -D_FILE_OFFSET_BITS=64 -D_REENTRANT -D_THREAD_SAFE -rdynamic -g -O2 -MT AuthAuthorizeHandler.o -MD -MP -MF .deps/AuthAuthorizeHandler.Tpo -c -o AuthAuthorizeHandler.o `test -f 'auth/AuthAuthorizeHandler.cc' || echo './'`auth/AuthAuthorizeHandler.cc
> In file included from ./msg/msg_types.h:19,
>  from auth/Auth.h:19,
>  from auth/AuthAuthorizeHandler.cc:1:
> ./include/blobhash.h: In member function 'size_t blobhash::operator()(const char*, unsigned int)':
> ./include/blobhash.h:42: error: no match for call to '(rjhash<long unsigned int>) (size_t&)'
> make[3]: *** [AuthAuthorizeHandler.o] Error 1

This isn't one I'm familiar with -- I'll look into it on Monday if nobody's got it before then. :)
-Greg

Re: building Ceph on various architectures

[Laszlo Boszormenyi]

On Sat, 2011-02-26 at 10:28 -0800, Gregory Farnum wrote:
> On Saturday, February 26, 2011 at 3:41 AM, Laszlo Boszormenyi wrote:
> This looks like an error that came up (and got fixed) on Ubuntu several months ago:
> https://bugs.launchpad.net/ubuntu/+source/ceph/+bug/635388
 Yes, it looks similar. But please note two things. The counterpart of
#635388 in Ubuntu is #597193 [1] in Debian. It was fixed on the 20th of
September, last year. Also, that bugs was:
'AO_fetch_and_sub1_write' was not declared in this scope
Compared to this one:
'AO_fetch_and_add_write' was not declared in this scope

I don't say it completely different, but doesn't look the exact same.

[ about the build failure on s390 ]
> This isn't one I'm familiar with -- I'll look into it on Monday if nobody's got it before then. :)
 OK, let it be. I may try to look into it, but please don't count on me.

Laszlo/GCS
[1] http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=597193

Re: building Ceph on various architectures

[Gregory Farnum]

On Saturday, February 26, 2011 at 11:40 AM, Laszlo Boszormenyi wrote: 
> On Sat, 2011-02-26 at 10:28 -0800, Gregory Farnum wrote:
> > This looks like an error that came up (and got fixed) on Ubuntu several months ago:
> > https://bugs.launchpad.net/ubuntu/+source/ceph/+bug/635388
>  Yes, it looks similar. But please note two things. The counterpart of
> #635388 in Ubuntu is #597193 [1] in Debian. It was fixed on the 20th of
> September, last year. Also, that bugs was:
> 'AO_fetch_and_sub1_write' was not declared in this scope
> Compared to this one:
> 'AO_fetch_and_add_write' was not declared in this scope
> 
> I don't say it completely different, but doesn't look the exact same.

Ah, I didn't notice that -- I'll look into it more closely! 

Re: building Ceph on various architectures

[Gregory Farnum]

On Saturday, February 26, 2011 at 11:52 AM, Gregory Farnum wrote:
On Saturday, February 26, 2011 at 11:40 AM, Laszlo Boszormenyi wrote: 
> > On Sat, 2011-02-26 at 10:28 -0800, Gregory Farnum wrote:
> > > This looks like an error that came up (and got fixed) on Ubuntu several months ago:
> > > https://bugs.launchpad.net/ubuntu/+source/ceph/+bug/635388
> >  Yes, it looks similar. But please note two things. The counterpart of
> > #635388 in Ubuntu is #597193 [1] in Debian. It was fixed on the 20th of
> > September, last year. Also, that bugs was:
> > 'AO_fetch_and_sub1_write' was not declared in this scope
> > Compared to this one:
> > 'AO_fetch_and_add_write' was not declared in this scope
> > 
> > I don't say it completely different, but doesn't look the exact same.
> 
> Ah, I didn't notice that -- I'll look into it more closely! 
>  I looked at this a bit more and there's nothing really obvious jumping out at me, but I do have a few observations/questions: 
1) It looks to me like this is the same error of definitions as previously. AO_fetch_and_sub1_write, AO_fetch_and_add_write, etc are in both bugs. (This is to be expected, since the most probable cause of atomic-ops issues are problems with definition chaining.)
2) I notice that this is in the compile output: "Kernel: Linux 2.6.31-rc9 armel (armv5tel)"
Does that mean it's building on an arm v5 machine?
I wonder if maybe there's an issue there -- my knowledge of the ARM architecture is extremely limited (basically just tracking issues like this down) but from what I've read there were some changes which could account for a bug like this. Notably, atomic-ops for arm depends on the "__arm__" preprocessor variable being defined, which I'd expect to be defined for any version but maybe isn't.

So, does anybody out there have more experience with ARM or the atomic-ops package? I can email the atomic-ops people but I'd like to make sure we aren't doing something odd ourselves!
-Greg

Re: building Ceph on various architectures

[Laszlo Boszormenyi]

On Sun, 2011-02-27 at 19:42 -0800, Gregory Farnum wrote:
> On Saturday, February 26, 2011 at 11:52 AM, Gregory Farnum wrote:
> On Saturday, February 26, 2011 at 11:40 AM, Laszlo Boszormenyi wrote: 
> > > On Sat, 2011-02-26 at 10:28 -0800, Gregory Farnum wrote:
> > > > This looks like an error that came up (and got fixed) on Ubuntu several months ago:
> > > > https://bugs.launchpad.net/ubuntu/+source/ceph/+bug/635388
> > >  Yes, it looks similar. But please note two things. The counterpart of
> > > #635388 in Ubuntu is #597193 [1] in Debian. It was fixed on the 20th of
> > > September, last year. Also, that bugs was:
> > > 'AO_fetch_and_sub1_write' was not declared in this scope
> > > Compared to this one:
> > > 'AO_fetch_and_add_write' was not declared in this scope
> > > 
> > > I don't say it completely different, but doesn't look the exact same.
> > 
> > Ah, I didn't notice that -- I'll look into it more closely! 
> >  I looked at this a bit more and there's nothing really obvious jumping out at me, but I do have a few observations/questions: 
> 1) It looks to me like this is the same error of definitions as previously. AO_fetch_and_sub1_write, AO_fetch_and_add_write, etc are in both bugs. (This is to be expected, since the most probable cause of atomic-ops issues are problems with definition chaining.)
 Well, can be. As I see, the original bug was fixed in Debian as
well[1].

> 2) I notice that this is in the compile output: "Kernel: Linux 2.6.31-rc9 armel (armv5tel)"
> Does that mean it's building on an arm v5 machine?
> I wonder if maybe there's an issue there -- my knowledge of the ARM architecture is extremely limited (basically just tracking issues like this down) but from what I've read there were some changes which could account for a bug like this. Notably, atomic-ops for arm depends on the "__arm__" preprocessor variable being defined, which I'd expect to be defined for any version but maybe isn't.
 I have to run and will have time only in the afternoon (CET). Until
then I can give you only pointers. The email list of ARM porters:
http://lists.debian.org/debian-arm/ . It's not moderated, so you can
post there. IRC can be faster, you can find #debian-arm on
irc.debian.org .

HTH,
Laszlo/GCS
[1] http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=597193

Re: building Ceph on various architectures

[Tommi Virtanen]

On Sat, Feb 26, 2011 at 12:41:18PM +0100, Laszlo Boszormenyi wrote:
> It also fails on s390, on the same file but with a different error
> message:
> g++ -DHAVE_CONFIG_H -I.    -Wall -D__CEPH__ -D_FILE_OFFSET_BITS=64 -D_REENTRANT -D_THREAD_SAFE -rdynamic -g -O2 -MT AuthAuthorizeHandler.o -MD -MP -MF .deps/AuthAuthorizeHandler.Tpo -c -o AuthAuthorizeHandler.o `test -f 'auth/AuthAuthorizeHandler.cc' || echo './'`auth/AuthAuthorizeHandler.cc
> In file included from ./msg/msg_types.h:19,
>                  from auth/Auth.h:19,
>                  from auth/AuthAuthorizeHandler.cc:1:
> ./include/blobhash.h: In member function 'size_t blobhash::operator()(const char*, unsigned int)':
> ./include/blobhash.h:42: error: no match for call to '(rjhash<long unsigned int>) (size_t&)'
> make[3]: *** [AuthAuthorizeHandler.o] Error 1

This is ticket http://tracker.newdream.net/issues/837 for us.

Can you try branch tv-blobhash-837 in ceph.git and see if it compiles
right?  I don't have convenient access to a zSeries box / Hercules.

-- 
:(){ :|:&};: