build fails on centos 5

build fails on centos 5

[Thomas Mueller]

hi 

downloaded 0.19.1 and tried to compile on centos 5.

compilation fails with this:


g++ -DHAVE_CONFIG_H -I.    -Wall -D_FILE_OFFSET_BITS=64 -D_REENTRANT -D_THREAD_SAFE -rdynamic -g -O2 -MT class_debug.o -MD -MP -MF .deps/class_debug.Tpo -c -o class_debug.o `test -f 'objclass/class_debug.cc' || echo './'`objclass/class_debug.cc
mv -f .deps/class_debug.Tpo .deps/class_debug.Po
g++ -DHAVE_CONFIG_H -I.    -Wall -D_FILE_OFFSET_BITS=64 -D_REENTRANT -D_THREAD_SAFE -rdynamic -g -O2 -MT class_api.o -MD -MP -MF .deps/class_api.Tpo -c -o class_api.o `test -f 'objclass/class_api.cc' || echo './'`objclass/class_api.cc
mv -f .deps/class_api.Tpo .deps/class_api.Po
/bin/sh ../libtool --tag=CXX   --mode=link g++ -Wall -D_FILE_OFFSET_BITS=64 -D_REENTRANT -D_THREAD_SAFE -rdynamic -g -O2   -o cosd cosd.o SimpleMessenger.o class_debug.o class_api.o libosd.a libos.a libcrush.a libcommon.a -ldl -lpthread -lm -lcrypto 
g++ -Wall -D_FILE_OFFSET_BITS=64 -D_REENTRANT -D_THREAD_SAFE -rdynamic -g -O2 -o cosd cosd.o SimpleMessenger.o class_debug.o class_api.o  libosd.a libos.a libcrush.a libcommon.a -ldl -lpthread -lm -lcrypto  
libos.a(FileStore.o): In function `FileStore::_write(coll_t, sobject_t const&, unsigned long, unsigned long, buffer::list const&)':
/usr/src/ceph-0.19.1/src/os/FileStore.cc:928: undefined reference to `sync_file_range'
libos.a(FileStore.o): In function `FileStore::flusher_entry()':
/usr/src/ceph-0.19.1/src/os/FileStore.cc:1108: undefined reference to `sync_file_range'
libos.a(FileJournal.o): In function `FileJournal::do_write(buffer::list&)':
/usr/src/ceph-0.19.1/src/os/FileJournal.cc:533: undefined reference to `sync_file_range'
/usr/src/ceph-0.19.1/src/os/FileJournal.cc:534: undefined reference to `sync_file_range'
/usr/src/ceph-0.19.1/src/os/FileJournal.cc:535: undefined reference to `sync_file_range'
libos.a(FileJournal.o):/usr/src/ceph-0.19.1/src/os/FileJournal.cc:536: more undefined references to `sync_file_range' follow
collect2: ld returned 1 exit status
make[1]: *** [cosd] Error 1
make[1]: Leaving directory `/usr/src/ceph-0.19.1/src'
make: *** [all] Error 2

could not find any hints with google on this error. 

centos package versions (maybe relevant, just guessing):
* gcc: 4.1.2
* glibc: 2.5
* fuse: 2.7.4
* boost: 1.33.1
* libedit: 2.11
* kernel (custom built, vanilla): 2.6.32.7

- Thomas



------------------------------------------------------------------------------
Download Intel® Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev

Re: build fails on centos 5

[Sage Weil]

Hi Thomas,

On Thu, 4 Mar 2010, Thomas Mueller wrote:
> /usr/src/ceph-0.19.1/src/os/FileStore.cc:928: undefined reference to `sync_file_range'
> libos.a(FileStore.o): In function `FileStore::flusher_entry()':

sync_file_range() appeared in 2.6.17.  It sounds like CentOS 5 was based 
on an older kernel.  Since your kernel is new, a recent glibc is probably 
all you need to get things going.

sage

------------------------------------------------------------------------------
Download Intel® Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev

Re: build fails on centos 5

[Thomas Mueller]

Am Thu, 04 Mar 2010 07:11:00 -0800 schrieb Sage Weil:

> Hi Thomas,
> 
> On Thu, 4 Mar 2010, Thomas Mueller wrote:
>> /usr/src/ceph-0.19.1/src/os/FileStore.cc:928: undefined reference to
>> `sync_file_range' libos.a(FileStore.o): In function
>> `FileStore::flusher_entry()':
> 
> sync_file_range() appeared in 2.6.17.  It sounds like CentOS 5 was based
> on an older kernel.  Since your kernel is new, a recent glibc is
> probably all you need to get things going.

original centos kernel is 2.6.18. anyhow, i found a RHEL bug (which 
centos is based on):

https://bugzilla.redhat.com/show_bug.cgi?id=518581

comment #1 from this bug:
It has only been added to glibc 2.6, and cannot be backported due to ABI
breakage.  You can always fall back to syscall(3).    

so, RHEL/CentOS 5 will never support this directly. 

what about implementing the mentioned "syscall(3)" workaround for RHEL/
CentOS?

- Thomas


------------------------------------------------------------------------------
Download Intel® Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev

Re: build fails on centos 5

[Sage Weil]

On Thu, 4 Mar 2010, Thomas Mueller wrote:
> Am Thu, 04 Mar 2010 07:11:00 -0800 schrieb Sage Weil:
> > Hi Thomas,
> > 
> > On Thu, 4 Mar 2010, Thomas Mueller wrote:
> >> /usr/src/ceph-0.19.1/src/os/FileStore.cc:928: undefined reference to
> >> `sync_file_range' libos.a(FileStore.o): In function
> >> `FileStore::flusher_entry()':
> > 
> > sync_file_range() appeared in 2.6.17.  It sounds like CentOS 5 was based
> > on an older kernel.  Since your kernel is new, a recent glibc is
> > probably all you need to get things going.
> 
> original centos kernel is 2.6.18. anyhow, i found a RHEL bug (which 
> centos is based on):
> 
> https://bugzilla.redhat.com/show_bug.cgi?id=518581
> 
> comment #1 from this bug:
> It has only been added to glibc 2.6, and cannot be backported due to ABI
> breakage.  You can always fall back to syscall(3).    
> 
> so, RHEL/CentOS 5 will never support this directly. 

Ah, makes sense.
 
> what about implementing the mentioned "syscall(3)" workaround for RHEL/
> CentOS?

I can add a check in configure.ac for sync_file_range(3) in the unstable 
branch.  In the meantime, the easiest thing would be to just replace 
those lines with a call to fdatasync() (or leave them off entirely in 
os/FileStore.cc, where it's purely an optimization).

sage



diff --git a/src/os/FileJournal.cc b/src/os/FileJournal.cc
index 639b7d7..e11e092 100644
--- a/src/os/FileJournal.cc
+++ b/src/os/FileJournal.cc
@@ -544,15 +544,18 @@ void FileJournal::do_write(bufferlist& bl)
     ::fsync(fd);
 #else
     if (is_bdev) {
-      if (split) {
+      ::fdatasync(fd);
+      /*if (split) {
 	::sync_file_range(fd, header.max_size - split, split, SYNC_FILE_RANGE_WAIT_BEFORE|SYNC_FILE_RANGE_WRITE);
 	::sync_file_range(fd, get_top(), bl.length() - split, SYNC_FILE_RANGE_WAIT_BEFORE|SYNC_FILE_RANGE_WRITE);
 	::sync_file_range(fd, header.max_size - split, split, SYNC_FILE_RANGE_WAIT_AFTER);
 	::sync_file_range(fd, get_top(), bl.length() - split, SYNC_FILE_RANGE_WAIT_AFTER);
+
       } else {
 	::sync_file_range(fd, write_pos, bl.length(),
 			  SYNC_FILE_RANGE_WAIT_BEFORE|SYNC_FILE_RANGE_WRITE|SYNC_FILE_RANGE_WAIT_AFTER);
       }
+	*/
     } else
       ::fdatasync(fd);
 #endif
diff --git a/src/os/FileStore.cc b/src/os/FileStore.cc
index 8dd7998..1f511c4 100644
--- a/src/os/FileStore.cc
+++ b/src/os/FileStore.cc
@@ -1327,8 +1327,8 @@ int FileStore::_write(coll_t cid, const sobject_t& oid,
 
     if (!g_conf.filestore_flusher ||
 	!queue_flusher(fd, offset, len)) {
-      if (g_conf.filestore_sync_flush)
-	::sync_file_range(fd, offset, len, SYNC_FILE_RANGE_WRITE);
+      /*if (g_conf.filestore_sync_flush)
+	::sync_file_range(fd, offset, len, SYNC_FILE_RANGE_WRITE);*/
       ::close(fd);
     }
     r = did;
@@ -1508,7 +1508,7 @@ void FileStore::flusher_entry()
 	q.pop_front();
 	if (!stop && ep == sync_epoch) {
 	  dout(10) << "flusher_entry flushing+closing " << fd << " ep " << ep << dendl;
-	  ::sync_file_range(fd, off, len, SYNC_FILE_RANGE_WRITE);
+	  //::sync_file_range(fd, off, len, SYNC_FILE_RANGE_WRITE);
 	} else 
 	  dout(10) << "flusher_entry JUST closing " << fd << " (stop=" << stop << ", ep=" << ep
 		   << ", sync_epoch=" << sync_epoch << ")" << dendl;


------------------------------------------------------------------------------
Download Intel&#174; Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev