Re: [Patch] chunkd: use port xxx82 to build

Re: [Patch] chunkd: use port xxx82 to build

[Rick Peralta]

Hi Jeff,

Here are the notes on the endian notes.  On first blush it looks like the basic problems are related to I/O and structure alignment.  Correcting the structure (in the compiler) might be enough (if is supports endian pragmas or some such.  The traditional way is to hard code endian conversions of the data before it goes out over the network and converting back on return.  There are macros in the networking code that do the core byte swaping function.  I was looking from net_write() & net_read() to see where they were called and what was passed to them.  struct chunksrv_req & struct chunksrv_resp_get seem to be the primary structures.  Where to optimally put the changes is less clear.

I cobbled on a VM to get the PPC vm working.  It seemed to boot, but needed more configuration than I had time to dope out.  I was cool to see a mini-laptop (using the atom processor) run windows (yetch), support a VM running Fedora, support a VM running PPC code!  I'm thinking a DVD could still run in the background, on  an external monitor...

 - Rick

>From: Jeff Garzik <jeff@garzik.org>
>Sent: Aug 4, 2009 9:26 PM
>To: Pete Zaitcev <zaitcev@redhat.com>
>Cc: Project Hail List <hail-devel@vger.kernel.org>, Rick Peralta <fbp@tiac.net>
>Subject: Re: [Patch] chunkd: use port xxx82 to build
>
>Pete Zaitcev wrote:
>> On Tue, 04 Aug 2009 21:00:36 -0400, Jeff Garzik <jeff@garzik.org> wrote:
>> 
>>> Speaking of chunkd...
>>>
>>> I don't think I will have time this week to fix chunkd's failure on big 
>>> endian machines.  If you (or anyone else listening...) have an 
>>> opportunity to look into that, that would be most helpful.
>> 
>> This looks like an ideal opportunity for Rick to read the code
>> with a specific goal in mind, if he doesn't mind me saying so.
>
>CC'd :)
>
>
>> Do we have a bug number for it?
>
>https://bugzilla.redhat.com/show_bug.cgi?id=514651
>
>	Jeff
>
>
>

Re: [Patch] chunkd: use port xxx82 to build

[Jeff Garzik]

Rick Peralta wrote:
> Hi Jeff,
> 
> Here are the notes on the endian notes.  On first blush it looks like the basic problems are related to I/O and structure alignment.  Correcting the structure (in the compiler) might be enough (if is supports endian pragmas or some such.  The traditional way is to hard code endian conversions of the data before it goes out over the network and converting back on return.  There are macros in the networking code that do the core byte swaping function.  I was looking from net_write() & net_read() to see where they were called and what was passed to them.  struct chunksrv_req & struct chunksrv_resp_get seem to be the primary structures.  Where to optimally put the changes is less clear.

In struct chunksrv_req, only one member has the possibility of endian 
problems: data_len.  And it appears to me that this is already properly 
converted:
> [jgarzik@bd chunkd]$ grep -w data_len */*.[ch]
> include/chunk_msg.h:	uint64_t		data_len;		/* len of addn'l data */
> lib/chunkdc.c:	*plen = GUINT64_FROM_LE(resp.req.data_len);
> lib/chunkdc.c:	req.data_len = GUINT64_TO_LE(content_len);
> lib/chunkdc.c:	req.data_len = GUINT64_TO_LE(cont_len);
> lib/chunkdc.c:	content_len = GUINT64_FROM_LE(resp.req.data_len);
> server/object.c:	uint64_t content_len = GUINT64_FROM_LE(cli->creq.data_len);
> server/object.c:	resp->req.data_len = GUINT32_TO_LE(obj->size);
> server/server.c:	resp->data_len = GUINT64_TO_LE(content_len);
> server/server.c:		       (long long) GUINT64_FROM_LE(req->data_len));

In struct chunksrv_resp_get, the only possibility of endian problem with 
is the 'mtime' struct member, which is only referenced once in
server/object.c (the other uses are native-endian internal variables):
> [jgarzik@bd chunkd]$ grep -w mtime */*.[ch]
> include/chunk_msg.h:	uint64_t		mtime;
> lib/chunkdc.c:	if (!net_read(stc, &resp.mtime, sizeof(resp) - sizeof(resp.req)))
> server/be-fs.c:	obj->bo.mtime = st.st_mtime;
> server/be-fs.c:			ve->mtime = st.st_mtime;
> server/chunkd.h:	time_t			mtime;
> server/chunkd.h:	time_t			mtime;		/* obj last-mod time */
> server/object.c:	resp->mtime = GUINT64_TO_LE(obj->mtime);
> server/server.c:			 time2str(timestr, ve->mtime),


What are the "basic problems of I/O and structure alignment"?


> I cobbled on a VM to get the PPC vm working.  It seemed to boot, but needed more configuration than I had time to dope out.  I was cool to see a mini-laptop (using the atom processor) run windows (yetch), support a VM running Fedora, support a VM running PPC code!  I'm thinking a DVD could still run in the background, on  an external monitor...

Great!

	Jeff

Re: [Patch] chunkd: use port xxx82 to build

[Pete Zaitcev]

On Fri, 7 Aug 2009 19:36:28 -0400 (EDT), Rick Peralta <fbp@tiac.net> wrote:

> I was looking from net_write() & net_read() to see where they were
> called and what was passed to them.  struct chunksrv_req & struct
> chunksrv_resp_get seem to be the primary structures.

I didn't know how serious you were about it, so I went ahead and
looked at the endian stuff. The immediate issue - the hang of
basic-object - was addressed with the following fix:

diff --git a/server/object.c b/server/object.c
index 4eb62f7..6ea1500 100644
--- a/server/object.c
+++ b/server/object.c
@@ -290,7 +290,7 @@ bool object_get(struct client *cli, bool want_body)
 
 	cli->in_len = obj->size;
 
-	resp->req.data_len = GUINT32_TO_LE(obj->size);
+	resp->req.data_len = GUINT64_TO_LE((uint64_t)obj->size);
 	memcpy(resp->req.checksum, obj->hashstr, sizeof(obj->hashstr));
 	resp->req.checksum[sizeof(obj->hashstr)] = 0;
 	resp->mtime = GUINT64_TO_LE(obj->mtime);
diff --git a/server/server.c b/server/server.c
index 2ad3db2..161dc67 100644
--- a/server/server.c
+++ b/server/server.c
@@ -639,7 +639,7 @@ static bool cli_resp_xml(struct client *cli, GList *content)
 {
 	int rc;
 	bool rcb;
-	int content_len = strlist_len(content);
+	size_t content_len = strlist_len(content);
 	struct chunksrv_req *resp = NULL;
 
 	resp = malloc(sizeof(*resp));
@@ -650,7 +650,7 @@ static bool cli_resp_xml(struct client *cli, GList *content)
 
 	memcpy(resp, &cli->creq, sizeof(cli->creq));
 
-	resp->data_len = GUINT64_TO_LE(content_len);
+	resp->data_len = GUINT64_TO_LE((uint64_t)content_len);
 
 	cli->state = evt_recycle;
 

We have two problems above:

 #1 the result of byteswap must not be allowed to be type-promoted,
    sign-extended, or otherwise changed by the language. Only bit-to-bit
    assignment will do. Therefore, we must only use 64-bit conversions
    when assigning to req->data_len.

 #2 for some reason swapping of a 32-bit value does not work either,
    I suspect clipping when shifting. So I had to fix that too.

With the above, basic-object passes, test advance further, and then
large-object fails like so:

test failed on line 134

        rcb = stc_get_start(stc, key, &rfd, &len);
        OK(len == (N_BUFS * BUFSZ));   <------- fails. how much is len?

Curiously, it fails on 32-bit PPC only and works on ppc64.

With this, I'm going to quit stepping on your toes and go work on
CLD interface for Chunkd, then data replication in tabled.
Sorry for the interference.

Good luck,
-- Pete

Re: [Patch] chunkd: use port xxx82 to build

[Jeff Garzik]

Pete Zaitcev wrote:
> On Fri, 7 Aug 2009 19:36:28 -0400 (EDT), Rick Peralta <fbp@tiac.net> wrote:
> 
>> I was looking from net_write() & net_read() to see where they were
>> called and what was passed to them.  struct chunksrv_req & struct
>> chunksrv_resp_get seem to be the primary structures.
> 
> I didn't know how serious you were about it, so I went ahead and
> looked at the endian stuff. The immediate issue - the hang of
> basic-object - was addressed with the following fix:
> 
> diff --git a/server/object.c b/server/object.c
> index 4eb62f7..6ea1500 100644
> --- a/server/object.c
> +++ b/server/object.c
> @@ -290,7 +290,7 @@ bool object_get(struct client *cli, bool want_body)
>  
>  	cli->in_len = obj->size;
>  
> -	resp->req.data_len = GUINT32_TO_LE(obj->size);
> +	resp->req.data_len = GUINT64_TO_LE((uint64_t)obj->size);
>  	memcpy(resp->req.checksum, obj->hashstr, sizeof(obj->hashstr));
>  	resp->req.checksum[sizeof(obj->hashstr)] = 0;
>  	resp->mtime = GUINT64_TO_LE(obj->mtime);
> diff --git a/server/server.c b/server/server.c
> index 2ad3db2..161dc67 100644
> --- a/server/server.c
> +++ b/server/server.c
> @@ -639,7 +639,7 @@ static bool cli_resp_xml(struct client *cli, GList *content)
>  {
>  	int rc;
>  	bool rcb;
> -	int content_len = strlist_len(content);
> +	size_t content_len = strlist_len(content);
>  	struct chunksrv_req *resp = NULL;
>  
>  	resp = malloc(sizeof(*resp));
> @@ -650,7 +650,7 @@ static bool cli_resp_xml(struct client *cli, GList *content)
>  
>  	memcpy(resp, &cli->creq, sizeof(cli->creq));
>  
> -	resp->data_len = GUINT64_TO_LE(content_len);
> +	resp->data_len = GUINT64_TO_LE((uint64_t)content_len);
>  
>  	cli->state = evt_recycle;

I think type-safety should win the day.  So, I am going to work on a set 
of 'static inline' functions that wrap GUINT* macros.  We shouldn't need 
to add casts to achieve type promotion, in that case.

I think I will name them le32_to_cpu, cpu_to_le16, le64_to_cpu, etc. :)

	Jeff

Re: [Patch] chunkd: use port xxx82 to build

[Pete Zaitcev]

On Mon, 10 Aug 2009 13:37:51 -0400, Jeff Garzik <jeff@garzik.org> wrote:

> I think type-safety should win the day.  So, I am going to work on a set 
> of 'static inline' functions that wrap GUINT* macros.  We shouldn't need 
> to add casts to achieve type promotion, in that case.

Sounds like a good idea, because we have better things to do than
carefuly type-match everything and then never, ever, change any
data sizes. But perhaps a fierce urgency is not in it.

-- Pete

Re: [Patch] chunkd: use port xxx82 to build

[Jeff Garzik]

Pete Zaitcev wrote:
> test failed on line 134
> 
>         rcb = stc_get_start(stc, key, &rfd, &len);
>         OK(len == (N_BUFS * BUFSZ));   <------- fails. how much is len?
> 
> Curiously, it fails on 32-bit PPC only and works on ppc64.


This is solved upstream, now, too:  'len' was a size_t, while 
stc_get_start wanted a uint64_t.

	Jeff

Re: [Patch] chunkd: use port xxx82 to build

[Rick Peralta]

>> There is support for all sorts of byte swaping.  And swapping bytes works in both directions, big to little or little to big is the same operation.   ntohll is a 64 bit byte swap, but might not naturally work based on architecture.  Check out be64_to_cpu and friends.  They are in the tree and should work reliably, regardless of architecture.

>ntohll does not appear to be part of glibc...
>be64_to_cpu and friends come from /usr/include/linux...
>In both cases, GLib provides idioms that are more portable, without 
>adding additional dependencies to this project.
>
>Certainly the underlying swap16/swap32/swap64 code is the same for big 
>or little endian...

Understood.

>> I've been dealing with issues that seem to be related to Fedora8.  Fixing them one at a time seems counter productive, as some of the libraries are not exactly backward compatible.  What OS/release and libraries do you use to build Hail? 
>
>Can you detail those issues?
>
>Just like we support other OS's, I would like to either (a) support an 
>older Linux distro, or (b) have configure fail gracefully, giving a 
>helpful message about why configure did not like your system.

understood.  The network wrappers were intended to be a starting point to find the plethora of existing byte swap code fragments.

With FC8, the libevent code that comes with yum is out of date:

configure:20147: gcc -o conftest -g -O2   conftest.c -levent   >&5
/tmp/ccmjVmle.o: In function `main':
/home/_/Desktop/CloudComputing/Hail/chunkd-0.3/conftest.c:34: undefined reference to `event_base_new'   (not in the older libraries)
collect2: ld returned 1 exit status      
configure:20154: $? = 1
configure: failed program was:
| /* confdefs.h.  */

The configure could be modified to include a message about updating to a more recent library.

 - Rick

Re: [Patch] chunkd: use port xxx82 to build

[Rick Peralta]

Hi Jeff,

There is support for all sorts of byte swaping.  And swapping bytes works in both directions, big to little or little to big is the same operation.   ntohll is a 64 bit byte swap, but might not naturally work based on architecture.  Check out be64_to_cpu and friends.  They are in the tree and should work reliably, regardless of architecture.

I've been dealing with issues that seem to be related to Fedora8.  Fixing them one at a time seems counter productive, as some of the libraries are not exactly backward compatible.  What OS/release and libraries do you use to build Hail? 

Cheers,

 - Rick

>From: Jeff Garzik <jeff@garzik.org>
>Sent: Aug 12, 2009 8:11 PM
>To: Rick Peralta <fbp@tiac.net>
>Cc: Pete Zaitcev <zaitcev@redhat.com>, Project Hail List <hail-devel@vger.kernel.org>
>Subject: Re: [Patch] chunkd: use port xxx82 to build
>
>Rick Peralta wrote:
>> There is code to byte swap in the network stack.  ntoh() and friends should do the job.
>
>Unfortunately that isn't the best solution for us, for a few reasons:
>
>* There is no 64-bit variant, which we need.
>
>* The system platform definitions of ntohl() etc. can vary between 
>macros and function calls depending on compiler's -O settings, which 
>complicates debugging (at least for me).
>
>* ntohl() forces big endian.  Given the current landscape of commodity 
>hardware, I intentionally chose little endian over big endian.  That 
>will completely eliminate byte-swapping for the primary platforms' byte 
>order.
>
>We use the GLib byte order facilities, which provide little endian 
>variants and are portable (along with the rest of GLib) across modern 
>platforms.  But GUINT*_{FROM,TO}_LE macros come with the minor 
>disadvantages of all macros, namely a lack of type safety, problems with 
>multiple evaluation, etc.
>
>Regards,
>
>	Jeff
>
>
>

Re: [Patch] chunkd: use port xxx82 to build

[Jeff Garzik]

Rick Peralta wrote:
> Hi Jeff,
> 
> There is support for all sorts of byte swaping.  And swapping bytes works in both directions, big to little or little to big is the same operation.   ntohll is a 64 bit byte swap, but might not naturally work based on architecture.  Check out be64_to_cpu and friends.  They are in the tree and should work reliably, regardless of architecture.

ntohll does not appear to be part of glibc, but rather nspr.  Adding a 
dependency on nspr would duplicate existing functionality in existing 
dependency GLib.

be64_to_cpu and friends come from /usr/include/linux, which is 
Linux-specific.  That doesn't work, because the project currently works 
on FreeBSD and Solaris, in addition to Linux.

In both cases, GLib provides idioms that are more portable, without 
adding additional dependencies to this project.

Certainly the underlying swap16/swap32/swap64 code is the same for big 
or little endian, but the APIs you cite (ntohl, ntohll, etc.) will 
compile to no-op on big endian, and swap on little endian, which gives 
the opposite result of the stated design choice (fixed little endian to 
match commodity hardware).


> I've been dealing with issues that seem to be related to Fedora8.  Fixing them one at a time seems counter productive, as some of the libraries are not exactly backward compatible.  What OS/release and libraries do you use to build Hail? 

Can you detail those issues?

Just like we support other OS's, I would like to either (a) support an 
older Linux distro, or (b) have configure fail gracefully, giving a 
helpful message about why configure did not like your system.

	Jeff

Re: [Patch] chunkd: use port xxx82 to build

[Rick Peralta]

There is code to byte swap in the network stack.  ntoh() and friends should do the job.

FYI: ntoh() and friends are usually wrappers, often many layers) for other code.

 - Rick

>From: Jeff Garzik <jeff@garzik.org>
>Sent: Aug 10, 2009 1:37 PM
>To: Pete Zaitcev <zaitcev@redhat.com>
>Cc: Rick Peralta <fbp@tiac.net>, Project Hail List <hail-devel@vger.kernel.org>
>Subject: Re: [Patch] chunkd: use port xxx82 to build
>
>Pete Zaitcev wrote:
>> On Fri, 7 Aug 2009 19:36:28 -0400 (EDT), Rick Peralta <fbp@tiac.net> wrote:
>> 
>>> I was looking from net_write() & net_read() to see where they were
>>> called and what was passed to them.  struct chunksrv_req & struct
>>> chunksrv_resp_get seem to be the primary structures.
>> 
>> I didn't know how serious you were about it, so I went ahead and
>> looked at the endian stuff. The immediate issue - the hang of
>> basic-object - was addressed with the following fix:
>> 
>> diff --git a/server/object.c b/server/object.c
>> index 4eb62f7..6ea1500 100644
>> --- a/server/object.c
>> +++ b/server/object.c
>> @@ -290,7 +290,7 @@ bool object_get(struct client *cli, bool want_body)
>>  
>>  	cli->in_len = obj->size;
>>  
>> -	resp->req.data_len = GUINT32_TO_LE(obj->size);
>> +	resp->req.data_len = GUINT64_TO_LE((uint64_t)obj->size);
>>  	memcpy(resp->req.checksum, obj->hashstr, sizeof(obj->hashstr));
>>  	resp->req.checksum[sizeof(obj->hashstr)] = 0;
>>  	resp->mtime = GUINT64_TO_LE(obj->mtime);
>> diff --git a/server/server.c b/server/server.c
>> index 2ad3db2..161dc67 100644
>> --- a/server/server.c
>> +++ b/server/server.c
>> @@ -639,7 +639,7 @@ static bool cli_resp_xml(struct client *cli, GList *content)
>>  {
>>  	int rc;
>>  	bool rcb;
>> -	int content_len = strlist_len(content);
>> +	size_t content_len = strlist_len(content);
>>  	struct chunksrv_req *resp = NULL;
>>  
>>  	resp = malloc(sizeof(*resp));
>> @@ -650,7 +650,7 @@ static bool cli_resp_xml(struct client *cli, GList *content)
>>  
>>  	memcpy(resp, &cli->creq, sizeof(cli->creq));
>>  
>> -	resp->data_len = GUINT64_TO_LE(content_len);
>> +	resp->data_len = GUINT64_TO_LE((uint64_t)content_len);
>>  
>>  	cli->state = evt_recycle;
>
>I think type-safety should win the day.  So, I am going to work on a set 
>of 'static inline' functions that wrap GUINT* macros.  We shouldn't need 
>to add casts to achieve type promotion, in that case.
>
>I think I will name them le32_to_cpu, cpu_to_le16, le64_to_cpu, etc. :)
>
>	Jeff
>
>
>

Re: [Patch] chunkd: use port xxx82 to build

[Jeff Garzik]

Rick Peralta wrote:
> There is code to byte swap in the network stack.  ntoh() and friends should do the job.

Unfortunately that isn't the best solution for us, for a few reasons:

* There is no 64-bit variant, which we need.

* The system platform definitions of ntohl() etc. can vary between 
macros and function calls depending on compiler's -O settings, which 
complicates debugging (at least for me).

* ntohl() forces big endian.  Given the current landscape of commodity 
hardware, I intentionally chose little endian over big endian.  That 
will completely eliminate byte-swapping for the primary platforms' byte 
order.

We use the GLib byte order facilities, which provide little endian 
variants and are portable (along with the rest of GLib) across modern 
platforms.  But GUINT*_{FROM,TO}_LE macros come with the minor 
disadvantages of all macros, namely a lack of type safety, problems with 
multiple evaluation, etc.

Regards,

	Jeff

Re: [Patch] chunkd: use port xxx82 to build

[Rick Peralta]

The build still fails:

# yum install git-core libevent libevent-devel glib2-devel openssl-devel zlib-devel libxml2-devel procps cld cld-devel
$ git clone git://git.kernel.org/pub/scm/daemon/cld/cld.git
$ ./autogen.sh 
$ ./configure  // fail
$ CFLAGS="-O2 -Wall -g" ./configure  // fail
$ cd chunkd
$ ./configure   // fail

Still fails in the same spot.  Where in the config.log did you see that it was a problem with libevent?

 - Rick

>From: Jeff Garzik <jeff@garzik.org>
>Sent: Aug 7, 2009 11:38 PM
>To: Rick Peralta <fbp@tiac.net>
>Cc: Project Hail List <hail-devel@vger.kernel.org>
>Subject: Re: [Patch] chunkd: use port xxx82 to build
>
>Rick Peralta wrote:
>>> We are definitely interested in the build failures, yes.
>> 
>> When builidng, chunkd fails in configure (so do the other Hail files).
>
>According to your config.log, you are missing the libevent dependency. 
>On Fedora, this will be "libevent-devel", which will also install the 
>libevent base package.
>
>configure intentionally stops when it cannot find something.
>
>I wouldn't advise bothering with the tarballs, though.  clone and build 
>each git repository, e.g.
>
>$ git clone git://git.kernel.org/pub/scm/daemon/cld/cld.git
>$ cd cld
>$ ./autogen.sh
>$ CFLAGS="-O2 -Wall -g" ./configure
>
>Because development is moving very rapidly, it is best for developers to 
>always look at the git repository for the latest source code.
>
>configure gives much more informative errors in the latest upstream (== 
>current source code in git repository), so this problem should be gone 
>in chunkd version 0.4, when released.
>
>
>> Running: Linux version 2.6.27.25-78.2.56.fc9.i686 (mockbuild@) (gcc version 4.3.0 20080428 (Red Hat 4.3.0-8) (GCC) ) #1 SMP Thu Jun 18 12:47:50 EDT 2009 Fedora release 9 (Sulphur)
>> Default install with all updates complete, all gcc and make updates
>> 
>> I asked a friend to run the build on a different distribution and it failed differently.
>> 
>>> Have you verified that all necessary dependencies are installed?
>> 
>> It is not clear what needs to be installed in order to build... is there a list?
>
>Here is the list from pkg/chunkd.spec:
>
>BuildRequires:  libevent-devel glib2-devel openssl-devel zlib-devel
>BuildRequires:  libxml2-devel procps cld cld-devel
>
>Regards,
>
>	Jeff
>
>

Re: [Patch] chunkd: use port xxx82 to build

[Jeff Garzik]

Rick Peralta wrote:
> The build still fails:
> 
> # yum install git-core libevent libevent-devel glib2-devel openssl-devel zlib-devel libxml2-devel procps cld cld-devel
> $ git clone git://git.kernel.org/pub/scm/daemon/cld/cld.git
> $ ./autogen.sh 
> $ ./configure  // fail
> $ CFLAGS="-O2 -Wall -g" ./configure  // fail
> $ cd chunkd
> $ ./configure   // fail
> 
> Still fails in the same spot.  Where in the config.log did you see that it was a problem with libevent?

Yeah, it will continue to fail until you install the needed dependencies.

Here is the relevant config.log section:
[...]
> configure:20418: checking for SSL_new in -lssl
> configure:20453: gcc -o conftest -g -O2   conftest.c -lssl   >&5
> configure:20460: $? = 0
> configure:20481: result: yes
> configure:20487: checking for event_base_new in -levent
> configure:20522: gcc -o conftest -g -O2   conftest.c -levent   >&5
> /usr/bin/ld: cannot find -levent
> collect2: ld returned 1 exit status
> configure:20529: $? = 1
> configure: failed program was:
[... followed by many lines of state dump ...]

Regards,

	Jeff

Re: [Patch] chunkd: use port xxx82 to build

[Rick Peralta]

[-- Attachment #1: Type: text/plain, Size: 1921 bytes --]

>We are definitely interested in the build failures, yes.

When builidng, chunkd fails in configure (so do the other Hail files).

Running: Linux version 2.6.27.25-78.2.56.fc9.i686 (mockbuild@) (gcc version 4.3.0 20080428 (Red Hat 4.3.0-8) (GCC) ) #1 SMP Thu Jun 18 12:47:50 EDT 2009 Fedora release 9 (Sulphur)
Default install with all updates complete, all gcc and make updates

I asked a friend to run the build on a different distribution and it failed differently.

>Have you verified that all necessary dependencies are installed?

It is not clear what needs to be installed in order to build... is there a list?

 - Rick

>From: Jeff Garzik <jeff@garzik.org>
>Sent: Aug 7, 2009 10:23 AM
>To: Rick Peralta <fbp@tiac.net>
>Cc: Pete Zaitcev <zaitcev@redhat.com>, Project Hail List <hail-devel@vger.kernel.org>
>Subject: Re: [Patch] chunkd: use port xxx82 to build
>
>Rick Peralta wrote:
>> Hi Jeff, Pete and all,
>> 
>> I've been called out of town and will not have time to look at the Hail stuff for a while.
>> 
>> Re: the endian issues, a quick look suggests that most of the code is OK and that the structs that are passed over the network need to be put into a standard endian format.  There seem to be only two structs that need attention.  It is less clear if there is a single place to make the byte order fix.
>
>Specifically which two structs need attention?
>
>
>> Note: The Hail builds are not working for me.  I've tried running a couple of difference distributions and they all fail in configure.  Running usual upgrades does not correct the problem.  Digging into the scripts a bit shows they generally fail due to library issues or what appear to be related to compiler issues (test... ac_nonexistent.h).  If there is interest,  I can forward the log files.
>
>Have you verified that all necessary dependencies are installed?
>
>We are definitely interested in the build failures, yes.
>
>	Jeff
>
>
>

[-- Attachment #2: config.log --]
[-- Type: application/octet-stream, Size: 29164 bytes --]

This file contains any messages produced by compilers while
running configure, to aid debugging if configure makes a mistake.

It was created by chunkd configure 0.3, which was
generated by GNU Autoconf 2.63.  Invocation command line was

  $ ./configure 

## --------- ##
## Platform. ##
## --------- ##

hostname = localhost.localdomain
uname -m = i686
uname -r = 2.6.27.25-78.2.56.fc9.i686
uname -s = Linux
uname -v = #1 SMP Thu Jun 18 12:47:50 EDT 2009

/usr/bin/uname -p = unknown
/bin/uname -X     = unknown

/bin/arch              = i686
/usr/bin/arch -k       = unknown
/usr/convex/getsysinfo = unknown
/usr/bin/hostinfo      = unknown
/bin/machine           = unknown
/usr/bin/oslevel       = unknown
/bin/universe          = unknown

PATH: /usr/lib/qt-3.3/bin
PATH: /usr/kerberos/bin
PATH: /usr/lib/ccache
PATH: /usr/local/bin
PATH: /usr/bin
PATH: /bin
PATH: /home/_/bin


## ----------- ##
## Core tests. ##
## ----------- ##

configure:2120: checking for a BSD-compatible install
configure:2188: result: /usr/bin/install -c
configure:2199: checking whether build environment is sane
configure:2242: result: yes
configure:2267: checking for a thread-safe mkdir -p
configure:2306: result: /bin/mkdir -p
configure:2319: checking for gawk
configure:2335: found /usr/bin/gawk
configure:2346: result: gawk
configure:2357: checking whether make sets $(MAKE)
configure:2379: result: yes
configure:2579: checking whether to enable maintainer-specific portions of Makefiles
configure:2588: result: no
configure:2650: checking for gcc
configure:2666: found /usr/lib/ccache/gcc
configure:2677: result: gcc
configure:2909: checking for C compiler version
configure:2917: gcc --version >&5
gcc (GCC) 4.3.0 20080428 (Red Hat 4.3.0-8)
Copyright (C) 2008 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

configure:2921: $? = 0
configure:2928: gcc -v >&5
Using built-in specs.
Target: i386-redhat-linux
Configured with: ../configure --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --with-bugurl=http://bugzilla.redhat.com/bugzilla --enable-bootstrap --enable-shared --enable-threads=posix --enable-checking=release --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-languages=c,c++,objc,obj-c++,java,fortran,ada --enable-java-awt=gtk --disable-dssi --enable-plugin --with-java-home=/usr/lib/jvm/java-1.5.0-gcj-1.5.0.0/jre --enable-libgcj-multifile --enable-java-maintainer-mode --with-ecj-jar=/usr/share/java/eclipse-ecj.jar --disable-libjava-multilib --with-cpu=generic --build=i386-redhat-linux
Thread model: posix
gcc version 4.3.0 20080428 (Red Hat 4.3.0-8) (GCC) 
configure:2932: $? = 0
configure:2939: gcc -V >&5
gcc: '-V' option must have argument
configure:2943: $? = 1
configure:2966: checking for C compiler default output file name
configure:2988: gcc    conftest.c  >&5
configure:2992: $? = 0
configure:3030: result: a.out
configure:3049: checking whether the C compiler works
configure:3059: ./a.out
configure:3063: $? = 0
configure:3082: result: yes
configure:3089: checking whether we are cross compiling
configure:3091: result: no
configure:3094: checking for suffix of executables
configure:3101: gcc -o conftest    conftest.c  >&5
configure:3105: $? = 0
configure:3131: result: 
configure:3137: checking for suffix of object files
configure:3163: gcc -c   conftest.c >&5
configure:3167: $? = 0
configure:3192: result: o
configure:3196: checking whether we are using the GNU C compiler
configure:3225: gcc -c   conftest.c >&5
configure:3232: $? = 0
configure:3249: result: yes
configure:3258: checking whether gcc accepts -g
configure:3288: gcc -c -g  conftest.c >&5
configure:3295: $? = 0
configure:3396: result: yes
configure:3413: checking for gcc option to accept ISO C89
configure:3487: gcc  -c -g -O2  conftest.c >&5
configure:3494: $? = 0
configure:3517: result: none needed
configure:3546: checking for style of include used by make
configure:3574: result: GNU
configure:3599: checking dependency style of gcc
configure:3690: result: gcc3
configure:3711: checking how to run the C preprocessor
configure:3751: gcc -E  conftest.c
configure:3758: $? = 0
configure:3789: gcc -E  conftest.c
conftest.c:10:28: error: ac_nonexistent.h: No such file or directory
configure:3796: $? = 1
configure: failed program was:
| /* confdefs.h.  */
| #define PACKAGE_NAME "chunkd"
| #define PACKAGE_TARNAME "chunkd"
| #define PACKAGE_VERSION "0.3"
| #define PACKAGE_STRING "chunkd 0.3"
| #define PACKAGE_BUGREPORT "http://www.kernel.org/pub/software/network/distsrv/"
| #define PACKAGE "chunkd"
| #define VERSION "0.3"
| /* end confdefs.h.  */
| #include <ac_nonexistent.h>
configure:3829: result: gcc -E
configure:3858: gcc -E  conftest.c
configure:3865: $? = 0
configure:3896: gcc -E  conftest.c
conftest.c:10:28: error: ac_nonexistent.h: No such file or directory
configure:3903: $? = 1
configure: failed program was:
| /* confdefs.h.  */
| #define PACKAGE_NAME "chunkd"
| #define PACKAGE_TARNAME "chunkd"
| #define PACKAGE_VERSION "0.3"
| #define PACKAGE_STRING "chunkd 0.3"
| #define PACKAGE_BUGREPORT "http://www.kernel.org/pub/software/network/distsrv/"
| #define PACKAGE "chunkd"
| #define VERSION "0.3"
| /* end confdefs.h.  */
| #include <ac_nonexistent.h>
configure:3943: checking for grep that handles long lines and -e
configure:4003: result: /bin/grep
configure:4008: checking for egrep
configure:4072: result: /bin/grep -E
configure:4078: checking whether gcc needs -traditional
configure:4120: result: no
configure:4205: checking build system type
configure:4223: result: i686-pc-linux-gnu
configure:4245: checking host system type
configure:4260: result: i686-pc-linux-gnu
configure:4282: checking for a sed that does not truncate output
configure:4338: result: /bin/sed
configure:4352: checking for ld used by gcc
configure:4419: result: /usr/bin/ld
configure:4428: checking if the linker (/usr/bin/ld) is GNU ld
configure:4443: result: yes
configure:4448: checking for /usr/bin/ld option to reload object files
configure:4455: result: -r
configure:4473: checking for BSD-compatible nm
configure:4522: result: /usr/bin/nm -B
configure:4526: checking whether ln -s works
configure:4530: result: yes
configure:4537: checking how to recognize dependent libraries
configure:4723: result: pass_all
configure:4967: checking for ANSI C header files
configure:4997: gcc -c -g -O2  conftest.c >&5
configure:5004: $? = 0
configure:5103: gcc -o conftest -g -O2   conftest.c  >&5
configure:5107: $? = 0
configure:5113: ./conftest
configure:5117: $? = 0
configure:5135: result: yes
configure:5159: checking for sys/types.h
configure:5180: gcc -c -g -O2  conftest.c >&5
configure:5187: $? = 0
configure:5204: result: yes
configure:5159: checking for sys/stat.h
configure:5180: gcc -c -g -O2  conftest.c >&5
configure:5187: $? = 0
configure:5204: result: yes
configure:5159: checking for stdlib.h
configure:5180: gcc -c -g -O2  conftest.c >&5
configure:5187: $? = 0
configure:5204: result: yes
configure:5159: checking for string.h
configure:5180: gcc -c -g -O2  conftest.c >&5
configure:5187: $? = 0
configure:5204: result: yes
configure:5159: checking for memory.h
configure:5180: gcc -c -g -O2  conftest.c >&5
configure:5187: $? = 0
configure:5204: result: yes
configure:5159: checking for strings.h
configure:5180: gcc -c -g -O2  conftest.c >&5
configure:5187: $? = 0
configure:5204: result: yes
configure:5159: checking for inttypes.h
configure:5180: gcc -c -g -O2  conftest.c >&5
configure:5187: $? = 0
configure:5204: result: yes
configure:5159: checking for stdint.h
configure:5180: gcc -c -g -O2  conftest.c >&5
configure:5187: $? = 0
configure:5204: result: yes
configure:5159: checking for unistd.h
configure:5180: gcc -c -g -O2  conftest.c >&5
configure:5187: $? = 0
configure:5204: result: yes
configure:5234: checking dlfcn.h usability
configure:5251: gcc -c -g -O2  conftest.c >&5
configure:5258: $? = 0
configure:5272: result: yes
configure:5276: checking dlfcn.h presence
configure:5291: gcc -E  conftest.c
configure:5298: $? = 0
configure:5312: result: yes
configure:5345: checking for dlfcn.h
configure:5354: result: yes
configure:5427: checking for g++
configure:5443: found /usr/lib/ccache/g++
configure:5454: result: g++
configure:5481: checking for C++ compiler version
configure:5489: g++ --version >&5
g++ (GCC) 4.3.0 20080428 (Red Hat 4.3.0-8)
Copyright (C) 2008 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

configure:5493: $? = 0
configure:5500: g++ -v >&5
Using built-in specs.
Target: i386-redhat-linux
Configured with: ../configure --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --with-bugurl=http://bugzilla.redhat.com/bugzilla --enable-bootstrap --enable-shared --enable-threads=posix --enable-checking=release --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-languages=c,c++,objc,obj-c++,java,fortran,ada --enable-java-awt=gtk --disable-dssi --enable-plugin --with-java-home=/usr/lib/jvm/java-1.5.0-gcj-1.5.0.0/jre --enable-libgcj-multifile --enable-java-maintainer-mode --with-ecj-jar=/usr/share/java/eclipse-ecj.jar --disable-libjava-multilib --with-cpu=generic --build=i386-redhat-linux
Thread model: posix
gcc version 4.3.0 20080428 (Red Hat 4.3.0-8) (GCC) 
configure:5504: $? = 0
configure:5511: g++ -V >&5
g++: '-V' option must have argument
configure:5515: $? = 1
configure:5518: checking whether we are using the GNU C++ compiler
configure:5547: g++ -c   conftest.cpp >&5
configure:5554: $? = 0
configure:5571: result: yes
configure:5580: checking whether g++ accepts -g
configure:5610: g++ -c -g  conftest.cpp >&5
configure:5617: $? = 0
configure:5718: result: yes
configure:5743: checking dependency style of g++
configure:5834: result: gcc3
configure:5859: checking how to run the C++ preprocessor
configure:5895: g++ -E  conftest.cpp
configure:5902: $? = 0
configure:5933: g++ -E  conftest.cpp
conftest.cpp:21:28: error: ac_nonexistent.h: No such file or directory
configure:5940: $? = 1
configure: failed program was:
| /* confdefs.h.  */
| #define PACKAGE_NAME "chunkd"
| #define PACKAGE_TARNAME "chunkd"
| #define PACKAGE_VERSION "0.3"
| #define PACKAGE_STRING "chunkd 0.3"
| #define PACKAGE_BUGREPORT "http://www.kernel.org/pub/software/network/distsrv/"
| #define PACKAGE "chunkd"
| #define VERSION "0.3"
| #define STDC_HEADERS 1
| #define HAVE_SYS_TYPES_H 1
| #define HAVE_SYS_STAT_H 1
| #define HAVE_STDLIB_H 1
| #define HAVE_STRING_H 1
| #define HAVE_MEMORY_H 1
| #define HAVE_STRINGS_H 1
| #define HAVE_INTTYPES_H 1
| #define HAVE_STDINT_H 1
| #define HAVE_UNISTD_H 1
| #define HAVE_DLFCN_H 1
| /* end confdefs.h.  */
| #include <ac_nonexistent.h>
configure:5973: result: g++ -E
configure:6002: g++ -E  conftest.cpp
configure:6009: $? = 0
configure:6040: g++ -E  conftest.cpp
conftest.cpp:21:28: error: ac_nonexistent.h: No such file or directory
configure:6047: $? = 1
configure: failed program was:
| /* confdefs.h.  */
| #define PACKAGE_NAME "chunkd"
| #define PACKAGE_TARNAME "chunkd"
| #define PACKAGE_VERSION "0.3"
| #define PACKAGE_STRING "chunkd 0.3"
| #define PACKAGE_BUGREPORT "http://www.kernel.org/pub/software/network/distsrv/"
| #define PACKAGE "chunkd"
| #define VERSION "0.3"
| #define STDC_HEADERS 1
| #define HAVE_SYS_TYPES_H 1
| #define HAVE_SYS_STAT_H 1
| #define HAVE_STDLIB_H 1
| #define HAVE_STRING_H 1
| #define HAVE_MEMORY_H 1
| #define HAVE_STRINGS_H 1
| #define HAVE_INTTYPES_H 1
| #define HAVE_STDINT_H 1
| #define HAVE_UNISTD_H 1
| #define HAVE_DLFCN_H 1
| /* end confdefs.h.  */
| #include <ac_nonexistent.h>
configure:6142: checking for g77
configure:6172: result: no
configure:6142: checking for xlf
configure:6172: result: no
configure:6142: checking for f77
configure:6172: result: no
configure:6142: checking for frt
configure:6172: result: no
configure:6142: checking for pgf77
configure:6172: result: no
configure:6142: checking for cf77
configure:6172: result: no
configure:6142: checking for fort77
configure:6172: result: no
configure:6142: checking for fl32
configure:6172: result: no
configure:6142: checking for af77
configure:6172: result: no
configure:6142: checking for xlf90
configure:6172: result: no
configure:6142: checking for f90
configure:6172: result: no
configure:6142: checking for pgf90
configure:6172: result: no
configure:6142: checking for pghpf
configure:6172: result: no
configure:6142: checking for epcf90
configure:6172: result: no
configure:6142: checking for gfortran
configure:6158: found /usr/bin/gfortran
configure:6169: result: gfortran
configure:6195: checking for Fortran 77 compiler version
configure:6203: gfortran --version >&5
GNU Fortran (GCC) 4.3.0 20080428 (Red Hat 4.3.0-8)
Copyright (C) 2008 Free Software Foundation, Inc.

GNU Fortran comes with NO WARRANTY, to the extent permitted by law.
You may redistribute copies of GNU Fortran
under the terms of the GNU General Public License.
For more information about these matters, see the file named COPYING

configure:6207: $? = 0
configure:6214: gfortran -v >&5
Using built-in specs.
Target: i386-redhat-linux
Configured with: ../configure --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --with-bugurl=http://bugzilla.redhat.com/bugzilla --enable-bootstrap --enable-shared --enable-threads=posix --enable-checking=release --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-languages=c,c++,objc,obj-c++,java,fortran,ada --enable-java-awt=gtk --disable-dssi --enable-plugin --with-java-home=/usr/lib/jvm/java-1.5.0-gcj-1.5.0.0/jre --enable-libgcj-multifile --enable-java-maintainer-mode --with-ecj-jar=/usr/share/java/eclipse-ecj.jar --disable-libjava-multilib --with-cpu=generic --build=i386-redhat-linux
Thread model: posix
gcc version 4.3.0 20080428 (Red Hat 4.3.0-8) (GCC) 
configure:6218: $? = 0
configure:6225: gfortran -V >&5
gfortran: '-V' option must have argument
configure:6229: $? = 1
configure:6237: checking whether we are using the GNU Fortran 77 compiler
configure:6256: gfortran -c  conftest.F >&5
configure:6263: $? = 0
configure:6280: result: yes
configure:6286: checking whether gfortran accepts -g
configure:6303: gfortran -c -g conftest.f >&5
configure:6310: $? = 0
configure:6326: result: yes
configure:6359: checking the maximum length of command line arguments
configure:6471: result: 1966080
configure:6483: checking command to parse /usr/bin/nm -B output from gcc object
configure:6588: gcc -c -g -O2  conftest.c >&5
configure:6591: $? = 0
configure:6595: /usr/bin/nm -B conftest.o \| sed -n -e 's/^.*[ 	]\([ABCDGIRSTW][ABCDGIRSTW]*\)[ 	][ 	]*\([_A-Za-z][_A-Za-z0-9]*\)$/\1 \2 \2/p' \> conftest.nm
configure:6598: $? = 0
configure:6650: gcc -o conftest -g -O2   conftest.c conftstm.o >&5
configure:6653: $? = 0
configure:6691: result: ok
configure:6695: checking for objdir
configure:6710: result: .libs
configure:6802: checking for ar
configure:6818: found /usr/bin/ar
configure:6829: result: ar
configure:6894: checking for ranlib
configure:6910: found /usr/bin/ranlib
configure:6921: result: ranlib
configure:6986: checking for strip
configure:7002: found /usr/bin/strip
configure:7013: result: strip
configure:7603: checking if gcc supports -fno-rtti -fno-exceptions
configure:7621: gcc -c -g -O2  -fno-rtti -fno-exceptions conftest.c >&5
cc1: warning: command line option "-fno-rtti" is valid for C++/ObjC++ but not for C
cc1: warning: command line option "-fno-rtti" is valid for C++/ObjC++ but not for C
configure:7625: $? = 0
configure:7638: result: no
configure:7653: checking for gcc option to produce PIC
configure:7885: result: -fPIC
configure:7893: checking if gcc PIC flag -fPIC works
configure:7911: gcc -c -g -O2  -fPIC -DPIC conftest.c >&5
configure:7915: $? = 0
configure:7928: result: yes
configure:7956: checking if gcc static flag -static works
configure:7984: result: yes
configure:7994: checking if gcc supports -c -o file.o
configure:8015: gcc -c -g -O2  -o out/conftest2.o conftest.c >&5
configure:8019: $? = 0
configure:8041: result: yes
configure:8067: checking whether the gcc linker (/usr/bin/ld) supports shared libraries
configure:9055: result: yes
configure:9076: checking whether -lc should be explicitly linked in
configure:9081: gcc -c -g -O2  conftest.c >&5
configure:9084: $? = 0
configure:9099: gcc -shared conftest.o  -v -Wl,-soname -Wl,conftest -o conftest 2\>\&1 \| grep  -lc  \>/dev/null 2\>\&1
configure:9102: $? = 0
configure:9114: result: no
configure:9122: checking dynamic linker characteristics
configure:9726: result: GNU/Linux ld.so
configure:9750: checking how to hardcode library paths into programs
configure:9775: result: immediate
configure:9789: checking whether stripping libraries is possible
configure:9794: result: yes
configure:10624: checking if libtool supports shared libraries
configure:10626: result: yes
configure:10629: checking whether to build shared libraries
configure:10650: result: yes
configure:10653: checking whether to build static libraries
configure:10657: result: yes
configure:10751: creating libtool
configure:11344: checking for ld used by g++
configure:11411: result: /usr/bin/ld
configure:11420: checking if the linker (/usr/bin/ld) is GNU ld
configure:11435: result: yes
configure:11486: checking whether the g++ linker (/usr/bin/ld) supports shared libraries
configure:12440: result: yes
configure:12457: g++ -c -g -O2  conftest.cpp >&5
configure:12460: $? = 0
configure:12616: checking for g++ option to produce PIC
configure:12900: result: -fPIC
configure:12908: checking if g++ PIC flag -fPIC works
configure:12926: g++ -c -g -O2  -fPIC -DPIC conftest.cpp >&5
configure:12930: $? = 0
configure:12943: result: yes
configure:12971: checking if g++ static flag -static works
configure:12999: result: yes
configure:13009: checking if g++ supports -c -o file.o
configure:13030: g++ -c -g -O2  -o out/conftest2.o conftest.cpp >&5
configure:13034: $? = 0
configure:13056: result: yes
configure:13082: checking whether the g++ linker (/usr/bin/ld) supports shared libraries
configure:13108: result: yes
configure:13175: checking dynamic linker characteristics
configure:13727: result: GNU/Linux ld.so
configure:13751: checking how to hardcode library paths into programs
configure:13776: result: immediate
configure:14315: checking if libtool supports shared libraries
configure:14317: result: yes
configure:14320: checking whether to build shared libraries
configure:14340: result: yes
configure:14343: checking whether to build static libraries
configure:14347: result: yes
configure:14357: checking for gfortran option to produce PIC
configure:14589: result: -fPIC
configure:14597: checking if gfortran PIC flag -fPIC works
configure:14615: gfortran -c -g -O2 -fPIC conftest.f >&5
configure:14619: $? = 0
configure:14632: result: yes
configure:14660: checking if gfortran static flag -static works
configure:14688: result: yes
configure:14698: checking if gfortran supports -c -o file.o
configure:14719: gfortran -c -g -O2 -o out/conftest2.o conftest.f >&5
configure:14723: $? = 0
configure:14745: result: yes
configure:14771: checking whether the gfortran linker (/usr/bin/ld) supports shared libraries
configure:15739: result: yes
configure:15806: checking dynamic linker characteristics
configure:16358: result: GNU/Linux ld.so
configure:16382: checking how to hardcode library paths into programs
configure:16407: result: immediate
configure:20018: checking for ANSI C header files
configure:20186: result: yes
configure:20212: checking sys/sendfile.h usability
configure:20229: gcc -c -g -O2  conftest.c >&5
configure:20236: $? = 0
configure:20250: result: yes
configure:20254: checking sys/sendfile.h presence
configure:20269: gcc -E  conftest.c
configure:20276: $? = 0
configure:20290: result: yes
configure:20323: checking for sys/sendfile.h
configure:20332: result: yes
configure:20349: checking for MD5_Init in -lcrypto
configure:20384: gcc -o conftest -g -O2   conftest.c -lcrypto   >&5
configure:20391: $? = 0
configure:20412: result: yes
configure:20418: checking for SSL_new in -lssl
configure:20453: gcc -o conftest -g -O2   conftest.c -lssl   >&5
configure:20460: $? = 0
configure:20481: result: yes
configure:20487: checking for event_base_new in -levent
configure:20522: gcc -o conftest -g -O2   conftest.c -levent   >&5
/usr/bin/ld: cannot find -levent
collect2: ld returned 1 exit status
configure:20529: $? = 1
configure: failed program was:
| /* confdefs.h.  */
| #define PACKAGE_NAME "chunkd"
| #define PACKAGE_TARNAME "chunkd"
| #define PACKAGE_VERSION "0.3"
| #define PACKAGE_STRING "chunkd 0.3"
| #define PACKAGE_BUGREPORT "http://www.kernel.org/pub/software/network/distsrv/"
| #define PACKAGE "chunkd"
| #define VERSION "0.3"
| #define STDC_HEADERS 1
| #define HAVE_SYS_TYPES_H 1
| #define HAVE_SYS_STAT_H 1
| #define HAVE_STDLIB_H 1
| #define HAVE_STRING_H 1
| #define HAVE_MEMORY_H 1
| #define HAVE_STRINGS_H 1
| #define HAVE_INTTYPES_H 1
| #define HAVE_STDINT_H 1
| #define HAVE_UNISTD_H 1
| #define HAVE_DLFCN_H 1
| #define STDC_HEADERS 1
| #define HAVE_SYS_SENDFILE_H 1
| /* end confdefs.h.  */
| 
| /* Override any GCC internal prototype to avoid an error.
|    Use char because int might match the return type of a GCC
|    builtin and then its argument prototype would still apply.  */
| #ifdef __cplusplus
| extern "C"
| #endif
| char event_base_new ();
| int
| main ()
| {
| return event_base_new ();
|   ;
|   return 0;
| }
configure:20550: result: no

## ---------------- ##
## Cache variables. ##
## ---------------- ##

ac_cv_build=i686-pc-linux-gnu
ac_cv_c_compiler_gnu=yes
ac_cv_cxx_compiler_gnu=yes
ac_cv_env_CCC_set=
ac_cv_env_CCC_value=
ac_cv_env_CC_set=
ac_cv_env_CC_value=
ac_cv_env_CFLAGS_set=
ac_cv_env_CFLAGS_value=
ac_cv_env_CPPFLAGS_set=
ac_cv_env_CPPFLAGS_value=
ac_cv_env_CPP_set=
ac_cv_env_CPP_value=
ac_cv_env_CXXCPP_set=
ac_cv_env_CXXCPP_value=
ac_cv_env_CXXFLAGS_set=
ac_cv_env_CXXFLAGS_value=
ac_cv_env_CXX_set=
ac_cv_env_CXX_value=
ac_cv_env_F77_set=
ac_cv_env_F77_value=
ac_cv_env_FFLAGS_set=
ac_cv_env_FFLAGS_value=
ac_cv_env_LDFLAGS_set=
ac_cv_env_LDFLAGS_value=
ac_cv_env_LIBS_set=
ac_cv_env_LIBS_value=
ac_cv_env_PKG_CONFIG_set=
ac_cv_env_PKG_CONFIG_value=
ac_cv_env_build_alias_set=
ac_cv_env_build_alias_value=
ac_cv_env_host_alias_set=
ac_cv_env_host_alias_value=
ac_cv_env_target_alias_set=
ac_cv_env_target_alias_value=
ac_cv_f77_compiler_gnu=yes
ac_cv_header_dlfcn_h=yes
ac_cv_header_inttypes_h=yes
ac_cv_header_memory_h=yes
ac_cv_header_stdc=yes
ac_cv_header_stdint_h=yes
ac_cv_header_stdlib_h=yes
ac_cv_header_string_h=yes
ac_cv_header_strings_h=yes
ac_cv_header_sys_sendfile_h=yes
ac_cv_header_sys_stat_h=yes
ac_cv_header_sys_types_h=yes
ac_cv_header_unistd_h=yes
ac_cv_host=i686-pc-linux-gnu
ac_cv_lib_crypto_MD5_Init=yes
ac_cv_lib_event_event_base_new=no
ac_cv_lib_ssl_SSL_new=yes
ac_cv_objext=o
ac_cv_path_EGREP='/bin/grep -E'
ac_cv_path_GREP=/bin/grep
ac_cv_path_install='/usr/bin/install -c'
ac_cv_path_mkdir=/bin/mkdir
ac_cv_prog_AWK=gawk
ac_cv_prog_CPP='gcc -E'
ac_cv_prog_CXXCPP='g++ -E'
ac_cv_prog_ac_ct_AR=ar
ac_cv_prog_ac_ct_CC=gcc
ac_cv_prog_ac_ct_CXX=g++
ac_cv_prog_ac_ct_F77=gfortran
ac_cv_prog_ac_ct_RANLIB=ranlib
ac_cv_prog_ac_ct_STRIP=strip
ac_cv_prog_cc_c89=
ac_cv_prog_cc_g=yes
ac_cv_prog_cxx_g=yes
ac_cv_prog_f77_g=yes
ac_cv_prog_gcc_traditional=no
ac_cv_prog_make_make_set=yes
am_cv_CC_dependencies_compiler_type=gcc3
am_cv_CXX_dependencies_compiler_type=gcc3
lt_cv_deplibs_check_method=pass_all
lt_cv_file_magic_cmd='$MAGIC_CMD'
lt_cv_file_magic_test_file=
lt_cv_ld_reload_flag=-r
lt_cv_objdir=.libs
lt_cv_path_LD=/usr/bin/ld
lt_cv_path_LDCXX=/usr/bin/ld
lt_cv_path_NM='/usr/bin/nm -B'
lt_cv_path_SED=/bin/sed
lt_cv_prog_compiler_c_o=yes
lt_cv_prog_compiler_c_o_CXX=yes
lt_cv_prog_compiler_c_o_F77=yes
lt_cv_prog_compiler_pic_works=yes
lt_cv_prog_compiler_pic_works_CXX=yes
lt_cv_prog_compiler_pic_works_F77=yes
lt_cv_prog_compiler_rtti_exceptions=no
lt_cv_prog_compiler_static_works=yes
lt_cv_prog_compiler_static_works_CXX=yes
lt_cv_prog_compiler_static_works_F77=yes
lt_cv_prog_gnu_ld=yes
lt_cv_prog_gnu_ldcxx=yes
lt_cv_sys_global_symbol_pipe='sed -n -e '\''s/^.*[ 	]\([ABCDGIRSTW][ABCDGIRSTW]*\)[ 	][ 	]*\([_A-Za-z][_A-Za-z0-9]*\)$/\1 \2 \2/p'\'''
lt_cv_sys_global_symbol_to_c_name_address='sed -n -e '\''s/^: \([^ ]*\) $/  {\"\1\", (lt_ptr) 0},/p'\'' -e '\''s/^[BCDEGRST] \([^ ]*\) \([^ ]*\)$/  {"\2", (lt_ptr) \&\2},/p'\'''
lt_cv_sys_global_symbol_to_cdecl='sed -n -e '\''s/^. .* \(.*\)$/extern int \1;/p'\'''
lt_cv_sys_lib_dlsearch_path_spec='/usr/lib /lib /usr/lib/mysql /usr/lib/qt-3.3/lib /usr/lib/xulrunner-1.9 /opt/qt/lib '
lt_cv_sys_lib_search_path_spec='/usr/lib /lib /usr/local/lib'
lt_cv_sys_max_cmd_len=1966080
lt_lt_cv_prog_compiler_c_o='"yes"'
lt_lt_cv_prog_compiler_c_o_CXX='"yes"'
lt_lt_cv_prog_compiler_c_o_F77='"yes"'
lt_lt_cv_sys_global_symbol_pipe='"sed -n -e '\''s/^.*[ 	]\\([ABCDGIRSTW][ABCDGIRSTW]*\\)[ 	][ 	]*\\([_A-Za-z][_A-Za-z0-9]*\\)\$/\\1 \\2 \\2/p'\''"'
lt_lt_cv_sys_global_symbol_to_c_name_address='"sed -n -e '\''s/^: \\([^ ]*\\) \$/  {\\\"\\1\\\", (lt_ptr) 0},/p'\'' -e '\''s/^[BCDEGRST] \\([^ ]*\\) \\([^ ]*\\)\$/  {\"\\2\", (lt_ptr) \\&\\2},/p'\''"'
lt_lt_cv_sys_global_symbol_to_cdecl='"sed -n -e '\''s/^. .* \\(.*\\)\$/extern int \\1;/p'\''"'

## ----------------- ##
## Output variables. ##
## ----------------- ##

ACLOCAL='${SHELL} /home/_/Desktop/CloudComputing/Hail/chunkd-0.3/missing --run aclocal-1.10'
AMDEPBACKSLASH='\'
AMDEP_FALSE='#'
AMDEP_TRUE=''
AMTAR='${SHELL} /home/_/Desktop/CloudComputing/Hail/chunkd-0.3/missing --run tar'
AR='ar'
ARGP_LIBS=''
AUTOCONF='${SHELL} /home/_/Desktop/CloudComputing/Hail/chunkd-0.3/missing --run autoconf'
AUTOHEADER='${SHELL} /home/_/Desktop/CloudComputing/Hail/chunkd-0.3/missing --run autoheader'
AUTOMAKE='${SHELL} /home/_/Desktop/CloudComputing/Hail/chunkd-0.3/missing --run automake-1.10'
AWK='gawk'
CC='gcc'
CCDEPMODE='depmode=gcc3'
CFLAGS='-g -O2'
CPP='gcc -E'
CPPFLAGS=''
CRYPTO_LIBS='-lcrypto'
CXX='g++'
CXXCPP='g++ -E'
CXXDEPMODE='depmode=gcc3'
CXXFLAGS='-g -O2'
CYGPATH_W='echo'
DEFS=''
DEPDIR='.deps'
DSYMUTIL=''
ECHO='echo'
ECHO_C=''
ECHO_N='-n'
ECHO_T=''
EGREP='/bin/grep -E'
EVENT_LIBS=''
EXEEXT=''
F77='gfortran'
FFLAGS='-g -O2'
GLIB_CFLAGS=''
GLIB_GENMARSHAL=''
GLIB_LIBS=''
GLIB_MKENUMS=''
GOBJECT_QUERY=''
GREP='/bin/grep'
INSTALL_DATA='${INSTALL} -m 644'
INSTALL_PROGRAM='${INSTALL}'
INSTALL_SCRIPT='${INSTALL}'
INSTALL_STRIP_PROGRAM='$(install_sh) -c -s'
LDFLAGS=''
LIBCHUNKDC_AGE='100'
LIBCHUNKDC_CURRENT='100'
LIBCHUNKDC_REVISION='1'
LIBOBJS=''
LIBS=''
LIBTOOL='$(SHELL) $(top_builddir)/libtool'
LN_S='ln -s'
LTLIBOBJS=''
MAINT='#'
MAINTAINER_MODE_FALSE=''
MAINTAINER_MODE_TRUE='#'
MAKEINFO='${SHELL} /home/_/Desktop/CloudComputing/Hail/chunkd-0.3/missing --run makeinfo'
MKDIR_P='/bin/mkdir -p'
NMEDIT=''
OBJEXT='o'
PACKAGE='chunkd'
PACKAGE_BUGREPORT='http://www.kernel.org/pub/software/network/distsrv/'
PACKAGE_NAME='chunkd'
PACKAGE_STRING='chunkd 0.3'
PACKAGE_TARNAME='chunkd'
PACKAGE_VERSION='0.3'
PATH_SEPARATOR=':'
PKG_CONFIG=''
RANLIB='ranlib'
SED='/bin/sed'
SET_MAKE=''
SHELL='/bin/sh'
SOCKET_LIBS=''
SSL_LIBS='-lssl'
STRIP='strip'
VERSION='0.3'
XML2_CONFIG=''
XML_CPPFLAGS=''
XML_LIBS=''
ac_ct_CC='gcc'
ac_ct_CXX='g++'
ac_ct_F77='gfortran'
am__fastdepCC_FALSE='#'
am__fastdepCC_TRUE=''
am__fastdepCXX_FALSE='#'
am__fastdepCXX_TRUE=''
am__include='include'
am__isrc=''
am__leading_dot='.'
am__quote=''
am__tar='${AMTAR} chof - "$$tardir"'
am__untar='${AMTAR} xf -'
bindir='${exec_prefix}/bin'
build='i686-pc-linux-gnu'
build_alias=''
build_cpu='i686'
build_os='linux-gnu'
build_vendor='pc'
datadir='${datarootdir}'
datarootdir='${prefix}/share'
docdir='${datarootdir}/doc/${PACKAGE_TARNAME}'
dvidir='${docdir}'
exec_prefix='NONE'
host='i686-pc-linux-gnu'
host_alias=''
host_cpu='i686'
host_os='linux-gnu'
host_vendor='pc'
htmldir='${docdir}'
includedir='${prefix}/include'
infodir='${datarootdir}/info'
install_sh='$(SHELL) /home/_/Desktop/CloudComputing/Hail/chunkd-0.3/install-sh'
libdir='${exec_prefix}/lib'
libexecdir='${exec_prefix}/libexec'
localedir='${datarootdir}/locale'
localstatedir='${prefix}/var'
mandir='${datarootdir}/man'
mkdir_p='/bin/mkdir -p'
oldincludedir='/usr/include'
pdfdir='${docdir}'
prefix='NONE'
program_transform_name='s,x,x,'
psdir='${docdir}'
sbindir='${exec_prefix}/sbin'
sharedstatedir='${prefix}/com'
sysconfdir='${prefix}/etc'
target_alias=''

## ----------- ##
## confdefs.h. ##
## ----------- ##

#define PACKAGE_NAME "chunkd"
#define PACKAGE_TARNAME "chunkd"
#define PACKAGE_VERSION "0.3"
#define PACKAGE_STRING "chunkd 0.3"
#define PACKAGE_BUGREPORT "http://www.kernel.org/pub/software/network/distsrv/"
#define PACKAGE "chunkd"
#define VERSION "0.3"
#define STDC_HEADERS 1
#define HAVE_SYS_TYPES_H 1
#define HAVE_SYS_STAT_H 1
#define HAVE_STDLIB_H 1
#define HAVE_STRING_H 1
#define HAVE_MEMORY_H 1
#define HAVE_STRINGS_H 1
#define HAVE_INTTYPES_H 1
#define HAVE_STDINT_H 1
#define HAVE_UNISTD_H 1
#define HAVE_DLFCN_H 1
#define STDC_HEADERS 1
#define HAVE_SYS_SENDFILE_H 1

configure: exit 1

[-- Attachment #3: yum.list.gz --]
[-- Type: application/x-gzip, Size: 128845 bytes --]

Re: [Patch] chunkd: use port xxx82 to build

[Jeff Garzik]

Rick Peralta wrote:
>> We are definitely interested in the build failures, yes.
> 
> When builidng, chunkd fails in configure (so do the other Hail files).

According to your config.log, you are missing the libevent dependency. 
On Fedora, this will be "libevent-devel", which will also install the 
libevent base package.

configure intentionally stops when it cannot find something.

I wouldn't advise bothering with the tarballs, though.  clone and build 
each git repository, e.g.

$ git clone git://git.kernel.org/pub/scm/daemon/cld/cld.git
$ cd cld
$ ./autogen.sh
$ CFLAGS="-O2 -Wall -g" ./configure

Because development is moving very rapidly, it is best for developers to 
always look at the git repository for the latest source code.

configure gives much more informative errors in the latest upstream (== 
current source code in git repository), so this problem should be gone 
in chunkd version 0.4, when released.


> Running: Linux version 2.6.27.25-78.2.56.fc9.i686 (mockbuild@) (gcc version 4.3.0 20080428 (Red Hat 4.3.0-8) (GCC) ) #1 SMP Thu Jun 18 12:47:50 EDT 2009 Fedora release 9 (Sulphur)
> Default install with all updates complete, all gcc and make updates
> 
> I asked a friend to run the build on a different distribution and it failed differently.
> 
>> Have you verified that all necessary dependencies are installed?
> 
> It is not clear what needs to be installed in order to build... is there a list?

Here is the list from pkg/chunkd.spec:

BuildRequires:  libevent-devel glib2-devel openssl-devel zlib-devel
BuildRequires:  libxml2-devel procps cld cld-devel

Regards,

	Jeff

Re: [Patch] chunkd: use port xxx82 to build

[Rick Peralta]

Hi Jeff, Pete and all,

I've been called out of town and will not have time to look at the Hail stuff for a while.

Re: the endian issues, a quick look suggests that most of the code is OK and that the structs that are passed over the network need to be put into a standard endian format.  There seem to be only two structs that need attention.  It is less clear if there is a single place to make the byte order fix.

Note: The Hail builds are not working for me.  I've tried running a couple of difference distributions and they all fail in configure.  Running usual upgrades does not correct the problem.  Digging into the scripts a bit shows they generally fail due to library issues or what appear to be related to compiler issues (test... ac_nonexistent.h).  If there is interest,  I can forward the log files.

 - Rick

>>> Speaking of chunkd...
>>>
>>> I don't think I will have time this week to fix chunkd's failure on big 
>>> endian machines.  If you (or anyone else listening...) have an 
>>> opportunity to look into that, that would be most helpful.
>> 
>> This looks like an ideal opportunity for Rick to read the code
>> with a specific goal in mind, if he doesn't mind me saying so.

Re: [Patch] chunkd: use port xxx82 to build

[Fabian Deutsch]

Rick,

Am Freitag, den 07.08.2009, 10:11 -0400 schrieb Rick Peralta:
> Note: The Hail builds are not working for me.  I've tried running a
> couple of difference distributions and they all fail in configure.
> Running usual upgrades does not correct the problem.  Digging into the
> scripts a bit shows they generally fail due to library issues or what
> appear to be related to compiler issues (test... ac_nonexistent.h).
> If there is interest,  I can forward the log files.


Would be nice if you can post those logiles somewhere. 

Cheers
- fabian

Re: [Patch] chunkd: use port xxx82 to build

[Jeff Garzik]

Rick Peralta wrote:
> Hi Jeff, Pete and all,
> 
> I've been called out of town and will not have time to look at the Hail stuff for a while.
> 
> Re: the endian issues, a quick look suggests that most of the code is OK and that the structs that are passed over the network need to be put into a standard endian format.  There seem to be only two structs that need attention.  It is less clear if there is a single place to make the byte order fix.

Specifically which two structs need attention?


> Note: The Hail builds are not working for me.  I've tried running a couple of difference distributions and they all fail in configure.  Running usual upgrades does not correct the problem.  Digging into the scripts a bit shows they generally fail due to library issues or what appear to be related to compiler issues (test... ac_nonexistent.h).  If there is interest,  I can forward the log files.

Have you verified that all necessary dependencies are installed?

We are definitely interested in the build failures, yes.

	Jeff

Re: [Patch] chunkd: use port xxx82 to build

[Pete Zaitcev]

On Tue, 04 Aug 2009 21:40:22 -0400, Jeff Garzik <jeff@garzik.org> wrote:

> Another avenue is to sign up for a Fedora Account 
> (http://fedoraproject.org/wiki/Join) and get access to scratch builds 
> via koji (https://fedoraproject.org/wiki/Using_the_Koji_build_system).

Chunk needs applog() treatment before Koji becomes useful for debugging.

BTW, Rick, please don't drop the list off CC: unless we discuss
something R34LLY S1KKR1T.

-- Pete

Re: [Patch] chunkd: use port xxx82 to build

[Jeff Garzik]

Pete Zaitcev wrote:
> On Tue, 04 Aug 2009 21:40:22 -0400, Jeff Garzik <jeff@garzik.org> wrote:
> 
>> Another avenue is to sign up for a Fedora Account 
>> (http://fedoraproject.org/wiki/Join) and get access to scratch builds 
>> via koji (https://fedoraproject.org/wiki/Using_the_Koji_build_system).
> 
> Chunk needs applog() treatment before Koji becomes useful for debugging.

huhwhuh?  :)

[jgarzik@bd chunkd]$ grep -wc applog server/*.[ch]
server/be-fs.c:26
server/chunkd.h:1
server/cldu.c:32
server/config.c:31
server/object.c:4
server/server.c:23
server/util.c:8


> BTW, Rick, please don't drop the list off CC: unless we discuss
> something R34LLY S1KKR1T.

Agreed.

	Jeff

[Patch] chunkd: use port xxx82 to build

[Pete Zaitcev]

Let's change build ports to xxx82 to deconflict with tabled.

Signed-Off-By: Pete Zaitcev <zaitcev@redhat.com>

diff --git a/test/server-test.cfg b/test/server-test.cfg
index 38fcc43..02c39d4 100644
--- a/test/server-test.cfg
+++ b/test/server-test.cfg
@@ -5,11 +5,11 @@
 </SSL>
 
 <Listen>
-	<Port>18080</Port>
+	<Port>18082</Port>
 </Listen>
 
 <Listen>
-	<Port>28080</Port>
+	<Port>28082</Port>
 	<Encrypt>true</Encrypt>
 </Listen>
 
diff --git a/test/test.h b/test/test.h
index 46667aa..d5407d4 100644
--- a/test/test.h
+++ b/test/test.h
@@ -9,8 +9,8 @@
 #define TEST_HOST "localhost"
 #define TEST_USER "testuser"
 #define TEST_USER_KEY "testuser"
-#define TEST_PORT 18080
-#define TEST_SSL_PORT 28080
+#define TEST_PORT 18082
+#define TEST_SSL_PORT 28082
 
 #define OK(expr)				\
 	do {					\

Re: [Patch] chunkd: use port xxx82 to build

[Jeff Garzik]

Pete Zaitcev wrote:
> Let's change build ports to xxx82 to deconflict with tabled.
> 
> Signed-Off-By: Pete Zaitcev <zaitcev@redhat.com>
> 
> diff --git a/test/server-test.cfg b/test/server-test.cfg
> index 38fcc43..02c39d4 100644
> --- a/test/server-test.cfg
> +++ b/test/server-test.cfg
> @@ -5,11 +5,11 @@
>  </SSL>
>  
>  <Listen>
> -	<Port>18080</Port>
> +	<Port>18082</Port>
>  </Listen>
>  
>  <Listen>
> -	<Port>28080</Port>
> +	<Port>28082</Port>
>  	<Encrypt>true</Encrypt>
>  </Listen>
>  
> diff --git a/test/test.h b/test/test.h
> index 46667aa..d5407d4 100644
> --- a/test/test.h
> +++ b/test/test.h
> @@ -9,8 +9,8 @@
>  #define TEST_HOST "localhost"
>  #define TEST_USER "testuser"
>  #define TEST_USER_KEY "testuser"
> -#define TEST_PORT 18080
> -#define TEST_SSL_PORT 28080
> +#define TEST_PORT 18082
> +#define TEST_SSL_PORT 28082

Speaking of conflicts... this patch does not apply to current upstream 
chunkd.

I manually fixed it up and applied it, working around recent changes in 
test/test.h

	Jeff

Re: [Patch] chunkd: use port xxx82 to build

[Jeff Garzik]

Speaking of chunkd...

I don't think I will have time this week to fix chunkd's failure on big 
endian machines.  If you (or anyone else listening...) have an 
opportunity to look into that, that would be most helpful.

	Jeff

Re: [Patch] chunkd: use port xxx82 to build

[Pete Zaitcev]

On Tue, 04 Aug 2009 21:00:36 -0400, Jeff Garzik <jeff@garzik.org> wrote:

> Speaking of chunkd...
> 
> I don't think I will have time this week to fix chunkd's failure on big 
> endian machines.  If you (or anyone else listening...) have an 
> opportunity to look into that, that would be most helpful.

This looks like an ideal opportunity for Rick to read the code
with a specific goal in mind, if he doesn't mind me saying so.

Do we have a bug number for it?

-- Pete

Re: [Patch] chunkd: use port xxx82 to build

[Jeff Garzik]

Pete Zaitcev wrote:
> On Tue, 04 Aug 2009 21:00:36 -0400, Jeff Garzik <jeff@garzik.org> wrote:
> 
>> Speaking of chunkd...
>>
>> I don't think I will have time this week to fix chunkd's failure on big 
>> endian machines.  If you (or anyone else listening...) have an 
>> opportunity to look into that, that would be most helpful.
> 
> This looks like an ideal opportunity for Rick to read the code
> with a specific goal in mind, if he doesn't mind me saying so.

CC'd :)


> Do we have a bug number for it?

https://bugzilla.redhat.com/show_bug.cgi?id=514651

	Jeff

Re: [Patch] chunkd: use port xxx82 to build

[Pete Zaitcev]

On Tue, 04 Aug 2009 20:57:11 -0400, Jeff Garzik <jeff@garzik.org> wrote:

> Speaking of conflicts... this patch does not apply to current upstream 
> chunkd.

This is quite strange, but indeed I just pulled and testuser2 appeared.
Sorry, this slipped somehow.

-- Pete