New 'hail' repository created, with major packaging rework

New 'hail' repository created, with major packaging rework

[Jeff Garzik]

A new git repository

	git://git.kernel.org/pub/scm/daemon/distsrv/hail.git

was created, preserving the full histories of cld.git and chunkd.git. 
The existing cld.git and chunkd.git repositories have been left 
untouched, for now.  I also have not yet updated tabled.git for this new 
work, though it should be an easy matter of linking against libhail 
rather than other libs.

This new repository creates hail-$VERSION.tar.gz tarballs via "make 
distcheck", producing libhail, cld and chunkd binaries.

libhail is a single shared library binary, linking together cldc, ncld, 
libtimer, and chunkdc modules.  In other words, libhail at present is a 
simplistic combination of cld/lib and chunkd/lib.

The RPM package specfile has been updated (pkg/hail.spec) to generate 
the following complement of packages on Fedora:

Wrote: /garz/rpm/SRPMS/hail-0.7-0.1.gc69acd63.fc12.src.rpm
Wrote: /garz/rpm/RPMS/x86_64/hail-0.7-0.1.gc69acd63.fc12.x86_64.rpm
	-> contains libhail
Wrote: /garz/rpm/RPMS/x86_64/hail-cld-0.7-0.1.gc69acd63.fc12.x86_64.rpm
	-> contains cld
Wrote: /garz/rpm/RPMS/x86_64/hail-chunkd-0.7-0.1.gc69acd63.fc12.x86_64.rpm
	-> contains chunkd
Wrote: /garz/rpm/RPMS/x86_64/hail-devel-0.7-0.1.gc69acd63.fc12.x86_64.rpm
	-> contains libhail devel libs, headers
Wrote: 
/garz/rpm/RPMS/x86_64/hail-debuginfo-0.7-0.1.gc69acd63.fc12.x86_64.rpm


rpmlint still issues several warnings about hail-cld and hail-chunkd 
packages.  That must be fixed before this package suite rename can be 
submitted to Fedora (pkg renames must be submitted as new packages, and 
go through the pkg review process all over again).

To produce hail*.rpm packages on Fedora, I would do something like this:

1) set up rpm build directories (== $RBD in this example)
2) git clone git://git.kernel.org/pub/scm/daemon/distsrv/hail.git
3) cd hail
4) ./autogen.sh
5) ./autogen.sh
6) ./configure
7) make -s dist
8) cp *.tar.gz pkg/*.init pkg/*.sysconf $RBD/SOURCES
9) cp pkg/hail.spec $RBD/SPECS
10) cd $RBD
11) rpmbuild -ba SPECS/hail.spec


As mentioned above, the {cld,chunkd}.git repositories have been left 
untouched, so if something goes wildly wrong with this scheme, we can 
easily backtrack.

Comments welcome.

	Jeff

Re: New 'hail' repository created, with major packaging rework

[Pete Zaitcev]

On Fri, 02 Jul 2010 02:59:20 -0400
Jeff Garzik <jeff@garzik.org> wrote:

> 4) ./autogen.sh
> 5) ./autogen.sh

One worked for me. Trying to pretend it's nothing :-)

> 11) rpmbuild -ba SPECS/hail.spec

Seems working perfectly, but there's one thing:

Wrote: /q/zaitcev/rpms/SRPMS/hail-0.8-0.1.z1.fc13.src.rpm
Wrote: /q/zaitcev/rpms/RPMS/x86_64/hail-0.8-0.1.z1.fc13.x86_64.rpm
Wrote: /q/zaitcev/rpms/RPMS/x86_64/cld-0.8-0.1.z1.fc13.x86_64.rpm
Wrote: /q/zaitcev/rpms/RPMS/x86_64/chunkd-0.8-0.1.z1.fc13.x86_64.rpm
Wrote: /q/zaitcev/rpms/RPMS/x86_64/hail-devel-0.8-0.1.z1.fc13.x86_64.rpm
Wrote: /q/zaitcev/rpms/RPMS/x86_64/hail-debuginfo-0.8-0.1.z1.fc13.x86_64.rpm

The debuginfos for cld and chunkd are not produced for some reason.

-- Pete

Re: New 'hail' repository created, with major packaging rework

[Jeff Garzik]

On 07/05/2010 02:31 PM, Pete Zaitcev wrote:
> On Fri, 02 Jul 2010 02:59:20 -0400
> Jeff Garzik<jeff@garzik.org>  wrote:
>
>> 4) ./autogen.sh
>> 5) ./autogen.sh
>
> One worked for me. Trying to pretend it's nothing :-)
>
>> 11) rpmbuild -ba SPECS/hail.spec
>
> Seems working perfectly, but there's one thing:
>
> Wrote: /q/zaitcev/rpms/SRPMS/hail-0.8-0.1.z1.fc13.src.rpm
> Wrote: /q/zaitcev/rpms/RPMS/x86_64/hail-0.8-0.1.z1.fc13.x86_64.rpm
> Wrote: /q/zaitcev/rpms/RPMS/x86_64/cld-0.8-0.1.z1.fc13.x86_64.rpm
> Wrote: /q/zaitcev/rpms/RPMS/x86_64/chunkd-0.8-0.1.z1.fc13.x86_64.rpm
> Wrote: /q/zaitcev/rpms/RPMS/x86_64/hail-devel-0.8-0.1.z1.fc13.x86_64.rpm
> Wrote: /q/zaitcev/rpms/RPMS/x86_64/hail-debuginfo-0.8-0.1.z1.fc13.x86_64.rpm
>
> The debuginfos for cld and chunkd are not produced for some reason.

Not produced...  or simply contained within hail-debuginfo?

Typically only one *-debuginfo.rpm is produced per SRPM, AFAIK.

	Jeff

Re: New 'hail' repository created, with major packaging rework

[Pete Zaitcev]

On Mon, 05 Jul 2010 14:37:48 -0400
Jeff Garzik <jeff@garzik.org> wrote:

> > Wrote: /q/zaitcev/rpms/RPMS/x86_64/hail-debuginfo-0.8-0.1.z1.fc13.x86_64.rpm
> >
> > The debuginfos for cld and chunkd are not produced for some reason.
> 
> Not produced...  or simply contained within hail-debuginfo?
> 
> Typically only one *-debuginfo.rpm is produced per SRPM, AFAIK.

Now I'm curious how abrtd finds them.

-- Pete

Re: New 'hail' repository created, with major packaging rework

[Pete Zaitcev]

On Fri, 02 Jul 2010 02:59:20 -0400
Jeff Garzik <jeff@garzik.org> wrote:

> 	git://git.kernel.org/pub/scm/daemon/distsrv/hail.git

> libhail is a single shared library binary, linking together cldc, ncld, 
> libtimer, and chunkdc modules.  In other words, libhail at present is a 
> simplistic combination of cld/lib and chunkd/lib.

[zaitcev@lembas hail-tip]$ ls lib include
include:
chunkc.h         chunksrv.h    cld-private.h  Makefile     ncld.h
chunk_msg.h      cldc.h        elist.h        Makefile.am  objcache.h
chunk-private.h  cld_common.h  hail_log.h     Makefile.in

lib:
chunkdc.c   cldc-udp.c     libhail.pc.in              Makefile
chunksrv.c  cld_msg_rpc.x  libhail-uninstalled.pc     Makefile.am
cldc.c      common.c       libhail-uninstalled.pc.in  Makefile.in
cldc-dns.c  libhail.pc     libtimer.c                 pkt.c
[zaitcev@lembas hail-tip]$ grep httpstor lib/*.c
[zaitcev@lembas hail-tip]$ 

What has happened to the plan to include httpstor into libhail?

-- Pete

Re: New 'hail' repository created, with major packaging rework

[Jeff Garzik]

On 07/05/2010 03:13 PM, Pete Zaitcev wrote:
> On Fri, 02 Jul 2010 02:59:20 -0400
> Jeff Garzik<jeff@garzik.org>  wrote:
>
>> 	git://git.kernel.org/pub/scm/daemon/distsrv/hail.git
>
>> libhail is a single shared library binary, linking together cldc, ncld,
>> libtimer, and chunkdc modules.  In other words, libhail at present is a
>> simplistic combination of cld/lib and chunkd/lib.
>
> [zaitcev@lembas hail-tip]$ ls lib include
> include:
> chunkc.h         chunksrv.h    cld-private.h  Makefile     ncld.h
> chunk_msg.h      cldc.h        elist.h        Makefile.am  objcache.h
> chunk-private.h  cld_common.h  hail_log.h     Makefile.in
>
> lib:
> chunkdc.c   cldc-udp.c     libhail.pc.in              Makefile
> chunksrv.c  cld_msg_rpc.x  libhail-uninstalled.pc     Makefile.am
> cldc.c      common.c       libhail-uninstalled.pc.in  Makefile.in
> cldc-dns.c  libhail.pc     libtimer.c                 pkt.c
> [zaitcev@lembas hail-tip]$ grep httpstor lib/*.c
> [zaitcev@lembas hail-tip]$
>
> What has happened to the plan to include httpstor into libhail?

Still planned, and can easily be done.  Important first step was getting 
the foundation laid -- creating hail.git, and synchronizing hail.git and 
tabled.git, and associated RPM packaging.

Moving libhttpstor is now a simple matter of simultaneous commits to 
hail.git and tabled.git, moving the code and updating build machinery.

I can release a hail 0.7.1 and tabled 0.5.1 with this change, if you 
feel versioning and pushing out this libhttpstor change is highly important.

(or you can do that yourself, doesn't make a difference to me)

	Jeff

Re: New 'hail' repository created, with major packaging rework

[Pete Zaitcev]

On Mon, 05 Jul 2010 15:22:40 -0400
Jeff Garzik <jeff@garzik.org> wrote:

> > What has happened to the plan to include httpstor into libhail?
> 
> Still planned, and can easily be done.  Important first step was getting 
> the foundation laid -- creating hail.git, and synchronizing hail.git and 
> tabled.git, and associated RPM packaging.

Thanks, I understand.

> I can release a hail 0.7.1 and tabled 0.5.1 with this change, if you 
> feel versioning and pushing out this libhttpstor change is highly important.

Well, I dunno, Darcy is the user of it but I don't know if he
uses released RPMs or pulls from git. If he's fine with git, we
may continue to procrastinate about releases.

I rebuilt all the stuff I have here and it seems to work.
I even blew out chunkds one at a time to account for the format
incompatibility that you mentioned and let tabled to rebuild them.

-- Pete

Re: New 'hail' repository created, with major packaging rework

[Jeff Darcy]

On 07/05/2010 03:57 PM, Pete Zaitcev wrote:
>> I can release a hail 0.7.1 and tabled 0.5.1 with this change, if you 
>> feel versioning and pushing out this libhttpstor change is highly important.
> 
> Well, I dunno, Darcy is the user of it but I don't know if he
> uses released RPMs or pulls from git. If he's fine with git, we
> may continue to procrastinate about releases.

I'm comfortable pulling stuff from git right now, but as my own stuff
gets pushed out more publicly I'll probably need to clean up some of the
packaging and dependencies.

Re: New 'hail' repository created, with major packaging rework

[Pete Zaitcev]

On Mon, 05 Jul 2010 15:22:40 -0400
Jeff Garzik <jeff@garzik.org> wrote:

> Moving libhttpstor is now a simple matter of simultaneous commits to 
> hail.git and tabled.git, moving the code and updating build machinery.

BTW, I suggest we do it differently: rename the functions and
the struct httpstor as they are introduced in libhail (without
changing anything else, to prevent accidential regressions).
This way, tabled and our out-of-tree tests can continue to build
for a couple of days and smoothly switch over to new libraries.

BTW 2, what is the license? I am wondering if we should adopt
a "Lesser GPL" for libhail. I heard some unhappy comments about
Fefe shipping libraries with GPL, and not necesserily from proprietary
vendors. There was some stuff wrt linking perfectly free code with MIT
license or something, but I do not remember details.

-- Pete

Re: New 'hail' repository created, with major packaging rework

[Jeff Garzik]

On 07/06/2010 11:24 AM, Pete Zaitcev wrote:
> On Mon, 05 Jul 2010 15:22:40 -0400
> Jeff Garzik<jeff@garzik.org>  wrote:
>
>> Moving libhttpstor is now a simple matter of simultaneous commits to
>> hail.git and tabled.git, moving the code and updating build machinery.
>
> BTW, I suggest we do it differently: rename the functions and
> the struct httpstor as they are introduced in libhail (without
> changing anything else, to prevent accidential regressions).
> This way, tabled and our out-of-tree tests can continue to build
> for a couple of days and smoothly switch over to new libraries.

To what should the functions be renamed?

We will want to copy over the req_sign* stuff as well as httpstor stuff, 
otherwise it's not a full S3 lib.


> BTW 2, what is the license? I am wondering if we should adopt
> a "Lesser GPL" for libhail. I heard some unhappy comments about
> Fefe shipping libraries with GPL, and not necesserily from proprietary
> vendors. There was some stuff wrt linking perfectly free code with MIT
> license or something, but I do not remember details.

A fair point.  It's GPL for the whole package due to laziness, which 
means linking apps MUST be GPL as well.  Even some open sourcers 
complain at this restriction, when it appears in other projects.

LGPL or MIT is fine with me, for libhail.

	Jeff

Re: New 'hail' repository created, with major packaging rework

[Jeff Garzik]

On 07/06/2010 11:24 AM, Pete Zaitcev wrote:
> On Mon, 05 Jul 2010 15:22:40 -0400
> Jeff Garzik<jeff@garzik.org>  wrote:
>
>> Moving libhttpstor is now a simple matter of simultaneous commits to
>> hail.git and tabled.git, moving the code and updating build machinery.
>
> BTW, I suggest we do it differently: rename the functions and
> the struct httpstor as they are introduced in libhail (without
> changing anything else, to prevent accidential regressions).
> This way, tabled and our out-of-tree tests can continue to build
> for a couple of days and smoothly switch over to new libraries.

OK, just pushed the following out to hail.git.  If people disagree with 
naming, now's the time to speak up.


commit 5188f48dd3c73ce86f2bc453a326ee0bf40fd6db
Author: Jeff Garzik <jeff@garzik.org>
Date:   Wed Jul 7 02:16:28 2010 -0400

     libhail: Import httpstor, httputil modules from tabled

     With the following transformations:

     s/req_/hreq_/
     s/httpstor_/hstor_/
     s//huri_/
     s//hutil_/

     Signed-off-by: Jeff Garzik <jgarzik@redhat.com>