RPM issues

RPM issues

[Colin McCabe]

Hi all,

 We have made some progress recently on getting a Ceph RPM that works
on CentOS 5.5 and other Red Hat-derived distros.

Now that our version number no longer includes a tilde, and the spec
file has been pruned a bit, I think we have a shot at building this
cleanly.

I just have a few questions that perhaps people more experienced with
RPM packaging can answer:

1. should we have a debug SPEC file and a regular SPEC file? I think
in the long term we want to have both a Ceph and a Ceph-devel RPM
built.

2. What is the best way to bundle the python bindings? Currently I am
getting this error when building the RPM:
> error: Installed (but unpackaged) file(s) found:
> /usr/lib/python2.4/site-packages/rados.py
> /usr/lib/python2.4/site-packages/rados.pyc
> /usr/lib/python2.4/site-packages/rados.pyo
However, I need to find an RPM variable that has a path to the python
site-packages directory. automake knows this, but so far I haven't
figured out where this information lives in RPM.

This guy suggests a workaround:
http://www.linux-archive.org/centos/171366-python-sybase-centos-5-x86_64-a.html
But his workaround seems kind of clumsy, and in fact doesn't work for
me on CentOS 5.5. Surely there is a standard way to refer to the
sit-packages directory in RPM without hacks?

3. How should we handle tcmalloc, if at all? Google-perftools is not
bundled for 64 bit on CentOS. (It seems like this decision was made
because some of the stuff in this package is buggy on 64 bit x86.
However, tcmalloc itself is not buggy on 64 bit.). And in general, how
do we handle things that are "good to have" but which shouldn't be
dependencies?

cheers,
Colin

Re: RPM issues

[Ruben Kerkhof]

Hi Colin,

On Sat, Mar 19, 2011 at 01:29, Colin McCabe <cmccabe@alumni.cmu.edu> wrote:
> Hi all,
>
>  We have made some progress recently on getting a Ceph RPM that works
> on CentOS 5.5 and other Red Hat-derived distros.
>
> Now that our version number no longer includes a tilde, and the spec
> file has been pruned a bit, I think we have a shot at building this
> cleanly.
>
> I just have a few questions that perhaps people more experienced with
> RPM packaging can answer:
>
> 1. should we have a debug SPEC file and a regular SPEC file? I think
> in the long term we want to have both a Ceph and a Ceph-devel RPM
> built.

A single spec can produce multiple binary rpms, so this shouldn't be necessary.

>
> 2. What is the best way to bundle the python bindings? Currently I am
> getting this error when building the RPM:
>> error: Installed (but unpackaged) file(s) found:
>> /usr/lib/python2.4/site-packages/rados.py
>> /usr/lib/python2.4/site-packages/rados.pyc
>> /usr/lib/python2.4/site-packages/rados.pyo
> However, I need to find an RPM variable that has a path to the python
> site-packages directory. automake knows this, but so far I haven't
> figured out where this information lives in RPM.
>
> This guy suggests a workaround:
> http://www.linux-archive.org/centos/171366-python-sybase-centos-5-x86_64-a.html
> But his workaround seems kind of clumsy, and in fact doesn't work for
> me on CentOS 5.5. Surely there is a standard way to refer to the
> sit-packages directory in RPM without hacks?

There are standard macros in recent versions of Fedora and RHEL6, but
this should work:

%if ! (0%{?fedora} > 12 || 0%{?rhel} > 5)
%{!?python_sitelib: %global python_sitelib %(%{__python} -c "from
distutils.sysconfig import get_python_lib; print(get_python_lib())")}
%{!?python_sitearch: %global python_sitearch %(%{__python} -c "from
distutils.sysconfig import get_python_lib; print(get_python_lib(1))")}
%endif

If you add that to the top of your spec you can use %{python_sitearch}
in your %files section.
There are more tips at http://fedoraproject.org/wiki/Packaging:Python

> 3. How should we handle tcmalloc, if at all? Google-perftools is not
> bundled for 64 bit on CentOS. (It seems like this decision was made
> because some of the stuff in this package is buggy on 64 bit x86.
> However, tcmalloc itself is not buggy on 64 bit.).

Is google-perftool an EPEL package? If so can you report a bug against
that package at bugzilla.redhat.com?
An option would be to split out tcmalloc into it's own rpm.

> And in general, how
> do we handle things that are "good to have" but which shouldn't be
> dependencies?

Ah, that's one of the problems of RPM :) There's no such thing as
Recommends or Suggests.
Some people handle this by adding conditional macros to the spec file.
This enables you to rebuild the source rpm by specifying rpmbuild
--with python for example. The libvirt spec file is a good example of
this: http://pkgs.fedoraproject.org/gitweb/?p=libvirt.git;a=blob;f=libvirt.spec

> cheers,
> Colin

Kind regards,

Ruben
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: RPM issues

[Tommi Virtanen]

On Fri, Mar 18, 2011 at 05:29:51PM -0700, Colin McCabe wrote:
> 3. How should we handle tcmalloc, if at all? Google-perftools is not
> bundled for 64 bit on CentOS. (It seems like this decision was made
> because some of the stuff in this package is buggy on 64 bit x86.
> However, tcmalloc itself is not buggy on 64 bit.). And in general, how
> do we handle things that are "good to have" but which shouldn't be
> dependencies?

Hmm. The whole point of commit a2c02d was that it'd be harder to
accidentally build a non-desired variant of Ceph. That is, just
because you reinstalled the machine you use for building the package,
and forgot to install libatomic-ops-dev, doesn't mean you should
create packages with completely different behavior in the future.

So, you need to figure out whether libatomic-ops is desired or not,
for the relevant platform/architecture. If yes, then builds must fail
without it (./configure will complain, debs have build-dependency). If
not, then say ./configure --without-libatomic-ops. But don't let it
depend on unpredictable factors.

-- 
:(){ :|:&};:

Re: RPM issues

[Colin McCabe]

On Mon, Mar 21, 2011 at 9:09 AM, Tommi Virtanen
<tommi.virtanen@dreamhost.com> wrote:
> On Fri, Mar 18, 2011 at 05:29:51PM -0700, Colin McCabe wrote:
>> 3. How should we handle tcmalloc, if at all? Google-perftools is not
>> bundled for 64 bit on CentOS. (It seems like this decision was made
>> because some of the stuff in this package is buggy on 64 bit x86.
>> However, tcmalloc itself is not buggy on 64 bit.). And in general, how
>> do we handle things that are "good to have" but which shouldn't be
>> dependencies?
>
> Hmm. The whole point of commit a2c02d was that it'd be harder to
> accidentally build a non-desired variant of Ceph. That is, just
> because you reinstalled the machine you use for building the package,
> and forgot to install libatomic-ops-dev, doesn't mean you should
> create packages with completely different behavior in the future.
>
> So, you need to figure out whether libatomic-ops is desired or not,
> for the relevant platform/architecture. If yes, then builds must fail
> without it (./configure will complain, debs have build-dependency). If
> not, then say ./configure --without-libatomic-ops. But don't let it
> depend on unpredictable factors.

I agree that libatomic-ops should be mandatory.

The more annoying dependencies are things like GTK2, which don't
really affect the behavior of the rest of the project, but which will
have to be "mandatory" in the RPM, it seems. Also, as I said, tcmalloc
will present somewhat of a problem because it's just not packaged on
RedHat/CentOS at the moment for 64-bit. I will report a bug with Red
Hat and see what they think...

cheers,
Colin

Re: RPM issues

[Gregory Farnum]

On Mon, Mar 21, 2011 at 9:22 AM, Colin McCabe <cmccabe@alumni.cmu.edu> wrote:
> The more annoying dependencies are things like GTK2, which don't
> really affect the behavior of the rest of the project, but which will
> have to be "mandatory" in the RPM, it seems. Also, as I said, tcmalloc
> will present somewhat of a problem because it's just not packaged on
> RedHat/CentOS at the moment for 64-bit. I will report a bug with Red
> Hat and see what they think...
I don't think it's okay to require gtk2 just for the gceph that nobody
uses...That at least should be an easy choice to kick out if
necessary.
tcmalloc is harder; we should get one of our lab users to submit a
ticket for that. If necessary I believe google-perftools makes it
pretty easy to build tcmalloc-minimal, which includes only the memory
allocator -- we'd have to adjust our machinery a bit so the heap
checker hooks are separate from the detection of tcmalloc but it'd be
doable, and unlike the other config bits wouldn't fundamentally change
non-dev behaviors.

libatomic-ops is separate and while Lazslo has disabled that on armel
for the moment I believe I will be able to get that fixed up pretty
soon (just need to check up on my QEMU instance once I get in and
handle the autobuild bits).
-Greg

Re: RPM issues

[Sage Weil]

On Mon, 21 Mar 2011, Colin McCabe wrote:
> On Mon, Mar 21, 2011 at 9:09 AM, Tommi Virtanen
> <tommi.virtanen@dreamhost.com> wrote:
> > On Fri, Mar 18, 2011 at 05:29:51PM -0700, Colin McCabe wrote:
> >> 3. How should we handle tcmalloc, if at all? Google-perftools is not
> >> bundled for 64 bit on CentOS. (It seems like this decision was made
> >> because some of the stuff in this package is buggy on 64 bit x86.
> >> However, tcmalloc itself is not buggy on 64 bit.). And in general, how
> >> do we handle things that are "good to have" but which shouldn't be
> >> dependencies?
> >
> > Hmm. The whole point of commit a2c02d was that it'd be harder to
> > accidentally build a non-desired variant of Ceph. That is, just
> > because you reinstalled the machine you use for building the package,
> > and forgot to install libatomic-ops-dev, doesn't mean you should
> > create packages with completely different behavior in the future.
> >
> > So, you need to figure out whether libatomic-ops is desired or not,
> > for the relevant platform/architecture. If yes, then builds must fail
> > without it (./configure will complain, debs have build-dependency). If
> > not, then say ./configure --without-libatomic-ops. But don't let it
> > depend on unpredictable factors.
> 
> I agree that libatomic-ops should be mandatory.
> 
> The more annoying dependencies are things like GTK2, which don't
> really affect the behavior of the rest of the project, but which will
> have to be "mandatory" in the RPM, it seems.

We should really break that into a separate package at some point.  The 
main roadblock there iirc is breaking it into a separate binary.  
Originally I thought we should have it launched via the ceph tool (similar 
to the git subcommands) but at this point I don't really care either way.

> Also, as I said, tcmalloc
> will present somewhat of a problem because it's just not packaged on
> RedHat/CentOS at the moment for 64-bit. I will report a bug with Red
> Hat and see what they think...

Yeah, getting it packaged would be ideal.  Let's see how that goes.  
Maybe stick something in the wiki that people can modify the .spec to 
build without it in the meantime?

sage

Re: RPM issues

[Tommi Virtanen]

On Mon, Mar 21, 2011 at 09:32:20AM -0700, Sage Weil wrote:
> > The more annoying dependencies are things like GTK2, which don't
> > really affect the behavior of the rest of the project, but which will
> > have to be "mandatory" in the RPM, it seems.
> 
> We should really break that into a separate package at some point.  The 
> main roadblock there iirc is breaking it into a separate binary.  
> Originally I thought we should have it launched via the ceph tool (similar 
> to the git subcommands) but at this point I don't really care either way.

Well, if you follow the git model, the subcommands are executables in
a specific directory; other source packages could install their
binaries there too.

And to avoid even protocol incompatibilities, gceph, or "ceph gui",
could actually run a git plumbing-style tool underneath, and just
consume it's stdout. Or just build against libceph-dev, that'll work
too.

-- 
:(){ :|:&};:

Re: RPM issues

[Colin McCabe]

On Sat, Mar 19, 2011 at 6:08 AM, Ruben Kerkhof <ruben@rubenkerkhof.com> wrote:
>> 2. What is the best way to bundle the python bindings? Currently I am
>> getting this error when building the RPM:
>>> error: Installed (but unpackaged) file(s) found:
>>> /usr/lib/python2.4/site-packages/rados.py
>>> /usr/lib/python2.4/site-packages/rados.pyc
>>> /usr/lib/python2.4/site-packages/rados.pyo
>> However, I need to find an RPM variable that has a path to the python
>> site-packages directory. automake knows this, but so far I haven't
>> figured out where this information lives in RPM.
>>
>> This guy suggests a workaround:
>> http://www.linux-archive.org/centos/171366-python-sybase-centos-5-x86_64-a.html
>> But his workaround seems kind of clumsy, and in fact doesn't work for
>> me on CentOS 5.5. Surely there is a standard way to refer to the
>> sit-packages directory in RPM without hacks?
>
> There are standard macros in recent versions of Fedora and RHEL6, but
> this should work:
>
> %if ! (0%{?fedora} > 12 || 0%{?rhel} > 5)
> %{!?python_sitelib: %global python_sitelib %(%{__python} -c "from
> distutils.sysconfig import get_python_lib; print(get_python_lib())")}
> %{!?python_sitearch: %global python_sitearch %(%{__python} -c "from
> distutils.sysconfig import get_python_lib; print(get_python_lib(1))")}
> %endif

Thanks for this workaround, Ruben. I committed it as
f2e146aa86f595b9acc1829e394abcd6570ce222

>
> If you add that to the top of your spec you can use %{python_sitearch}
> in your %files section.
> There are more tips at http://fedoraproject.org/wiki/Packaging:Python
>
>> 3. How should we handle tcmalloc, if at all? Google-perftools is not
>> bundled for 64 bit on CentOS. (It seems like this decision was made
>> because some of the stuff in this package is buggy on 64 bit x86.
>> However, tcmalloc itself is not buggy on 64 bit.).
>
> Is google-perftool an EPEL package? If so can you report a bug against
> that package at bugzilla.redhat.com?
> An option would be to split out tcmalloc into it's own rpm.

Filed as https://bugzilla.redhat.com/show_bug.cgi?id=689630

Hopefully we'll get a tcmalloc package!

cheers,
Colin