vstart and Python bindings

vstart and Python bindings

[Pete Zaitcev]

I did a git pull today and rebuilt. Ran vstart, and it produced a
Python traceback, ending with:

ImportError: No module named rados

Oooookay... This box never had Python bindings installed, but okay,
someone innovated something. Fine, I thought. Installed python-rados,
and ./ceph -w tracebacks:

Traceback (most recent call last):
  File "./ceph", line 953, in <module>
    retval = main()
  File "./ceph", line 778, in main
    run_in_thread(cluster_handle.monitor_log, level, watch_cb, 0)
AttributeError: 'Rados' object has no attribute 'monitor_log'

Of course. The system version of pybind is obsolete, but ./ceph gets
wrong one. Actually, it's not even getting built from .pyx anymore.

Could whoever did all this please undo?

-- Pete

Re: vstart and Python bindings

[Loic Dachary]

Hi Pete,

You need https://github.com/ceph/ceph/pull/7792 which is going to be merged real soon now ;-)

Cheers

On 25/02/2016 13:37, Pete Zaitcev wrote:
> I did a git pull today and rebuilt. Ran vstart, and it produced a
> Python traceback, ending with:
> 
> ImportError: No module named rados
> 
> Oooookay... This box never had Python bindings installed, but okay,
> someone innovated something. Fine, I thought. Installed python-rados,
> and ./ceph -w tracebacks:
> 
> Traceback (most recent call last):
>   File "./ceph", line 953, in <module>
>     retval = main()
>   File "./ceph", line 778, in main
>     run_in_thread(cluster_handle.monitor_log, level, watch_cb, 0)
> AttributeError: 'Rados' object has no attribute 'monitor_log'
> 
> Of course. The system version of pybind is obsolete, but ./ceph gets
> wrong one. Actually, it's not even getting built from .pyx anymore.
> 
> Could whoever did all this please undo?
> 
> -- Pete
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

-- 
Loïc Dachary, Artisan Logiciel Libre
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: vstart and Python bindings

[Willem Jan Withagen]

On 25-2-2016 08:04, Loic Dachary wrote:
> Hi Pete,
> 
> You need https://github.com/ceph/ceph/pull/7792 which is going to be merged real soon now ;-)
> 

I had a similar experience, but that was more due to Makefile.am in pybind:
if ENABLE_CLIENT
if WITH_RADOS
if WITH_RBD
if WITH_CYTHON

PY_DISTUTILS = \
        CPPFLAGS="-iquote \${abs_srcdir}/include ${AM_CPPFLAGS}
${CPPFLAGS}" \
        CFLAGS="-iquote \${abs_srcdir}/include ${AM_CFLAGS} ${CFLAGS}" \
        LDFLAGS="-L\${abs_builddir}/.libs $(subst -pie,,${AM_LDFLAGS})
${LDFLAGS}" \
        CYTHON_BUILD_DIR="$(shell readlink -f $(builddir))/build" \
        ${PYTHON} ./setup.py

include pybind/rbd/Makefile.am
include pybind/rados/Makefile.am

endif
endif
endif
endif

And since I'm not there for RBD, pybind/rados/Makefile.am does not get
included.
So I've reorganised the Makefile, put the WITH_{RADOS,RBD} as most inter
tests, and only with the includes.
But haven't gotten around actually compiling a full set again to see if
rados things are actually made.

--WjW


> Cheers
> 
> On 25/02/2016 13:37, Pete Zaitcev wrote:
>> I did a git pull today and rebuilt. Ran vstart, and it produced a
>> Python traceback, ending with:
>>
>> ImportError: No module named rados
>>
>> Oooookay... This box never had Python bindings installed, but okay,
>> someone innovated something. Fine, I thought. Installed python-rados,
>> and ./ceph -w tracebacks:
>>
>> Traceback (most recent call last):
>>   File "./ceph", line 953, in <module>
>>     retval = main()
>>   File "./ceph", line 778, in main
>>     run_in_thread(cluster_handle.monitor_log, level, watch_cb, 0)
>> AttributeError: 'Rados' object has no attribute 'monitor_log'
>>
>> Of course. The system version of pybind is obsolete, but ./ceph gets
>> wrong one. Actually, it's not even getting built from .pyx anymore.
>>
>> Could whoever did all this please undo?
>>
>> -- Pete
>> --
>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
> 

Re: vstart and Python bindings

[John Spray]

On Thu, Feb 25, 2016 at 6:37 AM, Pete Zaitcev <zaitcev@redhat.com> wrote:
> I did a git pull today and rebuilt. Ran vstart, and it produced a
> Python traceback, ending with:
>
> ImportError: No module named rados
>
> Oooookay... This box never had Python bindings installed, but okay,
> someone innovated something. Fine, I thought. Installed python-rados,
> and ./ceph -w tracebacks:
>
> Traceback (most recent call last):
>   File "./ceph", line 953, in <module>
>     retval = main()
>   File "./ceph", line 778, in main
>     run_in_thread(cluster_handle.monitor_log, level, watch_cb, 0)
> AttributeError: 'Rados' object has no attribute 'monitor_log'
>
> Of course. The system version of pybind is obsolete, but ./ceph gets
> wrong one. Actually, it's not even getting built from .pyx anymore.
>
> Could whoever did all this please undo?

I don't think we're undoing this, but I too would like to see prompt
fixes.  The cython changes broke cmake (amaredia mentioned in IRC he
was working on a fix?) and libcephfs's python bindings (fix
https://github.com/ceph/ceph/pull/7745).  With hindsight the cython
change needed more scrutiny before it merged.

John

Re: vstart and Python bindings

[Pete Zaitcev]

Dear Loic:

The PR 7792 is not the problem. The issue appears to be that cythonization
is not performed (anymore). There's a coupld of Makefile.am files in
src/pybind, but no relevant Makefile files are generated. I tried to make
it more explicit with this:

./configure --with-rados --with-cython --with-radosgw

Still no effect.

Greetings,
-- Pete

On Thu, 25 Feb 2016 14:04:13 +0700
Loic Dachary <loic@dachary.org> wrote:

> Hi Pete,
> 
> You need https://github.com/ceph/ceph/pull/7792 which is going to be merged real soon now ;-)
> 
> Cheers
> 
> On 25/02/2016 13:37, Pete Zaitcev wrote:
> > I did a git pull today and rebuilt. Ran vstart, and it produced a
> > Python traceback, ending with:
> > 
> > ImportError: No module named rados
> > 
> > Oooookay... This box never had Python bindings installed, but okay,
> > someone innovated something. Fine, I thought. Installed python-rados,
> > and ./ceph -w tracebacks:
> > 
> > Traceback (most recent call last):
> >   File "./ceph", line 953, in <module>
> >     retval = main()
> >   File "./ceph", line 778, in main
> >     run_in_thread(cluster_handle.monitor_log, level, watch_cb, 0)
> > AttributeError: 'Rados' object has no attribute 'monitor_log'
> > 
> > Of course. The system version of pybind is obsolete, but ./ceph gets
> > wrong one. Actually, it's not even getting built from .pyx anymore.
> > 
> > Could whoever did all this please undo?
> > 
> > -- Pete

Re: vstart and Python bindings

[Pete Zaitcev]

On Thu, 25 Feb 2016 11:41:46 +0000
John Spray <jspray@redhat.com> wrote:

> The cython changes broke cmake (amaredia mentioned in IRC he
> was working on a fix?) and libcephfs's python bindings (fix
> https://github.com/ceph/ceph/pull/7745).

The above seems to imply that something at least gets built and then
works, so for my general education I'm curious just how you do that.
Even if large parts, such as cephfs, are knocked out, I could be
okay with what's left, as long as I can get OSDs and monitor to run.
But I use vstart for that and it does not work (because rados.py
is not built/cythonized). Is there a simple workaround everyone
employs that I just don't see? Willem mentioned that he's tweaking
Makefile.am, which could help... if ./configure even tried to
source those in src/pybind. I'm missing something obvious here, am I?

-- Pete

Re: vstart and Python bindings

[Loic Dachary]

Hi Pete,

I know too little about cython to answer usefully I'm afraid. I'll let other, more knowledgeable developers answer to this problem.

Cheers

On 26/02/2016 07:27, Pete Zaitcev wrote:
> Dear Loic:
> 
> The PR 7792 is not the problem. The issue appears to be that cythonization
> is not performed (anymore). There's a coupld of Makefile.am files in
> src/pybind, but no relevant Makefile files are generated. I tried to make
> it more explicit with this:
> 
> ./configure --with-rados --with-cython --with-radosgw
> 
> Still no effect.
> 
> Greetings,
> -- Pete
> 
> On Thu, 25 Feb 2016 14:04:13 +0700
> Loic Dachary <loic@dachary.org> wrote:
> 
>> Hi Pete,
>>
>> You need https://github.com/ceph/ceph/pull/7792 which is going to be merged real soon now ;-)
>>
>> Cheers
>>
>> On 25/02/2016 13:37, Pete Zaitcev wrote:
>>> I did a git pull today and rebuilt. Ran vstart, and it produced a
>>> Python traceback, ending with:
>>>
>>> ImportError: No module named rados
>>>
>>> Oooookay... This box never had Python bindings installed, but okay,
>>> someone innovated something. Fine, I thought. Installed python-rados,
>>> and ./ceph -w tracebacks:
>>>
>>> Traceback (most recent call last):
>>>   File "./ceph", line 953, in <module>
>>>     retval = main()
>>>   File "./ceph", line 778, in main
>>>     run_in_thread(cluster_handle.monitor_log, level, watch_cb, 0)
>>> AttributeError: 'Rados' object has no attribute 'monitor_log'
>>>
>>> Of course. The system version of pybind is obsolete, but ./ceph gets
>>> wrong one. Actually, it's not even getting built from .pyx anymore.
>>>
>>> Could whoever did all this please undo?
>>>
>>> -- Pete
> 
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

-- 
Loïc Dachary, Artisan Logiciel Libre
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: vstart and Python bindings

[kefu chai]

On Fri, Feb 26, 2016 at 12:41 PM, Loic Dachary <loic@dachary.org> wrote:
> Hi Pete,
>
> I know too little about cython to answer usefully I'm afraid. I'll let other, more knowledgeable developers answer to this problem.
>
> Cheers
>
> On 26/02/2016 07:27, Pete Zaitcev wrote:
>> Dear Loic:
>>
>> The PR 7792 is not the problem. The issue appears to be that cythonization
>> is not performed (anymore). There's a coupld of Makefile.am files in
>> src/pybind, but no relevant Makefile files are generated. I tried to make
>> it more explicit with this:
>>
>> ./configure --with-rados --with-cython --with-radosgw
>>
>> Still no effect.

Pete, i use cythan and cmake. but the autotool case is sort of similar
when "ceph" cli is launched in the source tree. i am wondering if you
are building ceph out of source? which might break the way how "ceph"
detects the library and pybind python script paths. in the "ceph"
script, it is using a heuristic way to see if it is located in
$top_srcdir/src directory or in the $CMAKE_BINARY_DIR by checking the
fulle path of "__file__" and the content of CMakeCache.txt in $PWD.

in my case, the pybind path is correctly detected as
"/home/kefu/dev/ceph/src/pybind", and is appened to sys.path in
python.  see respawn_in_path() method in the ceph cli. maybe you can
tweak it a little bit to fit your needs. for example to adjust the
ceph.in so it is able to detect the $top_srcdir in a better way when
we are build ceph out of source tree? if that's your case.


-- 
Regards
Kefu Chai