Virtualenv tests still nort running...

Virtualenv tests still nort running...

[Willem Jan Withagen]

Hi,

I have rebased to the most recent HEAD before building verything again.
But the testrun still is unable to get the virtualenv correct.

Suggestions on how to diagnose this?

===========
cp -f ./fetch_config ./sample.fetch_config
cd ./ceph-detect-init ; ../tools/setup-virtualenv.sh ; test -d 
wheelhouse && export NO_INDEX=--no-index ; virtualenv/bin/pip install 
$NO_INDEX --use-wheel --find-links=file://$(pwd)/wheelhouse -e .
/bin/sh: ../tools/setup-virtualenv.sh: not found
/bin/sh: virtualenv/bin/pip: not found
Makefile:32259: recipe for target 'ceph-detect-init/virtualenv' failed
===========

path for setup-virtual
./src/tools/setup-virtualenv.sh

===========
cat ./src/tools/setup-virtualenv.sh
rm -fr virtualenv
virtualenv virtualenv
. virtualenv/bin/activate
# older versions of pip will not install wrap_console scripts
# when using wheel packages
pip --log virtualenv/log.txt install --upgrade 'pip >= 6.1'
if test -d wheelhouse ; then
     export NO_INDEX=--no-index
fi
pip --log virtualenv/log.txt install $NO_INDEX --use-wheel 
--find-links=file://$(pwd)/wheelhouse --upgrade di
stribute
pip --log virtualenv/log.txt install $NO_INDEX --use-wheel 
--find-links=file://$(pwd)/wheelhouse 'tox >=1.9'
if test -f requirements.txt ; then
     pip --log virtualenv/log.txt install $NO_INDEX --use-wheel 
--find-links=file://$(pwd)/wheelhouse -r requi
rements.txt
fi
===========
Perhaps here the virtualenv paths need to be converted into SHELLVARs as 
well?

--WjW

Re: Virtualenv tests still nort running...

[Loic Dachary]

Hi Willem,

This has undergone various fixes all of them wrong in a context or another. The last of them is (hopefully ;-) https://github.com/ceph/ceph/pull/8320

Cheers

On 29/03/2016 13:31, Willem Jan Withagen wrote:
> 
> Hi,
> 
> I have rebased to the most recent HEAD before building verything again.
> But the testrun still is unable to get the virtualenv correct.
> 
> Suggestions on how to diagnose this?
> 
> ===========
> cp -f ./fetch_config ./sample.fetch_config
> cd ./ceph-detect-init ; ../tools/setup-virtualenv.sh ; test -d wheelhouse && export NO_INDEX=--no-index ; virtualenv/bin/pip install $NO_INDEX --use-wheel --find-links=file://$(pwd)/wheelhouse -e .
> /bin/sh: ../tools/setup-virtualenv.sh: not found
> /bin/sh: virtualenv/bin/pip: not found
> Makefile:32259: recipe for target 'ceph-detect-init/virtualenv' failed
> ===========
> 
> path for setup-virtual
> ./src/tools/setup-virtualenv.sh
> 
> ===========
> cat ./src/tools/setup-virtualenv.sh
> rm -fr virtualenv
> virtualenv virtualenv
> . virtualenv/bin/activate
> # older versions of pip will not install wrap_console scripts
> # when using wheel packages
> pip --log virtualenv/log.txt install --upgrade 'pip >= 6.1'
> if test -d wheelhouse ; then
>     export NO_INDEX=--no-index
> fi
> pip --log virtualenv/log.txt install $NO_INDEX --use-wheel --find-links=file://$(pwd)/wheelhouse --upgrade di
> stribute
> pip --log virtualenv/log.txt install $NO_INDEX --use-wheel --find-links=file://$(pwd)/wheelhouse 'tox >=1.9'
> if test -f requirements.txt ; then
>     pip --log virtualenv/log.txt install $NO_INDEX --use-wheel --find-links=file://$(pwd)/wheelhouse -r requi
> rements.txt
> fi
> ===========
> Perhaps here the virtualenv paths need to be converted into SHELLVARs as well?
> 
> --WjW
> -- 
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

-- 
Loïc Dachary, Artisan Logiciel Libre
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Virtualenv tests still nort running...

[Willem Jan Withagen]

On 29-3-2016 14:28, Loic Dachary wrote:
> Hi Willem,
> 
> This has undergone various fixes all of them wrong in a context or
> another. The last of them is (hopefully ;-)
> https://github.com/ceph/ceph/pull/8320

Hi Loic,

I watched the discussion carefully, and was under the assemption that
everything was already merged. But I'll take your pull and see what happens.

--WjW

> Cheers
> 
> On 29/03/2016 13:31, Willem Jan Withagen wrote:
>> 
>> Hi,
>> 
>> I have rebased to the most recent HEAD before building verything
>> again. But the testrun still is unable to get the virtualenv
>> correct.
>> 
>> Suggestions on how to diagnose this?
>> 
>> =========== cp -f ./fetch_config ./sample.fetch_config cd
>> ./ceph-detect-init ; ../tools/setup-virtualenv.sh ; test -d
>> wheelhouse && export NO_INDEX=--no-index ; virtualenv/bin/pip
>> install $NO_INDEX --use-wheel --find-links=file://$(pwd)/wheelhouse
>> -e . /bin/sh: ../tools/setup-virtualenv.sh: not found /bin/sh:
>> virtualenv/bin/pip: not found Makefile:32259: recipe for target
>> 'ceph-detect-init/virtualenv' failed ===========
>> 
>> path for setup-virtual ./src/tools/setup-virtualenv.sh
>> 
>> =========== cat ./src/tools/setup-virtualenv.sh rm -fr virtualenv 
>> virtualenv virtualenv . virtualenv/bin/activate # older versions of
>> pip will not install wrap_console scripts # when using wheel
>> packages pip --log virtualenv/log.txt install --upgrade 'pip >=
>> 6.1' if test -d wheelhouse ; then export NO_INDEX=--no-index fi pip
>> --log virtualenv/log.txt install $NO_INDEX --use-wheel
>> --find-links=file://$(pwd)/wheelhouse --upgrade di stribute pip
>> --log virtualenv/log.txt install $NO_INDEX --use-wheel
>> --find-links=file://$(pwd)/wheelhouse 'tox >=1.9' if test -f
>> requirements.txt ; then pip --log virtualenv/log.txt install
>> $NO_INDEX --use-wheel --find-links=file://$(pwd)/wheelhouse -r
>> requi rements.txt fi =========== Perhaps here the virtualenv paths
>> need to be converted into SHELLVARs as well?
>> 
>> --WjW -- To unsubscribe from this list: send the line "unsubscribe
>> ceph-devel" in the body of a message to majordomo@vger.kernel.org 
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>> 
> 

Re: Virtualenv tests still nort running...

[Willem Jan Withagen]

On 29-3-2016 17:12, Willem Jan Withagen wrote:
> On 29-3-2016 14:28, Loic Dachary wrote:
>> Hi Willem,
>>
>> This has undergone various fixes all of them wrong in a context or
>> another. The last of them is (hopefully ;-)
>> https://github.com/ceph/ceph/pull/8320
> 
> Hi Loic,
> 
> I watched the discussion carefully, and was under the assemption that
> everything was already merged. But I'll take your pull and see what happens.

Hi Loic,

I saw that #8320 was merged and rebased. So running/compiling to see
what brings.

There might be one change that you'd care to take along in this as well,
which I have to keep my tests running ATM.
Otherwise I'll just keep it in my patchset to the time to come.

--WjW

: ${CEPH_DETECT_INIT_VIRTUALENV:=ceph-detect-init-virtualenv}

case `uname` in
 FreeBSD)
    echo FreeBSD init system has not been integrated.
    exit 0
    ;;
 *)
    # run from the ceph-detect-init directory or from its parent
    test -d ceph-detect-init && cd ceph-detect-init
    source ${CEPH_DETECT_INIT_VIRTUALENV}/bin/activate
    tox > ${CEPH_DETECT_INIT_VIRTUALENV}/tox.out 2>&1
    status=$?
    grep -v InterpreterNotFound < ${CEPH_DETECT_INIT_VIRTUALENV}/tox.out
    exit $status
    ;;
esac

Re: Virtualenv tests still nort running...

[Loic Dachary]

Hi Willem,

Would you mind proposing a pull request with that change ? It looks good to me.

Cheers

On 30/03/2016 11:25, Willem Jan Withagen wrote:
> On 29-3-2016 17:12, Willem Jan Withagen wrote:
>> On 29-3-2016 14:28, Loic Dachary wrote:
>>> Hi Willem,
>>>
>>> This has undergone various fixes all of them wrong in a context or
>>> another. The last of them is (hopefully ;-)
>>> https://github.com/ceph/ceph/pull/8320
>>
>> Hi Loic,
>>
>> I watched the discussion carefully, and was under the assemption that
>> everything was already merged. But I'll take your pull and see what happens.
> 
> Hi Loic,
> 
> I saw that #8320 was merged and rebased. So running/compiling to see
> what brings.
> 
> There might be one change that you'd care to take along in this as well,
> which I have to keep my tests running ATM.
> Otherwise I'll just keep it in my patchset to the time to come.
> 
> --WjW
> 
> : ${CEPH_DETECT_INIT_VIRTUALENV:=ceph-detect-init-virtualenv}
> 
> case `uname` in
>  FreeBSD)
>     echo FreeBSD init system has not been integrated.
>     exit 0
>     ;;
>  *)
>     # run from the ceph-detect-init directory or from its parent
>     test -d ceph-detect-init && cd ceph-detect-init
>     source ${CEPH_DETECT_INIT_VIRTUALENV}/bin/activate
>     tox > ${CEPH_DETECT_INIT_VIRTUALENV}/tox.out 2>&1
>     status=$?
>     grep -v InterpreterNotFound < ${CEPH_DETECT_INIT_VIRTUALENV}/tox.out
>     exit $status
>     ;;
> esac
> 
> 

-- 
Loïc Dachary, Artisan Logiciel Libre
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Virtualenv tests still nort running...

[Willem Jan Withagen]

On 30-3-2016 12:10, Loic Dachary wrote:
> Hi Willem,
> 
> Would you mind proposing a pull request with that change ? It looks good to me.
> 

Loic,

It does run from make.

But just running it on the commandline, does not really deliver....

[/usr/srcs/Ceph/work/ceph/src] wjw@freetest.digiware.nl>
ceph-disk/run-tox.sh
ceph-disk/run-tox.sh: line 21: ceph-disk-virtualenv/bin/activate: No
such file or directory
ceph-disk/run-tox.sh: line 22: ceph-disk-virtualenv/tox.out: No such
file or directory
ceph-disk/run-tox.sh: line 24: ceph-disk-virtualenv/tox.out: No such
file or directory

But perhaps the testscripts are not intended to run separate on the
commandline?

--WjW

Re: Virtualenv tests still nort running...

[Loic Dachary]

On 30/03/2016 22:39, Willem Jan Withagen wrote:
> On 30-3-2016 12:10, Loic Dachary wrote:
>> Hi Willem,
>>
>> Would you mind proposing a pull request with that change ? It looks good to me.
>>
> 
> Loic,
> 
> It does run from make.
> 
> But just running it on the commandline, does not really deliver....
> 
> [/usr/srcs/Ceph/work/ceph/src] wjw@freetest.digiware.nl>
> ceph-disk/run-tox.sh
> ceph-disk/run-tox.sh: line 21: ceph-disk-virtualenv/bin/activate: No
> such file or directory
> ceph-disk/run-tox.sh: line 22: ceph-disk-virtualenv/tox.out: No such
> file or directory
> ceph-disk/run-tox.sh: line 24: ceph-disk-virtualenv/tox.out: No such
> file or directory
> 
> But perhaps the testscripts are not intended to run separate on the
> commandline?

They are. There still is something wrong...

> 
> --WjW
> 
> 

-- 
Loïc Dachary, Artisan Logiciel Libre
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Virtualenv tests still nort running...

[Willem Jan Withagen]

On 31-3-2016 11:31, Loic Dachary wrote:
> 
> 
> On 30/03/2016 22:39, Willem Jan Withagen wrote:
>> On 30-3-2016 12:10, Loic Dachary wrote:
>>> Hi Willem,
>>>
>>> Would you mind proposing a pull request with that change ? It looks good to me.
>>>
>>
>> Loic,
>>
>> It does run from make.
>>
>> But just running it on the commandline, does not really deliver....
>>
>> [/usr/srcs/Ceph/work/ceph/src] wjw@freetest.digiware.nl>
>> ceph-disk/run-tox.sh
>> ceph-disk/run-tox.sh: line 21: ceph-disk-virtualenv/bin/activate: No
>> such file or directory
>> ceph-disk/run-tox.sh: line 22: ceph-disk-virtualenv/tox.out: No such
>> file or directory
>> ceph-disk/run-tox.sh: line 24: ceph-disk-virtualenv/tox.out: No such
>> file or directory
>>
>> But perhaps the testscripts are not intended to run separate on the
>> commandline?
> 
> They are. There still is something wrong...

Anything I can do?

--WjW

Re: Virtualenv tests still nort running...

[Loic Dachary]

On 31/03/2016 12:04, Willem Jan Withagen wrote:
> On 31-3-2016 11:31, Loic Dachary wrote:
>>
>>
>> On 30/03/2016 22:39, Willem Jan Withagen wrote:
>>> On 30-3-2016 12:10, Loic Dachary wrote:
>>>> Hi Willem,
>>>>
>>>> Would you mind proposing a pull request with that change ? It looks good to me.
>>>>
>>>
>>> Loic,
>>>
>>> It does run from make.
>>>
>>> But just running it on the commandline, does not really deliver....
>>>
>>> [/usr/srcs/Ceph/work/ceph/src] wjw@freetest.digiware.nl>
>>> ceph-disk/run-tox.sh
>>> ceph-disk/run-tox.sh: line 21: ceph-disk-virtualenv/bin/activate: No
>>> such file or directory
>>> ceph-disk/run-tox.sh: line 22: ceph-disk-virtualenv/tox.out: No such
>>> file or directory
>>> ceph-disk/run-tox.sh: line 24: ceph-disk-virtualenv/tox.out: No such
>>> file or directory
>>>
>>> But perhaps the testscripts are not intended to run separate on the
>>> commandline?
>>
>> They are. There still is something wrong...
> 
> Anything I can do?

modified   src/ceph-disk/run-tox.sh
@@ -16,7 +16,7 @@
 #

 # run from the ceph-disk directory or from its parent
-: ${CEPH_DISK_VIRTUALENV:=ceph-disk-virtualenv}
+: ${CEPH_DISK_VIRTUALENV:=/tmp/ceph-disk-virtualenv}
 test -d ceph-disk && cd ceph-disk
 source ${CEPH_DISK_VIRTUALENV}/bin/activate
 tox > ${CEPH_DISK_VIRTUALENV}/tox.out 2>&1

Does it work with the above patch ? The default location of the virtualenv was changed but this default was not updated.

Cheers

-- 
Loïc Dachary, Artisan Logiciel Libre
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Virtualenv tests still nort running...

[Loic Dachary]

[re-adding ceph-devel]

On 31/03/2016 12:35, Willem Jan Withagen wrote:
> On 31-3-2016 12:14, Loic Dachary wrote:
>> : ${CEPH_DISK_VIRTUALENV:=/tmp/ceph-disk-virtualenv}
> 
> That works when I run it like:
>  > cd src
>  > ./ceph-disk/run-tox.sh
> 
> -----------------------------------------------------
> TOTAL                    2302   1066    54%
> ___________________________________ summary
> ____________________________________
>   flake8: commands succeeded
>   py27: commands succeeded
>   congratulations :)
> 
> Still leaves remainder in /tmp so that needs to be added.
> And it will require a large enough temp with plenty freespace

It's only used for python virtualenv which is small.

> On a more philosophical note:
> The next thing to consider is that there is a fair chance that somebody
> mounted a tmpfs ram-disk on tmp. Which is again a different fs-beast
> from xfs, with different properties.

This location is only used to store virutalenv, not to run actual tests.

> Personally I do not really like the fact that the Ceph tests leave files
> lingering just about everyware: in .src, in /tmp.
> testdir is a real nice place to drop test things in.
> 
> Also the fact that gtest with EXPECT_DEATH do not cleanup the *cores
> after them is also something that would warrant some cleanup.

Yes. I expect we'll get something saner when we switch to cmake.

Cheers

-- 
Loïc Dachary, Artisan Logiciel Libre
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Virtualenv tests still nort running...

[Willem Jan Withagen]

On 31-3-2016 13:14, Loic Dachary wrote:
> [re-adding ceph-devel]
> 
> On 31/03/2016 12:35, Willem Jan Withagen wrote:
>> On 31-3-2016 12:14, Loic Dachary wrote:
>>> : ${CEPH_DISK_VIRTUALENV:=/tmp/ceph-disk-virtualenv}
>>
>> That works when I run it like:
>>  > cd src
>>  > ./ceph-disk/run-tox.sh
>>
>> -----------------------------------------------------
>> TOTAL                    2302   1066    54%
>> ___________________________________ summary
>> ____________________________________
>>   flake8: commands succeeded
>>   py27: commands succeeded
>>   congratulations :)
>>
>> Still leaves remainder in /tmp so that needs to be added.
>> And it will require a large enough temp with plenty freespace
> 
> It's only used for python virtualenv which is small.
> 
>> On a more philosophical note:
>> The next thing to consider is that there is a fair chance that somebody
>> mounted a tmpfs ram-disk on tmp. Which is again a different fs-beast
>> from xfs, with different properties.
> 
> This location is only used to store virutalenv, not to run actual tests.

Oke, good, because otherwise it would give room for strange results i guess.

>> Personally I do not really like the fact that the Ceph tests leave files
>> lingering just about everyware: in .src, in /tmp.
>> testdir is a real nice place to drop test things in.
>>
>> Also the fact that gtest with EXPECT_DEATH do not cleanup the *cores
>> after them is also something that would warrant some cleanup.
> 
> Yes. I expect we'll get something saner when we switch to cmake.

Oke, thanx for the pointer.
And I'll not bother hacking too much on this in the mean time.

--WjW