testing the teuthology OpenStack backend

testing the teuthology OpenStack backend

[Loic Dachary]

[-- Attachment #1: Type: text/plain, Size: 334 bytes --]

Hi,

I'm looking into testing the OpenStack backend for teuthology on a new cluster to verify it's portable. I think it is but ... ;-) I'm told you have an OpenStack cluster and would be interested in running teuthology workloads on it. Does it have a public facing API ? 

Cheers

-- 
Loïc Dachary, Artisan Logiciel Libre


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: testing the teuthology OpenStack backend

[Loic Dachary]

[-- Attachment #1: Type: text/plain, Size: 1480 bytes --]

Hi,

You should be able to follow the instructions at

   https://github.com/dachary/teuthology/tree/wip-6502-v3-dragonfly#openstack-backend

on your OpenStack cluster. I expect it to be portable accross OpenStack implemenations back to Havana. Please let me know if your OpenStack cluster has a non standard behavior and we will figure out a workaround. By using (note the --upload flag)

   teuthology-openstack --key-filename myself.pem --key-name myself \
                        --suite dummy --upload --teardown

the results of the teuthology run will be uploaded to 

    http://integration.ceph.dachary.org/

Cheers

On 08/08/2015 17:23, Loic Dachary wrote:
> 
> 
> On 08/08/2015 16:22, Li Wang wrote:
>> Hi Loic,
>>   Glad to talk with you on IRC about setting up
>> the teuthology OpenStack backend. Once it is ready
>> to run the test in private cloud, and the results
>> could be exported and uploaded to a public place,
>> please let me know asap :)
> 
> I'm working on it, should be ready RSN.
> 
>>
>> Cheers,
>> Li Wang
>>
>>
>> On 2015/8/6 23:05, Loic Dachary wrote:
>>> Hi,
>>>
>>> I'm looking into testing the OpenStack backend for teuthology on a new cluster to verify it's portable. I think it is but ... ;-) I'm told you have an OpenStack cluster and would be interested in running teuthology workloads on it. Does it have a public facing API ?
>>>
>>> Cheers
>>>
> 

-- 
Loïc Dachary, Artisan Logiciel Libre


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: testing the teuthology OpenStack backend

[Li Wang]

Hi Loic,
   Thanks very much, we will give a try asap.

Cheers,
Li Wang

On 2015/8/10 21:09, Loic Dachary wrote:
> Hi,
>
> You should be able to follow the instructions at
>
>     https://github.com/dachary/teuthology/tree/wip-6502-v3-dragonfly#openstack-backend
>
> on your OpenStack cluster. I expect it to be portable accross OpenStack implemenations back to Havana. Please let me know if your OpenStack cluster has a non standard behavior and we will figure out a workaround. By using (note the --upload flag)
>
>     teuthology-openstack --key-filename myself.pem --key-name myself \
>                          --suite dummy --upload --teardown
>
> the results of the teuthology run will be uploaded to
>
>      http://integration.ceph.dachary.org/
>
> Cheers
>
> On 08/08/2015 17:23, Loic Dachary wrote:
>>
>>
>> On 08/08/2015 16:22, Li Wang wrote:
>>> Hi Loic,
>>>    Glad to talk with you on IRC about setting up
>>> the teuthology OpenStack backend. Once it is ready
>>> to run the test in private cloud, and the results
>>> could be exported and uploaded to a public place,
>>> please let me know asap :)
>>
>> I'm working on it, should be ready RSN.
>>
>>>
>>> Cheers,
>>> Li Wang
>>>
>>>
>>> On 2015/8/6 23:05, Loic Dachary wrote:
>>>> Hi,
>>>>
>>>> I'm looking into testing the OpenStack backend for teuthology on a new cluster to verify it's portable. I think it is but ... ;-) I'm told you have an OpenStack cluster and would be interested in running teuthology workloads on it. Does it have a public facing API ?
>>>>
>>>> Cheers
>>>>
>>
>

Re: testing the teuthology OpenStack backend

[Loic Dachary]

[-- Attachment #1: Type: text/plain, Size: 2033 bytes --]

Hi,

[cc'ing the ceph-devel list for Zack or Andrew's input]

On 19/08/2015 11:32, Li Wang wrote:
> Hi Loic,
>   we did a test, the results are attached. We found that the
> network may be the bottleneck, it seems it will fetch all
> the packages from ceph website on the fly, that does take time
> and cause many failed jobs, and there are also many dead
> jobs, what do those mean? Do you have some suggestions,
> if possible to enable it fetch the packages from configurable
> mirror or even private package repositories?

You can ssh to the teuthology instance (the host that runs pulpito) and modify the ~/.teuthology.yaml with

   gitbuilder_host: gitbuilder.ubuntukylin.com

which will override the default of 

   gitbuilder_host: gitbuilder.ceph.com

I've never done that however and maybe Zack or Andrew have advices on how to establish a mirror properly.

I think having a private package repository that is useable by teuthology is complicated because the package locations, naming conventions and build methods are difficult to reproduce (and I don't fully understand what they are right now).

Cheers

> 
> Cheers,
> Li Wang
> 
> On 2015/8/8 23:23, Loic Dachary wrote:
>>
>>
>> On 08/08/2015 16:22, Li Wang wrote:
>>> Hi Loic,
>>>    Glad to talk with you on IRC about setting up
>>> the teuthology OpenStack backend. Once it is ready
>>> to run the test in private cloud, and the results
>>> could be exported and uploaded to a public place,
>>> please let me know asap :)
>>
>> I'm working on it, should be ready RSN.
>>
>>>
>>> Cheers,
>>> Li Wang
>>>
>>>
>>> On 2015/8/6 23:05, Loic Dachary wrote:
>>>> Hi,
>>>>
>>>> I'm looking into testing the OpenStack backend for teuthology on a new cluster to verify it's portable. I think it is but ... ;-) I'm told you have an OpenStack cluster and would be interested in running teuthology workloads on it. Does it have a public facing API ?
>>>>
>>>> Cheers
>>>>
>>

-- 
Loïc Dachary, Artisan Logiciel Libre


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

RE: testing the teuthology OpenStack backend

[Zhou, Yuan]

Hi Loic/Li,

We have a local Ceph binary repo for our local teuthology testing. I'm using some modified Ceph gitbuilder (https://github.com/ceph/gitbuilder) to create the repo. The things I've changed are:

1. modify branch-local to watch our branchs only, otherwise there'll be too many branches to build
2. modify make-debs.sh to add *test-dbg packages  (https://gist.github.com/zhouyuan/f4c1c671c1659ece04e1)
2. modify autobuilder.sh to make a proper directory and move the binaries there  (https://gist.github.com/zhouyuan/2a2c4d1b20ff42efc351)

It's still very simple (e,g, only Ubuntu Trusty included) but enough for our tests now. Hope this can help.

Thanks, -yuan


-----Original Message-----
From: ceph-devel-owner@vger.kernel.org [mailto:ceph-devel-owner@vger.kernel.org] On Behalf Of Loic Dachary
Sent: Wednesday, August 19, 2015 6:24 PM
To: Li Wang
Cc: Ceph Development
Subject: Re: testing the teuthology OpenStack backend

Hi,

[cc'ing the ceph-devel list for Zack or Andrew's input]

On 19/08/2015 11:32, Li Wang wrote:
> Hi Loic,
>   we did a test, the results are attached. We found that the network 
> may be the bottleneck, it seems it will fetch all the packages from 
> ceph website on the fly, that does take time and cause many failed 
> jobs, and there are also many dead jobs, what do those mean? Do you 
> have some suggestions, if possible to enable it fetch the packages 
> from configurable mirror or even private package repositories?

You can ssh to the teuthology instance (the host that runs pulpito) and modify the ~/.teuthology.yaml with

   gitbuilder_host: gitbuilder.ubuntukylin.com

which will override the default of 

   gitbuilder_host: gitbuilder.ceph.com

I've never done that however and maybe Zack or Andrew have advices on how to establish a mirror properly.

I think having a private package repository that is useable by teuthology is complicated because the package locations, naming conventions and build methods are difficult to reproduce (and I don't fully understand what they are right now).

Cheers

> 
> Cheers,
> Li Wang
> 
> On 2015/8/8 23:23, Loic Dachary wrote:
>>
>>
>> On 08/08/2015 16:22, Li Wang wrote:
>>> Hi Loic,
>>>    Glad to talk with you on IRC about setting up the teuthology 
>>> OpenStack backend. Once it is ready to run the test in private 
>>> cloud, and the results could be exported and uploaded to a public 
>>> place, please let me know asap :)
>>
>> I'm working on it, should be ready RSN.
>>
>>>
>>> Cheers,
>>> Li Wang
>>>
>>>
>>> On 2015/8/6 23:05, Loic Dachary wrote:
>>>> Hi,
>>>>
>>>> I'm looking into testing the OpenStack backend for teuthology on a new cluster to verify it's portable. I think it is but ... ;-) I'm told you have an OpenStack cluster and would be interested in running teuthology workloads on it. Does it have a public facing API ?
>>>>
>>>> Cheers
>>>>
>>

--
Loïc Dachary, Artisan Logiciel Libre

Re: testing the teuthology OpenStack backend

[Loic Dachary]

[-- Attachment #1: Type: text/plain, Size: 3500 bytes --]

Hi,

Thanks for sharing, it helps :-)

Cheers

On 21/08/2015 08:09, Zhou, Yuan wrote:
> Hi Loic/Li,
> 
> We have a local Ceph binary repo for our local teuthology testing. I'm using some modified Ceph gitbuilder (https://github.com/ceph/gitbuilder) to create the repo. The things I've changed are:
> 
> 1. modify branch-local to watch our branchs only, otherwise there'll be too many branches to build
> 2. modify make-debs.sh to add *test-dbg packages  (https://gist.github.com/zhouyuan/f4c1c671c1659ece04e1)
> 2. modify autobuilder.sh to make a proper directory and move the binaries there  (https://gist.github.com/zhouyuan/2a2c4d1b20ff42efc351)
> 
> It's still very simple (e,g, only Ubuntu Trusty included) but enough for our tests now. Hope this can help.
> 
> Thanks, -yuan
> 
> 
> -----Original Message-----
> From: ceph-devel-owner@vger.kernel.org [mailto:ceph-devel-owner@vger.kernel.org] On Behalf Of Loic Dachary
> Sent: Wednesday, August 19, 2015 6:24 PM
> To: Li Wang
> Cc: Ceph Development
> Subject: Re: testing the teuthology OpenStack backend
> 
> Hi,
> 
> [cc'ing the ceph-devel list for Zack or Andrew's input]
> 
> On 19/08/2015 11:32, Li Wang wrote:
>> Hi Loic,
>>   we did a test, the results are attached. We found that the network 
>> may be the bottleneck, it seems it will fetch all the packages from 
>> ceph website on the fly, that does take time and cause many failed 
>> jobs, and there are also many dead jobs, what do those mean? Do you 
>> have some suggestions, if possible to enable it fetch the packages 
>> from configurable mirror or even private package repositories?
> 
> You can ssh to the teuthology instance (the host that runs pulpito) and modify the ~/.teuthology.yaml with
> 
>    gitbuilder_host: gitbuilder.ubuntukylin.com
> 
> which will override the default of 
> 
>    gitbuilder_host: gitbuilder.ceph.com
> 
> I've never done that however and maybe Zack or Andrew have advices on how to establish a mirror properly.
> 
> I think having a private package repository that is useable by teuthology is complicated because the package locations, naming conventions and build methods are difficult to reproduce (and I don't fully understand what they are right now).
> 
> Cheers
> 
>>
>> Cheers,
>> Li Wang
>>
>> On 2015/8/8 23:23, Loic Dachary wrote:
>>>
>>>
>>> On 08/08/2015 16:22, Li Wang wrote:
>>>> Hi Loic,
>>>>    Glad to talk with you on IRC about setting up the teuthology 
>>>> OpenStack backend. Once it is ready to run the test in private 
>>>> cloud, and the results could be exported and uploaded to a public 
>>>> place, please let me know asap :)
>>>
>>> I'm working on it, should be ready RSN.
>>>
>>>>
>>>> Cheers,
>>>> Li Wang
>>>>
>>>>
>>>> On 2015/8/6 23:05, Loic Dachary wrote:
>>>>> Hi,
>>>>>
>>>>> I'm looking into testing the OpenStack backend for teuthology on a new cluster to verify it's portable. I think it is but ... ;-) I'm told you have an OpenStack cluster and would be interested in running teuthology workloads on it. Does it have a public facing API ?
>>>>>
>>>>> Cheers
>>>>>
>>>
> 
> --
> Loïc Dachary, Artisan Logiciel Libre
> 
> N�����r��y���b�X��ǧv�^�)޺{.n�+���z�]z���{ay�\x1dʇڙ�,j\a��f���h���z�\x1e�w���\f���j:+v���w�j�m����\a����zZ+�����ݢj"��!tml=
> 

-- 
Loïc Dachary, Artisan Logiciel Libre


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]