teuthology : ulimit: error

teuthology : ulimit: error

[Loic Dachary]

[-- Attachment #1: Type: text/plain, Size: 2321 bytes --]

Hi,

Trying to use Ubuntu precise virtual machines as teuthology targets ( making sure they have 2GB of RAM because ceph-test-dbg will not even install with 1GB of RAM ;-) and installing the key with

wget -q -O- 'https://ceph.com/git/?p=ceph.git;a=blob_plain;f=keys/release.asc' | sudo apt-key add -

teuthology runs with 

./virtualenv/bin/teuthology job.yaml

where job.yaml is

check-locks: false
roles:
- - mon.a
  - mon.c
  - osd.0
- - mon.b
  - osd.1
  - client.0
tasks:
- install:
   project: ceph
   branch: wip-5510
- ceph:
   fs: ext4
- workunit:
    clients:
      all:
      - filestore/filestore.sh
targets:
   ubuntu@91.121.254.229: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDOSit20EyZ2AKCvk2tnMsdQ6LmVRutvBmZb0awV9Z2EduJa0fYPrReYRb9ZhGRq2PJe0zgpFPKr8s4gGay+tL0+dFkju5uyABqMGeITlJCifd+RhM0MCVllgIzekDwVb0n6ydTS8k7GVFyYv8ZC0TPgbfcDcEtSEgqJNRJ0o1Bh8swuTn+cipanNDRVK39tOqJdfptUxak+TD+5QY8CGFdXdEQYP7VsYJ+jQHw73O2xbuPgfv5Shbmt+qGWLToxFqKca3owMtkvFeONgYUdujgg9qr7Q9p0+HhCFCXB8v4N2I7NSbWNdpGqyJLdLqwJ70gEeNlOhm5F7IsXfVxTapB
   ubuntu@91.121.254.237: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCXVzhedORtmEmCeZJ4Ssg8wfqpYyH9W/Aa+j6CvPHSAkzZ48zXqVBATxm6S8sIIqfKkz1hWpNssx6uUotbm8k/ZatMddsd932+Di136l/HUhp6O8iIFze56kjWpyDpRPw2MM0V+OKmsiHZDfMj9ATt6ChgXfRsm23MUlmnoFHvtfwvFBnvBjcZPN7qxMpFHDamZzACNvnis/OINJrud9VprOgjhZZ7mxcTbcVZaVgcTcnyC4b9d9PRrMG2aCv0BO1eb/qnlmSogQPfoKEORJcwaUcLgabww+Taa9hJSZ9l8yEQamj+XIgr6yzGKgCvlG4lTdHM2tQdpgATZvR7/pBz

and produces the following output

INFO:teuthology.orchestra.run.err:[91.121.254.229]: /home/ubuntu/cephtest/lo1308082328/adjust-ulimits: 4: ulimit: error setting limit (Operation not permitted) 

and the full output is at

http://pastealacon.com/32957

as if

/home/ubuntu/cephtest/lo1308082328/adjust-ulimits ceph-coverage /home/ubuntu/cephtest/lo1308082328/archive/coverage monmaptool --create --clobber --add a 91.121.254.229:6789 --add c 91.121.254.229:6790 --add b 91.121.254.237:6789 --print /home/ubuntu/cephtest/lo1308082328/monmap' 

was run without a leading sudo. I tried running sudo adjust-ulimits echo foo manually on the target and it works.

I would very much appreciate a hint ;-)

Cheers

-- 
Loïc Dachary, Artisan Logiciel Libre
All that is necessary for the triumph of evil is that good people do nothing.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 261 bytes --]

Re: teuthology : ulimit: error

[Dan Mick]

IIRC we had to adjust settings in /etc/security to allow ulimit 
adjustment of at least core:

sed -i 's/^#\*.*soft.*core.*0/\*                soft    core 
unlimited/g' /etc/security/limits.conf

or something like that.  That seems to apply to centos/fedora/redhat 
systems.


On 08/08/2013 02:52 PM, Loic Dachary wrote:
> Hi,
>
> Trying to use Ubuntu precise virtual machines as teuthology targets ( making sure they have 2GB of RAM because ceph-test-dbg will not even install with 1GB of RAM ;-) and installing the key with
>
> wget -q -O- 'https://ceph.com/git/?p=ceph.git;a=blob_plain;f=keys/release.asc' | sudo apt-key add -
>
> teuthology runs with
>
> ./virtualenv/bin/teuthology job.yaml
>
> where job.yaml is
>
> check-locks: false
> roles:
> - - mon.a
>    - mon.c
>    - osd.0
> - - mon.b
>    - osd.1
>    - client.0
> tasks:
> - install:
>     project: ceph
>     branch: wip-5510
> - ceph:
>     fs: ext4
> - workunit:
>      clients:
>        all:
>        - filestore/filestore.sh
> targets:
>     ubuntu@91.121.254.229: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDOSit20EyZ2AKCvk2tnMsdQ6LmVRutvBmZb0awV9Z2EduJa0fYPrReYRb9ZhGRq2PJe0zgpFPKr8s4gGay+tL0+dFkju5uyABqMGeITlJCifd+RhM0MCVllgIzekDwVb0n6ydTS8k7GVFyYv8ZC0TPgbfcDcEtSEgqJNRJ0o1Bh8swuTn+cipanNDRVK39tOqJdfptUxak+TD+5QY8CGFdXdEQYP7VsYJ+jQHw73O2xbuPgfv5Shbmt+qGWLToxFqKca3owMtkvFeONgYUdujgg9qr7Q9p0+HhCFCXB8v4N2I7NSbWNdpGqyJLdLqwJ70gEeNlOhm5F7IsXfVxTapB
>     ubuntu@91.121.254.237: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCXVzhedORtmEmCeZJ4Ssg8wfqpYyH9W/Aa+j6CvPHSAkzZ48zXqVBATxm6S8sIIqfKkz1hWpNssx6uUotbm8k/ZatMddsd932+Di136l/HUhp6O8iIFze56kjWpyDpRPw2MM0V+OKmsiHZDfMj9ATt6ChgXfRsm23MUlmnoFHvtfwvFBnvBjcZPN7qxMpFHDamZzACNvnis/OINJrud9VprOgjhZZ7mxcTbcVZaVgcTcnyC4b9d9PRrMG2aCv0BO1eb/qnlmSogQPfoKEORJcwaUcLgabww+Taa9hJSZ9l8yEQamj+XIgr6yzGKgCvlG4lTdHM2tQdpgATZvR7/pBz
>
> and produces the following output
>
> INFO:teuthology.orchestra.run.err:[91.121.254.229]: /home/ubuntu/cephtest/lo1308082328/adjust-ulimits: 4: ulimit: error setting limit (Operation not permitted)
>
> and the full output is at
>
> http://pastealacon.com/32957
>
> as if
>
> /home/ubuntu/cephtest/lo1308082328/adjust-ulimits ceph-coverage /home/ubuntu/cephtest/lo1308082328/archive/coverage monmaptool --create --clobber --add a 91.121.254.229:6789 --add c 91.121.254.229:6790 --add b 91.121.254.237:6789 --print /home/ubuntu/cephtest/lo1308082328/monmap'
>
> was run without a leading sudo. I tried running sudo adjust-ulimits echo foo manually on the target and it works.
>
> I would very much appreciate a hint ;-)
>
> Cheers
>

Re: teuthology : ulimit: error

[Loic Dachary]

[-- Attachment #1: Type: text/plain, Size: 3069 bytes --]



On 10/08/2013 07:35, Dan Mick wrote:
> IIRC we had to adjust settings in /etc/security to allow ulimit adjustment of at least core:
> 
> sed -i 's/^#\*.*soft.*core.*0/\*                soft    core unlimited/g' /etc/security/limits.conf
> 
> or something like that.  That seems to apply to centos/fedora/redhat systems.

Hi Dan,

This is Ubuntu and it does work with sudo. It probably is something about how it's called rather than a system limit.

Cheers

> 
> 
> On 08/08/2013 02:52 PM, Loic Dachary wrote:
>> Hi,
>>
>> Trying to use Ubuntu precise virtual machines as teuthology targets ( making sure they have 2GB of RAM because ceph-test-dbg will not even install with 1GB of RAM ;-) and installing the key with
>>
>> wget -q -O- 'https://ceph.com/git/?p=ceph.git;a=blob_plain;f=keys/release.asc' | sudo apt-key add -
>>
>> teuthology runs with
>>
>> ./virtualenv/bin/teuthology job.yaml
>>
>> where job.yaml is
>>
>> check-locks: false
>> roles:
>> - - mon.a
>>    - mon.c
>>    - osd.0
>> - - mon.b
>>    - osd.1
>>    - client.0
>> tasks:
>> - install:
>>     project: ceph
>>     branch: wip-5510
>> - ceph:
>>     fs: ext4
>> - workunit:
>>      clients:
>>        all:
>>        - filestore/filestore.sh
>> targets:
>>     ubuntu@91.121.254.229: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDOSit20EyZ2AKCvk2tnMsdQ6LmVRutvBmZb0awV9Z2EduJa0fYPrReYRb9ZhGRq2PJe0zgpFPKr8s4gGay+tL0+dFkju5uyABqMGeITlJCifd+RhM0MCVllgIzekDwVb0n6ydTS8k7GVFyYv8ZC0TPgbfcDcEtSEgqJNRJ0o1Bh8swuTn+cipanNDRVK39tOqJdfptUxak+TD+5QY8CGFdXdEQYP7VsYJ+jQHw73O2xbuPgfv5Shbmt+qGWLToxFqKca3owMtkvFeONgYUdujgg9qr7Q9p0+HhCFCXB8v4N2I7NSbWNdpGqyJLdLqwJ70gEeNlOhm5F7IsXfVxTapB
>>     ubuntu@91.121.254.237: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCXVzhedORtmEmCeZJ4Ssg8wfqpYyH9W/Aa+j6CvPHSAkzZ48zXqVBATxm6S8sIIqfKkz1hWpNssx6uUotbm8k/ZatMddsd932+Di136l/HUhp6O8iIFze56kjWpyDpRPw2MM0V+OKmsiHZDfMj9ATt6ChgXfRsm23MUlmnoFHvtfwvFBnvBjcZPN7qxMpFHDamZzACNvnis/OINJrud9VprOgjhZZ7mxcTbcVZaVgcTcnyC4b9d9PRrMG2aCv0BO1eb/qnlmSogQPfoKEORJcwaUcLgabww+Taa9hJSZ9l8yEQamj+XIgr6yzGKgCvlG4lTdHM2tQdpgATZvR7/pBz
>>
>> and produces the following output
>>
>> INFO:teuthology.orchestra.run.err:[91.121.254.229]: /home/ubuntu/cephtest/lo1308082328/adjust-ulimits: 4: ulimit: error setting limit (Operation not permitted)
>>
>> and the full output is at
>>
>> http://pastealacon.com/32957

Refreshed with a longer timeout.

http://pastealacon.com/32963
>>
>> as if
>>
>> /home/ubuntu/cephtest/lo1308082328/adjust-ulimits ceph-coverage /home/ubuntu/cephtest/lo1308082328/archive/coverage monmaptool --create --clobber --add a 91.121.254.229:6789 --add c 91.121.254.229:6790 --add b 91.121.254.237:6789 --print /home/ubuntu/cephtest/lo1308082328/monmap'
>>
>> was run without a leading sudo. I tried running sudo adjust-ulimits echo foo manually on the target and it works.
>>
>> I would very much appreciate a hint ;-)
>>
>> Cheers
>>

-- 
Loïc Dachary, Artisan Logiciel Libre
All that is necessary for the triumph of evil is that good people do nothing.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 261 bytes --]

Re: teuthology : ulimit: error

[Dan Mick]

Ah, there's another we apply universally to our test systems, apparently:

'/etc/security/limits.d/ubuntu.conf'

ubuntu hard nofile 16384

and the tests run as user "ubuntu".  Line 4 of the script is the nofile 
setting.

On 08/10/2013 01:34 AM, Loic Dachary wrote:
>
>
> On 10/08/2013 07:35, Dan Mick wrote:
>> IIRC we had to adjust settings in /etc/security to allow ulimit adjustment of at least core:
>>
>> sed -i 's/^#\*.*soft.*core.*0/\*                soft    core unlimited/g' /etc/security/limits.conf
>>
>> or something like that.  That seems to apply to centos/fedora/redhat systems.
>
> Hi Dan,
>
> This is Ubuntu and it does work with sudo. It probably is something about how it's called rather than a system limit.
>
> Cheers
>
>>
>>
>> On 08/08/2013 02:52 PM, Loic Dachary wrote:
>>> Hi,
>>>
>>> Trying to use Ubuntu precise virtual machines as teuthology targets ( making sure they have 2GB of RAM because ceph-test-dbg will not even install with 1GB of RAM ;-) and installing the key with
>>>
>>> wget -q -O- 'https://ceph.com/git/?p=ceph.git;a=blob_plain;f=keys/release.asc' | sudo apt-key add -
>>>
>>> teuthology runs with
>>>
>>> ./virtualenv/bin/teuthology job.yaml
>>>
>>> where job.yaml is
>>>
>>> check-locks: false
>>> roles:
>>> - - mon.a
>>>     - mon.c
>>>     - osd.0
>>> - - mon.b
>>>     - osd.1
>>>     - client.0
>>> tasks:
>>> - install:
>>>      project: ceph
>>>      branch: wip-5510
>>> - ceph:
>>>      fs: ext4
>>> - workunit:
>>>       clients:
>>>         all:
>>>         - filestore/filestore.sh
>>> targets:
>>>      ubuntu@91.121.254.229: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDOSit20EyZ2AKCvk2tnMsdQ6LmVRutvBmZb0awV9Z2EduJa0fYPrReYRb9ZhGRq2PJe0zgpFPKr8s4gGay+tL0+dFkju5uyABqMGeITlJCifd+RhM0MCVllgIzekDwVb0n6ydTS8k7GVFyYv8ZC0TPgbfcDcEtSEgqJNRJ0o1Bh8swuTn+cipanNDRVK39tOqJdfptUxak+TD+5QY8CGFdXdEQYP7VsYJ+jQHw73O2xbuPgfv5Shbmt+qGWLToxFqKca3owMtkvFeONgYUdujgg9qr7Q9p0+HhCFCXB8v4N2I7NSbWNdpGqyJLdLqwJ70gEeNlOhm5F7IsXfVxTapB
>>>      ubuntu@91.121.254.237: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCXVzhedORtmEmCeZJ4Ssg8wfqpYyH9W/Aa+j6CvPHSAkzZ48zXqVBATxm6S8sIIqfKkz1hWpNssx6uUotbm8k/ZatMddsd932+Di136l/HUhp6O8iIFze56kjWpyDpRPw2MM0V+OKmsiHZDfMj9ATt6ChgXfRsm23MUlmnoFHvtfwvFBnvBjcZPN7qxMpFHDamZzACNvnis/OINJrud9VprOgjhZZ7mxcTbcVZaVgcTcnyC4b9d9PRrMG2aCv0BO1eb/qnlmSogQPfoKEORJcwaUcLgabww+Taa9hJSZ9l8yEQamj+XIgr6yzGKgCvlG4lTdHM2tQdpgATZvR7/pBz
>>>
>>> and produces the following output
>>>
>>> INFO:teuthology.orchestra.run.err:[91.121.254.229]: /home/ubuntu/cephtest/lo1308082328/adjust-ulimits: 4: ulimit: error setting limit (Operation not permitted)
>>>
>>> and the full output is at
>>>
>>> http://pastealacon.com/32957
>
> Refreshed with a longer timeout.
>
> http://pastealacon.com/32963
>>>
>>> as if
>>>
>>> /home/ubuntu/cephtest/lo1308082328/adjust-ulimits ceph-coverage /home/ubuntu/cephtest/lo1308082328/archive/coverage monmaptool --create --clobber --add a 91.121.254.229:6789 --add c 91.121.254.229:6790 --add b 91.121.254.237:6789 --print /home/ubuntu/cephtest/lo1308082328/monmap'
>>>
>>> was run without a leading sudo. I tried running sudo adjust-ulimits echo foo manually on the target and it works.
>>>
>>> I would very much appreciate a hint ;-)
>>>
>>> Cheers
>>>
>

Re: teuthology : ulimit: error

[Loic Dachary]

[-- Attachment #1: Type: text/plain, Size: 3624 bytes --]

Hi Dan,

That was indeed the solution :-)

Thanks !

On 13/08/2013 04:53, Dan Mick wrote:
> Ah, there's another we apply universally to our test systems, apparently:
> 
> '/etc/security/limits.d/ubuntu.conf'
> 
> ubuntu hard nofile 16384
> 
> and the tests run as user "ubuntu".  Line 4 of the script is the nofile setting.
> 
> On 08/10/2013 01:34 AM, Loic Dachary wrote:
>>
>>
>> On 10/08/2013 07:35, Dan Mick wrote:
>>> IIRC we had to adjust settings in /etc/security to allow ulimit adjustment of at least core:
>>>
>>> sed -i 's/^#\*.*soft.*core.*0/\*                soft    core unlimited/g' /etc/security/limits.conf
>>>
>>> or something like that.  That seems to apply to centos/fedora/redhat systems.
>>
>> Hi Dan,
>>
>> This is Ubuntu and it does work with sudo. It probably is something about how it's called rather than a system limit.
>>
>> Cheers
>>
>>>
>>>
>>> On 08/08/2013 02:52 PM, Loic Dachary wrote:
>>>> Hi,
>>>>
>>>> Trying to use Ubuntu precise virtual machines as teuthology targets ( making sure they have 2GB of RAM because ceph-test-dbg will not even install with 1GB of RAM ;-) and installing the key with
>>>>
>>>> wget -q -O- 'https://ceph.com/git/?p=ceph.git;a=blob_plain;f=keys/release.asc' | sudo apt-key add -
>>>>
>>>> teuthology runs with
>>>>
>>>> ./virtualenv/bin/teuthology job.yaml
>>>>
>>>> where job.yaml is
>>>>
>>>> check-locks: false
>>>> roles:
>>>> - - mon.a
>>>>     - mon.c
>>>>     - osd.0
>>>> - - mon.b
>>>>     - osd.1
>>>>     - client.0
>>>> tasks:
>>>> - install:
>>>>      project: ceph
>>>>      branch: wip-5510
>>>> - ceph:
>>>>      fs: ext4
>>>> - workunit:
>>>>       clients:
>>>>         all:
>>>>         - filestore/filestore.sh
>>>> targets:
>>>>      ubuntu@91.121.254.229: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDOSit20EyZ2AKCvk2tnMsdQ6LmVRutvBmZb0awV9Z2EduJa0fYPrReYRb9ZhGRq2PJe0zgpFPKr8s4gGay+tL0+dFkju5uyABqMGeITlJCifd+RhM0MCVllgIzekDwVb0n6ydTS8k7GVFyYv8ZC0TPgbfcDcEtSEgqJNRJ0o1Bh8swuTn+cipanNDRVK39tOqJdfptUxak+TD+5QY8CGFdXdEQYP7VsYJ+jQHw73O2xbuPgfv5Shbmt+qGWLToxFqKca3owMtkvFeONgYUdujgg9qr7Q9p0+HhCFCXB8v4N2I7NSbWNdpGqyJLdLqwJ70gEeNlOhm5F7IsXfVxTapB
>>>>      ubuntu@91.121.254.237: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCXVzhedORtmEmCeZJ4Ssg8wfqpYyH9W/Aa+j6CvPHSAkzZ48zXqVBATxm6S8sIIqfKkz1hWpNssx6uUotbm8k/ZatMddsd932+Di136l/HUhp6O8iIFze56kjWpyDpRPw2MM0V+OKmsiHZDfMj9ATt6ChgXfRsm23MUlmnoFHvtfwvFBnvBjcZPN7qxMpFHDamZzACNvnis/OINJrud9VprOgjhZZ7mxcTbcVZaVgcTcnyC4b9d9PRrMG2aCv0BO1eb/qnlmSogQPfoKEORJcwaUcLgabww+Taa9hJSZ9l8yEQamj+XIgr6yzGKgCvlG4lTdHM2tQdpgATZvR7/pBz
>>>>
>>>> and produces the following output
>>>>
>>>> INFO:teuthology.orchestra.run.err:[91.121.254.229]: /home/ubuntu/cephtest/lo1308082328/adjust-ulimits: 4: ulimit: error setting limit (Operation not permitted)
>>>>
>>>> and the full output is at
>>>>
>>>> http://pastealacon.com/32957
>>
>> Refreshed with a longer timeout.
>>
>> http://pastealacon.com/32963
>>>>
>>>> as if
>>>>
>>>> /home/ubuntu/cephtest/lo1308082328/adjust-ulimits ceph-coverage /home/ubuntu/cephtest/lo1308082328/archive/coverage monmaptool --create --clobber --add a 91.121.254.229:6789 --add c 91.121.254.229:6790 --add b 91.121.254.237:6789 --print /home/ubuntu/cephtest/lo1308082328/monmap'
>>>>
>>>> was run without a leading sudo. I tried running sudo adjust-ulimits echo foo manually on the target and it works.
>>>>
>>>> I would very much appreciate a hint ;-)
>>>>
>>>> Cheers
>>>>
>>

-- 
Loïc Dachary, Artisan Logiciel Libre
All that is necessary for the triumph of evil is that good people do nothing.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 261 bytes --]