provisioning clients in teuthology with an extra local filesystem

provisioning clients in teuthology with an extra local filesystem

[Jeff Layton]

I've been working on a patch series to overhaul the fscache code in the
kclient. I also have this (really old) tracker to add fscache testing to
teuthology:

    https://tracker.ceph.com/issues/6373

It would be ideal if the clients in such testing had a dedicated
filesystem mounted on /var/cache/fscache, so that if it fills up it
doesn't take down the rootfs with it. We'll also need to have
cachefilesd installed and running in the client hosts.

Is it possible to do this in teuthology? How would I approach this?

Thanks,
-- 
Jeff Layton <jlayton@redhat.com>

Re: provisioning clients in teuthology with an extra local filesystem

[David Galloway]

On 12/1/20 9:55 AM, Jeff Layton wrote:
> I've been working on a patch series to overhaul the fscache code in the
> kclient. I also have this (really old) tracker to add fscache testing to
> teuthology:
> 
>     https://tracker.ceph.com/issues/6373
> 
> It would be ideal if the clients in such testing had a dedicated
> filesystem mounted on /var/cache/fscache, so that if it fills up it
> doesn't take down the rootfs with it. We'll also need to have
> cachefilesd installed and running in the client hosts.
> 
> Is it possible to do this in teuthology? How would I approach this?
> 

I think I can make this happen pretty easily in ceph-cm-ansible.  What
I'd need from you is desired filesystem type and size.  Once I'm done
with my end, we'll need to create a cephlab.ansible overrides yaml
fragment to stick in that suite's qa directory.

Re: provisioning clients in teuthology with an extra local filesystem

[Jeff Layton]

On Tue, 2020-12-01 at 10:22 -0500, David Galloway wrote:
> On 12/1/20 9:55 AM, Jeff Layton wrote:
> > I've been working on a patch series to overhaul the fscache code in the
> > kclient. I also have this (really old) tracker to add fscache testing to
> > teuthology:
> > 
> >     https://tracker.ceph.com/issues/6373
> > 
> > It would be ideal if the clients in such testing had a dedicated
> > filesystem mounted on /var/cache/fscache, so that if it fills up it
> > doesn't take down the rootfs with it. We'll also need to have
> > cachefilesd installed and running in the client hosts.
> > 
> > Is it possible to do this in teuthology? How would I approach this?
> > 
> 
> I think I can make this happen pretty easily in ceph-cm-ansible.  What
> I'd need from you is desired filesystem type and size.  Once I'm done
> with my end, we'll need to create a cephlab.ansible overrides yaml
> fragment to stick in that suite's qa directory.
> 

Ok, cool:

fstype: xfs or ext4 (either is fine)
size: ~50g or so would be ideal, but we can probably get away with less 
      if necessary

Thanks!
-- 
Jeff Layton <jlayton@redhat.com>

Re: provisioning clients in teuthology with an extra local filesystem

[David Galloway]

On 12/1/20 10:24 AM, Jeff Layton wrote:
> On Tue, 2020-12-01 at 10:22 -0500, David Galloway wrote:
>> On 12/1/20 9:55 AM, Jeff Layton wrote:
>>> I've been working on a patch series to overhaul the fscache code in the
>>> kclient. I also have this (really old) tracker to add fscache testing to
>>> teuthology:
>>>
>>>     https://tracker.ceph.com/issues/6373
>>>
>>> It would be ideal if the clients in such testing had a dedicated
>>> filesystem mounted on /var/cache/fscache, so that if it fills up it
>>> doesn't take down the rootfs with it. We'll also need to have
>>> cachefilesd installed and running in the client hosts.
>>>
>>> Is it possible to do this in teuthology? How would I approach this?
>>>
>>
>> I think I can make this happen pretty easily in ceph-cm-ansible.  What
>> I'd need from you is desired filesystem type and size.  Once I'm done
>> with my end, we'll need to create a cephlab.ansible overrides yaml
>> fragment to stick in that suite's qa directory.
>>
> 
> Ok, cool:
> 
> fstype: xfs or ext4 (either is fine)
> size: ~50g or so would be ideal, but we can probably get away with less 
>       if necessary
> 

Getting there...

https://github.com/ceph/ceph-cm-ansible/pull/592

Can you give me an optimal cachefilesd.conf and I'll set that up in
ceph-cm-ansible too?

Re: provisioning clients in teuthology with an extra local filesystem

[Jeff Layton]

On Tue, 2020-12-01 at 14:23 -0500, David Galloway wrote:
> 
> On 12/1/20 10:24 AM, Jeff Layton wrote:
> > On Tue, 2020-12-01 at 10:22 -0500, David Galloway wrote:
> > > On 12/1/20 9:55 AM, Jeff Layton wrote:
> > > > I've been working on a patch series to overhaul the fscache code in the
> > > > kclient. I also have this (really old) tracker to add fscache testing to
> > > > teuthology:
> > > > 
> > > >     https://tracker.ceph.com/issues/6373
> > > > 
> > > > It would be ideal if the clients in such testing had a dedicated
> > > > filesystem mounted on /var/cache/fscache, so that if it fills up it
> > > > doesn't take down the rootfs with it. We'll also need to have
> > > > cachefilesd installed and running in the client hosts.
> > > > 
> > > > Is it possible to do this in teuthology? How would I approach this?
> > > > 
> > > 
> > > I think I can make this happen pretty easily in ceph-cm-ansible.  What
> > > I'd need from you is desired filesystem type and size.  Once I'm done
> > > with my end, we'll need to create a cephlab.ansible overrides yaml
> > > fragment to stick in that suite's qa directory.
> > > 
> > 
> > Ok, cool:
> > 
> > fstype: xfs or ext4 (either is fine)
> > size: ~50g or so would be ideal, but we can probably get away with less 
> >       if necessary
> > 
> 
> Getting there...
> 
> https://github.com/ceph/ceph-cm-ansible/pull/592
> 
> Can you give me an optimal cachefilesd.conf and I'll set that up in
> ceph-cm-ansible too?
> 

I never actually tweak the defaults in my local testing, so let's start
with a default cachefilesd.conf as installed by the package. If we need
to tweak it later, then we'll go from there.

Thanks!
-- 
Jeff Layton <jlayton@redhat.com>

Re: provisioning clients in teuthology with an extra local filesystem

[David Galloway]

[-- Attachment #1: Type: text/plain, Size: 2017 bytes --]


On 12/1/20 3:07 PM, Jeff Layton wrote:
> On Tue, 2020-12-01 at 14:23 -0500, David Galloway wrote:
>>
>> On 12/1/20 10:24 AM, Jeff Layton wrote:
>>> On Tue, 2020-12-01 at 10:22 -0500, David Galloway wrote:
>>>> On 12/1/20 9:55 AM, Jeff Layton wrote:
>>>>> I've been working on a patch series to overhaul the fscache code in the
>>>>> kclient. I also have this (really old) tracker to add fscache testing to
>>>>> teuthology:
>>>>>
>>>>>     https://tracker.ceph.com/issues/6373
>>>>>
>>>>> It would be ideal if the clients in such testing had a dedicated
>>>>> filesystem mounted on /var/cache/fscache, so that if it fills up it
>>>>> doesn't take down the rootfs with it. We'll also need to have
>>>>> cachefilesd installed and running in the client hosts.
>>>>>
>>>>> Is it possible to do this in teuthology? How would I approach this?
>>>>>
>>>>
>>>> I think I can make this happen pretty easily in ceph-cm-ansible.  What
>>>> I'd need from you is desired filesystem type and size.  Once I'm done
>>>> with my end, we'll need to create a cephlab.ansible overrides yaml
>>>> fragment to stick in that suite's qa directory.
>>>>
>>>
>>> Ok, cool:
>>>
>>> fstype: xfs or ext4 (either is fine)
>>> size: ~50g or so would be ideal, but we can probably get away with less 
>>>       if necessary
>>>
>>
>> Getting there...
>>
>> https://github.com/ceph/ceph-cm-ansible/pull/592
>>
>> Can you give me an optimal cachefilesd.conf and I'll set that up in
>> ceph-cm-ansible too?
>>
> 
> I never actually tweak the defaults in my local testing, so let's start
> with a default cachefilesd.conf as installed by the package. If we need
> to tweak it later, then we'll go from there.
> 

K.  I made it optional and used the package-provided defaults.

Your requested changes are implemented in ceph-cm-ansible.  I've
attached the yaml fragments you'll need to pass on the
`teuthology-suite` command line to set up the partition, filesystem, and
cachefilesd service.

e.g., teuthology-suite ... ~/fs-cache-smithi.yml

[-- Attachment #2: fscache-smithi.yml --]
[-- Type: application/x-yaml, Size: 620 bytes --]

[-- Attachment #3: fscache-gibba.yml --]
[-- Type: application/x-yaml, Size: 611 bytes --]

Re: provisioning clients in teuthology with an extra local filesystem

[Jeff Layton]

On Tue, 2020-12-01 at 16:31 -0500, David Galloway wrote:
> On 12/1/20 3:07 PM, Jeff Layton wrote:
> > On Tue, 2020-12-01 at 14:23 -0500, David Galloway wrote:
> > > 
> > > On 12/1/20 10:24 AM, Jeff Layton wrote:
> > > > On Tue, 2020-12-01 at 10:22 -0500, David Galloway wrote:
> > > > > On 12/1/20 9:55 AM, Jeff Layton wrote:
> > > > > > I've been working on a patch series to overhaul the fscache code in the
> > > > > > kclient. I also have this (really old) tracker to add fscache testing to
> > > > > > teuthology:
> > > > > > 
> > > > > >     https://tracker.ceph.com/issues/6373
> > > > > > 
> > > > > > It would be ideal if the clients in such testing had a dedicated
> > > > > > filesystem mounted on /var/cache/fscache, so that if it fills up it
> > > > > > doesn't take down the rootfs with it. We'll also need to have
> > > > > > cachefilesd installed and running in the client hosts.
> > > > > > 
> > > > > > Is it possible to do this in teuthology? How would I approach this?
> > > > > > 
> > > > > 
> > > > > I think I can make this happen pretty easily in ceph-cm-ansible.  What
> > > > > I'd need from you is desired filesystem type and size.  Once I'm done
> > > > > with my end, we'll need to create a cephlab.ansible overrides yaml
> > > > > fragment to stick in that suite's qa directory.
> > > > > 
> > > > 
> > > > Ok, cool:
> > > > 
> > > > fstype: xfs or ext4 (either is fine)
> > > > size: ~50g or so would be ideal, but we can probably get away with less 
> > > >       if necessary
> > > > 
> > > 
> > > Getting there...
> > > 
> > > https://github.com/ceph/ceph-cm-ansible/pull/592
> > > 
> > > Can you give me an optimal cachefilesd.conf and I'll set that up in
> > > ceph-cm-ansible too?
> > > 
> > 
> > I never actually tweak the defaults in my local testing, so let's start
> > with a default cachefilesd.conf as installed by the package. If we need
> > to tweak it later, then we'll go from there.
> > 
> 
> K.  I made it optional and used the package-provided defaults.
> 
> Your requested changes are implemented in ceph-cm-ansible.  I've
> attached the yaml fragments you'll need to pass on the
> `teuthology-suite` command line to set up the partition, filesystem, and
> cachefilesd service.
> 
> e.g., teuthology-suite ... ~/fs-cache-smithi.yml

Many thanks! I now need to figure out how to pass arbitrary mount
options to the kclient. I think that might need some plumbing down in
ceph/qa. I'll sort that out soon and test this.

Cheers!
-- 
Jeff Layton <jlayton@redhat.com>