Re: CephFS use cases + MDS limitations

[Malcolm Haak <malcolm@sgi.com>]

Michael,

I haven't seen any on-list replies yet, so I wasn't sure if this was the 
right place. But I'll just reply and somebody will let me know if I am 
wrong.

The use cases I have encountered, in my clustered computing universe, 
were implemented with a different proprietary clustered file system. 
These file-systems were being used as home folders or "shared scratch" 
space. And the specific issues occur when you have users who 'misbehave' 
or have code that, by way of function create(and destroy) large numbers 
of files. And in the process bog down file-system access for everybody. 
I have not yet implemented ceph in production in this role but base 
testing shows it will encounter the same issues.

While it is ideal to not do such things to a clustered file system, it 
would be nice to be able to dedicate an MDS to specific sub folders 
without having to create a whole separate sub-file-system/mount-point 
(as is the current procedure with other solutions).

It would be really AWESOME to do this 'on the fly'. Having more than one 
MDS look after the whole file-system in an ACTIVE/ACTIVE fashion would 
be nice/ideal (as long as latency is not too negativity impacted), but 
really just being able to 'shard' the file-system up would be more than 
sufficient to solve most of the issues I usually encounter. Having this 
kind of functionality would be a 'killer feature' for this kind of workload.

I hope my wall of text makes sense. Please feel free to ping me with 
questions.

Regards

Malcolm Haak




On 04/11/13 09:53, Michael Sevilla wrote:
> Hi Ceph community,
>
> I’d like to get a feel for some of the problems that CephFS users are
> encountering with single MDS deployments. There were requests for
> stable distributed metadata/MDS services [1] and I’m guessing its
> because your workloads exhibit many, many metadata operations. Some of
> you mentioned opening many files in a directory for checkpointing,
> recursive stats on a directory, etc. [2] and I’d like more details,
> such as:
> - workloads/applications that stress the MDS service that would cause
> you to call for multi-MDS support
> - use cases for the Ceph file system (I’m not really too interested in
> users using CephFS to host VMs, since many of these use cases are
> migrating to RBD)
>
> I’m just trying to get an idea of what’s out there and the problems
> CephFS users encounter as a result of a bottlenecked MDS (single node
> or cluster).
>
> Thanks!
>
> Michael
>
> [1] CephFS MDS Status Discussion,
> http://ceph.com/dev-notes/cephfs-mds-status-discussion/
> [2] CephFS First Product Release Discussion,
> http://thread.gmane.org/gmane.comp.file-systems.ceph.devel/13524
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html