cephfs survey results

cephfs survey results

[Sage Weil]

In the Ceph session at the OpenStack summit someone asked what the CephFS 
survey results looked like.  Here's the link:

	https://www.surveymonkey.com/results/SM-L5JV7WXL/

In short, people want

fsck
multimds
snapshots
quotas

sage

Re: cephfs survey results

[Blair Bethwaite]

On 4 November 2014 01:50, Sage Weil <sage-BnTBU8nroG7k1uMJSBkQmQ@public.gmane.org> wrote:
> In the Ceph session at the OpenStack summit someone asked what the CephFS
> survey results looked like.

Thanks Sage, that was me!

>  Here's the link:
>
>         https://www.surveymonkey.com/results/SM-L5JV7WXL/
>
> In short, people want
>
> fsck
> multimds
> snapshots
> quotas

TBH I'm a bit surprised by a couple of these and hope maybe you guys
will apply a certain amount of filtering on this...

fsck and quotas were there for me, but multimds and snapshots are what
I'd consider "icing" features - they're nice to have but not on the
critical path to using cephfs instead of e.g. nfs in a production
setting. I'd have thought stuff like small file performance and
gateway support was much more relevant to uptake and
positive/pain-free UX. Interested to hear others rationale here.

-- 
Cheers,
~Blairo

Re: cephfs survey results

[Sage Weil]

On Tue, 4 Nov 2014, Blair Bethwaite wrote:
> On 4 November 2014 01:50, Sage Weil <sage@newdream.net> wrote:
> > In the Ceph session at the OpenStack summit someone asked what the CephFS
> > survey results looked like.
> 
> Thanks Sage, that was me!
> 
> >  Here's the link:
> >
> >         https://www.surveymonkey.com/results/SM-L5JV7WXL/
> >
> > In short, people want
> >
> > fsck
> > multimds
> > snapshots
> > quotas
> 
> TBH I'm a bit surprised by a couple of these and hope maybe you guys
> will apply a certain amount of filtering on this...
> 
> fsck and quotas were there for me, but multimds and snapshots are what
> I'd consider "icing" features - they're nice to have but not on the
> critical path to using cephfs instead of e.g. nfs in a production
> setting. I'd have thought stuff like small file performance and
> gateway support was much more relevant to uptake and
> positive/pain-free UX. Interested to hear others rationale here.

Yeah, I agree, and am taking the results with a grain of salt.  I 
think the results are heavily influenced by the order they were 
originally listed (I whish surveymonkey would randomize is for each 
person or something).

fsck is a clear #1.  Everybody wants multimds, but I think very few 
actually need it at this point.  We'll be merging a soft quota patch 
shortly, and things like performance (adding the inline data support to 
the kernel client, for instance) will probably compete with getting 
snapshots working (as part of a larger subvolume infrastructure).  That's 
my guess at least; for now, we're really focused on fsck and hard 
usability edges and haven't set priorities beyond that.

We're definitely interested in hearing feedback on this strategy, and on 
peoples' experiences with giant so far...

sage

Re: cephfs survey results

[Wido den Hollander]

On 11/04/2014 10:02 AM, Sage Weil wrote:
> On Tue, 4 Nov 2014, Blair Bethwaite wrote:
>> On 4 November 2014 01:50, Sage Weil <sage-BnTBU8nroG7k1uMJSBkQmQ@public.gmane.org> wrote:
>>> In the Ceph session at the OpenStack summit someone asked what the CephFS
>>> survey results looked like.
>>
>> Thanks Sage, that was me!
>>
>>>  Here's the link:
>>>
>>>         https://www.surveymonkey.com/results/SM-L5JV7WXL/
>>>
>>> In short, people want
>>>
>>> fsck
>>> multimds
>>> snapshots
>>> quotas
>>
>> TBH I'm a bit surprised by a couple of these and hope maybe you guys
>> will apply a certain amount of filtering on this...
>>
>> fsck and quotas were there for me, but multimds and snapshots are what
>> I'd consider "icing" features - they're nice to have but not on the
>> critical path to using cephfs instead of e.g. nfs in a production
>> setting. I'd have thought stuff like small file performance and
>> gateway support was much more relevant to uptake and
>> positive/pain-free UX. Interested to hear others rationale here.
> 
> Yeah, I agree, and am taking the results with a grain of salt.  I 
> think the results are heavily influenced by the order they were 
> originally listed (I whish surveymonkey would randomize is for each 
> person or something).
> 
> fsck is a clear #1.  Everybody wants multimds, but I think very few 
> actually need it at this point.  We'll be merging a soft quota patch 
> shortly, and things like performance (adding the inline data support to 
> the kernel client, for instance) will probably compete with getting 
> snapshots working (as part of a larger subvolume infrastructure).  That's 
> my guess at least; for now, we're really focused on fsck and hard 
> usability edges and haven't set priorities beyond that.
> 
> We're definitely interested in hearing feedback on this strategy, and on 
> peoples' experiences with giant so far...
> 

I think the approach is correct. Everybody I talk to wants to kick out
their NFS server, but you don't need multi MDS for that. Active/Standby
is just fine.

Wido

> sage
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 


-- 
Wido den Hollander
42on B.V.
Ceph trainer and consultant

Phone: +31 (0)20 700 9902
Skype: contact42on

Re: cephfs survey results

[Mark Kirkwood]

On 04/11/14 22:02, Sage Weil wrote:
> On Tue, 4 Nov 2014, Blair Bethwaite wrote:
>> On 4 November 2014 01:50, Sage Weil <sage-BnTBU8nroG7k1uMJSBkQmQ@public.gmane.org> wrote:
>>> In the Ceph session at the OpenStack summit someone asked what the CephFS
>>> survey results looked like.
>>
>> Thanks Sage, that was me!
>>
>>>   Here's the link:
>>>
>>>          https://www.surveymonkey.com/results/SM-L5JV7WXL/
>>>
>>> In short, people want
>>>
>>> fsck
>>> multimds
>>> snapshots
>>> quotas
>>
>> TBH I'm a bit surprised by a couple of these and hope maybe you guys
>> will apply a certain amount of filtering on this...
>>
>> fsck and quotas were there for me, but multimds and snapshots are what
>> I'd consider "icing" features - they're nice to have but not on the
>> critical path to using cephfs instead of e.g. nfs in a production
>> setting. I'd have thought stuff like small file performance and
>> gateway support was much more relevant to uptake and
>> positive/pain-free UX. Interested to hear others rationale here.
>
> Yeah, I agree, and am taking the results with a grain of salt.  I
> think the results are heavily influenced by the order they were
> originally listed (I whish surveymonkey would randomize is for each
> person or something).
>
> fsck is a clear #1.  Everybody wants multimds, but I think very few
> actually need it at this point.  We'll be merging a soft quota patch
> shortly, and things like performance (adding the inline data support to
> the kernel client, for instance) will probably compete with getting
> snapshots working (as part of a larger subvolume infrastructure).  That's
> my guess at least; for now, we're really focused on fsck and hard
> usability edges and haven't set priorities beyond that.
>
> We're definitely interested in hearing feedback on this strategy, and on
> peoples' experiences with giant so far...
>

Heh, not necessarily - I put multi mds in there, as we want the cephfs 
part to be of similar to the rest of ceph in its availability.

Maybe its because we are looking at plugging it in with an Openstack 
setup and for that you want everything to 'just look after itself'. If 
on the other hand we were wanting merely an nfs replacement, then sure 
multi mds not so important there.

regards

Mark

Re: cephfs survey results

[Sage Weil]

On Wed, 5 Nov 2014, Mark Kirkwood wrote:
> On 04/11/14 22:02, Sage Weil wrote:
> > On Tue, 4 Nov 2014, Blair Bethwaite wrote:
> > > On 4 November 2014 01:50, Sage Weil <sage-BnTBU8nroG7k1uMJSBkQmQ@public.gmane.org> wrote:
> > > > In the Ceph session at the OpenStack summit someone asked what the
> > > > CephFS
> > > > survey results looked like.
> > > 
> > > Thanks Sage, that was me!
> > > 
> > > >   Here's the link:
> > > > 
> > > >          https://www.surveymonkey.com/results/SM-L5JV7WXL/
> > > > 
> > > > In short, people want
> > > > 
> > > > fsck
> > > > multimds
> > > > snapshots
> > > > quotas
> > > 
> > > TBH I'm a bit surprised by a couple of these and hope maybe you guys
> > > will apply a certain amount of filtering on this...
> > > 
> > > fsck and quotas were there for me, but multimds and snapshots are what
> > > I'd consider "icing" features - they're nice to have but not on the
> > > critical path to using cephfs instead of e.g. nfs in a production
> > > setting. I'd have thought stuff like small file performance and
> > > gateway support was much more relevant to uptake and
> > > positive/pain-free UX. Interested to hear others rationale here.
> > 
> > Yeah, I agree, and am taking the results with a grain of salt.  I
> > think the results are heavily influenced by the order they were
> > originally listed (I whish surveymonkey would randomize is for each
> > person or something).
> > 
> > fsck is a clear #1.  Everybody wants multimds, but I think very few
> > actually need it at this point.  We'll be merging a soft quota patch
> > shortly, and things like performance (adding the inline data support to
> > the kernel client, for instance) will probably compete with getting
> > snapshots working (as part of a larger subvolume infrastructure).  That's
> > my guess at least; for now, we're really focused on fsck and hard
> > usability edges and haven't set priorities beyond that.
> > 
> > We're definitely interested in hearing feedback on this strategy, and on
> > peoples' experiences with giant so far...
> > 
> 
> Heh, not necessarily - I put multi mds in there, as we want the cephfs part to
> be of similar to the rest of ceph in its availability.
> 
> Maybe its because we are looking at plugging it in with an Openstack setup and
> for that you want everything to 'just look after itself'. If on the other hand
> we were wanting merely an nfs replacement, then sure multi mds not so
> important there.

Important clarification: "multimds" == multiple *active* MDS's.  "single 
mds" means 1 active MDS and N standy's.  One perfectly valid strategy, 
for example, is to run a ceph-mds on *every* node and let the mon pick 
whichever one is active.  (That works as long as you have sufficient 
memory on all nodes.)

sage

Re: cephfs survey results

[Mark Kirkwood]

On 05/11/14 11:47, Sage Weil wrote:
> On Wed, 5 Nov 2014, Mark Kirkwood wrote:
>> On 04/11/14 22:02, Sage Weil wrote:
>>> On Tue, 4 Nov 2014, Blair Bethwaite wrote:
>>>> On 4 November 2014 01:50, Sage Weil <sage@newdream.net> wrote:
>>>>> In the Ceph session at the OpenStack summit someone asked what the
>>>>> CephFS
>>>>> survey results looked like.
>>>>
>>>> Thanks Sage, that was me!
>>>>
>>>>>    Here's the link:
>>>>>
>>>>>           https://www.surveymonkey.com/results/SM-L5JV7WXL/
>>>>>
>>>>> In short, people want
>>>>>
>>>>> fsck
>>>>> multimds
>>>>> snapshots
>>>>> quotas
>>>>
>>>> TBH I'm a bit surprised by a couple of these and hope maybe you guys
>>>> will apply a certain amount of filtering on this...
>>>>
>>>> fsck and quotas were there for me, but multimds and snapshots are what
>>>> I'd consider "icing" features - they're nice to have but not on the
>>>> critical path to using cephfs instead of e.g. nfs in a production
>>>> setting. I'd have thought stuff like small file performance and
>>>> gateway support was much more relevant to uptake and
>>>> positive/pain-free UX. Interested to hear others rationale here.
>>>
>>> Yeah, I agree, and am taking the results with a grain of salt.  I
>>> think the results are heavily influenced by the order they were
>>> originally listed (I whish surveymonkey would randomize is for each
>>> person or something).
>>>
>>> fsck is a clear #1.  Everybody wants multimds, but I think very few
>>> actually need it at this point.  We'll be merging a soft quota patch
>>> shortly, and things like performance (adding the inline data support to
>>> the kernel client, for instance) will probably compete with getting
>>> snapshots working (as part of a larger subvolume infrastructure).  That's
>>> my guess at least; for now, we're really focused on fsck and hard
>>> usability edges and haven't set priorities beyond that.
>>>
>>> We're definitely interested in hearing feedback on this strategy, and on
>>> peoples' experiences with giant so far...
>>>
>>
>> Heh, not necessarily - I put multi mds in there, as we want the cephfs part to
>> be of similar to the rest of ceph in its availability.
>>
>> Maybe its because we are looking at plugging it in with an Openstack setup and
>> for that you want everything to 'just look after itself'. If on the other hand
>> we were wanting merely an nfs replacement, then sure multi mds not so
>> important there.
>
> Important clarification: "multimds" == multiple *active* MDS's.  "single
> mds" means 1 active MDS and N standy's.  One perfectly valid strategy,
> for example, is to run a ceph-mds on *every* node and let the mon pick
> whichever one is active.  (That works as long as you have sufficient
> memory on all nodes.)
>

Righty, so I think I've (plus a few others perhaps) misunderstood the 
nature of the 'promotion mechanism' for 1 active several standby design 
- I was under the (possibly wrong) impression that you needed to 'do 
something' to make a standby active? If not then yeah it would be fine, 
sorry!

Re: cephfs survey results

[Shain Miley]

+1 for fsck and snapshots, being able to have snapshot backups and 
protect against accidental deletion, etc is something we are really 
looking forward to.

Thanks,

Shain


On 11/04/2014 04:02 AM, Sage Weil wrote:
> On Tue, 4 Nov 2014, Blair Bethwaite wrote:
>> On 4 November 2014 01:50, Sage Weil <sage-BnTBU8nroG7k1uMJSBkQmQ@public.gmane.org> wrote:
>>> In the Ceph session at the OpenStack summit someone asked what the CephFS
>>> survey results looked like.
>> Thanks Sage, that was me!
>>
>>>   Here's the link:
>>>
>>>          https://www.surveymonkey.com/results/SM-L5JV7WXL/
>>>
>>> In short, people want
>>>
>>> fsck
>>> multimds
>>> snapshots
>>> quotas
>> TBH I'm a bit surprised by a couple of these and hope maybe you guys
>> will apply a certain amount of filtering on this...
>>
>> fsck and quotas were there for me, but multimds and snapshots are what
>> I'd consider "icing" features - they're nice to have but not on the
>> critical path to using cephfs instead of e.g. nfs in a production
>> setting. I'd have thought stuff like small file performance and
>> gateway support was much more relevant to uptake and
>> positive/pain-free UX. Interested to hear others rationale here.
> Yeah, I agree, and am taking the results with a grain of salt.  I
> think the results are heavily influenced by the order they were
> originally listed (I whish surveymonkey would randomize is for each
> person or something).
>
> fsck is a clear #1.  Everybody wants multimds, but I think very few
> actually need it at this point.  We'll be merging a soft quota patch
> shortly, and things like performance (adding the inline data support to
> the kernel client, for instance) will probably compete with getting
> snapshots working (as part of a larger subvolume infrastructure).  That's
> my guess at least; for now, we're really focused on fsck and hard
> usability edges and haven't set priorities beyond that.
>
> We're definitely interested in hearing feedback on this strategy, and on
> peoples' experiences with giant so far...
>
> sage
> _______________________________________________
> ceph-users mailing list
> ceph-users-idqoXFIVOFJgJs9I8MT0rw@public.gmane.org
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

-- 
Shain Miley | Manager of Systems and Infrastructure, Digital Media | 
smiley-VrEXfa0MiHk@public.gmane.org | 202.513.3649

Re: cephfs survey results

[Mariusz Gronczewski]

[-- Attachment #1.1: Type: text/plain, Size: 1351 bytes --]

On Tue, 4 Nov 2014 10:36:07 +1100, Blair Bethwaite
<blair.bethwaite-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:

> 
> TBH I'm a bit surprised by a couple of these and hope maybe you guys
> will apply a certain amount of filtering on this...
> 
> fsck and quotas were there for me, but multimds and snapshots are what
> I'd consider "icing" features - they're nice to have but not on the
> critical path to using cephfs instead of e.g. nfs in a production
> setting. I'd have thought stuff like small file performance and
> gateway support was much more relevant to uptake and
> positive/pain-free UX. Interested to hear others rationale here.
> 

Those are related; if small file performance will be enough for one
MDS to handle high load with a lot of small files (typical case of
webserver), having multiple acive MDS will be less of a priority;

And if someone currently have OSD on bunch of relatively weak nodes,
again, having active-active setup with MDS will be more interesting to
him than someone that can just buy new fast machine for it.


-- 
Mariusz Gronczewski, Administrator

Efigence S. A.
ul. Wołoska 9a, 02-583 Warszawa
T: [+48] 22 380 13 13
F: [+48] 22 380 13 14
E: mariusz.gronczewski-EWQ+Go7UwTxWk0Htik3J/w@public.gmane.org
<mailto:mariusz.gronczewski-EWQ+Go7UwTxWk0Htik3J/w@public.gmane.org>

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

[-- Attachment #2: Type: text/plain, Size: 178 bytes --]

_______________________________________________
ceph-users mailing list
ceph-users-idqoXFIVOFJgJs9I8MT0rw@public.gmane.org
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: cephfs survey results

[Scottix]

Agreed Multi-MDS is a nice to have but not required for full production use.
TBH stability and recovery will win any IT person dealing with filesystems.

On Tue, Nov 4, 2014 at 7:33 AM, Mariusz Gronczewski
<mariusz.gronczewski@efigence.com> wrote:
> On Tue, 4 Nov 2014 10:36:07 +1100, Blair Bethwaite
> <blair.bethwaite@gmail.com> wrote:
>
>>
>> TBH I'm a bit surprised by a couple of these and hope maybe you guys
>> will apply a certain amount of filtering on this...
>>
>> fsck and quotas were there for me, but multimds and snapshots are what
>> I'd consider "icing" features - they're nice to have but not on the
>> critical path to using cephfs instead of e.g. nfs in a production
>> setting. I'd have thought stuff like small file performance and
>> gateway support was much more relevant to uptake and
>> positive/pain-free UX. Interested to hear others rationale here.
>>
>
> Those are related; if small file performance will be enough for one
> MDS to handle high load with a lot of small files (typical case of
> webserver), having multiple acive MDS will be less of a priority;
>
> And if someone currently have OSD on bunch of relatively weak nodes,
> again, having active-active setup with MDS will be more interesting to
> him than someone that can just buy new fast machine for it.
>
>
> --
> Mariusz Gronczewski, Administrator
>
> Efigence S. A.
> ul. Wołoska 9a, 02-583 Warszawa
> T: [+48] 22 380 13 13
> F: [+48] 22 380 13 14
> E: mariusz.gronczewski@efigence.com
> <mailto:mariusz.gronczewski@efigence.com>
>
> _______________________________________________
> ceph-users mailing list
> ceph-users@lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>



-- 
Follow Me: @Scottix
http://about.me/scottix
Scottix@Gmail.com
_______________________________________________
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: cephfs survey results

[Blair Bethwaite]

[-- Attachment #1.1: Type: text/plain, Size: 1779 bytes --]

(Sorry for top posting - on phone)

Related to some extent. Multimds will help when your small file issue is
metadata bound, but I was actually thinking of data path random io latency.
So begs the question as to what the survey meant by it...?
On 04/11/2014 4:34 pm, "Mariusz Gronczewski" <
mariusz.gronczewski-EWQ+Go7UwTxWk0Htik3J/w@public.gmane.org> wrote:

> On Tue, 4 Nov 2014 10:36:07 +1100, Blair Bethwaite
> <blair.bethwaite-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>
> >
> > TBH I'm a bit surprised by a couple of these and hope maybe you guys
> > will apply a certain amount of filtering on this...
> >
> > fsck and quotas were there for me, but multimds and snapshots are what
> > I'd consider "icing" features - they're nice to have but not on the
> > critical path to using cephfs instead of e.g. nfs in a production
> > setting. I'd have thought stuff like small file performance and
> > gateway support was much more relevant to uptake and
> > positive/pain-free UX. Interested to hear others rationale here.
> >
>
> Those are related; if small file performance will be enough for one
> MDS to handle high load with a lot of small files (typical case of
> webserver), having multiple acive MDS will be less of a priority;
>
> And if someone currently have OSD on bunch of relatively weak nodes,
> again, having active-active setup with MDS will be more interesting to
> him than someone that can just buy new fast machine for it.
>
>
> --
> Mariusz Gronczewski, Administrator
>
> Efigence S. A.
> ul. Wołoska 9a, 02-583 Warszawa
> T: [+48] 22 380 13 13
> F: [+48] 22 380 13 14
> E: mariusz.gronczewski-EWQ+Go7UwTxWk0Htik3J/w@public.gmane.org
> <mailto:mariusz.gronczewski-EWQ+Go7UwTxWk0Htik3J/w@public.gmane.org>
>

[-- Attachment #1.2: Type: text/html, Size: 2567 bytes --]

[-- Attachment #2: Type: text/plain, Size: 178 bytes --]

_______________________________________________
ceph-users mailing list
ceph-users-idqoXFIVOFJgJs9I8MT0rw@public.gmane.org
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: cephfs survey results

[Patrick Hahn]

On Mon, Nov 3, 2014 at 6:36 PM, Blair Bethwaite
<blair.bethwaite@gmail.com> wrote:
> On 4 November 2014 01:50, Sage Weil <sage@newdream.net> wrote:
>> In the Ceph session at the OpenStack summit someone asked what the CephFS
>> survey results looked like.
>
> Thanks Sage, that was me!
>
>>  Here's the link:
>>
>>         https://www.surveymonkey.com/results/SM-L5JV7WXL/
>>
>> In short, people want
>>
>> fsck
>> multimds
>> snapshots
>> quotas
>
> TBH I'm a bit surprised by a couple of these and hope maybe you guys
> will apply a certain amount of filtering on this...
>
> fsck and quotas were there for me, but multimds and snapshots are what
> I'd consider "icing" features - they're nice to have but not on the
> critical path to using cephfs instead of e.g. nfs in a production
> setting. I'd have thought stuff like small file performance and
> gateway support was much more relevant to uptake and
> positive/pain-free UX. Interested to hear others rationale here.

For the use case we're looking at cephfs for at $dayjob we really need
snapshots. I think anyone building a cheap-and-deep cluster for
archival storage would like to be more than one errant rm -rf away
from a *very* long weekend.

Thanks,
-- 
Patrick Hahn