* Re: durability model [not found] <1401131148.7008.YahooMailNeo@web165003.mail.bf1.yahoo.com> @ 2014-05-27 6:50 ` Kyle Bader [not found] ` <1401197435.39621.YahooMailNeo@web165002.mail.bf1.yahoo.com> 0 siblings, 1 reply; 6+ messages in thread From: Kyle Bader @ 2014-05-27 6:50 UTC (permalink / raw) To: Koleos Fuskus; +Cc: loic@dachary.org, Ceph Development > Something that is not yet clear for me is how do we study the overlapping of different layers of reliability using the modelling tool. Is the idea to use erasure coding on top of another reliable layer like a RAID-based storage? If I understand correctly the current tool models disks, RAID, RADOS independently. But erasure coding is added on top of those layers, isn't it? You understand correctly in that the tool was originally written to analyze RADOS with replication and RAID, independently. Erasure coding can be used independently and I had considered the original scope to be modeling just that, a erasure coded pool in isolation. Being able to model durability of a tiered pool configuration would be an excellent next step, though. Good question! -- Kyle Bader - Inktank Senior Solution Architect ^ permalink raw reply [flat|nested] 6+ messages in thread
[parent not found: <1401197435.39621.YahooMailNeo@web165002.mail.bf1.yahoo.com>]
* Re: durability model [not found] ` <1401197435.39621.YahooMailNeo@web165002.mail.bf1.yahoo.com> @ 2014-05-27 14:24 ` Loic Dachary [not found] ` <1401284130.75330.YahooMailNeo@web165006.mail.bf1.yahoo.com> 0 siblings, 1 reply; 6+ messages in thread From: Loic Dachary @ 2014-05-27 14:24 UTC (permalink / raw) To: Koleos Fuskus; +Cc: Kyle Bader, Ceph Development [-- Attachment #1: Type: text/plain, Size: 2422 bytes --] On 27/05/2014 15:30, Koleos Fuskus wrote:> Hi Kyle, > Thanks for your answer! > I have other questions regarding the model. Sorry, I accumulated different doubts and the e-mail is a bit long. > > 1. What will be erasure model unit? Is it a pool or a placement group? I cannot see the best option here yet, probably because my inexperience with Ceph. A pool. > 2. Crush_ruleset for erasured pools: Are these crush-ruleset impacting on the reliability model? Should I define a default crush-ruleset? Unless I'm mistaken, the crush ruleset is only relevant to the durability model to the extent that it defines the failure domain. If a ruleset requires that no two shards of a given placement group (i.e. no two shards of a given object since a placement group is a set of object that share the same OSD for a given shard) are on the same host, it will lead to a model that is different from a ruleset that requires that no two shards are in the same datacenter. > 3. Metadata: Jerasure creates metadata files. How are those files stored in ceph (what kind of redundancy)? Should I care about them? I guess the jerasure metafiles are not important. If all objects stored in a pool share the same erasure-code-k and erasure-code-m, then is easy to re-generate the metadata file. However, if metadata file is lost and the pool contains erasured objects with different k and m, then repairing data will not be possible. Should I ignore this problem? Could you confirm that each pool has the same erasure parameters? Jerasure does not create a metadata file. > 4. Number of PG for an erasured-pool. The total PGs in the case of replication is calculated with this formula: Total PGs= (OSDs * 100)/Replicas. > > Should I replace Replicas for the stretch factor (storage overhead) in the case of erasure? > > 5.Parameters - Default values: > erasure-code-k (default 4) > erasure-code-m (default 2) > > OSD (default 200) > PG (default 100 x OSD) > stripes (default ????) I proposed an update to the documentation to clarify this : https://github.com/ceph/ceph/pull/1856 . Does it look sensible to you ? Cheers > 6. Units > It is hardcoded on the reliable tool: > KiB = 1000 > MiB = KiB * 1000 > GiB = MiB * 1000 > TiB = GiB * 1000 > PiB = TiB * 1000 > > Why KiB is 1000 when should be 1024? > Best, > Koleosfuscus > -- Loïc Dachary, Artisan Logiciel Libre [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 263 bytes --] ^ permalink raw reply [flat|nested] 6+ messages in thread
[parent not found: <1401284130.75330.YahooMailNeo@web165006.mail.bf1.yahoo.com>]
* Re: durability model [not found] ` <1401284130.75330.YahooMailNeo@web165006.mail.bf1.yahoo.com> @ 2014-05-28 14:05 ` Loic Dachary 2015-04-01 13:17 ` Milosz Tanski 0 siblings, 1 reply; 6+ messages in thread From: Loic Dachary @ 2014-05-28 14:05 UTC (permalink / raw) To: Koleos Fuskus; +Cc: Kyle Bader, Ceph Development [-- Attachment #1: Type: text/plain, Size: 1000 bytes --] On 28/05/2014 15:35, Koleos Fuskus wrote: > > >> 1. What will be erasure model unit? Is it a pool or a placement group? I cannot see the best option here yet, probably because my inexperience with Ceph. > > A pool. > OK. this is what I thought but some things are not clear enough. I am deploying Ceph to toying a bit more. > > >> 3. Metadata: Jerasure creates metadata files. How are those files stored in ceph (what kind of > Jerasure does not create a metadata file. > > It does on my machine! at least with jerasure 1.2 on mac os :) If you mean the files that are created by the command line examples provided with jerasure (1.2 or 2), they are not required and only meant for test purposes. Cheers > > > > > I proposed an update to the documentation to clarify this : https://github.com/ceph/ceph/pull/1856 . Does it look sensible to you ? > > > > THanks! > > Cheers, > > > Koleosfuscus > -- Loïc Dachary, Artisan Logiciel Libre [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 263 bytes --] ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: durability model 2014-05-28 14:05 ` Loic Dachary @ 2015-04-01 13:17 ` Milosz Tanski 2015-04-01 13:49 ` Sage Weil 0 siblings, 1 reply; 6+ messages in thread From: Milosz Tanski @ 2015-04-01 13:17 UTC (permalink / raw) To: Loic Dachary; +Cc: Koleos Fuskus, Kyle Bader, Ceph Development On Wed, May 28, 2014 at 10:05 AM, Loic Dachary <loic@dachary.org> wrote: > > > On 28/05/2014 15:35, Koleos Fuskus wrote: >> >> >>> 1. What will be erasure model unit? Is it a pool or a placement group? I cannot see the best option here yet, probably because my inexperience with Ceph. >> >> A pool. >> OK. this is what I thought but some things are not clear enough. I am deploying Ceph to toying a bit more. >> >> >>> 3. Metadata: Jerasure creates metadata files. How are those files stored in ceph (what kind of >> Jerasure does not create a metadata file. >> >> It does on my machine! at least with jerasure 1.2 on mac os :) > > If you mean the files that are created by the command line examples provided with jerasure (1.2 or 2), they are not required and only meant for test purposes. > > Cheers > >> >> >> >> >> I proposed an update to the documentation to clarify this : https://github.com/ceph/ceph/pull/1856 . Does it look sensible to you ? >> >> Recently while considering a hardware update on the Ceph cluster I've been thinking about the durability model of the new OSD nodes once the cluster is done migrating over. So I came looking through these old emails. I think it might be very useful to have a web tool that describes (or visualizes) the current cluster durability model, or better yet helps you play with the variables to visualize the changes. If we can formulate the rules model based on some common variables (about machines, OSDs, racks) and replication rules (count or erasure coding paramaters). Then we can spit out useful information like: - How many nodes / racks / osds do you need to lose to have data offline - What percentage of data is going to be unavailable if you lose X resources - What's the probability of data loss over time - How does the readability model impact your cluster network traffic, repair traffic I think it would be stair-forward to build the web app. In fact, I'll volunteer myself for that task. Before we start that I would suggest we try to get the model in something like Google Docs Sheets so we can use it as a basis. -- Milosz Tanski CTO 16 East 34th Street, 15th floor New York, NY 10016 p: 646-253-9055 e: milosz@adfin.com ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: durability model 2015-04-01 13:17 ` Milosz Tanski @ 2015-04-01 13:49 ` Sage Weil 2015-04-01 14:18 ` Milosz Tanski 0 siblings, 1 reply; 6+ messages in thread From: Sage Weil @ 2015-04-01 13:49 UTC (permalink / raw) To: Milosz Tanski; +Cc: Loic Dachary, Koleos Fuskus, Kyle Bader, Ceph Development On Wed, 1 Apr 2015, Milosz Tanski wrote: > On Wed, May 28, 2014 at 10:05 AM, Loic Dachary <loic@dachary.org> wrote: > > > > > > On 28/05/2014 15:35, Koleos Fuskus wrote: > >> > >> > >>> 1. What will be erasure model unit? Is it a pool or a placement group? I cannot see the best option here yet, probably because my inexperience with Ceph. > >> > >> A pool. > >> OK. this is what I thought but some things are not clear enough. I am deploying Ceph to toying a bit more. > >> > >> > >>> 3. Metadata: Jerasure creates metadata files. How are those files stored in ceph (what kind of > >> Jerasure does not create a metadata file. > >> > >> It does on my machine! at least with jerasure 1.2 on mac os :) > > > > If you mean the files that are created by the command line examples provided with jerasure (1.2 or 2), they are not required and only meant for test purposes. > > > > Cheers > > > >> > >> > >> > >> > >> I proposed an update to the documentation to clarify this : https://github.com/ceph/ceph/pull/1856 . Does it look sensible to you ? > >> > >> > > Recently while considering a hardware update on the Ceph cluster I've > been thinking about the durability model of the new OSD nodes once the > cluster is done migrating over. So I came looking through these old > emails. I think it might be very useful to have a web tool that > describes (or visualizes) the current cluster durability model, or > better yet helps you play with the variables to visualize the changes. > > If we can formulate the rules model based on some common variables > (about machines, OSDs, racks) and replication rules (count or erasure > coding paramaters). Then we can spit out useful information like: > - How many nodes / racks / osds do you need to lose to have data offline > - What percentage of data is going to be unavailable if you lose X resources > - What's the probability of data loss over time > - How does the readability model impact your cluster network traffic, > repair traffic > > I think it would be stair-forward to build the web app. In fact, I'll > volunteer myself for that task. Before we start that I would suggest > we try to get the model in something like Google Docs Sheets so we can > use it as a basis. If can you haven't seen this: https://github.com/ceph/ceph-tools/tree/master/models/reliability It's not web-based but it probably could be changed to be. I definitely like the idea of having a hosted version. sage ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: durability model 2015-04-01 13:49 ` Sage Weil @ 2015-04-01 14:18 ` Milosz Tanski 0 siblings, 0 replies; 6+ messages in thread From: Milosz Tanski @ 2015-04-01 14:18 UTC (permalink / raw) To: Sage Weil; +Cc: Loic Dachary, Koleos Fuskus, Kyle Bader, Ceph Development On Wed, Apr 1, 2015 at 9:49 AM, Sage Weil <sage@newdream.net> wrote: > On Wed, 1 Apr 2015, Milosz Tanski wrote: >> On Wed, May 28, 2014 at 10:05 AM, Loic Dachary <loic@dachary.org> wrote: >> > >> > >> > On 28/05/2014 15:35, Koleos Fuskus wrote: >> >> >> >> >> >>> 1. What will be erasure model unit? Is it a pool or a placement group? I cannot see the best option here yet, probably because my inexperience with Ceph. >> >> >> >> A pool. >> >> OK. this is what I thought but some things are not clear enough. I am deploying Ceph to toying a bit more. >> >> >> >> >> >>> 3. Metadata: Jerasure creates metadata files. How are those files stored in ceph (what kind of >> >> Jerasure does not create a metadata file. >> >> >> >> It does on my machine! at least with jerasure 1.2 on mac os :) >> > >> > If you mean the files that are created by the command line examples provided with jerasure (1.2 or 2), they are not required and only meant for test purposes. >> > >> > Cheers >> > >> >> >> >> >> >> >> >> >> >> I proposed an update to the documentation to clarify this : https://github.com/ceph/ceph/pull/1856 . Does it look sensible to you ? >> >> >> >> >> >> Recently while considering a hardware update on the Ceph cluster I've >> been thinking about the durability model of the new OSD nodes once the >> cluster is done migrating over. So I came looking through these old >> emails. I think it might be very useful to have a web tool that >> describes (or visualizes) the current cluster durability model, or >> better yet helps you play with the variables to visualize the changes. >> >> If we can formulate the rules model based on some common variables >> (about machines, OSDs, racks) and replication rules (count or erasure >> coding paramaters). Then we can spit out useful information like: >> - How many nodes / racks / osds do you need to lose to have data offline >> - What percentage of data is going to be unavailable if you lose X resources >> - What's the probability of data loss over time >> - How does the readability model impact your cluster network traffic, >> repair traffic >> >> I think it would be stair-forward to build the web app. In fact, I'll >> volunteer myself for that task. Before we start that I would suggest >> we try to get the model in something like Google Docs Sheets so we can >> use it as a basis. > > If can you haven't seen this: > > https://github.com/ceph/ceph-tools/tree/master/models/reliability > > It's not web-based but it probably could be changed to be. I definitely > like the idea of having a hosted version. Somehow I missed this. I'll take a stab to converting it into a web app. > > sage -- Milosz Tanski CTO 16 East 34th Street, 15th floor New York, NY 10016 p: 646-253-9055 e: milosz@adfin.com ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2015-04-01 14:18 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
[not found] <1401131148.7008.YahooMailNeo@web165003.mail.bf1.yahoo.com>
2014-05-27 6:50 ` durability model Kyle Bader
[not found] ` <1401197435.39621.YahooMailNeo@web165002.mail.bf1.yahoo.com>
2014-05-27 14:24 ` Loic Dachary
[not found] ` <1401284130.75330.YahooMailNeo@web165006.mail.bf1.yahoo.com>
2014-05-28 14:05 ` Loic Dachary
2015-04-01 13:17 ` Milosz Tanski
2015-04-01 13:49 ` Sage Weil
2015-04-01 14:18 ` Milosz Tanski
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.