* Re: Fwd: Reduce read latency and bandwidth for ec pool [not found] ` <CANE=7sVcY5sCaHUSFn+MDkAYzeewa767ziXXLqNiSCXhnG4s-g@mail.gmail.com> @ 2015-03-17 7:48 ` Loic Dachary [not found] ` <CANE=7sV1CQA=ZV5ELihrE4DZNGN5=4c07uP4Pv2w8LLpzqWabA@mail.gmail.com> 0 siblings, 1 reply; 8+ messages in thread From: Loic Dachary @ 2015-03-17 7:48 UTC (permalink / raw) To: Xinze Chi; +Cc: Ceph Development [-- Attachment #1: Type: text/plain, Size: 1310 bytes --] Hi, On 17/03/2015 08:27, Xinze Chi wrote: > hi, loic: > > I have an idea which could reduce read latency and bandwidth for ec pool. > > But, I don't know whether it is feasible. > > Such as ec pool stripe_width = 16384 = 4 * 4096, K = 4, M =2 > > So ceph will partition the total of 16384 bytes to 4 data chunk, > and encoding 2 parity chunk > > shard_0 include 0 - (4096-1) in original data; > shard_1 include 4096 - (4096*2 - 1) in original data; > shard_2 include 4096*2 - (4096 * 3 -1) in original data; > shard_3 include 4096*3 - (4096 * 4 - 1) in original data > shard_4 include parity chunk > shard_5 include parity chunk > > Now if client read (offset 0, len 4096) from object, it should > read 4 shard (from 0-3) and decode all this 4 chunk. > > But, this example, maybe we can compute the destination shard > based on ec pool config ,read offset and read len , we only read > > shard_0 and return it to client, because shard_0 has include all data > as client need. That optimization makes sense to me. I guess you're interested in having small objects in the pool and only read a few bytes at a time ? Cheers > > Wait for your comment. > > Thanks. > -- Loïc Dachary, Artisan Logiciel Libre [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 198 bytes --] ^ permalink raw reply [flat|nested] 8+ messages in thread
[parent not found: <CANE=7sV1CQA=ZV5ELihrE4DZNGN5=4c07uP4Pv2w8LLpzqWabA@mail.gmail.com>]
* Fwd: Fwd: Reduce read latency and bandwidth for ec pool [not found] ` <CANE=7sV1CQA=ZV5ELihrE4DZNGN5=4c07uP4Pv2w8LLpzqWabA@mail.gmail.com> @ 2015-03-17 7:52 ` Xinze Chi 2015-03-17 8:02 ` Loic Dachary 0 siblings, 1 reply; 8+ messages in thread From: Xinze Chi @ 2015-03-17 7:52 UTC (permalink / raw) To: ceph-devel@vger.kernel.org ---------- Forwarded message ---------- From: Xinze Chi <xmdxcxz@gmail.com> Date: 2015-03-17 15:52 GMT+08:00 Subject: Re: Fwd: Reduce read latency and bandwidth for ec pool To: Loic Dachary <loic@dachary.org> Yes, In my VDI environment, client read 4k every time. If we can read object from only shard. It would reduce the latency and bandwidth a lot. Thanks. 2015-03-17 15:48 GMT+08:00 Loic Dachary <loic@dachary.org>: > Hi, > > On 17/03/2015 08:27, Xinze Chi wrote: >> hi, loic: >> >> I have an idea which could reduce read latency and bandwidth for ec pool. >> >> But, I don't know whether it is feasible. >> >> Such as ec pool stripe_width = 16384 = 4 * 4096, K = 4, M =2 >> >> So ceph will partition the total of 16384 bytes to 4 data chunk, >> and encoding 2 parity chunk >> >> shard_0 include 0 - (4096-1) in original data; >> shard_1 include 4096 - (4096*2 - 1) in original data; >> shard_2 include 4096*2 - (4096 * 3 -1) in original data; >> shard_3 include 4096*3 - (4096 * 4 - 1) in original data >> shard_4 include parity chunk >> shard_5 include parity chunk >> >> Now if client read (offset 0, len 4096) from object, it should >> read 4 shard (from 0-3) and decode all this 4 chunk. >> >> But, this example, maybe we can compute the destination shard >> based on ec pool config ,read offset and read len , we only read >> >> shard_0 and return it to client, because shard_0 has include all data >> as client need. > > That optimization makes sense to me. I guess you're interested in having small objects in the pool and only read a few bytes at a time ? > > Cheers > >> >> Wait for your comment. >> >> Thanks. >> > > -- > Loïc Dachary, Artisan Logiciel Libre > -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Fwd: Fwd: Reduce read latency and bandwidth for ec pool 2015-03-17 7:52 ` Fwd: " Xinze Chi @ 2015-03-17 8:02 ` Loic Dachary [not found] ` <CANE=7sUR-Wdsat=3mBHW4SWrxSJPOC4svrZkwBMYQnWQFJ5VCw@mail.gmail.com> 0 siblings, 1 reply; 8+ messages in thread From: Loic Dachary @ 2015-03-17 8:02 UTC (permalink / raw) To: Xinze Chi, ceph-devel@vger.kernel.org [-- Attachment #1: Type: text/plain, Size: 2221 bytes --] On 17/03/2015 08:52, Xinze Chi wrote: > ---------- Forwarded message ---------- > From: Xinze Chi <xmdxcxz@gmail.com> > Date: 2015-03-17 15:52 GMT+08:00 > Subject: Re: Fwd: Reduce read latency and bandwidth for ec pool > To: Loic Dachary <loic@dachary.org> > > > Yes, In my VDI environment, client read 4k every time. If we can read > object from only shard. It would reduce the latency and bandwidth a > lot. I'm curious about your workload. Are you using RadosGW ? RBD ? > Thanks. > > 2015-03-17 15:48 GMT+08:00 Loic Dachary <loic@dachary.org>: >> Hi, >> >> On 17/03/2015 08:27, Xinze Chi wrote: >>> hi, loic: >>> >>> I have an idea which could reduce read latency and bandwidth for ec pool. >>> >>> But, I don't know whether it is feasible. >>> >>> Such as ec pool stripe_width = 16384 = 4 * 4096, K = 4, M =2 >>> >>> So ceph will partition the total of 16384 bytes to 4 data chunk, >>> and encoding 2 parity chunk >>> >>> shard_0 include 0 - (4096-1) in original data; >>> shard_1 include 4096 - (4096*2 - 1) in original data; >>> shard_2 include 4096*2 - (4096 * 3 -1) in original data; >>> shard_3 include 4096*3 - (4096 * 4 - 1) in original data >>> shard_4 include parity chunk >>> shard_5 include parity chunk >>> >>> Now if client read (offset 0, len 4096) from object, it should >>> read 4 shard (from 0-3) and decode all this 4 chunk. >>> >>> But, this example, maybe we can compute the destination shard >>> based on ec pool config ,read offset and read len , we only read >>> >>> shard_0 and return it to client, because shard_0 has include all data >>> as client need. >> >> That optimization makes sense to me. I guess you're interested in having small objects in the pool and only read a few bytes at a time ? >> >> Cheers >> >>> >>> Wait for your comment. >>> >>> Thanks. >>> >> >> -- >> Loïc Dachary, Artisan Logiciel Libre >> > -- > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > -- Loïc Dachary, Artisan Logiciel Libre [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 198 bytes --] ^ permalink raw reply [flat|nested] 8+ messages in thread
[parent not found: <CANE=7sUR-Wdsat=3mBHW4SWrxSJPOC4svrZkwBMYQnWQFJ5VCw@mail.gmail.com>]
* Re: Fwd: Fwd: Reduce read latency and bandwidth for ec pool [not found] ` <CANE=7sUR-Wdsat=3mBHW4SWrxSJPOC4svrZkwBMYQnWQFJ5VCw@mail.gmail.com> @ 2015-03-17 8:39 ` Loic Dachary 2015-03-17 8:45 ` Xinze Chi 0 siblings, 1 reply; 8+ messages in thread From: Loic Dachary @ 2015-03-17 8:39 UTC (permalink / raw) To: Xinze Chi; +Cc: Ceph Development [-- Attachment #1: Type: text/plain, Size: 2678 bytes --] On 17/03/2015 09:05, Xinze Chi wrote: > RBD. Did you measure that RBD does a significant amount of reads that would be optimized in this way ? > Maybe we could use tier pool. > > Thanks > > 2015-03-17 16:02 GMT+08:00 Loic Dachary <loic@dachary.org>: >> >> >> On 17/03/2015 08:52, Xinze Chi wrote: >>> ---------- Forwarded message ---------- >>> From: Xinze Chi <xmdxcxz@gmail.com> >>> Date: 2015-03-17 15:52 GMT+08:00 >>> Subject: Re: Fwd: Reduce read latency and bandwidth for ec pool >>> To: Loic Dachary <loic@dachary.org> >>> >>> >>> Yes, In my VDI environment, client read 4k every time. If we can read >>> object from only shard. It would reduce the latency and bandwidth a >>> lot. >> >> I'm curious about your workload. Are you using RadosGW ? RBD ? >> >>> Thanks. >>> >>> 2015-03-17 15:48 GMT+08:00 Loic Dachary <loic@dachary.org>: >>>> Hi, >>>> >>>> On 17/03/2015 08:27, Xinze Chi wrote: >>>>> hi, loic: >>>>> >>>>> I have an idea which could reduce read latency and bandwidth for ec pool. >>>>> >>>>> But, I don't know whether it is feasible. >>>>> >>>>> Such as ec pool stripe_width = 16384 = 4 * 4096, K = 4, M =2 >>>>> >>>>> So ceph will partition the total of 16384 bytes to 4 data chunk, >>>>> and encoding 2 parity chunk >>>>> >>>>> shard_0 include 0 - (4096-1) in original data; >>>>> shard_1 include 4096 - (4096*2 - 1) in original data; >>>>> shard_2 include 4096*2 - (4096 * 3 -1) in original data; >>>>> shard_3 include 4096*3 - (4096 * 4 - 1) in original data >>>>> shard_4 include parity chunk >>>>> shard_5 include parity chunk >>>>> >>>>> Now if client read (offset 0, len 4096) from object, it should >>>>> read 4 shard (from 0-3) and decode all this 4 chunk. >>>>> >>>>> But, this example, maybe we can compute the destination shard >>>>> based on ec pool config ,read offset and read len , we only read >>>>> >>>>> shard_0 and return it to client, because shard_0 has include all data >>>>> as client need. >>>> >>>> That optimization makes sense to me. I guess you're interested in having small objects in the pool and only read a few bytes at a time ? >>>> >>>> Cheers >>>> >>>>> >>>>> Wait for your comment. >>>>> >>>>> Thanks. >>>>> >>>> >>>> -- >>>> Loïc Dachary, Artisan Logiciel Libre >>>> >>> -- >>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >>> the body of a message to majordomo@vger.kernel.org >>> More majordomo info at http://vger.kernel.org/majordomo-info.html >>> >> >> -- >> Loïc Dachary, Artisan Logiciel Libre >> -- Loïc Dachary, Artisan Logiciel Libre [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 198 bytes --] ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Fwd: Fwd: Reduce read latency and bandwidth for ec pool 2015-03-17 8:39 ` Loic Dachary @ 2015-03-17 8:45 ` Xinze Chi 2015-03-17 8:58 ` Loic Dachary 0 siblings, 1 reply; 8+ messages in thread From: Xinze Chi @ 2015-03-17 8:45 UTC (permalink / raw) To: Loic Dachary, ceph-devel@vger.kernel.org Sorry, I have not measure it. But I think it should really reduce latency when hit miss in cache pool and do_proxy_read. 2015-03-17 16:39 GMT+08:00 Loic Dachary <loic@dachary.org>: > > > On 17/03/2015 09:05, Xinze Chi wrote: >> RBD. > > Did you measure that RBD does a significant amount of reads that would be optimized in this way ? > >> Maybe we could use tier pool. >> >> Thanks >> >> 2015-03-17 16:02 GMT+08:00 Loic Dachary <loic@dachary.org>: >>> >>> >>> On 17/03/2015 08:52, Xinze Chi wrote: >>>> ---------- Forwarded message ---------- >>>> From: Xinze Chi <xmdxcxz@gmail.com> >>>> Date: 2015-03-17 15:52 GMT+08:00 >>>> Subject: Re: Fwd: Reduce read latency and bandwidth for ec pool >>>> To: Loic Dachary <loic@dachary.org> >>>> >>>> >>>> Yes, In my VDI environment, client read 4k every time. If we can read >>>> object from only shard. It would reduce the latency and bandwidth a >>>> lot. >>> >>> I'm curious about your workload. Are you using RadosGW ? RBD ? >>> >>>> Thanks. >>>> >>>> 2015-03-17 15:48 GMT+08:00 Loic Dachary <loic@dachary.org>: >>>>> Hi, >>>>> >>>>> On 17/03/2015 08:27, Xinze Chi wrote: >>>>>> hi, loic: >>>>>> >>>>>> I have an idea which could reduce read latency and bandwidth for ec pool. >>>>>> >>>>>> But, I don't know whether it is feasible. >>>>>> >>>>>> Such as ec pool stripe_width = 16384 = 4 * 4096, K = 4, M =2 >>>>>> >>>>>> So ceph will partition the total of 16384 bytes to 4 data chunk, >>>>>> and encoding 2 parity chunk >>>>>> >>>>>> shard_0 include 0 - (4096-1) in original data; >>>>>> shard_1 include 4096 - (4096*2 - 1) in original data; >>>>>> shard_2 include 4096*2 - (4096 * 3 -1) in original data; >>>>>> shard_3 include 4096*3 - (4096 * 4 - 1) in original data >>>>>> shard_4 include parity chunk >>>>>> shard_5 include parity chunk >>>>>> >>>>>> Now if client read (offset 0, len 4096) from object, it should >>>>>> read 4 shard (from 0-3) and decode all this 4 chunk. >>>>>> >>>>>> But, this example, maybe we can compute the destination shard >>>>>> based on ec pool config ,read offset and read len , we only read >>>>>> >>>>>> shard_0 and return it to client, because shard_0 has include all data >>>>>> as client need. >>>>> >>>>> That optimization makes sense to me. I guess you're interested in having small objects in the pool and only read a few bytes at a time ? >>>>> >>>>> Cheers >>>>> >>>>>> >>>>>> Wait for your comment. >>>>>> >>>>>> Thanks. >>>>>> >>>>> >>>>> -- >>>>> Loïc Dachary, Artisan Logiciel Libre >>>>> >>>> -- >>>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >>>> the body of a message to majordomo@vger.kernel.org >>>> More majordomo info at http://vger.kernel.org/majordomo-info.html >>>> >>> >>> -- >>> Loïc Dachary, Artisan Logiciel Libre >>> > > -- > Loïc Dachary, Artisan Logiciel Libre > -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Fwd: Fwd: Reduce read latency and bandwidth for ec pool 2015-03-17 8:45 ` Xinze Chi @ 2015-03-17 8:58 ` Loic Dachary 2015-03-17 18:18 ` Josh Durgin 0 siblings, 1 reply; 8+ messages in thread From: Loic Dachary @ 2015-03-17 8:58 UTC (permalink / raw) To: Xinze Chi, ceph-devel@vger.kernel.org [-- Attachment #1: Type: text/plain, Size: 3221 bytes --] On 17/03/2015 09:45, Xinze Chi wrote: > Sorry, I have not measure it. > > But I think it should really reduce latency when hit miss in cache > pool and do_proxy_read. Interesting. I bet Jason or Josh have an opinion about this. Cheers > > 2015-03-17 16:39 GMT+08:00 Loic Dachary <loic@dachary.org>: >> >> >> On 17/03/2015 09:05, Xinze Chi wrote: >>> RBD. >> >> Did you measure that RBD does a significant amount of reads that would be optimized in this way ? >> >>> Maybe we could use tier pool. >>> >>> Thanks >>> >>> 2015-03-17 16:02 GMT+08:00 Loic Dachary <loic@dachary.org>: >>>> >>>> >>>> On 17/03/2015 08:52, Xinze Chi wrote: >>>>> ---------- Forwarded message ---------- >>>>> From: Xinze Chi <xmdxcxz@gmail.com> >>>>> Date: 2015-03-17 15:52 GMT+08:00 >>>>> Subject: Re: Fwd: Reduce read latency and bandwidth for ec pool >>>>> To: Loic Dachary <loic@dachary.org> >>>>> >>>>> >>>>> Yes, In my VDI environment, client read 4k every time. If we can read >>>>> object from only shard. It would reduce the latency and bandwidth a >>>>> lot. >>>> >>>> I'm curious about your workload. Are you using RadosGW ? RBD ? >>>> >>>>> Thanks. >>>>> >>>>> 2015-03-17 15:48 GMT+08:00 Loic Dachary <loic@dachary.org>: >>>>>> Hi, >>>>>> >>>>>> On 17/03/2015 08:27, Xinze Chi wrote: >>>>>>> hi, loic: >>>>>>> >>>>>>> I have an idea which could reduce read latency and bandwidth for ec pool. >>>>>>> >>>>>>> But, I don't know whether it is feasible. >>>>>>> >>>>>>> Such as ec pool stripe_width = 16384 = 4 * 4096, K = 4, M =2 >>>>>>> >>>>>>> So ceph will partition the total of 16384 bytes to 4 data chunk, >>>>>>> and encoding 2 parity chunk >>>>>>> >>>>>>> shard_0 include 0 - (4096-1) in original data; >>>>>>> shard_1 include 4096 - (4096*2 - 1) in original data; >>>>>>> shard_2 include 4096*2 - (4096 * 3 -1) in original data; >>>>>>> shard_3 include 4096*3 - (4096 * 4 - 1) in original data >>>>>>> shard_4 include parity chunk >>>>>>> shard_5 include parity chunk >>>>>>> >>>>>>> Now if client read (offset 0, len 4096) from object, it should >>>>>>> read 4 shard (from 0-3) and decode all this 4 chunk. >>>>>>> >>>>>>> But, this example, maybe we can compute the destination shard >>>>>>> based on ec pool config ,read offset and read len , we only read >>>>>>> >>>>>>> shard_0 and return it to client, because shard_0 has include all data >>>>>>> as client need. >>>>>> >>>>>> That optimization makes sense to me. I guess you're interested in having small objects in the pool and only read a few bytes at a time ? >>>>>> >>>>>> Cheers >>>>>> >>>>>>> >>>>>>> Wait for your comment. >>>>>>> >>>>>>> Thanks. >>>>>>> >>>>>> >>>>>> -- >>>>>> Loïc Dachary, Artisan Logiciel Libre >>>>>> >>>>> -- >>>>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >>>>> the body of a message to majordomo@vger.kernel.org >>>>> More majordomo info at http://vger.kernel.org/majordomo-info.html >>>>> >>>> >>>> -- >>>> Loïc Dachary, Artisan Logiciel Libre >>>> >> >> -- >> Loïc Dachary, Artisan Logiciel Libre >> -- Loïc Dachary, Artisan Logiciel Libre [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 198 bytes --] ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Fwd: Fwd: Reduce read latency and bandwidth for ec pool 2015-03-17 8:58 ` Loic Dachary @ 2015-03-17 18:18 ` Josh Durgin 2015-03-18 6:15 ` Ning Yao 0 siblings, 1 reply; 8+ messages in thread From: Josh Durgin @ 2015-03-17 18:18 UTC (permalink / raw) To: Loic Dachary, Xinze Chi, ceph-devel@vger.kernel.org On 03/17/2015 01:58 AM, Loic Dachary wrote: > > > On 17/03/2015 09:45, Xinze Chi wrote: >> Sorry, I have not measure it. >> >> But I think it should really reduce latency when hit miss in cache >> pool and do_proxy_read. > > Interesting. I bet Jason or Josh have an opinion about this. Yes, it sounds like a great idea! It seems like we'd need this for other potential optimizations in the future anyway: * partial-object promotes for cache tiers * client-side ec to eliminate another network hop This would also enable efficient reads from replicas for EC pools in general. That could be useful for rbd parent snapshots stored in a cache tier. This makes me wonder if it would be useful to add write-once append-only rbd images that could be stored directly on EC pools, for use as parent images. Josh > >> >> 2015-03-17 16:39 GMT+08:00 Loic Dachary <loic@dachary.org>: >>> >>> >>> On 17/03/2015 09:05, Xinze Chi wrote: >>>> RBD. >>> >>> Did you measure that RBD does a significant amount of reads that would be optimized in this way ? >>> >>>> Maybe we could use tier pool. >>>> >>>> Thanks >>>> >>>> 2015-03-17 16:02 GMT+08:00 Loic Dachary <loic@dachary.org>: >>>>> >>>>> >>>>> On 17/03/2015 08:52, Xinze Chi wrote: >>>>>> ---------- Forwarded message ---------- >>>>>> From: Xinze Chi <xmdxcxz@gmail.com> >>>>>> Date: 2015-03-17 15:52 GMT+08:00 >>>>>> Subject: Re: Fwd: Reduce read latency and bandwidth for ec pool >>>>>> To: Loic Dachary <loic@dachary.org> >>>>>> >>>>>> >>>>>> Yes, In my VDI environment, client read 4k every time. If we can read >>>>>> object from only shard. It would reduce the latency and bandwidth a >>>>>> lot. >>>>> >>>>> I'm curious about your workload. Are you using RadosGW ? RBD ? >>>>> >>>>>> Thanks. >>>>>> >>>>>> 2015-03-17 15:48 GMT+08:00 Loic Dachary <loic@dachary.org>: >>>>>>> Hi, >>>>>>> >>>>>>> On 17/03/2015 08:27, Xinze Chi wrote: >>>>>>>> hi, loic: >>>>>>>> >>>>>>>> I have an idea which could reduce read latency and bandwidth for ec pool. >>>>>>>> >>>>>>>> But, I don't know whether it is feasible. >>>>>>>> >>>>>>>> Such as ec pool stripe_width = 16384 = 4 * 4096, K = 4, M =2 >>>>>>>> >>>>>>>> So ceph will partition the total of 16384 bytes to 4 data chunk, >>>>>>>> and encoding 2 parity chunk >>>>>>>> >>>>>>>> shard_0 include 0 - (4096-1) in original data; >>>>>>>> shard_1 include 4096 - (4096*2 - 1) in original data; >>>>>>>> shard_2 include 4096*2 - (4096 * 3 -1) in original data; >>>>>>>> shard_3 include 4096*3 - (4096 * 4 - 1) in original data >>>>>>>> shard_4 include parity chunk >>>>>>>> shard_5 include parity chunk >>>>>>>> >>>>>>>> Now if client read (offset 0, len 4096) from object, it should >>>>>>>> read 4 shard (from 0-3) and decode all this 4 chunk. >>>>>>>> >>>>>>>> But, this example, maybe we can compute the destination shard >>>>>>>> based on ec pool config ,read offset and read len , we only read >>>>>>>> >>>>>>>> shard_0 and return it to client, because shard_0 has include all data >>>>>>>> as client need. >>>>>>> >>>>>>> That optimization makes sense to me. I guess you're interested in having small objects in the pool and only read a few bytes at a time ? >>>>>>> >>>>>>> Cheers >>>>>>> >>>>>>>> >>>>>>>> Wait for your comment. >>>>>>>> >>>>>>>> Thanks. >>>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Loïc Dachary, Artisan Logiciel Libre >>>>>>> >>>>>> -- >>>>>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >>>>>> the body of a message to majordomo@vger.kernel.org >>>>>> More majordomo info at http://vger.kernel.org/majordomo-info.html >>>>>> >>>>> >>>>> -- >>>>> Loïc Dachary, Artisan Logiciel Libre >>>>> >>> >>> -- >>> Loïc Dachary, Artisan Logiciel Libre >>> > -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Fwd: Fwd: Reduce read latency and bandwidth for ec pool 2015-03-17 18:18 ` Josh Durgin @ 2015-03-18 6:15 ` Ning Yao 0 siblings, 0 replies; 8+ messages in thread From: Ning Yao @ 2015-03-18 6:15 UTC (permalink / raw) To: Josh Durgin; +Cc: Loic Dachary, Xinze Chi, ceph-devel@vger.kernel.org I have another question about ecpool. Based on current strategy, we find that we need long time to train the hit-set(if we set it one hour per hit-set, if we set a small value hit_set_period, the training result would be bad), and then achieve a relatively high hit rate. While at the beginning of first several hours, it seems that some newest promoted objects are evicted. This is because list_object just scans the objects in the pg and does not concern which one is newest promoted or not. This means if the system is initialized , the performance of the cache_pool will be improved after three or four hours and tend to be stable after ten hours. So does LRU or LRU2 makes sense here? or other strategies to make training process converge faster? Regards Ning Yao 2015-03-18 2:18 GMT+08:00 Josh Durgin <jdurgin@redhat.com>: > On 03/17/2015 01:58 AM, Loic Dachary wrote: >> >> >> >> On 17/03/2015 09:45, Xinze Chi wrote: >>> >>> Sorry, I have not measure it. >>> >>> But I think it should really reduce latency when hit miss in cache >>> pool and do_proxy_read. >> >> >> Interesting. I bet Jason or Josh have an opinion about this. > > > Yes, it sounds like a great idea! It seems like we'd need this > for other potential optimizations in the future anyway: > > * partial-object promotes for cache tiers > * client-side ec to eliminate another network hop > > This would also enable efficient reads from replicas for > EC pools in general. That could be useful for rbd parent snapshots > stored in a cache tier. > > This makes me wonder if it would be useful to add write-once > append-only rbd images that could be stored directly on EC pools, for > use as parent images. > > Josh > > >> >>> >>> 2015-03-17 16:39 GMT+08:00 Loic Dachary <loic@dachary.org>: >>>> >>>> >>>> >>>> On 17/03/2015 09:05, Xinze Chi wrote: >>>>> >>>>> RBD. >>>> >>>> >>>> Did you measure that RBD does a significant amount of reads that would >>>> be optimized in this way ? >>>> >>>>> Maybe we could use tier pool. >>>>> >>>>> Thanks >>>>> >>>>> 2015-03-17 16:02 GMT+08:00 Loic Dachary <loic@dachary.org>: >>>>>> >>>>>> >>>>>> >>>>>> On 17/03/2015 08:52, Xinze Chi wrote: >>>>>>> >>>>>>> ---------- Forwarded message ---------- >>>>>>> From: Xinze Chi <xmdxcxz@gmail.com> >>>>>>> Date: 2015-03-17 15:52 GMT+08:00 >>>>>>> Subject: Re: Fwd: Reduce read latency and bandwidth for ec pool >>>>>>> To: Loic Dachary <loic@dachary.org> >>>>>>> >>>>>>> >>>>>>> Yes, In my VDI environment, client read 4k every time. If we can read >>>>>>> object from only shard. It would reduce the latency and bandwidth a >>>>>>> lot. >>>>>> >>>>>> >>>>>> I'm curious about your workload. Are you using RadosGW ? RBD ? >>>>>> >>>>>>> Thanks. >>>>>>> >>>>>>> 2015-03-17 15:48 GMT+08:00 Loic Dachary <loic@dachary.org>: >>>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> On 17/03/2015 08:27, Xinze Chi wrote: >>>>>>>>> >>>>>>>>> hi, loic: >>>>>>>>> >>>>>>>>> I have an idea which could reduce read latency and bandwidth >>>>>>>>> for ec pool. >>>>>>>>> >>>>>>>>> But, I don't know whether it is feasible. >>>>>>>>> >>>>>>>>> Such as ec pool stripe_width = 16384 = 4 * 4096, K = 4, M =2 >>>>>>>>> >>>>>>>>> So ceph will partition the total of 16384 bytes to 4 data >>>>>>>>> chunk, >>>>>>>>> and encoding 2 parity chunk >>>>>>>>> >>>>>>>>> shard_0 include 0 - (4096-1) in original data; >>>>>>>>> shard_1 include 4096 - (4096*2 - 1) in original data; >>>>>>>>> shard_2 include 4096*2 - (4096 * 3 -1) in original data; >>>>>>>>> shard_3 include 4096*3 - (4096 * 4 - 1) in original data >>>>>>>>> shard_4 include parity chunk >>>>>>>>> shard_5 include parity chunk >>>>>>>>> >>>>>>>>> Now if client read (offset 0, len 4096) from object, it should >>>>>>>>> read 4 shard (from 0-3) and decode all this 4 chunk. >>>>>>>>> >>>>>>>>> But, this example, maybe we can compute the destination shard >>>>>>>>> based on ec pool config ,read offset and read len , we only read >>>>>>>>> >>>>>>>>> shard_0 and return it to client, because shard_0 has include all >>>>>>>>> data >>>>>>>>> as client need. >>>>>>>> >>>>>>>> >>>>>>>> That optimization makes sense to me. I guess you're interested in >>>>>>>> having small objects in the pool and only read a few bytes at a time ? >>>>>>>> >>>>>>>> Cheers >>>>>>>> >>>>>>>>> >>>>>>>>> Wait for your comment. >>>>>>>>> >>>>>>>>> Thanks. >>>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Loïc Dachary, Artisan Logiciel Libre >>>>>>>> >>>>>>> -- >>>>>>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" >>>>>>> in >>>>>>> the body of a message to majordomo@vger.kernel.org >>>>>>> More majordomo info at http://vger.kernel.org/majordomo-info.html >>>>>>> >>>>>> >>>>>> -- >>>>>> Loïc Dachary, Artisan Logiciel Libre >>>>>> >>>> >>>> -- >>>> Loïc Dachary, Artisan Logiciel Libre >>>> >> > > -- > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2015-03-18 6:15 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
[not found] <CANE=7sUo7tMT=kyuN86vs5b13nnqY9wpOuNuf=_z=P7UXKc+sg@mail.gmail.com>
[not found] ` <CANE=7sVcY5sCaHUSFn+MDkAYzeewa767ziXXLqNiSCXhnG4s-g@mail.gmail.com>
2015-03-17 7:48 ` Fwd: Reduce read latency and bandwidth for ec pool Loic Dachary
[not found] ` <CANE=7sV1CQA=ZV5ELihrE4DZNGN5=4c07uP4Pv2w8LLpzqWabA@mail.gmail.com>
2015-03-17 7:52 ` Fwd: " Xinze Chi
2015-03-17 8:02 ` Loic Dachary
[not found] ` <CANE=7sUR-Wdsat=3mBHW4SWrxSJPOC4svrZkwBMYQnWQFJ5VCw@mail.gmail.com>
2015-03-17 8:39 ` Loic Dachary
2015-03-17 8:45 ` Xinze Chi
2015-03-17 8:58 ` Loic Dachary
2015-03-17 18:18 ` Josh Durgin
2015-03-18 6:15 ` Ning Yao
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.