RGW placement multi data pool support

RGW placement multi data pool support

[Yang Honggang]

Hello,

In the current RGW implementation, only one ‘data pool’ is allowed in the
placement target. So the size of a bucket is limited by the backend pool
size. We can add osds into the backend pool, but it would trigger PG data
rebalance. Also, we might have to do PG split when the OSD count is close
to the pre-defined PG number, however, it would trigger data rebalance as
well. We try to avoid data rebalance as many as we can by adding multiple
pools into one placement target. Then a scheduler between pools of a
placement target is needed.

I have a immature proposal would like to discuss with you. We can extend
placement's data pool to a pool list. When a new object is created through
RGW, we can choose one data pool from placement's data pool list. In order
to do that, the pool ID need to be added into the bucket index for each
object.

How do you guys think?

thx,

Joseph
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: RGW placement multi data pool support

[Matt Benjamin]

Hi,

In-progress work (planned for Kraken) will make storage layout more cleanly abstract, and so should make this kind of thing less intrusive to build.  

Matt

----- Original Message -----
> From: "Yang Honggang" <joseph.yang@xtaotech.com>
> To: "ceph-devel" <ceph-devel@vger.kernel.org>
> Cc: "Javen Wu" <javen.wu@xtaotech.com>
> Sent: Friday, March 18, 2016 7:07:09 AM
> Subject: RGW placement multi data pool support
> 
> Hello,
> 
> In the current RGW implementation, only one ‘data pool’ is allowed in the
> placement target. So the size of a bucket is limited by the backend pool
> size. We can add osds into the backend pool, but it would trigger PG data
> rebalance. Also, we might have to do PG split when the OSD count is close
> to the pre-defined PG number, however, it would trigger data rebalance as
> well. We try to avoid data rebalance as many as we can by adding multiple
> pools into one placement target. Then a scheduler between pools of a
> placement target is needed.
> 
> I have a immature proposal would like to discuss with you. We can extend
> placement's data pool to a pool list. When a new object is created through
> RGW, we can choose one data pool from placement's data pool list. In order
> to do that, the pool ID need to be added into the bucket index for each
> object.
> 
> How do you guys think?
> 
> thx,
> 
> Joseph
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

-- 
Matt Benjamin
Red Hat, Inc.
315 West Huron Street, Suite 140A
Ann Arbor, Michigan 48103

http://www.redhat.com/en/technologies/storage

tel.  734-707-0660
fax.  734-769-8938
cel.  734-216-5309
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: RGW placement multi data pool support

[Matt Benjamin]

(clarifying, this is refactoring specific to rgw by Dan, not cluster-level placement)

----- Original Message -----
> From: "Matt Benjamin" <mbenjamin@redhat.com>
> To: "Yang Honggang" <joseph.yang@xtaotech.com>
> Cc: "ceph-devel" <ceph-devel@vger.kernel.org>, "Javen Wu" <javen.wu@xtaotech.com>, "Daniel Gryniewicz"
> <dang@redhat.com>
> Sent: Monday, March 21, 2016 3:54:38 PM
> Subject: Re: RGW placement multi data pool support
> 
> Hi,
> 
> In-progress work (planned for Kraken) will make storage layout more cleanly
> abstract, and so should make this kind of thing less intrusive to build.
> 
> Matt
> 
> ----- Original Message -----
> > From: "Yang Honggang" <joseph.yang@xtaotech.com>
> > To: "ceph-devel" <ceph-devel@vger.kernel.org>
> > Cc: "Javen Wu" <javen.wu@xtaotech.com>
> > Sent: Friday, March 18, 2016 7:07:09 AM
> > Subject: RGW placement multi data pool support
> > 
> > Hello,
> > 
> > In the current RGW implementation, only one ‘data pool’ is allowed in the
> > placement target. So the size of a bucket is limited by the backend pool
> > size. We can add osds into the backend pool, but it would trigger PG data
> > rebalance. Also, we might have to do PG split when the OSD count is close
> > to the pre-defined PG number, however, it would trigger data rebalance as
> > well. We try to avoid data rebalance as many as we can by adding multiple
> > pools into one placement target. Then a scheduler between pools of a
> > placement target is needed.
> > 
> > I have a immature proposal would like to discuss with you. We can extend
> > placement's data pool to a pool list. When a new object is created through
> > RGW, we can choose one data pool from placement's data pool list. In order
> > to do that, the pool ID need to be added into the bucket index for each
> > object.
> > 
> > How do you guys think?
> > 
> > thx,
> > 
> > Joseph
> > --
> > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> > 
> 
> --
> Matt Benjamin
> Red Hat, Inc.
> 315 West Huron Street, Suite 140A
> Ann Arbor, Michigan 48103
> 
> http://www.redhat.com/en/technologies/storage
> 
> tel.  734-707-0660
> fax.  734-769-8938
> cel.  734-216-5309
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

-- 
Matt Benjamin
Red Hat, Inc.
315 West Huron Street, Suite 140A
Ann Arbor, Michigan 48103

http://www.redhat.com/en/technologies/storage

tel.  734-707-0660
fax.  734-769-8938
cel.  734-216-5309
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html