RBD layering needs to work across pools

RBD layering needs to work across pools

[Tommi Virtanen]

Hi list, hi Josh.

This came up in an OpenStack conversation today:

if we store "template images" and "volumes" both in RBD, it would make
a lot of sense prevent the Nova nodes from accidentally writing to the
golden images. That is, have the templates in a separate pool that is
writeable to Glance, but read-only for the Nova nodes.

To support this, we'd need to make sure the RBD layering we're
building is able to use base images in a separate pool. (And handle
the ugly case where after a misconfiguration the client can't access
the base image anymore...)

Josh, can you say whether that falls in line with what's been planned
for the layering?

Re: RBD layering needs to work across pools

[Tommi Virtanen]

On Fri, Dec 16, 2011 at 14:48, Tommi Virtanen
<tommi.virtanen@dreamhost.com> wrote:
> To support this, we'd need to make sure the RBD layering we're
> building is able to use base images in a separate pool. (And handle
> the ugly case where after a misconfiguration the client can't access
> the base image anymore...)
>
> Josh, can you say whether that falls in line with what's been planned
> for the layering?

Nevermind, I see parent_{pool, image_id, snap_id} in
http://tracker.newdream.net/issues/1772

Just keep that in mind as a hard requirement.

Re: RBD layering needs to work across pools

[Gregory Farnum]

On Fri, Dec 16, 2011 at 3:40 PM, Tommi Virtanen
<tommi.virtanen@dreamhost.com> wrote:
> On Fri, Dec 16, 2011 at 14:48, Tommi Virtanen
> <tommi.virtanen@dreamhost.com> wrote:
>> To support this, we'd need to make sure the RBD layering we're
>> building is able to use base images in a separate pool. (And handle
>> the ugly case where after a misconfiguration the client can't access
>> the base image anymore...)
>>
>> Josh, can you say whether that falls in line with what's been planned
>> for the layering?
>
> Nevermind, I see parent_{pool, image_id, snap_id} in
> http://tracker.newdream.net/issues/1772
>
> Just keep that in mind as a hard requirement.

We've needed it for other things too — one of the explicitly
enumerated ideas was being able to migrate VMs between pools that are
located primarily in different data centers/racks/whatever.
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: RBD layering needs to work across pools

[Josh Durgin]

On 12/16/2011 03:40 PM, Tommi Virtanen wrote:
> On Fri, Dec 16, 2011 at 14:48, Tommi Virtanen
> <tommi.virtanen@dreamhost.com>  wrote:
>> To support this, we'd need to make sure the RBD layering we're
>> building is able to use base images in a separate pool. (And handle
>> the ugly case where after a misconfiguration the client can't access
>> the base image anymore...)
>>
>> Josh, can you say whether that falls in line with what's been planned
>> for the layering?
>
> Nevermind, I see parent_{pool, image_id, snap_id} in
> http://tracker.newdream.net/issues/1772
>
> Just keep that in mind as a hard requirement.

Also note that the plan includes setting a read-only flag on
images that have clones, so there's no danger of a parent image
being modified underneath a child image.

There is one detail that we didn't realize while planning this
though - our current plan calls for the parent image to keep
track of a list of child images, which needs to be updated
whenever a child image is created or deleted.

We were planning on storing this in the parent's header, which
would be in the parent's pool. This means that cloning an image
or deleting a cloned image would require write access to the
parent's pool. We might need to rethink this a little.