rbd device would disappear after re-boot

rbd device would disappear after re-boot

[Eric_YH_Chen]

Dear All:

   Another question about the rbd device. All rbd device would disappear
after server re-boot.

   Do you have any plan to implement a feature that server would re-map
all the device during initialization?

   If not, do you have any suggestion (about technical part) if we want
to implement this feature? 

   Furthermore, we would like to send a pull request after we finish it,
any policy should we take-care?

   Thanks a lot!

Re: rbd device would disappear after re-boot

[Tommi Virtanen]

On Tue, Dec 6, 2011 at 01:31,  <Eric_YH_Chen@wistron.com> wrote:
>   Another question about the rbd device. All rbd device would disappear
> after server re-boot.
>
>   Do you have any plan to implement a feature that server would re-map
> all the device during initialization?
>
>   If not, do you have any suggestion (about technical part) if we want
> to implement this feature?
>
>   Furthermore, we would like to send a pull request after we finish it,
> any policy should we take-care?

Yes, all mappings need to be re-established after a reboot.

There is no convention yet on configuring what should be
re-established. Right now, putting an "rbd map" in something like
/etc/rc.local is a decent workaround.

I created http://tracker.newdream.net/issues/1790 to track this feature.
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: rbd device would disappear after re-boot

[Eric_YH_Chen]

Hi, Tommi:

Thanks for your suggestion.

We will try to implement a workaround solution for our internal experiment.

I have another little question. Could I map specific image to specific device?

For example:
Before re-boot
id      pool    image   snap    device
0       rbd     foo1    -       /dev/rbd0
1       rbd     foo3    -       /dev/rbd2

How could I re-map the image(foo3) to device(/dev/rbd2) but skip rbd1?  The command provided by ceph CLI cannot achieve this.

If I re-map the image to another device, it would not sync with iSCSI configuration and may cause problem.

Thanks a lot!


-----Original Message-----
From: Tommi Virtanen [mailto:tommi.virtanen@dreamhost.com] 
Sent: Wednesday, December 07, 2011 3:11 AM
To: Eric YH Chen/WHQ/Wistron
Cc: ceph-devel@vger.kernel.org; Chris YT Huang/WHQ/Wistron
Subject: Re: rbd device would disappear after re-boot

On Tue, Dec 6, 2011 at 01:31,  <Eric_YH_Chen@wistron.com> wrote:
>   Another question about the rbd device. All rbd device would disappear
> after server re-boot.
>
>   Do you have any plan to implement a feature that server would re-map
> all the device during initialization?
>
>   If not, do you have any suggestion (about technical part) if we want
> to implement this feature?
>
>   Furthermore, we would like to send a pull request after we finish it,
> any policy should we take-care?

Yes, all mappings need to be re-established after a reboot.

There is no convention yet on configuring what should be
re-established. Right now, putting an "rbd map" in something like
/etc/rc.local is a decent workaround.

I created http://tracker.newdream.net/issues/1790 to track this feature.

Re: rbd device would disappear after re-boot

[Damien Churchill]

On 7 December 2011 02:38,  <Eric_YH_Chen@wistron.com> wrote:
> Hi, Tommi:
>
> Thanks for your suggestion.
>
> We will try to implement a workaround solution for our internal experiment.
>
> I have another little question. Could I map specific image to specific device?
>
> For example:
> Before re-boot
> id      pool    image   snap    device
> 0       rbd     foo1    -       /dev/rbd0
> 1       rbd     foo3    -       /dev/rbd2
>
> How could I re-map the image(foo3) to device(/dev/rbd2) but skip rbd1?  The command provided by ceph CLI cannot achieve this.
>
> If I re-map the image to another device, it would not sync with iSCSI configuration and may cause problem.
>

You can use the symlinks created, if memory serves they end up as
something along the lines of:

/dev/rbd/<poolname>/<diskname>:0

> Thanks a lot!
>
>
> -----Original Message-----
> From: Tommi Virtanen [mailto:tommi.virtanen@dreamhost.com]
> Sent: Wednesday, December 07, 2011 3:11 AM
> To: Eric YH Chen/WHQ/Wistron
> Cc: ceph-devel@vger.kernel.org; Chris YT Huang/WHQ/Wistron
> Subject: Re: rbd device would disappear after re-boot
>
> On Tue, Dec 6, 2011 at 01:31,  <Eric_YH_Chen@wistron.com> wrote:
>>   Another question about the rbd device. All rbd device would disappear
>> after server re-boot.
>>
>>   Do you have any plan to implement a feature that server would re-map
>> all the device during initialization?
>>
>>   If not, do you have any suggestion (about technical part) if we want
>> to implement this feature?
>>
>>   Furthermore, we would like to send a pull request after we finish it,
>> any policy should we take-care?
>
> Yes, all mappings need to be re-established after a reboot.
>
> There is no convention yet on configuring what should be
> re-established. Right now, putting an "rbd map" in something like
> /etc/rc.local is a decent workaround.
>
> I created http://tracker.newdream.net/issues/1790 to track this feature.
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: rbd device would disappear after re-boot

[Tommi Virtanen]

On Tue, Dec 6, 2011 at 18:38,  <Eric_YH_Chen@wistron.com> wrote:
> I have another little question. Could I map specific image to specific device?
>
> For example:
> Before re-boot
> id      pool    image   snap    device
> 0       rbd     foo1    -       /dev/rbd0
> 1       rbd     foo3    -       /dev/rbd2
>
> How could I re-map the image(foo3) to device(/dev/rbd2) but skip rbd1?  The command provided by ceph CLI cannot achieve this.
>
> If I re-map the image to another device, it would not sync with iSCSI configuration and may cause problem.

I'm just confirming what Damien said. Don't rely on the numbering to
be consistent even from one boot to another; we provide a udev helper
that assigns more permanent names, based on the pool, image name, and
potentially snapshot name. You'll get a symlink like
/dev/rbd/POOL/IMAGE or /dev/rbd/POOL/IMAGE@SNAP, use that name.
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: rbd device would disappear after re-boot

[Eric_YH_Chen]

Hi, Tommi:

What I see is like this.

lrwxrwxrwx 1 root root 10 2011-12-07 16:48 foo1:0 -> ../../rbd0
lrwxrwxrwx 1 root root 10 2011-12-07 16:50 foo2:1 -> ../../rbd1

The extra number (:0 and :1) behind the image name make the problem still exists.

-----Original Message-----
From: Tommi Virtanen [mailto:tommi.virtanen@dreamhost.com] 
Sent: Thursday, December 08, 2011 1:31 AM
To: Eric YH Chen/WHQ/Wistron
Cc: ceph-devel@vger.kernel.org; Chris YT Huang/WHQ/Wistron
Subject: Re: rbd device would disappear after re-boot

On Tue, Dec 6, 2011 at 18:38,  <Eric_YH_Chen@wistron.com> wrote:
> I have another little question. Could I map specific image to specific device?
>
> For example:
> Before re-boot
> id      pool    image   snap    device
> 0       rbd     foo1    -       /dev/rbd0
> 1       rbd     foo3    -       /dev/rbd2
>
> How could I re-map the image(foo3) to device(/dev/rbd2) but skip rbd1?  The command provided by ceph CLI cannot achieve this.
>
> If I re-map the image to another device, it would not sync with iSCSI configuration and may cause problem.

I'm just confirming what Damien said. Don't rely on the numbering to
be consistent even from one boot to another; we provide a udev helper
that assigns more permanent names, based on the pool, image name, and
potentially snapshot name. You'll get a symlink like
/dev/rbd/POOL/IMAGE or /dev/rbd/POOL/IMAGE@SNAP, use that name.

Re: rbd device would disappear after re-boot

[Josh Durgin]

On 12/07/2011 11:31 PM, Eric_YH_Chen@wistron.com wrote:
> Hi, Tommi:
>
> What I see is like this.
>
> lrwxrwxrwx 1 root root 10 2011-12-07 16:48 foo1:0 ->  ../../rbd0
> lrwxrwxrwx 1 root root 10 2011-12-07 16:50 foo2:1 ->  ../../rbd1
>
> The extra number (:0 and :1) behind the image name make the problem still exists.

I'm not sure why we had the device number in the udev rule. I removed it 
in 891025e539a92b5d75011e2e75c475fc0c272042.

>
> -----Original Message-----
> From: Tommi Virtanen [mailto:tommi.virtanen@dreamhost.com]
> Sent: Thursday, December 08, 2011 1:31 AM
> To: Eric YH Chen/WHQ/Wistron
> Cc: ceph-devel@vger.kernel.org; Chris YT Huang/WHQ/Wistron
> Subject: Re: rbd device would disappear after re-boot
>
> On Tue, Dec 6, 2011 at 18:38,<Eric_YH_Chen@wistron.com>  wrote:
>> I have another little question. Could I map specific image to specific device?
>>
>> For example:
>> Before re-boot
>> id      pool    image   snap    device
>> 0       rbd     foo1    -       /dev/rbd0
>> 1       rbd     foo3    -       /dev/rbd2
>>
>> How could I re-map the image(foo3) to device(/dev/rbd2) but skip rbd1?  The command provided by ceph CLI cannot achieve this.
>>
>> If I re-map the image to another device, it would not sync with iSCSI configuration and may cause problem.
>
> I'm just confirming what Damien said. Don't rely on the numbering to
> be consistent even from one boot to another; we provide a udev helper
> that assigns more permanent names, based on the pool, image name, and
> potentially snapshot name. You'll get a symlink like
> /dev/rbd/POOL/IMAGE or /dev/rbd/POOL/IMAGE@SNAP, use that name.