krbd + format=2 ?

krbd + format=2 ?

[Chris Dunlop]

G'day,

Sage's recent pull message to Linus said:

----
Please pull the following Ceph patches from

  git://git.kernel.org/pub/scm/linux/kernel/git/sage/ceph-client.git for-linus

This is a big pull.  Most of it is culmination of Alex's work to implement
RBD image layering, which is now complete (yay!).
----

Am I correct in thinking "RBD image layering... is now complete"
implies there should be full(?) support for format=2?

I pulled the for-linus branch (@ 3abef3b) on top of 3.10.0-rc4, and it's
letting me map a format=2 image (created under bobtail), however reading
from the block device returns zeros rather than the data. The same image
correctly shows data (NTFS filesystem) when mounted into kvm using librbd.

----
# uname -r
3.10.0-rc4-00010-g0326739
# rbd ls -l
NAME                   SIZE PARENT                       FMT PROT LOCK 
xxx                   1536G                                2
# rbd map rbd/xxx
# rbd showmapped
id pool image   snap device    
1  rbd  xxx     -    /dev/rbd1 
# dd if=/dev/rbd1 of=/tmp/xxx count=20480
20480+0 records in
20480+0 records out
10485760 bytes (10 MB) copied, 0.757754 s, 13.8 MB/s
# od -c /tmp/xxx | less
0000000  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0
*
50000000
----


Cheers,

Chris

Re: krbd + format=2 ?

[Alex Elder]

On 06/03/2013 04:24 AM, Chris Dunlop wrote:
> G'day,
> 
> Sage's recent pull message to Linus said:
> 
> ----
> Please pull the following Ceph patches from
> 
>   git://git.kernel.org/pub/scm/linux/kernel/git/sage/ceph-client.git for-linus
> 
> This is a big pull.  Most of it is culmination of Alex's work to implement
> RBD image layering, which is now complete (yay!).
> ----
> 
> Am I correct in thinking "RBD image layering... is now complete"
> implies there should be full(?) support for format=2?
> 
> I pulled the for-linus branch (@ 3abef3b) on top of 3.10.0-rc4, and it's
> letting me map a format=2 image (created under bobtail), however reading
> from the block device returns zeros rather than the data. The same image
> correctly shows data (NTFS filesystem) when mounted into kvm using librbd.

Have you tried using a format 2 image that you created using
the Linux rbd environment?  It would be good to know whether
that works for you.

Can you also send me the result of running each of these commands:

foo=$(rados get --pool=rbd rbd_id.xxx - | strings)
rados ls --pool=rbd | grep rbd_data | grep "${foo}"
unset foo

Thanks.

					-Alex

> 
> ----
> # uname -r
> 3.10.0-rc4-00010-g0326739
> # rbd ls -l
> NAME                   SIZE PARENT                       FMT PROT LOCK 
> xxx                   1536G                                2
> # rbd map rbd/xxx
> # rbd showmapped
> id pool image   snap device    
> 1  rbd  xxx     -    /dev/rbd1 
> # dd if=/dev/rbd1 of=/tmp/xxx count=20480
> 20480+0 records in
> 20480+0 records out
> 10485760 bytes (10 MB) copied, 0.757754 s, 13.8 MB/s
> # od -c /tmp/xxx | less
> 0000000  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0
> *
> 50000000
> ----
> 
> 
> Cheers,
> 
> Chris
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Re: krbd + format=2 ?

[Chris Dunlop]

On Fri, Jun 07, 2013 at 11:54:20AM -0500, Alex Elder wrote:
> On 06/03/2013 04:24 AM, Chris Dunlop wrote:
>> I pulled the for-linus branch (@ 3abef3b) on top of 3.10.0-rc4, and it's
>> letting me map a format=2 image (created under bobtail), however reading
>> from the block device returns zeros rather than the data. The same image
>> correctly shows data (NTFS filesystem) when mounted into kvm using librbd.
> 
> Have you tried using a format 2 image that you created using
> the Linux rbd environment?  It would be good to know whether
> that works for you.

Sorry, how to you mean "created using the Linux rbd environment"?
The one I was trying was created using:

rbd create --format 2 xxx --size nnnnn

...then populated using qemu/librbd.

> Can you also send me the result of running each of these commands:
> 
> foo=$(rados get --pool=rbd rbd_id.xxx - | strings)
> rados ls --pool=rbd | grep rbd_data | grep "${foo}"
> unset foo

That results in a 3.2MB, 83973-line file, starting like:

rbd_data.e4d72ae8944a.0000000000027066
rbd_data.e4d72ae8944a.000000000001bc77
rbd_data.e4d72ae8944a.000000000002112e
rbd_data.e4d72ae8944a.000000000002550c
rbd_data.e4d72ae8944a.000000000001b6b3
rbd_data.e4d72ae8944a.000000000002875b

Complete file is at: https://www.dropbox.com/s/5bw6ba7ifatf1mu/rbd_data.out

(Note: it's a long w'end here on Oz, I won't be back at this till Tue.)

Cheers,

Chris

Re: krbd + format=2 ?

[Chris Dunlop]

On Sat, Jun 08, 2013 at 12:48:52PM +1000, Chris Dunlop wrote:
> On Fri, Jun 07, 2013 at 11:54:20AM -0500, Alex Elder wrote:
>> On 06/03/2013 04:24 AM, Chris Dunlop wrote:
>>> I pulled the for-linus branch (@ 3abef3b) on top of 3.10.0-rc4, and it's
>>> letting me map a format=2 image (created under bobtail), however reading
>>> from the block device returns zeros rather than the data. The same image
>>> correctly shows data (NTFS filesystem) when mounted into kvm using librbd.
>> 
>> Have you tried using a format 2 image that you created using
>> the Linux rbd environment?  It would be good to know whether
>> that works for you.
> 
> Sorry, how to you mean "created using the Linux rbd environment"?
> The one I was trying was created using:
> 
> rbd create --format 2 xxx --size nnnnn
> 
> ...then populated using qemu/librbd.

Looks like the kernel rbd and librbd aren't compatible, as at
3.10.0-rc4+ceph-client/for-linus@3abef3b vs librbd1 0.56.6-1~bpo70+1.

I can create a format=2 rbd image, map it, format and populate it using the
mapped device. I can then see that data after unmounting unmapping, then
re-mapping and re-mounting. However if I then attach that image via librbd to a
qemu-1.5.0~rc3 vm using the invocation below, the vm sees the whole device as
zeros.

Likewise, if I create a new format=2 rbd image, attach it via librbd to a
qemu-1.5.0~rc3 vm, format it and populate it, that data is thereafter visible
in the vm (e.g. after remounting etc.). But if I map that image using the
kernel client, again I get a device full of zeros.

# dpkg -l \*ceph\* \*rbd\* \*rados\* | grep ^i
ii  ceph           0.56.6-1~bpo70+1  amd64  distributed storage and file system
ii  ceph-common    0.56.6-1~bpo70+1  amd64  common utilities to mount and interact with a ceph storage cluster
ii  librados2      0.56.6-1~bpo70+1  amd64  RADOS distributed object store client library
ii  librbd1        0.56.6-1~bpo70+1  amd64  RADOS block device client library

##
## Attach disk to vm
##
# virsh attach-device test <(cat <<END
    <disk type='network' device='disk'>
      <driver name='qemu' type='raw'/>
      <auth username='vm'>
        <secret type='ceph' uuid='xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx'/>
      </auth> 
      <source protocol='rbd' name='rbd/crd-test'/>
      <target dev='vdc' bus='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x7' function='0x0'/>
    </disk>
END
)


Cheers,

Chris

Re: krbd + format=2 ?

[Josh Durgin]

On 06/11/2013 09:59 PM, Chris Dunlop wrote:
> On Sat, Jun 08, 2013 at 12:48:52PM +1000, Chris Dunlop wrote:
>> On Fri, Jun 07, 2013 at 11:54:20AM -0500, Alex Elder wrote:
>>> On 06/03/2013 04:24 AM, Chris Dunlop wrote:
>>>> I pulled the for-linus branch (@ 3abef3b) on top of 3.10.0-rc4, and it's
>>>> letting me map a format=2 image (created under bobtail), however reading
>>>> from the block device returns zeros rather than the data. The same image
>>>> correctly shows data (NTFS filesystem) when mounted into kvm using librbd.
>>>
>>> Have you tried using a format 2 image that you created using
>>> the Linux rbd environment?  It would be good to know whether
>>> that works for you.
>>
>> Sorry, how to you mean "created using the Linux rbd environment"?
>> The one I was trying was created using:
>>
>> rbd create --format 2 xxx --size nnnnn
>>
>> ...then populated using qemu/librbd.
>
> Looks like the kernel rbd and librbd aren't compatible, as at
> 3.10.0-rc4+ceph-client/for-linus@3abef3b vs librbd1 0.56.6-1~bpo70+1.

Thanks for the detailed report Chris. The kernel client was using the
wrong object names for format 2 (zero-padding them with a different
length than librbd). I just posted a patch fixing this.

Josh

Re: krbd + format=2 ?

[Chris Dunlop]

On Wed, Jun 12, 2013 at 08:56:50PM -0700, Josh Durgin wrote:
> On 06/11/2013 09:59 PM, Chris Dunlop wrote:
>> Looks like the kernel rbd and librbd aren't compatible, as at
>> 3.10.0-rc4+ceph-client/for-linus@3abef3b vs librbd1 0.56.6-1~bpo70+1.
> 
> Thanks for the detailed report Chris. The kernel client was using the
> wrong object names for format 2 (zero-padding them with a different
> length than librbd). I just posted a patch fixing this.

Works for me!

(I had to modify the first of your posted patches to apply it to
current linux master (@26e04462c) which pulled in Sage's
for-linus branch: see below.)

Cheers,

Chris.

----------------------------------------------------------------------
commit 932c369af21c9449649238f1b91d1062ab45384e
Author: Josh Durgin <josh.durgin@inktank.com>
Date:   Wed Jun 12 20:10:45 2013 -0700

    rbd: fetch object order before using it
    
    rbd_dev_v2_header_onetime() fetches striping information, and
    checks whether the image can be read by compariing the stripe unit
    to the object size. It determines the object size by shifting
    the object order, which is 0 at this point since it has not been
    read yet. Move the call to get the image size and object order
    before rbd_dev_v2_header_onetime() so it is set before use.
    
    Signed-off-by: Josh Durgin <josh.durgin@inktank.com>

diff --git a/drivers/block/rbd.c b/drivers/block/rbd.c
index 3063452..2fecab2 100644
--- a/drivers/block/rbd.c
+++ b/drivers/block/rbd.c
@@ -4239,6 +4239,10 @@ static int rbd_dev_v2_header_info(struct rbd_device *rbd_dev)
 
 	down_write(&rbd_dev->header_rwsem);
 
+	ret = rbd_dev_v2_image_size(rbd_dev);
+	if (ret)
+		goto out;
+
 	if (first_time) {
 		ret = rbd_dev_v2_header_onetime(rbd_dev);
 		if (ret)
@@ -4272,10 +4276,6 @@ static int rbd_dev_v2_header_info(struct rbd_device *rbd_dev)
 					"is EXPERIMENTAL!");
 	}
 
-	ret = rbd_dev_v2_image_size(rbd_dev);
-	if (ret)
-		goto out;
-
 	if (rbd_dev->spec->snap_id == CEPH_NOSNAP)
 		if (rbd_dev->mapping.size != rbd_dev->header.image_size)
 			rbd_dev->mapping.size = rbd_dev->header.image_size;

Re: krbd + format=2 ?

[Alex Elder]

On 06/12/2013 10:56 PM, Josh Durgin wrote:
> On 06/11/2013 09:59 PM, Chris Dunlop wrote:
>> On Sat, Jun 08, 2013 at 12:48:52PM +1000, Chris Dunlop wrote:
>>> On Fri, Jun 07, 2013 at 11:54:20AM -0500, Alex Elder wrote:
>>>> On 06/03/2013 04:24 AM, Chris Dunlop wrote:
>>>>> I pulled the for-linus branch (@ 3abef3b) on top of 3.10.0-rc4, and
>>>>> it's
>>>>> letting me map a format=2 image (created under bobtail), however
>>>>> reading
>>>>> from the block device returns zeros rather than the data. The same
>>>>> image
>>>>> correctly shows data (NTFS filesystem) when mounted into kvm using
>>>>> librbd.
>>>>
>>>> Have you tried using a format 2 image that you created using
>>>> the Linux rbd environment?  It would be good to know whether
>>>> that works for you.
>>>
>>> Sorry, how to you mean "created using the Linux rbd environment"?
>>> The one I was trying was created using:
>>>
>>> rbd create --format 2 xxx --size nnnnn
>>>
>>> ...then populated using qemu/librbd.
>>
>> Looks like the kernel rbd and librbd aren't compatible, as at
>> 3.10.0-rc4+ceph-client/for-linus@3abef3b vs librbd1 0.56.6-1~bpo70+1.
> 
> Thanks for the detailed report Chris. The kernel client was using the
> wrong object names for format 2 (zero-padding them with a different
> length than librbd). I just posted a patch fixing this.

I remember talking about this Josh.  I had suggested we
use a longer one so the full range of values would be
represented by the default name, and as I recall you said
it had already been done on the user space side.  I'm
sorry if no bug got opened to be sure that got resolved...

					-Alex

> Josh
> 

Re: krbd + format=2 ?

[Laurent Barbe]

Hello,

Since I upgrade kernel 3.10-rc6 to 3.10 final, it seems that the format 
of block device has changed and I can't mount them anymore.
I'm using rbd format 2 / xfs.

Are you aware of this incompatibility?


SGI XFS with ACLs, security attributes, realtime, large block/inode ug 
enabled
XFS (rbd1): bad magic number
ffff880037241000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  .
ffff880037241010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  .
ffff880037241020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  .
ffff880037241030: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  .
XFS (rbd1): Internal error xfs_sb_read_verify at line 730 of file t.c. 
Caller 0xffffffffa0522d95

CPU: 0 PID: 182 Comm: kworker/0:1H Not tainted 3.10.0-ccmbg1 #1
Hardware name: Dell Computer Corporation PowerEdge 860/0XM089, BIOS
Workqueue: xfslogd xfs_buf_iodone_work [xfs]
  ffffffff81362b9c 0000000000000071 ffffffffa0524c61 ffffffffa0522d95
  00000000000002da 0000000000000000 0000000000000016 ffff88007cbdde00
  ffff88003704c800 ffff88007fc1a500 ffffffffa0566222 ffffffffa0522d95
Call Trace:
  [<ffffffff81362b9c>] ? dump_stack+0xd/0x17
  [<ffffffffa0524c61>] ? xfs_corruption_error+0x54/0x6f [xfs]
  [<ffffffffa0522d95>] ? xfs_buf_iodone_work+0x3c/0x6a [xfs]
  [<ffffffffa0566222>] ? xfs_sb_read_verify+0xa4/0xbf [xfs]
  [<ffffffffa0522d95>] ? xfs_buf_iodone_work+0x3c/0x6a [xfs]
  [<ffffffffa0522d95>] ? xfs_buf_iodone_work+0x3c/0x6a [xfs]
  [<ffffffff81046588>] ? process_one_work+0x191/0x28f
  [<ffffffff813650f4>] ? __schedule+0x516/0x51b
  [<ffffffff81046a35>] ? worker_thread+0x121/0x1e7
  [<ffffffff81046914>] ? rescuer_thread+0x269/0x269
  [<ffffffff8104aedd>] ? kthread+0x81/0x89
  [<ffffffff8104ae5c>] ? __kthread_parkme+0x5d/0x5d
  [<ffffffff8136adec>] ? ret_from_fork+0x7c/0xb0
  [<ffffffff8104ae5c>] ? __kthread_parkme+0x5d/0x5d
XFS (rbd1): Corruption detected. Unmount and run xfs_repair
XFS (rbd1): SB validate failed with error 22.

Thanks,

Laurent Barbe


Le 13/06/2013 05:56, Josh Durgin a écrit :
> On 06/11/2013 09:59 PM, Chris Dunlop wrote:
>> On Sat, Jun 08, 2013 at 12:48:52PM +1000, Chris Dunlop wrote:
>>> On Fri, Jun 07, 2013 at 11:54:20AM -0500, Alex Elder wrote:
>>>> On 06/03/2013 04:24 AM, Chris Dunlop wrote:
>>>>> I pulled the for-linus branch (@ 3abef3b) on top of 3.10.0-rc4, and
>>>>> it's
>>>>> letting me map a format=2 image (created under bobtail), however
>>>>> reading
>>>>> from the block device returns zeros rather than the data. The same
>>>>> image
>>>>> correctly shows data (NTFS filesystem) when mounted into kvm using
>>>>> librbd.
>>>>
>>>> Have you tried using a format 2 image that you created using
>>>> the Linux rbd environment?  It would be good to know whether
>>>> that works for you.
>>>
>>> Sorry, how to you mean "created using the Linux rbd environment"?
>>> The one I was trying was created using:
>>>
>>> rbd create --format 2 xxx --size nnnnn
>>>
>>> ...then populated using qemu/librbd.
>>
>> Looks like the kernel rbd and librbd aren't compatible, as at
>> 3.10.0-rc4+ceph-client/for-linus@3abef3b vs librbd1 0.56.6-1~bpo70+1.
>
> Thanks for the detailed report Chris. The kernel client was using the
> wrong object names for format 2 (zero-padding them with a different
> length than librbd). I just posted a patch fixing this.
>
> Josh
>
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: krbd + format=2 ?

[Sage Weil]

On Thu, 4 Jul 2013, Laurent Barbe wrote:
> Hello,
> 
> Since I upgrade kernel 3.10-rc6 to 3.10 final, it seems that the format of
> block device has changed and I can't mount them anymore.
> I'm using rbd format 2 / xfs.
> 
> Are you aware of this incompatibility?

Hi Laurent,

There was a problem with earlier -rc's not interoperating with librbd 
because of the object naming.  To recover this image, boot into an -rc6 
kernel, copy the block device out of rbd or into a new image (e.g., rbd 
import /dev/rbd1 newrbd), and then use the new image with 3.10.

Sorry!
sage



> 
> 
> SGI XFS with ACLs, security attributes, realtime, large block/inode ug enabled
> XFS (rbd1): bad magic number
> ffff880037241000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  .
> ffff880037241010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  .
> ffff880037241020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  .
> ffff880037241030: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  .
> XFS (rbd1): Internal error xfs_sb_read_verify at line 730 of file t.c. Caller
> 0xffffffffa0522d95
> 
> CPU: 0 PID: 182 Comm: kworker/0:1H Not tainted 3.10.0-ccmbg1 #1
> Hardware name: Dell Computer Corporation PowerEdge 860/0XM089, BIOS
> Workqueue: xfslogd xfs_buf_iodone_work [xfs]
>  ffffffff81362b9c 0000000000000071 ffffffffa0524c61 ffffffffa0522d95
>  00000000000002da 0000000000000000 0000000000000016 ffff88007cbdde00
>  ffff88003704c800 ffff88007fc1a500 ffffffffa0566222 ffffffffa0522d95
> Call Trace:
>  [<ffffffff81362b9c>] ? dump_stack+0xd/0x17
>  [<ffffffffa0524c61>] ? xfs_corruption_error+0x54/0x6f [xfs]
>  [<ffffffffa0522d95>] ? xfs_buf_iodone_work+0x3c/0x6a [xfs]
>  [<ffffffffa0566222>] ? xfs_sb_read_verify+0xa4/0xbf [xfs]
>  [<ffffffffa0522d95>] ? xfs_buf_iodone_work+0x3c/0x6a [xfs]
>  [<ffffffffa0522d95>] ? xfs_buf_iodone_work+0x3c/0x6a [xfs]
>  [<ffffffff81046588>] ? process_one_work+0x191/0x28f
>  [<ffffffff813650f4>] ? __schedule+0x516/0x51b
>  [<ffffffff81046a35>] ? worker_thread+0x121/0x1e7
>  [<ffffffff81046914>] ? rescuer_thread+0x269/0x269
>  [<ffffffff8104aedd>] ? kthread+0x81/0x89
>  [<ffffffff8104ae5c>] ? __kthread_parkme+0x5d/0x5d
>  [<ffffffff8136adec>] ? ret_from_fork+0x7c/0xb0
>  [<ffffffff8104ae5c>] ? __kthread_parkme+0x5d/0x5d
> XFS (rbd1): Corruption detected. Unmount and run xfs_repair
> XFS (rbd1): SB validate failed with error 22.
> 
> Thanks,
> 
> Laurent Barbe
> 
> 
> Le 13/06/2013 05:56, Josh Durgin a ?crit :
> > On 06/11/2013 09:59 PM, Chris Dunlop wrote:
> > > On Sat, Jun 08, 2013 at 12:48:52PM +1000, Chris Dunlop wrote:
> > > > On Fri, Jun 07, 2013 at 11:54:20AM -0500, Alex Elder wrote:
> > > > > On 06/03/2013 04:24 AM, Chris Dunlop wrote:
> > > > > > I pulled the for-linus branch (@ 3abef3b) on top of 3.10.0-rc4, and
> > > > > > it's
> > > > > > letting me map a format=2 image (created under bobtail), however
> > > > > > reading
> > > > > > from the block device returns zeros rather than the data. The same
> > > > > > image
> > > > > > correctly shows data (NTFS filesystem) when mounted into kvm using
> > > > > > librbd.
> > > > > 
> > > > > Have you tried using a format 2 image that you created using
> > > > > the Linux rbd environment?  It would be good to know whether
> > > > > that works for you.
> > > > 
> > > > Sorry, how to you mean "created using the Linux rbd environment"?
> > > > The one I was trying was created using:
> > > > 
> > > > rbd create --format 2 xxx --size nnnnn
> > > > 
> > > > ...then populated using qemu/librbd.
> > > 
> > > Looks like the kernel rbd and librbd aren't compatible, as at
> > > 3.10.0-rc4+ceph-client/for-linus@3abef3b vs librbd1 0.56.6-1~bpo70+1.
> > 
> > Thanks for the detailed report Chris. The kernel client was using the
> > wrong object names for format 2 (zero-padding them with a different
> > length than librbd). I just posted a patch fixing this.
> > 
> > Josh
> > 
> > --
> > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
> 

Re: krbd + format=2 ?

[Laurent Barbe]

Hello Sage,

Thank you for your reply,
I will do like this.

Laurent


Le 05/07/2013 06:12, Sage Weil a écrit :
> On Thu, 4 Jul 2013, Laurent Barbe wrote:
>> Hello,
>>
>> Since I upgrade kernel 3.10-rc6 to 3.10 final, it seems that the format of
>> block device has changed and I can't mount them anymore.
>> I'm using rbd format 2 / xfs.
>>
>> Are you aware of this incompatibility?
>
> Hi Laurent,
>
> There was a problem with earlier -rc's not interoperating with librbd
> because of the object naming.  To recover this image, boot into an -rc6
> kernel, copy the block device out of rbd or into a new image (e.g., rbd
> import /dev/rbd1 newrbd), and then use the new image with 3.10.
>
> Sorry!
> sage
>
>
>
>>
>>
>> SGI XFS with ACLs, security attributes, realtime, large block/inode ug enabled
>> XFS (rbd1): bad magic number
>> ffff880037241000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  .
>> ffff880037241010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  .
>> ffff880037241020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  .
>> ffff880037241030: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  .
>> XFS (rbd1): Internal error xfs_sb_read_verify at line 730 of file t.c. Caller
>> 0xffffffffa0522d95
>>
>> CPU: 0 PID: 182 Comm: kworker/0:1H Not tainted 3.10.0-ccmbg1 #1
>> Hardware name: Dell Computer Corporation PowerEdge 860/0XM089, BIOS
>> Workqueue: xfslogd xfs_buf_iodone_work [xfs]
>>   ffffffff81362b9c 0000000000000071 ffffffffa0524c61 ffffffffa0522d95
>>   00000000000002da 0000000000000000 0000000000000016 ffff88007cbdde00
>>   ffff88003704c800 ffff88007fc1a500 ffffffffa0566222 ffffffffa0522d95
>> Call Trace:
>>   [<ffffffff81362b9c>] ? dump_stack+0xd/0x17
>>   [<ffffffffa0524c61>] ? xfs_corruption_error+0x54/0x6f [xfs]
>>   [<ffffffffa0522d95>] ? xfs_buf_iodone_work+0x3c/0x6a [xfs]
>>   [<ffffffffa0566222>] ? xfs_sb_read_verify+0xa4/0xbf [xfs]
>>   [<ffffffffa0522d95>] ? xfs_buf_iodone_work+0x3c/0x6a [xfs]
>>   [<ffffffffa0522d95>] ? xfs_buf_iodone_work+0x3c/0x6a [xfs]
>>   [<ffffffff81046588>] ? process_one_work+0x191/0x28f
>>   [<ffffffff813650f4>] ? __schedule+0x516/0x51b
>>   [<ffffffff81046a35>] ? worker_thread+0x121/0x1e7
>>   [<ffffffff81046914>] ? rescuer_thread+0x269/0x269
>>   [<ffffffff8104aedd>] ? kthread+0x81/0x89
>>   [<ffffffff8104ae5c>] ? __kthread_parkme+0x5d/0x5d
>>   [<ffffffff8136adec>] ? ret_from_fork+0x7c/0xb0
>>   [<ffffffff8104ae5c>] ? __kthread_parkme+0x5d/0x5d
>> XFS (rbd1): Corruption detected. Unmount and run xfs_repair
>> XFS (rbd1): SB validate failed with error 22.
>>
>> Thanks,
>>
>> Laurent Barbe
>>
>>
>> Le 13/06/2013 05:56, Josh Durgin a ?crit :
>>> On 06/11/2013 09:59 PM, Chris Dunlop wrote:
>>>> On Sat, Jun 08, 2013 at 12:48:52PM +1000, Chris Dunlop wrote:
>>>>> On Fri, Jun 07, 2013 at 11:54:20AM -0500, Alex Elder wrote:
>>>>>> On 06/03/2013 04:24 AM, Chris Dunlop wrote:
>>>>>>> I pulled the for-linus branch (@ 3abef3b) on top of 3.10.0-rc4, and
>>>>>>> it's
>>>>>>> letting me map a format=2 image (created under bobtail), however
>>>>>>> reading
>>>>>>> from the block device returns zeros rather than the data. The same
>>>>>>> image
>>>>>>> correctly shows data (NTFS filesystem) when mounted into kvm using
>>>>>>> librbd.
>>>>>>
>>>>>> Have you tried using a format 2 image that you created using
>>>>>> the Linux rbd environment?  It would be good to know whether
>>>>>> that works for you.
>>>>>
>>>>> Sorry, how to you mean "created using the Linux rbd environment"?
>>>>> The one I was trying was created using:
>>>>>
>>>>> rbd create --format 2 xxx --size nnnnn
>>>>>
>>>>> ...then populated using qemu/librbd.
>>>>
>>>> Looks like the kernel rbd and librbd aren't compatible, as at
>>>> 3.10.0-rc4+ceph-client/for-linus@3abef3b vs librbd1 0.56.6-1~bpo70+1.
>>>
>>> Thanks for the detailed report Chris. The kernel client was using the
>>> wrong object names for format 2 (zero-padding them with a different
>>> length than librbd). I just posted a patch fixing this.
>>>
>>> Josh
>>>
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>> --
>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
>>
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html