rbd-nbd 2T limit

rbd-nbd 2T limit

[Max Yehorov]

I was reading through release notes for 10.2.6 and there was a ticket

rbd: rbd-nbd: disallow mapping images >2TB in size (issue#17219,
pr#11870, Mykola Golub)

default nbd block size is 1k, valid values are from 512 to PAGE_SIZE

rbd_nbd.cc sets blocksize to 512 here:
https://github.com/ceph/ceph/blob/master/src/tools/rbd_nbd/rbd-nbd.cc#L635

Why 512 was chosen for rbd_nbd?
krbd also uses 512 sector size. Is there a reason to use 512 block
size vs 4k AF?

Re: rbd-nbd 2T limit

[Henrik Korkuc]

On 17-03-09 05:38, Max Yehorov wrote:
> I was reading through release notes for 10.2.6 and there was a ticket
>
> rbd: rbd-nbd: disallow mapping images >2TB in size (issue#17219,
> pr#11870, Mykola Golub)
>
> default nbd block size is 1k, valid values are from 512 to PAGE_SIZE
>
> rbd_nbd.cc sets blocksize to 512 here:
> https://github.com/ceph/ceph/blob/master/src/tools/rbd_nbd/rbd-nbd.cc#L635
>
> Why 512 was chosen for rbd_nbd?
> krbd also uses 512 sector size. Is there a reason to use 512 block
> size vs 4k AF?
I am also interested in >2TB NBD support and was disapointed when I 
couldn't mount 10TB images.

Based on issue in that commit for me it seems to be clear why it was 
done - some (most?) kernels in the wild do not support larger sizes. But 
it looks like support on unaffected kernels is already in master (not 
sure if it got backported to current releases). For reference:
http://tracker.ceph.com/issues/17219 -> 
https://github.com/ceph/ceph/pull/11741 -> 
http://tracker.ceph.com/issues/18335 -> 
https://github.com/ceph/ceph/pull/13229

Re: rbd-nbd 2T limit

[Mykola Golub]

On Thu, Mar 09, 2017 at 09:14:53AM +0200, Henrik Korkuc wrote:
> On 17-03-09 05:38, Max Yehorov wrote:
> >I was reading through release notes for 10.2.6 and there was a ticket
> >
> >rbd: rbd-nbd: disallow mapping images >2TB in size (issue#17219,
> >pr#11870, Mykola Golub)
> >
> >default nbd block size is 1k, valid values are from 512 to PAGE_SIZE
> >
> >rbd_nbd.cc sets blocksize to 512 here:
> >https://github.com/ceph/ceph/blob/master/src/tools/rbd_nbd/rbd-nbd.cc#L635
> >
> >Why 512 was chosen for rbd_nbd?
> >krbd also uses 512 sector size. Is there a reason to use 512 block
> >size vs 4k AF?
> I am also interested in >2TB NBD support and was disapointed when I couldn't
> mount 10TB images.
> 
> Based on issue in that commit for me it seems to be clear why it was done -
> some (most?) kernels in the wild do not support larger sizes. But it looks
> like support on unaffected kernels is already in master (not sure if it got
> backported to current releases). For reference:
> http://tracker.ceph.com/issues/17219 ->
> https://github.com/ceph/ceph/pull/11741 ->
> http://tracker.ceph.com/issues/18335 ->
> https://github.com/ceph/ceph/pull/13229

Yes, it removes the restriction for working kernels. It is going to be
backported to kraken and jewel:

http://tracker.ceph.com/issues/18910
http://tracker.ceph.com/issues/18911

-- 
Mykola Golub