* Ceph doesn't update the block device size while a rbd image is mounted [not found] <CAOLwVUkWUrjQTxQw5-umpWnY6G-AjReCwLTyQCOdCyNT175KgA@mail.gmail.com> @ 2012-07-19 14:55 ` Sébastien Han 2012-07-19 15:15 ` Wido den Hollander 0 siblings, 1 reply; 12+ messages in thread From: Sébastien Han @ 2012-07-19 14:55 UTC (permalink / raw) To: ceph-devel Hi Cephers! I'm working with rbd mapping. I figured out that the block device size of the rbd device is not update while the device is mounted. Here my tests: 1. Pick up a device and check his size # rbd ls size # rbd info test rbd image 'test': size 10000 MB in 2500 objects order 22 (4096 KB objects) block_name_prefix: rb.0.6 parent: (pool -1) 2. Map the device # rbd map --secret /etc/ceph/secret test # rbd showmapped id pool image snap device 1 rbd test - /dev/rbd1 3. Put a fs on it and check the block device size # mkfs.ext4 /dev/rdb1 ... ... # fdisk -l /dev/rbd1 Disk /dev/rbd1: 10.5 GB, 10485760000 bytes 4. Mount it # mount /dev/rbd1 /mnt # df -h /dev/rbd1 9.8G 277M 9.0G 3% /mnt 5. Change the image size # rbd resize --size 11000 test Resizing image: 100% complete...done. # rbd info test rbd image 'test': size 11000 MB in 2750 objects order 22 (4096 KB objects) block_name_prefix: rb.0.6 parent: (pool -1) At this point of time, if you perform the fdisk -l /dev/rbd1, the block device size will remain the same. 6. Unmount the device: # umount /media # fdisk -l /dev/rbd1 Disk /dev/rbd1: 11.5 GB, 11534336000 bytes Unmounting the directory did update the block device size. Of course you can do something really fast like: # umount /media && mount /dev/rbd1 /media That will work, it's a valid solution as long as there is no opened file. I won't use this trick in production... I also tried to "mount -o remount" and it didn't work. 7. Resize the fs (this can be performed while the fs is mounted): # e2fsck -f /dev/rbd1 e2fsck 1.42 (29-Nov-2011) Pass 1: Checking inodes, blocks, and sizes Pass 2: Checking directory structure Pass 3: Checking directory connectivity Pass 4: Checking reference counts Pass 5: Checking group summary information /dev/rbd1: 11/644640 files (0.0% non-contiguous), 77173/2560000 blocks # resize2fs /dev/rbd1 resize2fs 1.42 (29-Nov-2011) Resizing the filesystem on /dev/rbd1 to 2816000 (4k) blocks. The filesystem on /dev/rbd1 is now 2816000 blocks long. Did I miss something? Is this feature coming? Thank you in advance :) ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Ceph doesn't update the block device size while a rbd image is mounted 2012-07-19 14:55 ` Ceph doesn't update the block device size while a rbd image is mounted Sébastien Han @ 2012-07-19 15:15 ` Wido den Hollander 2012-07-19 15:26 ` Sébastien Han 0 siblings, 1 reply; 12+ messages in thread From: Wido den Hollander @ 2012-07-19 15:15 UTC (permalink / raw) To: Sébastien Han; +Cc: ceph-devel Hi, On 19-07-12 16:55, Sébastien Han wrote: > Hi Cephers! > > I'm working with rbd mapping. I figured out that the block device size > of the rbd device is not update while the device is mounted. Here my > tests: > iirc this is not something RBD specific, but since the device is in use it can't be re-read by the kernel. So when you unmount it the kernel can re-read the header and resize the device. Wido > 1. Pick up a device and check his size > > # rbd ls > size > > # rbd info test > rbd image 'test': > size 10000 MB in 2500 objects > order 22 (4096 KB objects) > block_name_prefix: rb.0.6 > parent: (pool -1) > > 2. Map the device > > # rbd map --secret /etc/ceph/secret test > # rbd showmapped > id pool image snap device > 1 rbd test - /dev/rbd1 > > 3. Put a fs on it and check the block device size > > # mkfs.ext4 /dev/rdb1 > ... > ... > > # fdisk -l /dev/rbd1 > > Disk /dev/rbd1: 10.5 GB, 10485760000 bytes > > 4. Mount it > > # mount /dev/rbd1 /mnt > # df -h > /dev/rbd1 9.8G 277M 9.0G 3% /mnt > > > 5. Change the image size > > # rbd resize --size 11000 test > Resizing image: 100% complete...done. > > # rbd info test > rbd image 'test': > size 11000 MB in 2750 objects > order 22 (4096 KB objects) > block_name_prefix: rb.0.6 > parent: (pool -1) > > At this point of time, if you perform the fdisk -l /dev/rbd1, the > block device size will remain the same. > > 6. Unmount the device: > > # umount /media > > # fdisk -l /dev/rbd1 > Disk /dev/rbd1: 11.5 GB, 11534336000 bytes > > Unmounting the directory did update the block device size. > > Of course you can do something really fast like: > > # umount /media && mount /dev/rbd1 /media > > That will work, it's a valid solution as long as there is no opened > file. I won't use this trick in production... > > I also tried to "mount -o remount" and it didn't work. > > 7. Resize the fs (this can be performed while the fs is mounted): > > # e2fsck -f /dev/rbd1 > e2fsck 1.42 (29-Nov-2011) > Pass 1: Checking inodes, blocks, and sizes > Pass 2: Checking directory structure > Pass 3: Checking directory connectivity > Pass 4: Checking reference counts > Pass 5: Checking group summary information > /dev/rbd1: 11/644640 files (0.0% non-contiguous), 77173/2560000 blocks > > # resize2fs /dev/rbd1 > resize2fs 1.42 (29-Nov-2011) > Resizing the filesystem on /dev/rbd1 to 2816000 (4k) blocks. > The filesystem on /dev/rbd1 is now 2816000 blocks long. > > > Did I miss something? > Is this feature coming? > > Thank you in advance :) > -- > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Ceph doesn't update the block device size while a rbd image is mounted 2012-07-19 15:15 ` Wido den Hollander @ 2012-07-19 15:26 ` Sébastien Han 2012-07-19 15:38 ` Tommi Virtanen 2012-07-19 15:47 ` Wido den Hollander 0 siblings, 2 replies; 12+ messages in thread From: Sébastien Han @ 2012-07-19 15:26 UTC (permalink / raw) To: Wido den Hollander; +Cc: ceph-devel Ok I got your point seems logic, but why is this possible with LVM for example? You can easily do this with LVM without un-mounting the device. Cheers. On Thu, Jul 19, 2012 at 5:15 PM, Wido den Hollander <wido@widodh.nl> wrote: > Hi, > > > On 19-07-12 16:55, Sébastien Han wrote: >> >> Hi Cephers! >> >> I'm working with rbd mapping. I figured out that the block device size >> of the rbd device is not update while the device is mounted. Here my >> tests: >> > > iirc this is not something RBD specific, but since the device is in use it > can't be re-read by the kernel. > > So when you unmount it the kernel can re-read the header and resize the > device. > > Wido > >> 1. Pick up a device and check his size >> >> # rbd ls >> size >> >> # rbd info test >> rbd image 'test': >> size 10000 MB in 2500 objects >> order 22 (4096 KB objects) >> block_name_prefix: rb.0.6 >> parent: (pool -1) >> >> 2. Map the device >> >> # rbd map --secret /etc/ceph/secret test >> # rbd showmapped >> id pool image snap device >> 1 rbd test - /dev/rbd1 >> >> 3. Put a fs on it and check the block device size >> >> # mkfs.ext4 /dev/rdb1 >> ... >> ... >> >> # fdisk -l /dev/rbd1 >> >> Disk /dev/rbd1: 10.5 GB, 10485760000 bytes >> >> 4. Mount it >> >> # mount /dev/rbd1 /mnt >> # df -h >> /dev/rbd1 9.8G 277M 9.0G 3% /mnt >> >> >> 5. Change the image size >> >> # rbd resize --size 11000 test >> Resizing image: 100% complete...done. >> >> # rbd info test >> rbd image 'test': >> size 11000 MB in 2750 objects >> order 22 (4096 KB objects) >> block_name_prefix: rb.0.6 >> parent: (pool -1) >> >> At this point of time, if you perform the fdisk -l /dev/rbd1, the >> block device size will remain the same. >> >> 6. Unmount the device: >> >> # umount /media >> >> # fdisk -l /dev/rbd1 >> Disk /dev/rbd1: 11.5 GB, 11534336000 bytes >> >> Unmounting the directory did update the block device size. >> >> Of course you can do something really fast like: >> >> # umount /media && mount /dev/rbd1 /media >> >> That will work, it's a valid solution as long as there is no opened >> file. I won't use this trick in production... >> >> I also tried to "mount -o remount" and it didn't work. >> >> 7. Resize the fs (this can be performed while the fs is mounted): >> >> # e2fsck -f /dev/rbd1 >> e2fsck 1.42 (29-Nov-2011) >> Pass 1: Checking inodes, blocks, and sizes >> Pass 2: Checking directory structure >> Pass 3: Checking directory connectivity >> Pass 4: Checking reference counts >> Pass 5: Checking group summary information >> /dev/rbd1: 11/644640 files (0.0% non-contiguous), 77173/2560000 blocks >> >> # resize2fs /dev/rbd1 >> resize2fs 1.42 (29-Nov-2011) >> Resizing the filesystem on /dev/rbd1 to 2816000 (4k) blocks. >> The filesystem on /dev/rbd1 is now 2816000 blocks long. >> >> >> Did I miss something? >> Is this feature coming? >> >> Thank you in advance :) >> -- >> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >> the body of a message to majordomo@vger.kernel.org >> More majordomo info at http://vger.kernel.org/majordomo-info.html >> > -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Ceph doesn't update the block device size while a rbd image is mounted 2012-07-19 15:26 ` Sébastien Han @ 2012-07-19 15:38 ` Tommi Virtanen 2012-07-19 15:39 ` Tommi Virtanen 2012-07-19 15:47 ` Wido den Hollander 1 sibling, 1 reply; 12+ messages in thread From: Tommi Virtanen @ 2012-07-19 15:38 UTC (permalink / raw) To: Sébastien Han; +Cc: Wido den Hollander, ceph-devel On Thu, Jul 19, 2012 at 8:26 AM, Sébastien Han <han.sebastien@gmail.com> wrote: > Ok I got your point seems logic, but why is this possible with LVM for example? > > You can easily do this with LVM without un-mounting the device. Do your LVM volumes have partition tables inside them? That might be the difference. Of course, you can put your filesystem straight on the RBD; that would be a good experiment to run. -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Ceph doesn't update the block device size while a rbd image is mounted 2012-07-19 15:38 ` Tommi Virtanen @ 2012-07-19 15:39 ` Tommi Virtanen 0 siblings, 0 replies; 12+ messages in thread From: Tommi Virtanen @ 2012-07-19 15:39 UTC (permalink / raw) To: Sébastien Han; +Cc: Wido den Hollander, ceph-devel On Thu, Jul 19, 2012 at 8:38 AM, Tommi Virtanen <tv@inktank.com> wrote: > Do your LVM volumes have partition tables inside them? That might be > the difference. > > Of course, you can put your filesystem straight on the RBD; that would > be a good experiment to run. Oops, I see you did put your fs straight on the RBD already; seeing "fdisk" tripped my brain. ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Ceph doesn't update the block device size while a rbd image is mounted 2012-07-19 15:26 ` Sébastien Han 2012-07-19 15:38 ` Tommi Virtanen @ 2012-07-19 15:47 ` Wido den Hollander 2012-07-19 15:50 ` Sébastien Han 1 sibling, 1 reply; 12+ messages in thread From: Wido den Hollander @ 2012-07-19 15:47 UTC (permalink / raw) To: Sébastien Han; +Cc: ceph-devel On 19-07-12 17:26, Sébastien Han wrote: > Ok I got your point seems logic, but why is this possible with LVM for example? > > You can easily do this with LVM without un-mounting the device. > LVM runs through the device mapper and are not regular block devices. If you resize the disk underneath LVM you won't see an increased VG or PV size unless you change the availability of the VG to unavailable and back to available again. I'm not a 100% sure what the exact root cause is, but the kernel won't read the new size of a block device as long as it is in use. Wido > Cheers. > > > On Thu, Jul 19, 2012 at 5:15 PM, Wido den Hollander <wido@widodh.nl> wrote: >> Hi, >> >> >> On 19-07-12 16:55, Sébastien Han wrote: >>> >>> Hi Cephers! >>> >>> I'm working with rbd mapping. I figured out that the block device size >>> of the rbd device is not update while the device is mounted. Here my >>> tests: >>> >> >> iirc this is not something RBD specific, but since the device is in use it >> can't be re-read by the kernel. >> >> So when you unmount it the kernel can re-read the header and resize the >> device. >> >> Wido >> >>> 1. Pick up a device and check his size >>> >>> # rbd ls >>> size >>> >>> # rbd info test >>> rbd image 'test': >>> size 10000 MB in 2500 objects >>> order 22 (4096 KB objects) >>> block_name_prefix: rb.0.6 >>> parent: (pool -1) >>> >>> 2. Map the device >>> >>> # rbd map --secret /etc/ceph/secret test >>> # rbd showmapped >>> id pool image snap device >>> 1 rbd test - /dev/rbd1 >>> >>> 3. Put a fs on it and check the block device size >>> >>> # mkfs.ext4 /dev/rdb1 >>> ... >>> ... >>> >>> # fdisk -l /dev/rbd1 >>> >>> Disk /dev/rbd1: 10.5 GB, 10485760000 bytes >>> >>> 4. Mount it >>> >>> # mount /dev/rbd1 /mnt >>> # df -h >>> /dev/rbd1 9.8G 277M 9.0G 3% /mnt >>> >>> >>> 5. Change the image size >>> >>> # rbd resize --size 11000 test >>> Resizing image: 100% complete...done. >>> >>> # rbd info test >>> rbd image 'test': >>> size 11000 MB in 2750 objects >>> order 22 (4096 KB objects) >>> block_name_prefix: rb.0.6 >>> parent: (pool -1) >>> >>> At this point of time, if you perform the fdisk -l /dev/rbd1, the >>> block device size will remain the same. >>> >>> 6. Unmount the device: >>> >>> # umount /media >>> >>> # fdisk -l /dev/rbd1 >>> Disk /dev/rbd1: 11.5 GB, 11534336000 bytes >>> >>> Unmounting the directory did update the block device size. >>> >>> Of course you can do something really fast like: >>> >>> # umount /media && mount /dev/rbd1 /media >>> >>> That will work, it's a valid solution as long as there is no opened >>> file. I won't use this trick in production... >>> >>> I also tried to "mount -o remount" and it didn't work. >>> >>> 7. Resize the fs (this can be performed while the fs is mounted): >>> >>> # e2fsck -f /dev/rbd1 >>> e2fsck 1.42 (29-Nov-2011) >>> Pass 1: Checking inodes, blocks, and sizes >>> Pass 2: Checking directory structure >>> Pass 3: Checking directory connectivity >>> Pass 4: Checking reference counts >>> Pass 5: Checking group summary information >>> /dev/rbd1: 11/644640 files (0.0% non-contiguous), 77173/2560000 blocks >>> >>> # resize2fs /dev/rbd1 >>> resize2fs 1.42 (29-Nov-2011) >>> Resizing the filesystem on /dev/rbd1 to 2816000 (4k) blocks. >>> The filesystem on /dev/rbd1 is now 2816000 blocks long. >>> >>> >>> Did I miss something? >>> Is this feature coming? >>> >>> Thank you in advance :) >>> -- >>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >>> the body of a message to majordomo@vger.kernel.org >>> More majordomo info at http://vger.kernel.org/majordomo-info.html >>> >> -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Ceph doesn't update the block device size while a rbd image is mounted 2012-07-19 15:47 ` Wido den Hollander @ 2012-07-19 15:50 ` Sébastien Han 2012-07-19 17:49 ` Calvin Morrow 2012-07-19 19:44 ` Sébastien Han 0 siblings, 2 replies; 12+ messages in thread From: Sébastien Han @ 2012-07-19 15:50 UTC (permalink / raw) To: Wido den Hollander; +Cc: ceph-devel Hum ok, I see. Thanks! But if you have any clue to force the kernel to re-read without unmont/mounting :) On Thu, Jul 19, 2012 at 5:47 PM, Wido den Hollander <wido@widodh.nl> wrote: > > > On 19-07-12 17:26, Sébastien Han wrote: >> >> Ok I got your point seems logic, but why is this possible with LVM for >> example? >> >> You can easily do this with LVM without un-mounting the device. >> > > > LVM runs through the device mapper and are not regular block devices. > > If you resize the disk underneath LVM you won't see an increased VG or PV > size unless you change the availability of the VG to unavailable and back to > available again. > > I'm not a 100% sure what the exact root cause is, but the kernel won't read > the new size of a block device as long as it is in use. > > Wido > > > >> Cheers. >> >> >> On Thu, Jul 19, 2012 at 5:15 PM, Wido den Hollander <wido@widodh.nl> >> wrote: >>> >>> Hi, >>> >>> >>> On 19-07-12 16:55, Sébastien Han wrote: >>>> >>>> >>>> Hi Cephers! >>>> >>>> I'm working with rbd mapping. I figured out that the block device size >>>> of the rbd device is not update while the device is mounted. Here my >>>> tests: >>>> >>> >>> iirc this is not something RBD specific, but since the device is in use >>> it >>> can't be re-read by the kernel. >>> >>> So when you unmount it the kernel can re-read the header and resize the >>> device. >>> >>> Wido >>> >>>> 1. Pick up a device and check his size >>>> >>>> # rbd ls >>>> size >>>> >>>> # rbd info test >>>> rbd image 'test': >>>> size 10000 MB in 2500 objects >>>> order 22 (4096 KB objects) >>>> block_name_prefix: rb.0.6 >>>> parent: (pool -1) >>>> >>>> 2. Map the device >>>> >>>> # rbd map --secret /etc/ceph/secret test >>>> # rbd showmapped >>>> id pool image snap device >>>> 1 rbd test - /dev/rbd1 >>>> >>>> 3. Put a fs on it and check the block device size >>>> >>>> # mkfs.ext4 /dev/rdb1 >>>> ... >>>> ... >>>> >>>> # fdisk -l /dev/rbd1 >>>> >>>> Disk /dev/rbd1: 10.5 GB, 10485760000 bytes >>>> >>>> 4. Mount it >>>> >>>> # mount /dev/rbd1 /mnt >>>> # df -h >>>> /dev/rbd1 9.8G 277M 9.0G 3% /mnt >>>> >>>> >>>> 5. Change the image size >>>> >>>> # rbd resize --size 11000 test >>>> Resizing image: 100% complete...done. >>>> >>>> # rbd info test >>>> rbd image 'test': >>>> size 11000 MB in 2750 objects >>>> order 22 (4096 KB objects) >>>> block_name_prefix: rb.0.6 >>>> parent: (pool -1) >>>> >>>> At this point of time, if you perform the fdisk -l /dev/rbd1, the >>>> block device size will remain the same. >>>> >>>> 6. Unmount the device: >>>> >>>> # umount /media >>>> >>>> # fdisk -l /dev/rbd1 >>>> Disk /dev/rbd1: 11.5 GB, 11534336000 bytes >>>> >>>> Unmounting the directory did update the block device size. >>>> >>>> Of course you can do something really fast like: >>>> >>>> # umount /media && mount /dev/rbd1 /media >>>> >>>> That will work, it's a valid solution as long as there is no opened >>>> file. I won't use this trick in production... >>>> >>>> I also tried to "mount -o remount" and it didn't work. >>>> >>>> 7. Resize the fs (this can be performed while the fs is mounted): >>>> >>>> # e2fsck -f /dev/rbd1 >>>> e2fsck 1.42 (29-Nov-2011) >>>> Pass 1: Checking inodes, blocks, and sizes >>>> Pass 2: Checking directory structure >>>> Pass 3: Checking directory connectivity >>>> Pass 4: Checking reference counts >>>> Pass 5: Checking group summary information >>>> /dev/rbd1: 11/644640 files (0.0% non-contiguous), 77173/2560000 blocks >>>> >>>> # resize2fs /dev/rbd1 >>>> resize2fs 1.42 (29-Nov-2011) >>>> Resizing the filesystem on /dev/rbd1 to 2816000 (4k) blocks. >>>> The filesystem on /dev/rbd1 is now 2816000 blocks long. >>>> >>>> >>>> Did I miss something? >>>> Is this feature coming? >>>> >>>> Thank you in advance :) >>>> -- >>>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >>>> the body of a message to majordomo@vger.kernel.org >>>> More majordomo info at http://vger.kernel.org/majordomo-info.html >>>> >>> > -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Ceph doesn't update the block device size while a rbd image is mounted 2012-07-19 15:50 ` Sébastien Han @ 2012-07-19 17:49 ` Calvin Morrow 2012-07-19 19:44 ` Sébastien Han 1 sibling, 0 replies; 12+ messages in thread From: Calvin Morrow @ 2012-07-19 17:49 UTC (permalink / raw) To: Sébastien Han; +Cc: ceph-devel I've had a little more luck using cfdisk than vanilla fdisk when it comes to detecting changes. You might try running partprobe and then cfdisk and seeing if you get anything different. Calvin On Thu, Jul 19, 2012 at 9:50 AM, Sébastien Han <han.sebastien@gmail.com> wrote: > Hum ok, I see. Thanks! > > But if you have any clue to force the kernel to re-read without > unmont/mounting :) > > On Thu, Jul 19, 2012 at 5:47 PM, Wido den Hollander <wido@widodh.nl> wrote: >> >> >> On 19-07-12 17:26, Sébastien Han wrote: >>> >>> Ok I got your point seems logic, but why is this possible with LVM for >>> example? >>> >>> You can easily do this with LVM without un-mounting the device. >>> >> >> >> LVM runs through the device mapper and are not regular block devices. >> >> If you resize the disk underneath LVM you won't see an increased VG or PV >> size unless you change the availability of the VG to unavailable and back to >> available again. >> >> I'm not a 100% sure what the exact root cause is, but the kernel won't read >> the new size of a block device as long as it is in use. >> >> Wido >> >> >> >>> Cheers. >>> >>> >>> On Thu, Jul 19, 2012 at 5:15 PM, Wido den Hollander <wido@widodh.nl> >>> wrote: >>>> >>>> Hi, >>>> >>>> >>>> On 19-07-12 16:55, Sébastien Han wrote: >>>>> >>>>> >>>>> Hi Cephers! >>>>> >>>>> I'm working with rbd mapping. I figured out that the block device size >>>>> of the rbd device is not update while the device is mounted. Here my >>>>> tests: >>>>> >>>> >>>> iirc this is not something RBD specific, but since the device is in use >>>> it >>>> can't be re-read by the kernel. >>>> >>>> So when you unmount it the kernel can re-read the header and resize the >>>> device. >>>> >>>> Wido >>>> >>>>> 1. Pick up a device and check his size >>>>> >>>>> # rbd ls >>>>> size >>>>> >>>>> # rbd info test >>>>> rbd image 'test': >>>>> size 10000 MB in 2500 objects >>>>> order 22 (4096 KB objects) >>>>> block_name_prefix: rb.0.6 >>>>> parent: (pool -1) >>>>> >>>>> 2. Map the device >>>>> >>>>> # rbd map --secret /etc/ceph/secret test >>>>> # rbd showmapped >>>>> id pool image snap device >>>>> 1 rbd test - /dev/rbd1 >>>>> >>>>> 3. Put a fs on it and check the block device size >>>>> >>>>> # mkfs.ext4 /dev/rdb1 >>>>> ... >>>>> ... >>>>> >>>>> # fdisk -l /dev/rbd1 >>>>> >>>>> Disk /dev/rbd1: 10.5 GB, 10485760000 bytes >>>>> >>>>> 4. Mount it >>>>> >>>>> # mount /dev/rbd1 /mnt >>>>> # df -h >>>>> /dev/rbd1 9.8G 277M 9.0G 3% /mnt >>>>> >>>>> >>>>> 5. Change the image size >>>>> >>>>> # rbd resize --size 11000 test >>>>> Resizing image: 100% complete...done. >>>>> >>>>> # rbd info test >>>>> rbd image 'test': >>>>> size 11000 MB in 2750 objects >>>>> order 22 (4096 KB objects) >>>>> block_name_prefix: rb.0.6 >>>>> parent: (pool -1) >>>>> >>>>> At this point of time, if you perform the fdisk -l /dev/rbd1, the >>>>> block device size will remain the same. >>>>> >>>>> 6. Unmount the device: >>>>> >>>>> # umount /media >>>>> >>>>> # fdisk -l /dev/rbd1 >>>>> Disk /dev/rbd1: 11.5 GB, 11534336000 bytes >>>>> >>>>> Unmounting the directory did update the block device size. >>>>> >>>>> Of course you can do something really fast like: >>>>> >>>>> # umount /media && mount /dev/rbd1 /media >>>>> >>>>> That will work, it's a valid solution as long as there is no opened >>>>> file. I won't use this trick in production... >>>>> >>>>> I also tried to "mount -o remount" and it didn't work. >>>>> >>>>> 7. Resize the fs (this can be performed while the fs is mounted): >>>>> >>>>> # e2fsck -f /dev/rbd1 >>>>> e2fsck 1.42 (29-Nov-2011) >>>>> Pass 1: Checking inodes, blocks, and sizes >>>>> Pass 2: Checking directory structure >>>>> Pass 3: Checking directory connectivity >>>>> Pass 4: Checking reference counts >>>>> Pass 5: Checking group summary information >>>>> /dev/rbd1: 11/644640 files (0.0% non-contiguous), 77173/2560000 blocks >>>>> >>>>> # resize2fs /dev/rbd1 >>>>> resize2fs 1.42 (29-Nov-2011) >>>>> Resizing the filesystem on /dev/rbd1 to 2816000 (4k) blocks. >>>>> The filesystem on /dev/rbd1 is now 2816000 blocks long. >>>>> >>>>> >>>>> Did I miss something? >>>>> Is this feature coming? >>>>> >>>>> Thank you in advance :) >>>>> -- >>>>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >>>>> the body of a message to majordomo@vger.kernel.org >>>>> More majordomo info at http://vger.kernel.org/majordomo-info.html >>>>> >>>> >> > -- > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Ceph doesn't update the block device size while a rbd image is mounted 2012-07-19 15:50 ` Sébastien Han 2012-07-19 17:49 ` Calvin Morrow @ 2012-07-19 19:44 ` Sébastien Han 2012-07-19 19:54 ` Calvin Morrow 2012-07-19 20:35 ` Andreas Kurz 1 sibling, 2 replies; 12+ messages in thread From: Sébastien Han @ 2012-07-19 19:44 UTC (permalink / raw) To: Wido den Hollander; +Cc: ceph-devel With LVM, you can re-scan the scsi bus to extend a physical drive and then run a pvextend. @Calvin: I tried your solution # partprobe /dev/rbd1 Unfortunatly nothing changed. Did you make it working? Cheers! On Thu, Jul 19, 2012 at 5:50 PM, Sébastien Han <han.sebastien@gmail.com> wrote: > Hum ok, I see. Thanks! > > But if you have any clue to force the kernel to re-read without > unmont/mounting :) > > On Thu, Jul 19, 2012 at 5:47 PM, Wido den Hollander <wido@widodh.nl> wrote: >> >> >> On 19-07-12 17:26, Sébastien Han wrote: >>> >>> Ok I got your point seems logic, but why is this possible with LVM for >>> example? >>> >>> You can easily do this with LVM without un-mounting the device. >>> >> >> >> LVM runs through the device mapper and are not regular block devices. >> >> If you resize the disk underneath LVM you won't see an increased VG or PV >> size unless you change the availability of the VG to unavailable and back to >> available again. >> >> I'm not a 100% sure what the exact root cause is, but the kernel won't read >> the new size of a block device as long as it is in use. >> >> Wido >> >> >> >>> Cheers. >>> >>> >>> On Thu, Jul 19, 2012 at 5:15 PM, Wido den Hollander <wido@widodh.nl> >>> wrote: >>>> >>>> Hi, >>>> >>>> >>>> On 19-07-12 16:55, Sébastien Han wrote: >>>>> >>>>> >>>>> Hi Cephers! >>>>> >>>>> I'm working with rbd mapping. I figured out that the block device size >>>>> of the rbd device is not update while the device is mounted. Here my >>>>> tests: >>>>> >>>> >>>> iirc this is not something RBD specific, but since the device is in use >>>> it >>>> can't be re-read by the kernel. >>>> >>>> So when you unmount it the kernel can re-read the header and resize the >>>> device. >>>> >>>> Wido >>>> >>>>> 1. Pick up a device and check his size >>>>> >>>>> # rbd ls >>>>> size >>>>> >>>>> # rbd info test >>>>> rbd image 'test': >>>>> size 10000 MB in 2500 objects >>>>> order 22 (4096 KB objects) >>>>> block_name_prefix: rb.0.6 >>>>> parent: (pool -1) >>>>> >>>>> 2. Map the device >>>>> >>>>> # rbd map --secret /etc/ceph/secret test >>>>> # rbd showmapped >>>>> id pool image snap device >>>>> 1 rbd test - /dev/rbd1 >>>>> >>>>> 3. Put a fs on it and check the block device size >>>>> >>>>> # mkfs.ext4 /dev/rdb1 >>>>> ... >>>>> ... >>>>> >>>>> # fdisk -l /dev/rbd1 >>>>> >>>>> Disk /dev/rbd1: 10.5 GB, 10485760000 bytes >>>>> >>>>> 4. Mount it >>>>> >>>>> # mount /dev/rbd1 /mnt >>>>> # df -h >>>>> /dev/rbd1 9.8G 277M 9.0G 3% /mnt >>>>> >>>>> >>>>> 5. Change the image size >>>>> >>>>> # rbd resize --size 11000 test >>>>> Resizing image: 100% complete...done. >>>>> >>>>> # rbd info test >>>>> rbd image 'test': >>>>> size 11000 MB in 2750 objects >>>>> order 22 (4096 KB objects) >>>>> block_name_prefix: rb.0.6 >>>>> parent: (pool -1) >>>>> >>>>> At this point of time, if you perform the fdisk -l /dev/rbd1, the >>>>> block device size will remain the same. >>>>> >>>>> 6. Unmount the device: >>>>> >>>>> # umount /media >>>>> >>>>> # fdisk -l /dev/rbd1 >>>>> Disk /dev/rbd1: 11.5 GB, 11534336000 bytes >>>>> >>>>> Unmounting the directory did update the block device size. >>>>> >>>>> Of course you can do something really fast like: >>>>> >>>>> # umount /media && mount /dev/rbd1 /media >>>>> >>>>> That will work, it's a valid solution as long as there is no opened >>>>> file. I won't use this trick in production... >>>>> >>>>> I also tried to "mount -o remount" and it didn't work. >>>>> >>>>> 7. Resize the fs (this can be performed while the fs is mounted): >>>>> >>>>> # e2fsck -f /dev/rbd1 >>>>> e2fsck 1.42 (29-Nov-2011) >>>>> Pass 1: Checking inodes, blocks, and sizes >>>>> Pass 2: Checking directory structure >>>>> Pass 3: Checking directory connectivity >>>>> Pass 4: Checking reference counts >>>>> Pass 5: Checking group summary information >>>>> /dev/rbd1: 11/644640 files (0.0% non-contiguous), 77173/2560000 blocks >>>>> >>>>> # resize2fs /dev/rbd1 >>>>> resize2fs 1.42 (29-Nov-2011) >>>>> Resizing the filesystem on /dev/rbd1 to 2816000 (4k) blocks. >>>>> The filesystem on /dev/rbd1 is now 2816000 blocks long. >>>>> >>>>> >>>>> Did I miss something? >>>>> Is this feature coming? >>>>> >>>>> Thank you in advance :) >>>>> -- >>>>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >>>>> the body of a message to majordomo@vger.kernel.org >>>>> More majordomo info at http://vger.kernel.org/majordomo-info.html >>>>> >>>> >> -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Ceph doesn't update the block device size while a rbd image is mounted 2012-07-19 19:44 ` Sébastien Han @ 2012-07-19 19:54 ` Calvin Morrow 2012-07-19 20:35 ` Andreas Kurz 1 sibling, 0 replies; 12+ messages in thread From: Calvin Morrow @ 2012-07-19 19:54 UTC (permalink / raw) To: Sébastien Han; +Cc: Wido den Hollander, ceph-devel I haven't tried resizing an rbd yet, but I was changing partitions on a non-ceph two-node cluster with shared storage yesterday while certain partitions were in use (partitions 1,2,5 were mounted, deleting partition ids 6+, adding new ones) and fdisk wasn't re-reading disk changes. Partprobe followed by cfdisk seemed to update the kernel with the on-disk changes. I realize its not exactly what you are doing, but I figured it might be close enough to be worth a shot. cfdisk seems to make a system call to refresh disk information that vanilla fdisk doesn't. Calvin On Thu, Jul 19, 2012 at 1:44 PM, Sébastien Han <han.sebastien@gmail.com> wrote: > With LVM, you can re-scan the scsi bus to extend a physical drive and > then run a pvextend. > > @Calvin: I tried your solution > > # partprobe /dev/rbd1 > > Unfortunatly nothing changed. > > Did you make it working? > > Cheers! > > > On Thu, Jul 19, 2012 at 5:50 PM, Sébastien Han <han.sebastien@gmail.com> wrote: >> Hum ok, I see. Thanks! >> >> But if you have any clue to force the kernel to re-read without >> unmont/mounting :) >> >> On Thu, Jul 19, 2012 at 5:47 PM, Wido den Hollander <wido@widodh.nl> wrote: >>> >>> >>> On 19-07-12 17:26, Sébastien Han wrote: >>>> >>>> Ok I got your point seems logic, but why is this possible with LVM for >>>> example? >>>> >>>> You can easily do this with LVM without un-mounting the device. >>>> >>> >>> >>> LVM runs through the device mapper and are not regular block devices. >>> >>> If you resize the disk underneath LVM you won't see an increased VG or PV >>> size unless you change the availability of the VG to unavailable and back to >>> available again. >>> >>> I'm not a 100% sure what the exact root cause is, but the kernel won't read >>> the new size of a block device as long as it is in use. >>> >>> Wido >>> >>> >>> >>>> Cheers. >>>> >>>> >>>> On Thu, Jul 19, 2012 at 5:15 PM, Wido den Hollander <wido@widodh.nl> >>>> wrote: >>>>> >>>>> Hi, >>>>> >>>>> >>>>> On 19-07-12 16:55, Sébastien Han wrote: >>>>>> >>>>>> >>>>>> Hi Cephers! >>>>>> >>>>>> I'm working with rbd mapping. I figured out that the block device size >>>>>> of the rbd device is not update while the device is mounted. Here my >>>>>> tests: >>>>>> >>>>> >>>>> iirc this is not something RBD specific, but since the device is in use >>>>> it >>>>> can't be re-read by the kernel. >>>>> >>>>> So when you unmount it the kernel can re-read the header and resize the >>>>> device. >>>>> >>>>> Wido >>>>> >>>>>> 1. Pick up a device and check his size >>>>>> >>>>>> # rbd ls >>>>>> size >>>>>> >>>>>> # rbd info test >>>>>> rbd image 'test': >>>>>> size 10000 MB in 2500 objects >>>>>> order 22 (4096 KB objects) >>>>>> block_name_prefix: rb.0.6 >>>>>> parent: (pool -1) >>>>>> >>>>>> 2. Map the device >>>>>> >>>>>> # rbd map --secret /etc/ceph/secret test >>>>>> # rbd showmapped >>>>>> id pool image snap device >>>>>> 1 rbd test - /dev/rbd1 >>>>>> >>>>>> 3. Put a fs on it and check the block device size >>>>>> >>>>>> # mkfs.ext4 /dev/rdb1 >>>>>> ... >>>>>> ... >>>>>> >>>>>> # fdisk -l /dev/rbd1 >>>>>> >>>>>> Disk /dev/rbd1: 10.5 GB, 10485760000 bytes >>>>>> >>>>>> 4. Mount it >>>>>> >>>>>> # mount /dev/rbd1 /mnt >>>>>> # df -h >>>>>> /dev/rbd1 9.8G 277M 9.0G 3% /mnt >>>>>> >>>>>> >>>>>> 5. Change the image size >>>>>> >>>>>> # rbd resize --size 11000 test >>>>>> Resizing image: 100% complete...done. >>>>>> >>>>>> # rbd info test >>>>>> rbd image 'test': >>>>>> size 11000 MB in 2750 objects >>>>>> order 22 (4096 KB objects) >>>>>> block_name_prefix: rb.0.6 >>>>>> parent: (pool -1) >>>>>> >>>>>> At this point of time, if you perform the fdisk -l /dev/rbd1, the >>>>>> block device size will remain the same. >>>>>> >>>>>> 6. Unmount the device: >>>>>> >>>>>> # umount /media >>>>>> >>>>>> # fdisk -l /dev/rbd1 >>>>>> Disk /dev/rbd1: 11.5 GB, 11534336000 bytes >>>>>> >>>>>> Unmounting the directory did update the block device size. >>>>>> >>>>>> Of course you can do something really fast like: >>>>>> >>>>>> # umount /media && mount /dev/rbd1 /media >>>>>> >>>>>> That will work, it's a valid solution as long as there is no opened >>>>>> file. I won't use this trick in production... >>>>>> >>>>>> I also tried to "mount -o remount" and it didn't work. >>>>>> >>>>>> 7. Resize the fs (this can be performed while the fs is mounted): >>>>>> >>>>>> # e2fsck -f /dev/rbd1 >>>>>> e2fsck 1.42 (29-Nov-2011) >>>>>> Pass 1: Checking inodes, blocks, and sizes >>>>>> Pass 2: Checking directory structure >>>>>> Pass 3: Checking directory connectivity >>>>>> Pass 4: Checking reference counts >>>>>> Pass 5: Checking group summary information >>>>>> /dev/rbd1: 11/644640 files (0.0% non-contiguous), 77173/2560000 blocks >>>>>> >>>>>> # resize2fs /dev/rbd1 >>>>>> resize2fs 1.42 (29-Nov-2011) >>>>>> Resizing the filesystem on /dev/rbd1 to 2816000 (4k) blocks. >>>>>> The filesystem on /dev/rbd1 is now 2816000 blocks long. >>>>>> >>>>>> >>>>>> Did I miss something? >>>>>> Is this feature coming? >>>>>> >>>>>> Thank you in advance :) >>>>>> -- >>>>>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >>>>>> the body of a message to majordomo@vger.kernel.org >>>>>> More majordomo info at http://vger.kernel.org/majordomo-info.html >>>>>> >>>>> >>> > -- > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Ceph doesn't update the block device size while a rbd image is mounted 2012-07-19 19:44 ` Sébastien Han 2012-07-19 19:54 ` Calvin Morrow @ 2012-07-19 20:35 ` Andreas Kurz 2012-08-09 19:32 ` Sébastien Han 1 sibling, 1 reply; 12+ messages in thread From: Andreas Kurz @ 2012-07-19 20:35 UTC (permalink / raw) To: ceph-devel; +Cc: han.sebastien [-- Attachment #1: Type: text/plain, Size: 5151 bytes --] On 07/19/2012 09:44 PM, Sébastien Han wrote: > With LVM, you can re-scan the scsi bus to extend a physical drive and > then run a pvextend. > > @Calvin: I tried your solution > > # partprobe /dev/rbd1 Did you try blockdev? # blockdev --rereadpt /dev/rbd1 Regards, Andreas > > Unfortunatly nothing changed. > > Did you make it working? > > Cheers! > > > On Thu, Jul 19, 2012 at 5:50 PM, Sébastien Han <han.sebastien@gmail.com> wrote: >> Hum ok, I see. Thanks! >> >> But if you have any clue to force the kernel to re-read without >> unmont/mounting :) >> >> On Thu, Jul 19, 2012 at 5:47 PM, Wido den Hollander <wido@widodh.nl> wrote: >>> >>> >>> On 19-07-12 17:26, Sébastien Han wrote: >>>> >>>> Ok I got your point seems logic, but why is this possible with LVM for >>>> example? >>>> >>>> You can easily do this with LVM without un-mounting the device. >>>> >>> >>> >>> LVM runs through the device mapper and are not regular block devices. >>> >>> If you resize the disk underneath LVM you won't see an increased VG or PV >>> size unless you change the availability of the VG to unavailable and back to >>> available again. >>> >>> I'm not a 100% sure what the exact root cause is, but the kernel won't read >>> the new size of a block device as long as it is in use. >>> >>> Wido >>> >>> >>> >>>> Cheers. >>>> >>>> >>>> On Thu, Jul 19, 2012 at 5:15 PM, Wido den Hollander <wido@widodh.nl> >>>> wrote: >>>>> >>>>> Hi, >>>>> >>>>> >>>>> On 19-07-12 16:55, Sébastien Han wrote: >>>>>> >>>>>> >>>>>> Hi Cephers! >>>>>> >>>>>> I'm working with rbd mapping. I figured out that the block device size >>>>>> of the rbd device is not update while the device is mounted. Here my >>>>>> tests: >>>>>> >>>>> >>>>> iirc this is not something RBD specific, but since the device is in use >>>>> it >>>>> can't be re-read by the kernel. >>>>> >>>>> So when you unmount it the kernel can re-read the header and resize the >>>>> device. >>>>> >>>>> Wido >>>>> >>>>>> 1. Pick up a device and check his size >>>>>> >>>>>> # rbd ls >>>>>> size >>>>>> >>>>>> # rbd info test >>>>>> rbd image 'test': >>>>>> size 10000 MB in 2500 objects >>>>>> order 22 (4096 KB objects) >>>>>> block_name_prefix: rb.0.6 >>>>>> parent: (pool -1) >>>>>> >>>>>> 2. Map the device >>>>>> >>>>>> # rbd map --secret /etc/ceph/secret test >>>>>> # rbd showmapped >>>>>> id pool image snap device >>>>>> 1 rbd test - /dev/rbd1 >>>>>> >>>>>> 3. Put a fs on it and check the block device size >>>>>> >>>>>> # mkfs.ext4 /dev/rdb1 >>>>>> ... >>>>>> ... >>>>>> >>>>>> # fdisk -l /dev/rbd1 >>>>>> >>>>>> Disk /dev/rbd1: 10.5 GB, 10485760000 bytes >>>>>> >>>>>> 4. Mount it >>>>>> >>>>>> # mount /dev/rbd1 /mnt >>>>>> # df -h >>>>>> /dev/rbd1 9.8G 277M 9.0G 3% /mnt >>>>>> >>>>>> >>>>>> 5. Change the image size >>>>>> >>>>>> # rbd resize --size 11000 test >>>>>> Resizing image: 100% complete...done. >>>>>> >>>>>> # rbd info test >>>>>> rbd image 'test': >>>>>> size 11000 MB in 2750 objects >>>>>> order 22 (4096 KB objects) >>>>>> block_name_prefix: rb.0.6 >>>>>> parent: (pool -1) >>>>>> >>>>>> At this point of time, if you perform the fdisk -l /dev/rbd1, the >>>>>> block device size will remain the same. >>>>>> >>>>>> 6. Unmount the device: >>>>>> >>>>>> # umount /media >>>>>> >>>>>> # fdisk -l /dev/rbd1 >>>>>> Disk /dev/rbd1: 11.5 GB, 11534336000 bytes >>>>>> >>>>>> Unmounting the directory did update the block device size. >>>>>> >>>>>> Of course you can do something really fast like: >>>>>> >>>>>> # umount /media && mount /dev/rbd1 /media >>>>>> >>>>>> That will work, it's a valid solution as long as there is no opened >>>>>> file. I won't use this trick in production... >>>>>> >>>>>> I also tried to "mount -o remount" and it didn't work. >>>>>> >>>>>> 7. Resize the fs (this can be performed while the fs is mounted): >>>>>> >>>>>> # e2fsck -f /dev/rbd1 >>>>>> e2fsck 1.42 (29-Nov-2011) >>>>>> Pass 1: Checking inodes, blocks, and sizes >>>>>> Pass 2: Checking directory structure >>>>>> Pass 3: Checking directory connectivity >>>>>> Pass 4: Checking reference counts >>>>>> Pass 5: Checking group summary information >>>>>> /dev/rbd1: 11/644640 files (0.0% non-contiguous), 77173/2560000 blocks >>>>>> >>>>>> # resize2fs /dev/rbd1 >>>>>> resize2fs 1.42 (29-Nov-2011) >>>>>> Resizing the filesystem on /dev/rbd1 to 2816000 (4k) blocks. >>>>>> The filesystem on /dev/rbd1 is now 2816000 blocks long. >>>>>> >>>>>> >>>>>> Did I miss something? >>>>>> Is this feature coming? >>>>>> >>>>>> Thank you in advance :) >>>>>> -- >>>>>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >>>>>> the body of a message to majordomo@vger.kernel.org >>>>>> More majordomo info at http://vger.kernel.org/majordomo-info.html >>>>>> >>>>> >>> > -- > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 222 bytes --] ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Ceph doesn't update the block device size while a rbd image is mounted 2012-07-19 20:35 ` Andreas Kurz @ 2012-08-09 19:32 ` Sébastien Han 0 siblings, 0 replies; 12+ messages in thread From: Sébastien Han @ 2012-08-09 19:32 UTC (permalink / raw) To: Andreas Kurz; +Cc: ceph-devel Hi Andreas, Sorry the long response, I also tried blockdev but without success :/ Cheers! On Thu, Jul 19, 2012 at 10:35 PM, Andreas Kurz <andreas@hastexo.com> wrote: > > On 07/19/2012 09:44 PM, Sébastien Han wrote: > > With LVM, you can re-scan the scsi bus to extend a physical drive and > > then run a pvextend. > > > > @Calvin: I tried your solution > > > > # partprobe /dev/rbd1 > > Did you try blockdev? > > # blockdev --rereadpt /dev/rbd1 > > Regards, > Andreas > > > > > Unfortunatly nothing changed. > > > > Did you make it working? > > > > Cheers! > > > > > > On Thu, Jul 19, 2012 at 5:50 PM, Sébastien Han <han.sebastien@gmail.com> wrote: > >> Hum ok, I see. Thanks! > >> > >> But if you have any clue to force the kernel to re-read without > >> unmont/mounting :) > >> > >> On Thu, Jul 19, 2012 at 5:47 PM, Wido den Hollander <wido@widodh.nl> wrote: > >>> > >>> > >>> On 19-07-12 17:26, Sébastien Han wrote: > >>>> > >>>> Ok I got your point seems logic, but why is this possible with LVM for > >>>> example? > >>>> > >>>> You can easily do this with LVM without un-mounting the device. > >>>> > >>> > >>> > >>> LVM runs through the device mapper and are not regular block devices. > >>> > >>> If you resize the disk underneath LVM you won't see an increased VG or PV > >>> size unless you change the availability of the VG to unavailable and back to > >>> available again. > >>> > >>> I'm not a 100% sure what the exact root cause is, but the kernel won't read > >>> the new size of a block device as long as it is in use. > >>> > >>> Wido > >>> > >>> > >>> > >>>> Cheers. > >>>> > >>>> > >>>> On Thu, Jul 19, 2012 at 5:15 PM, Wido den Hollander <wido@widodh.nl> > >>>> wrote: > >>>>> > >>>>> Hi, > >>>>> > >>>>> > >>>>> On 19-07-12 16:55, Sébastien Han wrote: > >>>>>> > >>>>>> > >>>>>> Hi Cephers! > >>>>>> > >>>>>> I'm working with rbd mapping. I figured out that the block device size > >>>>>> of the rbd device is not update while the device is mounted. Here my > >>>>>> tests: > >>>>>> > >>>>> > >>>>> iirc this is not something RBD specific, but since the device is in use > >>>>> it > >>>>> can't be re-read by the kernel. > >>>>> > >>>>> So when you unmount it the kernel can re-read the header and resize the > >>>>> device. > >>>>> > >>>>> Wido > >>>>> > >>>>>> 1. Pick up a device and check his size > >>>>>> > >>>>>> # rbd ls > >>>>>> size > >>>>>> > >>>>>> # rbd info test > >>>>>> rbd image 'test': > >>>>>> size 10000 MB in 2500 objects > >>>>>> order 22 (4096 KB objects) > >>>>>> block_name_prefix: rb.0.6 > >>>>>> parent: (pool -1) > >>>>>> > >>>>>> 2. Map the device > >>>>>> > >>>>>> # rbd map --secret /etc/ceph/secret test > >>>>>> # rbd showmapped > >>>>>> id pool image snap device > >>>>>> 1 rbd test - /dev/rbd1 > >>>>>> > >>>>>> 3. Put a fs on it and check the block device size > >>>>>> > >>>>>> # mkfs.ext4 /dev/rdb1 > >>>>>> ... > >>>>>> ... > >>>>>> > >>>>>> # fdisk -l /dev/rbd1 > >>>>>> > >>>>>> Disk /dev/rbd1: 10.5 GB, 10485760000 bytes > >>>>>> > >>>>>> 4. Mount it > >>>>>> > >>>>>> # mount /dev/rbd1 /mnt > >>>>>> # df -h > >>>>>> /dev/rbd1 9.8G 277M 9.0G 3% /mnt > >>>>>> > >>>>>> > >>>>>> 5. Change the image size > >>>>>> > >>>>>> # rbd resize --size 11000 test > >>>>>> Resizing image: 100% complete...done. > >>>>>> > >>>>>> # rbd info test > >>>>>> rbd image 'test': > >>>>>> size 11000 MB in 2750 objects > >>>>>> order 22 (4096 KB objects) > >>>>>> block_name_prefix: rb.0.6 > >>>>>> parent: (pool -1) > >>>>>> > >>>>>> At this point of time, if you perform the fdisk -l /dev/rbd1, the > >>>>>> block device size will remain the same. > >>>>>> > >>>>>> 6. Unmount the device: > >>>>>> > >>>>>> # umount /media > >>>>>> > >>>>>> # fdisk -l /dev/rbd1 > >>>>>> Disk /dev/rbd1: 11.5 GB, 11534336000 bytes > >>>>>> > >>>>>> Unmounting the directory did update the block device size. > >>>>>> > >>>>>> Of course you can do something really fast like: > >>>>>> > >>>>>> # umount /media && mount /dev/rbd1 /media > >>>>>> > >>>>>> That will work, it's a valid solution as long as there is no opened > >>>>>> file. I won't use this trick in production... > >>>>>> > >>>>>> I also tried to "mount -o remount" and it didn't work. > >>>>>> > >>>>>> 7. Resize the fs (this can be performed while the fs is mounted): > >>>>>> > >>>>>> # e2fsck -f /dev/rbd1 > >>>>>> e2fsck 1.42 (29-Nov-2011) > >>>>>> Pass 1: Checking inodes, blocks, and sizes > >>>>>> Pass 2: Checking directory structure > >>>>>> Pass 3: Checking directory connectivity > >>>>>> Pass 4: Checking reference counts > >>>>>> Pass 5: Checking group summary information > >>>>>> /dev/rbd1: 11/644640 files (0.0% non-contiguous), 77173/2560000 blocks > >>>>>> > >>>>>> # resize2fs /dev/rbd1 > >>>>>> resize2fs 1.42 (29-Nov-2011) > >>>>>> Resizing the filesystem on /dev/rbd1 to 2816000 (4k) blocks. > >>>>>> The filesystem on /dev/rbd1 is now 2816000 blocks long. > >>>>>> > >>>>>> > >>>>>> Did I miss something? > >>>>>> Is this feature coming? > >>>>>> > >>>>>> Thank you in advance :) > >>>>>> -- > >>>>>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > >>>>>> the body of a message to majordomo@vger.kernel.org > >>>>>> More majordomo info at http://vger.kernel.org/majordomo-info.html > >>>>>> > >>>>> > >>> > > -- > > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > > the body of a message to majordomo@vger.kernel.org > > More majordomo info at http://vger.kernel.org/majordomo-info.html > > > > -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 12+ messages in thread
end of thread, other threads:[~2012-08-09 19:32 UTC | newest]
Thread overview: 12+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
[not found] <CAOLwVUkWUrjQTxQw5-umpWnY6G-AjReCwLTyQCOdCyNT175KgA@mail.gmail.com>
2012-07-19 14:55 ` Ceph doesn't update the block device size while a rbd image is mounted Sébastien Han
2012-07-19 15:15 ` Wido den Hollander
2012-07-19 15:26 ` Sébastien Han
2012-07-19 15:38 ` Tommi Virtanen
2012-07-19 15:39 ` Tommi Virtanen
2012-07-19 15:47 ` Wido den Hollander
2012-07-19 15:50 ` Sébastien Han
2012-07-19 17:49 ` Calvin Morrow
2012-07-19 19:44 ` Sébastien Han
2012-07-19 19:54 ` Calvin Morrow
2012-07-19 20:35 ` Andreas Kurz
2012-08-09 19:32 ` Sébastien Han
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.