* rbd STDIN import does not work / wip-rbd-export-stdout @ 2012-11-26 13:20 Stefan Priebe - Profihost AG 2012-11-26 13:29 ` Oliver Francke 0 siblings, 1 reply; 15+ messages in thread From: Stefan Priebe - Profihost AG @ 2012-11-26 13:20 UTC (permalink / raw) To: ceph-devel@vger.kernel.org; +Cc: Dan Mick Hello list, i know branch wip-rbd-export-stdout is work in progress but it is more than useful ;-) When i try to import an image i get: # gzip -dc vm-101-disk-1.img.gz | rbd import --format=2 --size=42949672960 - kvmpool1/vm-101-disk-1 rbd: error reading file: (29) Illegal seek Importing image: 0% complete...failed. rbd: import failed: (29) Illegal seek Anything i've tried wrong? Greets, Stefan ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: rbd STDIN import does not work / wip-rbd-export-stdout 2012-11-26 13:20 rbd STDIN import does not work / wip-rbd-export-stdout Stefan Priebe - Profihost AG @ 2012-11-26 13:29 ` Oliver Francke 2012-11-26 13:35 ` Stefan Priebe - Profihost AG 0 siblings, 1 reply; 15+ messages in thread From: Oliver Francke @ 2012-11-26 13:29 UTC (permalink / raw) To: Stefan Priebe - Profihost AG; +Cc: ceph-devel@vger.kernel.org, Dan Mick Well... On 11/26/2012 02:20 PM, Stefan Priebe - Profihost AG wrote: > Hello list, > > i know branch wip-rbd-export-stdout is work in progress but it is more > than useful ;-) > > When i try to import an image i get: > > # gzip -dc vm-101-disk-1.img.gz | rbd import --format=2 > --size=42949672960 - kvmpool1/vm-101-disk-1 > rbd: error reading file: (29) Illegal seek > Importing image: 0% complete...failed. > rbd: import failed: (29) Illegal seek > > Anything i've tried wrong? I would assume, that size is already in MiB? Seems to be a slightly too big value... Not tried myself, though... Oliver. > > Greets, > Stefan > -- > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- Oliver Francke filoo GmbH Moltkestraße 25a 33330 Gütersloh HRB4355 AG Gütersloh Geschäftsführer: S.Grewing | J.Rehpöhler | C.Kunz Folgen Sie uns auf Twitter: http://twitter.com/filoogmbh -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: rbd STDIN import does not work / wip-rbd-export-stdout 2012-11-26 13:29 ` Oliver Francke @ 2012-11-26 13:35 ` Stefan Priebe - Profihost AG 2012-11-26 17:34 ` Dan Mick 0 siblings, 1 reply; 15+ messages in thread From: Stefan Priebe - Profihost AG @ 2012-11-26 13:35 UTC (permalink / raw) To: Oliver Francke; +Cc: ceph-devel@vger.kernel.org, Dan Mick Am 26.11.2012 14:29, schrieb Oliver Francke: > Well... makes no difference: gzip -dc vm-101-disk-1.img.gz | rbd import --format=2 --size=40960 - kvmpool1/vm-101-disk-1 rbd: error reading file: (29) Illegal seek Importing image: 0% complete...failed. rbd: import failed: (29) Illegal seek I think the problem is that ceph seeks in File when it get's zeros (not filled content). But seeking doesn't work on stdin. It has to read from stdin until there are no more zeros. Stefan > On 11/26/2012 02:20 PM, Stefan Priebe - Profihost AG wrote: >> Hello list, >> >> i know branch wip-rbd-export-stdout is work in progress but it is more >> than useful ;-) >> >> When i try to import an image i get: >> >> # gzip -dc vm-101-disk-1.img.gz | rbd import --format=2 >> --size=42949672960 - kvmpool1/vm-101-disk-1 >> rbd: error reading file: (29) Illegal seek >> Importing image: 0% complete...failed. >> rbd: import failed: (29) Illegal seek >> >> Anything i've tried wrong? > > I would assume, that size is already in MiB? Seems to be a slightly too > big value... Not tried myself, though... > > Oliver. > >> >> Greets, >> Stefan >> -- >> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >> the body of a message to majordomo@vger.kernel.org >> More majordomo info at http://vger.kernel.org/majordomo-info.html > > ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: rbd STDIN import does not work / wip-rbd-export-stdout 2012-11-26 13:35 ` Stefan Priebe - Profihost AG @ 2012-11-26 17:34 ` Dan Mick 2012-11-26 19:35 ` Stefan Priebe - Profihost AG 0 siblings, 1 reply; 15+ messages in thread From: Dan Mick @ 2012-11-26 17:34 UTC (permalink / raw) To: Stefan Priebe - Profihost AG; +Cc: Oliver Francke, ceph-devel@vger.kernel.org Yes, it's broken too (there's an issue open on it). I have some test code done over the weekend and will be pushing it after review. Be aware that import/export to stdin/stdout are limited in that they cannot preserve sparseness of the image, but they should both be working soon at least. On Mon, Nov 26, 2012 at 5:35 AM, Stefan Priebe - Profihost AG <s.priebe@profihost.ag> wrote: > Am 26.11.2012 14:29, schrieb Oliver Francke: >> >> Well... > > > makes no difference: > gzip -dc vm-101-disk-1.img.gz | rbd import --format=2 --size=40960 - > kvmpool1/vm-101-disk-1 > > rbd: error reading file: (29) Illegal seek > Importing image: 0% complete...failed. > rbd: import failed: (29) Illegal seek > > I think the problem is that ceph seeks in File when it get's zeros (not > filled content). But seeking doesn't work on stdin. It has to read from > stdin until there are no more zeros. > > Stefan > > >> On 11/26/2012 02:20 PM, Stefan Priebe - Profihost AG wrote: >>> >>> Hello list, >>> >>> i know branch wip-rbd-export-stdout is work in progress but it is more >>> than useful ;-) >>> >>> When i try to import an image i get: >>> >>> # gzip -dc vm-101-disk-1.img.gz | rbd import --format=2 >>> --size=42949672960 - kvmpool1/vm-101-disk-1 >>> rbd: error reading file: (29) Illegal seek >>> Importing image: 0% complete...failed. >>> rbd: import failed: (29) Illegal seek >>> >>> Anything i've tried wrong? >> >> >> I would assume, that size is already in MiB? Seems to be a slightly too >> big value... Not tried myself, though... >> >> Oliver. >> >>> >>> Greets, >>> Stefan >>> -- >>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >>> the body of a message to majordomo@vger.kernel.org >>> More majordomo info at http://vger.kernel.org/majordomo-info.html >> >> >> > ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: rbd STDIN import does not work / wip-rbd-export-stdout 2012-11-26 17:34 ` Dan Mick @ 2012-11-26 19:35 ` Stefan Priebe - Profihost AG 2012-11-26 20:00 ` Dan Mick 0 siblings, 1 reply; 15+ messages in thread From: Stefan Priebe - Profihost AG @ 2012-11-26 19:35 UTC (permalink / raw) To: Dan Mick; +Cc: Oliver Francke, ceph-devel@vger.kernel.org Hi, Export isn't a problem. But import should handle this correctly. Or does it write zeros to the object file But even if it write zeros to the file shouldn't a fstrim fix this? Stefan Am 26.11.2012 um 18:34 schrieb Dan Mick <dan.mick@inktank.com>: > Yes, it's broken too (there's an issue open on it). I have some test > code done over the weekend > and will be pushing it after review. > > Be aware that import/export to stdin/stdout are limited in that they > cannot preserve sparseness > of the image, but they should both be working soon at least. > > On Mon, Nov 26, 2012 at 5:35 AM, Stefan Priebe - Profihost AG > <s.priebe@profihost.ag> wrote: >> Am 26.11.2012 14:29, schrieb Oliver Francke: >>> >>> Well... >> >> >> makes no difference: >> gzip -dc vm-101-disk-1.img.gz | rbd import --format=2 --size=40960 - >> kvmpool1/vm-101-disk-1 >> >> rbd: error reading file: (29) Illegal seek >> Importing image: 0% complete...failed. >> rbd: import failed: (29) Illegal seek >> >> I think the problem is that ceph seeks in File when it get's zeros (not >> filled content). But seeking doesn't work on stdin. It has to read from >> stdin until there are no more zeros. >> >> Stefan >> >> >>> On 11/26/2012 02:20 PM, Stefan Priebe - Profihost AG wrote: >>>> >>>> Hello list, >>>> >>>> i know branch wip-rbd-export-stdout is work in progress but it is more >>>> than useful ;-) >>>> >>>> When i try to import an image i get: >>>> >>>> # gzip -dc vm-101-disk-1.img.gz | rbd import --format=2 >>>> --size=42949672960 - kvmpool1/vm-101-disk-1 >>>> rbd: error reading file: (29) Illegal seek >>>> Importing image: 0% complete...failed. >>>> rbd: import failed: (29) Illegal seek >>>> >>>> Anything i've tried wrong? >>> >>> >>> I would assume, that size is already in MiB? Seems to be a slightly too >>> big value... Not tried myself, though... >>> >>> Oliver. >>> >>>> >>>> Greets, >>>> Stefan >>>> -- >>>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >>>> the body of a message to majordomo@vger.kernel.org >>>> More majordomo info at http://vger.kernel.org/majordomo-info.html > -- > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: rbd STDIN import does not work / wip-rbd-export-stdout 2012-11-26 19:35 ` Stefan Priebe - Profihost AG @ 2012-11-26 20:00 ` Dan Mick 2012-11-27 9:16 ` Stefan Priebe - Profihost AG 0 siblings, 1 reply; 15+ messages in thread From: Dan Mick @ 2012-11-26 20:00 UTC (permalink / raw) To: Stefan Priebe - Profihost AG; +Cc: Oliver Francke, ceph-devel@vger.kernel.org It writes zeros; there's no way for it to know how many zeros are coming. It could make a half-hearted attempt depending on its buffer size and the amount of data the source is willing to buffer. As for fstrim: on the rbd image?....sure, if it's a filesystem, it ought to work (modulo some bugs I've heard about with 32-bit vs. 64-bit offsets in qemu... :) ) On 11/26/2012 11:35 AM, Stefan Priebe - Profihost AG wrote: > Hi, > > Export isn't a problem. But import should handle this correctly. Or does it write zeros to the object file > > But even if it write zeros to the file shouldn't a fstrim fix this? > > Stefan > > Am 26.11.2012 um 18:34 schrieb Dan Mick <dan.mick@inktank.com>: > >> Yes, it's broken too (there's an issue open on it). I have some test >> code done over the weekend >> and will be pushing it after review. >> >> Be aware that import/export to stdin/stdout are limited in that they >> cannot preserve sparseness >> of the image, but they should both be working soon at least. >> >> On Mon, Nov 26, 2012 at 5:35 AM, Stefan Priebe - Profihost AG >> <s.priebe@profihost.ag> wrote: >>> Am 26.11.2012 14:29, schrieb Oliver Francke: >>>> >>>> Well... >>> >>> >>> makes no difference: >>> gzip -dc vm-101-disk-1.img.gz | rbd import --format=2 --size=40960 - >>> kvmpool1/vm-101-disk-1 >>> >>> rbd: error reading file: (29) Illegal seek >>> Importing image: 0% complete...failed. >>> rbd: import failed: (29) Illegal seek >>> >>> I think the problem is that ceph seeks in File when it get's zeros (not >>> filled content). But seeking doesn't work on stdin. It has to read from >>> stdin until there are no more zeros. >>> >>> Stefan >>> >>> >>>> On 11/26/2012 02:20 PM, Stefan Priebe - Profihost AG wrote: >>>>> >>>>> Hello list, >>>>> >>>>> i know branch wip-rbd-export-stdout is work in progress but it is more >>>>> than useful ;-) >>>>> >>>>> When i try to import an image i get: >>>>> >>>>> # gzip -dc vm-101-disk-1.img.gz | rbd import --format=2 >>>>> --size=42949672960 - kvmpool1/vm-101-disk-1 >>>>> rbd: error reading file: (29) Illegal seek >>>>> Importing image: 0% complete...failed. >>>>> rbd: import failed: (29) Illegal seek >>>>> >>>>> Anything i've tried wrong? >>>> >>>> >>>> I would assume, that size is already in MiB? Seems to be a slightly too >>>> big value... Not tried myself, though... >>>> >>>> Oliver. >>>> >>>>> >>>>> Greets, >>>>> Stefan >>>>> -- >>>>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >>>>> the body of a message to majordomo@vger.kernel.org >>>>> More majordomo info at http://vger.kernel.org/majordomo-info.html >> -- >> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >> the body of a message to majordomo@vger.kernel.org >> More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: rbd STDIN import does not work / wip-rbd-export-stdout 2012-11-26 20:00 ` Dan Mick @ 2012-11-27 9:16 ` Stefan Priebe - Profihost AG 2012-11-27 22:50 ` Josh Durgin ` (2 more replies) 0 siblings, 3 replies; 15+ messages in thread From: Stefan Priebe - Profihost AG @ 2012-11-27 9:16 UTC (permalink / raw) To: Dan Mick; +Cc: Oliver Francke, ceph-devel@vger.kernel.org Am 26.11.2012 21:00, schrieb Dan Mick: > It writes zeros; there's no way for it to know how many zeros are > coming. It could make a half-hearted attempt depending on its buffer > size and the amount of data the source is willing to buffer. Yes OK that's correct. Maybe the buffer should be the size of a rados object / stripe size? So a whole stripe containing zeros could be ignored? > As for fstrim: on the rbd image?....sure, if it's a filesystem, it ought > to work (modulo some bugs I've heard about with 32-bit vs. 64-bit > offsets in qemu... :) ) it works fine with my patches i send to qemu. But right now they're no upstream an nobody of the qemu people answers anymore. So right now i'll keep my patches and be lucky others can't use RBD and fstrim. So maybe inktank should push this a bit more to the qemu people. Stefan > On 11/26/2012 11:35 AM, Stefan Priebe - Profihost AG wrote: >> Hi, >> >> Export isn't a problem. But import should handle this correctly. Or >> does it write zeros to the object file >> >> But even if it write zeros to the file shouldn't a fstrim fix this? >> >> Stefan >> >> Am 26.11.2012 um 18:34 schrieb Dan Mick <dan.mick@inktank.com>: >> >>> Yes, it's broken too (there's an issue open on it). I have some test >>> code done over the weekend >>> and will be pushing it after review. >>> >>> Be aware that import/export to stdin/stdout are limited in that they >>> cannot preserve sparseness >>> of the image, but they should both be working soon at least. >>> >>> On Mon, Nov 26, 2012 at 5:35 AM, Stefan Priebe - Profihost AG >>> <s.priebe@profihost.ag> wrote: >>>> Am 26.11.2012 14:29, schrieb Oliver Francke: >>>>> >>>>> Well... >>>> >>>> >>>> makes no difference: >>>> gzip -dc vm-101-disk-1.img.gz | rbd import --format=2 --size=40960 - >>>> kvmpool1/vm-101-disk-1 >>>> >>>> rbd: error reading file: (29) Illegal seek >>>> Importing image: 0% complete...failed. >>>> rbd: import failed: (29) Illegal seek >>>> >>>> I think the problem is that ceph seeks in File when it get's zeros (not >>>> filled content). But seeking doesn't work on stdin. It has to read from >>>> stdin until there are no more zeros. >>>> >>>> Stefan >>>> >>>> >>>>> On 11/26/2012 02:20 PM, Stefan Priebe - Profihost AG wrote: >>>>>> >>>>>> Hello list, >>>>>> >>>>>> i know branch wip-rbd-export-stdout is work in progress but it is >>>>>> more >>>>>> than useful ;-) >>>>>> >>>>>> When i try to import an image i get: >>>>>> >>>>>> # gzip -dc vm-101-disk-1.img.gz | rbd import --format=2 >>>>>> --size=42949672960 - kvmpool1/vm-101-disk-1 >>>>>> rbd: error reading file: (29) Illegal seek >>>>>> Importing image: 0% complete...failed. >>>>>> rbd: import failed: (29) Illegal seek >>>>>> >>>>>> Anything i've tried wrong? >>>>> >>>>> >>>>> I would assume, that size is already in MiB? Seems to be a slightly >>>>> too >>>>> big value... Not tried myself, though... >>>>> >>>>> Oliver. >>>>> >>>>>> >>>>>> Greets, >>>>>> Stefan >>>>>> -- >>>>>> To unsubscribe from this list: send the line "unsubscribe >>>>>> ceph-devel" in >>>>>> the body of a message to majordomo@vger.kernel.org >>>>>> More majordomo info at http://vger.kernel.org/majordomo-info.html >>> -- >>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >>> the body of a message to majordomo@vger.kernel.org >>> More majordomo info at http://vger.kernel.org/majordomo-info.html > -- > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: rbd STDIN import does not work / wip-rbd-export-stdout 2012-11-27 9:16 ` Stefan Priebe - Profihost AG @ 2012-11-27 22:50 ` Josh Durgin 2012-11-28 8:09 ` Stefan Priebe - Profihost AG 2012-11-28 14:53 ` Stefan Priebe - Profihost AG 2012-12-01 20:23 ` Stefan Priebe 2 siblings, 1 reply; 15+ messages in thread From: Josh Durgin @ 2012-11-27 22:50 UTC (permalink / raw) To: Stefan Priebe - Profihost AG Cc: Dan Mick, Oliver Francke, ceph-devel@vger.kernel.org On 11/27/2012 01:16 AM, Stefan Priebe - Profihost AG wrote: > Am 26.11.2012 21:00, schrieb Dan Mick: >> As for fstrim: on the rbd image?....sure, if it's a filesystem, it ought >> to work (modulo some bugs I've heard about with 32-bit vs. 64-bit >> offsets in qemu... :) ) > it works fine with my patches i send to qemu. But right now they're no > upstream an nobody of the qemu people answers anymore. So right now i'll > keep my patches and be lucky others can't use RBD and fstrim. So maybe > inktank should push this a bit more to the qemu people. If you could send another version with PATCHv3 instead of PATCH in the subject, and a description of the changes since the last version between the --- and the diffstat, the qemu folks would probably appreciate it. scripts/checkpatch.pl in qemu.git will warn you about style issues like the conditionals without braces. Josh ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: rbd STDIN import does not work / wip-rbd-export-stdout 2012-11-27 22:50 ` Josh Durgin @ 2012-11-28 8:09 ` Stefan Priebe - Profihost AG 2012-11-28 18:51 ` Josh Durgin 0 siblings, 1 reply; 15+ messages in thread From: Stefan Priebe - Profihost AG @ 2012-11-28 8:09 UTC (permalink / raw) To: Josh Durgin; +Cc: Dan Mick, Oliver Francke, ceph-devel@vger.kernel.org Am 27.11.2012 23:50, schrieb Josh Durgin: > On 11/27/2012 01:16 AM, Stefan Priebe - Profihost AG wrote: >> Am 26.11.2012 21:00, schrieb Dan Mick: >>> As for fstrim: on the rbd image?....sure, if it's a filesystem, it ought >>> to work (modulo some bugs I've heard about with 32-bit vs. 64-bit >>> offsets in qemu... :) ) >> it works fine with my patches i send to qemu. But right now they're no >> upstream an nobody of the qemu people answers anymore. So right now i'll >> keep my patches and be lucky others can't use RBD and fstrim. So maybe >> inktank should push this a bit more to the qemu people. > > If you could send another version with PATCHv3 instead of PATCH in the > subject, and a description of the changes since the last version between > the --- and the diffstat, the qemu folks would probably appreciate it. No idea how to archieve this with git send-email ;-( But still more important is also the patch for discards... > scripts/checkpatch.pl in qemu.git will warn you about style issues like > the conditionals without braces. Thank! 0001-rbd-block-driver-fix-race-between-aio-completition-a.patch has no obvious style problems and is ready for submission. Stefan ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: rbd STDIN import does not work / wip-rbd-export-stdout 2012-11-28 8:09 ` Stefan Priebe - Profihost AG @ 2012-11-28 18:51 ` Josh Durgin 2012-11-29 10:26 ` Stefan Priebe - Profihost AG 0 siblings, 1 reply; 15+ messages in thread From: Josh Durgin @ 2012-11-28 18:51 UTC (permalink / raw) To: Stefan Priebe - Profihost AG Cc: Dan Mick, Oliver Francke, ceph-devel@vger.kernel.org On 11/28/2012 12:09 AM, Stefan Priebe - Profihost AG wrote: > Am 27.11.2012 23:50, schrieb Josh Durgin: >> On 11/27/2012 01:16 AM, Stefan Priebe - Profihost AG wrote: >>> Am 26.11.2012 21:00, schrieb Dan Mick: >>>> As for fstrim: on the rbd image?....sure, if it's a filesystem, it >>>> ought >>>> to work (modulo some bugs I've heard about with 32-bit vs. 64-bit >>>> offsets in qemu... :) ) >>> it works fine with my patches i send to qemu. But right now they're no >>> upstream an nobody of the qemu people answers anymore. So right now i'll >>> keep my patches and be lucky others can't use RBD and fstrim. So maybe >>> inktank should push this a bit more to the qemu people. >> >> If you could send another version with PATCHv3 instead of PATCH in the >> subject, and a description of the changes since the last version between >> the --- and the diffstat, the qemu folks would probably appreciate it. > > No idea how to archieve this with git send-email ;-( But still more > important is also the patch for discards... Use git format-patch, edit the patch file (it includes the basic headers already), then send it with git send-email. Your return value type change was already merged into master of qemu.git as 08448d5195aeff49bf25fb62b4a6218f079f5284. Josh ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: rbd STDIN import does not work / wip-rbd-export-stdout 2012-11-28 18:51 ` Josh Durgin @ 2012-11-29 10:26 ` Stefan Priebe - Profihost AG 0 siblings, 0 replies; 15+ messages in thread From: Stefan Priebe - Profihost AG @ 2012-11-29 10:26 UTC (permalink / raw) To: Josh Durgin; +Cc: Dan Mick, Oliver Francke, ceph-devel@vger.kernel.org Hi Josh, Am 28.11.2012 19:51, schrieb Josh Durgin: >> No idea how to archieve this with git send-email ;-( But still more >> important is also the patch for discards... > > Use git format-patch, edit the patch file (it includes the basic > headers already), then send it with git send-email. done / fixed > Your return value type change was already merged into master of > qemu.git as 08448d5195aeff49bf25fb62b4a6218f079f5284. Oh thanks i didn't get a reply that it is applied. Stefan ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: rbd STDIN import does not work / wip-rbd-export-stdout 2012-11-27 9:16 ` Stefan Priebe - Profihost AG 2012-11-27 22:50 ` Josh Durgin @ 2012-11-28 14:53 ` Stefan Priebe - Profihost AG 2012-12-02 0:27 ` Dan Mick 2012-12-01 20:23 ` Stefan Priebe 2 siblings, 1 reply; 15+ messages in thread From: Stefan Priebe - Profihost AG @ 2012-11-28 14:53 UTC (permalink / raw) To: Dan Mick; +Cc: Oliver Francke, ceph-devel@vger.kernel.org Hi Dan, thanks for your update / new import-export branch. The import seems to work now. What i'm wondering is that it is really slow. I see around 10% CPU Usage on importing host (host where rbd import is started) and just 4MB every 2s raising Data on ceph. This means 40GB took around 5-6 hours. Night this be due to he size update? 4MB every 2s looks like the stripe size. Greets, Stefan ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: rbd STDIN import does not work / wip-rbd-export-stdout 2012-11-28 14:53 ` Stefan Priebe - Profihost AG @ 2012-12-02 0:27 ` Dan Mick 0 siblings, 0 replies; 15+ messages in thread From: Dan Mick @ 2012-12-02 0:27 UTC (permalink / raw) To: Stefan Priebe - Profihost AG; +Cc: Oliver Francke, ceph-devel@vger.kernel.org On 11/28/2012 6:53 AM, Stefan Priebe - Profihost AG wrote: > Hi Dan, > > thanks for your update / new import-export branch. > > The import seems to work now. What i'm wondering is that it is really > slow. > > I see around 10% CPU Usage on importing host (host where rbd import is > started) and just 4MB every 2s raising Data on ceph. This means 40GB > took around 5-6 hours. > > Night this be due to he size update? 4MB every 2s looks like the > stripe size. I did change the resize to double each time to help with that before pushing to master, but that still seems awfully slow. You should definitely use the version in the mainline, though; it's better in several ways. Are these particularly sparse images? I'm testing a "preserve sparseness" tweak that should help a lot if so (and save space in the cluster, obviously). ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: rbd STDIN import does not work / wip-rbd-export-stdout 2012-11-27 9:16 ` Stefan Priebe - Profihost AG 2012-11-27 22:50 ` Josh Durgin 2012-11-28 14:53 ` Stefan Priebe - Profihost AG @ 2012-12-01 20:23 ` Stefan Priebe 2012-12-02 0:28 ` Dan Mick 2 siblings, 1 reply; 15+ messages in thread From: Stefan Priebe @ 2012-12-01 20:23 UTC (permalink / raw) To: Dan Mick; +Cc: ceph-devel@vger.kernel.org Hi Dan, Am 27.11.2012 10:16, schrieb Stefan Priebe - Profihost AG: > Am 26.11.2012 21:00, schrieb Dan Mick: >> It writes zeros; there's no way for it to know how many zeros are >> coming. It could make a half-hearted attempt depending on its buffer >> size and the amount of data the source is willing to buffer. Yes but why not skip at leas if the WHOLE BUFFER is zero and then use the buffer size = stripe size? Right now a 40GB image which has just 3GB used rest is zeros due to using fstrim before doing the export takes a long time to import. We could speed this up a lot if we would skip parts with only zeros. Greets, Stefab ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: rbd STDIN import does not work / wip-rbd-export-stdout 2012-12-01 20:23 ` Stefan Priebe @ 2012-12-02 0:28 ` Dan Mick 0 siblings, 0 replies; 15+ messages in thread From: Dan Mick @ 2012-12-02 0:28 UTC (permalink / raw) To: Stefan Priebe; +Cc: ceph-devel@vger.kernel.org On 12/1/2012 12:23 PM, Stefan Priebe wrote: > Hi Dan, > > Am 27.11.2012 10:16, schrieb Stefan Priebe - Profihost AG: >> Am 26.11.2012 21:00, schrieb Dan Mick: >>> It writes zeros; there's no way for it to know how many zeros are >>> coming. It could make a half-hearted attempt depending on its buffer >>> size and the amount of data the source is willing to buffer. > > Yes but why not skip at leas if the WHOLE BUFFER is zero and then use > the buffer size = stripe size? > > Right now a 40GB image which has just 3GB used rest is zeros due to > using fstrim before doing the export takes a long time to import. We > could speed this up a lot if we would skip parts with only zeros. Great minds! I'm actually working on that now; as you say it's not hard. > Greets, > Stefab ^ permalink raw reply [flat|nested] 15+ messages in thread
end of thread, other threads:[~2012-12-02 0:28 UTC | newest] Thread overview: 15+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2012-11-26 13:20 rbd STDIN import does not work / wip-rbd-export-stdout Stefan Priebe - Profihost AG 2012-11-26 13:29 ` Oliver Francke 2012-11-26 13:35 ` Stefan Priebe - Profihost AG 2012-11-26 17:34 ` Dan Mick 2012-11-26 19:35 ` Stefan Priebe - Profihost AG 2012-11-26 20:00 ` Dan Mick 2012-11-27 9:16 ` Stefan Priebe - Profihost AG 2012-11-27 22:50 ` Josh Durgin 2012-11-28 8:09 ` Stefan Priebe - Profihost AG 2012-11-28 18:51 ` Josh Durgin 2012-11-29 10:26 ` Stefan Priebe - Profihost AG 2012-11-28 14:53 ` Stefan Priebe - Profihost AG 2012-12-02 0:27 ` Dan Mick 2012-12-01 20:23 ` Stefan Priebe 2012-12-02 0:28 ` Dan Mick
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.