* Ceph Ansible Repo @ 2014-03-02 23:26 Sebastien Han 2014-03-03 2:59 ` Ian Colle 2014-03-06 9:36 ` Gandalf Corvotempesta 0 siblings, 2 replies; 18+ messages in thread From: Sebastien Han @ 2014-03-02 23:26 UTC (permalink / raw) To: ceph-devel; +Cc: Weil Sage, Patrick McGarry [-- Attachment #1: Type: text/plain, Size: 508 bytes --] Hi, As discussed during the Ceph day, I achieved the Ansible playbook for Ceph. It currently lives on my github, but I’d like to see it on github.com/ceph. Does anyone can create the corresponding repo so I can send a pull request? Thanks! –––– Sébastien Han Cloud Engineer "Always give 100%. Unless you're giving blood.” Phone: +33 (0)1 49 70 99 72 Mail: sebastien.han@enovance.com Address : 11 bis, rue Roquépine - 75008 Paris Web : www.enovance.com - Twitter : @enovance [-- Attachment #2: Message signed with OpenPGP using GPGMail --] [-- Type: application/pgp-signature, Size: 496 bytes --] ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Ceph Ansible Repo 2014-03-02 23:26 Ceph Ansible Repo Sebastien Han @ 2014-03-03 2:59 ` Ian Colle 2014-03-03 8:49 ` Sebastien Han 2014-03-06 9:36 ` Gandalf Corvotempesta 1 sibling, 1 reply; 18+ messages in thread From: Ian Colle @ 2014-03-03 2:59 UTC (permalink / raw) To: Sebastien Han, ceph-devel; +Cc: Weil Sage, Patrick McGarry Sebastien, Have you seen https://github.com/alfredodeza/ceph-ansible ? Ian R. Colle Director of Engineering Inktank Delivering the Future of Storage http://www.linkedin.com/in/ircolle http://www.twitter.com/ircolle Cell: +1.303.601.7713 Email: ian@inktank.com On 3/2/14, 4:26 PM, "Sebastien Han" <sebastien.han@enovance.com> wrote: >Hi, > >As discussed during the Ceph day, I achieved the Ansible playbook for >Ceph. >It currently lives on my github, but I¹d like to see it on >github.com/ceph. > >Does anyone can create the corresponding repo so I can send a pull >request? > >Thanks! > > >Sébastien Han >Cloud Engineer > >"Always give 100%. Unless you're giving blood.² > >Phone: +33 (0)1 49 70 99 72 >Mail: sebastien.han@enovance.com >Address : 11 bis, rue Roquépine - 75008 Paris >Web : www.enovance.com - Twitter : @enovance > -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Ceph Ansible Repo 2014-03-03 2:59 ` Ian Colle @ 2014-03-03 8:49 ` Sebastien Han 2014-03-03 15:30 ` Travis Rhoden 0 siblings, 1 reply; 18+ messages in thread From: Sebastien Han @ 2014-03-03 8:49 UTC (permalink / raw) To: Ian Colle; +Cc: ceph-devel@vger.kernel.org, Weil Sage, Patrick McGarry [-- Attachment #1: Type: text/plain, Size: 1360 bytes --] Ian, Yes I’ve seen it, however it only installs Ceph. Cheers. –––– Sébastien Han Cloud Engineer "Always give 100%. Unless you're giving blood.” Phone: +33 (0)1 49 70 99 72 Mail: sebastien.han@enovance.com Address : 11 bis, rue Roquépine - 75008 Paris Web : www.enovance.com - Twitter : @enovance On 03 Mar 2014, at 03:59, Ian Colle <ian.colle@inktank.com> wrote: > Sebastien, > > Have you seen https://github.com/alfredodeza/ceph-ansible ? > > Ian R. Colle > Director of Engineering > Inktank > Delivering the Future of Storage > http://www.linkedin.com/in/ircolle > http://www.twitter.com/ircolle > Cell: +1.303.601.7713 > Email: ian@inktank.com > > > On 3/2/14, 4:26 PM, "Sebastien Han" <sebastien.han@enovance.com> wrote: > >> Hi, >> >> As discussed during the Ceph day, I achieved the Ansible playbook for >> Ceph. >> It currently lives on my github, but I¹d like to see it on >> github.com/ceph. >> >> Does anyone can create the corresponding repo so I can send a pull >> request? >> >> Thanks! >> >> >> Sébastien Han >> Cloud Engineer >> >> "Always give 100%. Unless you're giving blood.² >> >> Phone: +33 (0)1 49 70 99 72 >> Mail: sebastien.han@enovance.com >> Address : 11 bis, rue Roquépine - 75008 Paris >> Web : www.enovance.com - Twitter : @enovance >> > > [-- Attachment #2: Message signed with OpenPGP using GPGMail --] [-- Type: application/pgp-signature, Size: 496 bytes --] ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Ceph Ansible Repo 2014-03-03 8:49 ` Sebastien Han @ 2014-03-03 15:30 ` Travis Rhoden 2014-03-03 17:13 ` Ian Colle 0 siblings, 1 reply; 18+ messages in thread From: Travis Rhoden @ 2014-03-03 15:30 UTC (permalink / raw) To: Sebastien Han Cc: Ian Colle, ceph-devel@vger.kernel.org, Weil Sage, Patrick McGarry I had been working on something similar to Seb's. Alfredo's (which I had looked at before) seems to be a wrapper around ceph-deploy. Useful, but not as fully featured. It also hasn't been updated in 7 months. I would certainly like to see a more fully-featured set of Ansible playbooks in Ceph's main GitHub repo. I think Seb's work here is a great start! It's definitely a project I can help contribute to, as well. - Travis On Mon, Mar 3, 2014 at 3:49 AM, Sebastien Han <sebastien.han@enovance.com> wrote: > Ian, > > Yes I've seen it, however it only installs Ceph. > > Cheers. > ---- > Sébastien Han > Cloud Engineer > > "Always give 100%. Unless you're giving blood." > > Phone: +33 (0)1 49 70 99 72 > Mail: sebastien.han@enovance.com > Address : 11 bis, rue Roquépine - 75008 Paris > Web : www.enovance.com - Twitter : @enovance > > On 03 Mar 2014, at 03:59, Ian Colle <ian.colle@inktank.com> wrote: > >> Sebastien, >> >> Have you seen https://github.com/alfredodeza/ceph-ansible ? >> >> Ian R. Colle >> Director of Engineering >> Inktank >> Delivering the Future of Storage >> http://www.linkedin.com/in/ircolle >> http://www.twitter.com/ircolle >> Cell: +1.303.601.7713 >> Email: ian@inktank.com >> >> >> On 3/2/14, 4:26 PM, "Sebastien Han" <sebastien.han@enovance.com> wrote: >> >>> Hi, >>> >>> As discussed during the Ceph day, I achieved the Ansible playbook for >>> Ceph. >>> It currently lives on my github, but I¹d like to see it on >>> github.com/ceph. >>> >>> Does anyone can create the corresponding repo so I can send a pull >>> request? >>> >>> Thanks! >>> >>> >>> Sébastien Han >>> Cloud Engineer >>> >>> "Always give 100%. Unless you're giving blood.² >>> >>> Phone: +33 (0)1 49 70 99 72 >>> Mail: sebastien.han@enovance.com >>> Address : 11 bis, rue Roquépine - 75008 Paris >>> Web : www.enovance.com - Twitter : @enovance >>> >> >> > -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Ceph Ansible Repo 2014-03-03 15:30 ` Travis Rhoden @ 2014-03-03 17:13 ` Ian Colle 2014-03-03 18:13 ` Sebastien Han 0 siblings, 1 reply; 18+ messages in thread From: Ian Colle @ 2014-03-03 17:13 UTC (permalink / raw) To: Travis Rhoden, Sebastien Han, Alfredo Deza Cc: ceph-devel@vger.kernel.org, Weil Sage, Patrick McGarry Gents, Enjoy. Ready for PR’s. https://github.com/ceph/ceph-ansible Ian R. Colle Director of Engineering Inktank Delivering the Future of Storage http://www.linkedin.com/in/ircolle http://www.twitter.com/ircolle Cell: +1.303.601.7713 Email: ian@inktank.com On 3/3/14, 8:30 AM, "Travis Rhoden" <trhoden@gmail.com> wrote: >I had been working on something similar to Seb's. Alfredo's (which I >had looked at before) seems to be a wrapper around ceph-deploy. >Useful, but not as fully featured. It also hasn't been updated in 7 >months. > >I would certainly like to see a more fully-featured set of Ansible >playbooks in Ceph's main GitHub repo. I think Seb's work here is a >great start! It's definitely a project I can help contribute to, as >well. > > - Travis > >On Mon, Mar 3, 2014 at 3:49 AM, Sebastien Han ><sebastien.han@enovance.com> wrote: >> Ian, >> >> Yes I've seen it, however it only installs Ceph. >> >> Cheers. >> ---- >> Sébastien Han >> Cloud Engineer >> >> "Always give 100%. Unless you're giving blood." >> >> Phone: +33 (0)1 49 70 99 72 >> Mail: sebastien.han@enovance.com >> Address : 11 bis, rue Roquépine - 75008 Paris >> Web : www.enovance.com - Twitter : @enovance >> >> On 03 Mar 2014, at 03:59, Ian Colle <ian.colle@inktank.com> wrote: >> >>> Sebastien, >>> >>> Have you seen https://github.com/alfredodeza/ceph-ansible ? >>> >>> Ian R. Colle >>> Director of Engineering >>> Inktank >>> Delivering the Future of Storage >>> http://www.linkedin.com/in/ircolle >>> http://www.twitter.com/ircolle >>> Cell: +1.303.601.7713 >>> Email: ian@inktank.com >>> >>> >>> On 3/2/14, 4:26 PM, "Sebastien Han" <sebastien.han@enovance.com> wrote: >>> >>>> Hi, >>>> >>>> As discussed during the Ceph day, I achieved the Ansible playbook for >>>> Ceph. >>>> It currently lives on my github, but I¹d like to see it on >>>> github.com/ceph. >>>> >>>> Does anyone can create the corresponding repo so I can send a pull >>>> request? >>>> >>>> Thanks! >>>> >>>> >>>> Sébastien Han >>>> Cloud Engineer >>>> >>>> "Always give 100%. Unless you're giving blood.² >>>> >>>> Phone: +33 (0)1 49 70 99 72 >>>> Mail: sebastien.han@enovance.com >>>> Address : 11 bis, rue Roquépine - 75008 Paris >>>> Web : www.enovance.com - Twitter : @enovance >>>> >>> >>> >> -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Ceph Ansible Repo 2014-03-03 17:13 ` Ian Colle @ 2014-03-03 18:13 ` Sebastien Han 0 siblings, 0 replies; 18+ messages in thread From: Sebastien Han @ 2014-03-03 18:13 UTC (permalink / raw) To: Ian Colle Cc: Travis Rhoden, Alfredo Deza, ceph-devel@vger.kernel.org, Weil Sage, Patrick McGarry [-- Attachment #1: Type: text/plain, Size: 2918 bytes --] Thanks a lot Ian, here we go https://github.com/ceph/ceph-ansible/pull/1 :) –––– Sébastien Han Cloud Engineer "Always give 100%. Unless you're giving blood.” Phone: +33 (0)1 49 70 99 72 Mail: sebastien.han@enovance.com Address : 11 bis, rue Roquépine - 75008 Paris Web : www.enovance.com - Twitter : @enovance On 03 Mar 2014, at 18:13, Ian Colle <ian.colle@inktank.com> wrote: > Gents, > > Enjoy. Ready for PR’s. > > https://github.com/ceph/ceph-ansible > > > Ian R. Colle > Director of Engineering > Inktank > Delivering the Future of Storage > http://www.linkedin.com/in/ircolle > http://www.twitter.com/ircolle > Cell: +1.303.601.7713 > Email: ian@inktank.com > > > On 3/3/14, 8:30 AM, "Travis Rhoden" <trhoden@gmail.com> wrote: > >> I had been working on something similar to Seb's. Alfredo's (which I >> had looked at before) seems to be a wrapper around ceph-deploy. >> Useful, but not as fully featured. It also hasn't been updated in 7 >> months. >> >> I would certainly like to see a more fully-featured set of Ansible >> playbooks in Ceph's main GitHub repo. I think Seb's work here is a >> great start! It's definitely a project I can help contribute to, as >> well. >> >> - Travis >> >> On Mon, Mar 3, 2014 at 3:49 AM, Sebastien Han >> <sebastien.han@enovance.com> wrote: >>> Ian, >>> >>> Yes I've seen it, however it only installs Ceph. >>> >>> Cheers. >>> ---- >>> Sébastien Han >>> Cloud Engineer >>> >>> "Always give 100%. Unless you're giving blood." >>> >>> Phone: +33 (0)1 49 70 99 72 >>> Mail: sebastien.han@enovance.com >>> Address : 11 bis, rue Roquépine - 75008 Paris >>> Web : www.enovance.com - Twitter : @enovance >>> >>> On 03 Mar 2014, at 03:59, Ian Colle <ian.colle@inktank.com> wrote: >>> >>>> Sebastien, >>>> >>>> Have you seen https://github.com/alfredodeza/ceph-ansible ? >>>> >>>> Ian R. Colle >>>> Director of Engineering >>>> Inktank >>>> Delivering the Future of Storage >>>> http://www.linkedin.com/in/ircolle >>>> http://www.twitter.com/ircolle >>>> Cell: +1.303.601.7713 >>>> Email: ian@inktank.com >>>> >>>> >>>> On 3/2/14, 4:26 PM, "Sebastien Han" <sebastien.han@enovance.com> wrote: >>>> >>>>> Hi, >>>>> >>>>> As discussed during the Ceph day, I achieved the Ansible playbook for >>>>> Ceph. >>>>> It currently lives on my github, but I¹d like to see it on >>>>> github.com/ceph. >>>>> >>>>> Does anyone can create the corresponding repo so I can send a pull >>>>> request? >>>>> >>>>> Thanks! >>>>> >>>>> >>>>> Sébastien Han >>>>> Cloud Engineer >>>>> >>>>> "Always give 100%. Unless you're giving blood.² >>>>> >>>>> Phone: +33 (0)1 49 70 99 72 >>>>> Mail: sebastien.han@enovance.com >>>>> Address : 11 bis, rue Roquépine - 75008 Paris >>>>> Web : www.enovance.com - Twitter : @enovance >>>>> >>>> >>>> >>> > > [-- Attachment #2: Message signed with OpenPGP using GPGMail --] [-- Type: application/pgp-signature, Size: 496 bytes --] ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Ceph Ansible Repo 2014-03-02 23:26 Ceph Ansible Repo Sebastien Han 2014-03-03 2:59 ` Ian Colle @ 2014-03-06 9:36 ` Gandalf Corvotempesta 2014-03-06 10:14 ` Sebastien Han 2014-03-06 12:07 ` David McBride 1 sibling, 2 replies; 18+ messages in thread From: Gandalf Corvotempesta @ 2014-03-06 9:36 UTC (permalink / raw) To: Sebastien Han; +Cc: ceph-devel@vger.kernel.org, Weil Sage, Patrick McGarry Hi to all. Some question about ansible repo. 1. is possible to specify multiple journal devices for each OSD node? We have OSD with 12 spinning disks and 2 SSD as journal. We would like to use SSD1 for OSD1 to 6 and SSD2 for OSD 7 to 12. Is this possible ? 2. With this repo, each OSD node must be identical to each other, right? What if OSD server1 has 3 spinning disks and OSD server2 has just 2 ? 2014-03-03 0:26 GMT+01:00 Sebastien Han <sebastien.han@enovance.com>: > Hi, > > As discussed during the Ceph day, I achieved the Ansible playbook for Ceph. > It currently lives on my github, but I'd like to see it on github.com/ceph. > > Does anyone can create the corresponding repo so I can send a pull request? > > Thanks! > > ---- > Sébastien Han > Cloud Engineer > > "Always give 100%. Unless you're giving blood." > > Phone: +33 (0)1 49 70 99 72 > Mail: sebastien.han@enovance.com > Address : 11 bis, rue Roquépine - 75008 Paris > Web : www.enovance.com - Twitter : @enovance > -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Ceph Ansible Repo 2014-03-06 9:36 ` Gandalf Corvotempesta @ 2014-03-06 10:14 ` Sebastien Han 2014-03-06 10:28 ` Gandalf Corvotempesta 2014-03-06 12:07 ` David McBride 1 sibling, 1 reply; 18+ messages in thread From: Sebastien Han @ 2014-03-06 10:14 UTC (permalink / raw) To: Gandalf Corvotempesta Cc: ceph-devel@vger.kernel.org, Weil Sage, Patrick McGarry [-- Attachment #1: Type: text/plain, Size: 1877 bytes --] Hi Gandalf, Answers inline: –––– Sébastien Han Cloud Engineer "Always give 100%. Unless you're giving blood.” Phone: +33 (0)1 49 70 99 72 Mail: sebastien.han@enovance.com Address : 11 bis, rue Roquépine - 75008 Paris Web : www.enovance.com - Twitter : @enovance On 06 Mar 2014, at 10:36, Gandalf Corvotempesta <gandalf.corvotempesta@gmail.com> wrote: > Hi to all. > Some question about ansible repo. > > 1. is possible to specify multiple journal devices for each OSD node? > We have OSD with 12 spinning disks and 2 SSD as journal. We would like > to use SSD1 for OSD1 to 6 and SSD2 for OSD 7 to 12. Is this possible ? No this is currently not supported. We can only select on drive per server. Can you submit an issue on Github for enhancement? > > 2. With this repo, each OSD node must be identical to each other, > right? What if OSD server1 has 3 spinning disks and OSD server2 has > just 2 ? Yes they must be identical. If a device doesn’t exist Ansible will simply hang on the “parted” test, it will probably timeout at some point, however I haven’t tested this. However I just pushed a pull request to workaround this issue. https://github.com/ceph/ceph-ansible/pull/4 > > > 2014-03-03 0:26 GMT+01:00 Sebastien Han <sebastien.han@enovance.com>: >> Hi, >> >> As discussed during the Ceph day, I achieved the Ansible playbook for Ceph. >> It currently lives on my github, but I'd like to see it on github.com/ceph. >> >> Does anyone can create the corresponding repo so I can send a pull request? >> >> Thanks! >> >> ---- >> Sébastien Han >> Cloud Engineer >> >> "Always give 100%. Unless you're giving blood." >> >> Phone: +33 (0)1 49 70 99 72 >> Mail: sebastien.han@enovance.com >> Address : 11 bis, rue Roquépine - 75008 Paris >> Web : www.enovance.com - Twitter : @enovance >> [-- Attachment #2: Message signed with OpenPGP using GPGMail --] [-- Type: application/pgp-signature, Size: 496 bytes --] ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Ceph Ansible Repo 2014-03-06 10:14 ` Sebastien Han @ 2014-03-06 10:28 ` Gandalf Corvotempesta 2014-03-06 10:42 ` Sebastien Han 0 siblings, 1 reply; 18+ messages in thread From: Gandalf Corvotempesta @ 2014-03-06 10:28 UTC (permalink / raw) To: Sebastien Han; +Cc: ceph-devel@vger.kernel.org, Weil Sage, Patrick McGarry 2014-03-06 11:14 GMT+01:00 Sebastien Han <sebastien.han@enovance.com>: > No this is currently not supported. > We can only select on drive per server. > Can you submit an issue on Github for enhancement? I'll do in a couple of minutes. > Yes they must be identical. If a device doesn't exist Ansible will simply hang on the "parted" test, it will probably timeout at some point, however I haven't tested this. > However I just pushed a pull request to workaround this issue. > https://github.com/ceph/ceph-ansible/pull/4 Cool. In this case I'll be able to specify all 12 spinning disks for my servers even if they aren't installed. Is Ansible smart enough to detect and configure newly added disks in the next run ? ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Ceph Ansible Repo 2014-03-06 10:28 ` Gandalf Corvotempesta @ 2014-03-06 10:42 ` Sebastien Han 2014-03-06 10:43 ` Gandalf Corvotempesta 0 siblings, 1 reply; 18+ messages in thread From: Sebastien Han @ 2014-03-06 10:42 UTC (permalink / raw) To: Gandalf Corvotempesta Cc: ceph-devel@vger.kernel.org, Weil Sage, Patrick McGarry [-- Attachment #1: Type: text/plain, Size: 1301 bytes --] Cool :) Yes you will be able to pre-provision all the disks. During the next run it will retry all the disks, some will be installed some will not therefore it will configure them :). –––– Sébastien Han Cloud Engineer "Always give 100%. Unless you're giving blood.” Phone: +33 (0)1 49 70 99 72 Mail: sebastien.han@enovance.com Address : 11 bis, rue Roquépine - 75008 Paris Web : www.enovance.com - Twitter : @enovance On 06 Mar 2014, at 11:28, Gandalf Corvotempesta <gandalf.corvotempesta@gmail.com> wrote: > 2014-03-06 11:14 GMT+01:00 Sebastien Han <sebastien.han@enovance.com>: >> No this is currently not supported. >> We can only select on drive per server. >> Can you submit an issue on Github for enhancement? > > I'll do in a couple of minutes. > >> Yes they must be identical. If a device doesn't exist Ansible will simply hang on the "parted" test, it will probably timeout at some point, however I haven't tested this. >> However I just pushed a pull request to workaround this issue. >> https://github.com/ceph/ceph-ansible/pull/4 > > Cool. > In this case I'll be able to specify all 12 spinning disks for my > servers even if they aren't installed. > Is Ansible smart enough to detect and configure newly added disks in > the next run ? [-- Attachment #2: Message signed with OpenPGP using GPGMail --] [-- Type: application/pgp-signature, Size: 496 bytes --] ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Ceph Ansible Repo 2014-03-06 10:42 ` Sebastien Han @ 2014-03-06 10:43 ` Gandalf Corvotempesta 0 siblings, 0 replies; 18+ messages in thread From: Gandalf Corvotempesta @ 2014-03-06 10:43 UTC (permalink / raw) To: Sebastien Han; +Cc: ceph-devel@vger.kernel.org, Weil Sage, Patrick McGarry 2014-03-06 11:42 GMT+01:00 Sebastien Han <sebastien.han@enovance.com>: > Cool :) > > Yes you will be able to pre-provision all the disks. > During the next run it will retry all the disks, some will be installed some will not therefore it will configure them :). Perfect. Now the only missing thing is https://github.com/ceph/ceph-ansible/issues/5 ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Ceph Ansible Repo 2014-03-06 9:36 ` Gandalf Corvotempesta 2014-03-06 10:14 ` Sebastien Han @ 2014-03-06 12:07 ` David McBride 2014-03-06 13:19 ` Gandalf Corvotempesta 1 sibling, 1 reply; 18+ messages in thread From: David McBride @ 2014-03-06 12:07 UTC (permalink / raw) To: Gandalf Corvotempesta, Sebastien Han Cc: ceph-devel@vger.kernel.org, Weil Sage, Patrick McGarry On 06/03/14 09:36, Gandalf Corvotempesta wrote: > Hi to all. > Some question about ansible repo. > > 1. is possible to specify multiple journal devices for each OSD node? > We have OSD with 12 spinning disks and 2 SSD as journal. We would like > to use SSD1 for OSD1 to 6 and SSD2 for OSD 7 to 12. Is this possible ? One option I'm currently testing is whereby the two SSDs in a node are stitched together into a Linux software-RAID striped array, and then allocate the journal partitions on the resulting /dev/md device. This is currently working well, save that the default udev rules for mapping OSDs explicitly skip over md journal devices; removing the "|md*" term in the relevant ruleset resolves this problem. (This is in my workqueue to report as a bug.) This causes the IO load to be nicely balanced across the two SSDs, removing any hot spots, at the cost of enlarging the failure domain of the loss of an SSD from half a node to a full node. One potential optimization which I haven't yet investigated is to see whether moving the XFS filesystem journal, in addition to the OSd journal, might result in an improvement in performance — in particular on nodes which are equipped with slower 'green' drives. This is not something that ceph-deploy currently makes easy; if, as I expect, I find myself re-implementing the ceph-deploy functions in terms of Ansible rules — perhaps using LVM rather than GPT partitions? — I may then be able to try some experiments. Kind regards, David -- David McBride <dwm37@cam.ac.uk> Unix Specialist, University Computing Service -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Ceph Ansible Repo 2014-03-06 12:07 ` David McBride @ 2014-03-06 13:19 ` Gandalf Corvotempesta 2014-03-06 13:28 ` David McBride 0 siblings, 1 reply; 18+ messages in thread From: Gandalf Corvotempesta @ 2014-03-06 13:19 UTC (permalink / raw) To: David McBride Cc: Sebastien Han, ceph-devel@vger.kernel.org, Weil Sage, Patrick McGarry 2014-03-06 13:07 GMT+01:00 David McBride <dwm37@cam.ac.uk>: > This causes the IO load to be nicely balanced across the two SSDs, > removing any hot spots, at the cost of enlarging the failure domain of > the loss of an SSD from half a node to a full node. This is not a solution for me. Why not using LVM with a VG striped across both SSD ? I've never used LVM without raid, what happens in case of failure of a phisical disks? The whole VG is lost ? ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Ceph Ansible Repo 2014-03-06 13:19 ` Gandalf Corvotempesta @ 2014-03-06 13:28 ` David McBride 2014-03-06 16:31 ` Sebastien Han 0 siblings, 1 reply; 18+ messages in thread From: David McBride @ 2014-03-06 13:28 UTC (permalink / raw) To: Gandalf Corvotempesta Cc: Sebastien Han, ceph-devel@vger.kernel.org, Weil Sage, Patrick McGarry On 06/03/14 13:19, Gandalf Corvotempesta wrote: > 2014-03-06 13:07 GMT+01:00 David McBride <dwm37@cam.ac.uk>: >> This causes the IO load to be nicely balanced across the two SSDs, >> removing any hot spots, at the cost of enlarging the failure domain of >> the loss of an SSD from half a node to a full node. > > This is not a solution for me. > Why not using LVM with a VG striped across both SSD ? > I've never used LVM without raid, what happens in case of failure > of a phisical disks? The whole VG is lost ? Yes. A stripe-set depends on all of the members of an array, whether managed through MD or LVM. Thus, in a machine with two SSDs, which are striped together, the loss of *either* SSD will cause all of the OSDs hosted by that machine to be lost. (Note: if you want to use LVM rather than GPTs on MD, you will probably need to remove the '|dm-*' clause from the Ceph udev rules that govern OSD assembly before they will work as expected.) Kind regards, David -- David McBride <dwm37@cam.ac.uk> Unix Specialist, University Computing Service ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Ceph Ansible Repo 2014-03-06 13:28 ` David McBride @ 2014-03-06 16:31 ` Sebastien Han 2014-03-06 16:45 ` Gandalf Corvotempesta 0 siblings, 1 reply; 18+ messages in thread From: Sebastien Han @ 2014-03-06 16:31 UTC (permalink / raw) To: David McBride Cc: Gandalf Corvotempesta, ceph-devel@vger.kernel.org, Weil Sage, Patrick McGarry [-- Attachment #1: Type: text/plain, Size: 1956 bytes --] I believe there is a use case behind the lvm journals. For instance you can do: 2 SSDs: * tiny mdadm raid 1 setup for the system; let’s say /dev/sda1 and /dev/sdb1 * then you still have: * /dev/sda2 * /dev/sdb2 They can both host journals, and you usually want to manage them with lvm, this is easier than managing partition. I just open an issue as a feature/enhancement request. https://github.com/ceph/ceph-ansible/issues/9 This shouldn’t be that difficult to implement. –––– Sébastien Han Cloud Engineer "Always give 100%. Unless you're giving blood.” Phone: +33 (0)1 49 70 99 72 Mail: sebastien.han@enovance.com Address : 11 bis, rue Roquépine - 75008 Paris Web : www.enovance.com - Twitter : @enovance On 06 Mar 2014, at 14:28, David McBride <dwm37@cam.ac.uk> wrote: > On 06/03/14 13:19, Gandalf Corvotempesta wrote: >> 2014-03-06 13:07 GMT+01:00 David McBride <dwm37@cam.ac.uk>: >>> This causes the IO load to be nicely balanced across the two SSDs, >>> removing any hot spots, at the cost of enlarging the failure domain of >>> the loss of an SSD from half a node to a full node. >> >> This is not a solution for me. >> Why not using LVM with a VG striped across both SSD ? >> I've never used LVM without raid, what happens in case of failure >> of a phisical disks? The whole VG is lost ? > > Yes. A stripe-set depends on all of the members of an array, whether > managed through MD or LVM. > > Thus, in a machine with two SSDs, which are striped together, the loss > of *either* SSD will cause all of the OSDs hosted by that machine to be > lost. > > (Note: if you want to use LVM rather than GPTs on MD, you will probably > need to remove the '|dm-*' clause from the Ceph udev rules that govern > OSD assembly before they will work as expected.) > > Kind regards, > David > -- > David McBride <dwm37@cam.ac.uk> > Unix Specialist, University Computing Service [-- Attachment #2: Message signed with OpenPGP using GPGMail --] [-- Type: application/pgp-signature, Size: 496 bytes --] ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Ceph Ansible Repo 2014-03-06 16:31 ` Sebastien Han @ 2014-03-06 16:45 ` Gandalf Corvotempesta 2014-03-06 18:05 ` Sebastien Han 0 siblings, 1 reply; 18+ messages in thread From: Gandalf Corvotempesta @ 2014-03-06 16:45 UTC (permalink / raw) To: Sebastien Han Cc: David McBride, ceph-devel@vger.kernel.org, Weil Sage, Patrick McGarry 2014-03-06 17:31 GMT+01:00 Sebastien Han <sebastien.han@enovance.com>: > They can both host journals, and you usually want to manage them with lvm, this is easier than managing partition. > I just open an issue as a feature/enhancement request. Good idea! But first, check my request :-D when implemented I'll try the ansible recipe on my cluster ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Ceph Ansible Repo 2014-03-06 16:45 ` Gandalf Corvotempesta @ 2014-03-06 18:05 ` Sebastien Han 2014-03-07 13:09 ` Sebastien Han 0 siblings, 1 reply; 18+ messages in thread From: Sebastien Han @ 2014-03-06 18:05 UTC (permalink / raw) To: Gandalf Corvotempesta Cc: David McBride, ceph-devel@vger.kernel.org, Weil Sage, Patrick McGarry [-- Attachment #1: Type: text/plain, Size: 743 bytes --] Haha yes I will ASAP :) –––– Sébastien Han Cloud Engineer "Always give 100%. Unless you're giving blood.” Phone: +33 (0)1 49 70 99 72 Mail: sebastien.han@enovance.com Address : 11 bis, rue Roquépine - 75008 Paris Web : www.enovance.com - Twitter : @enovance On 06 Mar 2014, at 17:45, Gandalf Corvotempesta <gandalf.corvotempesta@gmail.com> wrote: > 2014-03-06 17:31 GMT+01:00 Sebastien Han <sebastien.han@enovance.com>: >> They can both host journals, and you usually want to manage them with lvm, this is easier than managing partition. >> I just open an issue as a feature/enhancement request. > > Good idea! > But first, check my request :-D when implemented I'll try the ansible > recipe on my cluster [-- Attachment #2: Message signed with OpenPGP using GPGMail --] [-- Type: application/pgp-signature, Size: 496 bytes --] ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Ceph Ansible Repo 2014-03-06 18:05 ` Sebastien Han @ 2014-03-07 13:09 ` Sebastien Han 0 siblings, 0 replies; 18+ messages in thread From: Sebastien Han @ 2014-03-07 13:09 UTC (permalink / raw) To: Gandalf Corvotempesta Cc: David McBride, ceph-devel@vger.kernel.org, Weil Sage, Patrick McGarry [-- Attachment #1: Type: text/plain, Size: 1178 bytes --] Here we go: https://github.com/ceph/ceph-ansible/pull/11 –––– Sébastien Han Cloud Engineer "Always give 100%. Unless you're giving blood.” Phone: +33 (0)1 49 70 99 72 Mail: sebastien.han@enovance.com Address : 11 bis, rue Roquépine - 75008 Paris Web : www.enovance.com - Twitter : @enovance On 06 Mar 2014, at 19:05, Sebastien Han <sebastien.han@enovance.com> wrote: > Haha yes I will ASAP :) > > –––– > Sébastien Han > Cloud Engineer > > "Always give 100%. Unless you're giving blood.” > > Phone: +33 (0)1 49 70 99 72 > Mail: sebastien.han@enovance.com > Address : 11 bis, rue Roquépine - 75008 Paris > Web : www.enovance.com - Twitter : @enovance > > On 06 Mar 2014, at 17:45, Gandalf Corvotempesta <gandalf.corvotempesta@gmail.com> wrote: > >> 2014-03-06 17:31 GMT+01:00 Sebastien Han <sebastien.han@enovance.com>: >>> They can both host journals, and you usually want to manage them with lvm, this is easier than managing partition. >>> I just open an issue as a feature/enhancement request. >> >> Good idea! >> But first, check my request :-D when implemented I'll try the ansible >> recipe on my cluster > [-- Attachment #2: Message signed with OpenPGP using GPGMail --] [-- Type: application/pgp-signature, Size: 496 bytes --] ^ permalink raw reply [flat|nested] 18+ messages in thread
end of thread, other threads:[~2014-03-07 13:09 UTC | newest] Thread overview: 18+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2014-03-02 23:26 Ceph Ansible Repo Sebastien Han 2014-03-03 2:59 ` Ian Colle 2014-03-03 8:49 ` Sebastien Han 2014-03-03 15:30 ` Travis Rhoden 2014-03-03 17:13 ` Ian Colle 2014-03-03 18:13 ` Sebastien Han 2014-03-06 9:36 ` Gandalf Corvotempesta 2014-03-06 10:14 ` Sebastien Han 2014-03-06 10:28 ` Gandalf Corvotempesta 2014-03-06 10:42 ` Sebastien Han 2014-03-06 10:43 ` Gandalf Corvotempesta 2014-03-06 12:07 ` David McBride 2014-03-06 13:19 ` Gandalf Corvotempesta 2014-03-06 13:28 ` David McBride 2014-03-06 16:31 ` Sebastien Han 2014-03-06 16:45 ` Gandalf Corvotempesta 2014-03-06 18:05 ` Sebastien Han 2014-03-07 13:09 ` Sebastien Han
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.