Re: problems creating new ceph cluster when using journal on block device

[Mark Nelson <mark.nelson@inktank.com>]

On 11/08/2012 11:36 AM, Travis Rhoden wrote:
> Solved!
>
> I stumbled into the solution while switching from block device to a
> file.  I was being bit by running mkcephfs multiple times -- it wasn't
> really failing on the journal, it was failing because the OSD data
> disk had been initialized before.  I couldn't see that until I used a
> file for the journal and then I see log output like:

Yeah, that was a change that landed a couple of months ago.  It's really 
important now to blow away the old data (I just reformat) if you want a 
totally clean ceph deployment rather than just running mkcephfs.

>
> === osd.0 ===
> 2012-11-08 16:41:37.677620 7ffc3cfcd780 -1 provided osd id 0 != superblock's -1
> 2012-11-08 16:41:37.678726 7ffc3cfcd780 -1  ** ERROR: error creating
> empty object store in /var/lib/ceph/osd/ceph-0: (22) Invalid argument
>
> I unmounted the OSD's that had been touched before, reformatted them,
> and then remounted.  I setup ceph.conf to use block devices for the
> journals, and then everything proceeded normally.
>
> So the final relevant bits from my ceph.conf file look like:
>
> [osd]
>          osd journal size = 0
>          journal dio = true
>          journal aio = true
>
> [osd.0]
>          host = ceph1
>          osd journal = /dev/sda5
>
> [osd.1]
>          host = ceph1
>          osd journal = /dev/sda6
> ...
>
> Thanks,
>
>   - Travis
>
> On Thu, Nov 8, 2012 at 10:08 AM, Travis Rhoden <trhoden@gmail.com> wrote:
>> One more thing -- Google search says this is harmless -- I see quite a
>> few of these in syslog:
>>
>> hdparm: sending ioctl 2285 to a partition!
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>