Ho to identify location of Primary Copy Of Obj ?

Ho to identify location of Primary Copy Of Obj ?

[hemant surale]

Hi Community,
          Is it possible to identify where exactly primary copy of obj
is stored ? I am using crushmaps to use specific osds for data
placement but i want to knw the primary capoy location. Or I need to
replace pseudo random function by some deterministic function to guide
ceph to utilize specific osd?


Regards,
Hemant Surale.

Re: Ho to identify location of Primary Copy Of Obj ?

[Dan Mick]

Hemant:

Yes, you can.  Use ceph osd getmap -o <file> to get the OSD map, and 
then use osdmaptool --find-object-map <objectname> <file> to output the
PG the object hashes to and the list of OSDs that PG maps to (primary 
first):

$ ceph osd getmap -o osdmap
got osdmap epoch 59
$ osdmaptool --test-map-object dmick.rbd osdmap
osdmaptool: osdmap file 'osdmap'
  object 'dmick.rbd' -> 0.69c8 -> [3,1]

shows dmick.rbd mapping to pg 0.69c8, which in turn maps to OSDs 3 and 
1, 3 being the primary.

On 09/25/2012 02:30 AM, hemant surale wrote:
> Hi Community,
>            Is it possible to identify where exactly primary copy of obj
> is stored ? I am using crushmaps to use specific osds for data
> placement but i want to knw the primary capoy location. Or I need to
> replace pseudo random function by some deterministic function to guide
> ceph to utilize specific osd?
>
>
> Regards,
> Hemant Surale.
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Re: Ho to identify location of Primary Copy Of Obj ?

[Sage Weil]

On Tue, 25 Sep 2012, Dan Mick wrote:
> Hemant:
> 
> Yes, you can.  Use ceph osd getmap -o <file> to get the OSD map, and then use
> osdmaptool --find-object-map <objectname> <file> to output the
> PG the object hashes to and the list of OSDs that PG maps to (primary first):
> 
> $ ceph osd getmap -o osdmap
> got osdmap epoch 59
> $ osdmaptool --test-map-object dmick.rbd osdmap
> osdmaptool: osdmap file 'osdmap'
>  object 'dmick.rbd' -> 0.69c8 -> [3,1]
> 
> shows dmick.rbd mapping to pg 0.69c8, which in turn maps to OSDs 3 and 1, 3
> being the primary.

You can do this more simply with:

 ceph osd map <poolname> <objectname>

sage

> 
> On 09/25/2012 02:30 AM, hemant surale wrote:
> > Hi Community,
> >            Is it possible to identify where exactly primary copy of obj
> > is stored ? I am using crushmaps to use specific osds for data
> > placement but i want to knw the primary capoy location. Or I need to
> > replace pseudo random function by some deterministic function to guide
> > ceph to utilize specific osd?
> > 
> > 
> > Regards,
> > Hemant Surale.
> > --
> > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> > 
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
> 

Re: Ho to identify location of Primary Copy Of Obj ?

[hemant surale]

Hi Sage, Dan ,
       When I tried to execute cmd "  osdmaptool --test-map-object
"anyrandomname" osdmap  " it still shows some random number as mapped
osds. Because I havnt created Object even for that result is shown as
some random numbers/ as osds where that Object is mapped.
      And with replication factor of 3 , I think " osdmaptool
--test-map-object  Objname osdmap" should show 3 nodes at least . But
currently it shows only 1 node .

        Please help me out .

Thanks & Regards,
Hemant Surale.


On Wed, Sep 26, 2012 at 2:59 AM, Sage Weil <sage@inktank.com> wrote:
> On Tue, 25 Sep 2012, Dan Mick wrote:
>> Hemant:
>>
>> Yes, you can.  Use ceph osd getmap -o <file> to get the OSD map, and then use
>> osdmaptool --find-object-map <objectname> <file> to output the
>> PG the object hashes to and the list of OSDs that PG maps to (primary first):
>>
>> $ ceph osd getmap -o osdmap
>> got osdmap epoch 59
>> $ osdmaptool --test-map-object dmick.rbd osdmap
>> osdmaptool: osdmap file 'osdmap'
>>  object 'dmick.rbd' -> 0.69c8 -> [3,1]
>>
>> shows dmick.rbd mapping to pg 0.69c8, which in turn maps to OSDs 3 and 1, 3
>> being the primary.
>
> You can do this more simply with:
>
>  ceph osd map <poolname> <objectname>
>
> sage
>
>>
>> On 09/25/2012 02:30 AM, hemant surale wrote:
>> > Hi Community,
>> >            Is it possible to identify where exactly primary copy of obj
>> > is stored ? I am using crushmaps to use specific osds for data
>> > placement but i want to knw the primary capoy location. Or I need to
>> > replace pseudo random function by some deterministic function to guide
>> > ceph to utilize specific osd?
>> >
>> >
>> > Regards,
>> > Hemant Surale.
>> > --
>> > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
>> > the body of a message to majordomo@vger.kernel.org
>> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
>> >
>> --
>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
>>

Re: Ho to identify location of Primary Copy Of Obj ?

[hemant surale]

Hi Dan ,
     I have set replication factor to 3 of pool 'newbyh' . Then when i
tried to execute cmds told by you I got that it reported like

----------------------------------------------------
root@third-virtual-machine:~# osdmaptool --test-map-object Obj1 osdmap
osdmaptool: osdmap file 'osdmap'
 object 'Obj1' -> 0.c3c4 -> [0,1]
-----------------------------------------------------

I even checked manually the dirs at every node . it shows proper data
available within osd0,osd1 & osd2 . ( i hve 3 node cluster using VM
Ceph v0.36 )

So my questions is why in above execution it shows that Obj1 is at
[0,1] .. it should report all nodes like [0,1,2] .



-
Hemant Surale.

On Wed, Sep 26, 2012 at 2:04 AM, Dan Mick <dan.mick@inktank.com> wrote:
> Hemant:
>
> Yes, you can.  Use ceph osd getmap -o <file> to get the OSD map, and then
> use osdmaptool --find-object-map <objectname> <file> to output the
> PG the object hashes to and the list of OSDs that PG maps to (primary
> first):
>
> $ ceph osd getmap -o osdmap
> got osdmap epoch 59
> $ osdmaptool --test-map-object dmick.rbd osdmap
> osdmaptool: osdmap file 'osdmap'
>  object 'dmick.rbd' -> 0.69c8 -> [3,1]
>
> shows dmick.rbd mapping to pg 0.69c8, which in turn maps to OSDs 3 and 1, 3
> being the primary.
>
>
> On 09/25/2012 02:30 AM, hemant surale wrote:
>>
>> Hi Community,
>>            Is it possible to identify where exactly primary copy of obj
>> is stored ? I am using crushmaps to use specific osds for data
>> placement but i want to knw the primary capoy location. Or I need to
>> replace pseudo random function by some deterministic function to guide
>> ceph to utilize specific osd?
>>
>>
>> Regards,
>> Hemant Surale.
>> --
>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
>

Re: Ho to identify location of Primary Copy Of Obj ?

[Sage Weil]

On Wed, 26 Sep 2012, hemant surale wrote:
> Hi Dan ,
>      I have set replication factor to 3 of pool 'newbyh' . Then when i
> tried to execute cmds told by you I got that it reported like
> 
> ----------------------------------------------------
> root@third-virtual-machine:~# osdmaptool --test-map-object Obj1 osdmap
> osdmaptool: osdmap file 'osdmap'
>  object 'Obj1' -> 0.c3c4 -> [0,1]
> -----------------------------------------------------
> 
> I even checked manually the dirs at every node . it shows proper data
> available within osd0,osd1 & osd2 . ( i hve 3 node cluster using VM
> Ceph v0.36 )
> 
> So my questions is why in above execution it shows that Obj1 is at
> [0,1] .. it should report all nodes like [0,1,2] .

The --test-map-object is currently somewhat useless because it assumes 
pool 0 ('data'), and your object is probably in a different pool.

sage

> 
> 
> 
> -
> Hemant Surale.
> 
> On Wed, Sep 26, 2012 at 2:04 AM, Dan Mick <dan.mick@inktank.com> wrote:
> > Hemant:
> >
> > Yes, you can.  Use ceph osd getmap -o <file> to get the OSD map, and then
> > use osdmaptool --find-object-map <objectname> <file> to output the
> > PG the object hashes to and the list of OSDs that PG maps to (primary
> > first):
> >
> > $ ceph osd getmap -o osdmap
> > got osdmap epoch 59
> > $ osdmaptool --test-map-object dmick.rbd osdmap
> > osdmaptool: osdmap file 'osdmap'
> >  object 'dmick.rbd' -> 0.69c8 -> [3,1]
> >
> > shows dmick.rbd mapping to pg 0.69c8, which in turn maps to OSDs 3 and 1, 3
> > being the primary.
> >
> >
> > On 09/25/2012 02:30 AM, hemant surale wrote:
> >>
> >> Hi Community,
> >>            Is it possible to identify where exactly primary copy of obj
> >> is stored ? I am using crushmaps to use specific osds for data
> >> placement but i want to knw the primary capoy location. Or I need to
> >> replace pseudo random function by some deterministic function to guide
> >> ceph to utilize specific osd?
> >>
> >>
> >> Regards,
> >> Hemant Surale.
> >> --
> >> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> >> the body of a message to majordomo@vger.kernel.org
> >> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> >>
> >
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
> 

Re: Ho to identify location of Primary Copy Of Obj ?

[Dan Mick]

Ah, yeah, that assumption would be a problem.

So, Hemant, does
	ceph osd dump <poolname> <objectname>

show you information that makes sense?

On 09/26/2012 08:21 AM, Sage Weil wrote:
> On Wed, 26 Sep 2012, hemant surale wrote:
>> Hi Dan ,
>>       I have set replication factor to 3 of pool 'newbyh' . Then when i
>> tried to execute cmds told by you I got that it reported like
>>
>> ----------------------------------------------------
>> root@third-virtual-machine:~# osdmaptool --test-map-object Obj1 osdmap
>> osdmaptool: osdmap file 'osdmap'
>>   object 'Obj1' -> 0.c3c4 -> [0,1]
>> -----------------------------------------------------
>>
>> I even checked manually the dirs at every node . it shows proper data
>> available within osd0,osd1 & osd2 . ( i hve 3 node cluster using VM
>> Ceph v0.36 )
>>
>> So my questions is why in above execution it shows that Obj1 is at
>> [0,1] .. it should report all nodes like [0,1,2] .
>
> The --test-map-object is currently somewhat useless because it assumes
> pool 0 ('data'), and your object is probably in a different pool.
>
> sage
>
>>
>>
>>
>> -
>> Hemant Surale.
>>
>> On Wed, Sep 26, 2012 at 2:04 AM, Dan Mick <dan.mick@inktank.com> wrote:
>>> Hemant:
>>>
>>> Yes, you can.  Use ceph osd getmap -o <file> to get the OSD map, and then
>>> use osdmaptool --find-object-map <objectname> <file> to output the
>>> PG the object hashes to and the list of OSDs that PG maps to (primary
>>> first):
>>>
>>> $ ceph osd getmap -o osdmap
>>> got osdmap epoch 59
>>> $ osdmaptool --test-map-object dmick.rbd osdmap
>>> osdmaptool: osdmap file 'osdmap'
>>>   object 'dmick.rbd' -> 0.69c8 -> [3,1]
>>>
>>> shows dmick.rbd mapping to pg 0.69c8, which in turn maps to OSDs 3 and 1, 3
>>> being the primary.
>>>
>>>
>>> On 09/25/2012 02:30 AM, hemant surale wrote:
>>>>
>>>> Hi Community,
>>>>             Is it possible to identify where exactly primary copy of obj
>>>> is stored ? I am using crushmaps to use specific osds for data
>>>> placement but i want to knw the primary capoy location. Or I need to
>>>> replace pseudo random function by some deterministic function to guide
>>>> ceph to utilize specific osd?
>>>>
>>>>
>>>> Regards,
>>>> Hemant Surale.
>>>> --
>>>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
>>>> the body of a message to majordomo@vger.kernel.org
>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>>
>>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
>>

Re: Ho to identify location of Primary Copy Of Obj ?

[Sage Weil]

On Wed, 26 Sep 2012, Dan Mick wrote:
> Ah, yeah, that assumption would be a problem.
> 
> So, Hemant, does
> 	ceph osd dump <poolname> <objectname>

Ahem,

	ceph osd map <poolname> <objectname>

:) sage

> 
> show you information that makes sense?
> 
> On 09/26/2012 08:21 AM, Sage Weil wrote:
> > On Wed, 26 Sep 2012, hemant surale wrote:
> > > Hi Dan ,
> > >       I have set replication factor to 3 of pool 'newbyh' . Then when i
> > > tried to execute cmds told by you I got that it reported like
> > > 
> > > ----------------------------------------------------
> > > root@third-virtual-machine:~# osdmaptool --test-map-object Obj1 osdmap
> > > osdmaptool: osdmap file 'osdmap'
> > >   object 'Obj1' -> 0.c3c4 -> [0,1]
> > > -----------------------------------------------------
> > > 
> > > I even checked manually the dirs at every node . it shows proper data
> > > available within osd0,osd1 & osd2 . ( i hve 3 node cluster using VM
> > > Ceph v0.36 )
> > > 
> > > So my questions is why in above execution it shows that Obj1 is at
> > > [0,1] .. it should report all nodes like [0,1,2] .
> > 
> > The --test-map-object is currently somewhat useless because it assumes
> > pool 0 ('data'), and your object is probably in a different pool.
> > 
> > sage
> > 
> > > 
> > > 
> > > 
> > > -
> > > Hemant Surale.
> > > 
> > > On Wed, Sep 26, 2012 at 2:04 AM, Dan Mick <dan.mick@inktank.com> wrote:
> > > > Hemant:
> > > > 
> > > > Yes, you can.  Use ceph osd getmap -o <file> to get the OSD map, and
> > > > then
> > > > use osdmaptool --find-object-map <objectname> <file> to output the
> > > > PG the object hashes to and the list of OSDs that PG maps to (primary
> > > > first):
> > > > 
> > > > $ ceph osd getmap -o osdmap
> > > > got osdmap epoch 59
> > > > $ osdmaptool --test-map-object dmick.rbd osdmap
> > > > osdmaptool: osdmap file 'osdmap'
> > > >   object 'dmick.rbd' -> 0.69c8 -> [3,1]
> > > > 
> > > > shows dmick.rbd mapping to pg 0.69c8, which in turn maps to OSDs 3 and
> > > > 1, 3
> > > > being the primary.
> > > > 
> > > > 
> > > > On 09/25/2012 02:30 AM, hemant surale wrote:
> > > > > 
> > > > > Hi Community,
> > > > >             Is it possible to identify where exactly primary copy of
> > > > > obj
> > > > > is stored ? I am using crushmaps to use specific osds for data
> > > > > placement but i want to knw the primary capoy location. Or I need to
> > > > > replace pseudo random function by some deterministic function to guide
> > > > > ceph to utilize specific osd?
> > > > > 
> > > > > 
> > > > > Regards,
> > > > > Hemant Surale.
> > > > > --
> > > > > To unsubscribe from this list: send the line "unsubscribe ceph-devel"
> > > > > in
> > > > > the body of a message to majordomo@vger.kernel.org
> > > > > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> > > > > 
> > > > 
> > > --
> > > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> > > the body of a message to majordomo@vger.kernel.org
> > > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> > > 
> > > 
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
> 

Re: Ho to identify location of Primary Copy Of Obj ?

[Dan Mick]

AGH!  So sorry Hemant.  I really was thinking 'map' when I typed that.

On 09/26/2012 04:41 PM, Sage Weil wrote:
> On Wed, 26 Sep 2012, Dan Mick wrote:
>> Ah, yeah, that assumption would be a problem.
>>
>> So, Hemant, does
>> 	ceph osd dump <poolname> <objectname>
>
> Ahem,
>
> 	ceph osd map <poolname> <objectname>
>
> :) sage
>
>>
>> show you information that makes sense?
>>
>> On 09/26/2012 08:21 AM, Sage Weil wrote:
>>> On Wed, 26 Sep 2012, hemant surale wrote:
>>>> Hi Dan ,
>>>>        I have set replication factor to 3 of pool 'newbyh' . Then when i
>>>> tried to execute cmds told by you I got that it reported like
>>>>
>>>> ----------------------------------------------------
>>>> root@third-virtual-machine:~# osdmaptool --test-map-object Obj1 osdmap
>>>> osdmaptool: osdmap file 'osdmap'
>>>>    object 'Obj1' -> 0.c3c4 -> [0,1]
>>>> -----------------------------------------------------
>>>>
>>>> I even checked manually the dirs at every node . it shows proper data
>>>> available within osd0,osd1 & osd2 . ( i hve 3 node cluster using VM
>>>> Ceph v0.36 )
>>>>
>>>> So my questions is why in above execution it shows that Obj1 is at
>>>> [0,1] .. it should report all nodes like [0,1,2] .
>>>
>>> The --test-map-object is currently somewhat useless because it assumes
>>> pool 0 ('data'), and your object is probably in a different pool.
>>>
>>> sage
>>>
>>>>
>>>>
>>>>
>>>> -
>>>> Hemant Surale.
>>>>
>>>> On Wed, Sep 26, 2012 at 2:04 AM, Dan Mick <dan.mick@inktank.com> wrote:
>>>>> Hemant:
>>>>>
>>>>> Yes, you can.  Use ceph osd getmap -o <file> to get the OSD map, and
>>>>> then
>>>>> use osdmaptool --find-object-map <objectname> <file> to output the
>>>>> PG the object hashes to and the list of OSDs that PG maps to (primary
>>>>> first):
>>>>>
>>>>> $ ceph osd getmap -o osdmap
>>>>> got osdmap epoch 59
>>>>> $ osdmaptool --test-map-object dmick.rbd osdmap
>>>>> osdmaptool: osdmap file 'osdmap'
>>>>>    object 'dmick.rbd' -> 0.69c8 -> [3,1]
>>>>>
>>>>> shows dmick.rbd mapping to pg 0.69c8, which in turn maps to OSDs 3 and
>>>>> 1, 3
>>>>> being the primary.
>>>>>
>>>>>
>>>>> On 09/25/2012 02:30 AM, hemant surale wrote:
>>>>>>
>>>>>> Hi Community,
>>>>>>              Is it possible to identify where exactly primary copy of
>>>>>> obj
>>>>>> is stored ? I am using crushmaps to use specific osds for data
>>>>>> placement but i want to knw the primary capoy location. Or I need to
>>>>>> replace pseudo random function by some deterministic function to guide
>>>>>> ceph to utilize specific osd?
>>>>>>
>>>>>>
>>>>>> Regards,
>>>>>> Hemant Surale.
>>>>>> --
>>>>>> To unsubscribe from this list: send the line "unsubscribe ceph-devel"
>>>>>> in
>>>>>> the body of a message to majordomo@vger.kernel.org
>>>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>>>>
>>>>>
>>>> --
>>>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
>>>> the body of a message to majordomo@vger.kernel.org
>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>>
>>>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
>>