XCP: python SR plugin questions

XCP: python SR plugin questions

[YAMAMOTO Takashi]

hi,

i'm writing a python sr plugin for our product and have a few questions.

- our product has 32-bit id, not uuid, for each volumes, which i want to map
  to xcp's vdi.  i generate name-based uuid from the 32-bit id in sr.scan and
  vdi.create.  is this a reasonable approach?

- how can i get name-label specified by "xe vdi-create" in my vdi.create?
  i want to pass it to our product.

if you are intersted, the "our product" is found at
http://sourceforge.net/projects/vastsky/

YAMAMOTO Takashi

Re: XCP: python SR plugin questions

[Jonathan Ludlam]

Hi there,

On 19 May 2010, at 03:35, YAMAMOTO Takashi wrote:

> hi,
> 
> i'm writing a python sr plugin for our product and have a few questions.
> 
> - our product has 32-bit id, not uuid, for each volumes, which i want to map
>  to xcp's vdi.  i generate name-based uuid from the 32-bit id in sr.scan and
>  vdi.create.  is this a reasonable approach?
> 

You can use the 'location' field in the VDI to contain your 32 bit id - then the uuid is irrelevant. I'd like to move more of the storage backends in this direction - currently I believe only the ISO SR does it this way.

> - how can i get name-label specified by "xe vdi-create" in my vdi.create?
>  i want to pass it to our product.
> 

Unfortunately this isn't possible at the moment. What you could do is pick the name up on a subsequent scan. This is ugly, I know, and might not even work if you need the name at VDI create time, but it's all we've got at the moment. This is on my list of things to fix about the SMAPI.

> if you are intersted, the "our product" is found at
> http://sourceforge.net/projects/vastsky/
> 
> YAMAMOTO Takashi
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

Re: XCP: python SR plugin questions

[YAMAMOTO Takashi]

hi,

thanks for quick reply.

> Hi there,
> 
> On 19 May 2010, at 03:35, YAMAMOTO Takashi wrote:
> 
>> hi,
>> 
>> i'm writing a python sr plugin for our product and have a few questions.
>> 
>> - our product has 32-bit id, not uuid, for each volumes, which i want to map
>>  to xcp's vdi.  i generate name-based uuid from the 32-bit id in sr.scan and
>>  vdi.create.  is this a reasonable approach?
>> 
> 
> You can use the 'location' field in the VDI to contain your 32 bit id - then the uuid is irrelevant. I'd like to move more of the storage backends in this direction - currently I believe only the ISO SR does it this way.

ok.  in that case, xapi generates random uuids for me, right?

> 
>> - how can i get name-label specified by "xe vdi-create" in my vdi.create?
>>  i want to pass it to our product.
>> 
> 
> Unfortunately this isn't possible at the moment. What you could do is pick the name up on a subsequent scan. This is ugly, I know, and might not even work if you need the name at VDI create time, but it's all we've got at the moment. This is on my list of things to fix about the SMAPI.

sad to hear. :(

our product needs a unique and preferably human readable name for
the VDI at create time.  do you have any suggestion?
probably i have to use uuid generated by xapi?

YAMAMOTO Takashi

> 
>> if you are intersted, the "our product" is found at
>> http://sourceforge.net/projects/vastsky/
>> 
>> YAMAMOTO Takashi
>> 
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xensource.com
>> http://lists.xensource.com/xen-devel

Re: XCP: python SR plugin questions

[Jonathan Ludlam]

On 19 May 2010, at 10:07, YAMAMOTO Takashi wrote:

> hi,
> 
> thanks for quick reply.
> 
>> Hi there,
>> 
>> On 19 May 2010, at 03:35, YAMAMOTO Takashi wrote:
>> 
>>> hi,
>>> 
>>> i'm writing a python sr plugin for our product and have a few questions.
>>> 
>>> - our product has 32-bit id, not uuid, for each volumes, which i want to map
>>> to xcp's vdi.  i generate name-based uuid from the 32-bit id in sr.scan and
>>> vdi.create.  is this a reasonable approach?
>>> 
>> 
>> You can use the 'location' field in the VDI to contain your 32 bit id - then the uuid is irrelevant. I'd like to move more of the storage backends in this direction - currently I believe only the ISO SR does it this way.
> 
> ok.  in that case, xapi generates random uuids for me, right?
> 

Actually the backends are responsible for creating the uuid.

>> 
>>> - how can i get name-label specified by "xe vdi-create" in my vdi.create?
>>> i want to pass it to our product.
>>> 
>> 
>> Unfortunately this isn't possible at the moment. What you could do is pick the name up on a subsequent scan. This is ugly, I know, and might not even work if you need the name at VDI create time, but it's all we've got at the moment. This is on my list of things to fix about the SMAPI.
> 
> sad to hear. :(
> 
> our product needs a unique and preferably human readable name for
> the VDI at create time.  do you have any suggestion?
> probably i have to use uuid generated by xapi?
> 

Unfortunately there's no alternative at the moment :-(

> YAMAMOTO Takashi
> 
>> 
>>> if you are intersted, the "our product" is found at
>>> http://sourceforge.net/projects/vastsky/
>>> 
>>> YAMAMOTO Takashi
>>> 
>>> _______________________________________________
>>> Xen-devel mailing list
>>> Xen-devel@lists.xensource.com
>>> http://lists.xensource.com/xen-devel

Re: XCP: python SR plugin questions

[YAMAMOTO Takashi]

hi,

>>>> i'm writing a python sr plugin for our product and have a few questions.
>>>> 
>>>> - our product has 32-bit id, not uuid, for each volumes, which i want to map
>>>> to xcp's vdi.  i generate name-based uuid from the 32-bit id in sr.scan and
>>>> vdi.create.  is this a reasonable approach?
>>>> 
>>> 
>>> You can use the 'location' field in the VDI to contain your 32 bit id - then the uuid is irrelevant. I'd like to move more of the storage backends in this direction - currently I believe only the ISO SR does it this way.
>> 
>> ok.  in that case, xapi generates random uuids for me, right?
>> 
> 
> Actually the backends are responsible for creating the uuid.

then, what is vdi_create's uuid argument for?
(my current plugin code simply ignores the argument and
return generated uuid via vdi_info.  is it ok?)

our product itself doesn't provide uuids at all.  so someone should
generate ones so that xapi can use them.
i plan to make my sr plugin generate name-based uuid from our product's
32-bit id.  does it sound ok for you?

i'm not sure how your suggestion to use vdi location is related to
the uuid issue.  can you please elaborate a bit?

YAMAMOTO Takashi

Re: XCP: python SR plugin questions

[Jonathan Ludlam]

Sure.

Here's a sample vdi_create call:

<methodCall>
  <methodName>vdi_create</methodName>
  <params>
    <param>
      <value>
        <struct>
          <member>
            <name>host_ref</name>
            <value>OpaqueRef:1a69645b-8981-b076-3439-ae40715168cb</value>
          </member>
          <member>
            <name>command</name>
            <value>vdi_create</value>
          </member>
          <member>
            <name>args</name>
            <value>
              <array>
                <data>
                  <value>104857600</value>
                </data>
              </array>
            </value>
          </member>
          <member>
            <name>device_config</name>
            <value>
              <struct>
                <member>
                  <name>SRmaster</name>
                  <value>true</value>
                </member>
                <member>
                  <name>device</name>
                  <value>/dev/sda3</value>
                </member>
              </struct>
            </value>
          </member>
          <member>
            <name>session_ref</name>
            <value>OpaqueRef:bb4ba642-1427-513d-f398-d07a07b56157</value>
          </member>
          <member>
            <name>sr_ref</name>
            <value>OpaqueRef:2fa93049-ee45-284c-e657-067e0c181789</value>
          </member>
          <member>
            <name>sr_uuid</name>
            <value>668da5be-32e8-1fac-6d1f-7590e2fa1c09</value>
          </member>
          <member>
            <name>vdi_sm_config</name>
            <value>
              <struct/>
            </value>
          </member>
          <member>
            <name>subtask_of</name>
            <value>OpaqueRef:cb7e281b-8430-1979-e36a-5dd17dab5c72</value>
          </member>
        </struct>
      </value>
    </param>
  </params>
</methodCall>

There is only a sr_uuid here, and no vdi_uuid at all. 

The way it works is:

1. Xapi receives a VDI.create XenAPI call
2. Xapi checks the SR, figures out which host should do the operation, checks it has its PBD attached and various other checks
3. On the host in question, Xapi calls out to the SM backend to do a vdi_create operation. It only tells the SM backend the size of the disk and the 'sm_config' map.
4. The SM backend creates the actual object - a vhd, or an ISO, or an LVM partition or whatever
5. The SM backend then introduces a VDI record using the XenAPI call 'VDI.db_introduce' - this API call takes a uuid which was made up by the SM backend. Xapi ensures that the uuid is globally unique and that the location field is unique within the SR. 
6. The SM backend returns control to Xapi, returning a small structure containing the uuid that it has just made up as well as the location.
7. Xapi looks up the VDI by uuid, and then overwrites several of the fields (including name_label) with the parameters of the XenAPI VDI.create call (from step 1).
8. The VDI.create call finishes, returning the reference of the VDI object.

On all subsequent calls to the backend associated with this VDI (vdi_attach, vdi_clone, etc.), Xapi will always pass in the reference, uuid and location of the VDI:

...
          <member>
            <name>vdi_ref</name>
            <value>OpaqueRef:67b70c91-8d6d-7c09-2a68-21aa5b3e3d07</value>
          </member>
          <member>
            <name>vdi_location</name>
            <value>f6397206-9dcb-f649-f893-a54de0ec60e4</value>
          </member>
          <member>
            <name>vdi_uuid</name>
            <value>1399962e-76d1-9303-86e5-98ae75708116</value>
          </member>
...

The idea is that the location field is used to identify which VDI xapi is talking about. For you, in the vdi_create call, the _db_introduce function would create
the VDI record with your 32 bit id in the location field. All subsequent calls about that VDI will contain that location field as well as the VDI's uuid, so the uuid isn't
really important as far as your backend is concerned - all you need is the location to identify your VDI. You *could* base the uuid of the VDI on your 32 bit id, but it's not necessary.

Hope this helps,

Jon


On 20 May 2010, at 02:46, YAMAMOTO Takashi wrote:

> hi,
> 
>>>>> i'm writing a python sr plugin for our product and have a few questions.
>>>>> 
>>>>> - our product has 32-bit id, not uuid, for each volumes, which i want to map
>>>>> to xcp's vdi.  i generate name-based uuid from the 32-bit id in sr.scan and
>>>>> vdi.create.  is this a reasonable approach?
>>>>> 
>>>> 
>>>> You can use the 'location' field in the VDI to contain your 32 bit id - then the uuid is irrelevant. I'd like to move more of the storage backends in this direction - currently I believe only the ISO SR does it this way.
>>> 
>>> ok.  in that case, xapi generates random uuids for me, right?
>>> 
>> 
>> Actually the backends are responsible for creating the uuid.
> 
> then, what is vdi_create's uuid argument for?
> (my current plugin code simply ignores the argument and
> return generated uuid via vdi_info.  is it ok?)
> 
> our product itself doesn't provide uuids at all.  so someone should
> generate ones so that xapi can use them.
> i plan to make my sr plugin generate name-based uuid from our product's
> 32-bit id.  does it sound ok for you?
> 
> i'm not sure how your suggestion to use vdi location is related to
> the uuid issue.  can you please elaborate a bit?
> 
> YAMAMOTO Takashi

Re: XCP: python SR plugin questions

[YAMAMOTO Takashi]

hi,

thanks for the detailed explanation!

YAMAMOTO Takashi

> Sure.
> 
> Here's a sample vdi_create call:
> 
> <methodCall>
>   <methodName>vdi_create</methodName>
>   <params>
>     <param>
>       <value>
>         <struct>
>           <member>
>             <name>host_ref</name>
>             <value>OpaqueRef:1a69645b-8981-b076-3439-ae40715168cb</value>
>           </member>
>           <member>
>             <name>command</name>
>             <value>vdi_create</value>
>           </member>
>           <member>
>             <name>args</name>
>             <value>
>               <array>
>                 <data>
>                   <value>104857600</value>
>                 </data>
>               </array>
>             </value>
>           </member>
>           <member>
>             <name>device_config</name>
>             <value>
>               <struct>
>                 <member>
>                   <name>SRmaster</name>
>                   <value>true</value>
>                 </member>
>                 <member>
>                   <name>device</name>
>                   <value>/dev/sda3</value>
>                 </member>
>               </struct>
>             </value>
>           </member>
>           <member>
>             <name>session_ref</name>
>             <value>OpaqueRef:bb4ba642-1427-513d-f398-d07a07b56157</value>
>           </member>
>           <member>
>             <name>sr_ref</name>
>             <value>OpaqueRef:2fa93049-ee45-284c-e657-067e0c181789</value>
>           </member>
>           <member>
>             <name>sr_uuid</name>
>             <value>668da5be-32e8-1fac-6d1f-7590e2fa1c09</value>
>           </member>
>           <member>
>             <name>vdi_sm_config</name>
>             <value>
>               <struct/>
>             </value>
>           </member>
>           <member>
>             <name>subtask_of</name>
>             <value>OpaqueRef:cb7e281b-8430-1979-e36a-5dd17dab5c72</value>
>           </member>
>         </struct>
>       </value>
>     </param>
>   </params>
> </methodCall>
> 
> There is only a sr_uuid here, and no vdi_uuid at all. 
> 
> The way it works is:
> 
> 1. Xapi receives a VDI.create XenAPI call
> 2. Xapi checks the SR, figures out which host should do the operation, checks it has its PBD attached and various other checks
> 3. On the host in question, Xapi calls out to the SM backend to do a vdi_create operation. It only tells the SM backend the size of the disk and the 'sm_config' map.
> 4. The SM backend creates the actual object - a vhd, or an ISO, or an LVM partition or whatever
> 5. The SM backend then introduces a VDI record using the XenAPI call 'VDI.db_introduce' - this API call takes a uuid which was made up by the SM backend. Xapi ensures that the uuid is globally unique and that the location field is unique within the SR. 
> 6. The SM backend returns control to Xapi, returning a small structure containing the uuid that it has just made up as well as the location.
> 7. Xapi looks up the VDI by uuid, and then overwrites several of the fields (including name_label) with the parameters of the XenAPI VDI.create call (from step 1).
> 8. The VDI.create call finishes, returning the reference of the VDI object.
> 
> On all subsequent calls to the backend associated with this VDI (vdi_attach, vdi_clone, etc.), Xapi will always pass in the reference, uuid and location of the VDI:
> 
> ...
>           <member>
>             <name>vdi_ref</name>
>             <value>OpaqueRef:67b70c91-8d6d-7c09-2a68-21aa5b3e3d07</value>
>           </member>
>           <member>
>             <name>vdi_location</name>
>             <value>f6397206-9dcb-f649-f893-a54de0ec60e4</value>
>           </member>
>           <member>
>             <name>vdi_uuid</name>
>             <value>1399962e-76d1-9303-86e5-98ae75708116</value>
>           </member>
> ...
> 
> The idea is that the location field is used to identify which VDI xapi is talking about. For you, in the vdi_create call, the _db_introduce function would create
> the VDI record with your 32 bit id in the location field. All subsequent calls about that VDI will contain that location field as well as the VDI's uuid, so the uuid isn't
> really important as far as your backend is concerned - all you need is the location to identify your VDI. You *could* base the uuid of the VDI on your 32 bit id, but it's not necessary.
> 
> Hope this helps,
> 
> Jon
> 
> 
> On 20 May 2010, at 02:46, YAMAMOTO Takashi wrote:
> 
>> hi,
>> 
>>>>>> i'm writing a python sr plugin for our product and have a few questions.
>>>>>> 
>>>>>> - our product has 32-bit id, not uuid, for each volumes, which i want to map
>>>>>> to xcp's vdi.  i generate name-based uuid from the 32-bit id in sr.scan and
>>>>>> vdi.create.  is this a reasonable approach?
>>>>>> 
>>>>> 
>>>>> You can use the 'location' field in the VDI to contain your 32 bit id - then the uuid is irrelevant. I'd like to move more of the storage backends in this direction - currently I believe only the ISO SR does it this way.
>>>> 
>>>> ok.  in that case, xapi generates random uuids for me, right?
>>>> 
>>> 
>>> Actually the backends are responsible for creating the uuid.
>> 
>> then, what is vdi_create's uuid argument for?
>> (my current plugin code simply ignores the argument and
>> return generated uuid via vdi_info.  is it ok?)
>> 
>> our product itself doesn't provide uuids at all.  so someone should
>> generate ones so that xapi can use them.
>> i plan to make my sr plugin generate name-based uuid from our product's
>> 32-bit id.  does it sound ok for you?
>> 
>> i'm not sure how your suggestion to use vdi location is related to
>> the uuid issue.  can you please elaborate a bit?
>> 
>> YAMAMOTO Takashi
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel