drbd: and hvm

drbd: and hvm

[James Harper]

What is it that prevents drbd: devices from not working under HVM? Could
the fix just be as simple as mapping 'drbd:name-of-resource' to
'/dev/drbd/by-res/<name-of-resource>'?

James

RE: drbd: and hvm

[James Harper]

> 
> What is it that prevents drbd: devices from not working under HVM?
Could
> the fix just be as simple as mapping 'drbd:name-of-resource' to
> '/dev/drbd/by-res/<name-of-resource>'?
> 

The following patch maps it correctly, but it only works if the device
is already 'primary', which kind of defeats the purpose... qemu-dm must
be trying to open the device before the drbd script switches the local
node to 'primary'. Might it still be a useful addition to qemu-dm
though? (eg it would work in a multiple-primary setup).

James

diff --git a/xenstore.c b/xenstore.c
index 05a1c22..b53f474 100644
--- a/xenstore.c
+++ b/xenstore.c
@@ -513,6 +513,14 @@ void xenstore_parse_domain_config(int hvm_domid)
             params = newparams;
            format = &bdrv_raw;
         }
+        /* handle drbd mapping */
+        if (!strcmp(drv, "drbd")) {
+            char *newparams = malloc(17 + strlen(params) + 1);
+            sprintf(newparams, "/dev/drbd/by-res/%s", params);
+            free(params);
+            params = newparams;
+           format = &bdrv_raw;
+        }

 #if 0
        /* Phantom VBDs are disabled because the use of paths

Re: drbd: and hvm

[Sauro Saltini]

James Harper <james.harper <at> bendigoit.com.au> writes:

> 
> > 
> > What is it that prevents drbd: devices from not working under HVM?
> Could
> > the fix just be as simple as mapping 'drbd:name-of-resource' to
> > '/dev/drbd/by-res/<name-of-resource>'?
> > 
> 
> The following patch maps it correctly, but it only works if the device
> is already 'primary', which kind of defeats the purpose... qemu-dm must
> be trying to open the device before the drbd script switches the local
> node to 'primary'. Might it still be a useful addition to qemu-dm
> though? (eg it would work in a multiple-primary setup).
> 
> James
> 
> diff --git a/xenstore.c b/xenstore.c
> index 05a1c22..b53f474 100644
> --- a/xenstore.c
> +++ b/xenstore.c
> @@ -513,6 +513,14 @@ void xenstore_parse_domain_config(int hvm_domid)
>              params = newparams;
>             format = &bdrv_raw;
>          }
> +        /* handle drbd mapping */
> +        if (!strcmp(drv, "drbd")) {
> +            char *newparams = malloc(17 + strlen(params) + 1);
> +            sprintf(newparams, "/dev/drbd/by-res/%s", params);
> +            free(params);
> +            params = newparams;
> +           format = &bdrv_raw;
> +        }
> 
>  #if 0
>         /* Phantom VBDs are disabled because the use of paths
> 


Hi, I've found your patch digging around, in the hope of making HVM DomU's
working with drbd: syntax and succesfully applied in my installation (Xen 4.0)

I've investigated a bit further, and found that the "xm create" of a such
configured DomU on a drbd "Secondary" node (i.e during initial startup of the
cluster) doesn't work as expected apparently only for timing issues...

In fact the block-drbd script is started and promotes the node to Primary, but
qemu-dm, which is fired a little time before in an asynchronous way, checks for
the resource state too early and finding it still not promoted stops the domU
creation; then the block-drbd is fired again (during DomU stop) and demotes the
node again to secondary.

I've managed to make the creation of domU work by simply putting a sleep(5);
statement in the middle of your conditional block !
In this way the asynchronous block-drbd launch is given the time to promote the
resource while qemu-drbd sleeps! Then the DomU creation can continue just fine.

I've also noticed that the problem happens only for initial creation (xm create)
of the domU, while a migration (even live) can succeed without the need of
adding the sleep statement, carrying out the correct state transitions of drbd
resource both on starting and destination hosts.

Any ideas to elaborate the patch making it less "hackish" than this ?

Sauro Saltini.

RE: Re: drbd: and hvm

[James Harper]

> 
> James Harper <james.harper <at> bendigoit.com.au> writes:
> 
> >
> > >
> > > What is it that prevents drbd: devices from not working under HVM?
> > Could
> > > the fix just be as simple as mapping 'drbd:name-of-resource' to
> > > '/dev/drbd/by-res/<name-of-resource>'?
> > >
> >
> > The following patch maps it correctly, but it only works if the
device
> > is already 'primary', which kind of defeats the purpose... qemu-dm
must
> > be trying to open the device before the drbd script switches the
local
> > node to 'primary'. Might it still be a useful addition to qemu-dm
> > though? (eg it would work in a multiple-primary setup).
> 
> I've managed to make the creation of domU work by simply putting a
sleep(5);
> statement in the middle of your conditional block !

I did exactly the same thing. It might still race on a really heavily
loaded system, but I haven't had a problem with it in a few weeks of
testing.

James

Re: Re: drbd: and hvm

[Sauro Saltini]

I've hit another problem, which seems to be related also to a race 
condition, but this time with Linux PVM's.
I've installed a Slack13 x86_64 PVM which works wery well, when I try to 
migrate (live or not) the PVM is moved on the destination host, but, 
after migration completes, seems hanged... no network, no console, but 
still responds correctly to a "xm shutdown" and closes cleanly.
In this state I've tried also to pause / unpause the VM without any effect.
This happens ONLY if I let block-drbd manage the promotion of drbd 
resource on destination host during the migration !
If I manually promote drbd resource on the destination host just before 
migrating the whole process go well and the PVM is still working on dest 
(not freezed), also the source host is correctly drbd-demoted.

Seems again something like a race condition between block-drbd and VM 
startup...but no qemu-dm now!

Any clues ?

Sauro Saltini


On 30/04/2010 01:44, James Harper wrote:
>> James Harper<james.harper<at>  bendigoit.com.au>  writes:
>>
>>      
>>>        
>>>> What is it that prevents drbd: devices from not working under HVM?
>>>>          
>>> Could
>>>        
>>>> the fix just be as simple as mapping 'drbd:name-of-resource' to
>>>> '/dev/drbd/by-res/<name-of-resource>'?
>>>>
>>>>          
>>> The following patch maps it correctly, but it only works if the
>>>        
> device
>    
>>> is already 'primary', which kind of defeats the purpose... qemu-dm
>>>        
> must
>    
>>> be trying to open the device before the drbd script switches the
>>>        
> local
>    
>>> node to 'primary'. Might it still be a useful addition to qemu-dm
>>> though? (eg it would work in a multiple-primary setup).
>>>        
>> I've managed to make the creation of domU work by simply putting a
>>      
> sleep(5);
>    
>> statement in the middle of your conditional block !
>>      
> I did exactly the same thing. It might still race on a really heavily
> loaded system, but I haven't had a problem with it in a few weeks of
> testing.
>
> James
>
>    

Re: drbd: and hvm

[Kamal Kishore]

Sauro Saltini <saltini <at> shc.it> writes:

> 
> Hi, I've found your patch digging around, in the hope of making HVM DomU's
> working with drbd: syntax and succesfully applied in my installation (Xen 4.0)
> 
> I've investigated a bit further, and found that the "xm create" of a such
> configured DomU on a drbd "Secondary" node (i.e during initial startup of the
> cluster) doesn't work as expected apparently only for timing issues...
> 
> In fact the block-drbd script is started and promotes the node to Primary, but
> qemu-dm, which is fired a little time before in an asynchronous way,
checks for
> the resource state too early and finding it still not promoted stops the domU
> creation; then the block-drbd is fired again (during DomU stop) and
demotes the
> node again to secondary.
> 
> I've managed to make the creation of domU work by simply putting a sleep(5);
> statement in the middle of your conditional block !
> In this way the asynchronous block-drbd launch is given the time to
promote the
> resource while qemu-drbd sleeps! Then the DomU creation can continue just
fine.
> 
> I've also noticed that the problem happens only for initial creation (xm
create)
> of the domU, while a migration (even live) can succeed without the need of
> adding the sleep statement, carrying out the correct state transitions of drbd
> resource both on starting and destination hosts.
> 
> Any ideas to elaborate the patch making it less "hackish" than this ?
> 
> Sauro Saltini.
> 

Hi Sauro Saltini,

           Was trying the same scenario as what you have accomplished.
I'm using XEN 4.1, as i'm not a developer I'm not sure where i can find the
patch you created or how to create one by myself.

Please suggest me the link to find your patch or provide me with the qemu-dm
file with the changes.

Thanks in advance.

Kamal Kishore