[Drbd-dev] DRBD-8: weirdness with netlink/connector

[Drbd-dev] DRBD-8: weirdness with netlink/connector

[Graham, Simon]

We have noticed that when issuing a sequence of drbdadm secondary &
primary commands, we get errors from the underlying drbdsetup command
like this:

[root@adam ~]# drbdadm secondary vm1.root.fs
[root@adam ~]# drbdadm primary vm1.root.fs
[root@adam ~]# drbdadm secondary vm1.root.fs
No response from the DRBD driver! Is the module loaded?

I noticed in the code that this error message is produced in a loop that
keeps reading from the netlink socket until it gets the expected reply
and also that there was some commented out trace code that would print
info if an unexpected reply was seen so I enabled this and now see
things like this:

[root@adam ~]# drbdadm secondary vm1.root.fs
[root@adam ~]# drbdadm primary vm1.root.fs
INFO: got other message 
got seq: 110 ; ack 0 
exp seq: 1 ; ack 1849768433 
[root@adam ~]# drbdadm secondary vm1.root.fs
INFO: got other message 
got seq: 111 ; ack 0 
exp seq: 1 ; ack 299691195 
No response from the DRBD driver! Is the module loaded?

This is very reproducible and is presumably either due to the wrong size
messages being sent by the kernel or some sort of data corruption in
drbdsetup - I figured you guys might have a better handle on fixing this
quickly!

Simon

Re: [Drbd-dev] DRBD-8: weirdness with netlink/connector

[Philipp Reisner]

Am Freitag, 13. Oktober 2006 21:02 schrieb Graham, Simon:
> We have noticed that when issuing a sequence of drbdadm secondary &
> primary commands, we get errors from the underlying drbdsetup command
> like this:
>
> [root@adam ~]# drbdadm secondary vm1.root.fs
> [root@adam ~]# drbdadm primary vm1.root.fs
> [root@adam ~]# drbdadm secondary vm1.root.fs
> No response from the DRBD driver! Is the module loaded?
>
> I noticed in the code that this error message is produced in a loop that
> keeps reading from the netlink socket until it gets the expected reply
> and also that there was some commented out trace code that would print
> info if an unexpected reply was seen so I enabled this and now see
> things like this:
>
> [root@adam ~]# drbdadm secondary vm1.root.fs
> [root@adam ~]# drbdadm primary vm1.root.fs
> INFO: got other message
> got seq: 110 ; ack 0
> exp seq: 1 ; ack 1849768433
> [root@adam ~]# drbdadm secondary vm1.root.fs
> INFO: got other message
> got seq: 111 ; ack 0
> exp seq: 1 ; ack 299691195
> No response from the DRBD driver! Is the module loaded?
>
> This is very reproducible and is presumably either due to the wrong size
> messages being sent by the kernel or some sort of data corruption in
> drbdsetup - I figured you guys might have a better handle on fixing this
> quickly!

Hi Simon,

I also noticed this already. It is simply a too small timeout in drbdsetup.
I just changed it from 300ms to 5 seconds. 

BTW, the "other messages" are the status events broad casted by DRBD 
on every state change. It is ok, that drbdsetup gets those messages.

-Phil
-- 
: Dipl-Ing Philipp Reisner                      Tel +43-1-8178292-50 :
: LINBIT Information Technologies GmbH          Fax +43-1-8178292-82 :
: Schönbrunnerstr 244, 1120 Vienna, Austria    http://www.linbit.com :