Potential race in dlm based messaging md-cluster.c

[Abhijit Bhopatkar <abhopatk@cisco.com>]

There is a possibility of a receiver losing out on messages in certain 
corner conditions. One of the buggy case is if there is are two sender 
ready with messages to be sent. Sender 1 initially gets the TOKEN lock 
and proceeds.
After initial processing the sender of message 1 _will_ release TOKEN as 
soon as receiver releases ACK, it does not wait till ACK CR is 
re-acquired by receiver.

To illustrate the problem consider timeline for two senders and one 
receiver (we will ignore receive part for Sender2 node)

Sender1              Sender2                         Receiver
Get EX on TOKEN       Get EX on TOKEN
<Granted>                    <Wait till granted>

Get EX on MSG
write LVB
down MSG to CR
Get EX of ACK
<wait till granted>                                                     
      BAST for ACK
                                                             Get CR on MSG
                     read LVB
                     process
                     release ACK
AST for ACK
down ACK to CR
release MSG
release TOKEN
                    <granted>
                    Get EX on MSG
                    <... proceed ...>
                    release TOKEN
  <lost one message>
^^^^^^^^^^^^^^^^^
                                                              Get EX on MSG
                                                              Get CR on ACK
release MSG


Abhijit