[Drbd-dev] switching between protocols A and C

[Drbd-dev] switching between protocols A and C

[Cristian Zamfir]

Hi,

Would it be possible to switch from protocol A to C and vice versa 
without disconnecting (for instance detach, switch protocol and re-attach)?

This would be useful when alternating between two primaries (only 
protocol C can be used) and a primary/secondary configuration where 
protocol A may increase write throughput on the primary node.

Right now this behaviour does not seem to be supported but I am 
interested if you consider it could be implemented easily.

Thank you.

Cristian

Re: [Drbd-dev] switching between protocols A and C

[Lars Ellenberg]

On Wed, Apr 11, 2007 at 01:13:32PM +0100, Cristian Zamfir wrote:
> 
> Hi,
> 
> Would it be possible to switch from protocol A to C and vice versa 
> without disconnecting (for instance detach, switch protocol and re-attach)?
> 
> This would be useful when alternating between two primaries (only 
> protocol C can be used) and a primary/secondary configuration where 
> protocol A may increase write throughput on the primary node.

protocol A does _NOT_ increase throughput at all.
it _MAY_ reduce latency somewhat sometimes.

> Right now this behaviour does not seem to be supported but I am 
> interested if you consider it could be implemented easily.

no. and it does not make much sense for me, either.

-- 
: Lars Ellenberg                            Tel +43-1-8178292-0  :
: LINBIT Information Technologies GmbH      Fax +43-1-8178292-82 :
: Vivenotgasse 48, A-1120 Vienna/Europe    http://www.linbit.com :
__
please use the "List-Reply" function of your email client.

Re: [Drbd-dev] switching between protocols A and C

[Cristian Zamfir]

Lars Ellenberg wrote:
> On Wed, Apr 11, 2007 at 01:13:32PM +0100, Cristian Zamfir wrote:
>> Hi,
>>
>> Would it be possible to switch from protocol A to C and vice versa 
>> without disconnecting (for instance detach, switch protocol and re-attach)?
>>
>> This would be useful when alternating between two primaries (only 
>> protocol C can be used) and a primary/secondary configuration where 
>> protocol A may increase write throughput on the primary node.
> 
> protocol A does _NOT_ increase throughput at all.
> it _MAY_ reduce latency somewhat sometimes.

I am testing with the bonnie++ benchmark over a 100Mbps link and I get a 
substantial increase for output in the case of protocol A at the expense 
of higher CPU ratio. From what I understand, it would make sense to get 
better throughput from the asynchronous protocol over low quality links. 
Or am I doing something wrong?



drbd connected protocol C

Version  1.03       ------Sequential Output------ --Sequential Input- 
--Random-
                     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- 
--Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP 
/sec %CP
raoul          464M 12764  52 13525   4  8401   0 18724  39 27174   0 
198.4   0
                     ------Sequential Create------ --------Random 
Create--------
                     -Create-- --Read--- -Delete-- -Create-- --Read--- 
-Delete--
               files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP 
/sec %CP
                  16 27245  94 +++++ +++ 27305  99 26368  98 +++++ +++ 
22775  95
raoul,464M,12764,52,13525,4,8401,0,18724,39,27174,0,198.4,0,16,27245,94,+++++,+++,27305,99,26368,98,+++++,+++,22775,95


drbd connected protocol A

Version  1.03       ------Sequential Output------ --Sequential Input- 
--Random-
                     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- 
--Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP 
/sec %CP
raoul          464M 27292  90 33632  10 11140   0 19785  46 28524   0 
218.0   0
                     ------Sequential Create------ --------Random 
Create--------
                     -Create-- --Read--- -Delete-- -Create-- --Read--- 
-Delete--
               files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP 
/sec %CP
                  16 27136  97 +++++ +++ 26985 100 26775  98 +++++ +++ 
23972 100
raoul,464M,27292,90,33632,10,11140,0,19785,46,28524,0,218.0,0,16,27136,97,+++++,+++,26985,100,26775,98,+++++,+++,23972,100 


> 
>> Right now this behaviour does not seem to be supported but I am 
>> interested if you consider it could be implemented easily.
> 
> no. and it does not make much sense for me, either.
> 

Cristian

Re: [Drbd-dev] switching between protocols A and C

[Lars Ellenberg]

On Wed, Apr 11, 2007 at 02:46:02PM +0100, Cristian Zamfir wrote:
> Lars Ellenberg wrote:
> >On Wed, Apr 11, 2007 at 01:13:32PM +0100, Cristian Zamfir wrote:
> >>Hi,
> >>
> >>Would it be possible to switch from protocol A to C and vice versa 
> >>without disconnecting (for instance detach, switch protocol and 
> >>re-attach)?
> >>
> >>This would be useful when alternating between two primaries (only 
> >>protocol C can be used) and a primary/secondary configuration where 
> >>protocol A may increase write throughput on the primary node.
> >
> >protocol A does _NOT_ increase throughput at all.
> >it _MAY_ reduce latency somewhat sometimes.
> 
> I am testing with the bonnie++ benchmark over a 100Mbps link and I get a 
> substantial increase for output in the case of protocol A at the expense 
> of higher CPU ratio. From what I understand, it would make sense to get 
> better throughput from the asynchronous protocol over low quality links. 
> Or am I doing something wrong?

and your exact bonnie++ command line is:

-- 
: Lars Ellenberg                            Tel +43-1-8178292-0  :
: LINBIT Information Technologies GmbH      Fax +43-1-8178292-82 :
: Vivenotgasse 48, A-1120 Vienna/Europe    http://www.linbit.com :
__
please use the "List-Reply" function of your email client.

Re: [Drbd-dev] switching between protocols A and C

[Cristian Zamfir]

Lars Ellenberg wrote:
> On Wed, Apr 11, 2007 at 02:46:02PM +0100, Cristian Zamfir wrote:
>> Lars Ellenberg wrote:
>>> On Wed, Apr 11, 2007 at 01:13:32PM +0100, Cristian Zamfir wrote:
>>>> Hi,
>>>>
>>>> Would it be possible to switch from protocol A to C and vice versa 
>>>> without disconnecting (for instance detach, switch protocol and 
>>>> re-attach)?
>>>>
>>>> This would be useful when alternating between two primaries (only 
>>>> protocol C can be used) and a primary/secondary configuration where 
>>>> protocol A may increase write throughput on the primary node.
>>> protocol A does _NOT_ increase throughput at all.
>>> it _MAY_ reduce latency somewhat sometimes.
>> I am testing with the bonnie++ benchmark over a 100Mbps link and I get a 
>> substantial increase for output in the case of protocol A at the expense 
>> of higher CPU ratio. From what I understand, it would make sense to get 
>> better throughput from the asynchronous protocol over low quality links. 
>> Or am I doing something wrong?
> 
> and your exact bonnie++ command line is:
> 
Bonnie++ is run with the default parameters:
bonnie++ -u zamf -d bonnie/

I have results from ./tiobench, with shows a 2.4x increase in 
performance in favour of protocol A:

/usr/bin/tiotest -t 8 -f 60 -r 500 -b 4096 -d . -T-T

Protocol A
Sequential Writes
                               File  Blk   Num                   Avg 
   Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency 
  Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- 
-----------  -------- -------- -----
2.6.18-xenU                   480   4096    1   24.27 11.73%     0.126 
     724.87   0.00000  0.00000   207
2.6.18-xenU                   480   4096    2   20.32 10.07%     0.312 
    1002.72   0.00000  0.00000   202
2.6.18-xenU                   480   4096    4   16.57 7.733%     0.648 
    4505.20   0.00570  0.00000   214
2.6.18-xenU                   480   4096    8   15.96 7.445%     1.107 
    6598.77   0.02279  0.00000   214


Protocol C

Sequential Writes
                               File  Blk   Num                   Avg 
   Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency 
  Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- 
-----------  -------- -------- -----
2.6.18-xenU                   480   4096    1   10.46 5.032%     0.301 
    1232.91   0.00000  0.00000   208
2.6.18-xenU                   480   4096    2   10.74 5.032%     0.581 
    1981.85   0.00000  0.00000   213
2.6.18-xenU                   480   4096    4   10.55 5.009%     0.849 
    3782.22   0.01302  0.00000   211
2.6.18-xenU                   480   4096    8   10.32 4.923%     1.841 
    8250.74   0.03581  0.00000   210

Re: [Drbd-dev] switching between protocols A and C

[Lars Ellenberg]

On Wed, Apr 11, 2007 at 04:04:33PM +0100, Cristian Zamfir wrote:
> Lars Ellenberg wrote:
> > On Wed, Apr 11, 2007 at 02:46:02PM +0100, Cristian Zamfir wrote:
> >> Lars Ellenberg wrote:
> >>> On Wed, Apr 11, 2007 at 01:13:32PM +0100, Cristian Zamfir wrote:
> >>>> Hi,
> >>>>
> >>>> Would it be possible to switch from protocol A to C and vice versa 
> >>>> without disconnecting (for instance detach, switch protocol and 
> >>>> re-attach)?
> >>>>
> >>>> This would be useful when alternating between two primaries (only 
> >>>> protocol C can be used) and a primary/secondary configuration where 
> >>>> protocol A may increase write throughput on the primary node.
> >>> protocol A does _NOT_ increase throughput at all.
> >>> it _MAY_ reduce latency somewhat sometimes.
> >> I am testing with the bonnie++ benchmark over a 100Mbps link and I get a 
> >> substantial increase for output in the case of protocol A at the expense 
> >> of higher CPU ratio. From what I understand, it would make sense to get 
> >> better throughput from the asynchronous protocol over low quality links. 
> >> Or am I doing something wrong?
> > 
> > and your exact bonnie++ command line is:
> > 
> Bonnie++ is run with the default parameters:
> bonnie++ -u zamf -d bonnie/
> 
> I have results from ./tiobench, with shows a 2.4x increase in 
> performance in favour of protocol A:
> 
> /usr/bin/tiotest -t 8 -f 60 -r 500 -b 4096 -d . -T-T

sorry, I don't have the time to interpret your benchmark results
in detail. it is likely that these benchmarks are just not testing
what I call "throuput".
which is sustained sequential block writes (streaming),
as can be measured with e.g. the "dm" tool from the benchmark sub
directory in the drbd tarball.

even so, regardless of benchmark results, the answer stands:

 either you allow two primaries,
   and are force to use protocol C.

 or you do not allow two primaries,
   and are free to chose the protocol.

protocol changes involve a disconnect/reconnect on both nodes.
I do not see any reason to change this.

-- 
: Lars Ellenberg                            Tel +43-1-8178292-0  :
: LINBIT Information Technologies GmbH      Fax +43-1-8178292-82 :
: Vivenotgasse 48, A-1120 Vienna/Europe    http://www.linbit.com :
__
please use the "List-Reply" function of your email client.