[Drbd-dev] Re: drbd in linux-ha cvs

[Drbd-dev] Re: drbd in linux-ha cvs

[Philipp Reisner]

On Monday 06 September 2004 16:52, you wrote:
> waere es sinnvoll sowas zu haben:
>
> drbdadm dump_meta_data r0 (oder info oder wie immer wir das nennen wollen)
>
> output would be bash sourceable:
> GEN_COUNT=SOME:LONGER:STRING:WITH:NUMBERS:AND:STUFF
> LOCAL_STATE={Primary,Secondary}
> REMOTE_STATE=...
> CSTATE=...
> LOCAL_STORAGE=...
>
>
> name space clashes können so gelöst werden:
> eval `drbdadm info r0 | sed 's/^/DRBD_/'`
>
> müsste einmal vernünftig überlegt werden, welche informationen wir
> exportieren wollen, und wie die variablen genannt werden.
> wie drbdadm an die info rankommt ist seine sache.
>
> deal?
>

Of course I am open to this idea.

Ideas:
 1) We move the read_gc.pl/write_gc.pl to the user directory. 
 2) Make them to one C program: drbdmeta
    -> in the future the module never creates the meta data
       block. One can use drbdmeta to create, read and 
       modify the drbdmeta block. drbdmeta refuses to write
       to it as long as the module is loaded (configured).
 3) drbdadm is the nice frontend to drbdmeta is it is to
    drbdsetup.

Currently we have these drbdadm commands that are not displayed
at the usage output:

  { "sh-resources",      sh_resources,0                  ,0,0 },
  { "sh-mod-parms",      sh_mod_parms,0                  ,0,0 },
  { "sh-dev",            sh_dev,      0                  ,0,1 },
  { "sh-ll-dev",         sh_ll_dev,   0                  ,0,1 },
  { "sh-md-dev",         sh_md_dev,   0                  ,0,1 },
  { "sh-md-idx",         sh_md_idx,   0                  ,0,1 }

... and we have state and cstate:

  { "state",             adm_generic_s,"state"           ,1,1 },
  { "cstate",            adm_generic_s,"cstate"          ,1,1 },

with the current interface you would do:

CSTATE=$(drbdadm cstate r0)
LOCAL_STATE=$( $DRBDADM state $RES 2> /dev/null )
LOCAL_STATE=${LOCAL_STATE%/*}
LOCAL_STORAGE= $(drbdadm sh-ll-dev r0)
...

With the current interface we have a lot of calls to drbdadm, so lets
have a look at the performance:

root@bloody:~/drbd07/user# time drbdadm sh-ll-dev r0
/dev/hdc1

real    0m0.170s
user    0m0.059s
sys     0m0.111s

... Quite slow!


So I disabled "verify_ips()". [ should probabely be really disabled. ]

root@bloody:~/drbd07/user# time ./drbdadm sh-ll-dev r0
/dev/hdc1

real    0m0.006s
user    0m0.001s
sys     0m0.005s

... now a call to drbdadm is as cheap as expected.

So with continuing the current way of doing things, I would 
extend the interface in this way:

drbdadm md-set-gc 1:2:3:4:5:6 r0
drbdadm md-get-gc 1:2:3:4:5:6 r0
drbdadm md-get/set-{la-size|consistent|etc...} resources....
drbdadm md-create r0

I rather prefer the current way, since it is also language
agnostic. In perl: $local_storage = `drbdadm sh-ll-dev r0`;
While the proposed interface is tied to bash.

-philipp
-- 
: Dipl-Ing Philipp Reisner                      Tel +43-1-8178292-50 :
: LINBIT Information Technologies GmbH          Fax +43-1-8178292-82 :
: Schönbrunnerstr 244, 1120 Vienna, Austria    http://www.linbit.com :

Re: [Drbd-dev] Re: drbd in linux-ha cvs

[Lars Ellenberg]

On Tue, Sep 07, 2004 at 09:57:57AM +0200, Philipp Reisner wrote:
> On Monday 06 September 2004 16:52, you wrote:
> > waere es sinnvoll sowas zu haben:
> >
> > drbdadm dump_meta_data r0 (oder info oder wie immer wir das nennen wollen)
> >
> > output would be bash sourceable:
> > GEN_COUNT=SOME:LONGER:STRING:WITH:NUMBERS:AND:STUFF
> > LOCAL_STATE={Primary,Secondary}
> > REMOTE_STATE=...
> > CSTATE=...
> > LOCAL_STORAGE=...
> >
> >
> > name space clashes können so gelöst werden:
> > eval `drbdadm info r0 | sed 's/^/DRBD_/'`
> >
> > müsste einmal vernünftig überlegt werden, welche informationen wir
> > exportieren wollen, und wie die variablen genannt werden.
> > wie drbdadm an die info rankommt ist seine sache.
> >
> > deal?
> >
> 
> Of course I am open to this idea.
> 
> Ideas:
>  1) We move the read_gc.pl/write_gc.pl to the user directory. 
>  2) Make them to one C program: drbdmeta
>     -> in the future the module never creates the meta data
>        block. One can use drbdmeta to create, read and 
>        modify the drbdmeta block. drbdmeta refuses to write
>        to it as long as the module is loaded (configured).
>  3) drbdadm is the nice frontend to drbdmeta is it is to
>     drbdsetup.

I don't want to make it easier to write generation counts.
heartbeat-TNG may have use for knowing the current ones, though.

> Currently we have these drbdadm commands that are not displayed
> at the usage output:
> 
>   { "sh-resources",      sh_resources,0                  ,0,0 },
>   { "sh-mod-parms",      sh_mod_parms,0                  ,0,0 },
>   { "sh-dev",            sh_dev,      0                  ,0,1 },
>   { "sh-ll-dev",         sh_ll_dev,   0                  ,0,1 },
>   { "sh-md-dev",         sh_md_dev,   0                  ,0,1 },
>   { "sh-md-idx",         sh_md_idx,   0                  ,0,1 }
> 
> ... and we have state and cstate:
> 
>   { "state",             adm_generic_s,"state"           ,1,1 },
>   { "cstate",            adm_generic_s,"cstate"          ,1,1 },
> 
> with the current interface you would do:
> 
> CSTATE=$(drbdadm cstate r0)
> LOCAL_STATE=$( $DRBDADM state $RES 2> /dev/null )
> LOCAL_STATE=${LOCAL_STATE%/*}
> LOCAL_STORAGE= $(drbdadm sh-ll-dev r0)
> ...
> 
> With the current interface we have a lot of calls to drbdadm, so lets
> have a look at the performance:
> 
> root@bloody:~/drbd07/user# time drbdadm sh-ll-dev r0
> /dev/hdc1
> 
> real    0m0.170s
> user    0m0.059s
> sys     0m0.111s
> 
> ... Quite slow!
> 
> 
> So I disabled "verify_ips()". [ should probabely be really disabled. ]

or only enabled with "verify config" or something like that.

> root@bloody:~/drbd07/user# time ./drbdadm sh-ll-dev r0
> /dev/hdc1
> 
> real    0m0.006s
> user    0m0.001s
> sys     0m0.005s
> 
> ... now a call to drbdadm is as cheap as expected.
> 
> So with continuing the current way of doing things, I would 
> extend the interface in this way:
> 
> drbdadm md-set-gc 1:2:3:4:5:6 r0
> drbdadm md-get-gc 1:2:3:4:5:6 r0
> drbdadm md-get/set-{la-size|consistent|etc...} resources....
> drbdadm md-create r0
> 
> I rather prefer the current way, since it is also language
> agnostic. In perl: $local_storage = `drbdadm sh-ll-dev r0`;
> While the proposed interface is tied to bash.

the point was to have an easy way to _read_ all available information in
a well defined way in one chunk. and having it in k=v form makes it
direktly useable by bash, and easily useable by perl:

  $_=`drbdadm info r0`; %drbdinfo = /^(.*?)=(.*?)$/mg; # :-)

I do NOT want to make it easy to _modify_ on-disk meta data.
if it is too easy (or even documented), people will screw up,
and blame us for it.

	lge

Re: [Drbd-dev] Re: drbd in linux-ha cvs

[Philipp Reisner]

>
> the point was to have an easy way to _read_ all available information in
> a well defined way in one chunk. and having it in k=v form makes it
> direktly useable by bash, and easily useable by perl:
>

Is it worth the trouble to maintain two interfaces in drbdadm ?

Is it less rouble to maintain the 

CSTATE=$(drbdadm cstate r0)
LOCAL_STATE=$( $DRBDADM state $RES 2> /dev/null )
LOCAL_STATE=${LOCAL_STATE%/*}
LOCAL_STORAGE= $(drbdadm sh-ll-dev r0)
...

lines in the bash script you do ?

-Philipp
-- 
: Dipl-Ing Philipp Reisner                      Tel +43-1-8178292-50 :
: LINBIT Information Technologies GmbH          Fax +43-1-8178292-82 :
: Schönbrunnerstr 244, 1120 Vienna, Austria    http://www.linbit.com :

Re: [Drbd-dev] Re: drbd in linux-ha cvs

[Philipp Reisner]

> > > waere es sinnvoll sowas zu haben:
> > >
> > > drbdadm dump_meta_data r0 (oder info oder wie immer wir das nennen
> > > wollen)
> > >
> > > output would be bash sourceable:
> > > GEN_COUNT=SOME:LONGER:STRING:WITH:NUMBERS:AND:STUFF
[...]
> > Ideas:
> >  1) We move the read_gc.pl/write_gc.pl to the user directory.
> >  2) Make them to one C program: drbdmeta
> >     -> in the future the module never creates the meta data
> >        block. One can use drbdmeta to create, read and
> >        modify the drbdmeta block. drbdmeta refuses to write
> >        to it as long as the module is loaded (configured).
> >  3) drbdadm is the nice frontend to drbdmeta is it is to
> >     drbdsetup.
[...]
> > So with continuing the current way of doing things, I would
> > extend the interface in this way:
> >
> > drbdadm md-set-gc 1:2:3:4:5:6 r0
> > drbdadm md-get-gc 1:2:3:4:5:6 r0
> > drbdadm md-get/set-{la-size|consistent|etc...} resources....
> > drbdadm md-create r0
[...]
> I do NOT want to make it easy to _modify_ on-disk meta data.
> if it is too easy (or even documented), people will screw up,
> and blame us for it.
>

Right! All (at least most) of the "set" methods must be undocumented. 
(And not visible in the usage)

-Philipp
-- 
: Dipl-Ing Philipp Reisner                      Tel +43-1-8178292-50 :
: LINBIT Information Technologies GmbH          Fax +43-1-8178292-82 :
: Schönbrunnerstr 244, 1120 Vienna, Austria    http://www.linbit.com :

Re: [Drbd-dev] Re: drbd in linux-ha cvs

[Lars Ellenberg]

On Tue, Sep 07, 2004 at 11:43:01AM +0200, Philipp Reisner wrote:
> >
> > the point was to have an easy way to _read_ all available information in
> > a well defined way in one chunk. and having it in k=v form makes it
> > direktly useable by bash, and easily useable by perl:
> >
> 
> Is it worth the trouble to maintain two interfaces in drbdadm ?
> 
> Is it less rouble to maintain the 
> 
> CSTATE=$(drbdadm cstate r0)
> LOCAL_STATE=$( $DRBDADM state $RES 2> /dev/null )
> LOCAL_STATE=${LOCAL_STATE%/*}
> LOCAL_STORAGE= $(drbdadm sh-ll-dev r0)
> ...
> 
> lines in the bash script you do ?

the drbdsetup code needs a cleanup anyways, we really need to
restructure what part ends up in some "public" header file, and what
part really still belongs to drbd_int.h ...

it is more clean to have one ioctl, get the full info, and output that.
I feel uneasy about introducing additional races by using several ioctls
in a row (even if they all finish within one second). of course, once we
have that info it could be stale already, but at least we have the
guarantee that at some point in time it was correct.
btw, one more reason to have some kind of "suspend", ask about
information, go figure, do some things, and only then "resume" to allow
changes to take effect... (due in 0.8)

as soon as we have a sane interface, I don't expect the internals to
change so often, so the "maintenance trouble" is very low, either way.

	lge

Re: [Drbd-dev] Re: drbd in linux-ha cvs

[Lars Marowsky-Bree]

On 2004-09-07T12:49:26,
   Lars Ellenberg <lars.ellenberg@linbit.com> said:

> it is more clean to have one ioctl, get the full info, and output that.
> I feel uneasy about introducing additional races by using several ioctls
> in a row (even if they all finish within one second). of course, once we
> have that info it could be stale already, but at least we have the
> guarantee that at some point in time it was correct.

Well, even if not correct, so at least it was consistent ;-)

It's also much easier to maintain in the external script, and less
likely to be gotten wrong in the script.

> as soon as we have a sane interface, I don't expect the internals to
> change so often, so the "maintenance trouble" is very low, either way.

Well, even if eventually change and export more data, that's more
maintainable.

I don't need writes to the generation counters, but getting at all that
information to know where to move things would be quite helpful.


Sincerely,
    Lars Marowsky-Brée <lmb@suse.de>

-- 
High Availability & Clustering	   \\\  /// 
SUSE Labs, Research and Development \honk/ 
SUSE LINUX AG - A Novell company     \\// 

Re: [Drbd-dev] Re: drbd in linux-ha cvs

[Philipp Reisner]

On Tuesday 07 September 2004 12:49, Lars Ellenberg wrote:
> On Tue, Sep 07, 2004 at 11:43:01AM +0200, Philipp Reisner wrote:
> > > the point was to have an easy way to _read_ all available information
> > > in a well defined way in one chunk. and having it in k=v form makes it
> > > direktly useable by bash, and easily useable by perl:
> >
> > Is it worth the trouble to maintain two interfaces in drbdadm ?
> >
> > Is it less rouble to maintain the
> >
> > CSTATE=$(drbdadm cstate r0)
> > LOCAL_STATE=$( $DRBDADM state $RES 2> /dev/null )
> > LOCAL_STATE=${LOCAL_STATE%/*}
> > LOCAL_STORAGE= $(drbdadm sh-ll-dev r0)
> > ...
> >
> > lines in the bash script you do ?
>
> the drbdsetup code needs a cleanup anyways, we really need to
> restructure what part ends up in some "public" header file, and what
> part really still belongs to drbd_int.h ...
>
> it is more clean to have one ioctl, get the full info, and output that.

Hmmm, the thing is, it is one ioctl. It is the DRBD_IOCTL_GET_CONFIG
ioctl. Only drbdsetup shows not show all the information in one call...

> I feel uneasy about introducing additional races by using several ioctls
> in a row (even if they all finish within one second). of course, once we
> have that info it could be stale already, but at least we have the
> guarantee that at some point in time it was correct.

This is of course an argument.

> btw, one more reason to have some kind of "suspend", ask about
> information, go figure, do some things, and only then "resume" to allow
> changes to take effect... (due in 0.8)
>
> as soon as we have a sane interface, I don't expect the internals to
> change so often, so the "maintenance trouble" is very low, either way.
>

I just not want to have two interfaces to get to the same stuff.

-Philipp
-- 
: Dipl-Ing Philipp Reisner                      Tel +43-1-8178292-50 :
: LINBIT Information Technologies GmbH          Fax +43-1-8178292-82 :
: Schönbrunnerstr 244, 1120 Vienna, Austria    http://www.linbit.com :

Re: [Drbd-dev] Re: drbd in linux-ha cvs

[Philipp Reisner]

On Tuesday 07 September 2004 14:03, Lars Marowsky-Bree wrote:
> On 2004-09-07T12:49:26,
>
>    Lars Ellenberg <lars.ellenberg@linbit.com> said:
> > it is more clean to have one ioctl, get the full info, and output that.
> > I feel uneasy about introducing additional races by using several ioctls
> > in a row (even if they all finish within one second). of course, once we
> > have that info it could be stale already, but at least we have the
> > guarantee that at some point in time it was correct.
>
> Well, even if not correct, so at least it was consistent ;-)
>
> It's also much easier to maintain in the external script, and less
> likely to be gotten wrong in the script.
>
> > as soon as we have a sane interface, I don't expect the internals to
> > change so often, so the "maintenance trouble" is very low, either way.
>
> Well, even if eventually change and export more data, that's more
> maintainable.
>
> I don't need writes to the generation counters, but getting at all that
> information to know where to move things would be quite helpful.
>

My initial understanding was that this read-generation counters
operation would read it off the disk (== run while the DRBD 
module is not loaded). Do you mean that ? Or do you want to have
an ioctl to ask the module for the current value of the countes ?

-philipp
-- 
: Dipl-Ing Philipp Reisner                      Tel +43-1-8178292-50 :
: LINBIT Information Technologies GmbH          Fax +43-1-8178292-82 :
: Schönbrunnerstr 244, 1120 Vienna, Austria    http://www.linbit.com :

Re: [Drbd-dev] Re: drbd in linux-ha cvs

[Lars Ellenberg]

On Tue, Sep 07, 2004 at 02:56:19PM +0200, Philipp Reisner wrote:
> On Tuesday 07 September 2004 14:03, Lars Marowsky-Bree wrote:
> > On 2004-09-07T12:49:26,
> >
> >    Lars Ellenberg <lars.ellenberg@linbit.com> said:
> > > it is more clean to have one ioctl, get the full info, and output that.
> > > I feel uneasy about introducing additional races by using several ioctls
> > > in a row (even if they all finish within one second). of course, once we
> > > have that info it could be stale already, but at least we have the
> > > guarantee that at some point in time it was correct.
> >
> > Well, even if not correct, so at least it was consistent ;-)
> >
> > It's also much easier to maintain in the external script, and less
> > likely to be gotten wrong in the script.
> >
> > > as soon as we have a sane interface, I don't expect the internals to
> > > change so often, so the "maintenance trouble" is very low, either way.
> >
> > Well, even if eventually change and export more data, that's more
> > maintainable.
> >
> > I don't need writes to the generation counters, but getting at all that
> > information to know where to move things would be quite helpful.
> >
> 
> My initial understanding was that this read-generation counters
> operation would read it off the disk (== run while the DRBD 
> module is not loaded). Do you mean that ? Or do you want to have
> an ioctl to ask the module for the current value of the countes ?

I think both ways make sense.
yes, I already added __u32 gen_cnt[5] to get_config here, but
I won't commit that too soon, I have some more suggestions in the ioctl
area, so we should wait for end of September first...

	lge

Re: [Drbd-dev] Re: drbd in linux-ha cvs

[Lars Marowsky-Bree]

On 2004-09-07T14:56:19,
   Philipp Reisner <philipp.reisner@linbit.com> said:

> My initial understanding was that this read-generation counters
> operation would read it off the disk (== run while the DRBD 
> module is not loaded). Do you mean that ? Or do you want to have
> an ioctl to ask the module for the current value of the countes ?

I need it in both scenarios and give me the proper information about the
state drbd is in - "Unconfigured" being one of them...


Sincerely,
    Lars Marowsky-Brée <lmb@suse.de>

-- 
High Availability & Clustering	   \\\  /// 
SUSE Labs, Research and Development \honk/ 
SUSE LINUX AG - A Novell company     \\//