RFC: Deprecating ceph-tool commands

RFC: Deprecating ceph-tool commands

[Joao Eduardo Luis]

All,

While working on #11545 (mon: have mon-specific commands under 'ceph mon
...') I crashed into a slightly tough brick wall.

The purpose of #11545 is to move certain commands, such as 'ceph scrub',
'ceph compact' and 'ceph sync force' to the 'mon' module of the ceph-tool.

These commands have long stood in this format because 'mon'-module
commands have been traditionally considered as being somehow related
with monmaps and/or the MonmapMonitor.  However, from a user
perspective, if they relate to the monitor itself (and not cluster-wide)
they should reside under the 'mon'-module.

As such, I decided they should be moved to 'ceph mon scrub', 'ceph mon
compact' and 'ceph mon sync force'.

Adding these commands and doing the correct mapping is not hard at all.
 However, backward compatibility must be maintained, and simply dropping
the old style commands doesn't seem reasonable at all.

Keeping the old style commands alongside with the new commands is
trivial enough to not pose a problem, but they must go away at some
point.  After everyone get used to the new commands, and as soon as the
vast majority of deployments support the new commands, the old style
commands will simply be clutter.

And while these commands are not widely used, and while most people
certainly have not ever needed to use them, this sort of thing can at
some point be required for any other (most commonly used) command.

As I have not been able to find any mentions to guidelines to
deprecating commands, I thus propose the following:

A command being DEPRECATED must be:

 - clearly marked as DEPRECATED in usage;
 - kept around for at least 2 major releases;
 - kept compatible for the duration of the deprecation period.

Once two major releases go by, the command will then enter the OBSOLETE
period.  This would be one major release, during which the command would
no longer work although still acknowledged.  A simple message down the
lines of 'This command is now obsolete; please check the docs' would
suffice to inform the user.

The command would no longer exist in the next major release.

This approach gives a lifespan of roughly 3 releases (at current rate,
roughly 1.5 years) before being completely dropped.  This should give
enough time to people to realize what has happened and adjust any
scripts they may have.

E.g., a command being deprecated in Infernallis would be completely
dropped in the L-release, spanning its existence to at least one
long-term stable (i.e., jewel) and being dropped as soon as the first
dev cycle for the L-release begins.

Any thoughts and comments are welcome.

Cheers!

  -Joao

p.s., If you want to take a look at how this would translate in terms of
code on the monitor, please check [1].

[1] - https://github.com/ceph/ceph/pull/4595

Re: RFC: Deprecating ceph-tool commands

[Gregory Farnum]

On Fri, May 8, 2015 at 4:55 PM, Joao Eduardo Luis <joao@suse.de> wrote:
> All,
>
> While working on #11545 (mon: have mon-specific commands under 'ceph mon
> ...') I crashed into a slightly tough brick wall.
>
> The purpose of #11545 is to move certain commands, such as 'ceph scrub',
> 'ceph compact' and 'ceph sync force' to the 'mon' module of the ceph-tool.
>
> These commands have long stood in this format because 'mon'-module
> commands have been traditionally considered as being somehow related
> with monmaps and/or the MonmapMonitor.  However, from a user
> perspective, if they relate to the monitor itself (and not cluster-wide)
> they should reside under the 'mon'-module.
>
> As such, I decided they should be moved to 'ceph mon scrub', 'ceph mon
> compact' and 'ceph mon sync force'.
>
> Adding these commands and doing the correct mapping is not hard at all.
>  However, backward compatibility must be maintained, and simply dropping
> the old style commands doesn't seem reasonable at all.
>
> Keeping the old style commands alongside with the new commands is
> trivial enough to not pose a problem, but they must go away at some
> point.  After everyone get used to the new commands, and as soon as the
> vast majority of deployments support the new commands, the old style
> commands will simply be clutter.
>
> And while these commands are not widely used, and while most people
> certainly have not ever needed to use them, this sort of thing can at
> some point be required for any other (most commonly used) command.
>
> As I have not been able to find any mentions to guidelines to
> deprecating commands, I thus propose the following:
>
> A command being DEPRECATED must be:
>
>  - clearly marked as DEPRECATED in usage;
>  - kept around for at least 2 major releases;
>  - kept compatible for the duration of the deprecation period.
>
> Once two major releases go by, the command will then enter the OBSOLETE
> period.  This would be one major release, during which the command would
> no longer work although still acknowledged.  A simple message down the
> lines of 'This command is now obsolete; please check the docs' would
> suffice to inform the user.
>
> The command would no longer exist in the next major release.
>
> This approach gives a lifespan of roughly 3 releases (at current rate,
> roughly 1.5 years) before being completely dropped.  This should give
> enough time to people to realize what has happened and adjust any
> scripts they may have.
>
> E.g., a command being deprecated in Infernallis would be completely
> dropped in the L-release, spanning its existence to at least one
> long-term stable (i.e., jewel) and being dropped as soon as the first
> dev cycle for the L-release begins.

Well, this is an interesting dilemma. "As a user", I think I'd want it
to be deprecated and warning me for one release I run — I'll see it
when I upgrade and then know I need to fix it — and then it can be
obsoleted in the next release I run.

It's that "release I run" bit that's tricky though, right? I presume
you made it two big releases because that should capture at least one
release supported by downstream providers? But the release marking it
as obsoleted will often not be captured by downstreams. So it seems
like either we should do two releases in each state, or else we should
as a community do one release in each state and if the downstreams
want to keep command mappings around for longer they can do that. In
general it's not like those are going to be tricky patches to apply...

Also, I think this set of commands is sufficiently special-purpose
that you could probably just make the swap and have the top-level ones
spit out an error referring to the move; we don't necessarily need to
make decisions on this right now.
-Greg
_______________________________________________
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: RFC: Deprecating ceph-tool commands

[Joao Eduardo Luis]

On 09/05/15 01:28, Gregory Farnum wrote:
> On Fri, May 8, 2015 at 4:55 PM, Joao Eduardo Luis <joao@suse.de> wrote:
>> All,
>>
>> While working on #11545 (mon: have mon-specific commands under 'ceph mon
>> ...') I crashed into a slightly tough brick wall.
>>
>> The purpose of #11545 is to move certain commands, such as 'ceph scrub',
>> 'ceph compact' and 'ceph sync force' to the 'mon' module of the ceph-tool.
>>
>> These commands have long stood in this format because 'mon'-module
>> commands have been traditionally considered as being somehow related
>> with monmaps and/or the MonmapMonitor.  However, from a user
>> perspective, if they relate to the monitor itself (and not cluster-wide)
>> they should reside under the 'mon'-module.
>>
>> As such, I decided they should be moved to 'ceph mon scrub', 'ceph mon
>> compact' and 'ceph mon sync force'.
>>
>> Adding these commands and doing the correct mapping is not hard at all.
>>   However, backward compatibility must be maintained, and simply dropping
>> the old style commands doesn't seem reasonable at all.
>>
>> Keeping the old style commands alongside with the new commands is
>> trivial enough to not pose a problem, but they must go away at some
>> point.  After everyone get used to the new commands, and as soon as the
>> vast majority of deployments support the new commands, the old style
>> commands will simply be clutter.
>>
>> And while these commands are not widely used, and while most people
>> certainly have not ever needed to use them, this sort of thing can at
>> some point be required for any other (most commonly used) command.
>>
>> As I have not been able to find any mentions to guidelines to
>> deprecating commands, I thus propose the following:
>>
>> A command being DEPRECATED must be:
>>
>>   - clearly marked as DEPRECATED in usage;
>>   - kept around for at least 2 major releases;
>>   - kept compatible for the duration of the deprecation period.
>>
>> Once two major releases go by, the command will then enter the OBSOLETE
>> period.  This would be one major release, during which the command would
>> no longer work although still acknowledged.  A simple message down the
>> lines of 'This command is now obsolete; please check the docs' would
>> suffice to inform the user.
>>
>> The command would no longer exist in the next major release.
>>
>> This approach gives a lifespan of roughly 3 releases (at current rate,
>> roughly 1.5 years) before being completely dropped.  This should give
>> enough time to people to realize what has happened and adjust any
>> scripts they may have.
>>
>> E.g., a command being deprecated in Infernallis would be completely
>> dropped in the L-release, spanning its existence to at least one
>> long-term stable (i.e., jewel) and being dropped as soon as the first
>> dev cycle for the L-release begins.
>
> Well, this is an interesting dilemma. "As a user", I think I'd want it
> to be deprecated and warning me for one release I run — I'll see it
> when I upgrade and then know I need to fix it — and then it can be
> obsoleted in the next release I run.
>
> It's that "release I run" bit that's tricky though, right? I presume
> you made it two big releases because that should capture at least one
> release supported by downstream providers? But the release marking it
> as obsoleted will often not be captured by downstreams. So it seems
> like either we should do two releases in each state, or else we should
> as a community do one release in each state and if the downstreams
> want to keep command mappings around for longer they can do that. In
> general it's not like those are going to be tricky patches to apply...

My initial thought was not even about downstream releases, and although 
that would be a valid point I believe downstream can just as well deal 
with it as they see fit.

However, AFACT those consuming directly from upstream tend to be in one 
of two categories: upgrade every release or, more commonly, skip 
intermediate releases and just go for the long-term stables.
Keeping a 2 releases time span + at least one other to obsolete the 
command will guarantee that we always get the deprecated command in at 
least one long-term stable.

I am not opposed to keep the command obsolete for another two major 
releases, but that does seem a bit excessive.  I'm much more concerned 
about keeping the command deprecated for at least two releases than I am 
with how long we keep it obsolete.

> Also, I think this set of commands is sufficiently special-purpose
> that you could probably just make the swap and have the top-level ones
> spit out an error referring to the move; we don't necessarily need to
> make decisions on this right now.

I could.  It would not be as clean, especially considering the 
purpose-specific logic to maintain backward compatibility, but I did try 
that at first -- and got annoyed by how ugly that was.

Rather than going that way though, I'm taking this as an opportunity to 
open this debate.  These are simple enough commands to allow us to 
rollback any major decisions we may take now and fine-tune the process 
if we find it lacking.

Instead of waiting for the next big command we may want to deprecate and 
end up doing it then.

   -Joao

_______________________________________________
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: RFC: Deprecating ceph-tool commands

[Loic Dachary]

[-- Attachment #1: Type: text/plain, Size: 784 bytes --]

Hi,

On 09/05/2015 01:55, Joao Eduardo Luis wrote:
> This approach gives a lifespan of roughly 3 releases (at current rate,
> roughly 1.5 years) before being completely dropped.  This should give
> enough time to people to realize what has happened and adjust any
> scripts they may have.

That sounds reasonable. 

In the python documentation I find it very convenient to have "New in X.Y.Z" next to a given API call or behavior, as well as "Deprectated since". Although release notes https://github.com/ceph/ceph/commit/e3b788a56c640fb19d8aa9464ebf5bc84c97ec6f are useful when upgrading (to answer the question "What API will break ?"), it's not the primary location I'll check when considering using something. 

Cheers

-- 
Loïc Dachary, Artisan Logiciel Libre


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: RFC: Deprecating ceph-tool commands

[Joao Eduardo Luis]

On 05/09/2015 09:57 AM, Loic Dachary wrote:
> Hi,
> 
> On 09/05/2015 01:55, Joao Eduardo Luis wrote:
>> This approach gives a lifespan of roughly 3 releases (at current rate,
>> roughly 1.5 years) before being completely dropped.  This should give
>> enough time to people to realize what has happened and adjust any
>> scripts they may have.
> 
> That sounds reasonable. 
> 
> In the python documentation I find it very convenient to have "New in X.Y.Z" next to a given API call or behavior, as well as "Deprectated since". Although release notes https://github.com/ceph/ceph/commit/e3b788a56c640fb19d8aa9464ebf5bc84c97ec6f are useful when upgrading (to answer the question "What API will break ?"), it's not the primary location I'll check when considering using something. 

Agreed, docs will have to be changed, including man pages.

  -Joao

Re: [ceph-users] RFC: Deprecating ceph-tool commands

[John Spray]

On 09/05/2015 00:55, Joao Eduardo Luis wrote:
> A command being DEPRECATED must be:
>
>   - clearly marked as DEPRECATED in usage;
>   - kept around for at least 2 major releases;
>   - kept compatible for the duration of the deprecation period.
>
> Once two major releases go by, the command will then enter the OBSOLETE
> period.  This would be one major release, during which the command would
> no longer work although still acknowledged.  A simple message down the
> lines of 'This command is now obsolete; please check the docs' would
> suffice to inform the user.
>
> The command would no longer exist in the next major release.
>
> This approach gives a lifespan of roughly 3 releases (at current rate,
> roughly 1.5 years) before being completely dropped.  This should give
> enough time to people to realize what has happened and adjust any
> scripts they may have.

+1, this seems like a reasonable timescale, but I think the important 
thing is that it's deprecated in at least one LTS release before it's 
actually removed.  So maybe we should just define it like that, and say 
"two stable releases or one LTS release, whichever is longer".  But I 
guess definition of LTS is a per-downstream-vendor thing, so maybe 
harder to define -- maybe the downstream part could be a guideline to 
downstream packagers, that will require no work from them as long as 
they are generating LTS releases on at least every other stable release.

An additional thought: it might be useful to have a "strict" flag for 
processes sending commands, so that e.g. management tools in QA can set 
that and fail out when they use something deprecated. Otherwise, 
automated things would tend not to notice messages about deprecation.

Cheers,
John