REST API docs

REST API docs

[Stewart Smith]

The documentation up at:
https://github.com/openbmc/docs/blob/master/host-management.md
says /org/openbmc but we here tales in internal bug reports that
everything is switching to /xyz/openbmc_project.

Except, I don't recall seeing any deprecation plan.... is there
one?

We're heavily using this now for op-test-framework, and there's a patch
to switch it, but I need to know how far back/forwards the existing/new
interfaces are supported.

See:
https://github.com/open-power/op-test-framework/pull/150/commits/e05b1a89e032b1bbe14d9a177afa46f1a2c6413c

Also, could some more effort please be put into communicating these
changes. These kind of surprises aren't great for our already
constrained team.

-- 
Stewart Smith
OPAL Architect, IBM.

Re: REST API docs

[Patrick Williams]

[-- Attachment #1: Type: text/plain, Size: 3548 bytes --]

On Tue, Aug 01, 2017 at 03:07:36PM +1000, Stewart Smith wrote:
> The documentation up at:
> https://github.com/openbmc/docs/blob/master/host-management.md
> says /org/openbmc but we here tales in internal bug reports that
> everything is switching to /xyz/openbmc_project.
> 
> Except, I don't recall seeing any deprecation plan.... is there
> one?
> 
> We're heavily using this now for op-test-framework, and there's a patch
> to switch it, but I need to know how far back/forwards the existing/new
> interfaces are supported.
> 
> See:
> https://github.com/open-power/op-test-framework/pull/150/commits/e05b1a89e032b1bbe14d9a177afa46f1a2c6413c
> 
> Also, could some more effort please be put into communicating these
> changes. These kind of surprises aren't great for our already
> constrained team.

Stewart,

I don't know why this is a surprise.  We have been communicating in
various avenues that the /org/openbmc APIs defined for Barreleye
were all deprecated the day we split the 2.0 development branch.
I alluded to this even in the email announcing the v1.0 tag:
    https://lists.ozlabs.org/pipermail/openbmc/2016-June/003707.html

It is a violation of the dbus spec for us to be using /org/openbmc since
it is not a domain name we have control over.

None of the /org/openbmc interfaces went through any proper interface
design.  They were all defined and implemented ad-hoc by individual
developers as part of the Barreleye proof-of-concept work.  There are
numerous inconsistencies in their original form.

One of the first pieces of work we defined and began working on was a
dbus interface YAML specification, from which we generate compile-time
enforced dbus bindings to ensure compatibility between applications.
All of the xyz.openbmc_project interfaces have formal documentation in
the phosphor-dbus-interfaces repository, which had its first commits in
October of last year.

The dbus interfaces you are _moving to_ were defined and implemented
in December of last year and we moved over to them as a whole within
the code base by Feb.  Your first commit "Initial OpenBMC support" in
your repository wasn't until late Feb, so you were already using the
wrong interfaces.

We have been working very closely with the automated test group that
works on the openbmc-test-automation repository.  I was pretty certain
that within IBM there is some affinity between the team working on this
repository and the team working on your repository, isn't there?  In
fact, I clearly recall discussions within IBM questioning why the
op-test-framework would not be building on top of the existing Robot
support provided by openbmc-test-automation rather than implement
a different custom framework.

I was not aware that you had written your own test automation suite that
was using REST APIs and you were not relying on the existing and
supported openbmc-test-automation repository.  I also did not hear any
inquiry from you, either publicly or privately, on which APIs you should
be targeting for work you were doing.

The deprecation plan is, and has for the entire 2.0 development branch,
that 1.0 and 2.0 have a complete API break and there is no attempt at
backwards compatibility.  A clear requirement of a 2.0 tag is that all
/org/openbmc interfaces have been removed and replaced by something
documented in phosphor-dbus-interfaces and implemented within the code
base.  We have been doing that at a fairly regular pace the entire
year.

-- 
Patrick Williams

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: REST API docs

[Stewart Smith]

Patrick Williams <patrick@stwcx.xyz> writes:

> On Tue, Aug 01, 2017 at 03:07:36PM +1000, Stewart Smith wrote:
>> The documentation up at:
>> https://github.com/openbmc/docs/blob/master/host-management.md
>> says /org/openbmc but we here tales in internal bug reports that
>> everything is switching to /xyz/openbmc_project.
>> 
>> Except, I don't recall seeing any deprecation plan.... is there
>> one?
>> 
>> We're heavily using this now for op-test-framework, and there's a patch
>> to switch it, but I need to know how far back/forwards the existing/new
>> interfaces are supported.
>> 
>> See:
>> https://github.com/open-power/op-test-framework/pull/150/commits/e05b1a89e032b1bbe14d9a177afa46f1a2c6413c
>> 
>> Also, could some more effort please be put into communicating these
>> changes. These kind of surprises aren't great for our already
>> constrained team.
>
> Stewart,
>
> I don't know why this is a surprise.  We have been communicating in
> various avenues that the /org/openbmc APIs defined for Barreleye
> were all deprecated the day we split the 2.0 development branch.
> I alluded to this even in the email announcing the v1.0 tag:
>     https://lists.ozlabs.org/pipermail/openbmc/2016-June/003707.html

API changes are reading a bit too much between the lines there to
reasonably be actionable on.

> It is a violation of the dbus spec for us to be using /org/openbmc since
> it is not a domain name we have control over.

ok, that's reasonable.

> None of the /org/openbmc interfaces went through any proper interface
> design.  They were all defined and implemented ad-hoc by individual
> developers as part of the Barreleye proof-of-concept work.  There are
> numerous inconsistencies in their original form.

ok.

> One of the first pieces of work we defined and began working on was a
> dbus interface YAML specification, from which we generate compile-time
> enforced dbus bindings to ensure compatibility between applications.
> All of the xyz.openbmc_project interfaces have formal documentation in
> the phosphor-dbus-interfaces repository, which had its first commits in
> October of last year.

So something like this
https://github.com/openbmc/phosphor-dbus-interfaces/tree/master/xyz/openbmc_project/Inventory
supercedes
https://github.com/openbmc/docs/blob/master/host-management.md#inventory
?

The former certainly isn't too clear, while the original docs are. In
fact, we explicitly call out to /usr/bin/curl and print the command line
we're running so that we have an easy way to see what our test suite was
doing in case somebody wants to manually recreate it for debugging purposes.

> The dbus interfaces you are _moving to_ were defined and implemented
> in December of last year and we moved over to them as a whole within
> the code base by Feb.  Your first commit "Initial OpenBMC support" in
> your repository wasn't until late Feb, so you were already using the
> wrong interfaces.

The inital commit (738db7943eb0f090fa3db07c5200df536d0e3a4a) just used
the SSH host console and that's it, it wasn't until June
(64312b69acff30de8d75d80ab208c759a522a8ee) that we started to use the
REST API at all as when we started to poke at the IPMI support in March
(b19d231096ac27fa137eaf07b22c6deb61032e24), we quickly discovered that
it wasn't going to be sufficient in a near enough timeframe to be
suitable for OPAL testing.

I personally spent a bunch of time getting our test suite to be BMC
agnostic just to accomodate the backwards incompatibility of OpenBMC
(notably P9 FSP systems and P9 SMC systems are backwards compatibile,
and I don't think we had to add *any* code to accomodate them over the
P8 systems, with one patch for SMC versus AMI BMCs).

As of the time I pulled in the code to speak the OpenBMC REST API, that
was what was in the docs, and indeed, it's what is still there. The docs
repository is exclusively /org/openbmc/.

[stewart@birb openbmc-docs]$ ack -l 'org/openbmc'
cheatsheet.md
code-update.md
dbus-interfaces.md
host-management.md
rest-api.md
rfc-obmc-service-iface.md
[stewart@birb openbmc-docs]$ ack -l 'xyz/openbmc_project'
LED-architecture.md
openbmc-systemd.md
sensor-architecture.md

> We have been working very closely with the automated test group that
> works on the openbmc-test-automation repository.  I was pretty certain
> that within IBM there is some affinity between the team working on this
> repository and the team working on your repository, isn't there?  In
> fact, I clearly recall discussions within IBM questioning why the
> op-test-framework would not be building on top of the existing Robot
> support provided by openbmc-test-automation rather than implement
> a different custom framework.

There hasn't been. Recently we re-synced a bit, but just on a discussion
level.

There are two very different goals of the test suites too. One is to
test OpenBMC itself, while the other is to test host firmware, and
largely incidentally test the BMC implementation.

A while ago (January) there was a pull request r.e. Robot tests for
op-test-framework
(https://github.com/open-power/op-test-framework/pull/74 ) but for a
couple of reasons this didn't go anywhere. My biggest issues were that it
just added Robot files to the repository and didn't integrate at all
with the existing tests, nor re-implement any of them, and certainly
didn't support running the same tests against systems based on FSP,
OpenBMC, AMI and SMC. We have to be able to run against any of them, and
preferably also against any/all of the simulators we use.

I'm certainly not fundamentally opposed to Robot, it's possibly useful
for writing some small tests. But we do get a lot of power from just
having python
available... e.g. https://github.com/open-power/op-test-framework/blob/v0.8/testcases/IpmiTorture.py
which was instrumental in finding and fixing a critical customer
issue. We can write quite sophisticated tests that do a lot of things
concurrently (in and out of band).

While the Robot syntax may be nicer for a bunch of basic tests, it's not
*too* bad with what we have
(e.g. https://github.com/open-power/op-test-framework/blob/v0.8/testcases/OpalMsglog.py
greps the OPAL log for warnings/errors on the host both at the petitboot
shell and in a booted OS).

> I was not aware that you had written your own test automation suite that
> was using REST APIs and you were not relying on the existing and
> supported openbmc-test-automation repository.  I also did not hear any
> inquiry from you, either publicly or privately, on which APIs you should
> be targeting for work you were doing.

We just read the docs, and code seemed to work written to the docs (if
it didn't, we likely would have filed a bug).

> The deprecation plan is, and has for the entire 2.0 development branch,
> that 1.0 and 2.0 have a complete API break and there is no attempt at
> backwards compatibility.  A clear requirement of a 2.0 tag is that all
> /org/openbmc interfaces have been removed and replaced by something
> documented in phosphor-dbus-interfaces and implemented within the code
> base.  We have been doing that at a fairly regular pace the entire
> year.

It'd be great if the docs repo could be updated so it was clear with the
examples and the transition. Additionally, it'd be good if there'd be
posts to the list saying, say "the new xyz/openbmc_project sensors API
is now in master, we'll keep the org/openbmc/ one for the next 60 days
before dropping it" or along those lines.

Reality is that different P9 platforms are on (slightly) different
OpenBMC builds, and not every machine we get pointed at is up to date
with OpenBMC, so we do need to support a range of OpenBMC
builds. Especially considering we're pretty agile in being able to swap
between host and BMC development whenever we spot a problem in the other
system to help someone debug (even if the solution is 'yeah, I've
upgraded it for you').

-- 
Stewart Smith
OPAL Architect, IBM.

Re: REST API docs

[Patrick Williams]

[-- Attachment #1: Type: text/plain, Size: 9808 bytes --]

On Wed, Aug 02, 2017 at 05:02:41PM +1000, Stewart Smith wrote:
> Patrick Williams <patrick@stwcx.xyz> writes:
> 
> > I don't know why this is a surprise.  We have been communicating in
> > various avenues that the /org/openbmc APIs defined for Barreleye
> > were all deprecated the day we split the 2.0 development branch.
> > I alluded to this even in the email announcing the v1.0 tag:
> >     https://lists.ozlabs.org/pipermail/openbmc/2016-June/003707.html
> 
> API changes are reading a bit too much between the lines there to
> reasonably be actionable on.

How about specifically when I pointed the exact APIs in question in this
Feb 28th email?

* Phosphor Inventory Manager
    * Inband IPMI and IPMI EEPROM parser moved to new interfaces
    * OpenPower VPD EEPROM parser
* Phosphor Sensors from HWmon devices
* Phosphor State Management interfaces and corresponding systemd targets

https://lists.ozlabs.org/pipermail/openbmc/2017-February/006785.html

As I said, this has been brought up in various avenues throughout the
project.

> > One of the first pieces of work we defined and began working on was a
> > dbus interface YAML specification, from which we generate compile-time
> > enforced dbus bindings to ensure compatibility between applications.
> > All of the xyz.openbmc_project interfaces have formal documentation in
> > the phosphor-dbus-interfaces repository, which had its first commits in
> > October of last year.
> 
> So something like this
> https://github.com/openbmc/phosphor-dbus-interfaces/tree/master/xyz/openbmc_project/Inventory
> supercedes
> https://github.com/openbmc/docs/blob/master/host-management.md#inventory
> ?
> 
> The former certainly isn't too clear, while the original docs are. In
> fact, we explicitly call out to /usr/bin/curl and print the command line
> we're running so that we have an easy way to see what our test suite was
> doing in case somebody wants to manually recreate it for debugging purposes.
> 
> > The dbus interfaces you are _moving to_ were defined and implemented
> > in December of last year and we moved over to them as a whole within
> > the code base by Feb.  Your first commit "Initial OpenBMC support" in
> > your repository wasn't until late Feb, so you were already using the
> > wrong interfaces.
> 
> The inital commit (738db7943eb0f090fa3db07c5200df536d0e3a4a) just used
> the SSH host console and that's it, it wasn't until June
> (64312b69acff30de8d75d80ab208c759a522a8ee) that we started to use the
> REST API at all as when we started to poke at the IPMI support in March
> (b19d231096ac27fa137eaf07b22c6deb61032e24), we quickly discovered that
> it wasn't going to be sufficient in a near enough timeframe to be
> suitable for OPAL testing.
> 
> I personally spent a bunch of time getting our test suite to be BMC
> agnostic just to accomodate the backwards incompatibility of OpenBMC
> (notably P9 FSP systems and P9 SMC systems are backwards compatibile,
> and I don't think we had to add *any* code to accomodate them over the
> P8 systems, with one patch for SMC versus AMI BMCs).

I don't know what you are referring to here with respect to 'backwards
incompatibility'.  You did not support any 1.0 machines.  All of the
interfaces you are moving to existed before any code you wrote.  There
was nothing to be 'backwards compatible' with.

> As of the time I pulled in the code to speak the OpenBMC REST API, that
> was what was in the docs, and indeed, it's what is still there. The docs
> repository is exclusively /org/openbmc/.
> 
> [stewart@birb openbmc-docs]$ ack -l 'org/openbmc'
> cheatsheet.md
> code-update.md
> dbus-interfaces.md
> host-management.md
> rest-api.md
> rfc-obmc-service-iface.md
> [stewart@birb openbmc-docs]$ ack -l 'xyz/openbmc_project'
> LED-architecture.md
> openbmc-systemd.md
> sensor-architecture.md

Generally speaking, docs are for stable releases, aren't they?  We
didn't make an obvious fork of the documentation between 1.0 and 2.0
partially because we were planning on documenting much of the APIs
through the YAML (which can generate .md themselves).  Based on your
experience we probably should be explicit and say these are all
deprecated for the 2.0 release.

> > We have been working very closely with the automated test group that
> > works on the openbmc-test-automation repository.  I was pretty certain
> > that within IBM there is some affinity between the team working on this
> > repository and the team working on your repository, isn't there?  In
> > fact, I clearly recall discussions within IBM questioning why the
> > op-test-framework would not be building on top of the existing Robot
> > support provided by openbmc-test-automation rather than implement
> > a different custom framework.
> 
> There hasn't been. Recently we re-synced a bit, but just on a discussion
> level.
> 
> There are two very different goals of the test suites too. One is to
> test OpenBMC itself, while the other is to test host firmware, and
> largely incidentally test the BMC implementation.
> 
> A while ago (January) there was a pull request r.e. Robot tests for
> op-test-framework
> (https://github.com/open-power/op-test-framework/pull/74 ) but for a
> couple of reasons this didn't go anywhere. My biggest issues were that it
> just added Robot files to the repository and didn't integrate at all
> with the existing tests, nor re-implement any of them, and certainly
> didn't support running the same tests against systems based on FSP,
> OpenBMC, AMI and SMC. We have to be able to run against any of them, and
> preferably also against any/all of the simulators we use.
> 
> I'm certainly not fundamentally opposed to Robot, it's possibly useful
> for writing some small tests. But we do get a lot of power from just
> having python
> available... e.g. https://github.com/open-power/op-test-framework/blob/v0.8/testcases/IpmiTorture.py
> which was instrumental in finding and fixing a critical customer
> issue. We can write quite sophisticated tests that do a lot of things
> concurrently (in and out of band).

Robot is python.

> 
> While the Robot syntax may be nicer for a bunch of basic tests, it's not
> *too* bad with what we have
> (e.g. https://github.com/open-power/op-test-framework/blob/v0.8/testcases/OpalMsglog.py
> greps the OPAL log for warnings/errors on the host both at the petitboot
> shell and in a booted OS).

This would be a Robot action which you would continue to codify in
python.  "Ensure No Warnings or Errors" might be the Robot directive
which would point to a python function similar to this.

> > I was not aware that you had written your own test automation suite that
> > was using REST APIs and you were not relying on the existing and
> > supported openbmc-test-automation repository.  I also did not hear any
> > inquiry from you, either publicly or privately, on which APIs you should
> > be targeting for work you were doing.
> 
> We just read the docs, and code seemed to work written to the docs (if
> it didn't, we likely would have filed a bug).
> 
> > The deprecation plan is, and has for the entire 2.0 development branch,
> > that 1.0 and 2.0 have a complete API break and there is no attempt at
> > backwards compatibility.  A clear requirement of a 2.0 tag is that all
> > /org/openbmc interfaces have been removed and replaced by something
> > documented in phosphor-dbus-interfaces and implemented within the code
> > base.  We have been doing that at a fairly regular pace the entire
> > year.
> 
> It'd be great if the docs repo could be updated so it was clear with the
> examples and the transition. Additionally, it'd be good if there'd be
> posts to the list saying, say "the new xyz/openbmc_project sensors API
> is now in master, we'll keep the org/openbmc/ one for the next 60 days
> before dropping it" or along those lines.

I agree the documentation should be updated.  As I said, I've been
pointing out new interfaces being available (with an implementation) in
tag messages.  There are on the order of 500 commits going in every
month.  It really isn't possible for me to know which ones someone on
the mailing list might be interested in.

As we've said since the beginning of the 2.0 stream, all work is being
tracked on github.  You can see every feature being worked on and see
exactly when it closes due to the code being merged.  I get that Linux
has amazing announcements and wikis for each release, like
kernelnewbies, but those aren't done by Linus himself and I certainly
don't have the bandwidth do it.

> Reality is that different P9 platforms are on (slightly) different
> OpenBMC builds, and not every machine we get pointed at is up to date
> with OpenBMC, so we do need to support a range of OpenBMC
> builds. Especially considering we're pretty agile in being able to swap
> between host and BMC development whenever we spot a problem in the other
> system to help someone debug (even if the solution is 'yeah, I've
> upgraded it for you').

Again, I'm not really sure how this is relevant.  All of the APIs you
were using have been available for months.

Internal to IBM, there is a firmware release group that puts together
firmware images based on our code (along side the host code) and they
support all the machines we have hardware of inside IBM.  If you
aren't running a recent one of those, it's pretty straight-forward to
recommend an update.  If you're running master yourself you're on your
own.

There have been very very few tight couplings between host firmware and
BMC firmware.  The only one that comes to my mind is the mailbox
protocol support.

-- 
Patrick Williams

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: REST API docs

[Benjamin Herrenschmidt]

On Wed, 2017-08-02 at 08:01 -0500, Patrick Williams wrote:
> 
> Generally speaking, docs are for stable releases, aren't they? 

Certainly not !

You cannot change an API in a backward compatible way and keep the doc
documenting the old/wrong way in the repo. That's just plain wrong.

Ben.

Re: REST API docs

[Benjamin Herrenschmidt]

On Wed, 2017-08-02 at 08:01 -0500, Patrick Williams wrote:
> he documentation should be updated.  As I said, I've been
> pointing out new interfaces being available (with an implementation) in
> tag messages.  There are on the order of 500 commits going in every
> month.  It really isn't possible for me to know which ones someone on
> the mailing list might be interested in.
> 
> As we've said since the beginning of the 2.0 stream, all work is being
> tracked on github.  You can see every feature being worked on and see
> exactly when it closes due to the code being merged.  I get that Linux
> has amazing announcements and wikis for each release, like
> kernelnewbies, but those aren't done by Linus himself and I certainly
> don't have the bandwidth do it.

Nobody has the bandwidth.

What you are saying is "if you use the project, you have to read and
track every commit message for breadcrumbs of documentation because we
aren't goin to keep the docs up to date".

This is just wrong and impossible for your users.

Ben.

Re: REST API docs

[Patrick Williams]

[-- Attachment #1: Type: text/plain, Size: 2660 bytes --]

On Thu, Aug 03, 2017 at 10:22:26AM +1000, Benjamin Herrenschmidt wrote:
> On Wed, 2017-08-02 at 08:01 -0500, Patrick Williams wrote:
> > he documentation should be updated.  As I said, I've been
> > pointing out new interfaces being available (with an implementation) in
> > tag messages.  There are on the order of 500 commits going in every
> > month.  It really isn't possible for me to know which ones someone on
> > the mailing list might be interested in.
> > 
> > As we've said since the beginning of the 2.0 stream, all work is being
> > tracked on github.  You can see every feature being worked on and see
> > exactly when it closes due to the code being merged.  I get that Linux
> > has amazing announcements and wikis for each release, like
> > kernelnewbies, but those aren't done by Linus himself and I certainly
> > don't have the bandwidth do it.
> 
> Nobody has the bandwidth.
> 
> What you are saying is "if you use the project, you have to read and
> track every commit message for breadcrumbs of documentation because we
> aren't goin to keep the docs up to date".
> 
> This is just wrong and impossible for your users.
> 
> Ben.

Ben,

That is not what I wrote at all.  What I tried to write was this:

   * I've been attempting to point out high importance features in the
     release notes for the tags.

   * There seems to still be complaints about this being insufficient.

   * I cannot write the equivalent of LinuxNewbies for each tag, giving
     a paragraph for each commit.  No one else has volunteered to do
     this either.

   * I cannot know a priori when I have failed to sufficiently point out
     a change that might interest someone in a sufficiently obvious and
     detailed way.

Do you have any guidelines to help me out here on what is significant
enough to write a better "announcement" of and how detailed it needs to
be?  I'm obviously failing and I don't have any clear advise other than
"this is a particular instance when we wanted to know more details."

What Stewart does for skiboot is great and I respect the effort he puts
into these:

   * https://lists.ozlabs.org/pipermail/skiboot/2017-July/008067.html

What he does here, to the best of my reading, is to write a paragraph
for nearly each commit sequence he merges.  Again, this is great.  I do
not have the availability to do this.  If someone would like to help
out, I'm all for it.

I will warn that we have on the order of 150 commits per 2 week sprint
spread across about two dozen repositories.  This would be a pretty
significant undertaking for anyone.

-- 
Patrick Williams

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: REST API docs

[Patrick Williams]

[-- Attachment #1: Type: text/plain, Size: 1017 bytes --]

On Thu, Aug 03, 2017 at 10:19:47AM +1000, Benjamin Herrenschmidt wrote:
> On Wed, 2017-08-02 at 08:01 -0500, Patrick Williams wrote:
> > 
> > Generally speaking, docs are for stable releases, aren't they? 
> 
> Certainly not !
> 
> You cannot change an API in a backward compatible way and keep the doc
> documenting the old/wrong way in the repo. That's just plain wrong.
> 
> Ben.
> 

Fair enough.  I honestly though calling this entire code stream 1.99.x
was sufficient "hear be dragons" to prevent people from coding to things
unless they speak up.  We've been actively engaging the teams that we
knew were affected by all of our API changes, and we were not aware of
this one.  The teams we keep informed outside of the direct development
community are: openbmc test-automation team, xCAT, and the GUI
development team.

I will make sure we go through the docs repository soon and clearly point
out everything that is deprecated in the 1.99.x code stream as such.

-- 
Patrick Williams

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: REST API docs

[Benjamin Herrenschmidt]

On Thu, 2017-08-03 at 01:08 -0500, Patrick Williams wrote:
> 
> Do you have any guidelines to help me out here on what is significant
> enough to write a better "announcement" of and how detailed it needs to
> be?  I'm obviously failing and I don't have any clear advise other than
> "this is a particular instance when we wanted to know more details."
> 
> What Stewart does for skiboot is great and I respect the effort he puts
> into these:
> 
>    * https://lists.ozlabs.org/pipermail/skiboot/2017-July/008067.html
> 
> What he does here, to the best of my reading, is to write a paragraph
> for nearly each commit sequence he merges.  Again, this is great.  I do
> not have the availability to do this.  If someone would like to help
> out, I'm all for it.

What might work better is to maintain some kind of wiki with the
highlights, or at least the stuff that can have a significant usability
impact, so people get used to go there.

For example, everybody's hitting obcmutil poweroff vs. chassisoff,
so that would be a good candidate to have up the top there.

> I will warn that we have on the order of 150 commits per 2 week sprint
> spread across about two dozen repositories.  This would be a pretty
> significant undertaking for anyone.

Ben.

Re: REST API docs

[Benjamin Herrenschmidt]

On Thu, 2017-08-03 at 01:13 -0500, Patrick Williams wrote:
> > You cannot change an API in a backward compatible way and keep the doc
> > documenting the old/wrong way in the repo. That's just plain wrong.
> > 
> > Ben.
> > 
> 
> Fair enough.  I honestly though calling this entire code stream 1.99.x
> was sufficient "hear be dragons" to prevent people from coding to things
> unless they speak up.  

Haha, yeah well ... I think you should at least delete the docs or move
them to a "deprecated" directory or something when you break what they
document.

It's understandable that you may not have the bandwidth to update all
the docs right away (though in OPAL at least we tend to push for
requiring docs in the commit with with the feature change these days),
but it would help to make it clear that those docs aren't to be
trusted.

> We've been actively engaging the teams that we
> knew were affected by all of our API changes, and we were not aware of
> this one.  The teams we keep informed outside of the direct development
> community are: openbmc test-automation team, xCAT, and the GUI
> development team.

Engaging with team is great but you can't always know everybody who's
looking at or playing with your stuff.

> I will make sure we go through the docs repository soon and clearly point
> out everything that is deprecated in the 1.99.x code stream as such.

Yup, or at least move the whole thing to a "deprecated-needs-update"
directory until you had a chance to do the filtering. Maybe a toplevel
"WARNING" file with a one-liner about the domain change.

Cheers,
Ben.