Recipe Renames, etc

Recipe Renames, etc

[Patrick Venture]

I've sent a bunch of recipe renames, but I'm still updating each with
an individual tested footer.  I will try marking them all as WIP now

Patrick

Re: Recipe Renames, etc

[Matt Spinler]

On 2018-10-02 10:28, Patrick Venture wrote:
> I've sent a bunch of recipe renames, but I'm still updating each with
> an individual tested footer.  I will try marking them all as WIP now
> 
> Patrick


Hi Patrick,

I think you broke master with these.  For example, the phosphor-rest 
recipe - it
has a default url-config.json, and then a layer specific file gets 
applied from
the meta-ibm layer, like:

meta-ibm/recipes-phosphor/interfaces/phosphor-rest/url_config.json

However, from the log.do_unpack from the phosphor-rest build, it doesn't 
look there,
it now only looks in phosphor-rest%.  So any override directory without 
a % isn't
being found.

I haven't looked to see what all is broken yet and in how many layers, 
at least
our phosphor-rest and phosphor-gevent, which is why the web UI stopped 
working
on witherspoon.  Not sure if any other systems are also having problems.

Re: Recipe Renames, etc

[Patrick Venture]

On Wed, Oct 10, 2018 at 11:38 AM Matt Spinler
<mspinler@linux.vnet.ibm.com> wrote:
>
> On 2018-10-02 10:28, Patrick Venture wrote:
> > I've sent a bunch of recipe renames, but I'm still updating each with
> > an individual tested footer.  I will try marking them all as WIP now
> >
> > Patrick
>
>
> Hi Patrick,
>
> I think you broke master with these.  For example, the phosphor-rest
> recipe - it has a default url-config.json, and then a layer specific file gets
> applied from the meta-ibm layer, like:
>
> meta-ibm/recipes-phosphor/interfaces/phosphor-rest/url_config.json
>
> However, from the log.do_unpack from the phosphor-rest build, it doesn't
> look there, it now only looks in phosphor-rest%.  So any override directory without
> a % isn't being found.

Interesting that the CI didn't catch the break since witherspoon is
one of the systems built via it.  But I'll take a look.

>
> I haven't looked to see what all is broken yet and in how many layers,
> at least our phosphor-rest and phosphor-gevent, which is why the web UI stopped
> working on witherspoon.  Not sure if any other systems are also having problems.
>
>

Re: Recipe Renames, etc

[Patrick Venture]

On Wed, Oct 10, 2018 at 11:45 AM Patrick Venture <venture@google.com> wrote:
>
> On Wed, Oct 10, 2018 at 11:38 AM Matt Spinler
> <mspinler@linux.vnet.ibm.com> wrote:
> >
> > On 2018-10-02 10:28, Patrick Venture wrote:
> > > I've sent a bunch of recipe renames, but I'm still updating each with
> > > an individual tested footer.  I will try marking them all as WIP now
> > >
> > > Patrick
> >
> >
> > Hi Patrick,
> >
> > I think you broke master with these.  For example, the phosphor-rest
> > recipe - it has a default url-config.json, and then a layer specific file gets
> > applied from the meta-ibm layer, like:
> >
> > meta-ibm/recipes-phosphor/interfaces/phosphor-rest/url_config.json
> >
> > However, from the log.do_unpack from the phosphor-rest build, it doesn't
> > look there, it now only looks in phosphor-rest%.  So any override directory without
> > a % isn't being found.
>
> Interesting that the CI didn't catch the break since witherspoon is
> one of the systems built via it.  But I'll take a look.

Although I have had configuration files disappear when the wildcard is
missing from the directory, the recipe normally complains about being
unable to find the file.  For a file override, if there's already that
file - presumably then it wouldn't complain.

>
> >
> > I haven't looked to see what all is broken yet and in how many layers,
> > at least our phosphor-rest and phosphor-gevent, which is why the web UI stopped
> > working on witherspoon.  Not sure if any other systems are also having problems.
> >
> >

Re: Recipe Renames, etc

[Patrick Venture]

On Wed, Oct 10, 2018 at 11:46 AM Patrick Venture <venture@google.com> wrote:
>
> On Wed, Oct 10, 2018 at 11:45 AM Patrick Venture <venture@google.com> wrote:
> >
> > On Wed, Oct 10, 2018 at 11:38 AM Matt Spinler
> > <mspinler@linux.vnet.ibm.com> wrote:
> > >
> > > On 2018-10-02 10:28, Patrick Venture wrote:
> > > > I've sent a bunch of recipe renames, but I'm still updating each with
> > > > an individual tested footer.  I will try marking them all as WIP now
> > > >
> > > > Patrick
> > >
> > >
> > > Hi Patrick,
> > >
> > > I think you broke master with these.  For example, the phosphor-rest
> > > recipe - it has a default url-config.json, and then a layer specific file gets
> > > applied from the meta-ibm layer, like:
> > >
> > > meta-ibm/recipes-phosphor/interfaces/phosphor-rest/url_config.json
> > >
> > > However, from the log.do_unpack from the phosphor-rest build, it doesn't
> > > look there, it now only looks in phosphor-rest%.  So any override directory without
> > > a % isn't being found.
> >
> > Interesting that the CI didn't catch the break since witherspoon is
> > one of the systems built via it.  But I'll take a look.
>
> Although I have had configuration files disappear when the wildcard is
> missing from the directory, the recipe normally complains about being
> unable to find the file.  For a file override, if there's already that
> file - presumably then it wouldn't complain.

I've fired off a witherspoon build locally from openbmc HEAD, and will
report back.  Very few of the recipes were updated in this fashion so
I'm not terribly worried about a general breakage.

>
> >
> > >
> > > I haven't looked to see what all is broken yet and in how many layers,
> > > at least our phosphor-rest and phosphor-gevent, which is why the web UI stopped
> > > working on witherspoon.  Not sure if any other systems are also having problems.
> > >
> > >

Re: Recipe Renames, etc

[Patrick Venture]

On Wed, Oct 10, 2018 at 11:50 AM Patrick Venture <venture@google.com> wrote:
>
> On Wed, Oct 10, 2018 at 11:46 AM Patrick Venture <venture@google.com> wrote:
> >
> > On Wed, Oct 10, 2018 at 11:45 AM Patrick Venture <venture@google.com> wrote:
> > >
> > > On Wed, Oct 10, 2018 at 11:38 AM Matt Spinler
> > > <mspinler@linux.vnet.ibm.com> wrote:
> > > >
> > > > On 2018-10-02 10:28, Patrick Venture wrote:
> > > > > I've sent a bunch of recipe renames, but I'm still updating each with
> > > > > an individual tested footer.  I will try marking them all as WIP now
> > > > >
> > > > > Patrick
> > > >
> > > >
> > > > Hi Patrick,
> > > >
> > > > I think you broke master with these.  For example, the phosphor-rest
> > > > recipe - it has a default url-config.json, and then a layer specific file gets
> > > > applied from the meta-ibm layer, like:
> > > >
> > > > meta-ibm/recipes-phosphor/interfaces/phosphor-rest/url_config.json
> > > >
> > > > However, from the log.do_unpack from the phosphor-rest build, it doesn't
> > > > look there, it now only looks in phosphor-rest%.  So any override directory without
> > > > a % isn't being found.
> > >
> > > Interesting that the CI didn't catch the break since witherspoon is
> > > one of the systems built via it.  But I'll take a look.
> >
> > Although I have had configuration files disappear when the wildcard is
> > missing from the directory, the recipe normally complains about being
> > unable to find the file.  For a file override, if there's already that
> > file - presumably then it wouldn't complain.
>
> I've fired off a witherspoon build locally from openbmc HEAD, and will
> report back.  Very few of the recipes were updated in this fashion so
> I'm not terribly worried about a general breakage.

https://gerrit.openbmc-project.xyz/q/topic:%22rename_step_1_of_3%22+(status:open%20OR%20status:merged)
- in case you're curious which recipes were updated in the 3-step
approach.

Looks like only the rest, gevent, and webui were updated in this fashion.

This shows https://gerrit.openbmc-project.xyz/q/topic:%22rename_step_2_of_3%22+(status:open%20OR%20status:merged),
those need the 3rd step to finish out the job and this should also fix
the build for those systems with phosphor-rest/gevent.

And these are those staged paches:
https://gerrit.openbmc-project.xyz/q/topic:%22rename_step_3_of_3%22+(status:open%20OR%20status:merged)

>
> >
> > >
> > > >
> > > > I haven't looked to see what all is broken yet and in how many layers,
> > > > at least our phosphor-rest and phosphor-gevent, which is why the web UI stopped
> > > > working on witherspoon.  Not sure if any other systems are also having problems.
> > > >
> > > >

Re: Recipe Renames, etc

[Matt Spinler]

On 2018-10-10 13:50, Patrick Venture wrote:
> On Wed, Oct 10, 2018 at 11:46 AM Patrick Venture <venture@google.com> 
> wrote:
>> 
>> On Wed, Oct 10, 2018 at 11:45 AM Patrick Venture <venture@google.com> 
>> wrote:
>> >
>> > On Wed, Oct 10, 2018 at 11:38 AM Matt Spinler
>> > <mspinler@linux.vnet.ibm.com> wrote:
>> > >
>> > > On 2018-10-02 10:28, Patrick Venture wrote:
>> > > > I've sent a bunch of recipe renames, but I'm still updating each with
>> > > > an individual tested footer.  I will try marking them all as WIP now
>> > > >
>> > > > Patrick
>> > >
>> > >
>> > > Hi Patrick,
>> > >
>> > > I think you broke master with these.  For example, the phosphor-rest
>> > > recipe - it has a default url-config.json, and then a layer specific file gets
>> > > applied from the meta-ibm layer, like:
>> > >
>> > > meta-ibm/recipes-phosphor/interfaces/phosphor-rest/url_config.json
>> > >
>> > > However, from the log.do_unpack from the phosphor-rest build, it doesn't
>> > > look there, it now only looks in phosphor-rest%.  So any override directory without
>> > > a % isn't being found.
>> >
>> > Interesting that the CI didn't catch the break since witherspoon is
>> > one of the systems built via it.  But I'll take a look.

Yea, our current automated test bucket can't catch this.  (shows there
is still room for improvement)


>> 
>> Although I have had configuration files disappear when the wildcard is
>> missing from the directory, the recipe normally complains about being
>> unable to find the file.  For a file override, if there's already that
>> file - presumably then it wouldn't complain.

The latter is what's happening here.

> 
> I've fired off a witherspoon build locally from openbmc HEAD, and will
> report back.  Very few of the recipes were updated in this fashion so
> I'm not terribly worried about a general breakage.

Thanks for looking.  I'm not sure if there's an easy way to find these 
cases.

> 
>> 
>> >
>> > >
>> > > I haven't looked to see what all is broken yet and in how many layers,
>> > > at least our phosphor-rest and phosphor-gevent, which is why the web UI stopped
>> > > working on witherspoon.  Not sure if any other systems are also having problems.
>> > >
>> > >

Re: Recipe Renames, etc

[Patrick Venture]

On Wed, Oct 10, 2018 at 12:00 PM Matt Spinler
<mspinler@linux.vnet.ibm.com> wrote:
>
> On 2018-10-10 13:50, Patrick Venture wrote:
> > On Wed, Oct 10, 2018 at 11:46 AM Patrick Venture <venture@google.com>
> > wrote:
> >>
> >> On Wed, Oct 10, 2018 at 11:45 AM Patrick Venture <venture@google.com>
> >> wrote:
> >> >
> >> > On Wed, Oct 10, 2018 at 11:38 AM Matt Spinler
> >> > <mspinler@linux.vnet.ibm.com> wrote:
> >> > >
> >> > > On 2018-10-02 10:28, Patrick Venture wrote:
> >> > > > I've sent a bunch of recipe renames, but I'm still updating each with
> >> > > > an individual tested footer.  I will try marking them all as WIP now
> >> > > >
> >> > > > Patrick
> >> > >
> >> > >
> >> > > Hi Patrick,
> >> > >
> >> > > I think you broke master with these.  For example, the phosphor-rest
> >> > > recipe - it has a default url-config.json, and then a layer specific file gets
> >> > > applied from the meta-ibm layer, like:
> >> > >
> >> > > meta-ibm/recipes-phosphor/interfaces/phosphor-rest/url_config.json
> >> > >
> >> > > However, from the log.do_unpack from the phosphor-rest build, it doesn't
> >> > > look there, it now only looks in phosphor-rest%.  So any override directory without
> >> > > a % isn't being found.
> >> >
> >> > Interesting that the CI didn't catch the break since witherspoon is
> >> > one of the systems built via it.  But I'll take a look.
>
> Yea, our current automated test bucket can't catch this.  (shows there
> is still room for improvement)
>
>
> >>
> >> Although I have had configuration files disappear when the wildcard is
> >> missing from the directory, the recipe normally complains about being
> >> unable to find the file.  For a file override, if there's already that
> >> file - presumably then it wouldn't complain.
>
> The latter is what's happening here.
>
> >
> > I've fired off a witherspoon build locally from openbmc HEAD, and will
> > report back.  Very few of the recipes were updated in this fashion so
> > I'm not terribly worried about a general breakage.
>
> Thanks for looking.  I'm not sure if there's an easy way to find these
> cases.

This specific case is difficult to catch but, it's something that's
avoided for most of the renames, so at least there's that.  Future
changes though -- one will just have to be more cognizant of matching
output.

>
> >
> >>
> >> >
> >> > >
> >> > > I haven't looked to see what all is broken yet and in how many layers,
> >> > > at least our phosphor-rest and phosphor-gevent, which is why the web UI stopped
> >> > > working on witherspoon.  Not sure if any other systems are also having problems.
> >> > >
> >> > >
>

Re: Recipe Renames, etc

[Patrick Venture]

On Wed, Oct 10, 2018 at 12:01 PM Patrick Venture <venture@google.com> wrote:
>
> On Wed, Oct 10, 2018 at 12:00 PM Matt Spinler
> <mspinler@linux.vnet.ibm.com> wrote:
> >
> > On 2018-10-10 13:50, Patrick Venture wrote:
> > > On Wed, Oct 10, 2018 at 11:46 AM Patrick Venture <venture@google.com>
> > > wrote:
> > >>
> > >> On Wed, Oct 10, 2018 at 11:45 AM Patrick Venture <venture@google.com>
> > >> wrote:
> > >> >
> > >> > On Wed, Oct 10, 2018 at 11:38 AM Matt Spinler
> > >> > <mspinler@linux.vnet.ibm.com> wrote:
> > >> > >
> > >> > > On 2018-10-02 10:28, Patrick Venture wrote:
> > >> > > > I've sent a bunch of recipe renames, but I'm still updating each with
> > >> > > > an individual tested footer.  I will try marking them all as WIP now
> > >> > > >
> > >> > > > Patrick
> > >> > >
> > >> > >
> > >> > > Hi Patrick,
> > >> > >
> > >> > > I think you broke master with these.  For example, the phosphor-rest
> > >> > > recipe - it has a default url-config.json, and then a layer specific file gets
> > >> > > applied from the meta-ibm layer, like:
> > >> > >
> > >> > > meta-ibm/recipes-phosphor/interfaces/phosphor-rest/url_config.json
> > >> > >
> > >> > > However, from the log.do_unpack from the phosphor-rest build, it doesn't
> > >> > > look there, it now only looks in phosphor-rest%.  So any override directory without
> > >> > > a % isn't being found.
> > >> >
> > >> > Interesting that the CI didn't catch the break since witherspoon is
> > >> > one of the systems built via it.  But I'll take a look.
> >
> > Yea, our current automated test bucket can't catch this.  (shows there
> > is still room for improvement)
> >
> >
> > >>
> > >> Although I have had configuration files disappear when the wildcard is
> > >> missing from the directory, the recipe normally complains about being
> > >> unable to find the file.  For a file override, if there's already that
> > >> file - presumably then it wouldn't complain.
> >
> > The latter is what's happening here.
> >
> > >
> > > I've fired off a witherspoon build locally from openbmc HEAD, and will
> > > report back.  Very few of the recipes were updated in this fashion so
> > > I'm not terribly worried about a general breakage.
> >
> > Thanks for looking.  I'm not sure if there's an easy way to find these
> > cases.
>
> This specific case is difficult to catch but, it's something that's
> avoided for most of the renames, so at least there's that.  Future
> changes though -- one will just have to be more cognizant of matching
> output.
>
> >
> > >
> > >>
> > >> >
> > >> > >
> > >> > > I haven't looked to see what all is broken yet and in how many layers,
> > >> > > at least our phosphor-rest and phosphor-gevent, which is why the web UI stopped
> > >> > > working on witherspoon.  Not sure if any other systems are also having problems.
> > >> > >
> > >> > >
> >

Given it looks like I've already staged the fixes to this, we just
need to move those forward and when it settles, it should all be
copacetic.

Re: Recipe Renames, etc

[Matt Spinler]

On 2018-10-10 14:02, Patrick Venture wrote:
> On Wed, Oct 10, 2018 at 12:01 PM Patrick Venture <venture@google.com> 
> wrote:
>> 
>> On Wed, Oct 10, 2018 at 12:00 PM Matt Spinler
>> <mspinler@linux.vnet.ibm.com> wrote:
>> >
>> > On 2018-10-10 13:50, Patrick Venture wrote:
>> > > On Wed, Oct 10, 2018 at 11:46 AM Patrick Venture <venture@google.com>
>> > > wrote:
>> > >>
>> > >> On Wed, Oct 10, 2018 at 11:45 AM Patrick Venture <venture@google.com>
>> > >> wrote:
>> > >> >
>> > >> > On Wed, Oct 10, 2018 at 11:38 AM Matt Spinler
>> > >> > <mspinler@linux.vnet.ibm.com> wrote:
>> > >> > >
>> > >> > > On 2018-10-02 10:28, Patrick Venture wrote:
>> > >> > > > I've sent a bunch of recipe renames, but I'm still updating each with
>> > >> > > > an individual tested footer.  I will try marking them all as WIP now
>> > >> > > >
>> > >> > > > Patrick
>> > >> > >
>> > >> > >
>> > >> > > Hi Patrick,
>> > >> > >
>> > >> > > I think you broke master with these.  For example, the phosphor-rest
>> > >> > > recipe - it has a default url-config.json, and then a layer specific file gets
>> > >> > > applied from the meta-ibm layer, like:
>> > >> > >
>> > >> > > meta-ibm/recipes-phosphor/interfaces/phosphor-rest/url_config.json
>> > >> > >
>> > >> > > However, from the log.do_unpack from the phosphor-rest build, it doesn't
>> > >> > > look there, it now only looks in phosphor-rest%.  So any override directory without
>> > >> > > a % isn't being found.
>> > >> >
>> > >> > Interesting that the CI didn't catch the break since witherspoon is
>> > >> > one of the systems built via it.  But I'll take a look.
>> >
>> > Yea, our current automated test bucket can't catch this.  (shows there
>> > is still room for improvement)
>> >
>> >
>> > >>
>> > >> Although I have had configuration files disappear when the wildcard is
>> > >> missing from the directory, the recipe normally complains about being
>> > >> unable to find the file.  For a file override, if there's already that
>> > >> file - presumably then it wouldn't complain.
>> >
>> > The latter is what's happening here.
>> >
>> > >
>> > > I've fired off a witherspoon build locally from openbmc HEAD, and will
>> > > report back.  Very few of the recipes were updated in this fashion so
>> > > I'm not terribly worried about a general breakage.
>> >
>> > Thanks for looking.  I'm not sure if there's an easy way to find these
>> > cases.
>> 
>> This specific case is difficult to catch but, it's something that's
>> avoided for most of the renames, so at least there's that.  Future
>> changes though -- one will just have to be more cognizant of matching
>> output.
>> 
>> >
>> > >
>> > >>
>> > >> >
>> > >> > >
>> > >> > > I haven't looked to see what all is broken yet and in how many layers,
>> > >> > > at least our phosphor-rest and phosphor-gevent, which is why the web UI stopped
>> > >> > > working on witherspoon.  Not sure if any other systems are also having problems.
>> > >> > >
>> > >> > >
>> >
> 
> Given it looks like I've already staged the fixes to this, we just
> need to move those forward and when it settles, it should all be
> copacetic.

Those patches do indeed fix it.  Glad everything is already done.  
Thanks!

Re: Recipe Renames, etc

[Patrick Venture]

On Wed, Oct 10, 2018 at 12:17 PM Matt Spinler
<mspinler@linux.vnet.ibm.com> wrote:
>
> On 2018-10-10 14:02, Patrick Venture wrote:
> > On Wed, Oct 10, 2018 at 12:01 PM Patrick Venture <venture@google.com>
> > wrote:
> >>
> >> On Wed, Oct 10, 2018 at 12:00 PM Matt Spinler
> >> <mspinler@linux.vnet.ibm.com> wrote:
> >> >
> >> > On 2018-10-10 13:50, Patrick Venture wrote:
> >> > > On Wed, Oct 10, 2018 at 11:46 AM Patrick Venture <venture@google.com>
> >> > > wrote:
> >> > >>
> >> > >> On Wed, Oct 10, 2018 at 11:45 AM Patrick Venture <venture@google.com>
> >> > >> wrote:
> >> > >> >
> >> > >> > On Wed, Oct 10, 2018 at 11:38 AM Matt Spinler
> >> > >> > <mspinler@linux.vnet.ibm.com> wrote:
> >> > >> > >
> >> > >> > > On 2018-10-02 10:28, Patrick Venture wrote:
> >> > >> > > > I've sent a bunch of recipe renames, but I'm still updating each with
> >> > >> > > > an individual tested footer.  I will try marking them all as WIP now
> >> > >> > > >
> >> > >> > > > Patrick
> >> > >> > >
> >> > >> > >
> >> > >> > > Hi Patrick,
> >> > >> > >
> >> > >> > > I think you broke master with these.  For example, the phosphor-rest
> >> > >> > > recipe - it has a default url-config.json, and then a layer specific file gets
> >> > >> > > applied from the meta-ibm layer, like:
> >> > >> > >
> >> > >> > > meta-ibm/recipes-phosphor/interfaces/phosphor-rest/url_config.json
> >> > >> > >
> >> > >> > > However, from the log.do_unpack from the phosphor-rest build, it doesn't
> >> > >> > > look there, it now only looks in phosphor-rest%.  So any override directory without
> >> > >> > > a % isn't being found.
> >> > >> >
> >> > >> > Interesting that the CI didn't catch the break since witherspoon is
> >> > >> > one of the systems built via it.  But I'll take a look.
> >> >
> >> > Yea, our current automated test bucket can't catch this.  (shows there
> >> > is still room for improvement)
> >> >
> >> >
> >> > >>
> >> > >> Although I have had configuration files disappear when the wildcard is
> >> > >> missing from the directory, the recipe normally complains about being
> >> > >> unable to find the file.  For a file override, if there's already that
> >> > >> file - presumably then it wouldn't complain.
> >> >
> >> > The latter is what's happening here.
> >> >
> >> > >
> >> > > I've fired off a witherspoon build locally from openbmc HEAD, and will
> >> > > report back.  Very few of the recipes were updated in this fashion so
> >> > > I'm not terribly worried about a general breakage.
> >> >
> >> > Thanks for looking.  I'm not sure if there's an easy way to find these
> >> > cases.
> >>
> >> This specific case is difficult to catch but, it's something that's
> >> avoided for most of the renames, so at least there's that.  Future
> >> changes though -- one will just have to be more cognizant of matching
> >> output.
> >>
> >> >
> >> > >
> >> > >>
> >> > >> >
> >> > >> > >
> >> > >> > > I haven't looked to see what all is broken yet and in how many layers,
> >> > >> > > at least our phosphor-rest and phosphor-gevent, which is why the web UI stopped
> >> > >> > > working on witherspoon.  Not sure if any other systems are also having problems.
> >> > >> > >
> >> > >> > >
> >> >
> >
> > Given it looks like I've already staged the fixes to this, we just
> > need to move those forward and when it settles, it should all be
> > copacetic.
>
> Those patches do indeed fix it.  Glad everything is already done.
> Thanks!

So rarely is a problem brought up the solution to which is already in
the queue.  It's like, winning $10 :D

>