[PATCH] Lower priority of debug messages that should not be shown on the console

[PATCH] Lower priority of debug messages that should not be shown on the console

[Adrian Bunk]

Signed-off-by: Adrian Bunk <bunk@stusta.de>
---
 bitbake/lib/bb/runqueue.py | 8 ++++----
 bitbake/lib/bb/siggen.py   | 6 +++---
 2 files changed, 7 insertions(+), 7 deletions(-)

diff --git a/bitbake/lib/bb/runqueue.py b/bitbake/lib/bb/runqueue.py
index 71108eeed7..f6bd8db8ce 100644
--- a/bitbake/lib/bb/runqueue.py
+++ b/bitbake/lib/bb/runqueue.py
@@ -2054,7 +2054,7 @@ class RunQueueExecute:
         self.update_holdofftasks()
 
         if not self.sq_live and not self.sqdone and not self.sq_deferred and not self.updated_taskhash_queue and not self.holdoff_tasks:
-            logger.info("Setscene tasks completed")
+            logger.debug(1, "Setscene tasks completed")
 
             err = self.summarise_scenequeue_errors()
             if err:
@@ -2259,7 +2259,7 @@ class RunQueueExecute:
             self.updated_taskhash_queue.remove((tid, unihash))
 
             if unihash != self.rqdata.runtaskentries[tid].unihash:
-                logger.info("Task %s unihash changed to %s" % (tid, unihash))
+                logger.debug(1, "Task %s unihash changed to %s" % (tid, unihash))
                 self.rqdata.runtaskentries[tid].unihash = unihash
                 bb.parse.siggen.set_unihash(tid, unihash)
                 toprocess.add(tid)
@@ -2304,7 +2304,7 @@ class RunQueueExecute:
                 elif tid in self.scenequeue_covered or tid in self.sq_live:
                     # Already ran this setscene task or it running. Report the new taskhash
                     bb.parse.siggen.report_unihash_equiv(tid, newhash, origuni, newuni, self.rqdata.dataCaches)
-                    logger.info("Already covered setscene for %s so ignoring rehash (remap)" % (tid))
+                    logger.debug(1, "Already covered setscene for %s so ignoring rehash (remap)" % (tid))
                     remapped = True
 
                 if not remapped:
@@ -2405,7 +2405,7 @@ class RunQueueExecute:
 
         for (tid, harddepfail, origvalid) in update_tasks:
             if tid in self.sqdata.valid and not origvalid:
-                logger.info("Setscene task %s became valid" % tid)
+                logger.debug(1, "Setscene task %s became valid" % tid)
             if harddepfail:
                 self.sq_task_failoutright(tid)
 
diff --git a/bitbake/lib/bb/siggen.py b/bitbake/lib/bb/siggen.py
index c2d0c736cf..d5c5c5c0af 100644
--- a/bitbake/lib/bb/siggen.py
+++ b/bitbake/lib/bb/siggen.py
@@ -606,7 +606,7 @@ class SignatureGeneratorUniHashMixIn(object):
                 method = method + self.extramethod[tid]
 
             data = self.client().report_unihash_equiv(taskhash, method, wanted_unihash, extra_data)
-            bb.note('Reported task %s as unihash %s to %s (%s)' % (tid, wanted_unihash, self.server, str(data)))
+            bb.debug(1, 'Reported task %s as unihash %s to %s (%s)' % (tid, wanted_unihash, self.server, str(data)))
 
             if data is None:
                 bb.warn("Server unable to handle unihash report")
@@ -615,9 +615,9 @@ class SignatureGeneratorUniHashMixIn(object):
             finalunihash = data['unihash']
 
             if finalunihash == current_unihash:
-                bb.note('Task %s unihash %s unchanged by server' % (tid, finalunihash))
+                bb.debug(1, 'Task %s unihash %s unchanged by server' % (tid, finalunihash))
             elif finalunihash == wanted_unihash:
-                bb.note('Task %s unihash changed %s -> %s as wanted' % (tid, current_unihash, finalunihash))
+                bb.debug(1, 'Task %s unihash changed %s -> %s as wanted' % (tid, current_unihash, finalunihash))
                 self.set_unihash(tid, finalunihash)
                 return True
             else:
-- 
2.17.1

Re: [PATCH] Lower priority of debug messages that should not be shown on the console

[Andreas Müller]

On Fri, Feb 21, 2020 at 10:00 PM Adrian Bunk <bunk@stusta.de> wrote:
>
> Signed-off-by: Adrian Bunk <bunk@stusta.de>
> ---
>  bitbake/lib/bb/runqueue.py | 8 ++++----
>  bitbake/lib/bb/siggen.py   | 6 +++---
>  2 files changed, 7 insertions(+), 7 deletions(-)
>
> diff --git a/bitbake/lib/bb/runqueue.py b/bitbake/lib/bb/runqueue.py
> index 71108eeed7..f6bd8db8ce 100644
> --- a/bitbake/lib/bb/runqueue.py
> +++ b/bitbake/lib/bb/runqueue.py
> @@ -2054,7 +2054,7 @@ class RunQueueExecute:
>          self.update_holdofftasks()
>
>          if not self.sq_live and not self.sqdone and not self.sq_deferred and not self.updated_taskhash_queue and not self.holdoff_tasks:
> -            logger.info("Setscene tasks completed")
> +            logger.debug(1, "Setscene tasks completed")
^ Could you please leave this one - I consider it helpful
>
>              err = self.summarise_scenequeue_errors()
>              if err:
> @@ -2259,7 +2259,7 @@ class RunQueueExecute:
>              self.updated_taskhash_queue.remove((tid, unihash))
>
>              if unihash != self.rqdata.runtaskentries[tid].unihash:
> -                logger.info("Task %s unihash changed to %s" % (tid, unihash))
> +                logger.debug(1, "Task %s unihash changed to %s" % (tid, unihash))
>                  self.rqdata.runtaskentries[tid].unihash = unihash
>                  bb.parse.siggen.set_unihash(tid, unihash)
>                  toprocess.add(tid)
> @@ -2304,7 +2304,7 @@ class RunQueueExecute:
>                  elif tid in self.scenequeue_covered or tid in self.sq_live:
>                      # Already ran this setscene task or it running. Report the new taskhash
>                      bb.parse.siggen.report_unihash_equiv(tid, newhash, origuni, newuni, self.rqdata.dataCaches)
> -                    logger.info("Already covered setscene for %s so ignoring rehash (remap)" % (tid))
> +                    logger.debug(1, "Already covered setscene for %s so ignoring rehash (remap)" % (tid))
>                      remapped = True
>
>                  if not remapped:
> @@ -2405,7 +2405,7 @@ class RunQueueExecute:
>
>          for (tid, harddepfail, origvalid) in update_tasks:
>              if tid in self.sqdata.valid and not origvalid:
> -                logger.info("Setscene task %s became valid" % tid)
> +                logger.debug(1, "Setscene task %s became valid" % tid)
>              if harddepfail:
>                  self.sq_task_failoutright(tid)
>
> diff --git a/bitbake/lib/bb/siggen.py b/bitbake/lib/bb/siggen.py
> index c2d0c736cf..d5c5c5c0af 100644
> --- a/bitbake/lib/bb/siggen.py
> +++ b/bitbake/lib/bb/siggen.py
> @@ -606,7 +606,7 @@ class SignatureGeneratorUniHashMixIn(object):
>                  method = method + self.extramethod[tid]
>
>              data = self.client().report_unihash_equiv(taskhash, method, wanted_unihash, extra_data)
> -            bb.note('Reported task %s as unihash %s to %s (%s)' % (tid, wanted_unihash, self.server, str(data)))
> +            bb.debug(1, 'Reported task %s as unihash %s to %s (%s)' % (tid, wanted_unihash, self.server, str(data)))
>
>              if data is None:
>                  bb.warn("Server unable to handle unihash report")
> @@ -615,9 +615,9 @@ class SignatureGeneratorUniHashMixIn(object):
>              finalunihash = data['unihash']
>
>              if finalunihash == current_unihash:
> -                bb.note('Task %s unihash %s unchanged by server' % (tid, finalunihash))
> +                bb.debug(1, 'Task %s unihash %s unchanged by server' % (tid, finalunihash))
>              elif finalunihash == wanted_unihash:
> -                bb.note('Task %s unihash changed %s -> %s as wanted' % (tid, current_unihash, finalunihash))
> +                bb.debug(1, 'Task %s unihash changed %s -> %s as wanted' % (tid, current_unihash, finalunihash))
>                  self.set_unihash(tid, finalunihash)
>                  return True
>              else:
> --
> 2.17.1
>
> --
> _______________________________________________
> bitbake-devel mailing list
> bitbake-devel@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/bitbake-devel

Re: [PATCH] Lower priority of debug messages that should not be shown on the console

[Adrian Bunk]

On Sat, Feb 22, 2020 at 12:57:47PM +0100, Andreas Müller wrote:
> On Fri, Feb 21, 2020 at 10:00 PM Adrian Bunk <bunk@stusta.de> wrote:
>...
> > --- a/bitbake/lib/bb/runqueue.py
> > +++ b/bitbake/lib/bb/runqueue.py
> > @@ -2054,7 +2054,7 @@ class RunQueueExecute:
> >          self.update_holdofftasks()
> >
> >          if not self.sq_live and not self.sqdone and not self.sq_deferred and not self.updated_taskhash_queue and not self.holdoff_tasks:
> > -            logger.info("Setscene tasks completed")
> > +            logger.debug(1, "Setscene tasks completed")
> ^ Could you please leave this one - I consider it helpful
>...

It would be helpful if it would be shown only once.

I've seen it 10 times during a build where
"Setscene task .. became valid" and "Already covered setscene for ..."
also happened a few times.


cu
Adrian

Re: [PATCH] Lower priority of debug messages that should not be shown on the console

[Richard Purdie]

On Fri, 2020-02-21 at 22:52 +0200, Adrian Bunk wrote:
> Signed-off-by: Adrian Bunk <bunk@stusta.de>
> ---
>  bitbake/lib/bb/runqueue.py | 8 ++++----
>  bitbake/lib/bb/siggen.py   | 6 +++---
>  2 files changed, 7 insertions(+), 7 deletions(-)

The trouble here is that if we remove these messages (and I can
understand why people want to), if there is a problem with hashequiv, I
can simply look at the result and say "yes, it broke" with zero ability
to understand why as we can't tell what happened in the build.

The correct fix is probably to change the console log messages to log
differently to the on disk log file. Even then, it means we can't debug
autobuilder failures unless we save those logs.

So in summary, there is a choice:

a) We can compel me to merge patches like this. I will then just have
to close all hashequiv bug reports as WORKSFORME.

b) We actually fix/improve the way logging works.

I'd prefer b).

Cheers,

Richard

Re: [PATCH] Lower priority of debug messages that should not be shown on the console

[Adrian Bunk]

On Sat, Feb 22, 2020 at 03:54:54PM +0000, Richard Purdie wrote:
>...
> b) We actually fix/improve the way logging works.
> 
> I'd prefer b).

Makes sense, I've submitted this as bug #13813.

> Cheers,
> 
> Richard

cu
Adrian

Re: [PATCH] Lower priority of debug messages that should not be shown on the console

[Richard Purdie]

On Sun, 2020-02-23 at 21:51 +0200, Adrian Bunk wrote:
> On Sat, Feb 22, 2020 at 03:54:54PM +0000, Richard Purdie wrote:
> > ...
> > b) We actually fix/improve the way logging works.
> > 
> > I'd prefer b).
> 
> Makes sense, I've submitted this as bug #13813.

I note you believe this has to be fixed before we reach M3. I guess I'd
agree, I just don't know who is going to do it.

Bitbake bugs get auto assigned to me. I physically cannot actually keep
up with the incoming bugs despite working more than I should.

I was actually trying to do something about my bug count as its
seriously depressing me. For each one I try and fix I seem to find
three more intermittent issues on the autobuilder :(.

I know you didn't mean it this way and we need to track the issues, I
just don't know how we're going to handle it.

Cheers,

Richard

Re: [PATCH] Lower priority of debug messages that should not be shown on the console

[Adrian Bunk]

On Sun, Feb 23, 2020 at 11:53:57PM +0000, Richard Purdie wrote:
> On Sun, 2020-02-23 at 21:51 +0200, Adrian Bunk wrote:
> > On Sat, Feb 22, 2020 at 03:54:54PM +0000, Richard Purdie wrote:
> > > ...
> > > b) We actually fix/improve the way logging works.
> > > 
> > > I'd prefer b).
> > 
> > Makes sense, I've submitted this as bug #13813.
> 
> I note you believe this has to be fixed before we reach M3. I guess I'd
> agree, I just don't know who is going to do it.

I would have said M4, but that doesn't really change anything.

> Bitbake bugs get auto assigned to me. I physically cannot actually keep
> up with the incoming bugs despite working more than I should.
> 
> I was actually trying to do something about my bug count as its
> seriously depressing me. For each one I try and fix I seem to find
> three more intermittent issues on the autobuilder :(.

I know this might sound brutal, but in hindsight the hash equivalence 
work was not a good idea with the current resourcing.

I fully understand why it is attractive to properly fix a long-standing
problem, but the hash equivalence work and followup fixes might have 
been the difference between an OKish workload and working more than
you should have done.

> I know you didn't mean it this way and we need to track the issues, I
> just don't know how we're going to handle it.

From my side the problem is that my initial patch is what I can do with 
reasonable effort.

> Cheers,
> 
> Richard

cu
Adrian

Re: [PATCH] Lower priority of debug messages that should not be shown on the console

[Richard Purdie]

On Mon, 2020-02-24 at 07:49 +0200, Adrian Bunk wrote:
> On Sun, Feb 23, 2020 at 11:53:57PM +0000, Richard Purdie wrote:
> > On Sun, 2020-02-23 at 21:51 +0200, Adrian Bunk wrote:
> > > On Sat, Feb 22, 2020 at 03:54:54PM +0000, Richard Purdie wrote:
> > > > ...
> > > > b) We actually fix/improve the way logging works.
> > > > 
> > > > I'd prefer b).
> > > 
> > > Makes sense, I've submitted this as bug #13813.
> > 
> > I note you believe this has to be fixed before we reach M3. I guess
> > I'd
> > agree, I just don't know who is going to do it.
> 
> I would have said M4, but that doesn't really change anything.

Technically its feature work although I guess it can be argued both
ways as its also a usability/bug fix.

> > Bitbake bugs get auto assigned to me. I physically cannot actually
> > keep
> > up with the incoming bugs despite working more than I should.
> > 
> > I was actually trying to do something about my bug count as its
> > seriously depressing me. For each one I try and fix I seem to find
> > three more intermittent issues on the autobuilder :(.
> 
> I know this might sound brutal, but in hindsight the hash
> equivalence work was not a good idea with the current resourcing.

Not brutal, in hindsight it hasn't been such a good idea.

> I fully understand why it is attractive to properly fix a long-
> standing
> problem, but the hash equivalence work and followup fixes might have 
> been the difference between an OKish workload and working more than
> you should have done.

Agreed, and now why I kind of feel guilty if I don't resolve some of
the issues. We could just turn it off again I guess, I keep wondering
about that.

> > I know you didn't mean it this way and we need to track the issues,
> > I
> > just don't know how we're going to handle it.
> 
> From my side the problem is that my initial patch is what I can do
> with reasonable effort.

Part of the reason I'm highlighting this is that I have a lot of
feedback that says people don't believe there is any problem with our
resourcing. Unless I can show where there are issues, its not going to
change :/.

Cheers,

Richard

Re: [PATCH] Lower priority of debug messages that should not be shown on the console

[Adrian Bunk]

On Mon, Feb 24, 2020 at 09:12:06AM +0000, Richard Purdie wrote:
> On Mon, 2020-02-24 at 07:49 +0200, Adrian Bunk wrote:
> > On Sun, Feb 23, 2020 at 11:53:57PM +0000, Richard Purdie wrote:
>...
> > > I know you didn't mean it this way and we need to track the issues,
> > > I
> > > just don't know how we're going to handle it.
> > 
> > From my side the problem is that my initial patch is what I can do
> > with reasonable effort.
> 
> Part of the reason I'm highlighting this is that I have a lot of
> feedback that says people don't believe there is any problem with our
> resourcing. Unless I can show where there are issues, its not going to
> change :/.

Why does YP give users false/misleading information regarding security
support of stable and LTS releases despite lack of resources to provide
security support?

There is no reason to believe claims that there would be any problem 
with resourcing when the public announcements from YP give the opposite 
impression.

> Cheers,
> 
> Richard

cu
Adrian

Re: [PATCH] Lower priority of debug messages that should not be shown on the console

[Richard Purdie]

On Fri, 2020-03-06 at 12:24 +0200, Adrian Bunk wrote:
> On Mon, Feb 24, 2020 at 09:12:06AM +0000, Richard Purdie wrote:
> > On Mon, 2020-02-24 at 07:49 +0200, Adrian Bunk wrote:
> > > On Sun, Feb 23, 2020 at 11:53:57PM +0000, Richard Purdie wrote:
> > ...
> > > > I know you didn't mean it this way and we need to track the
> > > > issues,
> > > > I
> > > > just don't know how we're going to handle it.
> > > 
> > > From my side the problem is that my initial patch is what I can
> > > do
> > > with reasonable effort.
> > 
> > Part of the reason I'm highlighting this is that I have a lot of
> > feedback that says people don't believe there is any problem with
> > our
> > resourcing. Unless I can show where there are issues, its not going
> > to
> > change :/.
> 
> Why does YP give users false/misleading information regarding
> security support of stable and LTS releases despite lack of resources
> to provide security support?

See my other longer reply to you on oe-arch.

I think the project is trying to improve the security support we can
offer on stable releases as they become an LTS. No its not perfect and
we're not offering guarantees but is is an improvement.

The project got to pick where it allocated funding and it chose to
improve LTS and security over for example helping me with day to day
patch review/testing and build failure work. I don't mind, we have to
start somewhere to improve things and LTS/security is important.

The project does already fund some "day to day" running by funding me
to work on bitbake/OE-Core. Yes, it would be helpful if it wasn't just
me, maybe in the future if we can show success of the LTS we may
attract more members and I can get some help and stop burning out, time
will tell.

> There is no reason to believe claims that there would be any problem 
> with resourcing when the public announcements from YP give the
> opposite impression.

As I've said in a few places now, there will be some comments from the
project about this. I have already drafted something about resourcing
but there were concerns it would cause panic and I've agreed to reword
it to try not to do that. There isn't reason to panic, we do need more
help and we will communicate that when its ready. You'd be amazed at
the work in writing public statements and your reaction to the LTS
announcement will sadly be held as an example of why I shouldn't talk
about resourcing.

Cheers,

Richard

Re: [PATCH] Lower priority of debug messages that should not be shown on the console

[Alexander Kanavin]

[-- Attachment #1: Type: text/plain, Size: 503 bytes --]

On Fri, 6 Mar 2020 at 11:48, Richard Purdie <
richard.purdie@linuxfoundation.org> wrote:

> You'd be amazed at
> the work in writing public statements and your reaction to the LTS
> announcement will sadly be held as an example of why I shouldn't talk
> about resourcing.
>

I am fairly sure that example is an outlier, and not a common data point.
Both in substance (claiming that it's easy to misrepresent the LTS
announcement - not true), and in tone (various toxic accusations).

Alex

[-- Attachment #2: Type: text/html, Size: 833 bytes --]