Re: [PATCH v2 2/2] perf test: Display number of remaining tests

[Namhyung Kim <namhyung@kernel.org>]

On Tue, Jul 02, 2024 at 09:30:44PM -0700, Ian Rogers wrote:
> On Tue, Jul 2, 2024 at 8:40 PM Namhyung Kim <namhyung@kernel.org> wrote:
> >
> > On Sun, Jun 30, 2024 at 09:42:36PM -0700, Ian Rogers wrote:
> > > Before polling or sleeping to wait for a test to complete, print out
> > > ": Running (<num> remaining)" where the number of remaining tests is
> > > determined by iterating over the remaining tests and seeing which
> > > return true for check_if_command_finished. After the delay, erase the
> > > line and either update it with the new number of remaining tests, or
> > > print the test's result. This allows a user to know a test is running
> > > and in parallel mode (default) how many of the tests are waiting to
> >
> > It's not default anymore. :)
> >
> >
> > > complete. If color mode is disabled then avoid displaying the
> > > "Running" message.
> > >
> > > Signed-off-by: Ian Rogers <irogers@google.com>
> > > ---
> > >  tools/perf/tests/builtin-test.c | 77 ++++++++++++++++++++++-----------
> > >  tools/perf/util/color.h         |  1 +
> > >  2 files changed, 53 insertions(+), 25 deletions(-)
> > >
> > > diff --git a/tools/perf/tests/builtin-test.c b/tools/perf/tests/builtin-test.c
> > > index c3d84b67ca8e..23be9139f229 100644
> > > --- a/tools/perf/tests/builtin-test.c
> > > +++ b/tools/perf/tests/builtin-test.c
> > > @@ -241,7 +241,10 @@ static int run_test_child(struct child_process *process)
> > >       return -err;
> > >  }
> > >
> > > -static int print_test_result(struct test_suite *t, int i, int subtest, int result, int width)
> > > +#define TEST_RUNNING -3
> > > +
> > > +static int print_test_result(struct test_suite *t, int i, int subtest, int result, int width,
> > > +                          int remaining)
> > >  {
> > >       if (has_subtests(t)) {
> > >               int subw = width > 2 ? width - 2 : width;
> > > @@ -251,6 +254,9 @@ static int print_test_result(struct test_suite *t, int i, int subtest, int resul
> > >               pr_info("%3d: %-*s:", i + 1, width, test_description(t, subtest));
> > >
> > >       switch (result) {
> > > +     case TEST_RUNNING:
> > > +             color_fprintf(stderr, PERF_COLOR_YELLOW, " Running (%d remaining)\n", remaining);
> > > +             break;
> > >       case TEST_OK:
> > >               pr_info(" Ok\n");
> > >               break;
> > > @@ -272,13 +278,15 @@ static int print_test_result(struct test_suite *t, int i, int subtest, int resul
> > >       return 0;
> > >  }
> > >
> > > -static int finish_test(struct child_test *child_test, int width)
> > > +static int finish_test(struct child_test **child_tests, int running_test, int child_test_num,
> > > +                    int width)
> > >  {
> > > +     struct child_test *child_test = child_tests[running_test];
> > >       struct test_suite *t = child_test->test;
> > >       int i = child_test->test_num;
> > >       int subi = child_test->subtest;
> > >       int err = child_test->process.err;
> > > -     bool err_done = err <= 0;
> > > +     bool err_done = false;
> > >       struct strbuf err_output = STRBUF_INIT;
> > >       int ret;
> > >
> > > @@ -293,7 +301,7 @@ static int finish_test(struct child_test *child_test, int width)
> > >        * Busy loop reading from the child's stdout/stderr that are set to be
> > >        * non-blocking until EOF.
> > >        */
> > > -     if (!err_done)
> > > +     if (err > 0)
> > >               fcntl(err, F_SETFL, O_NONBLOCK);
> > >       if (verbose > 1) {
> > >               if (has_subtests(t))
> > > @@ -307,29 +315,48 @@ static int finish_test(struct child_test *child_test, int width)
> > >                         .events = POLLIN | POLLERR | POLLHUP | POLLNVAL,
> > >                       },
> > >               };
> > > -             char buf[512];
> > > -             ssize_t len;
> > > -
> > > -             /* Poll to avoid excessive spinning, timeout set for 100ms. */
> > > -             poll(pfds, ARRAY_SIZE(pfds), /*timeout=*/100);
> > > -             if (!err_done && pfds[0].revents) {
> > > -                     errno = 0;
> > > -                     len = read(err, buf, sizeof(buf) - 1);
> > > -
> > > -                     if (len <= 0) {
> > > -                             err_done = errno != EAGAIN;
> > > -                     } else {
> > > -                             buf[len] = '\0';
> > > -                             if (verbose > 1)
> > > -                                     fprintf(stdout, "%s", buf);
> > > -                             else
> > > +             if (perf_use_color_default) {
> > > +                     int tests_in_progress = running_test;
> > > +
> > > +                     for (int y = running_test; y < child_test_num; y++) {
> > > +                             if (check_if_command_finished(&child_tests[y]->process))
> > > +                                     tests_in_progress++;
> > > +                     }
> > > +                     print_test_result(t, i, subi, TEST_RUNNING, width,
> > > +                                       child_test_num - tests_in_progress);
> > > +             }
> > > +
> > > +             err_done = true;
> > > +             if (err <= 0) {
> > > +                     /* No child stderr to poll, sleep for 10ms for child to complete. */
> > > +                     usleep(10 * 1000);
> > > +             } else {
> > > +                     /* Poll to avoid excessive spinning, timeout set for 100ms. */
> > > +                     poll(pfds, ARRAY_SIZE(pfds), /*timeout=*/100);
> >
> > When I tested this patch, I saw it refreshes too often in parallel mode.
> > Maybe 100ms is too short?  I don't know if it's from usleep (10ms) or
> > here.
> 
> It's usually the poll and I suspect it is the test writing a lot of
> output. I agree it can look a little flickery but it is also
> responsive in terms of not waiting too long before moving to the next
> test. I think it is possible to improve on the code here, the main
> thing I was after was making the output writing self contained and not
> split between start test and finish test, as that won't work well in
> the parallel case.

Is it possible to skip the rewriting if nothing is changed?

Thanks,
Namhyung