serious bugs in KUnit framework makes test completely useless

[Andy Shevchenko <andriy.shevchenko@intel.com>]

Hi!

I have stumbled over the kunit framework issues that make the respective test
cases useless.

Now to the details.
Consider having today's Linux Next.

Scenario 1 (good):

I run

	./tools/testing/kunit/kunit.py config
	./tools/testing/kunit/kunit.py run printf

Everything works as expected:

  [10:19:36] Testing complete. Ran 28 tests: passed: 28
  [10:19:36] Elapsed time: 15.929s total, 0.001s configuring, 15.761s building, 0.114s running


Scenario 2 (BAD):

I applied the following change:

--- a/lib/vsprintf.c
+++ b/lib/vsprintf.c
@@ -18,6 +18,7 @@
  */
 
 #include <linux/stdarg.h>
+#include <linux/bitops.h>
 #include <linux/build_bug.h>
 #include <linux/clk.h>
 #include <linux/clk-provider.h>
@@ -2904,12 +2889,17 @@ int vsnprintf(char *buf, size_t size, const char *fmt_str, va_list args)
 
 		case FORMAT_STATE_NUM: {
 			unsigned long long num;
+			u8 shift = fmt.size * 8 - 1;
 
 			if (fmt.size > sizeof(int))
 				num = va_arg(args, long long);
 			else
-				num = convert_num_spec(va_arg(args, int), fmt.size, spec);
-			str = number(str, end, num, spec);
+				num = va_arg(args, int);
+			num = sign_extend64(num, shift);
+			if (spec.flags & SIGN)
+				str = number(str, end, num, spec);
+			else
+				str = number(str, end, -(long long)num, spec);
 			continue;
 		}
 

Tests went into cosmos (I waited a few minutes and has to interrupt that):

  ^CERROR:root:Build interruption occurred. Cleaning console.
  ^CERROR:root:Build interruption occurred. Cleaning console.
  ^CERROR:root:Build interruption occurred. Cleaning console.
  Command '['.kunit/linux', 'kunit.filter_glob=printf', 'kunit.enable=1', 'mem=1G', 'console=tty', 'kunit_shutdown=halt']' timed out after 300 seconds
  [10:29:52] [ERROR] Test: <missing>: Could not find any KTAP output. Did any KUnit tests run?
  [10:29:52] ============================================================
  [10:29:52] Testing complete. Ran 0 tests: errors: 1
  [10:29:52] Elapsed time: 305.676s total, 0.001s configuring, 5.669s building, 300.006s running

NOTE!
Independently on how long I waited the Elapsed time is about 5 minutes
(Seems 300 seconds limit as stated in the output).


Scenario 3 (BAD):

Now I took again a clean tree and applied this change:

--- a/lib/vsprintf.c
+++ b/lib/vsprintf.c
@@ -18,6 +18,7 @@
  */
 
 #include <linux/stdarg.h>
+#include <linux/bitops.h>
 #include <linux/build_bug.h>
 #include <linux/clk.h>
 #include <linux/clk-provider.h>
@@ -2904,11 +2889,17 @@ int vsnprintf(char *buf, size_t size, const char *fmt_str, va_list args)
 
 		case FORMAT_STATE_NUM: {
 			unsigned long long num;
+			u8 shift = fmt.size * 8 - 1;
 
 			if (fmt.size > sizeof(int))
 				num = va_arg(args, long long);
+			else {
+				num = va_arg(args, int);
+			if ((spec.flags & SIGN))
+				num = sign_extend64(num, shift);
 			else
-				num = convert_num_spec(va_arg(args, int), fmt.size, spec);
+				num &= ~(BIT_ULL(shift) - 1);
+			}
 			str = number(str, end, num, spec);
 			continue;
 		}

and run tests again.

  [10:39:16] [ERROR] Test: <missing>: Could not find any KTAP output. Did any KUnit tests run?
  [10:39:16] ============================================================
  [10:39:16] Testing complete. Ran 0 tests: errors: 1
  [10:39:16] Elapsed time: 5.762s total, 0.001s configuring, 5.694s building, 0.067s running

it runs fast and completely useless. (There is no build error)

...

Please, fix this as it is a serious issue and really makes kunit useless.

-- 
With Best Regards,
Andy Shevchenko