Inconsistent results from script test case

Inconsistent results from script test case

[Paul Barker]

[-- Attachment #1: Type: text/plain, Size: 2088 bytes --]

Hi,

On the Yocto Project autobuilder we have seen occasional failures of the
util-linux test suite over the past year or so.

After investigating this, the failing test is the "size" subtest in
tests/ts/script/options. Occasionally, fewer characters than expected
are read from the pty and the test output doesn't match the expected
output.

Running the tested command in a loop locally, with heavy background load
on the CPUs, showed inconsistent results.

  for i in $(seq 0 4095); do \
    SCRIPT_TEST_SECOND_SINCE_EPOCH=1432489398 TZ=GMT \
    ../util-linux/test_script --output-limit 9 --command "echo 1:1234567890" test$i > /dev/null < /dev/null 2>&1 ; \
    done

This generated 3849 runs with the expected output, where there is a
blank line after the echo command output (94%):

  Script started on 2015-05-24 17:43:18+00:00 [COMMAND="echo 1:1234567890" <not executed on terminal>]
  1:1234567890
  
  Script done on 2015-05-24 17:43:18+00:00 [<max output size exceeded>]

And 247 runs without the blank line (6%):

  Script started on 2015-05-24 17:43:18+00:00 [COMMAND="echo 1:1234567890" <not executed on terminal>]
  1:1234567890
  Script done on 2015-05-24 17:43:18+00:00 [<max output size exceeded>]

This test was performed on the current HEAD of util-linux, commit
e4656fa9765f ("zramctl: Add note about column descriptions").

I see two possible ways to fix this flaky test:

1) Modify `script` to never print more characters than the requested
   output limit. This will ensure that the generated output is always
   consistent, regardless of the influence of I/O buffering.

2) Modify the test case to accept both of the above possible output
   variants.

We would like some input from folks more familiar with util-linux on
which of those options is preferred. For option (1) I should be able to
send a patch, for option (2) more invasive changes to the test framework
may be needed to allow for multiple expected outputs so I may not be
able to contribute that change myself.

Best regards,

-- 
Paul Barker


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 252 bytes --]