cancelling the tests says that they all passed

cancelling the tests says that they all passed

[Benno Schulenberg]

When one hits ^C when the compilation phase of 'make check'
finishes and the first tests start executing, the script prints

  All 177 tests PASSED

before aborting.  That... doesn't seem quite right.


When I leave the tests to run to completion, it says:

  4 tests of 177 FAILED

The failed ones are these:

         ipcs: mk-rm-shm                                     ... FAILED (ipcs/mk-rm-shm)
         misc: fallocate                                     ... FAILED (misc/fallocate)
         misc: swaplabel                                     ... FAILED (misc/swaplabel)
         misc: ul                                            ... FAILED (misc/ul)

Benno

-- 
http://www.fastmail.com - The way an email service should be

Re: cancelling the tests says that they all passed

[Ruediger Meier]

On Thursday 17 March 2016, Benno Schulenberg wrote:
> When one hits ^C when the compilation phase of 'make check'
> finishes and the first tests start executing, the script prints

I don't manage to reproduce that. How do you run make exactly (-j)?
If I hit ctrl-C during compilation then the test script does not start 
at all. Maybe there is a dependency problem and your tests start before 
compilation is finished?

>   All 177 tests PASSED

This is because the test script actually means this

  "None of 177 available tests failed"

It does not count the executed tests but failures only.

> before aborting.  That... doesn't seem quite right.

I've had already planned to improve/refactor the failure/success 
counting and also the handling of broken/aborted scripts. Probably this 
would also improve such cosmetical issues.

But this is something for the next release. Would be too risky to change 
something just a few days before releasing 2.28.


> When I leave the tests to run to completion, it says:
>
>   4 tests of 177 FAILED
>
> The failed ones are these:
>
>          ipcs: mk-rm-shm                                     ...
> FAILED (ipcs/mk-rm-shm) misc: fallocate                              
>       ... FAILED (misc/fallocate) misc: swaplabel                    
>                 ... FAILED (misc/swaplabel) misc: ul                 
>                           ... FAILED (misc/ul)
>



Could you show us the test diffs?
$ find tests/diff/ -type f | xargs -r cat

Maybe also warnings durning the test run (if any).

cu,
Rudi

Re: cancelling the tests says that they all passed

[Benno Schulenberg]

[-- Attachment #1: Type: text/plain, Size: 1248 bytes --]


On Thu, Mar 17, 2016, at 11:35, Ruediger Meier wrote:
> On Thursday 17 March 2016, Benno Schulenberg wrote:
> > When one hits ^C when the compilation phase of 'make check'
> > finishes and the first tests start executing, the script prints
> 
> I don't manage to reproduce that. How do you run make exactly (-j)?

Just plain 'make' and 'make check'.

> If I hit ctrl-C during compilation then the test script does not start 
> at all. Maybe there is a dependency problem and your tests start before 
> compilation is finished?

No, I don't hit ^C during compilation but when it has finished,
when that scary warning "Don't execute on production system!"
has been printed and the first "OK"s have appeared.

> Could you show us the test diffs?
> $ find tests/diff/ -type f | xargs -r cat

Attached.

> Maybe also warnings durning the test run (if any).

Only the ipcs one prints warnings:

         ipcs: mk-rm-msg                                     ... OK
         ipcs: mk-rm-sem                                     ... OK
ipcs: id 1048595 not found
ipcs: id 1081363 not found
         ipcs: mk-rm-shm                                     ... FAILED (ipcs/mk-rm-shm)

Benno

-- 
http://www.fastmail.com - A no graphics, no pop-ups email service


[-- Attachment #2: thediffs --]
[-- Type: application/octet-stream, Size: 1655 bytes --]

Re: cancelling the tests says that they all passed

[Ruediger Meier]

On Thursday 17 March 2016, Benno Schulenberg wrote:
> On Thu, Mar 17, 2016, at 11:35, Ruediger Meier wrote:
> > On Thursday 17 March 2016, Benno Schulenberg wrote:
> > > When one hits ^C when the compilation phase of 'make check'
> > > finishes and the first tests start executing, the script prints
> >
> > I don't manage to reproduce that. How do you run make exactly (-j)?
>
> Just plain 'make' and 'make check'.
>
> > If I hit ctrl-C during compilation then the test script does not
> > start at all. Maybe there is a dependency problem and your tests
> > start before compilation is finished?
>
> No, I don't hit ^C during compilation but when it has finished,
> when that scary warning "Don't execute on production system!"
> has been printed and the first "OK"s have appeared.
>
> > Could you show us the test diffs?
> > $ find tests/diff/ -type f | xargs -r cat
>
> Attached.
> 
> > Maybe also warnings durning the test run (if any).
>
> Only the ipcs one prints warnings:
>
>          ipcs: mk-rm-msg                                     ... OK
>          ipcs: mk-rm-sem                                     ... OK
> ipcs: id 1048595 not found
> ipcs: id 1081363 not found
>          ipcs: mk-rm-shm                                     ...
> FAILED (ipcs/mk-rm-shm)

Thanks!

I have some guesses about "ipcs" and "fallocate" but some more 
questions:

1. Which kernel version?
2. Which file system? (findmnt -n -o FSTYPE -T tests/output)
3. Is /proc mounted?
4  grep "FALL" config.h

"swaplabel" test fails because it's using again fallocate. Maybe we 
should use dd/truncate instead to not test fallocate again.

The misc/ul failure is something about different ncurses (or 
terminals?). I know it from BSD. Don't know how this should be fixed in 
our test suite because I don't know much about ncurses stuff.

This is the plain diff:

--- tests/expected/misc/ul
+++ tests/output/misc/ul
@@ -1,3 +1,3 @@
-^[[1ma^[(B^[[mb^[[4mc^[[24m
+^[[1ma^[[m^[(Bb^[[4mc^[[24m


Interpreted by terminal both looks ok: "abc" with bold "a" and 
underlined "c". I sombody confirmes that both strings in the diff are 
totally ok on any system, then I would fix the test by using sed.

cu,
Rudi

Re: cancelling the tests says that they all passed

[Benno Schulenberg]

On Thu, Mar 17, 2016, at 13:50, Ruediger Meier wrote:
> I have some guesses about "ipcs" and "fallocate" but some more 
> questions:
> 
> 1. Which kernel version?

2.6.32

> 2. Which file system? (findmnt -n -o FSTYPE -T tests/output)

ext3

> 3. Is /proc mounted?

$ ./findmnt -n -t proc
/proc  none   proc   rw,nosuid,nodev,noexec,relatime

> 4  grep "FALL" config.h

#define HAVE_FALLOCATE 1
#define HAVE_LINUX_FALLOC_H 1

Benno

-- 
http://www.fastmail.com - A fast, anti-spam email service.

Re: cancelling the tests says that they all passed

[Ruediger Meier]

On Thursday 17 March 2016, Benno Schulenberg wrote:
> On Thu, Mar 17, 2016, at 13:50, Ruediger Meier wrote:
> > I have some guesses about "ipcs" and "fallocate" but some more
> > questions:
> >
> > 1. Which kernel version?
>
> 2.6.32

Hehe my guess was right.

I've noticed already 2 years ago on Debian 6 that our scanf code in 
ipc_shm_get_info() is not compatible to this kernel version. (2.6.34 
worked AFAIR.)

Could you show us
$ cat /proc/sysvipc/shm

Maybe it's easy to fix.

Generally I think we should make a statement about "what is the minimum 
kernel version" for util-linux. Two years ago I decided for myself that 
I should fix bugs for newer kernels first. On the other hand 2.6.32 
would have been worth to support because it was officially maintained 
from 2009 until last month!

> > 2. Which file system? (findmnt -n -o FSTYPE -T tests/output)
>
> ext3

ext3 is not supported but the system call should return an error and the 
test should look like this:
   misc: fallocate ... SKIPPED ('ext3' not supported)

I guess the kernel syscall does not return error thus our fallocate 
succeed but the file is empty. It's a kernel bug we can't fix. Or 
should we add a stat() call to validate the syscall?

> > 3. Is /proc mounted?
>
> $ ./findmnt -n -t proc
> /proc  none   proc   rw,nosuid,nodev,noexec,relatime
>
> > 4  grep "FALL" config.h
>
> #define HAVE_FALLOCATE 1
> #define HAVE_LINUX_FALLOC_H 1
>
> Benno

Re: cancelling the tests says that they all passed

[Benno Schulenberg]

On Thu, Mar 17, 2016, at 18:18, Ruediger Meier wrote:
> Could you show us
> $ cat /proc/sysvipc/shm

       key      shmid perms       size  cpid  lpid nattch   uid   gid  cuid  cgid      atime      dtime      ctime
         0     131072  1600     393216  1409   921      2  1000  1000  1000  1000 1458201259          0 1458201259
         0     163841  1600     393216  1413 19472      2  1000  1000  1000  1000 1458231784 1458231784 1458201259
         0     196610  1600     393216  1399   921      2  1000  1000  1000  1000 1458201260          0 1458201260
         0     229379  1600     393216  1399   921      2  1000  1000  1000  1000 1458201260          0 1458201260
         0     262148  1600     393216  1419  1474      2  1000  1000  1000  1000 1458201262 1458201262 1458201261
         0     294917  1600     393216  1419  1474      2  1000  1000  1000  1000 1458201262 1458201262 1458201261
         0     327686  1600     393216  1388   921      2  1000  1000  1000  1000 1458201262          0 1458201262
         0     360455  1600     393216  1416  2127      2  1000  1000  1000  1000 1458204570 1458204570 1458201265
         0     393224  1600     393216  1424   921      2  1000  1000  1000  1000 1458201265          0 1458201265
         0     425993  1600     393216  1490   921      2  1000  1000  1000  1000 1458201268          0 1458201268
         0     458762  1600     393216  1495   921      2  1000  1000  1000  1000 1458201268          0 1458201268
         0     491531  1600     393216  1508   921      2  1000  1000  1000  1000 1458201268          0 1458201268
         0     524300  1600     393216  1497   921      2  1000  1000  1000  1000 1458201268          0 1458201268
         0     557069  1600     393216  1405 18482      2  1000  1000  1000  1000 1458215908 1458215908 1458201269
         0     589838  1600     393216  1492   921      2  1000  1000  1000  1000 1458201270          0 1458201270
         0     622607  1600     393216  1493   921      2  1000  1000  1000  1000 1458201273          0 1458201273
         0     655376  1600     393216  1496   921      2  1000  1000  1000  1000 1458201274          0 1458201274
         0     688145  1600     393216  1417   921      2  1000  1000  1000  1000 1458201282          0 1458201282
         0     720914  1600     393216  1413 19472      2  1000  1000  1000  1000 1458231784 1458231784 1458201449
 -60920149    1081363   644         12 10036     0      0  1000  1000  1000  1000          0          0 1458215137
         0    1114132  1600     393216  1417   921      2  1000  1000  1000  1000 1458215476          0 1458215476

> Generally I think we should make a statement about "what is the minimum 
> kernel version" for util-linux.

A statement.  And let the tests check the version and print
a warning in case it's too old.

> Two years ago I decided for myself that 
> I should fix bugs for newer kernels first. On the other hand 2.6.32 
> would have been worth to support because it was officially maintained 
> from 2009 until last month!

I'm not expecting you to fix things.  It's just nice to put
things in a harsher environment and see how they hold up.  :)

> I guess the kernel syscall does not return error thus our fallocate 
> succeed but the file is empty. It's a kernel bug we can't fix. Or 
> should we add a stat() call to validate the syscall?

That sounds like a good idea: check, and double check.

Benno

-- 
http://www.fastmail.com - Accessible with your email software
                          or over the web

Re: cancelling the tests says that they all passed

[Ruediger Meier]

On Thursday 17 March 2016, Benno Schulenberg wrote:
> On Thu, Mar 17, 2016, at 18:18, Ruediger Meier wrote:
> > Could you show us
> > $ cat /proc/sysvipc/shm

>   key      shmid perms       size  cpid  lpid nattch   uid   gid  cuid  cgid      atime      dtime      ctime
>     0     131072  1600     393216  1409   921      2  1000  1000  1000  1000 1458201259          0 1458201259

Ok, nowadays we have just two more columns at the end: "rss" and "swap".
Could be fixed probably. We have also still fallback code for the old
syscall which is used if there is no /proc at all.

Actually it was on my imaginary TODO list to review all /proc scanf
code for robustness against future column additions.

>
> > Generally I think we should make a statement about "what is the
> > minimum kernel version" for util-linux.
>
> A statement.  And let the tests check the version and print
> a warning in case it's too old.

Actually a failed test is the best "warning" (based on facts rather
than parsed version numbers):
  FAILED == "Stop! something is wrong with ul and your system."

But IMO ok to give users a hint like "your kernel might be too old".
This warning could be repeated when there are failed tests.

On the other hand this reminds me about all these wrong >2.6 version
checks, and why uname26 was invented :)

> > Two years ago I decided for myself that
> > I should fix bugs for newer kernels first. On the other hand 2.6.32
> > would have been worth to support because it was officially
> > maintained from 2009 until last month!
>
> I'm not expecting you to fix things.  It's just nice to put
> things in a harsher environment and see how they hold up.  :)

> > I guess the kernel syscall does not return error thus our fallocate
> > succeed but the file is empty. It's a kernel bug we can't fix. Or
> > should we add a stat() call to validate the syscall?
>
> That sounds like a good idea: check, and double check.
>
> Benno