Re: [PATCH 0/1] openssl: update to 1.0.2i (CVE-2016-6304 and more)

[Patrick Ohly <patrick.ohly@intel.com>]

On Fri, 2016-09-23 at 15:11 +0300, Alexander Kanavin wrote:
> On 09/23/2016 01:27 PM, Patrick Ohly wrote:
> >
> > There is one FAIL:
> >
> > ../util/shlib_wrap.sh ./dtlstest ../apps/server.pem ../apps/server.pem
> > Starting Test 0
> > Failed to load server certificate
> > Unable to create SSL_CTX pair
> > make[2]: Leaving directory '/usr/lib/openssl/ptest/test'
> > FAIL: test_dtls
> >
> > That's because server.pem wasn't installed. I'll fix that.
> >
> > However, ptest-runner returns with 0, i.e. success? Should it do that?
> 
> What does the failing test itself return? After checking the 
> ptest-runner source code, it shouldn't return 0 if one of the tests it 
> runs fails with a non-zero exit.

openssl's test/Makefile is the culprit:

alltests:               
        @(for i in $(all-tests); do \
        ( $(MAKE) $$i && echo "PASS: $$i" ) || echo "FAIL: $$i"; \                                                   
        done)                                                                                                      

If any test fails, it'll print FAIL, but won't cause make to fail and
thus the error never results in a non-zero exit code anywhere.

Here's a version which reports the problem via the return code:

alltests:               
        @(result=0; for i in $(all-tests); do \
        if $(MAKE) $$i; then echo "PASS: $$i"; else echo "FAIL: $$i"; result=1; fi; \                                 
        done; exit $$result)                

OpenSSL seems to rely on output checking. Not sure whether a patch
changing that would be accepted.

How are ptests used in the autobuilders? Does the return code of
ptest-runner matter, or is the output checked for ^PASS|SKIP|FAIL?

Speaking of the autobuilders and openssl-ptest in general, has no-one
noticed before that occasionally tests fail because file time stamps
imply that recompilation is needed? I got that a few times now and will
send a fix. I'm just wondering why that wasn't a problem earlier.

-- 
Best Regards, Patrick Ohly

The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.