RE: testmgr question

RE: testmgr question

[Pascal Van Leeuwen]

> -----Original Message-----
> From: Eric Biggers [mailto:ebiggers@google.com]
> Sent: Thursday, May 23, 2019 9:06 PM
> To: Pascal Van Leeuwen <pvanleeuwen@insidesecure.com>
> Cc: linux-crypto-owner@vger.kernel.org; Herbert Xu
> <herbert@gondor.apana.org.au>
> Subject: Re: testmgr question
>
> On Thu, May 23, 2019 at 07:01:46AM +0000, Pascal Van Leeuwen wrote:
> > > -----Original Message-----
> > > From: Eric Biggers [mailto:ebiggers@google.com]
> > > Sent: Wednesday, May 22, 2019 6:28 PM
> > > To: Pascal Van Leeuwen <pvanleeuwen@insidesecure.com>
> > > Cc: linux-crypto-owner@vger.kernel.org; Herbert Xu
> > > <herbert@gondor.apana.org.au>
> > > Subject: Re: testmgr question
> > >
> > > On Wed, May 22, 2019 at 01:32:43PM +0000, Pascal Van Leeuwen wrote:
> > > > Ugh,
> > > >
> > > > I just synced my development branch with Linus' mainline tree (5.2-rc1)
> and
> > > > apparently inherited some new testmgr tests that are now failing on the
> > > Inside
> > > > Secure driver. I managed to fix some trivial ones related to non-zero
> IV
> > > size
> > > > on ECB modes and error codes that differed from the expected ones, but
> now
> > > I'm
> > > > rather stuck with a hang case ... and I don't have a clue which
> particular
> > > test
> > > > is hanging or even which algorithm is being tested :-(
> > > >
> > > > Is there, by any chance, some magical debug switch available to make
> > > testmgr
> > > > print out which test it is actually *starting* to run?
> > > >
> > > > Regards,
> > > >
> > > > Pascal van Leeuwen
> > > > Silicon IP Architect, Multi-Protocol Engines @ Inside Secure
> > > > www.insidesecure.com
> > > >
> > >
> > > Not currently, but you can easily add some debugging messages to testmgr
> > > yourself.  E.g.,
> > >
> > > Print 'alg' and 'driver' at beginning of alg_test() to see which
> algorithm is
> > > starting to be tested.
> > >
> > > Print 'vec_name' and 'cfg->name' at beginning of test_hash_vec_cfg(),
> > > test_skcipher_vec_cfg(), and test_aead_vec_cfg() to see which test vector
> is
> > > starting to be tested and under what configuration.
> > >
> > Thanks. I guess adding such debugging statements to testmgr is what I've
> been
> > doing all along. Like everyone else having to debug these issues, I guess
> ...
> > Therefore I assumed by now, there might have been some standard
> infrastructure
> > for that. Or maybe it was just a hint that such a thing might be useful ;-)
> >
>
> testmgr already prints information when a test fails which is enough for most
> cases, and in my experience when it's not I need to add messages specific to
> tracking down the particular issue anyway.  So that's why I haven't
> personally
> added more messages.  Feel free to send a patch, though.  Also, please
> continue
> any further discussion of this on linux-crypto.
>
When developing hardware drivers, when things go wrong, odds are fairly
significant that the whole thing just hangs (or is that just me? :-).
So I can imagine I'm not the only one adding these debug print statements,
which means effort is probably wasted here. But I do notice that I keep
adding and removing them/commenting them out as they're pretty annoying when
things don't actually hang ...

Usually just knowing which specific case fails is enough for me  to reason about
why it's failing. I rarely need more debugging information than that. Following
a hash/HMAC operation is pretty impossible anyway unless you have a reference
implementation standing by to compare with.

Regards,
Pascal van Leeuwen
Silicon IP Architect, Multi-Protocol Engines @ Inside Secure
www.insidesecure.com

Re: testmgr question

[Eric Biggers]

On Thu, May 23, 2019 at 07:44:08PM +0000, Pascal Van Leeuwen wrote:
> > -----Original Message-----
> > From: Eric Biggers [mailto:ebiggers@google.com]
> > Sent: Thursday, May 23, 2019 9:06 PM
> > To: Pascal Van Leeuwen <pvanleeuwen@insidesecure.com>
> > Cc: linux-crypto-owner@vger.kernel.org; Herbert Xu
> > <herbert@gondor.apana.org.au>
> > Subject: Re: testmgr question
> >
> > On Thu, May 23, 2019 at 07:01:46AM +0000, Pascal Van Leeuwen wrote:
> > > > -----Original Message-----
> > > > From: Eric Biggers [mailto:ebiggers@google.com]
> > > > Sent: Wednesday, May 22, 2019 6:28 PM
> > > > To: Pascal Van Leeuwen <pvanleeuwen@insidesecure.com>
> > > > Cc: linux-crypto-owner@vger.kernel.org; Herbert Xu
> > > > <herbert@gondor.apana.org.au>
> > > > Subject: Re: testmgr question
> > > >
> > > > On Wed, May 22, 2019 at 01:32:43PM +0000, Pascal Van Leeuwen wrote:
> > > > > Ugh,
> > > > >
> > > > > I just synced my development branch with Linus' mainline tree (5.2-rc1)
> > and
> > > > > apparently inherited some new testmgr tests that are now failing on the
> > > > Inside
> > > > > Secure driver. I managed to fix some trivial ones related to non-zero
> > IV
> > > > size
> > > > > on ECB modes and error codes that differed from the expected ones, but
> > now
> > > > I'm
> > > > > rather stuck with a hang case ... and I don't have a clue which
> > particular
> > > > test
> > > > > is hanging or even which algorithm is being tested :-(
> > > > >
> > > > > Is there, by any chance, some magical debug switch available to make
> > > > testmgr
> > > > > print out which test it is actually *starting* to run?
> > > > >
> > > > > Regards,
> > > > >
> > > > > Pascal van Leeuwen
> > > > > Silicon IP Architect, Multi-Protocol Engines @ Inside Secure
> > > > > www.insidesecure.com
> > > > >
> > > >
> > > > Not currently, but you can easily add some debugging messages to testmgr
> > > > yourself.  E.g.,
> > > >
> > > > Print 'alg' and 'driver' at beginning of alg_test() to see which
> > algorithm is
> > > > starting to be tested.
> > > >
> > > > Print 'vec_name' and 'cfg->name' at beginning of test_hash_vec_cfg(),
> > > > test_skcipher_vec_cfg(), and test_aead_vec_cfg() to see which test vector
> > is
> > > > starting to be tested and under what configuration.
> > > >
> > > Thanks. I guess adding such debugging statements to testmgr is what I've
> > been
> > > doing all along. Like everyone else having to debug these issues, I guess
> > ...
> > > Therefore I assumed by now, there might have been some standard
> > infrastructure
> > > for that. Or maybe it was just a hint that such a thing might be useful ;-)
> > >
> >
> > testmgr already prints information when a test fails which is enough for most
> > cases, and in my experience when it's not I need to add messages specific to
> > tracking down the particular issue anyway.  So that's why I haven't
> > personally
> > added more messages.  Feel free to send a patch, though.  Also, please
> > continue
> > any further discussion of this on linux-crypto.
> >
> When developing hardware drivers, when things go wrong, odds are fairly
> significant that the whole thing just hangs (or is that just me? :-).
> So I can imagine I'm not the only one adding these debug print statements,
> which means effort is probably wasted here. But I do notice that I keep
> adding and removing them/commenting them out as they're pretty annoying when
> things don't actually hang ...
> 
> Usually just knowing which specific case fails is enough for me  to reason about
> why it's failing. I rarely need more debugging information than that. Following
> a hash/HMAC operation is pretty impossible anyway unless you have a reference
> implementation standing by to compare with.
> 

The verbose debugging messages could be behind a Kconfig option, or just printed
with pr_debug() so that they could be turned on with dynamic debug
(https://www.kernel.org/doc/html/latest/admin-guide/dynamic-debug-howto.html).
Again, feel free to send a patch.

- Eric

testmgr question

[Pascal Van Leeuwen]

Hi,

I'm currently busy adding support for some AES-CTR based authenticated encryption suites to the
inside-secure driver, and I got the distinct impression they were silently not getting tested at all.

Looking at testmgr.c, I noticed that they point to alg_test_null, confirming my suspicion.
I was wondering whether this was intentional and, if so, why that is the case?

I get that there are no reference vectors for these ciphersuites yet, but that shouldn't stop
testmgr from at least fuzzing the ciphersuite against the reference implementation?

Regards,
Pascal van Leeuwen
Silicon IP Architect, Multi-Protocol Engines @ Verimatrix
www.insidesecure.com