Re: af_alg01.c:36: TFAIL: instantiated nested hmac algorithm ('hmac(hmac(md5))')!

[Eric Biggers <ebiggers@kernel.org>]

On Tue, Sep 24, 2024 at 09:30:39PM +0800, Herbert Xu wrote:
> On Tue, Sep 24, 2024 at 03:16:09PM +0530, Naresh Kamboju wrote:
> >
> > Warning log:
> > ----------
> > tst_test.c:1617: TINFO: Timeout per run is 0h 02m 30s
> > af_alg01.c:36: TFAIL: instantiated nested hmac algorithm ('hmac(hmac(md5))')!
> > tst_af_alg.c:46: TBROK: unexpected error binding AF_ALG socket to hash
> > algorithm 'hmac(hmac(md5))': EINVAL (22)
> 
> This is expected.  You need to fix the test to not treat this as an
> error.
> 

So the error code changed from ENOENT to EINVAL.  These particular LTP tests
(af_alg01 and af_alg03) could check for either ENOENT or EINVAL, and it would
preserve the main point of the tests.

I do feel that the previous error code, ENOENT, was more logical though.  From
userspace's perspective, these invalid algorithms don't exist and don't need to
be treated any different from other algorithms that don't exist.

Has it been checked what else in userspace might need to be updated as a result
of this change?

Looking at libkcapi for example
(https://github.com/smuellerDD/libkcapi/blob/master/lib/kcapi.h#L125) it passes
the error code up and documents it as part of its API:

     * @return 0 upon success;
     *         -ENOENT - algorithm not available;
     *         -EOPNOTSUPP - AF_ALG family not available;
     *         -EINVAL - accept syscall failed
     *         -ENOMEM - cipher handle cannot be allocated
     */
    int kcapi_cipher_init(struct kcapi_handle **handle, const char *ciphername,
                          uint32_t flags);

So that is now wrong, as "algorithm not available" can now be either ENOENT or
EINVAL.

I don't really see the reason for this churn.  The kernel commit which made this
change (795f85fca229) provides no explanation either.

- Eric