Re: [PATCH 1/4] lib/crypto: Add ML-DSA verification support

[Eric Biggers <ebiggers@kernel.org>]

On Fri, Nov 21, 2025 at 11:29:16PM +0100, Lukas Wunner wrote:
> On Fri, Nov 21, 2025 at 10:23:09PM +0000, Eric Biggers wrote:
> > That list actually includes the same three files that use -EKEYREJECTED.
> > It looks like if the signature verification fails "early" it's -EBADMSG,
> > whereas if it fails "late" it's -EKEYREJECTED?
> 
> -EBADMSG denotes malformed data (e.g. incorrectly formatted ASN.1 payload).
> 
> -EKEYREJECTED denotes a well-formed, but incorrect signature (e.g. made
> by a wrong key).
> 
> I think it's important and useful to be able to differentiate that.

I guess.  The pseudocode in the ML-DSA specification is clear that
signature verification returns a boolean, regardless of whether the
signature is invalid due to the ctilde check, the coefficients of the
reponse vector being out of range, or the encoded hint vector being
malformed.  But if we really think it's useful we could disregard that
and use EKEYREJECTED for the ctilde check and EBADMSG for the other
cases.  I think that would align with what you're suggesting.  This is
inconsistent with the kernel's symmetric crypto code, but oh well.

- Eric