Re: [PATCH v2 2/4] block/ssh.c: Don't double-check that characters are hex digits

[Richard Henderson <richard.henderson@linaro.org>]

On 10/8/24 09:47, Peter Maydell wrote:
> In compare_fingerprint() we effectively check whether the characters
> in the fingerprint are valid hex digits twice: first we do so with
> qemu_isxdigit(), but then the hex2decimal() function also has a code
> path where it effectively detects an invalid digit and returns -1.
> This causes Coverity to complain because it thinks that we might use
> that -1 value in an expression where it would be an integer overflow.
> 
> Avoid the double-check of hex digit validity by testing the return
> values from hex2decimal() rather than doing separate calls to
> qemu_isxdigit().
> 
> Since this means we now use the illegal-character return value
> from hex2decimal(), rewrite it from "-1" to "UINT_MAX", which
> has the same effect since the return type is "unsigned" but
> looks less confusing at the callsites when we detect it with
> "c0 > 0xf".
> 
> Resolves: Coverity CID 1547813
> Signed-off-by: Peter Maydell<peter.maydell@linaro.org>
> Reviewed-by: Kevin Wolf<kwolf@redhat.com>
> ---
> v1->v2: make hex2decimal() return UINT_MAX, not -1
> ---
>   block/ssh.c | 12 +++++++-----
>   1 file changed, 7 insertions(+), 5 deletions(-)

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>

r~