Re: CAAM RSA breaks cfg80211 certificate verification on iMX8QXP

[Vitor Soares <ivitro@gmail.com>]

On Wed, 2025-11-26 at 13:59 +0100, Ahmad Fatoum wrote:
> Hello Vitor,
> 
> On 26.11.25 11:55, Vitor Soares wrote:
> > ++imx@lists.linux.dev
> > 
> > On Mon, 2025-11-24 at 19:03 +0000, Vitor Soares wrote:
> > > I’m currently investigating an issue on our Colibri iMX8QXP SoM running
> > > kernel
> > > 6.18-rc6 (also reproducible on v6.17), where cfg80211 fails to load the
> > > compiled-in X.509 certificates used to verify the regulatory database
> > > signature.
> > > 
> > > During boot, I consistently see the following messages:
> > >  cfg80211: Loading compiled-in X.509 certificates for regulatory database
> > >  Problem loading in-kernel X.509 certificate (-22)
> > >  Problem loading in-kernel X.509 certificate (-22)
> > >  cfg80211: loaded regulatory.db is malformed or signature is
> > > missing/invalid
> > > 
> > > As part of the debugging process, I removed the CAAM crypto drivers and
> > > manually
> > > reloaded cfg80211. In this configuration, the certificates load correctly
> > > and
> > > the regulatory database is validated with no errors.
> > > 
> > > With additional debugging enabled, I traced the failure to
> > > crypto_sig_verify(),
> > > which returns -22 (EINVAL).
> > > At this stage, I’m trying to determine whether:
> > >  - This is a known issue involving cfg80211 certificate validation when
> > > the
> > > CAAM
> > > hardware crypto engine is enabled on i.MX SoCs, or
> > >  - CAAM may be returning unexpected values to the X.509 verification
> > > logic.
> > > 
> > > If anyone has encountered similar behavior or can suggest areas to
> > > investigate—particularly around CAAM—I would greatly appreciate your
> > > guidance.
> > > 
> > > Thanks in advance for any insights,
> > > Vítor Soares
> > 
> > Following up with additional debugging findings.
> > 
> > I traced the -EINVAL to rsassa_pkcs1_verify() in the PKCS#1 v1.5
> > verification
> > path. The check that fails expects a leading 0x00 byte in the RSA output
> > buffer.
> > To investigate further, I poisoned the output buffer with 0xAA before the
> > RSA
> > operation. CAAM RSA operation returns success, but the output buffer is
> > never
> > written to.
> > 
> > During debugging, I loaded cfg80211 multiple times and observed that
> > sporadically one of the certificates gets verified correctly, but never
> > both.
> > 
> > I confirmed that other CAAM operations work correctly by testing hwrng via
> > /dev/hwrng, which produces valid random data.
> > 
> > Given that CAAM reports success but does not populate the RSA output buffer,
> > the
> > problem appears to be somewhere in the RSA execution flow (possibly in how
> > the
> > result buffer is handled or returned), but I don’t have enough insight into
> > CAAM's RSA implementation or firmware interaction to pinpoint the exact
> > cause.
> > 
> > As noted previously, blacklisting caam_pkc to force rsa-generic resolves the
> > issue.
> 
> Just a shot in the dark, because I have no experience with i.MX8 beyond
> i.MX8M:
> 
> Is the CAAM cache-coherent on your SoC? If so does the DT specify dma-coherent
> as it should? On i.MX8M, it's not cache-coherent, but on Layerscape it was and
> the mismatch with the DT leads to symptoms matching what you are observing.
> 

Thanks for the suggestion. I tested with dma-coherent added to the CAAM and job
ring nodes but the issue persists.
I traced through the DMA path in caampkc.c and confirmed:

- dma_map_sg() is called in rsa_edesc_alloc() with DMA_FROM_DEVICE
- dma_unmap_sg() is called in rsa_io_unmap() from rsa_pub_done() before
completion
- CAAM returns status err=0x00000000 (success)
- dst_nents=1 

Yet the output buffer remains untouched (still contains my 0xAA poison pattern).
The kernel DMA handling appears correct. CAAM accepts the job and reports
success, but never writes the RSA result. Given that CAAM reports success but
does not populate the RSA output buffer, the problem appears to be somewhere in
the RSA execution flow (possibly in how the result buffer is handled or
returned), but I don't have enough insight into CAAM's RSA implementation.

> Off-topic remark: If you have performance comparison between running with and
> without CAAM RSA acceleration, I'd be interested to hear about them.
> At least for the hashing algorithms, using the Cortex-A53 (+ CE) CPU was a lot
> faster than bothering with the CAAM "acceleration".
> 

I haven't done a kernel-level CAAM vs software RSA comparison, but OpenSSL with
ARM Crypto Extensions shows ~3100 verify ops/sec and ~80 sign ops/sec for RSA
2048 on the Cortex-A35.

Regards,
Vítor