CAAM RSA breaks cfg80211 certificate verification on iMX8QXP

CAAM RSA breaks cfg80211 certificate verification on iMX8QXP

[Vitor Soares]

I’m currently investigating an issue on our Colibri iMX8QXP SoM running kernel
6.18-rc6 (also reproducible on v6.17), where cfg80211 fails to load the
compiled-in X.509 certificates used to verify the regulatory database signature.

During boot, I consistently see the following messages:
 cfg80211: Loading compiled-in X.509 certificates for regulatory database
 Problem loading in-kernel X.509 certificate (-22)
 Problem loading in-kernel X.509 certificate (-22)
 cfg80211: loaded regulatory.db is malformed or signature is missing/invalid

As part of the debugging process, I removed the CAAM crypto drivers and manually
reloaded cfg80211. In this configuration, the certificates load correctly and
the regulatory database is validated with no errors.

With additional debugging enabled, I traced the failure to crypto_sig_verify(),
which returns -22 (EINVAL).
At this stage, I’m trying to determine whether:
 - This is a known issue involving cfg80211 certificate validation when the CAAM
hardware crypto engine is enabled on i.MX SoCs, or
 - CAAM may be returning unexpected values to the X.509 verification logic.

If anyone has encountered similar behavior or can suggest areas to
investigate—particularly around CAAM—I would greatly appreciate your guidance.

Thanks in advance for any insights,
Vítor Soares

Re: CAAM RSA breaks cfg80211 certificate verification on iMX8QXP

[Vitor Soares]

++imx@lists.linux.dev

On Mon, 2025-11-24 at 19:03 +0000, Vitor Soares wrote:
> I’m currently investigating an issue on our Colibri iMX8QXP SoM running kernel
> 6.18-rc6 (also reproducible on v6.17), where cfg80211 fails to load the
> compiled-in X.509 certificates used to verify the regulatory database
> signature.
> 
> During boot, I consistently see the following messages:
>  cfg80211: Loading compiled-in X.509 certificates for regulatory database
>  Problem loading in-kernel X.509 certificate (-22)
>  Problem loading in-kernel X.509 certificate (-22)
>  cfg80211: loaded regulatory.db is malformed or signature is missing/invalid
> 
> As part of the debugging process, I removed the CAAM crypto drivers and
> manually
> reloaded cfg80211. In this configuration, the certificates load correctly and
> the regulatory database is validated with no errors.
> 
> With additional debugging enabled, I traced the failure to
> crypto_sig_verify(),
> which returns -22 (EINVAL).
> At this stage, I’m trying to determine whether:
>  - This is a known issue involving cfg80211 certificate validation when the
> CAAM
> hardware crypto engine is enabled on i.MX SoCs, or
>  - CAAM may be returning unexpected values to the X.509 verification logic.
> 
> If anyone has encountered similar behavior or can suggest areas to
> investigate—particularly around CAAM—I would greatly appreciate your guidance.
> 
> Thanks in advance for any insights,
> Vítor Soares

Following up with additional debugging findings.

I traced the -EINVAL to rsassa_pkcs1_verify() in the PKCS#1 v1.5 verification
path. The check that fails expects a leading 0x00 byte in the RSA output buffer.
To investigate further, I poisoned the output buffer with 0xAA before the RSA
operation. CAAM RSA operation returns success, but the output buffer is never
written to.

During debugging, I loaded cfg80211 multiple times and observed that
sporadically one of the certificates gets verified correctly, but never both.

I confirmed that other CAAM operations work correctly by testing hwrng via
/dev/hwrng, which produces valid random data.

Given that CAAM reports success but does not populate the RSA output buffer, the
problem appears to be somewhere in the RSA execution flow (possibly in how the
result buffer is handled or returned), but I don’t have enough insight into
CAAM's RSA implementation or firmware interaction to pinpoint the exact cause.

As noted previously, blacklisting caam_pkc to force rsa-generic resolves the
issue.

Regards,
Vítor

Re: CAAM RSA breaks cfg80211 certificate verification on iMX8QXP

[Ahmad Fatoum]

Hello Vitor,

On 26.11.25 11:55, Vitor Soares wrote:
> ++imx@lists.linux.dev
> 
> On Mon, 2025-11-24 at 19:03 +0000, Vitor Soares wrote:
>> I’m currently investigating an issue on our Colibri iMX8QXP SoM running kernel
>> 6.18-rc6 (also reproducible on v6.17), where cfg80211 fails to load the
>> compiled-in X.509 certificates used to verify the regulatory database
>> signature.
>>
>> During boot, I consistently see the following messages:
>>  cfg80211: Loading compiled-in X.509 certificates for regulatory database
>>  Problem loading in-kernel X.509 certificate (-22)
>>  Problem loading in-kernel X.509 certificate (-22)
>>  cfg80211: loaded regulatory.db is malformed or signature is missing/invalid
>>
>> As part of the debugging process, I removed the CAAM crypto drivers and
>> manually
>> reloaded cfg80211. In this configuration, the certificates load correctly and
>> the regulatory database is validated with no errors.
>>
>> With additional debugging enabled, I traced the failure to
>> crypto_sig_verify(),
>> which returns -22 (EINVAL).
>> At this stage, I’m trying to determine whether:
>>  - This is a known issue involving cfg80211 certificate validation when the
>> CAAM
>> hardware crypto engine is enabled on i.MX SoCs, or
>>  - CAAM may be returning unexpected values to the X.509 verification logic.
>>
>> If anyone has encountered similar behavior or can suggest areas to
>> investigate—particularly around CAAM—I would greatly appreciate your guidance.
>>
>> Thanks in advance for any insights,
>> Vítor Soares
> 
> Following up with additional debugging findings.
> 
> I traced the -EINVAL to rsassa_pkcs1_verify() in the PKCS#1 v1.5 verification
> path. The check that fails expects a leading 0x00 byte in the RSA output buffer.
> To investigate further, I poisoned the output buffer with 0xAA before the RSA
> operation. CAAM RSA operation returns success, but the output buffer is never
> written to.
> 
> During debugging, I loaded cfg80211 multiple times and observed that
> sporadically one of the certificates gets verified correctly, but never both.
> 
> I confirmed that other CAAM operations work correctly by testing hwrng via
> /dev/hwrng, which produces valid random data.
> 
> Given that CAAM reports success but does not populate the RSA output buffer, the
> problem appears to be somewhere in the RSA execution flow (possibly in how the
> result buffer is handled or returned), but I don’t have enough insight into
> CAAM's RSA implementation or firmware interaction to pinpoint the exact cause.
> 
> As noted previously, blacklisting caam_pkc to force rsa-generic resolves the
> issue.

Just a shot in the dark, because I have no experience with i.MX8 beyond i.MX8M:

Is the CAAM cache-coherent on your SoC? If so does the DT specify dma-coherent
as it should? On i.MX8M, it's not cache-coherent, but on Layerscape it was and
the mismatch with the DT leads to symptoms matching what you are observing.

Off-topic remark: If you have performance comparison between running with and
without CAAM RSA acceleration, I'd be interested to hear about them.
At least for the hashing algorithms, using the Cortex-A53 (+ CE) CPU was a lot
faster than bothering with the CAAM "acceleration".

Cheers,
Ahmad



> 
> Regards,
> Vítor
> 
> 

Re: CAAM RSA breaks cfg80211 certificate verification on iMX8QXP

[Vitor Soares]

On Wed, 2025-11-26 at 13:59 +0100, Ahmad Fatoum wrote:
> Hello Vitor,
> 
> On 26.11.25 11:55, Vitor Soares wrote:
> > ++imx@lists.linux.dev
> > 
> > On Mon, 2025-11-24 at 19:03 +0000, Vitor Soares wrote:
> > > I’m currently investigating an issue on our Colibri iMX8QXP SoM running
> > > kernel
> > > 6.18-rc6 (also reproducible on v6.17), where cfg80211 fails to load the
> > > compiled-in X.509 certificates used to verify the regulatory database
> > > signature.
> > > 
> > > During boot, I consistently see the following messages:
> > >  cfg80211: Loading compiled-in X.509 certificates for regulatory database
> > >  Problem loading in-kernel X.509 certificate (-22)
> > >  Problem loading in-kernel X.509 certificate (-22)
> > >  cfg80211: loaded regulatory.db is malformed or signature is
> > > missing/invalid
> > > 
> > > As part of the debugging process, I removed the CAAM crypto drivers and
> > > manually
> > > reloaded cfg80211. In this configuration, the certificates load correctly
> > > and
> > > the regulatory database is validated with no errors.
> > > 
> > > With additional debugging enabled, I traced the failure to
> > > crypto_sig_verify(),
> > > which returns -22 (EINVAL).
> > > At this stage, I’m trying to determine whether:
> > >  - This is a known issue involving cfg80211 certificate validation when
> > > the
> > > CAAM
> > > hardware crypto engine is enabled on i.MX SoCs, or
> > >  - CAAM may be returning unexpected values to the X.509 verification
> > > logic.
> > > 
> > > If anyone has encountered similar behavior or can suggest areas to
> > > investigate—particularly around CAAM—I would greatly appreciate your
> > > guidance.
> > > 
> > > Thanks in advance for any insights,
> > > Vítor Soares
> > 
> > Following up with additional debugging findings.
> > 
> > I traced the -EINVAL to rsassa_pkcs1_verify() in the PKCS#1 v1.5
> > verification
> > path. The check that fails expects a leading 0x00 byte in the RSA output
> > buffer.
> > To investigate further, I poisoned the output buffer with 0xAA before the
> > RSA
> > operation. CAAM RSA operation returns success, but the output buffer is
> > never
> > written to.
> > 
> > During debugging, I loaded cfg80211 multiple times and observed that
> > sporadically one of the certificates gets verified correctly, but never
> > both.
> > 
> > I confirmed that other CAAM operations work correctly by testing hwrng via
> > /dev/hwrng, which produces valid random data.
> > 
> > Given that CAAM reports success but does not populate the RSA output buffer,
> > the
> > problem appears to be somewhere in the RSA execution flow (possibly in how
> > the
> > result buffer is handled or returned), but I don’t have enough insight into
> > CAAM's RSA implementation or firmware interaction to pinpoint the exact
> > cause.
> > 
> > As noted previously, blacklisting caam_pkc to force rsa-generic resolves the
> > issue.
> 
> Just a shot in the dark, because I have no experience with i.MX8 beyond
> i.MX8M:
> 
> Is the CAAM cache-coherent on your SoC? If so does the DT specify dma-coherent
> as it should? On i.MX8M, it's not cache-coherent, but on Layerscape it was and
> the mismatch with the DT leads to symptoms matching what you are observing.
> 

Thanks for the suggestion. I tested with dma-coherent added to the CAAM and job
ring nodes but the issue persists.
I traced through the DMA path in caampkc.c and confirmed:

- dma_map_sg() is called in rsa_edesc_alloc() with DMA_FROM_DEVICE
- dma_unmap_sg() is called in rsa_io_unmap() from rsa_pub_done() before
completion
- CAAM returns status err=0x00000000 (success)
- dst_nents=1 

Yet the output buffer remains untouched (still contains my 0xAA poison pattern).
The kernel DMA handling appears correct. CAAM accepts the job and reports
success, but never writes the RSA result. Given that CAAM reports success but
does not populate the RSA output buffer, the problem appears to be somewhere in
the RSA execution flow (possibly in how the result buffer is handled or
returned), but I don't have enough insight into CAAM's RSA implementation.

> Off-topic remark: If you have performance comparison between running with and
> without CAAM RSA acceleration, I'd be interested to hear about them.
> At least for the hashing algorithms, using the Cortex-A53 (+ CE) CPU was a lot
> faster than bothering with the CAAM "acceleration".
> 

I haven't done a kernel-level CAAM vs software RSA comparison, but OpenSSL with
ARM Crypto Extensions shows ~3100 verify ops/sec and ~80 sign ops/sec for RSA
2048 on the Cortex-A35.

Regards,
Vítor

Re: CAAM RSA breaks cfg80211 certificate verification on iMX8QXP

[Ahmad Fatoum]

Hi Vitor,

On 11/26/25 7:35 PM, Vitor Soares wrote:
> On Wed, 2025-11-26 at 13:59 +0100, Ahmad Fatoum wrote:
>> Is the CAAM cache-coherent on your SoC? If so does the DT specify dma-coherent
>> as it should? On i.MX8M, it's not cache-coherent, but on Layerscape it was and
>> the mismatch with the DT leads to symptoms matching what you are observing.
>>
> 
> Thanks for the suggestion. I tested with dma-coherent added to the CAAM and job
> ring nodes but the issue persists.
> I traced through the DMA path in caampkc.c and confirmed:
> 
> - dma_map_sg() is called in rsa_edesc_alloc() with DMA_FROM_DEVICE
> - dma_unmap_sg() is called in rsa_io_unmap() from rsa_pub_done() before
> completion
> - CAAM returns status err=0x00000000 (success)
> - dst_nents=1 
> 
> Yet the output buffer remains untouched (still contains my 0xAA poison pattern).
> The kernel DMA handling appears correct. CAAM accepts the job and reports
> success, but never writes the RSA result. Given that CAAM reports success but
> does not populate the RSA output buffer, the problem appears to be somewhere in
> the RSA execution flow (possibly in how the result buffer is handled or
> returned), but I don't have enough insight into CAAM's RSA implementation.

Ok.. That was the only thing off the top of my head right now.

>> Off-topic remark: If you have performance comparison between running with and
>> without CAAM RSA acceleration, I'd be interested to hear about them.
>> At least for the hashing algorithms, using the Cortex-A53 (+ CE) CPU was a lot
>> faster than bothering with the CAAM "acceleration".
>>
> 
> I haven't done a kernel-level CAAM vs software RSA comparison, but OpenSSL with
> ARM Crypto Extensions shows ~3100 verify ops/sec and ~80 sign ops/sec for RSA
> 2048 on the Cortex-A35.

I see, thanks.

Cheers,
Ahmad

> 
> Regards,
> Vítor
> 
> 
> 
> 

-- 
Pengutronix e.K.                  |                             |
Steuerwalder Str. 21              | http://www.pengutronix.de/  |
31137 Hildesheim, Germany         | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686  | Fax:   +49-5121-206917-5555 |