Hi Everyone,

I’m trying to integrate the mbedTLS library into my existing project and facing some issues while trying to create the RSA keys.

Before I get into the issue, I would like to give some insight about the hardware and the software:

- Kinetis K24 ARM Cortex-M4
- Using MQXLite (RTOS)
- Compiling and running through Kinetis Design Studio (based on Eclipse)
- using the latest from the github branch of mbedtls

I initially tested the mbedTLS lib on Ubuntu and it worked perfectly. I even created my own custom example of rsa init, encrypt and decrypt based on the 3 rsa example files in `programs/pkey`

so I can easily integrate it in my exisiting firmware.

While trying to integrate the library in my firmware, I provided a custom random generator function as well by using the `mbedtls_hardware_poll()`

. Tested that out and works okay!

Now coming to the issue:

The `calloc()`

call inside the `mbedtls_rsa_gen_key()`

is always failing specifically from `line 559`

from `rsa.c`

which is `MBEDTLS_MPI_CHK( mbedtls_mpi_lset( &ctx->E, exponent ) );`

This eventually is leading to a hard fault as well (maybe not related to calloc() failing but I’m not sure)

**When I put a break point where it’s failing on the calloc(), I see it’s trying to grow one limb of size 4 bytes. I tried calling calloc() and free() in my init function and allocated 10 bytes of uint32_t and it worked fine.**

## This is my init code:

```
/**
* \brief This function initializes the RSA.
*
* \note The function creates all the necessary keys (public, private)
* and other equation values required for RSA encryption/decryption
*
* \param rsa The RSA context to initialize. This must not be \c NULL.
* \param entropy The entropy context to initialize. This must not be \c NULL.
* \param ctr_drbg The CTR_DRBG context to initialize. This also must not be \c NULL.
* \param pers Personalization data, that is device-specific identifiers. Can be NULL.
*
* \return \c 1 on success
* \return \c -1 on failure
*/
int rsa_init(mbedtls_rsa_context *rsa, mbedtls_entropy_context *entropy,
mbedtls_ctr_drbg_context *ctr_drbg, const char *pers) {
int ret = 1;
// init rnga module
init_RNGA_module();
// init rsa contexts
mbedtls_ctr_drbg_init(ctr_drbg);
mbedtls_rsa_init(rsa, MBEDTLS_RSA_PKCS_V15, 0);
mbedtls_entropy_init(entropy);
if( ( ret = mbedtls_ctr_drbg_seed(ctr_drbg, mbedtls_entropy_func, entropy,
(const unsigned char *) pers,
strlen( pers ) ) ) != 0 )
{
//print_nonblocking(TIME_STAMP, DEBUG, " failed\n ! mbedtls_ctr_drbg_seed returned %d\n", ret );
return -1;
}
if( ( ret = mbedtls_rsa_gen_key(rsa, mbedtls_ctr_drbg_random, ctr_drbg, KEY_SIZE,
EXPONENT ) ) != 0 )
{
//print_nonblocking(TIME_STAMP, DEBUG, " failed\n ! mbedtls_rsa_gen_key returned %d\n\n", ret );
return -1;
}
return 1;
}
```

## The parent function (defined in another file) that calls the rsa init:

```
static mbedtls_rsa_context rsa;
static mbedtls_entropy_context entropy;
static mbedtls_ctr_drbg_context ctr_drbg;
static bool is_key_generated = false;
/**
* \brief API to create the RSA keys
*
* \param none
*
* \return \c SUCCESS on success
* \return \c FAILURE on failure
*/
int generate_rsa_keys() {
// if key is already generated, we need to de-init the contexts first
if (is_key_generated) {
rsa_deinit(&rsa, &entropy, &ctr_drbg);
is_key_generated = false;
}
// init rsa module
int ret = rsa_init(&rsa, &entropy, &ctr_drbg, "rsa_keygen");
if (ret == ERR) {
print_nonblocking(TIME_STAMP, ERROR, "RSA INIT ERROR!");
return ERR;
}
// print the keys
print_nonblocking(TIME_STAMP, DEBUG, "\n Public key (N - %d): ", rsa.N.n);
for (uint16_t i=0; i<rsa.N.n; i++) {
print_nonblocking(NO_TIME_STAMP, CLI_RESPONSE, "%02lX ", rsa.N.p[i]);
}
print_nonblocking(TIME_STAMP, DEBUG, "\n Public exponent (E - %d): ", rsa.E.n);
for (uint16_t i=0; i<rsa.E.n; i++) {
print_nonblocking(NO_TIME_STAMP, CLI_RESPONSE, "%02lX", rsa.E.p[i]);
}
print_nonblocking(TIME_STAMP, DEBUG, "\n Private key (D - %d): ", rsa.D.n);
for (uint16_t i=0; i<rsa.D.n; i++) {
print_nonblocking(NO_TIME_STAMP, CLI_RESPONSE, "%02lX", rsa.D.p[i]);
}
print_nonblocking(NO_TIME_STAMP, CLI_RESPONSE, "\n");
return ret;
}
```

## Flags currently enabled (changed by me) in `config.h`

:

`MBEDTLS_ENTROPY_HARDWARE_ALT`

`MBEDTLS_AES_ROM_TABLES`

Any ideas or suggestion why this might be happening?

I would really appreciate any help!

Thank you!