MbedTLS issue when using with with HTTPs server (RSA certificates, buffer overflow, MAGIC1 corruption)

Hi everyone,

I recently switch to version 3.0.0 and now I got a problem when I try to access to my HTTP server.

The problem appears with RSA certificates (2048 or 4096). In that case, the firmware fails into mbedtls_exit after verifying the header of a buffer during buffer_alloc_free.

After verification it seems that something creates an overflow of one byte in the buffer and override the first byte of MAGIC1 value in the buffer’s header.

I found a fix by modifying the rsa_prepare_blinding function, but I am not very confident in my correction. I simply remove some -1 operations and in that case the buffer is correctly filled by mbedtls_mpi_fill_random without overflow.

Here the code:

/*
 * Generate or update blinding values, see section 10 of:
 *  KOCHER, Paul C. Timing attacks on implementations of Diffie-Hellman, RSA,
 *  DSS, and other systems. In : Advances in Cryptology-CRYPTO'96. Springer
 *  Berlin Heidelberg, 1996. p. 104-113.
 */
static int rsa_prepare_blinding( mbedtls_rsa_context *ctx,
                 int (*f_rng)(void *, unsigned char *, size_t), void *p_rng )
{
    int ret, count = 0;
    mbedtls_mpi R;

    mbedtls_mpi_init( &R );

    if( ctx->Vf.p != NULL )
    {
        /* We already have blinding values, just update them by squaring */
        MBEDTLS_MPI_CHK( mbedtls_mpi_mul_mpi( &ctx->Vi, &ctx->Vi, &ctx->Vi ) );
        MBEDTLS_MPI_CHK( mbedtls_mpi_mod_mpi( &ctx->Vi, &ctx->Vi, &ctx->N ) );
        MBEDTLS_MPI_CHK( mbedtls_mpi_mul_mpi( &ctx->Vf, &ctx->Vf, &ctx->Vf ) );
        MBEDTLS_MPI_CHK( mbedtls_mpi_mod_mpi( &ctx->Vf, &ctx->Vf, &ctx->N ) );

        goto cleanup;
    }

    /* Unblinding value: Vf = random number, invertible mod N */
    do {
        if( count++ > 10 )
        {
            ret = MBEDTLS_ERR_RSA_RNG_FAILED;
            goto cleanup;
        }

        MBEDTLS_MPI_CHK( mbedtls_mpi_fill_random( &ctx->Vf, ctx->len, f_rng, p_rng ) );

        /* Compute Vf^-1 as R * (R Vf)^-1 to avoid leaks from inv_mod. */
        MBEDTLS_MPI_CHK( mbedtls_mpi_fill_random( &R, ctx->len, f_rng, p_rng ) );
        MBEDTLS_MPI_CHK( mbedtls_mpi_mul_mpi( &ctx->Vi, &ctx->Vf, &R ) );
        MBEDTLS_MPI_CHK( mbedtls_mpi_mod_mpi( &ctx->Vi, &ctx->Vi, &ctx->N ) );

        /* At this point, Vi is invertible mod N if and only if both Vf and R
         * are invertible mod N. If one of them isn't, we don't need to know
         * which one, we just loop and choose new values for both of them.
         * (Each iteration succeeds with overwhelming probability.) */
        ret = mbedtls_mpi_inv_mod( &ctx->Vi, &ctx->Vi, &ctx->N );
        if( ret != 0 && ret != MBEDTLS_ERR_MPI_NOT_ACCEPTABLE )
            goto cleanup;

    } while( ret == MBEDTLS_ERR_MPI_NOT_ACCEPTABLE );

    /* Finish the computation of Vf^-1 = R * (R Vf)^-1 */
    MBEDTLS_MPI_CHK( mbedtls_mpi_mul_mpi( &ctx->Vi, &ctx->Vi, &R ) );
    MBEDTLS_MPI_CHK( mbedtls_mpi_mod_mpi( &ctx->Vi, &ctx->Vi, &ctx->N ) );

    /* Blinding value: Vi = Vf^(-e) mod N
     * (Vi already contains Vf^-1 at this point) */
    MBEDTLS_MPI_CHK( mbedtls_mpi_exp_mod( &ctx->Vi, &ctx->Vi, &ctx->E, &ctx->N, &ctx->RN ) );


cleanup:
    mbedtls_mpi_free( &R );

    return( ret );
}

Could you please tell my if I made any mistake ?

Thanks in advance !

Regards,
Thomas

Below more information with the buffer data and the callstack status: