Bignum.c in mbedTLS 2.16.1 and later not working

We had mbedTLS 2.5.2 working well on our embedded hardware which utilizes the Analog Devices Blackfin 51x family of CPUs, which operates little-endan. When we upgraded to 2.16.3 our embedded HTTPS server did not work anymore. It just reset the connection not too far into the TLS negotiations with browsers.

Upon investigating I found that bignum.c is to blame. The version of bignum.c in mbedTLS 2.16.0 works and I can use it in 2.16.2 or 2.16.3 with good results. But the versions of bignum.c in any of mbedTLS 2.16.1, 2, or 3 do not work well.

I think the root problem is likely related to the endian conversions in the later versions but it isn’t quite clear. I know that for many years our software uses a macro for the endian conversion and on a little-endian system it is

#define Ntohl(val)
( (((u32)(val) >> 24) & 0x000000FF) |
(((u32)(val) >> 8) & 0x0000FF00) |
(((u32)(val) << 8) & 0x00FF0000) |
(((u32)(val) << 24) & 0xFF000000) )

This appears to be quite different from the mpi_uint_bigendian_to_host_c() function in 2.16.3 but
I can’t spend more time on it. I’ve already spent quite a lot tracking down the problem.

We build the code with Visual DSP 5.1.2 with build date May 30 2013. It is worth noting that it
does NOT define BYTE_ORDER so mpi_uint_bigendian_to_host_c() is used.

Hi Mike,
Thank you for reporting this issue!

I have created a ticket in our repository to track this.
Mbed TLS Support