Public key read failure due to possible endian-ness issue

Mbed TLS
1

Hi All:

I am using mbedtls 2.16.1. I am attempting to migrate from using openssl to using mbedtls.

I am using a test key which has a public exponent (??) of 0x10001. I am using the same pem data in 2 tests. Really it is the same test run on both x86 and ppc. The pem data is stored in memory before calling mbedtls.

I am compiling mbedtls in Linux (gcc) for both x86 and ppc architectures. I have multiple other executables compiled for both platforms. I know my flags are good as far as ppc are concerned (at least in realtion to everything pother than mbedtls). I am not certain about mbedtls flags. I used this same test with openssl way-back-when without issue: i.e. also same pem data.

  1. The x86 test works fine.

  2. The ppc test fails while attempting to validate the public key. Specifically, it fails while valdiating the public exponent. The failure is detected in mbedtls_rsa_check_pubkey.

The calling function “mbedtls_rsa_check_pubkey” is making this call (line 651 of rsa.c):

if( mbedtls_mpi_get_bit( &ctx->E, 0 ) == 0 ||

I did validate the other calls/values in same if statement: they looked good.

On x86 I am getting a public exponent of 0x10001
where the memory footprint looks like: 01000100
On x86, mbedtls_mpi_get_bit returns 1. This is good.

But on ppc I am getting a public exponent of 16777472 (01000100)
where the memory footprint looks like: 01000100
On ppc, mbedtls_mpi_get_bit returns 0: The least significant bit is 0, so no wonder.

This looks like an endianness issue to me, and I have no idea how to handle this.

Am I suppoosed to let mbedtls read in the data? Thoughts?

Cheers.

Solar.

2

Hi solar,

Thank you for your question.
I believe that you may be correct, and I created an issue in our Mbed crypto repository on your behalf.

Regards,
Mbed Support
Ron

3

Ron:

We just new that we could not be the first big endian mbedtls users, so we kept digging. After further head scratching, a peer pointed out that our outdated version of gcc may not directly support the appropriate flags required for proper endianness in mbedtls. After some digging we found this to be the case. Our version of $(CC) was setting

__BIG_ENDIAN__ 1

… where as mbedtls requires

__BYTE_ORDER__=__ORDER_BIG_ENDIAN__

… for ppc. This fixed the issue: no bug in mbedtls.

Cheers.

4

Hi Dave,

Thank you for your follow up and information.

Since our main boards are little endian architecture, it is always possible there are bugs in BE. It is good to know this isn’t an issue in Mbed TLS

Regards,

Ron