mbedTLS equivalent for Java's "RSA/ECB/OAEPWithSHA-256AndMGF1Padding"

Mbed TLS Crypto and SSL questions
1

I have a server that sends down a key encrypted with “RSA/ECB/OAEPWithSHA-256AndMGF1Padding” (via Java).

I’m currently trying to convert my client application from Java to C++, but when decrypting the key mbedTLS returns back to me -0x4100 (MBEDTLS_ERR_RSA_INVALID_PADDING). I’ve tried using mbedtls_rsa_set_padding in conjunction with mbedtls_pk_decrypt, mbedtls_rsa_pkcs1_decrypt, and mbedtls_rsa_rsaes_oaep_decrypt all to no avail. Can someone help guide me in the right direction so I can successfully decrypt my key with mbedTLS?

2

Hi @matthu1
Thank you for your question!

According to this post, te MGF1 is instantiated with SHA1. and the SHA256 is only for the hashing of the label.

Assuming you have MBEDTLS_PKCS1_V21 defined, and set the padding to MBEDTLS_RSA_PKCS_V21, have you set the hash_id to MBEDTLS_MD_SHA1?

Regards,
Mbed TLS Team member
Ron

1 Like
3

Hi @roneld01 , I met similar problem with @matthu1 .

currently i want to encrypt data with mbedTLS rsa library and decrypt with “RSA/ECB/OAEPWithSHA-256AndMGF1Padding” (java).
In my code, mbedtls_rsa_context is initted as PKCS_V21, and hash id is MD_SHA1. I cannot decrypt data with java(RSA/ECB/OAEPWithSHA-256AndMGF1Padding), but can with java(RSA/ECB/OAEPWithSHA-1AndMGF1Padding). That is to say mbedTLS RSA seems only support RSA/ECB/OAEPWithSHA-1AndMGF1Padding"? Any settings or configuration to make mbedTLS can work as “RSA/ECB/OAEPWithSHA-256AndMGF1Padding”?
Thanks for any help.

5

From the code (/library/rsa.c, mbedtls_rsa_rsaes_oaep_encrypt), I understand that the same hash function is being used for the label hashing and MGF1 (From RFC8017, MGF1 is a particular MGF constructed with a hash algorithm, which can be anything secure, not necessarily the same as RSAES-OAEP).

If this is correct, can we document that mbedTLS does not support using different hashing algorithms for OAEP’s label hashing and MGF1?