I have been having a problems with https web server running mbedTLS. I am getting a lot of Certificate Unknown failures. Then at some point things start working…
This is a self signed certificate, and does not change between when it works and does not. I have registered certificate with client as well.
Does anyone have a suggestion as to why this is happening?
Hi @trampas
As I answered you in your other post, the client cannot handle the certificate your server is sending.
There could be many reasons. However, if sometimes the certificate verification succeeds and sometimes it doesn’t, the usual suspect is memory leak.
The question is whether it is a memory issue on the server side or on the client side.
I would suggest you compare the certificate message between success attempts and failed attempts, to see if there is difference. If there was difference, so there is probably a memory issue on the server side. If the certificate that is sent wasn’t tampered, then it might be a memory issue on the client side.
Regards,
Mbed TLS Support
Ron
Ron, Thanks for the reply.
The only library that is doing any memory allocation is mbedtls, everything else is statically allocated memory. I have also added additional checks on the calloc for mbedtls, and verified that I am not having heap run into stack.
The interesting thing is that when the program first starts I get the client hello from Chrome and my server sends the server hello and then chrome sends the Alert 46 (Certificate Unknown). I get several of these before chrome accepts the certificate.
I assume that chrome is sending a private key, and then mebtls is using the entropy and private key to encrypt the certificate key exchange in the server hello, as that the server hello messages have different data in each message, and different number of bytes in payload.
Is it possible to encrypt with a known random number and see if the server hello data is correct? That is I assume the problem is that either the encryption is wrong, or data is being transmitted incorrectly.
I also thought of third case where chrome is intentionally doing this to see if the server uses random number entropy. That is it does the key exchange several times to make sure the keys are changing.
Under IE and Edge I get different results
The main problem I am having is that sometimes the browsers will get all the page’s data, and sometimes not. That is it might get the HTML but not the large CSS package. Sometimes it gets both. When not using https the server works perfect. Hence I assume it is a problem with the https webserver or the mbedtls.
This behavior is different on reboots, that is after reboot sometimes it will work and sometimes it will not. Which leads me to believe that it is something that is not static, like the entropy key changing timing or even timing on the browser side. Note that the different results happen with IE,Edge, and Chrome so I assume it is not browser but server issues.
Thanks
Trampas
Hi trampas,
The key exchange message is sent after the certificate message.
The image you are showing is 5 messages sent together from the server.
The server hello message contain random data, which differs between every new handshake.
The relevant message is the certificate message that is being sent by the server, and this message should not be different between runs. Have you compared these messages?
Regards
The left one shows a server key as 181 the right is 180 byte, the left failed the right passed.
I had assume the server key was randomly generated and thus could be different is this not the case?
HI @trampas
This difference may cause problem, however we need to dig further on reason for difference in the length. The length should remain the same, as the key exchange have same size parameters.
However, the alert you are receiving is about certificate unknown.
Have you compared the previous TLS record(Certificate message)? Are there any differences?
Regards
So I did some checking and difference in key sizes are a result of the mbedtls_pk_sign() function call from ssl_write_server_key_exchange() in ssl_srv.c.
I also went through the certificate and verified between passing and failing packets the certificates are the same.
Maybe I have the wrong options or keys?
7987: altcp_tcp.c 110: RX 517 bytes, port 56759
17997: ssl_srv.c 735: candidate certificate chain, certificate
18004: ssl_srv.c 797: selected certificate chain, certificate
18013: ssl_srv.c 2547: server hello, session id len.: 32
18020: ssl_srv.c 2636: write server hello 87 bytes
18027: altcp_tcp.c 366: write 92
18032: ssl_tls.c 4457: own certificate
18039: altcp_tcp.c 366: write 311
18689: ssl_srv.c 3067: N is 69 bytes
18695: ssl_srv.c 3205: N is 71 bytes
19408: ssl_srv.c 3216: signature_len is 72 bytes
19415: ssl_srv.c 3217: hashlen is 0 bytes
19420: ssl_srv.c 3225: N is 145 bytes
19427: altcp_tcp.c 366: write 154
19432: ssl_srv.c 3246: write server key exchange, 149 bytes
19439: altcp_tcp.c 366: write 19
19444: altcp_tcp.c 366: write 9
19449: WARNING: altcp_tcp.c 257: REceived 517,56759
19458: altcp_tcp.c 110: RX 517 bytes, port 56756
19467: ssl_srv.c 735: candidate certificate chain, certificate
19475: ssl_srv.c 797: selected certificate chain, certificate
19484: ssl_srv.c 2547: server hello, session id len.: 32
19491: ssl_srv.c 2636: write server hello 87 bytes
19498: altcp_tcp.c 366: write 92
19503: ssl_tls.c 4457: own certificate
19510: altcp_tcp.c 366: write 311
20160: ssl_srv.c 3067: N is 69 bytes
20165: ssl_srv.c 3205: N is 71 bytes
20880: ssl_srv.c 3216: signature_len is 70 bytes
20886: ssl_srv.c 3217: hashlen is 0 bytes
20892: ssl_srv.c 3225: N is 143 bytes
20898: altcp_tcp.c 366: write 152
20903: ssl_srv.c 3246: write server key exchange, 147 bytes
20911: altcp_tcp.c 366: write 19
20916: altcp_tcp.c 366: write 9
HI @Trrampas,
The differnce in lengths could also be because of the encoding of the signature.
Quoting this answer:
Since rr and ss are positive values, this means that the top bit of the first byte of each encoding must be zero. In your example, the top byte of rr is 0xB2, whose top bit is a 1, so an extra 0x00 byte is added, while the top byte of ss is 0x22, whose top bit is a 0, so no extra 0x00 byte here.
So, the additional byte could be because of an extra 0x00 byte, which is ok.
What is the Mbed TLS version you are using?
I also went through the certificate and verified between passing and failing packets the certificates are the same.
Just to confirm, have you compared the certificate message and verified the messages are the same, or the actual certificate that you load and think you send?
Regards
Just to confirm, have you compared the certificate message and verified the messages are the same, or the actual certificate that you load and think you send?
I verified the bytes that were in the failing certificate and the passing certificate were the same certificate bytes.
I am currently using mbedtls 2.7.13, I also tried 2.16.4 with the same results.
Again the root problem is that I think these certificates failures are preventing file transfers.
Here is a link to the Wireshark capture file:
https://drive.google.com/open?id=1y5d6-DOa2hQaXlfBiIGHTho_iK2Rz6Bu
Here is some debug log from my server, if it helps.
79361: altcp_tcp.c 110: RX 517 bytes, port 63162
679371: ssl_srv.c 735: candidate certificate chain, certificate
679379: ssl_srv.c 797: selected certificate chain, certificate
679387: ssl_srv.c 2547: server hello, session id len.: 32
679395: ssl_srv.c 2636: write server hello 87 bytes
679402: altcp_tcp.c 367: write 92, port 63162
679408: ssl_tls.c 4457: own certificate
679416: altcp_tcp.c 367: write 311, port 63162
680064: ssl_srv.c 3067: N is 69 bytes
680070: ssl_srv.c 3205: N is 71 bytes
680778: ssl_srv.c 3216: signature_len is 70 bytes
680784: ssl_srv.c 3217: hashlen is 0 bytes
680790: ssl_srv.c 3225: N is 143 bytes
680796: altcp_tcp.c 367: write 152, port 63162
680802: ssl_srv.c 3246: write server key exchange, 147 bytes
680810: altcp_tcp.c 367: write 19, port 63162
680816: altcp_tcp.c 367: write 9, port 63162
680822: WARNING: altcp_tcp.c 257: REceived 517,63162
680831: altcp_tcp.c 110: RX 517 bytes, port 63163
680841: ssl_srv.c 735: candidate certificate chain, certificate
680849: ssl_srv.c 797: selected certificate chain, certificate
680857: ssl_srv.c 2547: server hello, session id len.: 32
680864: ssl_srv.c 2636: write server hello 87 bytes
680872: altcp_tcp.c 367: write 92, port 63163
680878: ssl_tls.c 4457: own certificate
680885: altcp_tcp.c 367: write 311, port 63163
681533: ssl_srv.c 3067: N is 69 bytes
681539: ssl_srv.c 3205: N is 71 bytes
682250: ssl_srv.c 3216: signature_len is 72 bytes
682256: ssl_srv.c 3217: hashlen is 0 bytes
682262: ssl_srv.c 3225: N is 145 bytes
682268: altcp_tcp.c 367: write 154, port 63163
682274: ssl_srv.c 3246: write server key exchange, 149 bytes
682282: altcp_tcp.c 367: write 19, port 63163
682288: altcp_tcp.c 367: write 9, port 63163
682294: WARNING: altcp_tcp.c 257: REceived 517,63163
682361: altcp_tcp.c 128: TX 583 bytes, port 63162
682367: altcp_tcp.c 110: RX 7 bytes, port 63162
682374: WARNING: ssl_tls.c 4253: got an alert message, type: [2:46]
682382: ERROR: ssl_tls.c 4263: is a fatal alert message (msg 46)
682390: WARNING: altcp_tcp.c 257: REceived 7,63162
682397: ERROR: altcp_tls_mbedtls.c 287: mbedtls_ssl_handshake failed: -0x7780, port 63162
682606: altcp_tcp.c 128: TX 585 bytes, port 63163
682612: altcp_tcp.c 110: RX 7 bytes, port 63163
682619: WARNING: ssl_tls.c 4253: got an alert message, type: [2:46]
682627: ERROR: ssl_tls.c 4263: is a fatal alert message (msg 46)
682635: WARNING: altcp_tcp.c 257: REceived 7,63163
682642: ERROR: altcp_tls_mbedtls.c 287: mbedtls_ssl_handshake failed: -0x7780, port 63163
682883: altcp_tcp.c 110: RX 517 bytes, port 63168
682892: ssl_srv.c 735: candidate certificate chain, certificate
682900: ssl_srv.c 797: selected certificate chain, certificate
682909: ssl_srv.c 2547: server hello, session id len.: 32
682916: ssl_srv.c 2636: write server hello 87 bytes
682923: altcp_tcp.c 367: write 92, port 63168
682929: ssl_tls.c 4457: own certificate
682937: altcp_tcp.c 367: write 311, port 63168
683588: ssl_srv.c 3067: N is 69 bytes
683593: ssl_srv.c 3205: N is 71 bytes
684305: ssl_srv.c 3216: signature_len is 72 bytes
684311: ssl_srv.c 3217: hashlen is 0 bytes
684317: ssl_srv.c 3225: N is 145 bytes
684323: altcp_tcp.c 367: write 154, port 63168
684329: ssl_srv.c 3246: write server key exchange, 149 bytes
684337: altcp_tcp.c 367: write 19, port 63168
684343: altcp_tcp.c 367: write 9, port 63168
684349: WARNING: altcp_tcp.c 257: REceived 517,63168
684667: altcp_tcp.c 128: TX 585 bytes, port 63168
684674: altcp_tcp.c 110: RX 138 bytes, port 63168
684680: ERROR: ssl_tls.c 4605: TLSv1 client has no certificate
685358: altcp_tcp.c 367: write 6, port 63168
685367: altcp_tcp.c 367: write 45, port 63168
685373: WARNING: altcp_tcp.c 257: REceived 138,63168
685382: altcp_tcp.c 110: RX 517 bytes, port 63167
685391: ssl_srv.c 735: candidate certificate chain, certificate
685399: ssl_srv.c 797: selected certificate chain, certificate
685408: ssl_srv.c 2547: server hello, session id len.: 32
685415: ssl_srv.c 2636: write server hello 87 bytes
685422: altcp_tcp.c 367: write 92, port 63167
685428: ssl_tls.c 4457: own certificate
685436: altcp_tcp.c 367: write 311, port 63167
686086: ssl_srv.c 3067: N is 69 bytes
686092: ssl_srv.c 3205: N is 71 bytes
686804: ssl_srv.c 3216: signature_len is 71 bytes
686810: ssl_srv.c 3217: hashlen is 0 bytes
686816: ssl_srv.c 3225: N is 144 bytes
686822: altcp_tcp.c 367: write 153, port 63167
686829: ssl_srv.c 3246: write server key exchange, 148 bytes
686836: altcp_tcp.c 367: write 19, port 63167
686843: altcp_tcp.c 367: write 9, port 63167
686849: WARNING: altcp_tcp.c 257: REceived 517,63167
687254: altcp_tcp.c 128: TX 51 bytes, port 63168
687260: altcp_tcp.c 110: RX 570 bytes, port 63168
687269: WARNING: altcp_tcp.c 257: REceived 29,63168
687276: altcp_tls_mbedtls.c 1054: Received 541, port 63168
687283: WARNING: altcp_tcp.c 257: REceived 541,63168
687291: httpd.c 2434: Opening /dashboard.html
687299: fs.cpp 92: reading SD /dashboard.html: 2920, 18732
687324: altcp_tcp.c 367: write 2949, port 63168
687465: altcp_tcp.c 128: TX 2920 bytes, port 63168
687472: fs.cpp 92: reading SD /dashboard.html: 2920, 15812
687498: altcp_tcp.c 367: write 2949, port 63168
687519: altcp_tcp.c 128: TX 2920 bytes, port 63168
687525: fs.cpp 92: reading SD /dashboard.html: 2920, 12892
687552: altcp_tcp.c 367: write 2949, port 63168
687698: altcp_tcp.c 128: TX 2920 bytes, port 63168
687704: fs.cpp 92: reading SD /dashboard.html: 2920, 9972
687730: altcp_tcp.c 367: write 2949, port 63168
687738: altcp_tcp.c 128: TX 2920 bytes, port 63168
687744: fs.cpp 92: reading SD /dashboard.html: 2920, 7052
687771: altcp_tcp.c 367: write 2949, port 63168
687866: altcp_tcp.c 128: TX 2920 bytes, port 63168
687873: fs.cpp 92: reading SD /dashboard.html: 2920, 4132
687899: altcp_tcp.c 367: write 2949, port 63168
687910: altcp_tcp.c 128: TX 2920 bytes, port 63168
687916: fs.cpp 92: reading SD /dashboard.html: 1212, 1212
687933: altcp_tcp.c 367: write 1241, port 63168
687940: httpd.c 1713: end of file /dashboard.html
687946: WARNING: httpd.c 644: Closing connection, 63168
687959: altcp_tcp.c 110: RX 517 bytes, port 63173
687969: ssl_srv.c 735: candidate certificate chain, certificate
687977: ssl_srv.c 797: selected certificate chain, certificate
687985: ssl_srv.c 2547: server hello, session id len.: 32
687992: ssl_srv.c 2636: write server hello 87 bytes
687999: altcp_tcp.c 367: write 92, port 63173
688005: ssl_tls.c 4457: own certificate
688013: altcp_tcp.c 367: write 311, port 63173
688661: ssl_srv.c 3067: N is 69 bytes
688667: ssl_srv.c 3205: N is 71 bytes
689380: ssl_srv.c 3216: signature_len is 72 bytes
689386: ssl_srv.c 3217: hashlen is 0 bytes
689392: ssl_srv.c 3225: N is 145 bytes
689398: altcp_tcp.c 367: write 154, port 63173
689405: ssl_srv.c 3246: write server key exchange, 149 bytes
689412: altcp_tcp.c 367: write 19, port 63173
689418: altcp_tcp.c 367: write 9, port 63173
689424: WARNING: altcp_tcp.c 257: REceived 517,63173
689732: altcp_tcp.c 128: TX 585 bytes, port 63173
689739: altcp_tcp.c 110: RX 7 bytes, port 63173
689745: WARNING: ssl_tls.c 4253: got an alert message, type: [2:46]
689753: ERROR: ssl_tls.c 4263: is a fatal alert message (msg 46)
689761: WARNING: altcp_tcp.c 257: REceived 7,63173
689768: ERROR: altcp_tls_mbedtls.c 287: mbedtls_ssl_handshake failed: -0x7780, port 63173
689972: altcp_tcp.c 110: RX 517 bytes, port 63179
689981: ssl_srv.c 735: candidate certificate chain, certificate
689989: ssl_srv.c 797: selected certificate chain, certificate
689998: ssl_srv.c 2547: server hello, session id len.: 32
690005: ssl_srv.c 2636: write server hello 87 bytes
690012: altcp_tcp.c 367: write 92, port 63179
690018: ssl_tls.c 4457: own certificate
690026: altcp_tcp.c 367: write 311, port 63179
690676: ssl_srv.c 3067: N is 69 bytes
690682: ssl_srv.c 3205: N is 71 bytes
691400: ssl_srv.c 3216: signature_len is 72 bytes
691406: ssl_srv.c 3217: hashlen is 0 bytes
691412: ssl_srv.c 3225: N is 145 bytes
691418: altcp_tcp.c 367: write 154, port 63179
691424: ssl_srv.c 3246: write server key exchange, 149 bytes
691432: altcp_tcp.c 367: write 19, port 63179
691438: altcp_tcp.c 367: write 9, port 63179
691444: WARNING: altcp_tcp.c 257: REceived 517,63179
691952: altcp_tcp.c 128: TX 584 bytes, port 63167
691959: altcp_tcp.c 128: TX 585 bytes, port 63179
691965: altcp_tcp.c 110: RX 7 bytes, port 63179
691972: WARNING: ssl_tls.c 4253: got an alert message, type: [2:46]
691980: ERROR: ssl_tls.c 4263: is a fatal alert message (msg 46)
691988: WARNING: altcp_tcp.c 257: REceived 7,63179
691995: ERROR: altcp_tls_mbedtls.c 287: mbedtls_ssl_handshake failed: -0x7780, port 63179
692455: altcp_tcp.c 110: RX 517 bytes, port 63182
692465: ssl_srv.c 735: candidate certificate chain, certificate
692473: ssl_srv.c 797: selected certificate chain, certificate
692481: ssl_srv.c 2547: server hello, session id len.: 32
692488: ssl_srv.c 2636: write server hello 87 bytes
692495: altcp_tcp.c 367: write 92, port 63182
692501: ssl_tls.c 4457: own certificate
692509: altcp_tcp.c 367: write 311, port 63182
693158: ssl_srv.c 3067: N is 69 bytes
693164: ssl_srv.c 3205: N is 71 bytes
693879: ssl_srv.c 3216: signature_len is 71 bytes
693885: ssl_srv.c 3217: hashlen is 0 bytes
693891: ssl_srv.c 3225: N is 144 bytes
693897: altcp_tcp.c 367
Thanks
Trampas
Hi,
Thanks for the info
Again the root problem is that I think these certificates failures are preventing file transfers.
You are correct that the client failing to verify certificate is the problem. However, the server key exchange is not related.
The logs do not tell much, as they are out of context. Please show Mbed TLS logs with debug_level=4.
The wireshark capture file does not extract. I tried gunzip and tar. Please check
I verified the bytes that were in the failing certificate and the passing certificate were the same certificate bytes.
I understand, but where did you look at the bytes? Was it in wireshark or your own logs?
Regards
I checked in Wireshark that they matched.
Try these links for wireshark capture and detail debug log.
https://drive.google.com/open?id=1i1bA0N-eoPHcnx2Jn542qnUlqcvZib88
https://drive.google.com/open?id=1c-EiehHZCh5u2P8CyoYBXymG0_CuYvyZ
I have been adding in other logging, so the line numbers might slightly different but should be close.
Thanks
Trampas
Hi @trampas
From the wireshark, I see that you are sending your certificate ( Is there a reason why the subject is outlook.office.com? Are you using your device certificate?) and its parent intermediate certificate.
The intermediate certificate has a baisc constraint: “PathlenConstraint: 0”
As you can see from RFC 5280 section 4.2.1.9:
The pathLenConstraint field is meaningful only if the cA boolean is asserted and the key usage extension, if present, asserts the keyCertSign bit (Section 4.2.1.3). In this case, it gives the maximum number of non-self-issued intermediate certificates that may follow this certificate in a valid certification path.
This means, that since it’s 0 in your case, it is invalid to have another certificate follow it.
How did you generate your certificate? How did you set your certificate chain?
Regards
I was following guide here for key generation
openssl ecparam -genkey -name prime256v1 -out ca.key
openssl req -x509 -new -SHA256 -nodes -key ca.key -days 3650 -out ca.crt
I had also tried to use SHA384 as well, not sure which key I have at the moment (256 or 384).
To set the key I am using the example from LWIP example which calls:
#define LWIP_HTTPD_EXAMPLE_HTTPS_KEY_FILE_PASS NULL
#define LWIP_HTTPD_EXAMPLE_HTTPS_KEY_FILE_PASS_LEN 0
_conf = altcp_tls_create_config_server_privkey_cert(_privkey, privkey_size,LWIP_HTTPD_EXAMPLE_HTTPS_KEY_FILE_PASS,
LWIP_HTTPD_EXAMPLE_HTTPS_KEY_FILE_PASS_LEN, _cert, cert_size);
Following the multiple layers for code:
/** Create new TLS configuration
if (altcp_tls_config_server_add_privkey_cert(conf, privkey, privkey_len,
privkey_pass, privkey_pass_len, cert, cert_len) != ERR_OK) {
altcp_mbedtls_free_config(conf);
return NULL;
}
return conf;
}
err_t altcp_tls_config_server_add_privkey_cert(struct altcp_tls_config *config,
const u8_t *privkey, size_t privkey_len,
const u8_t *privkey_pass, size_t privkey_pass_len,
const u8_t *cert, size_t cert_len)
{
int ret;
mbedtls_x509_crt *srvcert;
mbedtls_pk_context *pkey;
if (config->cert_count >= config->cert_max) {
return ERR_MEM;
}
if (config->pkey_count >= config->pkey_max) {
return ERR_MEM;
}
srvcert = config->cert + config->cert_count;
mbedtls_x509_crt_init(srvcert);
pkey = config->pkey + config->pkey_count;
mbedtls_pk_init(pkey);
/* Load the certificates and private key */
ret = mbedtls_x509_crt_parse(srvcert, cert, cert_len);
if (ret != 0) {
LWIP_DEBUGF(ALTCP_MBEDTLS_DEBUG, (“mbedtls_x509_crt_parse failed: %d\n”, ret));
return ERR_VAL;
}
ret = mbedtls_pk_parse_key(pkey, (const unsigned char *) privkey, privkey_len, privkey_pass, privkey_pass_len);
if (ret != 0) {
LWIP_DEBUGF(ALTCP_MBEDTLS_DEBUG, (“mbedtls_pk_parse_public_key failed: %d\n”, ret));
mbedtls_x509_crt_free(srvcert);
return ERR_VAL;
}
ret = mbedtls_ssl_conf_own_cert(&config->conf, srvcert, pkey);
if (ret != 0) {
LWIP_DEBUGF(ALTCP_MBEDTLS_DEBUG, (“mbedtls_ssl_conf_own_cert failed: %d\n”, ret));
mbedtls_x509_crt_free(srvcert);
mbedtls_pk_free(pkey);
return ERR_VAL;
}
config->cert_count++;
config->pkey_count++;
return ERR_OK;
}
Thanks
Trampas
Hi Trampas,
The instruction you referenced were on how to create a self signed CA certificate.
How did you generate your device certificate?
I think I tried to set the IP address for the certificate to help with this problem. Here is a link to my certificate and key files:
https://drive.google.com/open?id=1el8FuwHeKZR1E33os2VPHYLv64Stzchx
https://drive.google.com/open?id=1z4pgOseulR0nJO5vELOQOPDfh8AZaAOi
I did not see the outlook link anywhere, so I am not sure where that came from?
I was looking at the certificate chain and their is a only one entry and then the pnext is null.
Trampas
Hi Trampas,
Please do not share private keys, as these are secrets
Please replace them once you finish your development, and use them for test only
Hi Trampas,
This certificate does not make sense.
It is a self signed certificate, and no basic constraints, so it’s not even a CA certtificate.
This certificate will not work on your TLS handshake
As for the outlook.office.com certificate, from the wireshark you sent:
If the certificate your server is sending to the clioent, then it is reasonable the client will not accept it, as it is not issued by any trusted CA
The keys and cert are just for development and not used in production…
Trampas
Yes, I later followed the IP of the sender, and I got the same certificate message.
However, my previous answer that you cannot use this certificate still holds. You cannot use a self signed certificate as a server certificate, as it is not issued by a trusted CA.
Regards
