Mbed-https: Unable to resolve https-address

A_B · July 12, 2021, 8:58am

Hello,

I am currently trying to connect to an https webserver via mbed-http.
Unfortunately, I am encountering the error that my HTTPS request returns an NSAPI_ERROR_NO_CONNECTION (-3004) error. If I change the URL to the IP of the webserver, the error changes to NSAPI_ERROR_AUTH_FAILURE] (-3011), this is my code:

std::string http_request_str(staging_backend_url);
http_request_str.append(endpoint).append(api_key);

printf("HTTP Request String: %s\r\n", http_request_str.c_str());

HttpRequestBase *get_req;
if(https) {
	get_req = new HttpsRequest(network, tls_certificates, HTTP_GET, http_request_str.c_str());
} else {
	get_req = new HttpRequest(network, HTTP_GET, http_request_str.c_str());
}

HttpResponse *get_res = get_req->send();
if (!get_res) {
	printf("HttpRequest failed with error code %d\r\n", get_req->get_error());
	return false;
}

To me it looks like that mbed-os is unable to resolve the URL. The second problem is that the webservice is behind Cloudflare which is probably the reason why direct IP accesses don’t work.

Any ideas how to fix this?

JohnnyK · July 12, 2021, 9:53am

Hello,

What a version for MbedOS and mbed-http do you use?

You probably need to call method of the NetworkInterface gethostbyname for your URL address, that will translate it to the IP address (SocketAddress).

I took original mbed-http and made some updates, so you can try updated version of mbed-http and two basic examples for MbedOS 6+.

The mbed-http has two options (overloaded constructor):

pass NetworkInterface - the gethostbyname is called inside the library (only updated no original)
pass TCPsocket - the gethostbyname must be called by you

When the certification is not valid/outdated, the check this - Adding TLS Sockets to Mbed OS | Mbed

BR, Jan

A_B · July 14, 2021, 5:47am

Hello Jan,

thank you so much for your reply.

I use Mbed-OS 6.12.0 and mbed-http tip from here: mbed-http - HTTP and HTTPS library for Mbed OS 5 | Mbed.

I also use BSP_DISCO_F746NG tip.

If I change my code to this (added the gethostbyname as you recommended):

std::string http_request_str;
if(https) {
	http_request_str = backend_url;
} else {
	http_request_str = dev_backend_url;
}

http_request_str.append(endpoint).append(device_key);

printf("HTTP Request String: %s\r\n", http_request_str.c_str());

SocketAddress temp_socket;
const nsapi_error_t error = network->gethostbyname(http_request_str.c_str(), &temp_socket);
printf("NSAPI RESULT: %d\r\n", error);

printf("Address: %s\r\n", temp_socket.get_ip_address());

HttpRequestBase *get_req;
if(https) {
	get_req = new HttpsRequest(network, tls_certificates, HTTP_GET, http_request_str.c_str());
} else {
	get_req = new HttpRequest(network, HTTP_GET, http_request_str.c_str());
}

HttpResponse *get_res = get_req->send();
if (!get_res) {
	printf("HttpRequest failed with error code %d\r\n", get_req->get_error());
	return false;
}

it takes about two minutes or so until the function returns with

NSAPI RESULT: -3009

Any ideas what else I can do?

JohnnyK · July 14, 2021, 6:42am

Ye, that is the original library but is not compatible with MbedOS 6+.

NSAPI RESULT: -3009

NSAPI_ERROR_DNS_FAILURE (DNS failed to complete successfully)

DNS Resolver - API references and tutorials | Mbed OS 6 Documentation

NSAPI_ERROR_DNS_FAILURE indicates that DNS failed to complete successfully. Check the host name and network connection.

Also there were a bug NSAPI DNS queries failing for some websites · Issue #3926 · ARMmbed/mbed-os · GitHub in the past.

BR, Jan

A_B · July 14, 2021, 7:13am

Okay, that is unfortunate.

Is there a replacement-library for Mbed OS6 which supports https-requests?

JohnnyK · July 14, 2021, 7:44am

I tested it only with URLs what are in examples what are in TEST folder.
Please, let me know.

BR, Jan

A_B · July 14, 2021, 7:49am

I removed the original mbed-http library and installed your version of it but I am getting the same error.

JohnnyK · July 14, 2021, 8:04am

Ok, that seems to to be a not library related issue but for sure you can run the exmple code from TEST folder.

BR, Jan

A_B · July 14, 2021, 10:50am

The HTTPS-GET example returns after 2 minutes or so with an error -3003
which means " NSAPI_ERROR_PARAMETER: invalid configuration "

JohnnyK · July 14, 2021, 12:32pm

Strange, I will check it at home, also with Disco-F746NG.

BR, Jan

JohnnyK · July 14, 2021, 5:29pm

Ok, I tried mbed-http with MbedOS 6.12 on Nucleo-F429ZI and Disco-F746NG and executed this code. They both seem to work.

See log

Example of a HTTPS client
[NWK] Connecting to network…
[NWK] Connected to the network
[NWK] IP address: 10.0.1.26

----- HTTPS GET request -----

----- HTTPS GET response -----
Status: 200 - OK
Headers:
Date: Wed, 14 Jul 2021 17:23:24 GMT
Content-Type: text/plain
Content-Length: 14
Connection: keep-alive
Last-Modified: Sat, 17 Nov 2018 12:02:53 GMT
ETag: “5bf0036d-e”
Expires: Thu, 15 Jul 2021 03:23:24 GMT
Cache-Control: max-age=36000
Accept-Ranges: bytes
Strict-Transport-Security: max-age=31536000; includeSubDomains; preload

Body (14 bytes):

Hello world!

----- HTTPS POST request -----

----- HTTPS POST response -----
Status: 200 - OK
Headers:
Date: Wed, 14 Jul 2021 17:23:26 GMT
Content-Type: application/json
Content-Length: 376
Connection: keep-alive
Server: gunicorn/19.9.0
Access-Control-Allow-Origin: *
Access-Control-Allow-Credentials: true

Body (376 bytes):

{
“args”: {},
“data”: “{"hello":"world"}”,
“files”: {},
“form”: {},
“headers”: {
“Content-Length”: “17”,
“Content-Type”: “application/json”,
“Host”: “httpbin.org”,
“X-Amzn-Trace-Id”: “Root=1-60ef1d8e-022c176b6fc8e29b53c9c1f2”
},
“json”: {
“hello”: “world”
},
“origin”: “90.182.255.79”,
“url”: “https://httpbin.org/post”
}

Do you have a public URL what I can test?

BR, Jan

A_B · July 16, 2021, 6:20am

Thanks for your reply.

I get this response:

----- HTTPS GET request -----
HttpRequest failed (error code -3003)

Maybe something is wrong with my internal DNS server. I’ll try to add Google DNS

Edit: Also doesn’t work if I add Google DNS

JohnnyK · July 21, 2021, 5:43pm

Hello,

I wanted to see a list of DNS (I thought there were some defaults ) it but when I tried to call get_dns_server the result is No DNS -3006 = IP address is not known and when I add one, then the result is same.

   SocketAddress *dnsAddr;
   int result = 0;
   if((result = network->get_dns_server(0,dnsAddr)) == NSAPI_ERROR_OK){
       printf("DNS IP %d: %s\n", 0, dnsAddr->get_ip_address());
   }
   else{
       printf("No DNS %d\n", result);
   }

I also tried add a DNS server, especially CloudFlare public DNS server

network->add_dns_server(SocketAddress("1.1.1.1"),name);

It want to pass an interface name, but that cause a MbedOS crash when you tried to read it back.

    SocketAddress *dnsAddr;
    char name[5];
    network->get_interface_name(name);
    int result = 0;
    result = network->add_dns_server(SocketAddress("1.1.1.1"),name);
    printf("DNS add %d\n", result);

    int atempt = 0;

    while(1){
        if((result = network->get_dns_server(atempt,dnsAddr,name)) == NSAPI_ERROR_OK){
            printf("DNS IP %d: %s\n", atempt++, dnsAddr->get_ip_address());
        }
        else{
            printf("No DNS %d\n", result);
            break;
        }
    }

Dear @JojoS, do you have any idea?

BR, Jan

JojoS · July 23, 2021, 2:41pm

this is some testcode that I used:

    // print DNS entries
    err = NSAPI_ERROR_OK;
    int index = 0;
    while(err == NSAPI_ERROR_OK) {
        err = net.get_dns_server(index++, &sockAddr, "st0");
        if ((err == NSAPI_ERROR_OK) || (index == 1)) {
        switch (sockAddr.get_ip_version()) {
            case NSAPI_IPv4: printf("DNS #%d IPv4: ", index); break;
            case NSAPI_IPv6: printf("DNS #%d IPv6: ", index); break;
            default: printf ("DNS #%d IPv unknown: ", index);
        }
            (err == NSAPI_ERROR_OK) ? printf(" %s\n", sockAddr.get_ip_address()) : printf("get DNS err: %d\n", err);
        }
    }

With a fixed name for ST boards, your solution with getting the interface name looks cleaner.

I remember that there is a fixed DNS built in the code, the google address 8.8.8.8:

github.com

ARMmbed/mbed-os/blob/efaf159ca44f62ca665f50312a410360a85abc70/connectivity/lwipstack/source/LWIPInterface.cpp#L56


      
          {
              for (Interface *interface = list; interface; interface = interface->next) {
                  if (netif == &interface->netif) {
                      return interface;
                  }
              }
          
          
    return NULL;
          }
          
          
static void add_dns_addr_to_dns_list_index(const u8_t addr_type, const u8_t index, struct netif *netif)
          {
          #if LWIP_IPV6
              if (addr_type == IPADDR_TYPE_V6) {
                  /* 2001:4860:4860::8888 google */
                  ip_addr_t ipv6_dns_addr = IPADDR6_INIT(
                                                PP_HTONL(0x20014860UL),
                                                PP_HTONL(0x48600000UL),
                                                PP_HTONL(0x00000000UL),
                                                PP_HTONL(0x00008888UL));
                  dns_setserver(index, &ipv6_dns_addr, netif);

JohnnyK · July 24, 2021, 8:02am

Ah, the if and switch statements with IP version sorting are probably necessary, I knew you will advise

    nsapi_error_t err;
    char name[10];
    network->get_interface_name(name);
    // add CloudFlare public DNS resolver
    if ((err = network->add_dns_server(SocketAddress("1.1.1.1"),name))== NSAPI_ERROR_OK) {
        printf("DNS added successfully\n");
    }
    else printf("DNS add error: %d\n", err);

    SocketAddress sockAddr;
    err = NSAPI_ERROR_OK;
    int index = 0;
    while(err == NSAPI_ERROR_OK) {
        err = network->get_dns_server(index++, &sockAddr, name);
        if ((err == NSAPI_ERROR_OK) || (index == 1)) {
        switch (sockAddr.get_ip_version()) {
            case NSAPI_IPv4: printf("DNS #%d IPv4: ", index); break;
            case NSAPI_IPv6: printf("DNS #%d IPv6: ", index); break;
            default: printf ("DNS #%d IPv unknown: ", index);
        }
            (err == NSAPI_ERROR_OK) ? printf(" %s\n", sockAddr.get_ip_address()) : printf("get DNS err: %d\n", err);
        }
    }

Thank you for the response and your help

BR, Jan

A_B · July 30, 2021, 6:40am

I tried this code:

    SocketAddress sockAddr;
    connect_status = NSAPI_ERROR_OK;
    int index = 0;
    while(connect_status == NSAPI_ERROR_OK) {
        connect_status = network->get_dns_server(index++, &sockAddr, "st0");
        if ((connect_status == NSAPI_ERROR_OK) || (index == 1)) {
            switch (sockAddr.get_ip_version()) {
                case NSAPI_IPv4: printf("[NWKH] DNS #%d IPv4: ", index); break;
                case NSAPI_IPv6: printf("[NWKH] DNS #%d IPv6: ", index); break;
                default: printf ("[NWKH] DNS #%d IPv unknown: ", index);
            }

            (connect_status == NSAPI_ERROR_OK) ? printf(" %s:%d\n", sockAddr.get_ip_address(), sockAddr.get_port()) : printf("get DNS err: %d\n", connect_status);
        }
    }

And sockAddr.get_port() is always 0. Can someone confirm this? Shouldn’t this be the typical DNS port 53?

My problem still is that line 62 of mbed-http - Fork of SandBox’s original mbed-http (https://os… | Mbed fails with error -3009. Any ideas where to start debugging?

Edit:

I changed NetworkInterface::gethostbyname to this:

nsapi_error_t NetworkInterface::gethostbyname(const char *name, SocketAddress *address, nsapi_version_t version, const char *interface_name)

{

    printf("STACK NAME: %s\r\n", typeid(get_stack()).name());

    return get_stack()->gethostbyname(name, address, version, interface_name);

}

with the result P12NetworkStack. After grep-ing for this name I only get entries in .map-files so it looks to me like this is some kind of binary-blob which is linked at compile time? But where is the source-code?
Can I switch the network-stack?

Edit: Okay, I think I found the solution thanks to this post: Gethostbyname() question - Mbed OS - Arm Mbed OS support forum

network->gethostbyname(_parsed_url->host(), &address, NSAPI_IPv4, "st0");

Notice the two additional parameters at the end.

JohnnyK · July 30, 2021, 8:40am

Hello,

just for clarification, after this

the rest is working and do you are able to connect?

Thank you
BR, Jan

A_B · July 30, 2021, 8:53am

Your example code is working fine (accessing https://os.mbed.com/media/uploads/mbed_official/hello.txt) but I get -3011 error for accessing my cloudflare-website.

Basically what I did was openssl s_client -connect <my_page>:443 -showcerts and then copy the two certificates into my mbed-project.

Maybe I am using the wrong certificates?

JohnnyK · July 30, 2021, 9:13am

Yes, I did it same and cope&pase the last one, according to Adding TLS Sockets to Mbed OS | Mbed

BR, Jan

A_B · July 30, 2021, 9:22am

How did you get mbed_trace working? I set "mbed-trace.enable": 1 in mbed_app.json and added this library to my project: mbed-trace - Tracing Library | Mbed as well as mbed_trace_init(); but it fails with several compiler errors in mbed_trace.c.

Edit: Oh okay, it looks like mbed_trace is built-in now so I don’t need the library.

Something isn’t working though:

[INFO][TLSW]: Starting TLS handshake with ��1zg
                                               ��
w�6v}��߅�>��V¤Tr�z���78�L¬±����X�]�c�A�@°�?s�m��c³O¯%¨¹'#a269dt�z¹b�^����6Q6��� )Dz�:T�§E
[ERR ][TLSW]: mbedtls_ssl_handshake() failed: -0x7780 (-30592): SSL - A fatal alert message was received from our peer

Topic		Replies	Views
HTTP request using IP instead of URL Mbed OS	2	621	March 7, 2019
Any suggestion for mbed HTTP library (mbed-http not work with recent mbed-os) Mbed OS	2	1210	March 8, 2020
Unable to connect using TLSSocket and https Mbed TLS mbed_tls	9	1200	August 23, 2022
DISCO_L475VG_IOT01 WIFI HTTP SERVER example Mbed OS	5	843	November 18, 2020
Program with mbed-http doesn't run on L-Tek FF-LPC546XX Mbed OS mbed_tls	0	362	December 26, 2019

Mbed-https: Unable to resolve https-address

Related topics