UDP socket in blocking mode missing incoming messages

Mbed OS Bugs
1

We use the UDP socket in blocking mode to communicate with our cloud (DTLS encrypted). This has been going well for years, but since a week, we experience that after the DTLS handshake some devices get stuck at exchanging Application Data. Sometimes at the first exchange, but sometimes at the second or third. We only see this behaviour on devices that have (LTE-M) SIM cards from a specific telco, while none of our devices with SIM cards of other telcos are affected. We have not changed anything on our cloud side (or done FOTA’s recently). All devices are STM32F4 with a Ublox SARA-R410M modem module. We’re on mbed OS 5.13.4.

We found out that the behaviour is caused by UDPSocket::recvfrom getting stuck on the line flag = _event_flag.wait_any(READ_FLAG, _timeout);. If we dive a bit deeper, we see that there is no API_EVENT/ sys_mbox_trypost being done and that even UARTSerial::rx_irq is not firing. If we read out the rx line at that time with a second serial connection (to a desktop), we don’t see any data going through it after the transmit of the third exchange (after which it is expected to receive).

We’re pretty sure the Application Data has been received by the modem. In the cloud we see the application data coming in and it sending a response. More importantly, if we put the device in non-blocking mode, data comes in through the rx line, UARTSerial::rx_irq fires as it should and the data can be read.

As I have difficulty fully understanding IP stack, I have a few questions:

  • Do I understand correctly that when you create a connection, a new thread ‘ppp_lwip’ is started (if it didn’t exist already) that listens to rx data and uses sys_mbox_trypost to send it to a mailbox?
  • Is it another thread (the one you created the connection with) that runs UDPSocket::recvfrom and in the end uses sys_mbox_fetch to read from that mailbox?
  • Can the ppp_lwip thread be blocked when UDPSocket::recvfrom is stuck, causing the data being sent by the modem to be lost?
  • Can it be that the rx line is set to 0V when UDPSocket::recvfrom is stuck and with that, causing the data being sent by the modem to be lost?

Any tips or guidance would be highly appreciated, we’re looking at this issue with 2 people for about a full week now and we don’t have the idea we’re getting closer.