Possible/How to use ESP32/WiFi to download large files over HTTP?

Mbed OS
1

Hi,

I have been trying to download some relatively large files (~1MB) over HTTP via ESP32-WiFi or Ethernet. The same set of code and files seem work just fine over Ethernet, but I constantly loose data randomly over ESP32/WiFi. The following are the results of 3 attempts to download a file = 704KB:

2791: fota, tried to verify downloaded fw, length = 577816, calculatedCrc32 = C754732A, expectedCrc32 = A96F7467
2376: fota, tried to verify downloaded fw, length = 564676, calculatedCrc32 = 17AD1251, expectedCrc32 = A96F7467
3039: fota, tried to verify downloaded fw, length = 590956, calculatedCrc32 = DB4A6F2A, expectedCrc32 = A96F7467

The length of actually downloaded file varies from one attempt to another, but generally it is about 120KB shorter than expected file length, and of course the CRC32 doesn’t match expected value.

I am using the ESP32-driver from the following link:

Has anyone tried to use ESP32/WiFi module to transfer large files? Or the other WiFi module from Inventek that is installed on one of DISCO board from STM?

At this point, I am not sure if this is a problem in the driver, or I am not using it correct, or the limitation of ESP32 module itself. Please advise.

Thanks,
ZL

2

The problem seems to be in SPIFBlockDevice that downloaded chunks of data are saved into. If I only track length and calculate CRC32 of downloaded data and don’t try to save them into SPIFBlockDevice, both length and CRC32 of downloaded data seems to be OK. ESP32/WiFi is slower than Ethernet, but downloaded data comes in larger chunks and sometimes in burst. My guess is when system is busy saving previously downloaded data into SPIF, some chunks of new data are ignored.

3

Hello,
a time ago i tried something similar but with binaries, pictures and html pages. I do not know what is your possibility and requirements but i solve that with controlled transfer. Input file was divided to chunks with IDs and CRCs. When client received a chunk, then did a CRC check and send back to server ID and CRC result. The server did an evaluation and if everything was ok then sent another chunk. That also make a space to slowly store this in memory.

BR, Jan

1 Like
4

Hi,

Thanks for your reply. Your solution is probably a more robust and at the same more complicated way to solve the problem. Because we will have a large external SDRAM anyway, my plan is to save into SDRAM first then move into SPIF. I haven’t tested it yet, I think that will work.

Too bad the hardware flow control in the library doesn’t seem to work, at least when I tested it on our customized F767 board. If hardware flow control works, that may prevent such problem as well.

Best,
ZL

1 Like
5

I ended up implementing a similar way to download those large files chunk by chunk. If each chunk is larger than ~300KB, the chance that a portion of data missing increases rapidly. I think the root problem is that UART is just not suitable to continuously transfer large amount of data. Because it is asynchronous, the larger/longer each transfers takes, the bigger the accumulated error rate is. In that sense, I bet those DISCO boards with WiFi module connected via SPI will work better for downloading large files.

6

From my point of view the ideology, what was applied to Mbed, is not applicable for everything. I mean sometimes you have to make your application more specific for your hardware.

Just a tip because I have no experiences with ESP32. The UART can be the bottleneck of course, however the UART it self is not so slow. So try to look for some limitations of UART’s packet size or buffer size in this mode in a EPS32 reference manual (or something around). I did this with the small ESP8622 and I remember there was a packet size limitation what I found in AT command manual or something like that. Then I decided to sending packets not bigger than 100 bytes.

BR, Jan

7

What if you try disabling sleep on your board? I have battled with this bug in the past, it causes UART corruption on STM32 devices when sleep is enabled and has been unfixed for years. Maybe it’s causing issues with AT commands to the modem?

8

I don’t think the issue I had was caused by the sleep mode. It is more likely due to the limitation of UART itself. Due to its asynchronous nature, there is an inevitable non-trivial error rate. If large files are involved, SPI, as used in DISCO-L475VG , seems like a better option. If I control chunk size under 250KB, which is still quite large by embedded system standard, and add checksum to each chunk as suggested by @JohnnyK , I think I am good now.

Several years ago, someone asked for support to use ESP32 via SPI, but that didn’t go anyway unfortunately.

9

Honestly, I am suspicious of that. I use UART at my job with a fancy DMA driver that I spent quite some time putting together, and we can run it for hours without anything being dropped. Unless you are running your UART so fast that you actually are getting corruption due to electrical limitations of the board, it should be able to go quite some time without any corruption happening. In my experience, corruption is much more likely to happen due to firmware error, e.g. missing bytes from the UART peripheral because it does not have a FIFO and the application has to grab the byte within 10s of us or it will be lost.

Just to rule it out, I’d appreciate if you could try turning off sleep and seeing if it makes any difference at all. Would help me determine how serious this issue is!

10

I think you are right in the sense that UART should be able to handle relatively large data transfer without problem. My impression that UART can be unreliable came from experience working with MSP430. There is usually a table detailing expected error rate with different modulation configuration and baud rate, something like the following:

image
image1670Ă—592 57.6 KB

The error rate is nontrivial. Because MSP430 is slow and takes time to wake up from sleep, we have to limit baud rate to 9600. Otherwise we would run into issues you described in the bug related to STM32 UART.

The root cause of issues downloading large chunk of data with ESP32 turns out quite silly.

static void fotaFwDownloadCallback(const char* chunkData, uint32_t chunkLen)
{
    static uint16_t chunkCount = 0;
    // code to copy chunkData into SDRAM
    //printf("%d: fota downloaded chunk #%d, length = %d, accumulated length = %d\r\n", chunkCount++, chunkLen, fotaDownloadedFwLength);
    printf("\n");
}

Printing out a long debug message takes long enough time that it delays/messes up receiving data from ESP32. If I reduce the debug message to a newline, or remove the debug message altogether, I can actually reliably download some 700+KB file with one request:

// without printf debug message in callback
8531: fotaVerifyDownloadedFw, expectedLength = 720896, actualLength = 720896, expectedCrc32 = 866798E3, actualCrc32 = 866798E3
// with full debug message in callback, sdio baud rate = 115200
8924: fotaVerifyDownloadedFw, expectedLength = 720896, actualLength = 633281, expectedCrc32 = 866798E3, actualCrc32 = 00000000
// with full debug message in callback, sdio baud rate = 460800
9560: fotaVerifyDownloadedFw, expectedLength = 720896, actualLength = 611336, expectedCrc32 = 866798E3, actualCrc32 = 00000000
// with newline printf in callback, sdio baud rate = 460800
9972: fotaVerifyDownloadedFw, expectedLength = 720896, actualLength = 720896, expectedCrc32 = 866798E3, actualCrc32 = 866798E3

The tests above were done on a DISCO_F746NG board + external ESP32-S2 module connected via some 60CM loose wires, no hardware flow control, baud rate = 460800.
At this point, I think we can have the case closed. We just have to minimize what we do in the callback. I previously had code that saves chunkData into block device in of the iterations. This didn’t help as well.

11

That’s actually pretty weird. The only reason I can think of is that printing the debug message would cause the UART interrupt handler to activate a lot (especially so if at a high baudrate), which would cause delays when the ESP32 UART handler tries to execute.

Wow, 460800 is a pretty high baudrate! Since every single character requires the UART Rx ISR to fire before the next character arrives, I suspect that just about any significant critical section or interrupt handler is going to give you these types of issues :frowning: . If it were me, I would turn that baudrate down if I’m doing anything critical with it.

12

In production code, we primarily use Ethernet and only use cellular or ESP32/WiFi, both over UART with baud rate at 115200, if Ethernet is not available. I used 460800 to prove your point that even at high baud rate ESP32/WiFi over UART can still work if we are careful with the callback routine.

I poked a little further, printing out a few more characters will start to cause data loss. With the following line in callback routine:

printf("chunk #\n");

I get a 50-50 chance of data loss, sometimes I can download the whole 700KB, sometimes ~100KB data loss will happen. Basically moving data from RAM to external SDRAM is about the only stuff we can do in the callback routine for long period of time(10s of seconds).

13

In your application, is it possible to put the ESP32 on the LPUART peripheral? That one does have a FIFO, unlike the other STM32F7 UART peripherals. If you could put it on the LPUART, and we update Mbed to actually use the FIFO, then that should improve this situation greatly.

14

Did you mean that LPUART is available on STM32F7? I briefly went through the reference manual for F7 and didn’t see any mention of it. LPUART seems only available on L4 and probably a few other lines of STM32 MCUs. My take is that switching to LPUART is probably going to open another can of worms.

The only time we need to download large files is FOTA. If we only download the firmware images one small piece at a time, 16KB or 32KB and stick with 115200 baud rate, I think we should be fine.

15

Ah, rip. I guess truly the only decent way to do this is with DMA…