Struggling to record serial input. UnbufferedSerial loses data. BufferedSerial crashes

Mbed OS
#1

I’m trying to record incoming data on a serial port from a GPS receiver to a file on an NXP LPC1768 board. For context, the GPS sends 4-6 lines of text (NMEA) every second, each around 30-75 characters long. In other words, I can expect to receive up to 450 characters per second, arriving in short bursts. The baud rate of 9600 can’t be changed easily.

When using UnbufferedSerial the program prints and records data without crashing, but it very often loses characters. The output is incomplete and invalid. There seems to be a pattern to it, since it only loses the end of some of the lines, never the beginning. It’s as if the program can’t keep up, even when commenting out fprintf() and doing nothing more than GPS.read() and printf().

When using BufferedSerial all the data is received and printed to the terminal, as long as fprintf() is commented out. As soon as fprintf() is introduced, the program crashes shortly or immediately after receiving data. The error code is 0x80FF013D (System is out of memory). Considering how little memory the program allocates, this seems odd. Figured it might refer to the serial buffer usage, but increasing drivers.uart-serial-rxbuf-size to 512 or even 1024 doesn’t help.

Any ideas how to resolve this issue? What am I doing wrong? :slight_smile:

#include "mbed.h"

BufferedSerial GPS(p13, p14, 9600);
LocalFileSystem local("local");
FILE* fd;

int main()
{
  fd = fopen("local/log.txt", "w");
  if (fd==NULL) {
    printf("fopen() failed\n");
  }
  else
  {
    char input;
    while (1)
    {
      GPS.read(&input, 1);
      printf("%c", input);
      // Program crashes when fprintf is introduced while using BufferedSerial.
      // Error code 0x80FF013D (System is out of memory)
      fprintf(fd, "%c", input);

      if (input == '\n')
      {
        // Closing and re-opening on every line-break, to save the
        // data and make it available to the computer
        fclose(fd);
        fd = fopen("local/log.txt", "a");
        if (fd == NULL)
        {
          printf("Re-opening failed\n");
          break;
        }
      }
    }
  }
}

Error message:

++ MbedOS Fault Handler ++

FaultType: HardFault

Context:
R   0: 00000009
R   1: 100027AC
R   2: 10002B28
R   3: 00000001
R   4: 00000009
R   5: 100027AC
R   6: 00000000
R   7: 80000000
R   8: 00000000
R   9: 00000000
R  10: 00000000
R  11: 00000000
R  12: 00000E65
SP   : 100027A8
LR   : 000021F5
PC   : 00003360
xPSR : 01000000
PSP  : 10002788
MSP  : 10007FA0
CPUID: 412FC230
HFSR : 80000000
MMFSR: 00000000
BFSR : 00000000
UFSR : 00000000
DFSR : 0000000A
AFSR : 00000000
Mode : Thread
Priv : Privileged
Stack: PSP

-- MbedOS Fault Handler --



++ MbedOS Error Info ++
Error Status: 0x80FF013D Code: 317 Module: 255
Error Message: Fault exception
Location: 0x3360
Error Value: 0x100028D0
Current Thread: main Id: 0x100011A4 Entry: 0x1C25 StackSize: 0x1000 StackMem: 0x100018D0 SP: 0x100027A8 
For more info, visit: https://mbed.com/s/error?error=0x80FF013D&tgt=LPC1768
-- MbedOS Error Info --
#2

Hello Jesper,

I can expect to receive up to 450 characters per second, arriving in short bursts. The baud rate of 9600 can’t be changed easily.

When using UnbufferedSerial the program prints and records data without crashing, but it very often loses characters.

In the loop of main thread you have:

GPS.read(&input, 1);
printf("%c", input);

Let’s assume there are 10 bits per byte (1 start bit + 8 data bits + 1 stop bit + 1 gape bit between data) needed for the serial communication. Then at the the speed of 9600 bits/second it takes

(450 bytes * 10 bits (received) + 450 bytes * 10 bits (printed) ) / 9600 = 937.5 ms

to complete the two functions above . Hence, for the rest of the program remain 1000 - 937.5 = 62.5 ms per second. It might be enough. But to get more, since the GPS speed is difficult to change, try to speed up the console printf.

You can either add a mbed_app.json :

{
    "target_overrides": {
        "*": {
            "platform.stdio-baud-rate": 460800
        }
    }
}

or to override the stdout console by adding:

FileHandle* mbed::mbed_override_console(int)
{
    static BufferedSerial   myConsole(USBTX, USBRX, 460800);
    return &myConsole;
}

For more useful info about serial communication have a look at this post.

#3

Thank you, that provided me with a lot of insight, in particular how to calculate how much time it takes.

However, the same thing happens even without printf("%c", input);, which should cut down the time spent significantly. UnbufferedSerial still loses characters and BufferedSerial still crashes :frowning:

Tried narrowing it down even further, and it turns out that the program crashes with BufferedSerial even when only a single byte is written to the file, while the rest of the program spins to keep the buffer empty.

#include "mbed.h"

BufferedSerial GPS(p13, p14, 9600);
LocalFileSystem local("local");

int main()
{
  FILE *fd;
  fd = fopen("/local/log.txt", "w");
  if (fd == NULL)
  {
    printf("fopen() failed\n");
  }
  else
  {
    char input;
    GPS.read(&input, 1);
    
    // Write a single byte to the file
    fprintf(fd, "%c", input);
    
    // Try to keep the buffer empty
    while(1) {
      GPS.read(&input, 1);
    }
  }
}

I’m starting to think that there’s some sort of interrupt conflict going on?

#4

Hello,

for one char per loop try to use fputc()

Or did you tried something like below?

    while (1) {
        if (uint32_t num = GPS.read(buf, sizeof(buf))) {
            fd = fopen("local/log.txt", "a+");
            fwrite(buf, sizeof(char), num, fd);
            fclose(fd);
        }
    }

BR, Jan

#5

Since BufferedSerial is blocking by default calling the GPS.read function blocks the main thread. Try to set it not blocking:

int main()
{
    GPS.set_blocking(false);
    ...
#6 
// Program crashes when fprintf is introduced while using BufferedSerial.
// Error code 0x80FF013D (System is out of memory)

How about the obvious, not enough ram? LPC1768 has only 32 kbit of ram and for me it seems that this tiny code consumes a lot already due file operations