How do I get more RAM/storage for a data buffer - do I just buy a better board?

Mbed OS
1

I have an LPC1768 and I am trying to use it to log data and save to an SD card or stream to PC.

I have an external 12bit ADC with 6 multiplexed channels and I want to sample these at 5kHz.
So that is 6 channels at 5kHz + a time stamp potentially.

I need to log this data for up to several minutes at a time. I would like the option to either stream it to PC or save it to SD card / USB to then put in a PC (ideally I want to do both, but not necessarily at the same time).

So first thing I tried is streaming to a PC through teraterm and the max rate I can get on this is 1kHz even with printf or putc. This is in real time as well so quite good but I need a higher sample rate

Using SD card is also too slow even when increasing the SPI frequency to 25MHz

If I try to use a buffer I can only get about 1000 samples for each channel before it stops compiling, so that’s just a few milliseconds

So how do I store this much data? Is it possible to buy external memory and is this easy to use as a buffer? I have looked at buying better boards but it seems most have less than 100kb of RAM so I dont think buying these would work
Or are there other methods to use? I think I am looking at close to 400kB/s of data I need to store or stream

2

Hello Bakez,

Considering that 12bit data can be stored in two bytes you can calculate the minimum rate as:

2 (Bytes) * 5 (kHz) * 6 (channels) = 60kByte/s (480kbit/s).

But because a data transfer involves also some additional bits (like start and check bits etc.), pause between bytes and between packets you’ll probably need a bit higher speed. Let’s assume 100kByte/s (800kbit/s).

The LPC1768 MCU supports full-speed USB, which is 12Mbit/s (12000kbit/s). So 800kbit/s should be achievable. The USBCDC, USBCDC_ECM, USBSerial or USBHID are good candidates. Maybe the USBAudio could be used as well if you are able to dig out your data from an audio file on PC.

Best regards, Zoltan

3

Dear Zoltan
Thank you for your reply.
I had looked into these USB files eg USBSerial.
However isn’t this already what I’ve been doing? I am already viewing data through teraterm by using SerialPC. I upped the baud to 921600 and can still only really max out at 1-1.5kHz depending on the rest of the code outside of the printf or putc.

How would I capture the data on a PC from USBSerial and how would the speed be different?

4

Hello Bakez,

I agree, It’s not obvious but when the mbed LPC1768 is connected to the PC over a USB cable plugged into the board’s USB-mini connector then the connection involves two sections, RS-232 TTL + USB:

LPC1768 ← (RS-232 TTL serial line) → (UART to USB converter) ← (USB mini connector + cable) → PC

The bottleneck of such connection is the RS-232 section. When you use 921600 bit/s for the RS-232 (UART interface) no matter how fast the USB line could be the final speed is limited to 921600 bit/s. As I already said you should account also for the delays between bytes and packets. So I’d recommend to try a pure USB connection:

LPC1768 ← (USB line) → PC

To avoid the RS-232 section you should connect an external USB connector to the LPC1768’s USB pins (p31, p32) as shown here and connect the PC over this external USB connector. Then you can achieve a 12Mbit/s USB full speed rate. The USB device classes listed above work only over this type of connection.

How would I capture the data on a PC from USBSerial.

With USBCDC or USBSerial you can use the same code as you did for the serial (UART/RS-232) connection. You should just use a different virtual serial (COM) port which is associated with mbed pure USB connection (speed settings will be ignored for such port - USB full-speed will be used). On MS Windows 10, Mac and Linux no manual driver installation is needed. However, a mbed USB driver shall be installed on Windows 7 and earlier.

Best regards, Zoltan

5

If streaming to a PC is acceptable, maybe you can use Ethernet? I often find TCP the easiest way to transfer data. TCP is reliable. I think it is fast enough for your project.

6

From your query title - “How do I get more RAM/storage…”

You mention you are using the LPC1768. If you are not already using the upper 32K RAM bank for other purposes, there’s a free 32K right there. The normal mbed behavior is to not use that. Do take note that some drivers use the upper RAM, and this is often not well documented. On the mbed site you’ll find references to Ethernet and CAN as users of those services, but there may be others.

A small example, for the LPC1768 is this. I have a typedef for some Log Data, and then I define a pointer to that data in the upper RAM (using the fixed address as shown).

LogData_T* LogData = (LogData_T*)(0x2007C000);

After that, I can simply use LogData as an array:

LogData[logEntries++] = latestData;

Size-limiting or bounds-checking is up to your code of course.

1 Like
7

Hi Zoltan
I have tried this method using a USB cable which I stripped the wires from and it seems to be slower than using the normal serial through the programming cable
It is taking 2.28 seconds to send 1000 lines of the same data = 500Hz max
Am using TeraTerm to view the data but using the different COM port like you said

In fact after further tests it is about 5x slower. Took 16seconds vs 3seconds on stress test on putc and printfs i just did

8

Hello Bakez,

I built the following test program with GCC ARM on Ubuntu 18.04 and used GTKTerm 1.0 as serial terminal:

#include "mbed.h"
#include "USBSerial.h"

#define RS232

#define BUFF_SIZE  (2*5000) // 2 bytes * 5 kHz = 10000 bytes (per channel)

uint8_t     buff[BUFF_SIZE];
Timer       timer;
#ifndef RS232
USBSerial   usbSerial;
#endif

int main()
{
    // Fill the buffer with characters
    for (int i = 0; i < BUFF_SIZE; i++) {
        buff[i] = (i % 24) + 65;  // capital letter
    }

    while (1) {
        timer.reset();
        timer.start();
        // Send data of 6 channels: 6 * 10000 = 60000 bytes
        for (int i = 0; i < 6; i++) {
#ifdef RS232
            for (int j = 0; j < BUFF_SIZE; j++) {
                putc(buff[j], stdout);
            }
#else
            usbSerial.send(buff, BUFF_SIZE);
#endif
        }
        timer.stop();
        auto elapsedTime = chrono::duration_cast<chrono::milliseconds>(timer.elapsed_time()).count();

#ifdef RS232
        printf("\r\nIt took %llu ms to send.\r\n", elapsedTime);
#else
        usbSerial.printf("\r\nIt took %llu ms to send.\r\n", elapsedTime);
#endif
        ThisThread::sleep_for(5000);
    }
}

and this mbed_app.json:

{
    "target_overrides": {
        "*": {
            "platform.stdio-baud-rate": 921600
        }
    }
}

The result was:

It took 714 ms to send.

After deleting (or commenting out) #define RS232 the result was:

It took 85 ms to send.

Please note that when sending data over RS232 the signal on the serial line is same with a connected receiver as without a receiver. However, the USB connection is different. It waits for the receiver to receive and “process” the data. Consequently, you will get different results with the TeraTerm window maximized and with a small TeraTerm window.

Best regards, Zoltan

1 Like