Serial communication with a Python UI: I'm failing effortfully

I am trying to get an LPC1768 to accept multi-character commands from a python interface on a PC. Days of troubleshooting have passed, and I’ve distilled the problem into the demo program attached below. I am unable to debug normally, because, well, I can’t even get one-way communication to work correctly, and I imagine it’d be a whole other project to receive messages properly in the python terminal. So, what I’ve got to work with is the four LEDs on the board itself. In the attached program, they blink out the received message. Here is the result: (“-” stands for an unknown non-digit, non-upper-case-letter character):

      "TT1234S" yields "T-----T1234S"
      "S1234T" yields "S-----1234T-"
      "S123T" yields "S-----123T--"

So, it seems that it takes the first character, fills in the rest of the six-long buffer with something, then, next time around, reads in the remaining characters, filling in again as necessary to fill up the buffer. In other experiments, I have seen the second character dropped entirely, or all but the first and last character dropped entirely. I’ve seen it always print the previous message.

I’m guessing that the problem is some sort of basic misunderstanding, but I’ve been unable to find any helpful online tutorials on this topic. I do understand that the problem could very well be on the python end, but I doubt it, because a previous version of this MBED program worked fine with the same Python program.


#include "mbed.h"
#include "ThisThread.h"

BufferedSerial pc(USBTX, USBRX); // tx, rx
char charBuffer[6] = {0};
DigitalOut First_LED(LED1);
DigitalOut Second_LED(LED2);
DigitalOut Third_LED(LED3);
DigitalOut Fourth_LED(LED4);

void blinkItOut (int encoded) {
    Second_LED = false;
    Third_LED = false;
    Fourth_LED = false;
    if (encoded == 83 || encoded == 84) {        
        encoded -= 82;
        for (; encoded > 0; --encoded) {
            Second_LED = true;
            ThisThread::sleep_for(200);
            Second_LED = false;
            ThisThread::sleep_for(200);
        }
    }
    else if (encoded > 47 && encoded < 58) {
        encoded -= 47;
        for (; encoded > 0; --encoded) {
            Third_LED = true;
            ThisThread::sleep_for(200);
            Third_LED = false;
            ThisThread::sleep_for(200);
        }
    }
    else {
        Fourth_LED = true;
        ThisThread::sleep_for(500);
        Fourth_LED = false;
    }
    
}

int main() {
    pc.set_baud(9600);
    pc.set_format(8, BufferedSerial::None, 1);
    First_LED = 1;
    ThisThread::sleep_for(1000);
    First_LED = 0;
    while (true) {
        if(pc.readable()) {
            pc.read(&charBuffer, 6);
            for (int i = 0; i < 6; ++i) {
                blinkItOut(charBuffer[i]);
                ThisThread::sleep_for(2000);
            }
            for (int i = 0; i < 6; ++i) {
               charBuffer[i] = 0;
            }   
        }
    }
}

Hi there,

that is because the BufferedSerial has different behavior than you think. It does not must wait for 6 chars always. So the dashes comes from your for loop.
Better will be use

        if (uint32_t num = serial_port.read(buf, sizeof(buf))) {
          // collect received num of chars and proceed
        }

Also the multiple waits in the function is not best chose, this just slows down the whole loop. So if you need it then you need use different method like eventqueue for example.

BR, Jan

1 Like

Thank you for the quick reply. To clarify, are you suggesting that “if” statement as a replacement for–

if(pc.readable()) {
            pc.read(&charBuffer, 6);
.
.

?
And are you saying that the mystery characters come from the second for-loop, which I intended to blank out the buffer with zeros? Why aren’t those characters zeros, then? How does the way that the buffer is read effect the behavior of a separate for-loop which writes to the buffer?

Also, I’ve never seen an if statement used with an assignment “=” operator as the condition. How does that evaluate to true/false?

-Aidan

To be honest, in your code is not any method for printout characters so it is not same what you use. So I can not say “dashes are created exactly here”.
But I see problem in your first for loop. You assume the buffer is always filled with 6 chars, what is not always true.

Nope, I mean something like this.

   while (true) {
      if(pc.readable()) {
         if(uint32_t num = pc.read(&charBuffer, 6)) {
            //change the fixed size 6 to num, what represent number of really read chars
            //for (int i = 0; i < 6; ++i) {
            for (int i = 0; i < num; ++i) { 
               blinkItOut(charBuffer[i]);
               ThisThread::sleep_for(2000);
            }
            for (int i = 0; i < num-1; ++i) {
               charBuffer[i] = 0;
            }   
         }
      }
   }

It is same like if(pc.read(&charBuffer, 6)), but with variable. I am also learning every day.

Have a nice day
Jan

Jan,
I think the following example, made with consideration for your prior suggested method, will illustrate my current confusion well:

while (1) {
        if (serial_port.readable()) {
            if (uint32_t recievedMessageLength = serial_port.read(fromPCBuffer, 4)) {
                blinkItOut(recievedMessageLength);                           
                ThisThread::sleep_for(3000);
                for (int i = 0; i < recievedMessageLength - 1; ++i) {
                    blinkItOut(fromPCBuffer[i]);               
                    ThisThread::sleep_for(1000);
                }                           
                ThisThread::sleep_for(3000);
                blinkItOut(recievedMessageLength);
            }
        }
    }

Sent message: “S350”
Expected result: “4” … “S” “3” “5” … “4”
Actual result: “0” … (skips readout) … “4” … “3” “5” … “3”

I can’t make any sense of that. It’s as though just reading from the buffer changes it somehow.

Also note that my expected result does not include a “0”. This is because I’m following your technique, despite not understanding a key part of it: why is the continue-condition in tho for-loop “i < num-1”? Don’t we want the character in the index of “num-1”? If the received message is 4 long, the condition as you wrote it would only have the program print (or, in this case, blink) the characters at indexes 0, 1, and 2, right?

Thanks again for your help,
Aidan

Nah, I was wrong, sorry. Put the -1 away.
Why do you have every where so much and so long delays? And what that have to do?

BR, Jan

The delays are there because I have to do all debugging via the onboard LEDs, so they have to be slow enough to be counted. When I listed “actual” and “expected” results, I was referring to what I saw when the thing blinked out ASCII codes. Do you think that the delays are what’s messing things up? I did actually test that, trying a variant of the program which didn’t do any delays until the message had been read in to a separate array. That didn’t help. It was still a mess: reading massages multiple times, etc.

The delays are really long and slow down everyting. One loop takes 10s + more inside blinkItOut function.

Option one: use another UART, but for that you need another hardware - USB to TTL Covertor (something with FTDI chip or similar, or Arduino UNO is usable or another board ofc)
Option two: much simple, use Echo like it is in example of BufferedSerial

BR, Jan

FWIW, if you use Mbed CE, you can debug the LPC1768 using OpenOCD. You could also debug it on the command line using the config file I put together: mbed-os/lpc1768.cfg at master · mbed-ce/mbed-os · GitHub

Sorry, I may be too much of a novice to understand: do you mean “echo” as in the linux terminal command?

It’s ok :slight_smile:
With the Echo I mean.

PC   > "123"
Mbed < "123"
//Mbed received, process it and send it back to PC
Mbed > "123"
PC   < "123"

You send a string from your PC to your Mbed board. The board read it and send it back to PC over same UART.

BR, Jan

To investigate, here are some examples without any delays. Instead, I use an analog output attached to a multimeter:

This one works fine, supplied with single-digit commands:

if (uint32_t recievedMessageLength = serial_port.read(fromPCBuffer, 4)) {
        recievedNumber = 0;
        recievedNumber += fromPCBuffer[0] - 48;
        aOut = recievedNumber / 10.0;
}

Supplied with a 2-digit command, this one interprets all numbers with a least significant digit of 0-6 as “0” (or 1; this setup confuses those values), numbers with a least significant digit of 7 as “2”, 8 as “3”, and 9 as “4”:

if (uint32_t recievedMessageLength = serial_port.read(fromPCBuffer, 4)) {
        recievedNumber = 0;
        recievedNumber += (fromPCBuffer[0] - 48) * 10;
        recievedNumber += fromPCBuffer[1] - 48;
        aOut = recievedNumber / 10.0;
}

Supplied with a three-digit command, this one did nothing at all:

if (uint32_t recievedMessageLength = serial_port.read(fromPCBuffer, sizeof(fromPCBuffer))) {
        recievedNumber = 0;
        recievedNumber += (fromPCBuffer[0] - 48) * 100;
        recievedNumber += (fromPCBuffer[1] - 48) * 10;
        recievedNumber += fromPCBuffer[2] - 48;
        aOut = recievedNumber / 1000.0;
}

It is still same story your second and third example have fixed positions of array, but that will not work when the read method will process just one char.

if (uint32_t recievedMessageLength = serial_port.read(fromPCBuffer, sizeof(fromPCBuffer))) {
   recievedNumber = 0;
   recievedNumber += (fromPCBuffer[0] - 48) * 100;
   if(recievedMessageLength > 1) recievedNumber += (fromPCBuffer[1] - 48) * 10;
   if(recievedMessageLength > 2) recievedNumber += fromPCBuffer[2] - 48;
   aOut = recievedNumber / 1000.0;
}

If this should be just for debugging then it is Interesting method, but too much complicated from my point of view.
Much simple

if (uint32_t recievedMessageLength = serial_port.read(fromPCBuffer, sizeof(fromPCBuffer))) {
   // ECHO back to PC and then you exactly know what and how your Mbed received
   serial_port.write("Received: ", 9);
   serial_port.write(fromPCBuffer,recievedMessageLength);
}

BR, Jan

I’ve been very dumb: it didn’t occur to me that I can send serial commands from within the MBED Studio. I was trying to come up with other methods to do it, because I thought I was limited by having to have the microcontroller connected to the computer which will ultimately be used; the one with the python program on it.

I’ll report back after I play around using “echo” for debugging.

Thanks again, everybody.

You say, “the read method will process just one char.”

This is confusing to me: read() returns the number of characters in the buffer, right? I’ve seen recievedMessageLength be higher than 1, even though the documentation says it should only return 0 or -1…

Is it like-- it reads in one character at a time into the buffer until the buffer gets read? How does it see that the buffer is read?

Nope, you cut-out my sentence out of context.

That does not mean it will read one char every time, that means it can read from one char to max size of your buffer. One time 1byte, second time 3 and so on.

Documentation of read says. The number of bytes read, 0 at end of file, negative error on failure.
That means:

  • Positive value > 0 is number of readed bytes.
  • Zero is end of file
  • Negative value is an error

BR, Jan

Ah, it looks like I was looking at the documentation for read(uint8_t buffer, int length…),* not read(void buffer, size_t length ).*

Reading your comment carefully, and assuming you meant to say “one time 1 byte, second time 2 and so on,” it sounds like what you’re saying is that read() reads one character farther into the receiving buffer every time, presumably until it hits a new-line character or something, at which point it resets. And that each time, it returns an integer to report how far into the buffer it is. I made the following program to test this understanding. (note: I couldn’t figure out how to send more than one character at a time with MBED Studio’s terminal interface, so I used a combination of it and a program called CuteCom)

if (uint32_t depthIntoBuffer = serial_port.read(fromPCBuffer, sizeof(fromPCBuffer))) {
   workingBuffer[0] = depthIntoBuffer + 48;
   serial_port.write(workingBuffer, 1);
}

The expected behavior, if it received a five-character message*, would be for it to reply with “12345.” Instead, it returns “1” a number of times equal to the length of the message plus one. Presumably the “plus one” is because there’s a new-line character in there.

This seems contrary to what I’ve seen before, where recievedMessageLength (renamed in the above example to “depthIntoBuffer” usually matches the size of the character-bundle sent. In several other tests, including your above-supplied suggestion to read in three-digit numbers with some "if"s, I can’t get it to act as though the integer returned by read() is ever more than 1. Strange because I definitely got it to confirm counts above one before. Any ideas?

*What’s the proper terminology here? Do characters have any sort of grouping in UART? In other words, if I send “hello world” as one string from one buffer with one command, is that received any differently than if I had sent each character 100ms apart?

Here is maybe a clearer version of the previous example:

    uint32_t depthIntoBuffer;
    while (1) {
        depthIntoBuffer = serial_port.read(fromPCBuffer, sizeof(fromPCBuffer));
        if (depthIntoBuffer) {
            workingBuffer[0] = depthIntoBuffer + 48;
        }
        serial_port.write(workingBuffer, depthIntoBuffer);
    }

Expected output, for input “12345”:
“12{-}3{-}{-}4{-}{-}{-}5{-}{-}{-}{-}6{-}{-}{-}{-}{-}”
({-} being whatever exists in the unwritten buffer slots; six rounds because remember that newline character at the end)
Actual output:
“111111”

Another data point: if I have nothing in my main loop except this:

            serial_port.write("hi", 3);

– it works as expected, generating a stream of "hi"s. But if I make it this:

            depthIntoBuffer = serial_port.read(fromPCBuffer, sizeof(fromPCBuffer));
            serial_port.write("hi", 3);

– suddenly the printing is conditional. It only prints upon receiving a message, and then only as many times as the number of characters received plus one.