Hitchhiker's Guide to Printf in Mbed 6

Since Mbed 6 was released, there’s been a lot of confusion about printf. This is understandable, as the Mbed developers removed the previous method of printing stuff (Serial) that people have been using for years. What’s more, the developers have been very tight-lipped about what exactly to use as an equivalent. There are several possible replacements (none of which is a 1:1 port), and it’s often unclear which to use.

I have worked with Mbed OS for years, and did extensive research and testing into all of the current methods, analyzing how to implement them correctly, how they perform, and what they can’t do. In this guide I will present my results and my recommendations on which to use.

TL;DR

If you just want to print things to the serial port normally and performantly in Mbed 6, just put this code into mbed_app.json:

{
    "target_overrides": {
        "*": {
            "platform.stdio-baud-rate": 115200,
            "platform.stdio-buffered-serial": 1
        }
    }
}

You no longer need to create an object to call printf() on, like Serial in Mbed 5. You can just use the default printf() function and it will work fine. You can select whatever baudrate you need by changing the baud-rate option, and this baudrate will be used from bootup.

Also, if you need ISO C compliant floating point formatting in printf(), add this line into the target overrides block:

"target.printf_lib": "std"

Otherwise, by default, floating point numbers cannot be printed at all.

The Whole Story

Printf() in mbed is a complicated story, and we’ll need to go through a bit of background before I can explain how the current methods work.

Mbed File Handles

Mbed OS interoperates with the C library’s IO functions in a somewhat complicated way. Essentially, Mbed OS provides the abstraction of file descriptors. These are specific integers that represent I/O streams that can be read to and/or written from – just like in Unix C. These include the standard streams: stdin, stdout, and stderr (which have file descriptors 0, 1, and 2 respectively). You can also create a file descriptor for any object that extends FileHandle (such as BufferedSerial) using the mbed::bind_to_fd() function.

In contrast, the C library provides struct FILE. This is essentially an opaque struct that provides information about an I/O stream to the C library – for example, it might contain function pointers for writing to it, a pointer to the buffer, etc. However, all that you as a user can do with it is pass it to functions that take a FILE *. This includes things like fwrite() and fprintf().

To work with the C library, Mbed OS provides a key function:

std::FILE *fdopen(mbed::FileHandle *fh, const char *mode)

This is like the Unix fdopen(), except that it produces a C FILE * for a FileHandle object. This allows you to use the FileHandle with all C IO functions that take a FILE *. Since FileHandle lacks printf() itself, using FILE * plus fprintf() is the recommended way to call printf() on a FileHandle in Mbed 6.

Overriding the Console

Thankfully, the Mbed devs realized that carrying around FILE* variables for every print statement would get annoying. So, they created a way to override the behavior of the default printf(). Since printf() prints to stdout, it is equivalent to writing to file descriptor 1 (stdout). Similarly, scanf() is equivalent to reading from file descriptor 0 (stdin). Mbed allows you to define a specific function to configure the stream that is referred to by those file descriptors. This looks like:

namespace mbed
{
	FileHandle *mbed_override_console(int fd)
	{
		return /* some file handle pointer */;
	}
}

Defining this function in your code overrides a weak symbol (it’s complicated…) in Mbed OS, and your implementation will be used instead of the default one. This function will get called three times at bootup, once each with fd equal to 0, 1, and 2. The FileHandle that you return will then be connected to that file descriptor.

Note: 99% of the time you will want to return the same FileHandle for all three, as FileHandles like BufferedSerial support both input and output in the same object.

Direct and Buffered Serial

If you don’t provide mbed_override_console(), Mbed OS provides code equivalent to this:

namespace mbed
{
	FileHandle *mbed_override_console(int fd)
	{
	    static DirectSerial console(USBTX, USBRX, MBED_CONF_PLATFORM_STDIO_BAUD_RATE);
		return &console;
	}
}

DirectSerial is an undocumented Mbed OS class that works similarly to UnbufferedSerial. Since it’s the default stream, by default printfs won’t be buffered – they will block until the entire string has been transmitted. If you try to print a 100 character line at 115200 baud, your code is going to block for 8.7 ms until every single character has been transmitted out of the device. This is, understandably, not desirable in most situations.
This is why, in Mbed 6, ARM created the BufferedSerial class. This class (similar to the popular MODSERIAL community library for Mbed 5) keeps a software buffer of characters to be transmitted. It then uses an interrupt to detect when the serial port peripheral in the CPU can accept more data, then gradually feeds in characters in the background. This means, effectively, that you can have the serial port send characters at a low rate in the background while your code executes in the forground. Much better than just waiting, right?

To use BufferedSerial instead of DirectSerial, you need to add the following line to your mbed_app.json:

"platform.stdio-buffered-serial": 1

This will cause the console override to be declared equivalent to:

namespace mbed
{
	FileHandle *mbed_override_console(int fd)
	{
	    static BufferedSerial console(USBTX, USBRX, MBED_CONF_PLATFORM_STDIO_BAUD_RATE);
		return &console;
	}
}

In most cases, this is what you will want for printf() support in your own apps.

Note that the baud rate can be configured by this setting in mbed_app.json:

"platform.stdio-baud-rate": 115200

The default, for some ungodly reason, is 9600 baud, so you will probably want to increase this to escape dialup speeds.

Porting Code that Uses Serial

In Mbed 5, the Serial class acted as a one-stop-shop for all serial port communication needs. But when porting to Mbed 6, you will need to divide your usage of the Serial class into two categories:

  • Usage of Serial to provide standard output (e.g. pc.printf("Hello World\n"))
  • Usage of Serial to communicate with external devices (GPSs, modems, radios, etc.)

If Serial is being used as standard output, just change the calls to regular printfs (and scanfs), and use the JSON method discussed above. Bam, done.

If Serial is being used for I/O with a device, then you will need to replace it with calls to UnbufferedSerial or BufferedSerial. UnbufferedSerial is a direct equivalent to old Serial and is not buffered, but, as discussed above, BufferedSerial can provide better performance in many situations. For most devices, BufferedSerial in blocking mode should be a straight upgrade from Serial with few gotchas.

For example, if you only need to read and write fixed numbers of bytes, you can simply use (Un)BufferedSerial Directly:

BufferedSerial myRadio(PIN_RADIO_TX, PIN_RADIO_RX);

char* data = "TEST";
myRadio.write(data, strlen(data));

However, if you need more advanced IO functions like printf, you have to use the FILE * API:

FILE* myRadioFile = fdopen(&myRadio, "r+");

fprintf(myRadioFile, "Test #%d\n", 2);

NOTE: A while ago I released the SerialStream library, which provides a Stream subclass that sends data to a BufferedSerial class. This is also an option for porting code, but the Mbed developers have indicated that the Stream class as a whole is on the chopping block for deprecation so it’s probably best to avoid too much reliance on this system.

Blocking vs Non-Blocking

One final topic to cover: In Mbed 6, the developers introduced the extremely useful option for serial ports to be non-blocking. Consider the problem of checking if a char has been received on the serial port. If one was received, we want to save it to a buffer, but if not, we want to continue with the program. In Mbed 5, this was very difficult: reading from Serial was blocking, so getc() would wait forever until someone typed a character. So, the only way to implement this was using interrupts on Serial, e.g. calling pc.attach(<...>, Serial::RxIrq) and then reading chars in the interrupt.
In Mbed 6, this functionality can be implemented much more easily using BufferedSerial::set_blocking() (which affects both the class functions and the FILE * API).

myRadio.set_blocking(false);

while(true)
{
    <do other stuff...>
    
    char received;
    while(myRadio.read(&received, 1) > 0)
    {
        addToBuffer(received);
    }
}

BufferedSerial::readable() is also a convenient way of doing the same thing.

while(true)
{
    <do other stuff...>
    
    char received;
    while(myRadio.readable())
    {
        myRadio.read(&received, 1); // guaranteed to return because the port is readable
        addToBuffer(received);
    }
}

Note: both of these methods only work for BufferedSerial, not UnbufferedSerial.

A Note about Performance

Before publishing this guide, I ran a benchmark of all 3 methods of printing characters (console override, FILE *, and SerialStream). Using BufferedSerial, I tested both long (>100 chars) and short (1 char) strings, as well as medium (115200) and fast (921600) baudrates. I found that there was hardly any significant difference between the three methods. The console override was a few percent faster than the others for one character strings, but the difference was barely noticable, and all three methods were able to reach the maximum transfer rate for their baudrates.

Anyway, hopefully this guide provides all you ever wanted (and didn’t want) to know about serial ports in Mbed 6. Let me know in the comments if there are any questions.

24 Likes

Great, this should be integrated somehow in the offical Mbed docs. The API documentation is just a reference with some examples, but some insights are missing.

3 Likes

That’s a great post! Thanks a lot @MultipleMonomials – I’ll share it to our team and review our use of serial :slight_smile:

For the Blocking vs Non-Blocking part, I would highlight the fact that you are talking about BufferedSerial at the beginning, put it in bold and add the initialization in the code block so we can have the full picture.

I would love at the end a full working example with initialization, write, fprintf and non-blocking stuff so that once again we could have the full picture, easy to copy/paste into a test program to play with :slight_smile:

1 Like

Great post @MultipleMonomials!

If I’m not mistaken, the original author was using TeraTerm which has a default baud rate of 9600. Even if it it was many years ago (more than a decade ?) it sticked and Mbed OS default baud rate still is 9600 today.

1 Like

Should we change it?

2 Likes

Nice post, thanks @MultipleMonomials.

I would also add mbed_trace and redirection to SWO as they are usually even better practice than pure printf and UART respectively.

3 Likes

Yes this! I’d like to know more about that :slight_smile:

1 Like

A nice Mbed SWO viewer library along with info is available here. It can be used also with the STM Studio viewer.

1 Like

Add this information to the official Mbed 6 doc’s ?
This guy’s done all the hard work, what are you waiting for?
115200 should be default now, cant remember the last time I used 9600, but tbh easy enough to specify in the json config file.

1 Like

This is not required. See mbed-os/mbed_retarget.h at 96e19afdd196c6c99edd58fddd44e2c691cdca2f · ARMmbed/mbed-os · GitHub

Another inadequately documented feature of Mbed.

3 Likes

Thank you for the info. It seems promising but it requires a target with ITM defined in the targets.json. Unfortunately there are just few of them at the moment (mbed-os-6.9.0):

EFM32GG_STK3700
EFM32GG11_STK3701
MCU_NRF52832 (some children remove ITM)
MCU_NRF52840 (some children remove ITM)

The missing piece seems to be a target specific implementation of the following function:

/**
 * @brief      Target specific initialization function.
 *             This function is responsible for initializing and configuring
 *             the debug clock for the ITM and setting up the SWO pin for
 *             debug output.
 *
 *             The only Cortex-M register that should be modified is the clock
 *             prescaler in TPI->ACPR.
 *
 *             The generic mbed_itm_init initialization function will setup:
 *
 *                  ITM->LAR
 *                  ITM->TPR
 *                  ITM->TCR
 *                  ITM->TER
 *                  TPI->SPPR
 *                  TPI->FFCR
 *                  DWT->CTRL
 *
 *             for SWO output on stimulus port 0.
 */
void itm_init(void);
1 Like

That’s correct. When working on a custom target it is not a big deal to add the ITM init code. On dev boards you usually have a UART output available so it might not be worth the effort.

1 Like

Great post ! Many thanks. I will pin this at the top to make it easy to find.

2 Likes

It seems like there’s a problem with scanf() if I declare a BufferedSerial object using USBTX, USBRX (but without redirecting the console). An example code below:

    BufferedSerial serial_port(USBTX, USBRX, 9600);

    int main() {
      char buffer[1024];
      printf("helloworld\n");
      while(1) {
        scanf("%s", buffer);
        printf("got:\n");
        printf("%s\n", buffer);
}

When I send a short string over serial, scanf() does not pick up anything. If I send a really long string, scanf() pick up the last few characters and printf() back. But after a few times sending, scanf() start acting like normal again (pickup whole string and printf() back whole string).
If I have to guess, the BufferedSerial object I defined is swallowing up the serial buffer to fill up its own circular buffer, preventing the scanf()'s DirecSerial object to get any. But once my own BufferedSerial object circular buffer filled up, somehow it relax the priority and let DirectSerial have the priority.

Adding override make everything works normally:
FileHandle *mbed::mbed_override_console(int fd) {
return &serial_port;
}

1 Like

Yeah your analysis sounds about right. Having two serial port objects active seems to be a recipe for trouble.

1 Like

Useful thread, thanks.
Simple question about buffer sizes,
/** Software serial buffers
* By default buffer size is 256 for TX and 256 for RX. Configurable
* through mbed_app.json
*/

What exactly do I need to insert into mbed_app.json ?

1 Like

Wait you mean you didn’t get your psychic powers? They were supposed to be included with each download of Mbed OS, that’s how you’re supposed to find these config option names. /s

Looking through the source code (mbed-os/drivers/mbed_lib.json), the option names seem to be drivers.uart-serial-txbuf-size and drivers.uart-serial-rxbuf-size.

5 Likes

For those who don’t work with Mbed frequently it could take some effort and time (linked with frustration) to figure out which parameters can be configured and what to put into the “mbed_app.json”.

The configuration system is documented at The configuration system - Program setup | Mbed OS 6 Documentation but the info below might help.

  1. Search the actual “mbed-os” (including subfolders) for “mbed_lib.json” files (189 in Mbed OS 6.9.0).

  2. Filter out those related to “test” (139 remain in Mbed OS 6.9.0).

  3. If you are looking for something very specific (target, device etc.) you can easily narrow the list.

  4. For “common options” it’s worth to check:

    mbed-os/drivers/mbed_lib.json
    mbed-os/events/mbed_lib.json
    mbed-os/platform/mbed_lib.json

  5. To set a parameter listed in a “mbed_lib.json” file it is important to know also the name of the associated library. That’s indicated as first field in the “mbed_lib.json” file. In your “mbed_app.json” file first put the library’s name followed by a dot and the parameter’s name. The new value you’d like to use goes after a colon.

    For example, to set the “uart-serial-txbuf-size” which is listed in the “mbed-os/drivers/mbed_lib.json”
    and the “stdio-baud-rate” listed in the “mbed-os/platform/mbed_lib.json” file
    put the following into your “mbed_app.json” file located in the root directory of your project:

{
    "target_overrides": {
        "*": {
            "drivers.uart-serial-txbuf-size": 128,
            "platform.stdio-baud-rate": 115200
        }
    }
}

You can add any number of parameters. The order doesn’t matter. However, make sure they are separated by a comma.
There is also indicated a “help” field for each parameter in the “mbed_lib.json” which provides a description and a “value” showing it’s default value.

You can verify your settings after compilation by checking the “mbed_config.h” file. It is automatically generated and saved to the root directory of your project. Parameters are defined as upper case macros preceded by “MBED_CONF_”.

3 Likes

Thanks Jamie, thanks Zoltan.

1 Like