TL;DR: Is there a way to clue in mbed test --unittests that there are unit tests outside of the mbed-os directory? Because I don’t want to put my own (non-OS-related) code in there.
I’m writing a bit of code that may eventually become a library, but the only really straightforward way to even begin to write it is to start by writing tests.
I’d like to make my code a good citizen in the mbed-os ecosystem, so my first thought was to search for something like “unit test mbed” but 100% of the links are how to write unit tests for the guts of the Mbed OS itself. So my assumption is that we’re intended to write and run our tests entirely outside of the mbed CLI. Is that correct?
If so, can someone point me to a tutorial on this? Or point me to an mbed library with good unit tests that I can use as a reference? My C++/embedded experience predates wide adoption of TDD so I’m kind of out of my depth here.
Personally, I use TDD with Google Test and gMock. I use CMake to set them up and run them, I don’t use the mbed-cli at all for that.
In the past I tried copying the mbed unit tests to my project, removing all the tests and using only mine but it was tool complicated to integrate in the mbed workflow.
Starting from scratch with Google Test was way easier. Unit tests are not supposed to run on hardware, so you won’t have any issues.
Was also my question here, but unfortunately it does not seem to be possible. The recent CMake integration does not support this use case yet. Use an external build tool such as CMake to include the available stubs in the UNITTESTS directory and try building on that.
@ladislas, I’m also working on setting up a unit test system for my code. I looked into Google Mock but as far as I’ve seen it isn’t really suitable for “C-style” C++ code, where the dependencies are not implementations of abstract classes, which is usually the case in Mbed.
I will probably use CppUTest and its accompanying CppUMock, which is mostly aimed at C but written in C++. Therefore IMHO it fits better to lower level code. How is your experience with Google Mock so far?
You’re right, you basically always need to have a base class that you can mock. you can have a regular class with virtual methods and a default implementation, but in my experience, it makes it clearer to just have a base class in that case.
For the C-style C++, we usually add a shallow wrapper class around the functionality. One recent example: our product is based on the STM32F769 and needs to drive a MIPI-DSI LCD-TFT screen. We started with the ST’s examples, got them running and then we rebuilt the system to fit our needs. To unit test all that we put a class wrapper around ST’s HAL functions that just calls the underlying HAL functions but that we can very easily mock wit Google Mock. We then use dependency injection to pass our hal object to classes that need it.
The code is not public yet but we are planning on releasing it soon, I’ll share it here when it’s done.
I’ve hear a lot of good things about CppUTest/CppUMock, but I haven’t used it yet. When unit testing code linked to mbed using underlying C API, we usually just provide a stub, for example for void gpio_init_out(gpio_t *gpio, PinName pin) to make the compiler happy and a spy if we need to look into what is happening.
In the end I think it really is just a matter of taste, Google Test/Mock works great for us. If needed we could also use CppUMock, they work well together I think. I found the documentation less clear than Google Test which works great with CMake.
The mock functionalities of Google Mock are also very powerful and you can expand the system to suit your needs/types/structs, etc.
If you haven’t read it yet, I recommend Test Driven Development for Embedded C, a real page turner (no kidding) – the guy uses CppUTest/CppUMock.
Just finished reading that one Wonderful book indeed.
I found the documentation less clear than Google Test which works great with CMake.
The documentation of CppUTest is a little bit outdated, you’re right. Some new stuff are only documented in their GitHub repo. At the bottom there is a snippet for CMake to automatically fetch and configure it.
For the C-style C++, we usually add a shallow wrapper class around the functionality.
Adding wrappers just for testing feels like a lot of hassle for me. Are there other benefits of using this approach?
it might be a bit artificial but the separation of concerns this way makes TDD and unit testing easier for the team. but I’m not saying it’s the only way.
for the LCD stuff, we also use google mock for “specification” – we know that the DSI command must be called 101 times for the LCD to work correctly, the functional test shows us that.
If you modify the driver and remove 1 DSI write call, the tests will complain. Doesn’t mean the screen won’t work (we had the experience) but we use it as a warning that something has been changed and that it might break some other unexpected things. It also means that functional tests must be run thoroughly in this case. If all checks up green in the end, the unit test specs can be changed to expect only 100 calls.
I’ve been reading this thread and I’ve found it very interesting. Beyond the framework, I have been wondering about some things.
As @ladislas said, low-level stubs can be very interesting when you want to test new peripherals or libraries which work against mbed OS drivers such as I2C, SPI, and so on.
But, what is your approach when you are trying to test high level code? I mean, making stubs for connectivity APIs seems to be a difficult work.
As an example, if I want to test an AWS connector, how would you do? Writing low-level stubs seems not to be the best option here…
I think I will create a new thread to discuss this.
I don’t know if you are already working with out-of-tree unit testing over Mbed OS 6, but I have just configured an example project which runs unit tests using mbed stubs and fakes but without including it in Mbed OS folder. This allows me to include tests in each library without modying mbed references.
As @ladislas said in this comment, it’s a good point copying Mbed OS configuration and modifying it to get out of the tree.
Of course, here you have unit-test-mbed-os-example. I think code structure and comments are enough, I have cleaned CMake files and I think project is very simple.
Wonderful book indeed.
