CLI1 vs CLI2 build size + time

For those who a curious please find below the build statistics of a simple Mbed program built with Mbed CLI1 versus Mbed CLI2.

Program source code:

/* mbed Microcontroller Library
 * Copyright (c) 2021 ARM Limited
 * SPDX-License-Identifier: Apache-2.0
 */

#include "mbed.h"

int main()
{
    printf("Hello, Mbed!\n");
    return 0;
}
================================================================
Host computer
Processor:        Intel® Pentium(R) CPU G2130 @ 3.20GHz × 2 
OS type:          64-bit
Memory:           15.5 GiB
Operating system: Linux (Ubuntu 18.04.6 LTS)

================================================================
Target board: LPC1768
Mbed OS:      6.15.0

================================================================

CLI1

Build system: GNU Make 4.1
Compiler: gcc-arm-none-eabi-10.3-2021.07

RESULTS:

| Module           |         .text |       .data |        .bss |
|------------------|---------------|-------------|-------------|
| [fill]           |     614(+614) |       0(+0) |     19(+19) |
| [lib]/c.a        |   7268(+7268) | 2108(+2108) |     58(+58) |
| [lib]/gcc.a      |     776(+776) |       0(+0) |       0(+0) |
| [lib]/misc       |     188(+188) |       4(+4) |     28(+28) |
| main.o           |       16(+16) |       0(+0) |       0(+0) |
| mbed-os/cmsis    |   6306(+6306) |   168(+168) | 5953(+5953) |
| mbed-os/drivers  |       58(+58) |       0(+0) |       0(+0) |
| mbed-os/hal      |     470(+470) |       0(+0) |       2(+2) |
| mbed-os/platform |   3680(+3680) |   260(+260) |   348(+348) |
| mbed-os/targets  |   2248(+2248) |       4(+4) |   240(+240) |
| Subtotals        | 21624(+21624) | 2544(+2544) | 6648(+6648) |
Total Static RAM memory (data + bss): 9192(+9192) bytes
Total Flash memory (text + data): 24168(+24168) bytes

Build times:

real  0m27.727s
user  0m40.369s
sys	  0m4.686s

================================================================

CLI2

Build system: CMake 3.20.6 + Ninja 1.8.2
Compiler: gcc-arm-none-eabi-10.3-2021.07

RESULTS:

| Module           |         .text |       .data |        .bss |
|------------------|---------------|-------------|-------------|
| [fill]           |     616(+616) |       0(+0) |     19(+19) |
| [lib]/c.a        |   7268(+7268) | 2108(+2108) |     58(+58) |
| [lib]/gcc.a      |     776(+776) |       0(+0) |       0(+0) |
| [lib]/misc       |     188(+188) |       4(+4) |     28(+28) |
| main.cpp.obj     |       16(+16) |       0(+0) |       0(+0) |
| mbed-os/cmsis    |   6438(+6438) |   168(+168) | 5953(+5953) |
| mbed-os/drivers  |       58(+58) |       0(+0) |       0(+0) |
| mbed-os/hal      |     578(+578) |       0(+0) |       2(+2) |
| mbed-os/platform |   4282(+4282) |   260(+260) |   348(+348) |
| mbed-os/targets  |   2388(+2388) |       4(+4) |   240(+240) |
| Subtotals        | 22608(+22608) | 2544(+2544) | 6648(+6648) |
Total Static RAM memory (data + bss): 9192(+9192) bytes
Total Flash memory (text + data): 25152(+25152) bytes

Build times:

real  0m23.339s
user  0m34.688s
sys	  0m3.444s
1 Like

thanks, not so much faster than my first impression was, but maybe it will be more difference with larger projects. And compiling under windows with NTFS is much slower, in WSL2 its nearly as fast as with native linux.
Its also interesting that the codesize with CLI2 is larger, it should use the same compiler flags.

In my experience it was most of the time the opposite, see mbed cli vs mbed tools bin size comparision · Issue #13974 · ARMmbed/mbed-os · GitHub

@hudakz for CLI1 are you using .mbedignore? If so, can you try with an empty one?

Again, CMake and static libs will make a lot of sense for real project using multiple targets, bootloaders, etc., not just for blinky examples.

We have around 20 spikes + bootloader + main firmware and once mbed-os is built, adding a new spikes only adds one or two seconds to the compilation time.

Those results also improve a lot when using ccache which I highly recommend.

for CLI1 are you using .mbedignore ?

Yes, I use this .mbedignore file in each CLI1 project. Actually, what CMake does in CLI2 (selects which files shall be compiled) the .mbedignore file can do in CLI1. The main difference is that while CMake applies a distributed approach (directory wise CMakeLists.txt files) the .mbedignore file enables a sort of centralized configuration concentrated in one file. Of course CMake offers much more but it’s also more complex than the .mbedignore file.

But my main interest was to compare the performance of the Ninja build system with the GNU make.

If so, can you try with an empty one?

Sure. But in this case the number of uselessly compiled files (files which are not used by the linker to create the final binary file) is much higher in CLI1 than in CLI2:

| Module           |         .text |       .data |        .bss |
|------------------|---------------|-------------|-------------|
| [fill]           |     614(+614) |       0(+0) |     19(+19) |
| [lib]/c.a        |   7268(+7268) | 2108(+2108) |     58(+58) |
| [lib]/gcc.a      |     776(+776) |       0(+0) |       0(+0) |
| [lib]/misc       |     188(+188) |       4(+4) |     28(+28) |
| main.o           |       16(+16) |       0(+0) |       0(+0) |
| mbed-os/cmsis    |   6306(+6306) |   168(+168) | 5953(+5953) |
| mbed-os/drivers  |       70(+70) |       0(+0) |       0(+0) |
| mbed-os/hal      |     470(+470) |       0(+0) |       2(+2) |
| mbed-os/platform |   3668(+3668) |   260(+260) |   348(+348) |
| mbed-os/targets  |   2248(+2248) |       4(+4) |   240(+240) |
| Subtotals        | 21624(+21624) | 2544(+2544) | 6648(+6648) |
Total Static RAM memory (data + bss): 9192(+9192) bytes
Total Flash memory (text + data): 24168(+24168) bytes

Build times:

real	2m44.892s
user	4m15.217s
sys	    0m52.540s

yes, with mbed2 I was also happy with the simple build system. It was easy to use, just copy the files to the project and no hustle with include path. But with the large growth of Mbed, it wasted a lot of time with compiling unused stuff. mbedignore was also no perfect solution, the depencies made a lot of trial and error necessary. I think you spent a lot of time to create your sets of ignorefiles :slight_smile:
Then you want to use network or other components and start again fixing dependencies. So cmake is a big milestone for Mbed. For small libraries, the configuration is really manageable,

The main difference is the way they work:

  • .mbedignore prevents files from being compiled by ignoring them
  • CMake lets you choose the files you need to compile by choosing them, packaging them into library targets and then choosing to link against a target or not

There is no logic behing .mbedignore, you just ignore stuff.

There is logic with cmake in how you architecture your project, how you modularize your code, etc.

That’s why the two approaches are vastly different.

yes, and that made it necessary to define and clean up dependencies in mbed-os repo, a great side effect :slight_smile:

and look how Espressifs ESP evolved, you can add components by picking them from a list and cofigure items easily. Hope Mbed gets this also managed.

I tried to use the GNU make build system rather than ninja. CMake was used to generate the build environment and manage the build process with CLI2. It turned out to be working too:

...
cmake -S . -B cmake_build/LPC1768/develop/GCC_ARM -G "Unix Makefiles"
...

The results are below:

| Module           |         .text |       .data |        .bss |
|------------------|---------------|-------------|-------------|
| [fill]           |     616(+616) |       0(+0) |     19(+19) |
| [lib]/c.a        |   7268(+7268) | 2108(+2108) |     58(+58) |
| [lib]/gcc.a      |     776(+776) |       0(+0) |       0(+0) |
| [lib]/misc       |     188(+188) |       4(+4) |     28(+28) |
| main.cpp.obj     |       16(+16) |       0(+0) |       0(+0) |
| mbed-os/cmsis    |   6438(+6438) |   168(+168) | 5953(+5953) |
| mbed-os/drivers  |       58(+58) |       0(+0) |       0(+0) |
| mbed-os/hal      |     578(+578) |       0(+0) |       2(+2) |
| mbed-os/platform |   4282(+4282) |   260(+260) |   348(+348) |
| mbed-os/targets  |   2388(+2388) |       4(+4) |   240(+240) |
| Subtotals        | 22608(+22608) | 2544(+2544) | 6648(+6648) |
Total Static RAM memory (data + bss): 9192(+9192) bytes
Total Flash memory (text + data): 25152(+25152) bytes

real	0m41.714s
user	0m34.633s
sys	    0m4.145s
1 Like