MbedOS Programmatic Backtrace of Call Stack When Crash Occurs

Mbed OS
1

As a very useful enhancement, could we have a way of programmatically generating a backtrace when a crash occurs and outputting that on the serial? The ability to turn that feature on or off could even be controlled by one of the configuration options. Also the depth of the backtrace too.

execinfo. h’s backtrace() and backtrace_symbols() could be used to accomplish this. Alternatively, there is libunwind which could also be leveraged to implement this. It should be very straightforward. Thanks.

I tried implementing this myself by adding a hook into __verbose_terminate_handler() but unfortunately, execinfo.h is not included in MbedOS even though, surprisingly, MbedTLS conditionally includes that file. Anyway, I will attach my attempt as an addendum to this post and perhaps it can help expedite this enhancement being implemented. Thanks again.

2

Here is my attempt below:


inline void DisplayBacktrace()
{
    const size_t MAX_DEPTH = 30;
    size_t stack_depth;
    void *stack_addrs[MAX_DEPTH];
    char **stack_strings;
    std::ostringstream oss; 
    
    oss << "\nBacktrace Crash Call Stack Using GLIBC Utilities:\n" << std::endl;

    stack_depth = backtrace(stack_addrs, MAX_DEPTH);
    
    // Restore the faulting address:
    #if defined(TOOLCHAIN_GCC_ARM)
        if (mbed_fault_context)
        {
            stack_addrs[2] = mbed_fault_context->PC_reg;
        }
    #else
        #error "Unsupported platform."
    #endif
    
    stack_strings = backtrace_symbols(stack_addrs, stack_depth);

    for (size_t i = 0; i < stack_depth; i++)
    {
        // TBD Nuertey Odzeyem; A guess for now. Enhance later due
        // to in-depth research as template names will be much wider.
        size_t sz = 256; 
        char *function = (char*)malloc(sz);
        char *begin = 0, *end = 0;

        // Find the parentheses and address offset surrounding the
        // mangled name.
        for (char *j = stack_strings[i]; *j; ++j)
        {
            if (*j == '(')
            {
                begin = j;
            }
            else if (*j == '+')
            {
                end = j;
            }
        }

        if (begin && end)
        {
            *begin++ = 0;
            std::string full_func_name(end);
            *end = 0; // Found our mangled name, now in [begin, end)

            int status;
            
            // New ABI-mandated entry point in the C++ runtime library for demangling.
            // 
            // char* abi::__cxa_demangle(const char *   mangled_name,
            //                           char *         output_buffer,
            //                           size_t *       length,
            //                           int *          status   
            //                          )   
            char *ret = abi::__cxa_demangle(begin, function, &sz, &status);

            if (ret)
            {
                // Return value may be a realloc() of the input.
                function = ret;
            }
            else
            {
                // Demangling failed, just pretend it's a C function with no args.
                std::strncpy(function, begin, sz);
                std::strncat(function, "()", sz);
                function[sz-1] = 0;
            }

            oss << "    " << stack_strings[i] << ":" << function 
                << " {" << full_func_name << "}" << std::endl;
        }
        else
        {
            // Did not find the mangled name, just print the whole line.
            oss << "    " << stack_strings[i] << std::endl;
        }
        free(function);
    }

    free(stack_strings); // malloc()ed by backtrace_symbols
    
    printf("\n\n%s\n\n", oss.str().c_str());
}

#if defined(TOOLCHAIN_GCC)
/* prevents the exception handling name demangling code getting pulled in */
#include "mbed_error.h"
namespace __gnu_cxx 
{
    void __verbose_terminate_handler()
    {
        DisplayBacktrace();
        
        printf("%s called from %p\n", __func__, MBED_CALLER_ADDR());
        MBED_ERROR1(MBED_MAKE_ERROR(MBED_MODULE_PLATFORM, MBED_ERROR_CODE_CLIB_EXCEPTION), "Exception", 0);
    }
}
3

If you glance over my code, you will realize that it is all practically there; all we now need is to have execinfo.h in the code base. So perhaps the task simply devolves into porting execinfo.h into MbedOS. Thanks.

4

Interesting issue, but AFAIK this is in general not trivial for Cortex-M microcontrollers, let alone Mbed. A simpler method would be to generate a core dump and analyze it with respect to the ELF file at the host side. There are projects which do this, but of course it would be much easier to have at least a guide to integrate it into Mbed.

5

Indeed @boraozgen, it is not as straightforward as other platforms, say for example, Linux. I wonder if Cortex-M microcontrollers can be ‘helped along in that direction’ by leveraging intrinsics such as the below which MbedOS is already leveraging. And on that note there are also other projects that I came across and researched that are trying to accomplish similar goals for embedded platforms. I will share their links too in the addendum.

Regards, and thanks for responding.
Nuertey

Excerpted from “./mbed-os/platform/include/platform/mbed_toolchain.h”


#define MBED_CALLER_ADDR() __builtin_extract_return_addr(__builtin_return_address(0))
6

@boraozgen here are the relevant links below. I think that several great folks have been taking a stab at this “particular interesting issue” for some time now. Even the last link’s author’s code (and comments) on Stackoverflow is very instructive. I will excerpt his code in the next addendum but his whole comment (on ARM processors in general) needs to be read hence being linked by me; he and his team apparently leveraged the intrinsics, see? I think the intrinsics have all the information that we need, and the Mbed OS team ought to know this better than us, so I am trusting that they have insight into the issue better than us. Delving even within their backlog of closed issues even, I see that other folks have also attempted to request this same feature :slight_smile: … Great minds thinking (and needing certain useful features in such a great embedded OS such as Mbed OS) after all :).

7 
extern void * __libc_stack_end;

struct backtrace_frame_t
{
    void * fp;
    void * sp;
    void * lr;
    void * pc;
};

int backtrace(void ** array, int size)
{
    void * top_frame_p;
    void * current_frame_p;
    struct backtrace_frame_t * frame_p;
    int frame_count;

    top_frame_p = __builtin_frame_address(0);
    current_frame_p = top_frame_p;
    frame_p = (struct backtrace_frame_t*)((void**)(current_frame_p)-3);
    frame_count = 0;

    if (__builtin_return_address(0) != frame_p->lr)
    {
        fprintf(stderr, "backtrace error: __builtin_return_address(0) != frame_p->lr\n");
        return frame_count;
    }

    if (current_frame_p != NULL
        && current_frame_p > (void*)&frame_count
        && current_frame_p < __libc_stack_end)
    {
        while (frame_count < size
               && current_frame_p != NULL
               && current_frame_p > (void*)&frame_count
               && current_frame_p < __libc_stack_end)
        {
            frame_p = (struct backtrace_frame_t*)((void**)(current_frame_p)-3);
            array[frame_count] = frame_p->lr;
            frame_count++;
            current_frame_p = frame_p->fp;
        }
    }

    return frame_count;
}