Hi guys,
I want to integrate executorch into Mbed OS to run a pre-trained PyTorch model. The libraries from executorch are .a-files. Where do I have to incorporate those files, or how would you integrate the executor_runner (How do you get executorch to run within Mbed OS? · Issue #2226 · pytorch/executorch · GitHub) into Mbed OS?
Cheers,
Christoph
If you use CMake (either via Mbed CLI 2 or via Mbed CE), you can set them as imported libraries:
add_library(MyImportedModel STATIC IMPORTED)
set_property(TARGET MyImportedModel PROPERTY IMPORTED_LOCATION ${CMAKE_SOURCE_DIR}/path/to/MyImportedModel.a)
Then later you can link it to your executable
target_link_libraries(MyApplication MyImportedModel)
However, with this approach, you must be careful that the .a file is built with the correct architecture flags for the CPU you are targeting. It’s pretty easy to get it wrong and end up with code that won’t run on your device.
Looking at the github, I see you are running into an error with a missing sys/mman.h. Code that uses that header will be a non starter to run on embedded devices, because mmap is a feature of full size OSs with processes such as Linux. Before it can work on Mbed, the code will need to be modified to remove usage of mmap.
Thank you very much for your quick response. I will try the first proposal (Mbed CLI2) today and reply if I have reached some results.
@MultipleMonomials In the meantime I compiled the libraries and added the dir exeuctor_runner (executorch/examples/arm/executor_runner at main · pytorch/executorch · GitHub) to mbed os. Additionally I added
add_subdirectory(executor_runner)
in the root CMakeLists.txt of mbed. Then I executed the following commands in the arm folder:
mbed-tools configure -m NUCLEO_WB55RG -t GCC_ARM
and
cmake \
-S . \
-B cmake_build/NUCLEO_WB55RG/develop/GCC_ARM \
-DET_PTE_FILE_PATH:PATH=/home/chris/et-tut-1/executorch/softmax.pte \
-DCMAKE_TOOLCHAIN_FILE=/home/chris/et-tut-1/executorch/examples/arm/ethos-u-setup/arm-none-eabi-gcc.cmake \
-DTARGET_CPU=cortex-m4 \
-DET_DIR_PATH:PATH=/home/chris/et-tut-1/executorch \
-DET_BUILD_DIR_PATH:PATH=/home/chris/et-tut-1/executorch/cmake-out \
-DPYTHON_EXECUTABLE=$(which python3)
But I get the following error:
$ cmake \
-S . \
-B cmake_build/NUCLEO_WB55RG/develop/GCC_ARM \
-DET_PTE_FILE_PATH:PATH=/home/chris/et-tut-1/executorch/softmax.pte \
-DCMAKE_TOOLCHAIN_FILE=/home/chris/et-tut-1/executorch/examples/arm/ethos-u-setup/arm-none-eabi-gcc.cmake \
-DTARGET_CPU=cortex-m4 \
-DET_DIR_PATH:PATH=/home/chris/et-tut-1/executorch \
-DET_BUILD_DIR_PATH:PATH=/home/chris/et-tut-1/executorch/cmake-out \
-DPYTHON_EXECUTABLE=$(which python3)
-- The C compiler identification is GNU 13.2.1
-- The CXX compiler identification is GNU 13.2.1
-- The ASM compiler identification is GNU
-- Found assembler: /home/chris/et-tut-1/arm-gnu-toolchain-13.2.rel1-x86_64-arm-none-eabi/bin/arm-none-eabi-gcc
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /home/chris/et-tut-1/arm-gnu-toolchain-13.2.rel1-x86_64-arm-none-eabi/bin/arm-none-eabi-gcc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /home/chris/et-tut-1/arm-gnu-toolchain-13.2.rel1-x86_64-arm-none-eabi/bin/arm-none-eabi-g++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found Python3: /home/chris/et-tut-1/executorch/.executorch/bin/python3.10 (found version "3.10.12") found components: Interpreter
-- Checking for Python package prettytable -- found
-- Checking for Python package future -- found
-- Checking for Python package jinja2 -- found
-- Checking for Python package intelhex -- found
CMake Error at mbed-os/CMakeLists.txt:88 (mbed_set_cpu_core_definitions):
Unknown CMake command "mbed_set_cpu_core_definitions".
-- Configuring incomplete, errors occurred!
What am I doing wrong here?
I solved the error :). It happens because Mbed uses its own cmake_install.cmake file
In the meantime, it compiles successfully. Now, I am getting the following error on the board after I flashed it:
executorch:main.cpp:68] Model PTE file loaded. Size: 960 bytes.
E executorch:program.cpp:108] Program identifier 'V�r�' != expected 'ET12'
I executorch:main.cpp:72] Program loading failed @ 0x0x200018A0: 0x23
F executorch:result.h:165] In function CheckOk(), assert failed: hasValue_
++ MbedOS Fault Handler ++
FaultType: HardFault
Context:
R 0: 0000004B
R 1: 0000000A
R 2: 00000000
R 3: 0000004B
R 4: 0000087C
R 5: 8AF0BE00
R 6: 08016204
R 7: 080161DF
R 8: 00000000
R 9: 00000000
R 10: 00000000
R 11: 00000000
R 12: 00000001
SP : 200032C8
LR : 0800F027
PC : 0800045E
xPSR : 010F0000
PSP : 200032A8
MSP : 2002FFC0
CPUID: 410FC241
HFSR : 40000000
MMFSR: 00000000
BFSR : 00000000
UFSR : 00000001
DFSR : 00000008
AFSR : 00000000
Mode : Thread
Priv : Privileged
Stack: PSP
-- MbedOS Fault Handler --
++ MbedOS Error Info ++
Error Status: 0x80FF013D Code: 317 Module: 255
Error Message: Fault exception
Location: 0x800045E
Error Value: 0x20003538
Current Thread: main Id: 0x20002450 Entry: 0x800251D StackSize: 0x1000 StackMem: 0x20002498 SP: 0x200032C8
For more info, visit: https://mbed.com/s/error?error=0x80FF013D&tgt=NUCLEO_WB55RG
-- MbedOS Error Info --
This is the first time I am going to debug such an error. Which procedure is suitable for finding the cause of such an error?
Hmm it certainly sounds like it’s loading bad data and then crashing. I would start by using a debugger to step through program.cpp:108 and see what’s going on.
@MultipleMonomials That is the point. I can compile it with Mbed cli2 but not with Mbed Studio. Therefore, I need another debugging tool like OpenOCDs. Or do you see a chance to do the debugging in Mbed Studio?
Well, shameless plug here, but if you use Mbed CE, you can use CMake for your build configuration, but use VS Code to develop and debug code!
Can you tell me the solution please?
Hi @edwin_lawanda,
As soon as I have run a torch model in Mbed OS, I will post the solution here.
Cheers,
Christoph