Builder tweak to improve Mbed CE support

[multiplemonomials]

Hi! I'm working on adding support for the Mbed CE framework to PlatformIO (forum thread for context). I've nearly gotten it working, but I have to make two small tweaks to the platformio builder logic to get everything running.

Tweak 1: PIO_EXTRA_APP_OBJS

Update: This is not needed anymore and is removed in the latest version of the PR.

Tweak 2: PIO_EXTRA_APP_SOURCE_DEPS

Mbed CE has a ton of #defines that need to be added for every single source file, and rather than make the compiler command line tens of kilobytes long, we write these out to an include file and then force every source file to include this using the -include compiler flag.

The problem is that SCons doesn't seem to consider -include when scanning dependencies, so if the header gets modified, the source files that depend on it aren't rebuilt. This is a big issue with Mbed CE because it means that a manual clean would be required after editing the mbed_app.json file rather than things rebuilding automatically.

To fix this problem, I added a new PIO_EXTRA_APP_SOURCE_DEPS env var. Every application obj file is made to depend on all files in this list, so the app sources will be recompiled if any of these files are changed. This way, I can set the variable to point to the -include-ed header and the dependencies will be handled properly.

P.S. I am still new to PlatformIO so please let me know if there's a better name or location for this functionality and I will happily rework it!

[CLAassistant]

All committers have signed the CLA.

[maxgerhardt]

static library needs to be wrapped in -Wl,--whole-archive.

Sounds like what is already done in existing builder scripts for e.g. ArduinoCore-mbed

https://github.com/platformio/builder-framework-arduino-core-mbed/blob/029e576c57331f1baa41802b7684710a587f3ef8/arduino-core-mbed.py#L209-L211

Not sure if this really really needs that PlatformIO core tweak.

[maxgerhardt]

For Tweak 2: Did you try setting up only a dependency of the final ELF to the include file? Like

env.Depends(target_elf, "path/to/global_include.h")

in the main.py of the platform.

[multiplemonomials]

Did you try setting up only a dependency of the final ELF to the include file?

Yes I did. That just causes the elf file to get relinked, without compiling any of its sources

[multiplemonomials]

@maxgerhardt Excellent find! You are right, using _LIBFLAGS allows me to avoid needing the first tweak.

[maxgerhardt]

I could have sworn I've seen something in the builder script logic for ESP-IDF trigger an entire rebuild of the project if the sdkconfig.h (autogenerated from a sdkconfig file full of config option) changes, but that logic seems to be in the CMake files that PlatformIO calls into, not the SCons build itself (espidf.py). So the tweak to get the dependency on the global config file may seem unavoidable.

Does the -include'd file only contain #define xxx or also other #include? Theoretically, you could write all macro definitions into env.CPPDEFINES. Even if this gets very large, PlatformIO will write the command line arguments to a file and make GCC read its arguments from that file, bypassing the limit on the maximum length of a shell command. And, since they're then in env.CPPDEFINES, PlatformIO should automatically trigger a rebuild if any of those flags changes.