[RFC] Zephyr RTOS as a build target for MeshCore

[marekmatej]

Summary

MeshCore's protocol is small and already well separated from the hardware. The surrounding firmware is neither, and almost none of that weight is protocol work: 87 board variants, 586 build environments, ~12000 lines of PlatformIO configuration, 44 handwritten board JSON files, 40 display driver files, a fork of Adafruit's nRF52 Arduino core, a second ESP32 platform provider because the first went stale, and dependencies pinned to personal forks with no tag and no commit SHA.

That is the real maintenance surface. It grows with every board added, and essentially none of it is
reusable by anyone outside this project.

This RFC argues that Arduino/PlatformIO is the wrong long-term platform for a project of MeshCore's
scope, and that MeshCore should be buildable with Zephyr. Zephyr is an RTOS whose entire design center is
board support, drivers, and dependency management.

It proposes no rewrite, no flag day, and no removal of anything. The Arduino build stays, every board
keeps working, and no 'on air' or 'on disk' format changes. What it asks for is agreement on a direction,
so that the work of making src/ platform neutral can begin as a series of small, independently
useful pull requests.

---

1. What Arduino/PlatformIO costs this project

1.1 A board cannot be described once

Adding a board today means writing a block of preprocessor defines. Pin assignments live in build_flags, 50 -D ...PIN_... macros across the variants. A display choice is a build_src_filter line naming a .cpp. A radio choice is a set of -D RADIOLIB_EXCLUDE_* flags.
The firmware's role is another build_src_filter line. Multiply those, and you get 586 environments across 87 variants/*/platformio.ini files totaling over 14k lines. 44 boards additionally need a handwritten boards/*.json describing the MCU to PlatformIO.

Three consequences follow, and they are structural rather than incidental:

The same hardware appears six or seven times: companion USB, companion BLE, repeater, room server, sensor, and KISS modem, each repeating pin definitions by inheritance through extends chains up to four levels deep. Change a pin, and you change it in one place if the inheritance is right, or in seven if it is not.

The description is not machine-readable. -D PIN_LORA_NSS=8 is handed to a compiler. Nothing can validate that the pin exists on the package, that it does not collide with the SPI bus declared three lines below, or that the display and the radio have not been given the same chip select. These are caught by flashing the board and noticing it does not work.

Board support cannot be shared. A variants/ directory is useful only to MeshCore, and MeshCore
benefits from nobody else.

1.2 The dependency graph is a liability

MeshCore depends on roughly thirty Arduino libraries. The interesting part is not the number; it is
how they are sourced.

• The project forks its own vendor BSP. framework-arduinoadafruitnrf52 is pinned to a MeshCore maintained fork of Adafruit's nRF52 core carrying a BLE stack patch. Every nRF52 build now depends on this organization maintaining a fork of somebody else's Arduino core, indefinitely.
• Two ESP32 platform providers. Most ESP32 builds use platformio/espressif32 @ 6.11.0. ESP32-C6 cannot - it needs Arduino-ESP32 3.x - so it uses a community fork carrying an in tree comment reading "WARNING: experimental. May not work as stable as other platforms."
• Dependencies pinned to nothing. SoulOfNoob/GxEPD2 and maxgerhardt/platform-raspberrypi are personal forks referenced with no tag and no SHA. Whatever is on their default branch today is what builds today. An entire platform - the RP2040 toolchain - is sourced this way.
• The same library from two sources. GxEPD2 appears both as registry zinggjm/GxEPD2 @ 1.6.2 and as the unpinned fork above. heltec-eink-modules appears as two different forks at two different commits. MicroNMEA is ^2.0.6 in some variants and ~2.0.6 in others.

None of these decisions was wrong when it was made! Each was the pragmatic fix for a real problem. But the accumulation is a supply chain nobody audits, that cannot be reproduced from a git tag, and whose failure mode is somebody else force pushing a branch.

Arduino has no dependency story beyond "name a library and hope." There is no per target resolution, no way to say "this library, but only on nRF52," no mechanism for carrying a patch with provenance.

1.3 There is no hardware abstraction below "Arduino, approximately"

Because each Arduino library brings its own idea of how to talk to hardware, src/ carries roughly 560 preprocessor conditional lines and platform macros (ESP32, NRF52_PLATFORM, RP2040_PLATFORM, STM32_PLATFORM) scattered through files that should not know what an MCU is.
IdentityStore.h picks its filesystem type by #ifdef.
CommonCLI.cpp opens the same file three different ways depending on platform.

And there are boards MeshCore simply cannot reach, rn. There is no Arduino core for nRF54L15, EFR32MG24 or STM32WL. Zephyr supports all three today.

2. Why Zephyr-RTOS answers this

Zephyr's devicetree exists precisely to solve the problem in 1.1. A board is described once, in a .dts declaring the SOC, the buses, and what is attached to them. Pin conflicts become build time errors. Drivers bind to nodes by compatible string, so choosing a display is deleting one node and adding another rather than editing a source filter. The role becomes a Kconfig symbol rather than a file list.

Critically, the board description is the same artifact whether the firmware is MeshCore or anything else - which is why most of the boards MeshCore supports already have upstream Zephyr definitions maintained by silicon vendors and board makers, at no cost to this project.
The asymmetry is worth stating plainly: MeshCore currently pays for board support it could largely inherit.

On dependencies (1.2), a west manifest pins every project to an explicit revision, module dependencies are declared and resolved, and downstream patches are also tracked activity rather than ad-hoc work: Zephyr ships a west patch extension command whose patches.yml records, per patch, the module it applies to, its author and date, whether it is considered upstreamable, and the issue, PR and merge commit once it has been submitted. That last part is a convention a project opts into, not something west does on its own - but the option exists, which is more than the current arrangement offers. Together this is not nicer syntax for the same thing; it is the difference between a dependency set you can reason about and one you cannot.

On abstraction (1.3), Zephyr provides a device driver model with proper initialization ordering, plus subsystems MeshCore has no equivalent for: settings/NVS storage, power management, a GNSS driver class with NMEA parsing, and a LoRa driver API. Two further consequences matter for a mesh node specifically:

• The idle loop goes away. k_event_wait/k_poll replaces polling, so an idle node sits in WFI between events instead of spinning. On a battery powered repeater, this is the largest power win available.
• The firmware runs on a PC. A native_sim build compiles the real firmware as a Linux process debuggable without flashing anything, and able to drive a real SX1262 over SPI on an SBC.

3. This is not a rewrite - the seam already exists

The single most important fact in this RFC is that MeshCore's protocol core is already written against abstract interfaces:

interface	declared in
MainBoard, RTCClock	src/MeshCore.h:45, src/MeshCore.h:87
MillisecondClock, Radio, PacketManager	src/Dispatcher.h:14,22,87

All are pure virtual. src/helpers/ is simply one implementation of that seam - the Arduino.

So a Zephyr build does not require restructuring MeshCore. It requires implementing interfaces that already exist and removing the incidental Arduino coupling that has accumulated around them.

4. What an audit of src/ found

Before proposing anything, we audited src/ for platform coupling with a syntax-only host compile:

g++ -fsyntax-only -std=c++17 -I src -I test/mocks -I lib/ed25519 <file>

That command needs a caution, because it is easy to over read. test/mocks/ is 258 lines of stubs, and four of its headers: Arduino.h (17 lines), Stream.h (72), AES.h (13), SHA256.h (35) stand in for headers that exist nowhere else in the tree. Remove -I test/mocks and nothing in src/ compiles at all. So the probe does not show that the core is platform neutral. What it shows is how small the substitute has to be:

file	what it needs that a host toolchain does not provide
Mesh.cpp, Dispatcher.cpp	Arduino's Stream type
Packet.cpp	rweather SHA256.h
Utils.cpp	Arduino's Stream, rweather AES.h + SHA256.h
helpers/StaticPoolPacketManager.cpp	Arduino's Stream type
helpers/ConfigSerializer.cpp	Arduino.h (for millis) and Stream
Identity.cpp	Arduino's Stream, and rweather Ed25519.h - unstubbed, so this one fails
helpers/{IdentityStore,ClientACL,RegionMap,TransportKeyStore}.cpp	IdentityStore.h:15: 'FILESYSTEM' does not name a type
helpers/{AdvertDataHelpers,BaseChatMesh}.cpp	sprintf - <stdio.h> never included
helpers/TxtDataHelpers.cpp	ltoa - an avr-libc extension, absent from glibc and Zephyr's libc
helpers/CommonCLI.cpp	RTClib.h

Read that way, the result is still encouraging, and more defensible than a pass/fail count. The protocol core reaches for exactly two things outside the C++ standard library: Arduino's Print/Stream classes, and the rweather crypto headers. Nothing in Mesh.cpp, Dispatcher.cpp, Packet.cpp or Utils.cpp needs Serial, String, GPIO, or any Arduino behaviour - a 72-line Stream stub with no Arduino code in it is a sufficient substitute. The coupling is shallow, and PR 3 and PR 5 are what remove the two dependencies that remain.

The helper files are a different matter, but their blockers are narrow and largely clerical: one FILESYSTEM typedef at IdentityStore.h:15 accounts for four files, none of which performs any file IO itself - they fail only because IdentityStore.h is on their include path. Two more are missing <stdio.h>. One uses an avr libc function that has a standard replacement. One pulls an Arduino RTC library into the CLI.

Separately, and independently of any of the above: 43 files in src/ carry an uncommented #include <Arduino.h>. Sixteen are inside a #if / #ifdef guard and thirteen genuinely use Arduino symbols, but fourteen are unguarded and use no Arduino symbol at all. Four of them annotated // needed for PlatformIO, which they are not. Deleting those fourteen changes nothing and is PR 1.

The audit also turned up two problems worth naming here - not as complaints, but as evidence. Both are the kind of thing a second platform and a host test build make visible, and neither was visible without them.

4.1 The crypto primitives are an Arduino dependency, selected by #ifdef

Identity.cpp includes <Ed25519.h>, and Utils.cpp includes <SHA256.h> and <AES.h>. All three resolve to rweather/Crypto @ ^0.4.0, a PlatformIO registry package, pulled in by lib_deps. The second implementation, orlp's ed_25519.h, is vendored in lib/ed25519. This is why Identity.cpp is the one file in 4's table that fails even with the stubs in place: Ed25519.h has no stub, and outside PlatformIO there is no lib_deps to resolve the real one.

Which implementation runs is then decided by preprocessor branch. USE_CC310_HW_CRYPTO routes verify, sha256 and AES to the nRF52 CC310 accelerator; everything else falls through to rweather or orlp.
The selection is per operation, not per platform. MeshCore signs with orlp and verifies with rweather, and HMAC-SHA256 is always rweather regardless of accelerator. A second platform makes this a three way #ifdef, and Zephyr's PSA Crypto would make it four.

Two consequences for a port. The mechanical one is that the dependency needs an answer - vendored, declared as a west module, or replaced - before anything compiles. The structural one is that nothing in the tree tests any of this. There are no known answer vectors, and no test that a signature produced by one backend verifies under another - even though that mixed path is what ships today. Changing a backend is currently an unverifiable act, which is precisely why 6 defers PSA and why PR 5 adds vectors before anyone touches it.

(A disabled branch at Identity.cpp:34 carries the comment memory corruption bug was found in this function!! over an ed25519_verify call. PR 5 carries it across untouched; 10 asks what should
happen to it.)

4.2 The existing host tests do not reach the protocol core

[env:native] sets build_src_filter to exactly three files: Utils.cpp, Packet.cpp and helpers/ConfigSerializer.cpp. Mesh.cpp, Dispatcher.cpp and Identity.cpp are not compiled on the host at all, by any environment.

The two files that are tested are tested against the stubs. mocks/AES.h::encryptBlock has an empty body, so every AES call in Utils.cpp is a no-op under test. mocks/SHA256.h is an XOR and rotate mixing function - deterministic and good enough to make two different packets hash differently, which is what calculatePacketHash needs, but not SHA256 - and its finalizeHMAC writes nothing at all. So the packet hash tests verify that the hash is stable, never that it is correct, and no test anywhere exercises a real transport key.

Four more stub headers: Mesh.h, Identity.h, Utils.h, CayenneLPP.h are dead: no test includes them, and -I src precedes -I test/mocks so the real headers win regardless.

None of this is an argument that the tests are bad. It is an argument that there has never been a build in which the protocol core runs against real primitives on a machine a developer can attach a debugger to.
PR 6 creates one, and a Zephyr native_sim target is a second.

A second platform and a real host build would have surfaced both long time ago.

5. What is proposed

A sequence of small pull requests against main (OR 'dev'), one concern each. Each one is independently
useful, and none changes on air behaviour or any on disk format.

#	PR	scope	risk
1	includes (#3420)	delete the 14 unguarded, unused <Arduino.h> includes; add the <stdio.h> two files rely on transitively; ltoa -> snprintf	mechanical
2	logging	route MESH_DEBUG_PRINT* / MESH_PACKET_LOGGING through one MESHCORE_LOG_PRINTF that each platform defines	low
3	FileStore	a mesh::FileStore interface; port IdentityStore and CommonCLI prefs I/O onto it; drop constrain/map/byte/RTClib from the portable path	medium
4	identity format	move LocalIdentity serialisation off Stream& onto FileStore, and name the on-flash and wire layouts separately	medium
5	crypto seam	src/crypto/ behind one internal header, with known answer vectors, fixing 4.1 lack of coverage	medium
6	host harness	widen [env:native], delete the shadow mocks, add a loopback radio so the protocol core is actually tested	low
7	platform layer	document the platform layer convention, and add the Zephyr layer as a Zephyr module, inert by default	low

PRs 1, 2 and 7 are independent. Then 3 depends on 4, and 5 depends on 6.

PRs 1 and 2 are gated on producing a binary identical .elf. PR 4 renames the function that reads the *.id file; the bytes in the file do not move, and the two layouts become separately named so neither reader can be handed the other's data.
PR 3 keeps /prefs.json byte-identical. PR 5 keeps each primitive's backend exactly as it is today, including the per-operation split and the disabled #elif 0 branch - it puts a seam and a test suite around the current behaviour, it does not decide what that behaviour should be.

Note on PR 2, because it looks like a no-op: MESH_DEBUG=1 is set in most of the 586 environments, so this is shipping code, not a debug path. The macros expand to Serial.printf on Arduino exactly as before, and the binary identical .elf gate is what proves that expansion is unchanged.

6. Scope of the first Zephyr build

Deliberately narrow. The goal is a demonstrably working build, not a second board matrix.

A Zephyr platform layer packaged as a proper Zephyr module module.yml, Kconfig, CMakeLists.txt, west.yml Implementing the par3 interfaces: a MainBoard, an RTCClock, a MillisecondClock over k_uptime_get(), a FileStore over LittleFS or the settings subsystem, and a console backed serial interface. Everything sits under a top level menuconfig MESHCORE defaulting to n, so it is inert for anyone building with PlatformIO. Where that layer lives in the tree is a detail to settle on in review.

The target is native_sim plus one real board, in one role. RAK4631 as a repeater is the natural pick: it is already supported by the Arduino build and has an upstream Zephyr board definition, and the repeater role has the smallest surface of the six.

Three scoping decisions worth stating upfront, because they are the ones reviewers will ask about:

Radio: keep RadioLib. RadioLib has an official non-Arduino HAL interface (RadioLibHal).
Providing a Zephyr implementation over spi/gpio/k_uptime_get means MeshCore keeps RadioLib and all seven src/helpers/radiolib/Custom*Wrapper.h files, along with every bit of accumulated radio tuning. It also means no patches to Zephyr's own LoRa drivers are needed, which removes by far the largest source of downstream patches against Zephyr itself. This is less idiomatic than Zephyr's LoRa API, and that trade is deliberate: correctness and continuity first, idiom later if it ever proves worth it.

Display: headless first. src/helpers/ui/'s 40 files are Adafruit/U8g2/LovyanGFX - based and cannot be used under Zephyr. Zephyr has in tree drivers for SSD1306, ST7735R, ST7789V and the SSD16xx ePaper
family; SH1106 and NV3001B have none. Display support is follow up work via Zephyr's display subsystem, not part of the first build.

Crypto: unchanged. The existing rweather/orlp backends compile fine under Zephyr. Routing SHA256, AES and HMAC to PSA Crypto - and thus to whatever accelerator the SoC has - is a genuine later win, but it is an option PR 5's seam makes available, not a promise this RFC makes.

7. Evidence that this works

A complete Zephyr port of MeshCore exists as a separate project, ZephCore. It is wire and file compatible with this firmware. Same packet format, same binary prefs and contacts layout. It pairs with the existing MeshCore mobile apps, it has been shipping releases for months, and it already runs on nRF54L15, EFR32MG24 and STM32WL.

This RFC does not propose merging ZephCore, or adopting any of its code.
ZephCore is a hard fork carrying its own copies of Mesh.cpp, Dispatcher.cpp, Packet.cpp, Identity.cpp and Utils.cpp, along with a substantial stack of downstream patches against Zephyr itself. Bringing that in wholesale would import a large body of divergence to solve a problem MeshCore can solve directly.

What ZephCore establishes is the one thing worth knowing before starting: the protocol core needs no
changes to run under Zephyr. It has been proven working, against this firmware, on real meshes. The work in par5 and par6 is about reaching the same destination from MeshCore's own sources.

8. Non-goals

• No removal or deprecation of the Arduino/PlatformIO build.
• No board dropped. No change to the 586 environments.
• No change to the packet format, the prefs/contacts/channels binary layout, or the CLI grammar.
• No behaviour change in any preparation PR.
• No relocation of src/helpers/. Renaming it to mark it as the Arduino platform layer would be tidier, but it would touch every variant's build_src_filter for no functional gain.
• No attempt to share examples/* with a Zephyr application layer.
• No requirement that existing contributors learn Zephyr to keep contributing.

9. The counterarguments, stated fairly

Zephyr has a steeper learning curve.
True, and the strongest objection. Devicetree and Kconfig are genuinely harder than -D PIN_LORA_NSS=8, and a contributor who can add a variants/ directory in an evening may not be able to write a .dts in one. The mitigation is that most boards already have upstream definitions, so the common case is an overlay rather than a board definition. But the cost is real and should not be talked away.

The Arduino ecosystem is larger for hobbyist peripherals.
Also true. If someone wants a specific obscure OLED working tomorrow, an Arduino library probably exists and a Zephyr driver probably does not.

586 environments represent a lot of validated hardware.
Nobody should propose throwing that away, and this RFC does not.

Zephyr brings its own dependency weight.
A west workspace clones Zephyr, its modules and HAL blobs (tens of thousands of files). Bigger checkout, slower first build, in exchange for a pinned and reproducible one.

Keeping RadioLib means inheriting less of Zephyr's driver ecosystem.
Correct, and deliberate (6). The board support and dependency management arguments are unaffected; the "stop re-implementing drivers" argument becomes directional rather than immediate.

10. The question for maintainers

Setting aside implementation detail, the question is:

Does the project accept that its long-term platform is Zephyr, that board support, driver work and dependency management should increasingly happen there rather than in variants/, src/helpers/ui/ and platformio.ini, while the Arduino build continues for as long as it carries users?

A yes makes the par5 pull requests the obvious next step, and they can begin immediately: PRs 1 and 2 are mechanical, and both are worth having whatever is decided afterward.

A no is a legitimate answer. But it should be made deliberately, with the cost of the current surface understood, rather than arrived at by not deciding.

Secondary questions, only if the answer is yes:

1. Do releases and tags stay shared between Arduino and Zephyr artifacts, or does the Zephyr build get its own release track? A governance question, not a technical one.
2. The #elif 0 ed25519 verify branch (4.1): fix it, delete it, or leave it exactly as is? PR 5 deliberately carries it across untouched rather than deciding.
3. How much CI time is acceptable for a Zephyr build? Suggested starting point: native_sim plus the single board in 6.

[marekmatej]

@liquidraver you might be interested.