Fork Notes — birdingman0626/extender

This fork diverges from upstream defold/extender to support running the extender on a Windows host without Docker, plus warning cleanup, two cherry-picked bug-fix issues, and dependency bumps.

Base commit at last sync: 97f20e2313a77959c3c35ce1440ef26e198c1b4f (dev branch, "Retag base winsdk image" — git describe --match 'extender-v*' → extender-v2.12.4-46-g97f20e2, i.e. 46 commits past upstream's published extender-v2.12.4)
Fork version scheme: <closest-upstream-release>-fork.<N>. Bump N when re-cutting on the same upstream base; reset to 1 whenever the fork rebases onto a newer upstream release. Current fork tag: fork-v2.12.4-fork.1.
Total divergence: 29 files, ~226 insertions / 28 deletions

Maintainership intent: continuous merge from upstream/dev is expected. This document is the rebase ledger — every divergence is listed with what changed, why, and whether it's safe to drop after a future upstream merge.

How to merge from upstream

cd D:\DevTools\extender
git fetch upstream
git merge upstream/dev      # or: git rebase upstream/dev
# resolve conflicts using this document as a guide
./gradlew :server:bootJar --no-daemon --warning-mode all   # must stay zero-warning
cp server/build/libs/extender-*.jar server/app/extender.jar

For each conflicting file, find its entry in the Touched files section below to decide what to keep.

1. Windows-host compatibility patches (MUST KEEP)

These three patches are mandatory for the extender to run on Windows without Docker. They have no matching upstream issue — pure additions. If upstream rewrites any of these three methods, re-apply the patch.

1.1 `ZipUtils.java` — POSIX file-attribute guard

Why: NTFS does not implement POSIX file attributes; upstream's Files.setPosixFilePermissions(...) throws UnsupportedOperationException during the first SDK unzip on Windows. Patch:

Added import java.nio.file.FileSystems.

Added class-level constant:

private static final boolean POSIX_FS =
    FileSystems.getDefault().supportedFileAttributeViews().contains("posix");

Guarded the chmod call in unzip(...):

if (POSIX_FS) {
    Files.setPosixFilePermissions(entryTargetFile.toPath(), s);
}

On merge: if upstream introduces an alternative chmod path, keep the guard; the Linux container is unaffected because POSIX_FS == true there.

1.2 `Extender.java` — upper-case env aliases

Why: Windows preserves env-var casing (Path, not PATH). Mustache template lookups are case-sensitive, so {{env.PATH}} in build.yml misses on Windows hosts. Patch: in the constructor's env-context loop (around line 280), for every entry from System.getenv() and builder.env, also publish an upper-case alias when the upper differs:

String upper = key.toUpperCase();
if (!upper.equals(key)) {
    envContext.putIfAbsent("env." + upper, value);
}

On merge: if upstream changes how envContext is populated, port the same upper-case-alias loop. Linux is unaffected because canonical Linux env keys already match upper-case.

1.3 `TemplateExecutor.java` — `defaultValue("")` on Mustache compiler

Why: build.yml references env vars for cross-compilers we don't install on Windows (ZIG_PATH_0_11, Xcode vars, emsdk vars). Strict Mustache aborts the whole build on the first unresolved {{env.X}} even when X is irrelevant to the target platform. Patch: replaced both compile sites:

Mustache.compiler().defaultValue("").compile(template).execute(context)

(unset keys substitute as empty string instead of throwing). On merge: if upstream switches templating libraries or adds its own default-handling, drop this; otherwise keep. Side-effect on Linux: the cloud setup has every env var populated, so behaviour is unchanged there.

2. Bug fixes — selected from upstream open issues

2.1 `ExtensionManifestValidator.java` (closes #532 — emscriptenLinkFlags validation)

Why: upstream issue #532 asks for an allowlist on emscripten link flags; currently any string passes through to the linker. Patch: added VALID_EMSCRIPTEN_LINK_FLAG regex + dedicated case "emscriptenLinkFlags" in validate(...) that rejects entries containing shell metacharacters, whitespace, or that don't start with - / --. Test: ExtensionManifestValidatorTest.testEmscriptenLinkFlagsValidation (new). On merge: if upstream lands their own validation for this flag category, drop our regex/case; otherwise keep until #532 closes.

2.2 `services/DataCacheService.java` (closes #646 — build fail when cache enabled)

Why: the LOCAL cache mode fails the build when a previously cached blob is missing from disk (e.g. cache dir emptied between builds, or worker extender's cache lags the frontend's). Reporter: zotrix, 2025-01. Patch: in getCachedFiles(...), re-check dataCache.exists(key) for every entry the client claims is cached. On miss, log a warning, set entry.setCached(false), and continue — the post-build cacheFiles path will re-upload the blob (self-healing). Tests: added testDownloadHandlesMissingBlob + extended the existing testDownload mocks to stub exists(). On merge: check whether upstream has shipped its own fix to #646 (none as of base commit); if so, prefer upstream's, drop ours.

3. Warning cleanup (zero-warning build)

./gradlew :server:bootJar --warning-mode all is currently warning-free. None of these changes alter logic. If upstream regresses any of these, they reappear as new warnings — port the same annotation forward.

3.1 Build config — `server/build.gradle`

Added tasks.withType(JavaCompile).configureEach { options.compilerArgs += ['-Xlint:-processing'] } (Lombok requires the processor on; Spring annotations otherwise trigger the "no processor matches" note).
Rewrote bootJar { doLast { copy {...} } } to capture layout.buildDirectory.file(...) / layout.projectDirectory.dir("app") / versionTag at configuration time instead of reading project.projectDir / $project.ext.extenderVersion at execution time. The latter is deprecated in Gradle 9 and removed in Gradle 10 (incompatible with the configuration cache).

3.2 `serialVersionUID` — 5 exception classes

Added private static final long serialVersionUID = 1L; to:

ExtenderException.java
PlatformNotSupportedException.java (also tightened private static String ERROR_MESSAGE to private static final String)
VersionNotSupportedException.java (same tightening)
remote/RemoteBuildException.java
services/cocoapods/PodfileParsingException.java

3.3 Redundant casts removed

TemplateExecutor.java lines 25 / 37 — (String)template (the var is already String).
services/HealthReporterService.java:142 — (OperationalStatus)statuses.get(...) (Map<String, OperationalStatus> returns the right type).
services/cocoapods/CocoaPodsService.java — removed two explicit writer.close() calls inside try-with-resources.

3.4 Deprecation handling

ExtenderController.java:138 — replaced HttpStatus.UNPROCESSABLE_ENTITY with HttpStatus.valueOf(422).
cache/LocalDiskDataCache.java:60 — inlined StringUtils.removeEnd(s, suffix) as s.endsWith(suffix) ? s.substring(0, s.length() - suffix.length()) : s.
Extender.java (1 method, near line 2691): @SuppressWarnings("deprecation") for ResolvedPods.getPodsPrivacyManifests() (no upstream replacement yet).
services/GCPInstanceService.java constructor — @SuppressWarnings("deprecation") for Builder.setExecutorProvider(...) (Google Cloud SDK still routes here).
tracing/ExtenderExecutor.java — @SuppressWarnings("deprecation") on getAsyncExecutor, initializeExecutor, getScheduledExecutor for Micrometer's deprecated ContextSnapshot.captureAll(Object...) and ContextScheduledExecutorService.wrap(...).

3.5 `@SuppressWarnings("unchecked")` for json-simple raw types

Added at the narrowest method scope (json-simple's JSONObject/JSONArray extend raw HashMap/ArrayList so every put/get/cast is unchecked by design). Touched files:

Extender.java — 9 methods
ExtenderUtil.java — 6 methods (one was previously a commented-out annotation; activated)
ExtensionManifestValidator.java — 1 method covers 4 warning sites
builders/CSharpBuilder.java — updateContext
services/cocoapods/CocoaPodsService.java — installPods
services/cocoapods/PodSpecParser.java — 4 methods
utils/PodBuildUtil.java — 3 methods

3.6 `@SuppressWarnings("this-escape")` for Spring `@Autowired` constructors

Benign Java 21 lint on constructors that register this before the class finishes initializing. One annotation per constructor:

services/GradleService.java
services/UserUpdateService.java
services/cocoapods/PodBuildSpec.java
services/cocoapods/ResolvedPods.java
client/.../ExtenderClientCache.java — ctor + private loadCache() (called from ctor)

3.7 Client + manifestmergetool warnings

The CI's Linux build compiles :client:compileJava and :manifestmergetool:compileJava, neither of which :server:bootJar touches. Fixed:

client/.../ExtenderClient.java — @SuppressWarnings("unchecked") on queryCache and getCachedFiles (json-simple raw types, same pattern as §3.5).
client/.../ExtenderClientException.java — serialVersionUID.
manifestmergetool/.../PlistMergeException.java — extracted from InfoPlistMerger.java into its own file (silences the [auxiliaryclass] warning emitted at the use sites in PrivacyManifestMerger.java); also got serialVersionUID.
manifestmergetool/.../AndroidManifestMerger.java — qualified two static-field assignments in the constructor with the class name instead of this. so the [static] warning goes away.
manifestmergetool/.../ManifestMergeTool.java — converted the leading /** ... */ block (which wasn't attached to any declaration) into a plain // ... comment to clear [dangling-doc-comments].

4. Dependency version bumps

Equivalent to open dependabot PRs #927/#929/#931/#932/#933 — applied directly rather than merging the bot branches.

Module	Dependency	From	To
`server/build.gradle`	`com.google.cloud:google-cloud-compute`	`1.100.0`	`1.101.0`
`server/build.gradle`	`org.projectlombok:lombok`	`1.18.44`	`1.18.46`
`client/build.gradle`	`commons-io:commons-io` (testImpl)	`2.21.0`	`2.22.0`
`server/manifestmergetool/build.gradle`	`org.jsoup:jsoup`	`1.22.1`	`1.22.2`
`server/manifestmergetool/build.gradle`	`commons-codec:commons-codec`	`1.21.0`	`1.22.0`
`server/manifestmergetool/build.gradle`	`commons-io:commons-io` (testImpl)	`2.21.0`	`2.22.0`

On merge: if upstream has merged the dependabot PRs, take theirs. If upstream's versions are newer than ours, take upstream's. Conflicts here are pure version strings — no code change.

5. Triaged but not fixed (25 upstream issues)

Skipped because they require cross-repo coordination (defold/defold, defold/bob), Docker image changes, GCP/S3 infra, or are larger features than reasonable on a fork.

#	Title	Reason
163	Throw on incompatible bob version	SCOPE
232	Cache extension build results	SCOPE
318	Thread NE subtasks	SCOPE
353	Mac SDK path missing under Cocoapods	INFRA
360	Build .c with C compiler	SCOPE
566	Replace Proguard with R8	SCOPE
579	AWS S3 data migration	INFRA
580	Anonymous extension stats	SCOPE
581	Cancel a job	SCOPE
582	Job progress	SCOPE
585	Cleanup toolchain pins in Dockerfiles	INFRA
591	Custom Jetty response handler (R&D)	SCOPE
619	Rework RemoteEngineBuilder tests	SCOPE
630	Rework user-facing errors	SCOPE
659	Inconsistent ios/osx min version	INFRA
670	Android local-path extension how-to	QUESTION
688	Update Zig version	INFRA
726	Editable NDK API version	INFRA
787	CMake as project generator (R&D)	SCOPE
805	Update Wine	INFRA
823	Swift language support (iOS/macOS)	SCOPE
824	Swift Package Manager support	SCOPE
829	Use official bob.jar (not bob-light)	SCOPE
861	Local `.aar` support	SCOPE
940	`--start-group` / `--end-group` link support	INFRA

Re-triage these after each upstream sync — some will become fixable once upstream lands prerequisites.

6. Touched files — quick rebase index

When a merge conflicts, the file's row tells you which sections above own the change.

File	Owning sections
`client/build.gradle`	§4 (commons-io bump)
`server/build.gradle`	§3.1, §4
`server/manifestmergetool/build.gradle`	§4
`server/src/main/java/com/defold/extender/Extender.java`	§1.2, §3.4, §3.5
`server/src/main/java/com/defold/extender/ExtenderController.java`	§3.4
`server/src/main/java/com/defold/extender/ExtenderException.java`	§3.2
`server/src/main/java/com/defold/extender/ExtenderUtil.java`	§3.5
`server/src/main/java/com/defold/extender/ExtensionManifestValidator.java`	§2.1, §3.5
`server/src/main/java/com/defold/extender/PlatformNotSupportedException.java`	§3.2
`server/src/main/java/com/defold/extender/TemplateExecutor.java`	§1.3, §3.3
`server/src/main/java/com/defold/extender/VersionNotSupportedException.java`	§3.2
`server/src/main/java/com/defold/extender/ZipUtils.java`	§1.1
`server/src/main/java/com/defold/extender/builders/CSharpBuilder.java`	§3.5
`server/src/main/java/com/defold/extender/cache/LocalDiskDataCache.java`	§3.4
`server/src/main/java/com/defold/extender/remote/RemoteBuildException.java`	§3.2
`server/src/main/java/com/defold/extender/services/DataCacheService.java`	§2.2
`server/src/main/java/com/defold/extender/services/GCPInstanceService.java`	§3.4
`server/src/main/java/com/defold/extender/services/GradleService.java`	§3.6
`server/src/main/java/com/defold/extender/services/HealthReporterService.java`	§3.3
`server/src/main/java/com/defold/extender/services/UserUpdateService.java`	§3.6
`server/src/main/java/com/defold/extender/services/cocoapods/CocoaPodsService.java`	§3.3, §3.5
`server/src/main/java/com/defold/extender/services/cocoapods/PodBuildSpec.java`	§3.6
`server/src/main/java/com/defold/extender/services/cocoapods/PodSpecParser.java`	§3.5
`server/src/main/java/com/defold/extender/services/cocoapods/PodfileParsingException.java`	§3.2
`server/src/main/java/com/defold/extender/services/cocoapods/ResolvedPods.java`	§3.6
`server/src/main/java/com/defold/extender/tracing/ExtenderExecutor.java`	§3.4
`server/src/main/java/com/defold/extender/utils/PodBuildUtil.java`	§3.5
`server/src/test/java/com/defold/extender/ExtensionManifestValidatorTest.java`	§2.1 (new test)
`server/src/test/java/com/defold/extender/services/DataCacheServiceTest.java`	§2.2 (new test + mock fix)

7. Acceptance checks after every merge

cd D:\DevTools\extender
./gradlew :server:bootJar :manifestmergetool:build --no-daemon \
    --rerun-tasks --warning-mode all
# expect: BUILD SUCCESSFUL, zero "warning"/"note"/"deprecated"/"警告"/"注" lines

# boot test
Start-Process -FilePath "$env:TEMP\extender_launch.bat" \
    -RedirectStandardOutput "$env:TEMP\extender_server.log" \
    -RedirectStandardError "$env:TEMP\extender_server.log.err" \
    -WindowStyle Hidden
curl http://localhost:9010/actuator/health   # expect: status UP

# Windows-host build test (consumer side: defold_vn repo)
# see .agentdocs/build/local_extender.md in that repo

If any of the three §1 patches are dropped, the boot test or the first build through --build-server http://localhost:9010 will fail with one of these signatures:

§1.1 dropped: UnsupportedOperationException at ZipUtils.unzip during the first request.
§1.2 dropped: MustacheException: No method or field with name 'env' on line 1 while substituting {{env.PATH}}.
§1.3 dropped: same Mustache error but on {{env.ZIG_PATH_0_11}} or another unset toolchain var.

8. OHOS native target — work in progress

Context: why a native port

The defold_vn WebView wrapper (birdingman0626/defold-vn → .agentdocs/build/ohos_webview.md) produces a signed .hap that installs via hdc install and runs in ArkWeb, but the OHOS x86_64 emulator's software GLES makes Defold's engine assert internally at dmArray::operator[] during init — black screen, no recovery from outside the engine WASM. The same bundle runs cleanly in Chrome desktop, so the bundle is fine; the emulator is the limit. To make HarmonyOS a first-class Defold target (and bypass the emulator's WebView entirely), the path is a native arm64-ohos engine.

What this fork already has (recipe-side)

A working OHOS recipe is patched into the locally-cached SDK build.yml under D:\DevTools\extender\sdk\<sha>\defoldsdk\extender\build.yml. It mirrors arm64-android with these changes:

Field	OHOS value
Compiler	`{{env.OHOS_NDK_BIN_PATH}}/clang++.exe -target aarch64-linux-ohos`
Sysroot	`{{env.OHOS_NDK_SYSROOT}}` (musl-based)
Linker	`{{env.OHOS_NDK_BIN_PATH}}/ld.lld`
Archiver	`{{env.OHOS_NDK_BIN_PATH}}/llvm-ar`
defines	`DM_PLATFORM_OHOS, __aarch64__, __MUSL__`
dynamicLibs	`EGL GLESv2 OpenSLES m c` (no `-llog` / `-landroid`)

The bundled aarch64-unknown-linux-ohos-clang++ is a POSIX shell script (#!/bin/sh exec clang++ -target … --sysroot=… -D__MUSL__ …), which Windows can't CreateProcess. We call clang++.exe directly with the equivalent flags. Smoke-test produced a valid ELF aarch64 shared object:

clang++.exe -target aarch64-linux-ohos --sysroot=<sysroot> \
            -D__MUSL__ -O2 -fpic -shared hello.cpp -o libhello.so
# → ELF 64-bit LSB shared object, ARM aarch64

The launcher .bat (%TEMP%\extender_launch.bat) now exports:

OHOS_NDK_PATH        = D:\DevTools\command-line-tools\sdk\default\openharmony\native
OHOS_NDK_BIN_PATH    = D:\DevTools\command-line-tools\sdk\default\openharmony\native\llvm\bin
OHOS_NDK_SYSROOT     = D:\DevTools\command-line-tools\sdk\default\openharmony\native\sysroot

Limit: the SDK's build.yml is downloaded by extender per-build from the Defold release archive; my local edit only affects the already-cached SDK. To make this recipe shippable, the same block needs to land in defold/defold's share/extender/build_input.yml (line ~485, alongside arm64-android). Tracked below.

What's still needed (cross-repo)

Repo	File(s)	Change	Status
`defold/defold`	`share/extender/build_input.yml`	Append `ohos:` + `arm64-ohos:` block (same content as my local SDK edit). Without this, every fresh extender build downloads a build.yml that doesn't know about OHOS.	pending — fork created at `birdingman0626/defold`
`defold/defold`	`engine/graphics/proto/graphics/graphics_ddf.proto`	Add `OS_ID_OHOS = 11;` to the `PlatformProfile.OS` enum (currently ends at `OS_ID_XBOX = 10`).	pending
`defold/defold`	`com.dynamo.cr/com.dynamo.cr.bob/src/com/dynamo/bob/Platform.java`	Add an `Arm64Ohos` Platform constant alongside `Arm64Android`, wired to `OS.OS_ID_OHOS`. Without this, bob.jar rejects `--platform=arm64-ohos` as "Platform … not supported" before the extender is even consulted.	pending
`defold/defold`	`build_tools/BuildUtility.py`	Add `{'platform': 'arm64-ohos', 'os': TargetOS.OHOS, 'arch': 'arm64'}` next to the Android entries; add `TargetOS.OHOS` enum value.	pending
`defold/defold`	`build_tools/sdk.py`	OHOS-SDK-detection block: look for `OHOS_NDK_PATH` / standard install locations, set the proper paths returned to waf.	pending
`defold/defold`	`scripts/build.py`	Add `arm64-ohos` to `BASE_PLATFORMS`.	pending
`defold/defold`	`engine/platform`, `engine/graphics`, `engine/hid`, `engine/sound`, `engine/sys`, `engine/window`, `engine/dlib`	Port platform-layer code. HarmonyOS Native shares much with POSIX/Linux for syscalls (musl libc) but has its own NDK headers (`napi/native_api.h`, `multimedia/audio_framework/`) and graphics surface model (NativeWindow via `OH_NativeWindow_`). Largest chunk of work — multi-week.	pending — out of scope for current session
`defold/defold`	`engine/extension/src/dmsdk/extension/extension.h`	Add `DM_PLATFORM_OHOS` conditional next to `DM_PLATFORM_ANDROID`. Trivial once preceding work is done.	pending
`defold/bob` (lives inside `defold/defold/com.dynamo.cr/com.dynamo.cr.bob`)	bundle producer	Add `.hap` bundle target (or leave bob producing the `.so` and let `scripts/build_ohos.ps1` wrap into the HAP — simpler short-term).	pending

Validation prerequisite

Before applying any of these patches, I'm validating that the fork at birdingman0626/defold can compile the engine for at least one existing platform via GitHub Actions. Once that's green for arm64-android (the closest analogue), I add the OHOS entries and re-run CI for arm64-ohos. The fork's CI workflow lives at birdingman0626/defold/.github/workflows/fork-build.yml and is deliberately stripped of upstream Defold's S3/notarization/Slack secrets — it only verifies the engine compiles.

Why this lives in the extender fork's notes

The extender fork is the entry point: anyone reading FORK_NOTES.md here while debugging a Windows-host build is the most likely audience for "and here's the next step toward native OHOS." Once the engine fork has its own README/notes, this section will shorten to a link.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fork Notes — birdingman0626/extender

How to merge from upstream

1. Windows-host compatibility patches (MUST KEEP)

1.1 `ZipUtils.java` — POSIX file-attribute guard

1.2 `Extender.java` — upper-case env aliases

1.3 `TemplateExecutor.java` — `defaultValue("")` on Mustache compiler

2. Bug fixes — selected from upstream open issues

2.1 `ExtensionManifestValidator.java` (closes #532 — emscriptenLinkFlags validation)

2.2 `services/DataCacheService.java` (closes #646 — build fail when cache enabled)

3. Warning cleanup (zero-warning build)

3.1 Build config — `server/build.gradle`

3.2 `serialVersionUID` — 5 exception classes

3.3 Redundant casts removed

3.4 Deprecation handling

3.5 `@SuppressWarnings("unchecked")` for json-simple raw types

3.6 `@SuppressWarnings("this-escape")` for Spring `@Autowired` constructors

3.7 Client + manifestmergetool warnings

4. Dependency version bumps

5. Triaged but not fixed (25 upstream issues)

6. Touched files — quick rebase index

7. Acceptance checks after every merge

8. OHOS native target — work in progress

Context: why a native port

What this fork already has (recipe-side)

What's still needed (cross-repo)

Validation prerequisite

Why this lives in the extender fork's notes

FilesExpand file tree

FORK_NOTES.md

Latest commit

History

FORK_NOTES.md

File metadata and controls

Fork Notes — birdingman0626/extender

How to merge from upstream

1. Windows-host compatibility patches (MUST KEEP)

1.1 ZipUtils.java — POSIX file-attribute guard

1.2 Extender.java — upper-case env aliases

1.3 TemplateExecutor.java — defaultValue("") on Mustache compiler

2. Bug fixes — selected from upstream open issues

2.1 ExtensionManifestValidator.java (closes #532 — emscriptenLinkFlags validation)

2.2 services/DataCacheService.java (closes #646 — build fail when cache enabled)

3. Warning cleanup (zero-warning build)

3.1 Build config — server/build.gradle

3.2 serialVersionUID — 5 exception classes

3.3 Redundant casts removed

3.4 Deprecation handling

3.5 @SuppressWarnings("unchecked") for json-simple raw types

3.6 @SuppressWarnings("this-escape") for Spring @Autowired constructors

3.7 Client + manifestmergetool warnings

4. Dependency version bumps

5. Triaged but not fixed (25 upstream issues)

6. Touched files — quick rebase index

7. Acceptance checks after every merge

8. OHOS native target — work in progress

Context: why a native port

What this fork already has (recipe-side)

What's still needed (cross-repo)

Validation prerequisite

Why this lives in the extender fork's notes

1.1 `ZipUtils.java` — POSIX file-attribute guard

1.2 `Extender.java` — upper-case env aliases

1.3 `TemplateExecutor.java` — `defaultValue("")` on Mustache compiler

2.1 `ExtensionManifestValidator.java` (closes #532 — emscriptenLinkFlags validation)

2.2 `services/DataCacheService.java` (closes #646 — build fail when cache enabled)

3.1 Build config — `server/build.gradle`

3.2 `serialVersionUID` — 5 exception classes

3.5 `@SuppressWarnings("unchecked")` for json-simple raw types

3.6 `@SuppressWarnings("this-escape")` for Spring `@Autowired` constructors