Add CI, E2E Tests, and Pre-commit Hooks for Java SDK

[jason810496]

Add CI, E2E Tests, and Pre-commit Hooks for Java SDK

• Merge order: This is PR 3 of 3 — merge this after Add Coordinator Layer and Java Coordinator #65958

1. Add Java SDK #65956
2. Add Coordinator Layer and Java Coordinator #65958
3. Add CI, E2E Tests, and Pre-commit Hooks for Java SDK #65959 (this PR)

• Try it out: A combined PoC branch with all changes cherry-picked is available at [DON'T MERGE] Java SDK All #65960 for reviewers who want to test the full integration end-to-end.

Why

The Java SDK and Language Coordinator Layer (PRs 1 and 2) introduce cross-language serialization and a new execution path. Without CI coverage, regressions in the serialization contract or the end-to-end execution flow would go undetected.

What

E2E Tests (airflow-e2e-tests/)

• Add java-sdk.yml Docker Compose overlay that provisions the Java SDK, builds the example JAR bundle, and configures Airflow with the Java provider
• Add test_java_sdk.py with end-to-end tests for both integration patterns:
  • Python stub DAG (stub_dag.py): python_task_1 → extract (Java) → transform (Java) → python_task_2 — validates Python-Java XCom interop, getConnection, getVariable, and bidirectional data flow
  • Pure Java DAG (JavaExample.java): extract → transform → load — validates Java-only DAG parsing, task execution, and expected failure handling (load throws RuntimeException)
• Extend conftest.py and constants.py with Java SDK fixtures, DAG IDs, and test utilities

Pre-commit Hook (scripts/ci/prek/)

• Add check_java_serialization_compatibility.py pre-commit hook that:
  1. Runs the Java SDK SerializationCompatibilityTest via Gradle to produce serialized_java.json
  2. Runs serialize_python.py to produce serialized_python.json
  3. Runs compare.py to deep-diff the two outputs
  4. Fails the commit if any DAG serializes differently across the two SDKs
• Register the hook in .pre-commit-config.yaml

Breeze Integration (dev/breeze/, scripts/)

• Add --sdk java flag to breeze start-airflow for one-command Java development setup
• Add java_sdk_build.sh — builds the Java SDK and example JAR from source inside the Breeze container
• Add java_sdk_setup.sh — installs JDK, builds JARs, and configures DAG bundle paths
• Cache Gradle dependencies in Breeze to speed up repeated builds
• Extend entrypoint_ci.sh with Java SDK environment initialization

---

Was generative AI tooling used to co-author this PR?

• Yes, with help of Claude Code Ops 4.6 following the guidelines

Co-authored-by: Tzu-ping Chung uranusjr@gmail.com