Concept: expose a user's Telegram account as a file tree — folder per chat (named after the chat/user, in Telegram dialog order), containing all downloaded media plus the chat history exported as text. The primary product is a native drive resembling Dropbox/Google Drive:
- desktop and mobile apps that integrate with Finder, Explorer, iOS Files, Android's system picker, and Linux mounts;
- the Telegram engine embedded per device for desktop/Android v1, with an interchangeable remote source retained for iOS cold hydration, self-hosting, or a later hosted tier;
- no web application or rich Telegram-like UI requirement for the initial product.
Early implementation. The technology survey, specification baseline, and complete service decomposition are committed; architecture decisions marked provisional still require their explicit decision tasks. Product code so far: the shared Rust core workspace skeleton (crates/, see below) — crate boundaries, dependency-direction rules, and quality gates; domain logic is still to come.
Start with the specification index and the generated project plan. Product implementation is intentionally deferred until the plan is reviewed and approved.
Naming (DEC-019, POL-7): the public product name is GramDrive, and every shipped identifier is derived from the com.reluxworks.gramdrive namespace. tgfs remains the internal repository/codename only — it collides with TheodoreKrypton/tgfs and must not appear in user-visible strings, marketing, or store listings. The repository is deliberately not renamed. Trademark/handle check happens before public release.
- Shared client core: Rust — virtual tree, local SQLite/cache, change cursors, hydration, range downloads, retry/recovery, offline state, naming and generated files. Swift/Kotlin bindings through UniFFI; direct use on Windows/Linux. Verified precedents: Element X (matrix-rust-sdk), Dropbox Nucleus, Firefox (
.research/260715-shared-core-feasibility.md). - Telegram source — behind one provider-neutral Rust trait, two interchangeable implementations:
- Local-first: TDLib (BSL-1.0) embedded per device via tdjson FFI — zero infrastructure, Dropbox UX on desktop/Android; on iOS TDLib fits only in the main app (FP extension ~20 MB cap), and TDLib has no takeout API (normal-API backfill with flood-wait pacing).
- Remote: Go + gotd/td (MIT) — optional for takeout backfill, one canonical archive, and iOS cold hydration; requires an always-on instance (self-hosted or SaaS with auth-key custody). Patterns from iyear/tdl (study only — AGPL); Telethon (Codeberg, MIT) is the takeout-worker alternative.
- OS integration: thin native adapters: File Provider replicated extensions (iOS/macOS, Swift), Cloud Files API (Windows, Rust), DocumentsProvider/SAF (Android, Kotlin), FUSE (Linux, Rust). Read-only first.
- UI: native and minimal; web and rich chat UI deferred.
- Storage: SQLite/PostgreSQL canonical metadata + content-addressed blob store; text export as NDJSON (lossless) + Markdown (human-readable).
- Treat official Telegram clients as reference implementations. A commercial GPL fork is legally possible, but a proprietary/closed-source fork is generally incompatible with their copyleft obligations and needs a deliberate licensing review.
| Artifact | Contents |
|---|---|
.spec/architecture.md |
Native-drive architecture with the shared Rust core and interchangeable local/remote sources |
.spec/policies.md |
Accepted product policies POL-1…POL-8 (ordering, media/Archive Mode, retention, restricted content, support matrix, licensing, naming, approval gates) |
.research/260715-telegram-filesystem-landscape.md |
Synthesized library, API, platform, architecture, and prior-art report |
.research/260715-core-libraries.md |
MTProto/TDLib library landscape |
.research/260715-oss-clients.md |
Official/OSS Telegram clients, licenses, and API terms |
.research/260715-filesystem-integration.md |
File Provider / Cloud Files / SAF / FUSE survey |
.research/260715-prior-art.md |
Exporter, archive, Telegram-FUSE, and WebDAV prior art |
.research/260715-shared-core-feasibility.md |
Rust/UniFFI precedents, TDLib extension constraints, gomobile and grammers analysis |
docs/OPEN_QUESTIONS.md |
Open product and architecture decisions |
docs/TELEGRAM_API_COMPLIANCE.md |
Telegram API terms → verifiable controls, rule-to-task mapping (TGC-nn) |
docs/TRACEABILITY.md |
Requirement coverage matrix: every PRD/DOM/SYNC/PLAT/SEC/NFR/DEC/POL ID mapped to board elements |
The canonical local board is stored in .task-board/ and must be changed through the task-board CLI. The current baseline contains 11 epics, 53 stories, and 142 atomic tasks; all remain unstarted.
Human-only work is isolated in the manual-actions epic (EPIC-260716-3vc5ay): product decisions and ADR ratification, external credentials (Telegram api_id/api_hash, Apple signing assets, Windows signing identity, test devices), and manual on-device release validation. Every other epic is designed to run autonomously in an agent loop once its manual-actions dependencies are done.
The project-level dependency plan has four phases:
- manual decisions/credentials plus autonomous product-foundation analysis;
- shared Rust core;
- local TDLib source and the optional remote tier;
- native drive integrations plus cross-platform quality, security, and release work.
Detailed generated plans for every epic are in .planning/. The remote tier is decomposed to preserve interface and sizing clarity, but remains optional and does not authorize hosted-service implementation.
Conventions:
.spec/— product and architecture source of truth.crates/— the shared Rust core workspace (crates/README.mddocuments layers, dependency direction, and feature policy).apple/— Apple-native packages; currentlyapple/GramDriveSupport, the provider-support Swift package every GramDrive process links (App Group container resolution, shared-state access per process role, the cross-process change doorbell), which also ships thegramdrive-agentcompanion agent and thegramdrive-companionmenu-bar shell (authorization, status, cache/Archive settings, diagnostics, repair, removal —apple/GramDriveSupport/README.md)..research/— permanent research archive..task-board/and.planning/— project decomposition and generated execution plans..scripts/— reusable repo utilities..temp/— ignored local agent/runtime artifacts only.- Research documents cite primary-source URLs inline; verify against them before relying on a claim.
Every automated gate runs through one entrypoint, and CI invokes the same script with the same suite names — a gate that exists only inside a CI config cannot be run before pushing, and drifts the first time either side is edited.
make check # pre-push gate: the core suite plus the repo suite
make check-core # Rust core only: toolchain, format, lint, test, architecture, supply chain
make check-security # gitleaks secret scan of committed history (needs gitleaks)
make check-live-content # combined synthetic Rust/Swift date-first acceptance
make gates # print every suite and the exact command behind each step
make fmt # apply rustfmt (the gate only checks formatting)The non-shipping shared-state fixture seeder drives the coordinator's real Rust write path for cross-language smoke and File Provider boundary tests:
cargo run -p gramdrive-ffi --example shared_state_seed -- \
.temp/shared-state-fixture seed
cargo run -p gramdrive-ffi --example shared_state_seed -- \
.temp/generated-boundary-fixture generated-initialBoth modes write only beneath the supplied data root. The generated-boundary
mode is consumed by the Swift regression and emits synthetic identifiers plus
aggregate journal provenance; disposable outputs belong under .temp/.
make check is shorthand for
python3 .scripts/acceptance/run_automated.py --suite all --run-id local-all.
Every step runs even after one fails, because the useful output of a gate run is
the full list of what is broken. Each run writes provenance to
.temp/acceptance/<run-id>/ — summary.json (commit, worktree state, tool
versions, per-step exit codes and durations) plus one log per step. CI uploads
that directory as an artifact and passes --require-clean, which refuses to run
against a dirty worktree so the recorded commit describes what was actually
tested (NFR-052).
Prerequisites: rustup (it reads rust-toolchain.toml and installs the pinned
toolchain automatically), cargo-deny (brew install cargo-deny), and Python
3.11+. The toolchain step fails with an explicit message if any of them is
missing or is the wrong version, rather than letting an unpinned compiler
quietly produce a different verdict. The security suite additionally needs
gitleaks (brew install gitleaks); it is deliberately kept out of make check
so the everyday pre-push gate does not require it.
.github/workflows/ci.yml runs the same acceptance entrypoint on every pull
request (and on main after merge). It invents no step of its own — each job is
run_automated.py --suite <x> --require-clean, so a check that fails in CI fails
the same way locally:
| Job | Runner | Suite | Provenance artifact |
|---|---|---|---|
rust-core |
macos-15 (arm64, POL-5 reference host) |
all — toolchain, format, lint, test, architecture, cargo-deny (POL-6), traceability, script self-tests |
acceptance-ci-all |
secret-scan |
ubuntu-24.04 |
security — gitleaks over committed history |
acceptance-ci-security |
Design notes:
- Least privilege. The workflow grants
contents: readand nothing else; no job writes to the repo or mints a token. Release signing/attestation is a separate tag-triggered workflow. - Pinned, so cache cannot change a verdict. The Rust toolchain is pinned by
rust-toolchain.toml, dependencies byCargo.lock,cargo-denyandgitleaksby exact version (gitleaks additionally by sha256), and every action by commit SHA. The cargo cache is keyed on the toolchain and lockfile, so a hit can only ever hold artifacts built from identical inputs — it speeds up a run, it cannot alter its result. - Self-healing Rust bootstrap. Every Rust-consuming self-hosted job runs
.github/scripts/bootstrap-rust-toolchain.sh. If the runner-ownedrustupbinary is absent, it downloads the fixed rustup 1.29.0 archive and verifies its committed SHA-256 before execution, then installs the exact channel/profile/components fromrust-toolchain.toml, and publishes Cargo's bin directory throughGITHUB_PATH. It never sources a login profile, and a repeated invocation only reconciles the same pinned toolchain. - No secrets in logs. Neither job needs a repository secret, and the secret
scan runs gitleaks with
--redactso a matched value never reaches the uploaded log. Verified false positives are pinned by fingerprint in.gitleaksignore;.gitleaks.tomlis the committed, shared rule config. - Provenance. Each job uploads
.temp/acceptance/<run-id>/(if: always(),if-no-files-found: error, 14-day retention) so a result — green or red — is attributable to a commit (NFR-052).
Required checks (branch protection, one-time repo-admin setup). A workflow
file cannot make itself blocking. Mark rust-core and secret-scan as required
status checks on main so a pull request cannot merge while either fails.
.github/workflows/native-ci.yml extends the same pattern to the macOS native
drive (POL-5 / DEC-017 reference target). It is not the fast per-PR gate: its
jobs build TDLib from source and stage the core XCFramework, so it runs on a
schedule (nightly), on demand (workflow_dispatch), on main, and on
release-bound PRs (release/**) — which is what "release branches require native
acceptance evidence" needs without taxing every feature PR. Each job builds from
a clean checkout and, like core-ci, runs the pinned entrypoint (or a packaging
script that writes its own manifest), never an ad-hoc swift command list.
| Job | Runner | Suite / script | Provenance artifact |
|---|---|---|---|
tdlib |
macos-15 (arm64) |
pinned TDLib built from source (cache-first, keyed on the pin) + Rust link smoke | native-tdlib |
apple-build-test |
macos-15 (arm64) |
apple — swift build + swift test of apple/GramDriveSupport over the staged core (make check-apple) |
acceptance-ci-apple |
apple-package-unsigned |
macos-15 (arm64) |
build_app_bundle.py --unsigned — assembles GramDrive.app (nested appex, Info.plists, entitlement plists) with no Developer ID (make package-app-unsigned) |
native-app-package-unsigned |
Design notes:
- No secrets, ever. Native-ci only assembles an unsigned bundle. Signing and
notarization (the Developer ID identity in a keychain) live in the separate
tag-triggered release workflow (
TASK-260715-3bhbkv); this workflow keepspermissions: contents: readand consumes no repository secret. - TDLib cached on the pin. The
tdlibjob keys its cache onbuild_tdlib.py(which holds the pinned commit), so a warm run restores the artifact and re-runs only the fast link smoke; the from-source C++ build happens on a cold cache. - Support matrix (POL-5), documented not silently missing. macOS is the v1
native leg. iOS (
EPIC-260715-3uynbw), Windows (EPIC-260715-1mlv5j), Linux (EPIC-260715-1hnglv) and Android (EPIC-260715-y0fshx) legs are deferred with their backlog EPIC ids in the workflow header, and enter native-ci when the platform EPIC starts — not stubbed, because a build path nothing runs rots.
The bindings smoke (make smoke-bindings, not part of make check)
additionally needs swiftc (Xcode command line tools), kotlinc
(brew install kotlin), and Java 17+ (brew install openjdk); it downloads
its two JVM runtime jars (JNA, kotlinx-coroutines) from Maven Central once,
pinned by version and sha256 in the runner script.
The shared-state smoke (make smoke-shared-state, also not part of
make check) needs Xcode: it runs a Rust coordinator process, two
concurrent Swift provider processes, and a change-watcher process over one
substitute App Group container, through the packaged artifact (staging
make package first when none is present).
The agent-lifecycle smoke (make smoke-agent-lifecycle, also not part of
make check) needs Xcode: it runs the gramdrive-agent companion binary
as real processes over a substitute container and proves the lifecycle
contract — health over the bounded IPC channel, single-instance refusal of
a second agent, SIGTERM drain of a hosted transfer, and instant successor
startup after SIGKILL.
.github/workflows/release.yml is the tag-triggered release (TASK-260715-3bhbkv):
push a v* tag and it produces one signed, notarized GramDrive.app + .dmg and
its provenance. It is the only workflow with elevated permissions
(contents: write, id-token: write, attestations: write) and the only one
that holds a secret, so it is isolated to that single trigger. It re-implements no
signing command — it invokes the same reusable scripts native-ci proves
unsigned.
Pipeline (macos-15, environment: release):
- Supply-chain gate first (POL-6).
run_automated.py --suite supply-chainfails the release closed — before any credential is imported — if a dependency pulls in a disallowed license or an open advisory. - Import into a throwaway keychain. The Developer ID Application identity
(from
MACOS_CERT_P12/MACOS_CERT_PASSWORD) and thegramdrive-notaryprofile (fromAPPSTORE_KEY_ID/APPSTORE_ISSUER_ID/APPSTORE_PRIVATE_KEY) go into oneRUNNER_TEMPkeychain, deleted in analways()step. No secret is echoed; no key material touches the workspace, a cache, or the login keychain. - Build the signed artifact.
build_app_bundle.py --notarizesigns inside-out with the hardened runtime + a trusted timestamp, then notarizes and staples both the.appand the.dmg(so an app dragged out of the dmg verifies offline). Version stamping reads the reviewed three-componentapple/GramDriveSupport/Version.json; ordinary local packaging uses the git rev-count forCFBundleVersion. Candidate CI selects the greater of that floor and the highest applicable public feed build plus one, then revalidates the exact selection immediately before immutable handoff (Sparkle ordering). - Build the release provenance.
build_release_provenance.pyemits the CycloneDX SBOM (POL-6 dependency inventory), a changelog since the last tag, rollback metadata, a release manifest tying every artifact to a sha256, and a credential scrub that fails the release if any produced file looks like it carries a secret. - Attest + publish.
actions/attest-build-provenancerecords an OIDC-bound attestation for the dmg (verifiable withgh attestation verify), thengh release createpublishes the dmg, checksums, SBOM, changelog, rollback and release manifest with the changelog as the release notes.
The human approval gate (POL-8 / DEC-020: public release is the single
mandatory human stop) is environment: release — its required-reviewer protection
rule (owner sign-off) is a repo-admin setting, the same transparent limitation the
required-status-check rule has above.
Local dry-run of the non-signing half (needs a prior make package-app /
make package-app-unsigned for the manifest; no signing identity, Xcode, or
network — only git and cargo):
make release-provenance # SBOM + changelog + rollback + release manifest + scrub → .temp/release/make package # build + verify the artifacts native hosts consume
make package-reproducible # build the shipped library at two paths, compare bytesmake package produces a self-contained SwiftPM package in .temp/packaging/:
GramDriveCore.xcframework (macOS 14+ arm64 static library plus headers), the
generated Swift bindings, a manifest, and checksums — then proves it by
resolving and running a real minimal Swift package against the result. Needs
Xcode (xcodebuild, swift); like the smoke it is not part of make check,
because it needs a release build and produces artifacts rather than a verdict on
the source.
Per platform: macOS consumes the XCFramework; Windows and Linux consume
the gramdrive-ffi crate directly as a Rust dependency and need no artifact at
all (which is why the crate keeps its rlib crate-type); Android (.so +
Kotlin) and iOS slices are deferred until those platforms enter scope
(POL-5/DEC-017), not stubbed — a build path nothing runs is a build path that
rots. Full rationale, the measured reproducibility and size numbers, and the
crate-type and debug-info decisions: .scripts/packaging/README.md.
GramDrive is licensed under the Apache License 2.0. Attribution and notice obligations are collected in NOTICE. Contributions are accepted under that license; see CONTRIBUTING.md, the Code of Conduct, and SECURITY.md for the contribution, community, and private-vulnerability-reporting policies.
The repository's private-to-public conversion is controlled by
the public-readiness checklist. In
particular, public release and Sparkle-feed publication require reviewer
acceptance; private-era v0.1.0 and v0.1.1 are retained as GitHub drafts
(with their tags and assets preserved) and are excluded from all feeds.
Update credential bootstrap, independent rotation/revocation, versioned-feed bridges, emergency freeze, and forward recovery are documented in the update operations runbook. It provides stdin/owner-only-file setter commands and a value-free inventory preflight.
Available utilities:
| Tool | Purpose | Run | Output |
|---|---|---|---|
.scripts/release/check_update_secret_inventory.py |
Value-safe bootstrap/rotation setter plus initial-inventory preflight for the seven update-delivery names and versioned Sparkle generations; setter values arrive only via stdin or owner-only files and are never argv/output | make updates-secret-inventory; make updates-secret-inventory CHECK_GITHUB=1; for provisioning, use the exact owner-only-file and generate_keys --account … -p / -x <owner-only-private-file> sequences in docs/UPDATE_OPERATIONS.md with --set, --set-developer-id-from, and --set-notary-from. The same runbook gives exact post-bridge gh secret delete NAME --env ENV and account-scoped Keychain cleanup commands. |
Exit 0 when the requested storage/preflight action succeeds; exit 1 on invalid input, missing/unexpected names, or unavailable/failed gh; no credential-bearing artifacts |
.scripts/acceptance/run_automated.py |
The single gate entrypoint: runs a named suite, records provenance. Used identically by make and by CI |
python3 .scripts/acceptance/run_automated.py --suite core --run-id local-core; --list prints suites and steps |
Exit 0 pass / 1 failed step / 2 could not start; .temp/acceptance/<run-id>/ with summary.json + per-step logs |
.scripts/acceptance/run_live_content.py |
Privacy-safe pre-install acceptance matrix composing the focused Rust history/render/fidelity/hydration/story/retention suites with the full Swift package/provider regressions. Child output is discarded; evidence is allow-listed and bounded | make check-live-content, or through run_automated.py --suite live-content --run-id local-live-content after staging make package |
Exit 0 all matrix legs passed / 1 one or more failed; .temp/acceptance/<run-id>/live-content.json contains only fixed labels, counts, booleans, timings, versions, and bounds |
.scripts/acceptance/run_installed_live_content.py |
Privacy-safe installed-profile hierarchy, one-read hydration, generated-cache, additive identity, cursor, metadata, namespace, and relaunch proof. Each phase has fixed stage timings and a 120-second hard deadline; TERM, exact-group SIGKILL, leader reap, and pipe drain are bounded by that same deadline. The schema-v25 partial index streams at most 20 small live attachment candidates without an account-wide temp sort; each Finder st_flags probe is isolated to a 500ms child and classified as dataless, materialized, missing, platform error, timeout, or identity/path mismatch. A missing indexed item gets one exact-ItemId URL resolution through the installed companion's bounded, pre-AppKit --acceptance-resolve-placeholder command; the command validates with the authoritative core parser and round-trips the returned URL to the same provider identity before the harness performs an exact dataless re-probe. Bulk identity/cursor evidence stays in an indexed private SQLite sidecar, while generated references are streamed and physical inventory is capped at 1,000,000 entries |
python3 .scripts/acceptance/run_installed_live_content.py {before|stability-snapshot|after} --state <private.json> --evidence <public.json> [--deadline-seconds 120]; preserve the same private state and relaunch without resetting the profile between phases |
Exit 0 only when every phase-specific assertion passes; exit 1 on a fixed acceptance, bounded selection, stage-timeout, or worker-cleanup-deadline-exceeded category; exit 2 for an invalid deadline. Public JSON contains fixed aggregate fields/timings and placeholder-state counts only; private JSON plus <state>.snapshot.sqlite3 contain comparison keys |
.scripts/acceptance/run_installed_index_metadata.py |
Installed authorized-profile probe for the historical chat index (BUG-260728-2qfzbd): how much of the listed backlog background history work can actually reach, whether every chat/month directory publishes a size rollup equal to its indexed descendants, whether any directory is still undated (the state Finder renders as 1 Jan 1970) as distinct from faithfully epoch-dated, and whether every cursor window stayed monotonic across an app + agent relaunch. The rollup and date checks are scoped to the kinds that own a rollup — chat, month, Active Stories — because a chat list or folder catalog is deliberately left NULL. Opens no content and downloads nothing |
python3 .scripts/acceptance/run_installed_index_metadata.py {before|after|relaunch} --output <file.json> --private <dir> --now-ms <ms> |
Exit 0 all acceptance booleans true / 1 any false; the JSON output holds counts, booleans, and byte totals only, while salted cursor digests stay under --private. A phase that asserts nothing (before) reports "passed": null, not true |
.scripts/acceptance/run_installed_foreground_demand.py |
Installed authorized-profile probe for the foreground-demand path (BUG-260728-2qfzbd): whether using a chat in Finder buys it a history turn, with no control-socket hint anywhere. It picks the reachable incomplete chat the background rotation will reach last, watches it for one window untouched, performs one gesture, then watches it again — so a turn the rotation would have handed out regardless fails the probe instead of passing it. --gesture read (default) reads one generated document inside the chat, which is the interaction that reliably reaches the extension; --gesture open only lists the folder, which macOS usually answers from its own replica without calling the extension at all. It also reads the agent's hint counters either side of the gesture. Downloads no Telegram payload bytes |
python3 .scripts/acceptance/run_installed_foreground_demand.py --output <file.json> --private <file.json> [--gesture read|open] [--window <seconds>] [--socket <path>] |
Exit 0 when the gesture's boolean (content_read_granted_a_turn / foreground_open_granted_a_turn) is true, 1 when false; the JSON output holds counts, booleans, seconds, and message deltas only, while the chosen chat's id, folder, document, and raw cursor readings stay in --private |
.scripts/acceptance/run_installed_generated_hydration.py |
Installed authorized-profile saturation probe for 20 distinct dataless generated documents across Markdown, NDJSON, and chat JSON, with exact-cache verification, fixed per-read deadlines, backfill/hint/cursor fairness, and a before/after relaunch identity check | python3 .scripts/acceptance/run_installed_generated_hydration.py before --private <private.json> --evidence <before.json>; relaunch without resetting the profile/domain, then repeat with after, the same private path, and a new evidence path |
Exit 0 only for zero timeouts/errnos, exact bytes, p95 <1s, p99 <3s, saturated backfill, balanced requested/background hints, a target chat turn, monotonic cursors, and preserved identities. Public evidence is aggregate-only; identifiers, paths, and digests stay private |
.scripts/acceptance/run_native_macos.py |
The macOS native manual acceptance harness (TASK-260715-3oe2nr): the ten File Provider Finder flows the release gate requires (register, enumerate, hydrate, cancel, pin, update, restart, repair, upgrade, remove). One scenario catalog drives the run-sheet, the evidence form, and the machine probes. Human-in-the-loop by necessity — it preflights the host, captures evidence, and never reports a scenario passed; a person runs Finder and signs off. Pipeline: .scripts/acceptance/README.md |
make accept-macos, make accept-macos-runsheet, or python3 .scripts/acceptance/run_native_macos.py --run-id accept-YYYY-MM-DD [--app-path …] [--require-ready]; --list prints the catalog |
Exit 0 prepared (not a pass) / 2 could not start / 3 --require-ready host off-matrix; .temp/acceptance/<run-id>/ with runsheet.md, evidence-template.md, summary.json + per-probe evidence logs |
make |
Shorthand for the entrypoint plus the non-gate inner loop (fmt, build, test) |
make check, make check-core, make check-repo, make gates |
Delegates to the entrypoint; never re-defines a gate command |
cargo (pinned to Rust 1.91.0 by rust-toolchain.toml, edition 2024) |
Build and test the shared core workspace | cargo build --workspace / cargo test --workspace (repo root); per-crate commands in each crates/*/README.md |
Binaries/test results under target/ (gitignored) |
rustfmt + clippy (pinned components) |
Formatting and lints. Config: rustfmt.toml, clippy.toml, and [workspace.lints] in Cargo.toml — levels live in the manifest so editors agree with the gate |
cargo fmt --all to fix; the format and lint gate steps to check |
Exit non-zero on a formatting diff or any warning (-D warnings) |
swift + swift-format |
Build/test the macOS support package and lint task-scoped Swift changes using the repository's existing four-space style and accepted retroactive File Provider conformance | After make package: cd apple/GramDriveSupport && swift test; lint a changed file with swift-format lint --strict --configuration '{"version":1,"indentation":{"spaces":4},"lineLength":100,"respectsExistingLineBreaks":true,"rules":{"AvoidRetroactiveConformances":false}}' PATH |
SwiftPM artifacts and coverage profiles under apple/GramDriveSupport/.build/; formatter emits diagnostics only |
.scripts/check_toolchain.py |
Asserts the pinned toolchain is actually in effect — rust-toolchain.toml only binds when rustup drives cargo — and that cargo-deny meets the minimum version |
python3 .scripts/check_toolchain.py (repo root; stdlib only) |
Exit 0 + summary line, or exit 1 with itemized errors (CI-suitable) |
.github/scripts/bootstrap-rust-toolchain.sh |
Restores missing rustup on a clean self-hosted macOS runner with the fixed Rustup 1.29.0 archive verified by committed SHA-256, then installs the channel/components from rust-toolchain.toml and exports Cargo's bin directory for later CI steps |
.github/scripts/bootstrap-rust-toolchain.sh (GitHub Actions macOS job; requires curl and shasum) |
Exit 0 with active toolchain, rustc, and cargo versions; no shell-profile mutation or execution of an unverified download |
.scripts/check_crate_architecture.py |
Enforces crates/README.md: dependency direction, no cycles, no platform leakage in core crates, testkit dev-only, per-crate README sections, shared lint-set opt-in |
python3 .scripts/check_crate_architecture.py (repo root; stdlib only, needs cargo on PATH) |
Exit 0 + summary line, or exit 1 with itemized errors (CI-suitable) |
cargo-deny (installed via brew install cargo-deny) |
Supply-chain gate, config in deny.toml: POL-6 licenses (permissive-only), RustSec advisories, bans, and sources (crates.io only) |
cargo deny check (repo root), or one check: cargo deny check licenses |
advisories ok, bans ok, licenses ok, sources ok, or non-zero exit with the offending dependency tree |
.scripts/validate_traceability.py |
Validates docs/TRACEABILITY.md against .spec/ and .task-board/: every requirement mapped exactly once, no orphan board elements, no stale requirement references on the board |
python3 .scripts/validate_traceability.py (repo root; stdlib only) |
Exit 0 + summary line, or exit 1 with itemized errors (CI-suitable) |
.scripts/tests/ |
Self-tests for the gate scripts themselves — an untested runner is a gate with no gate | python3 -m unittest discover -s .scripts/tests -t .scripts/tests, or the scripts gate step |
Standard unittest output |
uniffi-bindgen (workspace-local, crates/gramdrive-ffi/src/bin/) |
Generates Swift + Kotlin bindings from the compiled library, version-locked to the linked uniffi crate; pipeline documented in crates/gramdrive-ffi/README.md |
make bindings, or cargo run -p gramdrive-ffi --features bindgen --bin uniffi-bindgen -- generate --library target/debug/libgramdrive_ffi.dylib --language swift --language kotlin --out-dir .temp/bindings |
Generated sources in .temp/bindings/ (build artifacts, never committed) |
.scripts/smoke/run_bindings_smoke.py |
End-to-end bindings smoke: builds the FFI library, generates bindings, compiles and runs the Swift and Kotlin smoke consumers (.scripts/smoke/{swift,kotlin}/) asserting async, progress, error, and cancellation round-trips |
make smoke-bindings, or python3 .scripts/smoke/run_bindings_smoke.py [--skip-swift] [--skip-kotlin] (needs swiftc, kotlinc, java) |
Exit 0 + BINDINGS SMOKE PASSED, or non-zero with the failing step's log; artifacts and per-step logs in .temp/bindings-smoke/ |
.scripts/smoke/run_shared_state_smoke.py |
Multi-process shared-state smoke (TASK-260715-gnsa2s): a Rust coordinator process seeds a substitute App Group container, two concurrent Swift provider processes (apple/GramDriveSupport over the packaged artifact) must read byte-identical item metadata, and a watcher process must observe the Darwin change doorbell plus the dataVersion probe across a foreign commit |
make smoke-shared-state, or python3 .scripts/smoke/run_shared_state_smoke.py [--repackage] (macOS; needs Xcode; stages make package when no artifact is present) |
Exit 0 + SHARED-STATE SMOKE PASSED, or non-zero with the failing step's output; container and per-step logs in .temp/shared-state-smoke/ |
.scripts/packaging/build_core_artifacts.py |
Builds what native consumers ship against: release staticlib (LTO restored via a crate-type override), Swift bindings generated from that exact binary, XCFramework, manifest (contract version read from the built artifact, git describe, toolchain), checksums, and a deterministic zip; verifies it all by resolving and running a real minimal SwiftPM package (.scripts/packaging/swift-consumer/). Owns the shipped-target list |
make package, make package-reproducible, or python3 .scripts/packaging/build_core_artifacts.py [--skip-verify] [--check-reproducible] (macOS; needs xcodebuild, swift) |
Exit 0 + PACKAGING PASSED, or non-zero with the failing step's log; artifacts, manifest.json, CHECKSUMS.sha256 and per-step logs in .temp/packaging/ |
.scripts/tdlib/restore_pinned_artifact.py |
Restores the recipe-keyed runner-local arm64 TDLib artifact on the dedicated x86_64 CI/signing runner; refuses a cold cache, symlinks, wrong pin/target/install-name/architecture, missing or mismatched static-OpenSSL attribution, and any file/checksum/manifest inventory mismatch before atomically replacing the staged artifact | python3 .scripts/tdlib/restore_pinned_artifact.py [--cache-root DIR] [--out-dir DIR]; CI follows it with make tdlib-smoke-link |
Exit 0 plus the privacy-safe cache key and library SHA-256; verified artifact in .temp/tdlib/out, or exit 2 for an unseeded cache |
.scripts/release/build_release_provenance.py |
The release provenance bundle (TASK-260715-3bhbkv), derived from the signed .app manifest + git history + cargo metadata: CycloneDX SBOM (POL-6 dependency inventory, license per crate), changelog since the last tag, rollback metadata, a release manifest tying every artifact to a sha256, and a credential scrub that fails on any secret-shaped content. POL-6 is enforced by cargo deny (core CI), not re-adjudicated here. Invoked by the tag-triggered release.yml |
make release-provenance, or python3 .scripts/release/build_release_provenance.py [--package-dir DIR] [--out-dir DIR] [--tag vX.Y.Z] (needs a prior packaging run for the manifest; only git + cargo, no signing/Xcode/network) |
Exit 0; sbom.json, CHANGELOG.md, rollback.json, release-manifest.json, RELEASE-CHECKSUMS.sha256 in .temp/release/ |
.scripts/release/build_candidate_package.py |
Candidate trust-boundary verifier (TASK-260810-fwgmr4): rejects non-live, wrong-channel, non-arm64, non-monotonic, incorrectly identified, incompletely signed/notarized/stapled/Gatekeeper-assessed inputs; requires complete per-Mach-O architecture/team/authority readback and exact app/core/TDLib checksum inventories; copies the exact DMG bytes under the immutable version-and-build asset name; emits privacy-safe provenance; binds the subject inventory to the GitHub Sigstore bundle; and re-verifies the immutable handoff | Used by .github/workflows/candidate-build.yml; downstream verification is python3 .scripts/release/build_candidate_package.py verify --out-dir PATH |
Exact build-named DMG plus app/core/TDLib manifests and checksums, candidate-manifest.json, candidate-provenance.json, verification.json, SUBJECTS.sha256, candidate-attestation.json, finalization.json, and CANDIDATE-CHECKSUMS.sha256 in .temp/candidate/ |
.scripts/release/publish_sparkle.py |
Deterministic Sparkle publication verifier and renderer: validates exact candidate intake, endpoint-isolated appcasts, authenticated complete-site archives, and the stable generation state machine. Stable candidates and tags bind to reviewed .github/sparkle-stable.json; authenticated prior-site state permits only same-generation promotion or a one-time forward rotation with an old-key bridge. Workflows source .github/scripts/run-with-optional-file-argument.sh so initial publication and prior-feed updates are both safe under the runner's Bash 3.2 with set -u. Stable promotion freezes the authenticated site as immutable Release assets on macOS and stops without Pages capability. A separately approved redeploy-site dispatch from protected main checks out the exact dispatch SHA, re-verifies those immutable assets on Ubuntu, uploads the authenticated site, and hands it to the isolated Pages deployment job without exposing the stable key or Release mutation capability |
Used by .github/workflows/candidate-build.yml and .github/workflows/release.yml; focused regression command: python3 .scripts/tests/test_publish_sparkle.py; inspect subcommands with python3 .scripts/release/publish_sparkle.py --help |
Exit 0 plus SPARKLE PUBLICATION … PASSED, or exit 1 fail-closed; workflow artifacts remain under run-scoped .temp/publication/ and .temp/stable/ |
.scripts/release/release_asset_inventory.py |
Deterministic Release rerun guard: fully captures gh release view --json assets, normalizes one safe unique name per asset, and rejects CLI, JSON, schema, unsafe-name, or duplicate-inventory failures before immutable upload decisions. Existing assets remain download-and-compare only; only an explicit absent state may enter the upload branch. Stable dispatch seals the helper's exact git-object bytes from the protected-main workflow revision before detaching to an older immutable tag; push tags seal it from their exact tag workflow revision. Every use rechecks canonical runner-temp containment and both workflow-output/environment SHA-256 bindings |
python3 .scripts/release/release_asset_inventory.py capture --release TAG --output FILE, then python3 .scripts/release/release_asset_inventory.py state --inventory FILE --name ASSET; used directly by test publication and through the sealed workflow-control path by stable publication |
Normalized JSON inventory plus present or absent; untrusted workflow refs, missing/tampered/path-escaped helpers, hash mismatches, or indeterminate inventory state exit nonzero before Release mutation |
.scripts/release/select_stable_release.py |
Deterministic offline selector for the complete paginated GitHub Releases API response. It strictly flattens the gh api --paginate --slurp page array and validates stable publication timestamps as timezone-aware RFC3339. Candidate preflight chooses the newest published exact-semver stable Release as a fail-closed state head and requires singular manifest/attestation evidence. Stable publication instead excludes its current source tag, refuses any newer published semver, and restores the newest prior site's singular complete asset set, so current Releases remain resumable when absent, partial, or complete without falling back past an incomplete predecessor |
Workflows save the paginated page array to a file, then run python3 .scripts/release/select_stable_release.py --release-pages FILE --mode candidate-state-head, or --release-pages FILE --mode stable-prior --current-tag vX.Y.Z; no standalone jq prerequisite |
Exit 0 and the selected tag (or empty output when no state exists); malformed pages, records, timestamps, duplicated/incomplete evidence, or incompatible state exits non-zero before download or mutation |
.scripts/release/check_candidate_build_order.py |
Read-only monotonic-build selector and immutable-handoff guard for candidate production; test candidates inspect the rolling test feed, while stable candidates derive every versioned stable endpoint through the active generation from .github/sparkle-stable.json. The latest published semver stable Release is the fail-closed state head: its GitHub-attested prior-site manifest must have an exact canonical archive/file/key inventory, and each recorded Pages feed must match the attested SHA-256 and byte count before its builds can raise the floor. An unrecorded next generation is 404-only. Present feeds accept exactly one canonical positive build on each direct RSS item, using an attribute-free child-free exact Sparkle namespace leaf or the legacy enclosure attribute; malformed/unavailable feeds fail closed |
Selection: python3 .scripts/release/check_candidate_build_order.py --mode test|stable-candidate --git-build N [--stable-site-manifest VERIFIED.json]; final race check: repeat with --candidate-build N |
Exit 0 and the highest applicable published numeric build; selection also writes build_number=max(git_build, published_highest+1) to GITHUB_OUTPUT; candidate and stable workflows share a non-cancelling concurrency group, and validation immediately before artifact upload exits non-zero if the serialized public floor consumed that build; no feed mutation |
.scripts/apple-app/build_app_bundle.py |
Assembles, signs (Developer ID, hardened runtime, inside-out), embeds/signs Sparkle 2.9.5 and its helpers, notarizes + staples the .app and .dmg, and records a manifest with per-binary entitlements/cdhashes + checksums. Pipeline: .scripts/apple-app/README.md |
make package-app (sign+verify, no notarize), make package-app-unsigned (assemble only), make package-app-notarize (full), or `python3 .scripts/apple-app/build_app_bundle.py --update-channel test |
stable [--build-number N] [--notarize] [--unsigned] [--notary-keychain PATH]` (macOS; needs Xcode, a Developer ID identity, and staged core; candidate CI alone supplies the reviewed public-feed-aware build override; each channel's reviewed public trust anchor is checked in) |
.scripts/smoke/run_agent_lifecycle_smoke.py |
Multi-process agent-lifecycle smoke (TASK-260715-1yx9ly): the gramdrive-agent companion binary over a substitute container — startup with health served over the bounded UNIX-socket IPC, single-instance refusal (exit 2) of a second agent, SIGTERM drain cancelling a hosted transfer through its token (exit 0, endpoint removed), and a successor starting immediately after SIGKILL with healthy durable state |
make smoke-agent-lifecycle, or python3 .scripts/smoke/run_agent_lifecycle_smoke.py [--repackage] (macOS; needs Xcode; stages make package when no artifact is present) |
Exit 0 + PASSED: agent lifecycle smoke, or non-zero with the failing step's output; container and per-step logs in .temp/agent-lifecycle-smoke/ |
.scripts/smoke/run_control_auth_smoke.py |
Control-channel auth smoke (BUG-260720-3i74u1): real end-to-end sign-in against Telegram's test DC through the packaged bundle's gramdrive-agent — the control socket up, phone → code → ready over the same NDJSON wire the companion speaks, the durable account row in status, and after an agent restart the stored session still authorizing (repair completes). Since mid-2025 Telegram rejects the documented auto-code for shared 99966XYYYY numbers with third-party api ids (tdlib/td#3361), so completing sign-in needs --phone with a dedicated test-DC account (created once via an official app in test mode) and one interactive code entry; the kept authorized session makes later legs and re-runs unattended |
make smoke-control-auth, or python3 .scripts/smoke/run_control_auth_smoke.py [--agent PATH] [--dc N] [--attempts N] [--keep] [--phone +NUMBER] (macOS; needs make package-app, keychain api credentials readable by the packaged agent — see the keychain provisioning row — and network to the test DC) |
Exit 0 + control-auth smoke: PASS with the signed-in test account; agent logs in .temp/control-auth-smoke/ |
.scripts/smoke/run_sparkle_manual_update_smoke.py |
Packages an isolated LSUIElement accessory host against pinned Sparkle 2.9.5, signs a local appcast and enclosure with a fixture-only EdDSA key, and proves the production manual-update action presents a visible key-capable standard Sparkle window from zero windows within a bounded deadline. Shipping feed URLs and keys are never used or changed |
make smoke-sparkle-manual-update, or run make package followed by python3 .scripts/smoke/run_sparkle_manual_update_smoke.py [--work-dir .temp/PATH] on macOS |
Exit 0 + sparkle manual update smoke: PASS; privacy-safe JSON, local feed, isolated preferences, and the packaged fixture host in .temp/sparkle-manual-update-smoke/ |
.scripts/keychain/provision_telegram_credentials.py |
Rewrites the gramdrive-telegram keychain items (api_id/api_hash, TASK-260716-1iypv4) so the signed product binaries read them without a consent prompt: compiles and Developer ID-signs the companion Swift tool, which recreates the items with the team's partition list and a trusted-application ACL naming the packaged gramdrive-agent/GramDrive (items made by the security CLI are partition-locked to apple-tool: and always prompt) |
python3 .scripts/keychain/provision_telegram_credentials.py [--identity NAME] [--bundle PATH] (macOS; needs the packaged bundle, a Developer ID identity, and the values in GRAMDRIVE_API_ID/GRAMDRIVE_API_HASH env or in the existing keychain items) |
Exit 0 + provisioned: ...; secrets never touch argv, logs, or the repo |
.scripts/tdlib/build_tdlib.py |
Reproducible build of the pinned TDLib tdjson artifact the local Telegram source links against (BSL-1.0 recorded per POL-6): fetch at the pinned commit, CMake build, staged libtdjson.dylib + headers + license, manifest and checksums, proved by the link-smoke/ Rust binary; pipeline documented in .scripts/tdlib/README.md |
make tdlib, make tdlib-smoke (re-run only the link smoke), make tdlib-verify (same-path reproducibility), or python3 .scripts/tdlib/build_tdlib.py (macOS arm64; needs Xcode clang, cmake, gperf, Homebrew openssl@3) |
Staged artifact, manifest.json and CHECKSUMS.sha256 in .temp/tdlib/out/; smoke prints the running library's version |
make tdjson-smoke |
Real-linkage smoke of the gramdrive-source-tdjson runtime (crate docs: crates/gramdrive-source-tdjson/README.md): with GRAMDRIVE_TDLIB_ARTIFACT_DIR set, the crate's env-gated build.rs links the staged libtdjson.dylib and the otherwise-empty real_tdjson_smoke test drives correlation, client close, and shutdown against the real library. Every make check runs the same runtime mock-only, artifact-free |
make tdjson-smoke (after make tdlib staged the artifact) |
cargo test output: 1 test against the real library, exit non-zero on failure |