Skip to content

Latest commit

 

History

156 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

GramDrive

Concept: expose a user's Telegram account as a file tree — folder per chat (named after the chat/user, in Telegram dialog order), containing all downloaded media plus the chat history exported as text. The primary product is a native drive resembling Dropbox/Google Drive:

  • desktop and mobile apps that integrate with Finder, Explorer, iOS Files, Android's system picker, and Linux mounts;
  • the Telegram engine embedded per device for desktop/Android v1, with an interchangeable remote source retained for iOS cold hydration, self-hosting, or a later hosted tier;
  • no web application or rich Telegram-like UI requirement for the initial product.

Status

Early implementation. The technology survey, specification baseline, and complete service decomposition are committed; architecture decisions marked provisional still require their explicit decision tasks. Product code so far: the shared Rust core workspace skeleton (crates/, see below) — crate boundaries, dependency-direction rules, and quality gates; domain logic is still to come.

Start with the specification index and the generated project plan. Product implementation is intentionally deferred until the plan is reviewed and approved.

Naming (DEC-019, POL-7): the public product name is GramDrive, and every shipped identifier is derived from the com.reluxworks.gramdrive namespace. tgfs remains the internal repository/codename only — it collides with TheodoreKrypton/tgfs and must not appear in user-visible strings, marketing, or store listings. The repository is deliberately not renamed. Trademark/handle check happens before public release.

Current direction (from research, 2026-07-15)

  • Shared client core: Rust — virtual tree, local SQLite/cache, change cursors, hydration, range downloads, retry/recovery, offline state, naming and generated files. Swift/Kotlin bindings through UniFFI; direct use on Windows/Linux. Verified precedents: Element X (matrix-rust-sdk), Dropbox Nucleus, Firefox (.research/260715-shared-core-feasibility.md).
  • Telegram source — behind one provider-neutral Rust trait, two interchangeable implementations:
    • Local-first: TDLib (BSL-1.0) embedded per device via tdjson FFI — zero infrastructure, Dropbox UX on desktop/Android; on iOS TDLib fits only in the main app (FP extension ~20 MB cap), and TDLib has no takeout API (normal-API backfill with flood-wait pacing).
    • Remote: Go + gotd/td (MIT) — optional for takeout backfill, one canonical archive, and iOS cold hydration; requires an always-on instance (self-hosted or SaaS with auth-key custody). Patterns from iyear/tdl (study only — AGPL); Telethon (Codeberg, MIT) is the takeout-worker alternative.
  • OS integration: thin native adapters: File Provider replicated extensions (iOS/macOS, Swift), Cloud Files API (Windows, Rust), DocumentsProvider/SAF (Android, Kotlin), FUSE (Linux, Rust). Read-only first.
  • UI: native and minimal; web and rich chat UI deferred.
  • Storage: SQLite/PostgreSQL canonical metadata + content-addressed blob store; text export as NDJSON (lossless) + Markdown (human-readable).
  • Treat official Telegram clients as reference implementations. A commercial GPL fork is legally possible, but a proprietary/closed-source fork is generally incompatible with their copyleft obligations and needs a deliberate licensing review.

Research artifacts

Artifact Contents
.spec/architecture.md Native-drive architecture with the shared Rust core and interchangeable local/remote sources
.spec/policies.md Accepted product policies POL-1…POL-8 (ordering, media/Archive Mode, retention, restricted content, support matrix, licensing, naming, approval gates)
.research/260715-telegram-filesystem-landscape.md Synthesized library, API, platform, architecture, and prior-art report
.research/260715-core-libraries.md MTProto/TDLib library landscape
.research/260715-oss-clients.md Official/OSS Telegram clients, licenses, and API terms
.research/260715-filesystem-integration.md File Provider / Cloud Files / SAF / FUSE survey
.research/260715-prior-art.md Exporter, archive, Telegram-FUSE, and WebDAV prior art
.research/260715-shared-core-feasibility.md Rust/UniFFI precedents, TDLib extension constraints, gomobile and grammers analysis
docs/OPEN_QUESTIONS.md Open product and architecture decisions
docs/TELEGRAM_API_COMPLIANCE.md Telegram API terms → verifiable controls, rule-to-task mapping (TGC-nn)
docs/TRACEABILITY.md Requirement coverage matrix: every PRD/DOM/SYNC/PLAT/SEC/NFR/DEC/POL ID mapped to board elements

Delivery decomposition

The canonical local board is stored in .task-board/ and must be changed through the task-board CLI. The current baseline contains 11 epics, 53 stories, and 142 atomic tasks; all remain unstarted.

Human-only work is isolated in the manual-actions epic (EPIC-260716-3vc5ay): product decisions and ADR ratification, external credentials (Telegram api_id/api_hash, Apple signing assets, Windows signing identity, test devices), and manual on-device release validation. Every other epic is designed to run autonomously in an agent loop once its manual-actions dependencies are done.

The project-level dependency plan has four phases:

  1. manual decisions/credentials plus autonomous product-foundation analysis;
  2. shared Rust core;
  3. local TDLib source and the optional remote tier;
  4. native drive integrations plus cross-platform quality, security, and release work.

Detailed generated plans for every epic are in .planning/. The remote tier is decomposed to preserve interface and sizing clarity, but remains optional and does not authorize hosted-service implementation.

Tools

Conventions:

  • .spec/ — product and architecture source of truth.
  • crates/ — the shared Rust core workspace (crates/README.md documents layers, dependency direction, and feature policy).
  • apple/ — Apple-native packages; currently apple/GramDriveSupport, the provider-support Swift package every GramDrive process links (App Group container resolution, shared-state access per process role, the cross-process change doorbell), which also ships the gramdrive-agent companion agent and the gramdrive-companion menu-bar shell (authorization, status, cache/Archive settings, diagnostics, repair, removal — apple/GramDriveSupport/README.md).
  • .research/ — permanent research archive.
  • .task-board/ and .planning/ — project decomposition and generated execution plans.
  • .scripts/ — reusable repo utilities.
  • .temp/ — ignored local agent/runtime artifacts only.
  • Research documents cite primary-source URLs inline; verify against them before relying on a claim.

Running the checks

Every automated gate runs through one entrypoint, and CI invokes the same script with the same suite names — a gate that exists only inside a CI config cannot be run before pushing, and drifts the first time either side is edited.

make check          # pre-push gate: the core suite plus the repo suite
make check-core     # Rust core only: toolchain, format, lint, test, architecture, supply chain
make check-security # gitleaks secret scan of committed history (needs gitleaks)
make check-live-content # combined synthetic Rust/Swift date-first acceptance
make gates          # print every suite and the exact command behind each step
make fmt            # apply rustfmt (the gate only checks formatting)

The non-shipping shared-state fixture seeder drives the coordinator's real Rust write path for cross-language smoke and File Provider boundary tests:

cargo run -p gramdrive-ffi --example shared_state_seed -- \
  .temp/shared-state-fixture seed
cargo run -p gramdrive-ffi --example shared_state_seed -- \
  .temp/generated-boundary-fixture generated-initial

Both modes write only beneath the supplied data root. The generated-boundary mode is consumed by the Swift regression and emits synthetic identifiers plus aggregate journal provenance; disposable outputs belong under .temp/.

make check is shorthand for python3 .scripts/acceptance/run_automated.py --suite all --run-id local-all. Every step runs even after one fails, because the useful output of a gate run is the full list of what is broken. Each run writes provenance to .temp/acceptance/<run-id>/summary.json (commit, worktree state, tool versions, per-step exit codes and durations) plus one log per step. CI uploads that directory as an artifact and passes --require-clean, which refuses to run against a dirty worktree so the recorded commit describes what was actually tested (NFR-052).

Prerequisites: rustup (it reads rust-toolchain.toml and installs the pinned toolchain automatically), cargo-deny (brew install cargo-deny), and Python 3.11+. The toolchain step fails with an explicit message if any of them is missing or is the wrong version, rather than letting an unpinned compiler quietly produce a different verdict. The security suite additionally needs gitleaks (brew install gitleaks); it is deliberately kept out of make check so the everyday pre-push gate does not require it.

Continuous integration

.github/workflows/ci.yml runs the same acceptance entrypoint on every pull request (and on main after merge). It invents no step of its own — each job is run_automated.py --suite <x> --require-clean, so a check that fails in CI fails the same way locally:

Job Runner Suite Provenance artifact
rust-core macos-15 (arm64, POL-5 reference host) all — toolchain, format, lint, test, architecture, cargo-deny (POL-6), traceability, script self-tests acceptance-ci-all
secret-scan ubuntu-24.04 security — gitleaks over committed history acceptance-ci-security

Design notes:

  • Least privilege. The workflow grants contents: read and nothing else; no job writes to the repo or mints a token. Release signing/attestation is a separate tag-triggered workflow.
  • Pinned, so cache cannot change a verdict. The Rust toolchain is pinned by rust-toolchain.toml, dependencies by Cargo.lock, cargo-deny and gitleaks by exact version (gitleaks additionally by sha256), and every action by commit SHA. The cargo cache is keyed on the toolchain and lockfile, so a hit can only ever hold artifacts built from identical inputs — it speeds up a run, it cannot alter its result.
  • Self-healing Rust bootstrap. Every Rust-consuming self-hosted job runs .github/scripts/bootstrap-rust-toolchain.sh. If the runner-owned rustup binary is absent, it downloads the fixed rustup 1.29.0 archive and verifies its committed SHA-256 before execution, then installs the exact channel/profile/components from rust-toolchain.toml, and publishes Cargo's bin directory through GITHUB_PATH. It never sources a login profile, and a repeated invocation only reconciles the same pinned toolchain.
  • No secrets in logs. Neither job needs a repository secret, and the secret scan runs gitleaks with --redact so a matched value never reaches the uploaded log. Verified false positives are pinned by fingerprint in .gitleaksignore; .gitleaks.toml is the committed, shared rule config.
  • Provenance. Each job uploads .temp/acceptance/<run-id>/ (if: always(), if-no-files-found: error, 14-day retention) so a result — green or red — is attributable to a commit (NFR-052).

Required checks (branch protection, one-time repo-admin setup). A workflow file cannot make itself blocking. Mark rust-core and secret-scan as required status checks on main so a pull request cannot merge while either fails.

Native platform CI

.github/workflows/native-ci.yml extends the same pattern to the macOS native drive (POL-5 / DEC-017 reference target). It is not the fast per-PR gate: its jobs build TDLib from source and stage the core XCFramework, so it runs on a schedule (nightly), on demand (workflow_dispatch), on main, and on release-bound PRs (release/**) — which is what "release branches require native acceptance evidence" needs without taxing every feature PR. Each job builds from a clean checkout and, like core-ci, runs the pinned entrypoint (or a packaging script that writes its own manifest), never an ad-hoc swift command list.

Job Runner Suite / script Provenance artifact
tdlib macos-15 (arm64) pinned TDLib built from source (cache-first, keyed on the pin) + Rust link smoke native-tdlib
apple-build-test macos-15 (arm64) appleswift build + swift test of apple/GramDriveSupport over the staged core (make check-apple) acceptance-ci-apple
apple-package-unsigned macos-15 (arm64) build_app_bundle.py --unsigned — assembles GramDrive.app (nested appex, Info.plists, entitlement plists) with no Developer ID (make package-app-unsigned) native-app-package-unsigned

Design notes:

  • No secrets, ever. Native-ci only assembles an unsigned bundle. Signing and notarization (the Developer ID identity in a keychain) live in the separate tag-triggered release workflow (TASK-260715-3bhbkv); this workflow keeps permissions: contents: read and consumes no repository secret.
  • TDLib cached on the pin. The tdlib job keys its cache on build_tdlib.py (which holds the pinned commit), so a warm run restores the artifact and re-runs only the fast link smoke; the from-source C++ build happens on a cold cache.
  • Support matrix (POL-5), documented not silently missing. macOS is the v1 native leg. iOS (EPIC-260715-3uynbw), Windows (EPIC-260715-1mlv5j), Linux (EPIC-260715-1hnglv) and Android (EPIC-260715-y0fshx) legs are deferred with their backlog EPIC ids in the workflow header, and enter native-ci when the platform EPIC starts — not stubbed, because a build path nothing runs rots.

The bindings smoke (make smoke-bindings, not part of make check) additionally needs swiftc (Xcode command line tools), kotlinc (brew install kotlin), and Java 17+ (brew install openjdk); it downloads its two JVM runtime jars (JNA, kotlinx-coroutines) from Maven Central once, pinned by version and sha256 in the runner script.

The shared-state smoke (make smoke-shared-state, also not part of make check) needs Xcode: it runs a Rust coordinator process, two concurrent Swift provider processes, and a change-watcher process over one substitute App Group container, through the packaged artifact (staging make package first when none is present).

The agent-lifecycle smoke (make smoke-agent-lifecycle, also not part of make check) needs Xcode: it runs the gramdrive-agent companion binary as real processes over a substitute container and proves the lifecycle contract — health over the bounded IPC channel, single-instance refusal of a second agent, SIGTERM drain of a hosted transfer, and instant successor startup after SIGKILL.

Release (signed, notarized, attested)

.github/workflows/release.yml is the tag-triggered release (TASK-260715-3bhbkv): push a v* tag and it produces one signed, notarized GramDrive.app + .dmg and its provenance. It is the only workflow with elevated permissions (contents: write, id-token: write, attestations: write) and the only one that holds a secret, so it is isolated to that single trigger. It re-implements no signing command — it invokes the same reusable scripts native-ci proves unsigned.

Pipeline (macos-15, environment: release):

  1. Supply-chain gate first (POL-6). run_automated.py --suite supply-chain fails the release closed — before any credential is imported — if a dependency pulls in a disallowed license or an open advisory.
  2. Import into a throwaway keychain. The Developer ID Application identity (from MACOS_CERT_P12 / MACOS_CERT_PASSWORD) and the gramdrive-notary profile (from APPSTORE_KEY_ID / APPSTORE_ISSUER_ID / APPSTORE_PRIVATE_KEY) go into one RUNNER_TEMP keychain, deleted in an always() step. No secret is echoed; no key material touches the workspace, a cache, or the login keychain.
  3. Build the signed artifact. build_app_bundle.py --notarize signs inside-out with the hardened runtime + a trusted timestamp, then notarizes and staples both the .app and the .dmg (so an app dragged out of the dmg verifies offline). Version stamping reads the reviewed three-component apple/GramDriveSupport/Version.json; ordinary local packaging uses the git rev-count for CFBundleVersion. Candidate CI selects the greater of that floor and the highest applicable public feed build plus one, then revalidates the exact selection immediately before immutable handoff (Sparkle ordering).
  4. Build the release provenance. build_release_provenance.py emits the CycloneDX SBOM (POL-6 dependency inventory), a changelog since the last tag, rollback metadata, a release manifest tying every artifact to a sha256, and a credential scrub that fails the release if any produced file looks like it carries a secret.
  5. Attest + publish. actions/attest-build-provenance records an OIDC-bound attestation for the dmg (verifiable with gh attestation verify), then gh release create publishes the dmg, checksums, SBOM, changelog, rollback and release manifest with the changelog as the release notes.

The human approval gate (POL-8 / DEC-020: public release is the single mandatory human stop) is environment: release — its required-reviewer protection rule (owner sign-off) is a repo-admin setting, the same transparent limitation the required-status-check rule has above.

Local dry-run of the non-signing half (needs a prior make package-app / make package-app-unsigned for the manifest; no signing identity, Xcode, or network — only git and cargo):

make release-provenance    # SBOM + changelog + rollback + release manifest + scrub → .temp/release/

Packaging the core for native consumers

make package               # build + verify the artifacts native hosts consume
make package-reproducible  # build the shipped library at two paths, compare bytes

make package produces a self-contained SwiftPM package in .temp/packaging/: GramDriveCore.xcframework (macOS 14+ arm64 static library plus headers), the generated Swift bindings, a manifest, and checksums — then proves it by resolving and running a real minimal Swift package against the result. Needs Xcode (xcodebuild, swift); like the smoke it is not part of make check, because it needs a release build and produces artifacts rather than a verdict on the source.

Per platform: macOS consumes the XCFramework; Windows and Linux consume the gramdrive-ffi crate directly as a Rust dependency and need no artifact at all (which is why the crate keeps its rlib crate-type); Android (.so + Kotlin) and iOS slices are deferred until those platforms enter scope (POL-5/DEC-017), not stubbed — a build path nothing runs is a build path that rots. Full rationale, the measured reproducibility and size numbers, and the crate-type and debug-info decisions: .scripts/packaging/README.md.

License and public repository policy

GramDrive is licensed under the Apache License 2.0. Attribution and notice obligations are collected in NOTICE. Contributions are accepted under that license; see CONTRIBUTING.md, the Code of Conduct, and SECURITY.md for the contribution, community, and private-vulnerability-reporting policies.

The repository's private-to-public conversion is controlled by the public-readiness checklist. In particular, public release and Sparkle-feed publication require reviewer acceptance; private-era v0.1.0 and v0.1.1 are retained as GitHub drafts (with their tags and assets preserved) and are excluded from all feeds.

Update credential bootstrap, independent rotation/revocation, versioned-feed bridges, emergency freeze, and forward recovery are documented in the update operations runbook. It provides stdin/owner-only-file setter commands and a value-free inventory preflight.

Available utilities:

Tool Purpose Run Output
.scripts/release/check_update_secret_inventory.py Value-safe bootstrap/rotation setter plus initial-inventory preflight for the seven update-delivery names and versioned Sparkle generations; setter values arrive only via stdin or owner-only files and are never argv/output make updates-secret-inventory; make updates-secret-inventory CHECK_GITHUB=1; for provisioning, use the exact owner-only-file and generate_keys --account … -p / -x <owner-only-private-file> sequences in docs/UPDATE_OPERATIONS.md with --set, --set-developer-id-from, and --set-notary-from. The same runbook gives exact post-bridge gh secret delete NAME --env ENV and account-scoped Keychain cleanup commands. Exit 0 when the requested storage/preflight action succeeds; exit 1 on invalid input, missing/unexpected names, or unavailable/failed gh; no credential-bearing artifacts
.scripts/acceptance/run_automated.py The single gate entrypoint: runs a named suite, records provenance. Used identically by make and by CI python3 .scripts/acceptance/run_automated.py --suite core --run-id local-core; --list prints suites and steps Exit 0 pass / 1 failed step / 2 could not start; .temp/acceptance/<run-id>/ with summary.json + per-step logs
.scripts/acceptance/run_live_content.py Privacy-safe pre-install acceptance matrix composing the focused Rust history/render/fidelity/hydration/story/retention suites with the full Swift package/provider regressions. Child output is discarded; evidence is allow-listed and bounded make check-live-content, or through run_automated.py --suite live-content --run-id local-live-content after staging make package Exit 0 all matrix legs passed / 1 one or more failed; .temp/acceptance/<run-id>/live-content.json contains only fixed labels, counts, booleans, timings, versions, and bounds
.scripts/acceptance/run_installed_live_content.py Privacy-safe installed-profile hierarchy, one-read hydration, generated-cache, additive identity, cursor, metadata, namespace, and relaunch proof. Each phase has fixed stage timings and a 120-second hard deadline; TERM, exact-group SIGKILL, leader reap, and pipe drain are bounded by that same deadline. The schema-v25 partial index streams at most 20 small live attachment candidates without an account-wide temp sort; each Finder st_flags probe is isolated to a 500ms child and classified as dataless, materialized, missing, platform error, timeout, or identity/path mismatch. A missing indexed item gets one exact-ItemId URL resolution through the installed companion's bounded, pre-AppKit --acceptance-resolve-placeholder command; the command validates with the authoritative core parser and round-trips the returned URL to the same provider identity before the harness performs an exact dataless re-probe. Bulk identity/cursor evidence stays in an indexed private SQLite sidecar, while generated references are streamed and physical inventory is capped at 1,000,000 entries python3 .scripts/acceptance/run_installed_live_content.py {before|stability-snapshot|after} --state <private.json> --evidence <public.json> [--deadline-seconds 120]; preserve the same private state and relaunch without resetting the profile between phases Exit 0 only when every phase-specific assertion passes; exit 1 on a fixed acceptance, bounded selection, stage-timeout, or worker-cleanup-deadline-exceeded category; exit 2 for an invalid deadline. Public JSON contains fixed aggregate fields/timings and placeholder-state counts only; private JSON plus <state>.snapshot.sqlite3 contain comparison keys
.scripts/acceptance/run_installed_index_metadata.py Installed authorized-profile probe for the historical chat index (BUG-260728-2qfzbd): how much of the listed backlog background history work can actually reach, whether every chat/month directory publishes a size rollup equal to its indexed descendants, whether any directory is still undated (the state Finder renders as 1 Jan 1970) as distinct from faithfully epoch-dated, and whether every cursor window stayed monotonic across an app + agent relaunch. The rollup and date checks are scoped to the kinds that own a rollup — chat, month, Active Stories — because a chat list or folder catalog is deliberately left NULL. Opens no content and downloads nothing python3 .scripts/acceptance/run_installed_index_metadata.py {before|after|relaunch} --output <file.json> --private <dir> --now-ms <ms> Exit 0 all acceptance booleans true / 1 any false; the JSON output holds counts, booleans, and byte totals only, while salted cursor digests stay under --private. A phase that asserts nothing (before) reports "passed": null, not true
.scripts/acceptance/run_installed_foreground_demand.py Installed authorized-profile probe for the foreground-demand path (BUG-260728-2qfzbd): whether using a chat in Finder buys it a history turn, with no control-socket hint anywhere. It picks the reachable incomplete chat the background rotation will reach last, watches it for one window untouched, performs one gesture, then watches it again — so a turn the rotation would have handed out regardless fails the probe instead of passing it. --gesture read (default) reads one generated document inside the chat, which is the interaction that reliably reaches the extension; --gesture open only lists the folder, which macOS usually answers from its own replica without calling the extension at all. It also reads the agent's hint counters either side of the gesture. Downloads no Telegram payload bytes python3 .scripts/acceptance/run_installed_foreground_demand.py --output <file.json> --private <file.json> [--gesture read|open] [--window <seconds>] [--socket <path>] Exit 0 when the gesture's boolean (content_read_granted_a_turn / foreground_open_granted_a_turn) is true, 1 when false; the JSON output holds counts, booleans, seconds, and message deltas only, while the chosen chat's id, folder, document, and raw cursor readings stay in --private
.scripts/acceptance/run_installed_generated_hydration.py Installed authorized-profile saturation probe for 20 distinct dataless generated documents across Markdown, NDJSON, and chat JSON, with exact-cache verification, fixed per-read deadlines, backfill/hint/cursor fairness, and a before/after relaunch identity check python3 .scripts/acceptance/run_installed_generated_hydration.py before --private <private.json> --evidence <before.json>; relaunch without resetting the profile/domain, then repeat with after, the same private path, and a new evidence path Exit 0 only for zero timeouts/errnos, exact bytes, p95 <1s, p99 <3s, saturated backfill, balanced requested/background hints, a target chat turn, monotonic cursors, and preserved identities. Public evidence is aggregate-only; identifiers, paths, and digests stay private
.scripts/acceptance/run_native_macos.py The macOS native manual acceptance harness (TASK-260715-3oe2nr): the ten File Provider Finder flows the release gate requires (register, enumerate, hydrate, cancel, pin, update, restart, repair, upgrade, remove). One scenario catalog drives the run-sheet, the evidence form, and the machine probes. Human-in-the-loop by necessity — it preflights the host, captures evidence, and never reports a scenario passed; a person runs Finder and signs off. Pipeline: .scripts/acceptance/README.md make accept-macos, make accept-macos-runsheet, or python3 .scripts/acceptance/run_native_macos.py --run-id accept-YYYY-MM-DD [--app-path …] [--require-ready]; --list prints the catalog Exit 0 prepared (not a pass) / 2 could not start / 3 --require-ready host off-matrix; .temp/acceptance/<run-id>/ with runsheet.md, evidence-template.md, summary.json + per-probe evidence logs
make Shorthand for the entrypoint plus the non-gate inner loop (fmt, build, test) make check, make check-core, make check-repo, make gates Delegates to the entrypoint; never re-defines a gate command
cargo (pinned to Rust 1.91.0 by rust-toolchain.toml, edition 2024) Build and test the shared core workspace cargo build --workspace / cargo test --workspace (repo root); per-crate commands in each crates/*/README.md Binaries/test results under target/ (gitignored)
rustfmt + clippy (pinned components) Formatting and lints. Config: rustfmt.toml, clippy.toml, and [workspace.lints] in Cargo.toml — levels live in the manifest so editors agree with the gate cargo fmt --all to fix; the format and lint gate steps to check Exit non-zero on a formatting diff or any warning (-D warnings)
swift + swift-format Build/test the macOS support package and lint task-scoped Swift changes using the repository's existing four-space style and accepted retroactive File Provider conformance After make package: cd apple/GramDriveSupport && swift test; lint a changed file with swift-format lint --strict --configuration '{"version":1,"indentation":{"spaces":4},"lineLength":100,"respectsExistingLineBreaks":true,"rules":{"AvoidRetroactiveConformances":false}}' PATH SwiftPM artifacts and coverage profiles under apple/GramDriveSupport/.build/; formatter emits diagnostics only
.scripts/check_toolchain.py Asserts the pinned toolchain is actually in effect — rust-toolchain.toml only binds when rustup drives cargo — and that cargo-deny meets the minimum version python3 .scripts/check_toolchain.py (repo root; stdlib only) Exit 0 + summary line, or exit 1 with itemized errors (CI-suitable)
.github/scripts/bootstrap-rust-toolchain.sh Restores missing rustup on a clean self-hosted macOS runner with the fixed Rustup 1.29.0 archive verified by committed SHA-256, then installs the channel/components from rust-toolchain.toml and exports Cargo's bin directory for later CI steps .github/scripts/bootstrap-rust-toolchain.sh (GitHub Actions macOS job; requires curl and shasum) Exit 0 with active toolchain, rustc, and cargo versions; no shell-profile mutation or execution of an unverified download
.scripts/check_crate_architecture.py Enforces crates/README.md: dependency direction, no cycles, no platform leakage in core crates, testkit dev-only, per-crate README sections, shared lint-set opt-in python3 .scripts/check_crate_architecture.py (repo root; stdlib only, needs cargo on PATH) Exit 0 + summary line, or exit 1 with itemized errors (CI-suitable)
cargo-deny (installed via brew install cargo-deny) Supply-chain gate, config in deny.toml: POL-6 licenses (permissive-only), RustSec advisories, bans, and sources (crates.io only) cargo deny check (repo root), or one check: cargo deny check licenses advisories ok, bans ok, licenses ok, sources ok, or non-zero exit with the offending dependency tree
.scripts/validate_traceability.py Validates docs/TRACEABILITY.md against .spec/ and .task-board/: every requirement mapped exactly once, no orphan board elements, no stale requirement references on the board python3 .scripts/validate_traceability.py (repo root; stdlib only) Exit 0 + summary line, or exit 1 with itemized errors (CI-suitable)
.scripts/tests/ Self-tests for the gate scripts themselves — an untested runner is a gate with no gate python3 -m unittest discover -s .scripts/tests -t .scripts/tests, or the scripts gate step Standard unittest output
uniffi-bindgen (workspace-local, crates/gramdrive-ffi/src/bin/) Generates Swift + Kotlin bindings from the compiled library, version-locked to the linked uniffi crate; pipeline documented in crates/gramdrive-ffi/README.md make bindings, or cargo run -p gramdrive-ffi --features bindgen --bin uniffi-bindgen -- generate --library target/debug/libgramdrive_ffi.dylib --language swift --language kotlin --out-dir .temp/bindings Generated sources in .temp/bindings/ (build artifacts, never committed)
.scripts/smoke/run_bindings_smoke.py End-to-end bindings smoke: builds the FFI library, generates bindings, compiles and runs the Swift and Kotlin smoke consumers (.scripts/smoke/{swift,kotlin}/) asserting async, progress, error, and cancellation round-trips make smoke-bindings, or python3 .scripts/smoke/run_bindings_smoke.py [--skip-swift] [--skip-kotlin] (needs swiftc, kotlinc, java) Exit 0 + BINDINGS SMOKE PASSED, or non-zero with the failing step's log; artifacts and per-step logs in .temp/bindings-smoke/
.scripts/smoke/run_shared_state_smoke.py Multi-process shared-state smoke (TASK-260715-gnsa2s): a Rust coordinator process seeds a substitute App Group container, two concurrent Swift provider processes (apple/GramDriveSupport over the packaged artifact) must read byte-identical item metadata, and a watcher process must observe the Darwin change doorbell plus the dataVersion probe across a foreign commit make smoke-shared-state, or python3 .scripts/smoke/run_shared_state_smoke.py [--repackage] (macOS; needs Xcode; stages make package when no artifact is present) Exit 0 + SHARED-STATE SMOKE PASSED, or non-zero with the failing step's output; container and per-step logs in .temp/shared-state-smoke/
.scripts/packaging/build_core_artifacts.py Builds what native consumers ship against: release staticlib (LTO restored via a crate-type override), Swift bindings generated from that exact binary, XCFramework, manifest (contract version read from the built artifact, git describe, toolchain), checksums, and a deterministic zip; verifies it all by resolving and running a real minimal SwiftPM package (.scripts/packaging/swift-consumer/). Owns the shipped-target list make package, make package-reproducible, or python3 .scripts/packaging/build_core_artifacts.py [--skip-verify] [--check-reproducible] (macOS; needs xcodebuild, swift) Exit 0 + PACKAGING PASSED, or non-zero with the failing step's log; artifacts, manifest.json, CHECKSUMS.sha256 and per-step logs in .temp/packaging/
.scripts/tdlib/restore_pinned_artifact.py Restores the recipe-keyed runner-local arm64 TDLib artifact on the dedicated x86_64 CI/signing runner; refuses a cold cache, symlinks, wrong pin/target/install-name/architecture, missing or mismatched static-OpenSSL attribution, and any file/checksum/manifest inventory mismatch before atomically replacing the staged artifact python3 .scripts/tdlib/restore_pinned_artifact.py [--cache-root DIR] [--out-dir DIR]; CI follows it with make tdlib-smoke-link Exit 0 plus the privacy-safe cache key and library SHA-256; verified artifact in .temp/tdlib/out, or exit 2 for an unseeded cache
.scripts/release/build_release_provenance.py The release provenance bundle (TASK-260715-3bhbkv), derived from the signed .app manifest + git history + cargo metadata: CycloneDX SBOM (POL-6 dependency inventory, license per crate), changelog since the last tag, rollback metadata, a release manifest tying every artifact to a sha256, and a credential scrub that fails on any secret-shaped content. POL-6 is enforced by cargo deny (core CI), not re-adjudicated here. Invoked by the tag-triggered release.yml make release-provenance, or python3 .scripts/release/build_release_provenance.py [--package-dir DIR] [--out-dir DIR] [--tag vX.Y.Z] (needs a prior packaging run for the manifest; only git + cargo, no signing/Xcode/network) Exit 0; sbom.json, CHANGELOG.md, rollback.json, release-manifest.json, RELEASE-CHECKSUMS.sha256 in .temp/release/
.scripts/release/build_candidate_package.py Candidate trust-boundary verifier (TASK-260810-fwgmr4): rejects non-live, wrong-channel, non-arm64, non-monotonic, incorrectly identified, incompletely signed/notarized/stapled/Gatekeeper-assessed inputs; requires complete per-Mach-O architecture/team/authority readback and exact app/core/TDLib checksum inventories; copies the exact DMG bytes under the immutable version-and-build asset name; emits privacy-safe provenance; binds the subject inventory to the GitHub Sigstore bundle; and re-verifies the immutable handoff Used by .github/workflows/candidate-build.yml; downstream verification is python3 .scripts/release/build_candidate_package.py verify --out-dir PATH Exact build-named DMG plus app/core/TDLib manifests and checksums, candidate-manifest.json, candidate-provenance.json, verification.json, SUBJECTS.sha256, candidate-attestation.json, finalization.json, and CANDIDATE-CHECKSUMS.sha256 in .temp/candidate/
.scripts/release/publish_sparkle.py Deterministic Sparkle publication verifier and renderer: validates exact candidate intake, endpoint-isolated appcasts, authenticated complete-site archives, and the stable generation state machine. Stable candidates and tags bind to reviewed .github/sparkle-stable.json; authenticated prior-site state permits only same-generation promotion or a one-time forward rotation with an old-key bridge. Workflows source .github/scripts/run-with-optional-file-argument.sh so initial publication and prior-feed updates are both safe under the runner's Bash 3.2 with set -u. Stable promotion freezes the authenticated site as immutable Release assets on macOS and stops without Pages capability. A separately approved redeploy-site dispatch from protected main checks out the exact dispatch SHA, re-verifies those immutable assets on Ubuntu, uploads the authenticated site, and hands it to the isolated Pages deployment job without exposing the stable key or Release mutation capability Used by .github/workflows/candidate-build.yml and .github/workflows/release.yml; focused regression command: python3 .scripts/tests/test_publish_sparkle.py; inspect subcommands with python3 .scripts/release/publish_sparkle.py --help Exit 0 plus SPARKLE PUBLICATION … PASSED, or exit 1 fail-closed; workflow artifacts remain under run-scoped .temp/publication/ and .temp/stable/
.scripts/release/release_asset_inventory.py Deterministic Release rerun guard: fully captures gh release view --json assets, normalizes one safe unique name per asset, and rejects CLI, JSON, schema, unsafe-name, or duplicate-inventory failures before immutable upload decisions. Existing assets remain download-and-compare only; only an explicit absent state may enter the upload branch. Stable dispatch seals the helper's exact git-object bytes from the protected-main workflow revision before detaching to an older immutable tag; push tags seal it from their exact tag workflow revision. Every use rechecks canonical runner-temp containment and both workflow-output/environment SHA-256 bindings python3 .scripts/release/release_asset_inventory.py capture --release TAG --output FILE, then python3 .scripts/release/release_asset_inventory.py state --inventory FILE --name ASSET; used directly by test publication and through the sealed workflow-control path by stable publication Normalized JSON inventory plus present or absent; untrusted workflow refs, missing/tampered/path-escaped helpers, hash mismatches, or indeterminate inventory state exit nonzero before Release mutation
.scripts/release/select_stable_release.py Deterministic offline selector for the complete paginated GitHub Releases API response. It strictly flattens the gh api --paginate --slurp page array and validates stable publication timestamps as timezone-aware RFC3339. Candidate preflight chooses the newest published exact-semver stable Release as a fail-closed state head and requires singular manifest/attestation evidence. Stable publication instead excludes its current source tag, refuses any newer published semver, and restores the newest prior site's singular complete asset set, so current Releases remain resumable when absent, partial, or complete without falling back past an incomplete predecessor Workflows save the paginated page array to a file, then run python3 .scripts/release/select_stable_release.py --release-pages FILE --mode candidate-state-head, or --release-pages FILE --mode stable-prior --current-tag vX.Y.Z; no standalone jq prerequisite Exit 0 and the selected tag (or empty output when no state exists); malformed pages, records, timestamps, duplicated/incomplete evidence, or incompatible state exits non-zero before download or mutation
.scripts/release/check_candidate_build_order.py Read-only monotonic-build selector and immutable-handoff guard for candidate production; test candidates inspect the rolling test feed, while stable candidates derive every versioned stable endpoint through the active generation from .github/sparkle-stable.json. The latest published semver stable Release is the fail-closed state head: its GitHub-attested prior-site manifest must have an exact canonical archive/file/key inventory, and each recorded Pages feed must match the attested SHA-256 and byte count before its builds can raise the floor. An unrecorded next generation is 404-only. Present feeds accept exactly one canonical positive build on each direct RSS item, using an attribute-free child-free exact Sparkle namespace leaf or the legacy enclosure attribute; malformed/unavailable feeds fail closed Selection: python3 .scripts/release/check_candidate_build_order.py --mode test|stable-candidate --git-build N [--stable-site-manifest VERIFIED.json]; final race check: repeat with --candidate-build N Exit 0 and the highest applicable published numeric build; selection also writes build_number=max(git_build, published_highest+1) to GITHUB_OUTPUT; candidate and stable workflows share a non-cancelling concurrency group, and validation immediately before artifact upload exits non-zero if the serialized public floor consumed that build; no feed mutation
.scripts/apple-app/build_app_bundle.py Assembles, signs (Developer ID, hardened runtime, inside-out), embeds/signs Sparkle 2.9.5 and its helpers, notarizes + staples the .app and .dmg, and records a manifest with per-binary entitlements/cdhashes + checksums. Pipeline: .scripts/apple-app/README.md make package-app (sign+verify, no notarize), make package-app-unsigned (assemble only), make package-app-notarize (full), or `python3 .scripts/apple-app/build_app_bundle.py --update-channel test stable [--build-number N] [--notarize] [--unsigned] [--notary-keychain PATH]` (macOS; needs Xcode, a Developer ID identity, and staged core; candidate CI alone supplies the reviewed public-feed-aware build override; each channel's reviewed public trust anchor is checked in)
.scripts/smoke/run_agent_lifecycle_smoke.py Multi-process agent-lifecycle smoke (TASK-260715-1yx9ly): the gramdrive-agent companion binary over a substitute container — startup with health served over the bounded UNIX-socket IPC, single-instance refusal (exit 2) of a second agent, SIGTERM drain cancelling a hosted transfer through its token (exit 0, endpoint removed), and a successor starting immediately after SIGKILL with healthy durable state make smoke-agent-lifecycle, or python3 .scripts/smoke/run_agent_lifecycle_smoke.py [--repackage] (macOS; needs Xcode; stages make package when no artifact is present) Exit 0 + PASSED: agent lifecycle smoke, or non-zero with the failing step's output; container and per-step logs in .temp/agent-lifecycle-smoke/
.scripts/smoke/run_control_auth_smoke.py Control-channel auth smoke (BUG-260720-3i74u1): real end-to-end sign-in against Telegram's test DC through the packaged bundle's gramdrive-agent — the control socket up, phone → code → ready over the same NDJSON wire the companion speaks, the durable account row in status, and after an agent restart the stored session still authorizing (repair completes). Since mid-2025 Telegram rejects the documented auto-code for shared 99966XYYYY numbers with third-party api ids (tdlib/td#3361), so completing sign-in needs --phone with a dedicated test-DC account (created once via an official app in test mode) and one interactive code entry; the kept authorized session makes later legs and re-runs unattended make smoke-control-auth, or python3 .scripts/smoke/run_control_auth_smoke.py [--agent PATH] [--dc N] [--attempts N] [--keep] [--phone +NUMBER] (macOS; needs make package-app, keychain api credentials readable by the packaged agent — see the keychain provisioning row — and network to the test DC) Exit 0 + control-auth smoke: PASS with the signed-in test account; agent logs in .temp/control-auth-smoke/
.scripts/smoke/run_sparkle_manual_update_smoke.py Packages an isolated LSUIElement accessory host against pinned Sparkle 2.9.5, signs a local appcast and enclosure with a fixture-only EdDSA key, and proves the production manual-update action presents a visible key-capable standard Sparkle window from zero windows within a bounded deadline. Shipping feed URLs and keys are never used or changed make smoke-sparkle-manual-update, or run make package followed by python3 .scripts/smoke/run_sparkle_manual_update_smoke.py [--work-dir .temp/PATH] on macOS Exit 0 + sparkle manual update smoke: PASS; privacy-safe JSON, local feed, isolated preferences, and the packaged fixture host in .temp/sparkle-manual-update-smoke/
.scripts/keychain/provision_telegram_credentials.py Rewrites the gramdrive-telegram keychain items (api_id/api_hash, TASK-260716-1iypv4) so the signed product binaries read them without a consent prompt: compiles and Developer ID-signs the companion Swift tool, which recreates the items with the team's partition list and a trusted-application ACL naming the packaged gramdrive-agent/GramDrive (items made by the security CLI are partition-locked to apple-tool: and always prompt) python3 .scripts/keychain/provision_telegram_credentials.py [--identity NAME] [--bundle PATH] (macOS; needs the packaged bundle, a Developer ID identity, and the values in GRAMDRIVE_API_ID/GRAMDRIVE_API_HASH env or in the existing keychain items) Exit 0 + provisioned: ...; secrets never touch argv, logs, or the repo
.scripts/tdlib/build_tdlib.py Reproducible build of the pinned TDLib tdjson artifact the local Telegram source links against (BSL-1.0 recorded per POL-6): fetch at the pinned commit, CMake build, staged libtdjson.dylib + headers + license, manifest and checksums, proved by the link-smoke/ Rust binary; pipeline documented in .scripts/tdlib/README.md make tdlib, make tdlib-smoke (re-run only the link smoke), make tdlib-verify (same-path reproducibility), or python3 .scripts/tdlib/build_tdlib.py (macOS arm64; needs Xcode clang, cmake, gperf, Homebrew openssl@3) Staged artifact, manifest.json and CHECKSUMS.sha256 in .temp/tdlib/out/; smoke prints the running library's version
make tdjson-smoke Real-linkage smoke of the gramdrive-source-tdjson runtime (crate docs: crates/gramdrive-source-tdjson/README.md): with GRAMDRIVE_TDLIB_ARTIFACT_DIR set, the crate's env-gated build.rs links the staged libtdjson.dylib and the otherwise-empty real_tdjson_smoke test drives correlation, client close, and shutdown against the real library. Every make check runs the same runtime mock-only, artifact-free make tdjson-smoke (after make tdlib staged the artifact) cargo test output: 1 test against the real library, exit non-zero on failure

About

Native Telegram-backed drive: architecture, specifications, and delivery plan

Topics

Resources

Code of conduct

Contributing

Security policy

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Used by

Contributors

Languages