Skip to content
211 changes: 211 additions & 0 deletions docs/funding/LTFF_APPLICATION_DRAFT.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,211 @@
# Long-Term Future Fund — AEGIS Ω Application Draft

Status: submission-ready draft, NOT SUBMITTED
Target fund: Long-Term Future Fund (EA Funds)
Current public application surface: always open; general EA Funds range USD 1,000–500,000.
Primary source: https://funds.effectivealtruism.org/apply-for-funding

## Project title

**Evidence-Bound Runtime Authority for Autonomous AI Agents**

## Funding request

**USD 120,000 / 12 months**

This request is intentionally below the program-wide maximum. The goal is to fund one narrow, independently testable safety program rather than the entire AEGIS Ω roadmap.

## One-sentence summary

Build and independently evaluate an open-source runtime layer that prevents autonomous AI agents from converting model confidence, credentials, or tool availability into consequential authority without task-, state-, policy-, and evidence-bound admission.

## Problem

As AI agents gain the ability to modify code, infrastructure, databases, workflows, and external services, traditional identity and access controls are necessary but incomplete. A valid identity may possess a valid credential and still attempt an action that is no longer authorized, is based on stale state, exceeds task scope, or produces an outcome that cannot be verified.

Most agent-safety approaches concentrate on model behavior, prompt-level policy, or monitoring. This project studies a complementary system property: **can consequential authority be mechanically bounded outside the model and independently reconstructed after execution?**

The central safety invariant is:

```text
model capability != authority
model confidence != evidence
execution != verified outcome
```

## Proposed work

We will develop a bounded open-source reference implementation and adversarial benchmark around five transitions:

1. **Typed action proposal** — actor, task, resource, capability, intended side effect.
2. **Authority admission** — delegated scope, lease/fence, policy, state freshness.
3. **Consequential execution** — no external side effect without an admitted action.
4. **Execution receipt** — deterministic, tamper-evident evidence binding proposal to observed execution.
5. **Outcome verification** — explicit separation of `SUCCESS`, `FAILURE`, and `OUTCOME_UNKNOWN`, with replay or compensation where feasible.

## Research questions

1. Can task-level authority prevent failures that identity/IAM alone cannot distinguish?
2. Which agent actions can be deterministically replayed or independently verified despite stochastic model reasoning?
3. How should stale state, late receipts, ambiguous operator intent, and partial provider failures affect authority?
4. How often do latent model/user-state inferences improperly influence routing or policy, and can evidence thresholds reduce that failure mode?
5. What minimum evidence is sufficient for an external reviewer to reproduce an agent-action verdict without trusting the originating model?

## Existing technical base

AEGIS Ω is already a public open-source repository with runnable reference proofs for:

- deterministic canonicalization and SHA-256 lineage;
- replay-verifiable governance envelopes;
- tamper detection;
- cross-language verification across Python, Node.js, and Rust;
- fail-closed admission and governance primitives;
- large existing TypeScript/Rust/Python test surfaces.

Public reproduction commands currently include:

```bash
python3 genomics/test_replay_proof.py
python3 verifiable/test_generality.py
bash verifiable/cross_language/verify.sh
python3 verifiable/certify_all.py --twice
```

The proposal does **not** rely on claims of AGI, consciousness, universal hallucination elimination, or production deployment.

## Deliverables

### D1 — Consequential Action Contract

A stable open schema for:
- actor identity;
- task identity;
- capability;
- delegated authority;
- resource scope;
- state preconditions;
- lease/fence;
- execution intent;
- expected outcome;
- evidence references.

### D2 — Runtime Admission Layer

Reference implementation supporting:

`ADMIT | DENY | REVIEW | BLOCKED`

with fail-closed handling for unresolved consequential ambiguity.

### D3 — Adversarial Benchmark

At least 100 reproducible negative-control scenarios covering:
- valid identity / invalid task authority;
- stale lease or fence;
- stale target state;
- privilege expansion;
- prompt/tool injection;
- provider timeout;
- late receipt;
- duplicate action/retry;
- outcome unknown;
- unsupported latent user/operator-state inference.

### D4 — Receipt + Replay Package

Cross-language deterministic verification and public reproduction harness.

### D5 — Independent Evaluation

Recruit at least two external technical reviewers or organizations to run the benchmark independently and publish or return reproducibility results.

### D6 — Research Write-up

Open technical paper covering architecture, threat model, empirical results, failure cases, limitations, and falsification criteria.

## Milestones

### Months 1–2
- freeze action/receipt schemas;
- isolate canonical reference implementation;
- threat model and baseline negative controls.

### Months 3–5
- implement state/authority admission and adversarial harness;
- cross-runtime replay;
- automated receipts.

### Months 6–8
- benchmark agent/tool/provider failure modes;
- measure false-admit / false-deny behavior;
- operator-model calibration experiments.

### Months 9–10
- independent reproduction wave;
- red-team fixes;
- deterministic release package.

### Months 11–12
- final paper;
- public benchmark;
- deployment guidance and limitations.

## Success metrics

Primary metrics:

- False Unauthorized Admission Rate;
- Unverifiable Action Execution Rate;
- replay success rate;
- stale-state rejection rate;
- receipt completeness;
- independent reproduction rate;
- time-to-reconstruct an incident;
- policy-elevation errors caused by unsupported latent inference.

A negative result is useful. If the architecture cannot provide reliable evidence-bound authority under realistic provider behavior, the benchmark should expose that clearly.

## Budget — USD 120,000

- Operator/research engineering: $60,000
- External reviewers / independent reproduction / contracted security review: $20,000
- Compute, API and cloud experimentation: $15,000
- Test infrastructure and hardware/edge targets: $10,000
- Research dissemination, conference/travel and reproducibility support: $7,500
- Legal/accounting/administrative and contingency: $7,500

Exact budget can be resized after funder feedback.

## Why this may matter for advanced-AI risk

As agents become more capable, preventing every model-level error may be unrealistic. A complementary safety strategy is to ensure that **error does not automatically become authority**.

The project therefore focuses on the boundary between cognition and consequential action. Even if an agent reasons incorrectly, the system should be able to deny an inadmissible action, preserve evidence of the attempted transition, and avoid silently classifying an unknown outcome as success.

## Public-benefit plan

Core schemas, benchmark cases, verifier, reference runtime, and research outputs will be released openly. The goal is to make the work reusable by other AI-safety researchers, open-source agent frameworks, and organizations evaluating agent controls.

## Key risks and limitations

- deterministic evidence does not make stochastic cognition deterministic;
- a receipt can prove what the instrumented system observed, not unknowable external facts;
- provider black boxes limit end-to-end verification;
- safety benefits depend on external actions actually being routed through the control boundary;
- benchmarks may underrepresent future high-autonomy workflows;
- this project does not claim a complete alignment solution.

## Operator

Tarik Skalić
Bihać, Bosnia and Herzegovina
AEGIS Ω / Aegis-Omega

## Submission checklist

- [ ] confirm requested amount and personal/entity recipient structure;
- [ ] add concise prior-work links;
- [ ] select 3 strongest runnable proof links;
- [ ] add 12-month personal runway/budget assumptions if form requests them;
- [ ] operator review;
- [ ] submit through LTFF application surface.
186 changes: 186 additions & 0 deletions docs/funding/OPENAI_CYBERSECURITY_GRANT_DRAFT.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,186 @@
# OpenAI Cybersecurity Grant Program — AEGIS Ω Draft

Status: READY FOR OPERATOR REVIEW — NOT SUBMITTED
Program: OpenAI Cybersecurity Grant Program / Trusted Access for Cyber ecosystem
Primary sources:
- https://openai.com/index/openai-cybersecurity-grant-program/
- https://openai.com/index/trusted-access-for-cyber/

## Project title

**Capability-Bound Agent Execution: Preventing Tool Misuse and Privilege Drift in Autonomous Security Workflows**

## Requested support

**$50,000 equivalent support**, preferably a mix of API credits and other available program support.

The amount is intentionally aligned to a bounded defensive open-source project and can be resized to the current program mechanism.

## Project summary

Security agents increasingly operate with repository, cloud, CI/CD, scanner, ticketing, and remediation tools. Existing identity controls can establish who an agent is and what credential it possesses, but a valid identity does not prove that a particular action is authorized for the current task, target, state, or point in time.

AEGIS Ω will build an open-source defensive control layer that binds security-agent actions to task-scoped capabilities, delegated authority, resource/state preconditions, and deterministic execution receipts. The project will demonstrate that an agent can have valid credentials yet still be denied when the requested action is stale, excessive, ambiguous, or unsupported by admissible evidence.

The system is intended for authorized defensive workflows only.

## Defensive use cases

The initial benchmark will cover:

1. secure code review and finding validation;
2. patch proposal and patch verification;
3. security backlog triage;
4. threat-model generation and evidence updates;
5. controlled CI security checks;
6. defensive MCP/tool invocation;
7. remediation-plan execution in isolated test environments.

No project milestone requires exploitation or unauthorized testing of third-party systems.

## Core security hypothesis

```text
VALID_IDENTITY != VALID_AUTHORITY
VALID_AUTHORITY != VALID_ACTION_NOW
VALID_ACTION != VERIFIED_OUTCOME
```

The project tests whether a deterministic external governance layer can reduce security-agent privilege drift and confused-deputy behavior without depending on a model to reliably self-police its own authority.

## Technical design

A proposed security-agent action enters an admission contract containing:

- actor/workload identity;
- task identity;
- target resource;
- requested capability;
- delegated authority;
- lease/fence and freshness information;
- observed target state;
- policy/evidence references;
- expected side effect and outcome.

AEGIS evaluates the contract and returns one of:

`ADMIT | DENY | REVIEW | BLOCKED`

An admitted execution emits a canonical, tamper-evident receipt. The verifier separately evaluates the observed outcome and preserves `OUTCOME_UNKNOWN` when a provider or environment cannot establish success or failure.

## Adversarial scenarios

The benchmark will include defensive failure cases such as:

- valid repository token, wrong repository/task scope;
- valid cloud identity, expired delegated lease;
- target changed after approval;
- tool request expands beyond approved remediation;
- malicious instructions embedded in issue/code/tool output;
- retry causes duplicate side effect;
- security scanner returns ambiguous or stale evidence;
- provider timeout after a possibly committed action;
- an agent claims a patch was applied but outcome verification disagrees;
- one agent's assertion is incorrectly treated as independent verification by another.

## Existing AEGIS base

The public AEGIS Ω repository already contains reference implementations and tests for replay-verifiable, tamper-evident evidence handling and deterministic governance envelopes, including cross-language verification.

Public proof commands include:

```bash
python3 genomics/test_replay_proof.py
python3 verifiable/test_generality.py
bash verifiable/cross_language/verify.sh
python3 verifiable/certify_all.py --twice
```

The funded project would turn these general primitives into a focused security-agent benchmark and integration layer.

## Expected deliverables

1. **Open Agent Security Action Contract**
- typed schema for actor/task/capability/authority/state/evidence/outcome.

2. **Defensive Admission Gateway**
- read/decision layer usable around MCP and other tool adapters.

3. **Security-Agent Adversarial Benchmark**
- at least 100 deterministic or receipt-bound scenarios.

4. **Receipt and Replay Verifier**
- independent validation of action lineage and declared outcomes where observable.

5. **Reference Integrations**
- at least two bounded integrations, prioritizing open-source repository/AppSec workflows.

6. **Open Evaluation Report**
- false-admit/false-deny behavior, bypass attempts, limitations, and reproduction guide.

## Why OpenAI support is high leverage

The project directly concerns defensive agent execution and the growing transition from advisory security models to agents capable of acting through tools. OpenAI API/Codex access would enable testing across realistic long-horizon defensive workflows while AEGIS supplies the external authority/evidence boundary.

The research is intentionally compatible with a trust-based access model: increased model capability should not imply increased unbounded authority.

## Public-benefit / open-source plan

The reference schemas, benchmark, verifier, and core integration code will be released in the public AEGIS Ω repository under its existing open-source licensing model or a maximally reusable compatible subcomponent license where necessary.

Results will include negative findings and bypasses rather than only successful demonstrations.

## Success metrics

- unauthorized-action admission rate;
- stale-authority rejection rate;
- duplicate-side-effect prevention/detection rate;
- evidence-chain replay rate;
- outcome-misclassification rate;
- prompt/tool-injection containment rate at the action boundary;
- external reproduction by at least two reviewers.

## Explicit scope limits

This proposal does not request permission for:

- credential theft;
- malware deployment;
- persistence or stealth;
- destructive testing;
- exploitation of systems without explicit authorization;
- bypassing platform safeguards.

All active testing will occur against owned, isolated, open-source, or explicitly authorized targets.

## 6-month execution plan

### Month 1
Contract freeze, threat model, benchmark skeleton.

### Months 2–3
Admission gateway, OpenAI/Codex defensive workflow adapter, first 50 negative controls.

### Month 4
Remaining benchmark cases, outcome verification and retry/timeout semantics.

### Month 5
Independent reproduction and red-team review.

### Month 6
Public release, evaluation report, partner integration guide.

## Operator

Tarik Skalić — AEGIS Ω
Bihać, Bosnia and Herzegovina
Public repository: Aegis-Omega/AEGIS-OMEGA

## Before submission

- [ ] map final answers to current OpenAI application form fields;
- [ ] choose exact request amount/support mix;
- [ ] reference strongest current security/replay demos;
- [ ] include Codex Security integration only after its preflight/execution status is accurately represented;
- [ ] operator review;
- [ ] submit.
Loading
Loading