Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 25 additions & 3 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,6 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).

---

<<<<<<< feature/deep-scan-governance-rollout
## [Unreleased]

### Added
Expand All @@ -18,7 +17,31 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
- Local Docker defaults no longer rely on a pre-created `.env`; `.env.example` is now the explicit starting point.
- Documentation now treats `documentation/WORKBOARD.md` as the source of truth for milestone, issue, and branch tracking.
- Terraform, Docker, and config baselines were corrected to match the current repository layout and runtime behavior.
=======
- Expanded GitHub Actions OIDC deploy-role permissions for newly managed security resources (CloudTrail event selectors, additional Secrets Manager read/list/restore actions, and WAFv2 resource lookups) to avoid `AccessDenied` during Terraform apply.

---

## [1.3.0] - 2026-04-04

### Added
- **AWS WAF v2** (`waf.tf`): CloudFront-scoped Web ACL (us-east-1) with `AWSManagedRulesCommonRuleSet`, `AWSManagedRulesKnownBadInputsRuleSet`, and a per-IP rate-based rule (300 requests / 5 minutes). Controlled by `enable_waf` variable (default `true`).
- **AWS Secrets Manager** (`secrets_manager.tf`): Flask `SECRET_KEY` and `MSAL_CLIENT_SECRET` are now stored in a Secrets Manager secret (`{prefix}/app-secrets`). The Lambda receives `SECRET_ARN` instead of the plaintext values, preventing credentials from appearing in the AWS console environment variables view.
- **AWS CloudTrail** (`cloudtrail.tf`): Multi-region trail writing to a dedicated AES256-encrypted S3 bucket with log-file validation enabled and S3 data-event logging for the application bucket. Controlled by `enable_cloudtrail` variable (default `true`) with a 7-year retention policy (transition to Glacier after 90 days) and no force-destroy on the log bucket.
- **DynamoDB PITR**: Point-in-Time Recovery enabled on all 11 DynamoDB tables for a 35-day recovery window.
- **SQS DLQ alarms**: Two new CloudWatch alarms (`{prefix}-registration-dlq-depth` and `{prefix}-campaign-dlq-depth`) fire on the SNS alerts topic when either dead-letter queue accumulates ≥ 1 message.
- **`config.py` Secrets Manager resolver**: shared secret payload is fetched once and cached; `SECRET_KEY` now fails closed when `SECRET_ARN` is configured but the key cannot be retrieved, while local development and tests still use env-var fallbacks when `SECRET_ARN` is absent.
- **Terraform variables**: `enable_waf` (bool, default `true`) and `enable_cloudtrail` (bool, default `true`) added to `variables.tf`.
- **Terraform outputs**: canonical outputs `waf_web_acl_arn`, `app_secrets_arn`, and `cloudtrail_bucket` exposed for the new infrastructure components.

### Changed
- `lambda.tf`: Lambda env var `SECRET_KEY` replaced with `SECRET_ARN`; `MSAL_CLIENT_SECRET` removed from plaintext env vars (now fetched at runtime from Secrets Manager).
- `iam.tf`: Lambda execution role now includes `secretsmanager:GetSecretValue` on the application secrets ARN.
Comment on lines +28 to +38

Copilot AI Apr 5, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This entry says MSAL_CLIENT_SECRET was removed from plaintext Lambda env vars and that Lambda receives only SECRET_ARN, but phishing-platform-infra/terraform/lambda.tf still sets MSAL_CLIENT_SECRET = var.msal_client_secret in the environment. Either update the infrastructure to actually remove it, or adjust the changelog to reflect the current behavior.

Copilot uses AI. Check for mistakes.
- `cloudfront.tf`: `web_acl_id` now references the WAF Web ACL ARN when `enable_waf = true`.
- `terraform.tfvars.example`: Documents `enable_waf` and `enable_cloudtrail` optional overrides.
- `documentation/ARCHITECTURE.md`: Updated System Overview and AWS Infrastructure diagrams to show WAF, Secrets Manager, CloudTrail, SQS DLQs for both queues, 11 DynamoDB tables, and 8 CloudWatch alarms.

---

## [1.2.6] - 2026-04-04

### Changed
Expand All @@ -31,7 +54,6 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
### Added
- **`documentation/REPO_SEPARATION.md`**: step-by-step guide for splitting the Flask application and AWS infrastructure into two standalone repositories, including `git filter-repo` commands, CI/CD handoff strategy, and CODEOWNERS alternative.
- **Documentation index updates**: `documentation/README.md`, `documentation/dev/README.md`, and `documentation/operator/README.md` now list all files in their respective directories.
>>>>>>> main

---

Expand Down
2 changes: 1 addition & 1 deletion VERSION
Original file line number Diff line number Diff line change
@@ -1 +1 @@
1.2.5
1.3.0
101 changes: 99 additions & 2 deletions config.py
Original file line number Diff line number Diff line change
@@ -1,8 +1,105 @@
import json
import logging
import os
from typing import Any

logger = logging.getLogger(__name__)

_APP_SECRETS_CACHE: dict[str, Any] | None = None
_APP_SECRETS_CACHE_ARN: str | None = None


def _get_app_secrets() -> dict[str, Any]:
"""Fetch and cache the shared app secret bundle from Secrets Manager."""
global _APP_SECRETS_CACHE, _APP_SECRETS_CACHE_ARN

secret_arn = os.environ.get("SECRET_ARN", "").strip()
if not secret_arn:
return {}

if (
_APP_SECRETS_CACHE is not None
and _APP_SECRETS_CACHE_ARN == secret_arn
):
return _APP_SECRETS_CACHE

import boto3
from botocore.exceptions import BotoCoreError, ClientError

try:
region = os.environ.get("AWS_REGION_NAME", "eu-west-3")
client = boto3.client("secretsmanager", region_name=region)
resp = client.get_secret_value(SecretId=secret_arn)
data = json.loads(resp["SecretString"])
if not isinstance(data, dict):
raise ValueError("secret payload must be a JSON object")
except (
ClientError,
BotoCoreError,
json.JSONDecodeError,
KeyError,
ValueError,
) as exc:
raise RuntimeError(
f"Failed to load application secrets from {secret_arn}: {exc}"
) from exc

_APP_SECRETS_CACHE = data
_APP_SECRETS_CACHE_ARN = secret_arn
return data


def _resolve_secret_key() -> str:
"""Return the Flask SECRET_KEY.

When running on AWS Lambda the ``SECRET_ARN`` environment variable points
to a Secrets Manager secret that holds the key as a JSON object
``{"SECRET_KEY": "..."}`` alongside other credentials. We fetch the value
at startup so the plaintext key is never stored as a Lambda environment
variable (where it is visible in the AWS console).

Falls back to the ``SECRET_KEY`` environment variable (or the insecure
development default) when ``SECRET_ARN`` is absent — this preserves
backward-compatibility for local development and unit tests.
"""
secret_arn = os.environ.get("SECRET_ARN", "").strip()
if secret_arn:
secret_key = _get_app_secrets().get("SECRET_KEY")
if secret_key:
return secret_key

env_secret_key = os.environ.get("SECRET_KEY", "")
if env_secret_key:
logger.warning(
"SECRET_ARN is set but SECRET_KEY was not found in Secrets Manager; "
"falling back to SECRET_KEY env var"
)
return env_secret_key

raise RuntimeError(
"SECRET_ARN is set but SECRET_KEY could not be resolved from Secrets "
"Manager and no SECRET_KEY env fallback is configured"
)
return os.environ.get("SECRET_KEY", "dev-secret-key-change-in-production")

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Fail closed when SECRET_KEY cannot be retrieved

When SECRET_ARN is present and Secrets Manager retrieval fails (for example due to IAM, throttling, or regional issues), this path falls back to os.environ.get("SECRET_KEY", "dev-secret-key-change-in-production"). In the same commit lambda.tf replaces SECRET_KEY with SECRET_ARN, so production can start with the known dev key instead of refusing to boot, which weakens session and CSRF signing security.

Useful? React with 👍 / 👎.



def _resolve_msal_client_secret() -> str:
"""Return the MSAL client secret from Secrets Manager or env var."""
secret_arn = os.environ.get("SECRET_ARN", "").strip()
if secret_arn:
msal_client_secret = _get_app_secrets().get("MSAL_CLIENT_SECRET")
if msal_client_secret:
return msal_client_secret

logger.warning(
"SECRET_ARN is set but MSAL_CLIENT_SECRET was not found in Secrets "
"Manager; falling back to MSAL_CLIENT_SECRET env var"
)
return os.environ.get("MSAL_CLIENT_SECRET", "")
Comment on lines +52 to +98

Copilot AI Apr 5, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both _resolve_secret_key() and _resolve_msal_client_secret() call Secrets Manager independently, which results in two network calls during cold start (and repeats JSON parsing). Consider factoring this into a single helper that fetches/parses the secret once and memoizes the result (module-level cache), then have both resolvers read from that cached dict.

Copilot uses AI. Check for mistakes.


class Config:
SECRET_KEY = os.environ.get('SECRET_KEY', 'dev-secret-key-change-in-production')
SECRET_KEY = _resolve_secret_key()

# Flask-WTF CSRF — allow requests through CloudFront where the Referer
# header is the CloudFront domain, not the API Gateway origin.
Expand Down Expand Up @@ -54,7 +151,7 @@ class Config:
# Set all three variables to enable the "Sign in with Microsoft" button.
# Leave them empty (default) to disable SSO entirely.
MSAL_CLIENT_ID = os.environ.get('MSAL_CLIENT_ID', '')
MSAL_CLIENT_SECRET = os.environ.get('MSAL_CLIENT_SECRET', '')
MSAL_CLIENT_SECRET = _resolve_msal_client_secret()
# e.g. "https://login.microsoftonline.com/<tenant-id>/v2.0" or
# "https://login.microsoftonline.com/common/v2.0" for multi-tenant.
MSAL_AUTHORITY = os.environ.get(
Expand Down
44 changes: 36 additions & 8 deletions documentation/ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,31 +31,37 @@ graph TB
end

subgraph AWS["AWS — eu-west-3"]
WAF["WAF v2\nOWASP rules + rate limiting"]
CF["CloudFront\nCDN + HTTPS"]
APIGW["API Gateway v2\nHTTP API"]
LambdaApp["Lambda Flask App\nPython 3.12 · 512 MB"]
LambdaWorker["Lambda Registration Worker\nPython 3.12 · 256 MB"]
DDB[("DynamoDB\n9 Tables")]
DDB[("DynamoDB\n11 Tables + PITR")]
S3[("S3 Bucket\nEML + Videos + Reports")]
SQS["SQS Queue\nRegistration"]
SES["SES\nEmail"]
SNS["SNS\nAlerts + Events"]
CW["CloudWatch\nAlarms + Dashboard + X-Ray"]
SM["Secrets Manager\nSECRET_KEY + MSAL"]
CT["CloudTrail\nAudit Log"]
end

Student -->|HTTPS| CF
Admin -->|HTTPS| CF
Student -->|HTTPS| WAF
Admin -->|HTTPS| WAF
WAF --> CF
CF --> APIGW
APIGW --> LambdaApp
LambdaApp --> DDB
LambdaApp --> S3
LambdaApp --> SQS
LambdaApp --> SM
SQS --> LambdaWorker
LambdaWorker --> DDB
LambdaWorker --> SES
LambdaWorker --> SNS
LambdaApp --> CW
SNS --> CW
CT --> S3
GHA -->|Terraform apply| LambdaApp
GHA -->|S3 sync| S3
GHA -->|seed_dynamodb.py| DDB
Expand All @@ -73,6 +79,9 @@ graph TB
![CloudFront](https://img.shields.io/badge/CloudFront-232F3E?logo=amazonaws&logoColor=white)
![API Gateway](https://img.shields.io/badge/API_Gateway-FF4F8B?logo=amazonaws&logoColor=white)
![CloudWatch](https://img.shields.io/badge/CloudWatch-FF4F8B?logo=amazonaws&logoColor=white)
![AWS WAF](https://img.shields.io/badge/AWS_WAF-DD344C?logo=amazonaws&logoColor=white)
![Secrets Manager](https://img.shields.io/badge/Secrets_Manager-DD344C?logo=amazonaws&logoColor=white)
![CloudTrail](https://img.shields.io/badge/CloudTrail-232F3E?logo=amazonaws&logoColor=white)

Mermaid does not support embedding official AWS architecture logos inside nodes. Keep AWS service badges or a legend outside the diagram, and generate companion SVG/PNG assets only when logo-rich visuals are needed.

Expand All @@ -85,8 +94,12 @@ graph LR
ACM["ACM Cert\nus-east-1"]
end

subgraph Security["WAF v2 (us-east-1)"]
WAF["Web ACL\nAWSManagedRulesCommonRuleSet\nAWSManagedRulesKnownBadInputsRuleSet\nRate limit: 300 req/5 min per IP"]
end

subgraph CDN["CloudFront"]
CF["Distribution\nTTL=0 · Compress\nredirect-to-https"]
CF["Distribution\nTTL=0 · Compress\nredirect-to-https · WAF attached"]
end

subgraph APIGW_["API Gateway v2"]
Expand All @@ -100,9 +113,10 @@ graph LR

subgraph Storage["S3"]
S3["phishing-app-{env}-eu-west-3\nVersioned · AES256\nPublic read: videos/* (dev only)\nPrivate: eml-samples/ reports/"]
S3Trail["phishing-app-{env}-cloudtrail-{acct}\nAES256 · Public access blocked"]
end

subgraph DB["DynamoDB — 9 tables (PAY_PER_REQUEST)"]
subgraph DB["DynamoDB — 11 tables (PAY_PER_REQUEST + PITR)"]
TUsers["users\nPK: username\nGSI: email-index, group-index"]
TQuizzes["quizzes\nPK: quiz_id"]
TAttempts["attempts\nPK: username+quiz_id\nGSI: quiz-index, group-index"]
Expand All @@ -112,45 +126,59 @@ graph LR
TBugs["bugs\nPK: bug_id"]
TAnswerKey["answer-key-overrides\nPK: email_file"]
TCohort["cohort-tokens\nPK: token · TTL: expires_at (90 days)"]
TThreatCache["threat-cache\nPK: cache_key · TTL: ttl"]
TCampaigns["campaigns\nPK: campaign_id · GSI: cohort-index"]
end

subgraph Async["SQS + SES + SNS"]
SQS["Registration Queue\nDLQ · SSE · 60s visibility · 1-day retention"]
DLQQ["Registration DLQ\n14-day retention · maxReceiveCount=4"]
CampaignQ["Campaign Queue\nDLQ · SSE · 300s visibility · 7-day retention"]
CampaignDLQ["Campaign DLQ\n14-day retention"]
SES["SES Email Identity\nno-reply@..."]
SNSReg["SNS Registration Topic\nFuture fan-out"]
SNSAlerts["SNS Alerts Topic\n+ Email subscription (optional)"]
end

subgraph Observe["CloudWatch"]
CWLogs["Log Groups\n/aws/lambda/{app,worker} · /aws/apigateway/*\n14-day retention"]
CWAlarms["6 Alarms\nLambda errors(>=5)/duration-p95(>=25s)/throttles(>=1)\nAPI GW 4xx(>=50)/5xx(>=3) · DynamoDB SystemErrors(>=1)"]
CWAlarms["8 Alarms\nLambda errors/duration-p95/throttles\nAPI GW 4xx/5xx · DynamoDB SystemErrors\nRegistration DLQ depth · Campaign DLQ depth"]
CWDash["Dashboard\nphishing-app-{env}-overview\n3 rows: Lambda · API GW · DynamoDB"]
end

subgraph SecretsAudit["Secrets + Audit"]
SM["Secrets Manager\n{prefix}/app-secrets\nSECRET_KEY + MSAL_CLIENT_SECRET"]
CT["CloudTrail\n{prefix}-trail · multi-region\nS3 data events for app bucket"]
end

subgraph IAM_["IAM"]
RoleLambda["phishing-app-{env}-lambda-role\nDynamoDB (9 tables+GSIs) · S3 · SQS:SendMessage · X-Ray"]
RoleLambda["phishing-app-{env}-lambda-role\nDynamoDB (11 tables+GSIs) · S3 · SQS:SendMessage\nX-Ray · secretsmanager:GetSecretValue"]
RoleWorker["phishing-app-{env}-registration-worker-role\nDynamoDB:users · SES:SendEmail · SQS:Receive+Delete · SNS:Publish"]
RoleGHA["phishing-app-{env}-github-actions-deploy\nOIDC · Lambda · IAM · DynamoDB · S3 · API GW\nCloudFront · CloudWatch · SNS · SQS · SES · X-Ray · ACM · Route53"]
OIDC["OIDC Provider\ntoken.actions.githubusercontent.com\nrepo: Panacota96/master-Project-Phishing:*"]
end

R53 --> CF
ACM --> CF
WAF --> CF
CF --> APIGW
APIGW --> LApp
LApp --> TUsers & TQuizzes & TAttempts & TResponses & TInspector & TInspectorAnon & TBugs & TAnswerKey & TCohort
LApp --> TUsers & TQuizzes & TAttempts & TResponses & TInspector & TInspectorAnon & TBugs & TAnswerKey & TCohort & TThreatCache & TCampaigns
LApp --> S3
LApp --> SQS
LApp --> CampaignQ
LApp --> SM
SQS --> LWorker
SQS --> DLQQ
CampaignQ --> CampaignDLQ
LWorker --> TUsers
LWorker --> SES
LWorker --> SNSReg
SNSAlerts --> CWAlarms
LApp --> CWLogs
LWorker --> CWLogs
APIGW --> CWLogs
CT --> S3Trail
RoleLambda --> LApp
RoleWorker --> LWorker
OIDC --> RoleGHA
Expand Down
1 change: 1 addition & 0 deletions phishing-platform-infra/terraform/cloudfront.tf
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ resource "aws_cloudfront_distribution" "app" {
comment = "${local.prefix} – stable login URL"
default_root_object = ""
aliases = var.domain_name != "" ? [var.domain_name] : []
web_acl_id = var.enable_waf ? aws_wafv2_web_acl.app[0].arn : null

origin {
domain_name = replace(aws_apigatewayv2_api.app.api_endpoint, "https://", "")
Expand Down
Loading
Loading