Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 25 additions & 0 deletions .github/workflows/secret-scan.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
name: Secret Scanning

on:
push:
branches: [ "main", "feat/*", "feature/*" ]
pull_request:
branches: [ "main", "feat/*", "feature/*" ]

jobs:
gitleaks:
name: Detect Secrets with Gitleaks
runs-on: ubuntu-latest
permissions:
contents: read

steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
fetch-depth: 0

- name: Run Gitleaks
uses: gitleaks/gitleaks-action@v2
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
30 changes: 30 additions & 0 deletions .gitleaks.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
title = "Gitleaks Config for SoroTask"

# Extend the default ruleset
[extend]
useDefault = true

# Define custom rules
[[rules]]
id = "sorolab-api-key"
description = "SoroLabs API Key pattern"
regex = '''(?i)(sorolab[_-]?api[_-]?key|sorolab[_-]?secret)\s*=\s*['\"]?[a-z0-9]{32,}['\"]?'''
secretGroup = 3
entropy = 5.0

# Allowlist patterns - files or paths to exclude
[allowlist]
description = "Allowlist configuration"
paths = [
# Test fixtures and mocks
'''(test_snapshots|__tests__|test|spec)''',
# Documentation examples
'''(docs|README|CHANGELOG)''',
# Config files
'''\.(env\.example|env\.template|env\.sample)$''',
# Dependencies and build artifacts
'''(node_modules|target|dist|build)''',
]

commits = []
regexes = []
10 changes: 10 additions & 0 deletions .gitleaksignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
# Gitleaks ignore file for false positives
# Add patterns or specific exceptions here

# Example format:
# <commit_hash>:<line_number>:<rule_name>

# Test credentials and fixtures that are intentionally exposed
# Format: <commit>:<line>:<rule> or <rule_name>

# Add mock keys used in tests or documentation here
25 changes: 25 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
repos:
- repo: https://github.com/gitleaks/gitleaks
rev: v8.18.0
hooks:
- id: gitleaks
name: Detect secrets with gitleaks
entry: gitleaks detect --source . --verbose
language: golang
stages: [commit]
pass_filenames: false

- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.5.0
hooks:
- id: detect-private-key
name: Detect private keys
- id: check-added-large-files
name: Check for large files
args: [ '--maxkb=1000' ]
- id: check-json
name: Check JSON files
- id: check-yaml
name: Check YAML files
- id: check-toml
name: Check TOML files
173 changes: 173 additions & 0 deletions SECRET_SCANNING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,173 @@
# Secret Scanning and Prevention Guide

This document explains how secret scanning and prevention are implemented in the SoroTask repository.

## Overview

We use **Gitleaks** for automated secret scanning to prevent accidental commits of sensitive credentials (API keys, private keys, etc.).

## CI/CD Integration

### GitHub Actions Workflow

A dedicated GitHub Actions workflow (`.github/workflows/secret-scan.yml`) runs on every push and pull request. This workflow:

- Uses the official Gitleaks Action
- Scans the entire repository history
- Fails the CI if secrets are detected
- Produces detailed reports listing detected patterns

The workflow triggers on:
- All pushes to `main`, `feat/*`, and `feature/*` branches
- All pull requests to `main`, `feat/*`, and `feature/*` branches

## Pre-commit Hooks

### Setup Instructions

To enable pre-commit hooks locally:

1. Install pre-commit:
```bash
pip install pre-commit
```

2. Install the git hooks:
```bash
pre-commit install
```

3. Verify installation:
```bash
pre-commit run --all-files
```

### What Gets Checked

The `.pre-commit-config.yaml` configures these checks:

- **Gitleaks Detection**: Scans for secret patterns before committing
- **Private Key Detection**: Identifies private key files
- **Large Files**: Prevents committing files larger than 1MB
- **JSON Validation**: Ensures JSON files are valid
- **YAML Validation**: Ensures YAML files are valid
- **TOML Validation**: Ensures TOML files are valid

## Configuration Files

### `.gitleaks.toml`

Main Gitleaks configuration file that defines:
- Detection rules (includes default rules from Gitleaks)
- Custom rules for project-specific patterns
- Allowlist configurations for excluding false positives

Key allowlists:
- Test snapshots and fixture directories
- Documentation and README files
- Example environment files (`.env.example`, `.env.template`)
- Build artifacts and dependencies

### `.gitleaksignore`

File for ignoring false positives. Format options:
- `<commit_hash>:<line_number>:<rule_name>` - Ignore specific lines
- `<rule_name>` - Ignore all detections of a specific rule

### `.pre-commit-config.yaml`

Defines all pre-commit hooks that run locally before commits.

## Handling False Positives

If a legitimate value is flagged as a secret:

### Option 1: Add to `.gitleaksignore`
Add an entry to ignore the specific detection:
```
abc123def456:42:potential-api-key
```

### Option 2: Update `.gitleaks.toml`
Add patterns to the allowlist to exclude files or rules:
```toml
[allowlist]
paths = [
'''path/to/false/positive''',
'''test/fixtures''',
]
```

### Option 3: Move Sensitive Data
If it's a real secret that was accidentally committed:
1. Rotate all compromised credentials immediately
2. Use tools like `git-filter-branch` to remove from history
3. Force push the cleaned history

## Development Workflow

### Making Commits

When working locally:

1. Your pre-commit hooks will automatically run
2. If secrets are detected, the commit will fail
3. Fix the issues (remove secrets or add to allowlist)
4. Try committing again

### Bypassing Hooks (Not Recommended)

In rare cases where you need to bypass hooks:
```bash
git commit --no-verify
```

⚠️ **Warning**: This should only be used when absolutely necessary and with caution.

## Testing Secret Detection

To manually test if Gitleaks is working:

```bash
# Test with gitleaks CLI (if installed)
gitleaks detect --source . --verbose

# Test pre-commit hooks
pre-commit run gitleaks --all-files
```

## Secrets to Avoid

Never commit:
- Private/secret keys (RSA, Ed25519, etc.)
- API keys and tokens
- Database passwords
- AWS/GCP/Azure credentials
- Private cryptocurrency keys
- OAuth tokens
- SSH keys
- NPM/PyPI authentication tokens

## Additional Resources

- [Gitleaks Documentation](https://github.com/gitleaks/gitleaks)
- [Pre-commit Framework](https://pre-commit.com/)
- [GitHub Secret Scanning](https://docs.github.com/en/code-security/secret-scanning)

## Troubleshooting

### Gitleaks Action Fails in CI

1. Check the workflow output for specific rules that triggered
2. If it's a false positive, update `.gitleaksignore` or `.gitleaks.toml`
3. Push the fix and re-run the workflow

### Pre-commit Hooks Not Running

1. Verify installation: `pre-commit --version`
2. Reinstall hooks: `pre-commit install`
3. Run manually: `pre-commit run --all-files`

### Need to Update Rules

Edit `.gitleaks.toml` and add new rules following [Gitleaks rule documentation](https://github.com/gitleaks/gitleaks#rules).