Skip to content

AxiosLeo/webhook-cd

Repository files navigation

Webhook-CD: Continuous Deployment via Webhooks

English | 简体中文

NPM version node version License

⚠️ 重要警告 / IMPORTANT WARNING

🚫 本工具严禁在生产环境中使用!仅限开发环境使用!

🚫 DO NOT USE THIS TOOL IN PRODUCTION ENVIRONMENTS! DEVELOPMENT USE ONLY!

此工具为开发阶段的自动化部署工具,未经过生产环境的安全性和稳定性验证。在生产环境中使用可能导致数据丢失、服务中断或安全风险。

This tool is designed for development automation and has not been validated for production security and stability. Using it in production may result in data loss, service interruption, or security risks.

Webhook-CD is a tool designed to automate continuous deployment workflows by listening to webhooks from both Coding.net and GitHub platforms. It processes Git merge/pull request events to trigger deployment actions on configured repositories.

Overview

The system works by:

  1. Listening for Webhooks: An HTTP server listens for incoming webhook events from Coding.net and GitHub, specifically related to merge/pull requests (created, updated, merged, closed).
  2. Automated Deployments: Based on repository configurations, the tool monitors for deployable events (like a merged merge/pull request). When such an event occurs for a configured repository and branch, it performs Git operations (e.g., checkout, pull) in a specified local directory to deploy the changes.
  3. CLI for Manual Control: A command-line interface (CLI) tool, wcd, is provided for manual task inspection and triggering.

Features

  • Webhook integration with Coding.net and GitHub (for Merge/Pull Request events).
  • Automated Git operations for deployment based on webhook triggers.
  • Message queue integration with RabbitMQ for reliable task processing.
  • Configuration via environment variables for flexibility.
  • CLI tool (wcd) for manual interaction and task management.

Prerequisites

  • Node.js and npm
  • Access to Coding.net and/or GitHub repositories you wish to deploy.
  • RabbitMQ server (recommended: use Docker for development environment)
  • Docker (optional, for running RabbitMQ in development)

Setup & Configuration

1. Clone the Repository

git clone <repository-url>
cd webhook-cd

2. Configure Environment Variables

The application uses environment variables for its configuration. Create a .env file in the root of the project or set these variables in your deployment environment.

RabbitMQ Configuration:

  • RABBITMQ_HOST: Hostname of your RabbitMQ server (default: localhost).
  • RABBITMQ_PORT: Port for RabbitMQ (default: 5672).
  • RABBITMQ_USER: RabbitMQ username (default: guest).
  • RABBITMQ_PASS: RabbitMQ password (default: guest).

Coding.net Configuration:

  • CODING_USER: Your Coding.net username.
  • CODING_TOKEN: Your Coding.net user token for API access.

GitHub Configuration:

  • GITHUB_USER: Your GitHub username.
  • GITHUB_TOKEN: Your GitHub personal access token for API access.
  • GITHUB_WEBHOOK_SECRET: (Optional) Secret token for webhook signature verification.

Application Configuration:

  • WORKSPACE: (Optional) The base workspace directory. Defaults to runtime/repos relative to the application root.
  • LISTEN_HOST: Host for the webhook listener (default: 0.0.0.0).
  • LISTEN_PORT: Port for the webhook listener to run on (default: 8800).

3. Start RabbitMQ (Development Environment)

For development, you can easily start a RabbitMQ container using Docker:

docker run -d --name rabbitmq \
  -p 5672:5672 \
  -p 15672:15672 \
  -e RABBITMQ_DEFAULT_USER=guest \
  -e RABBITMQ_DEFAULT_PASS=guest \
  rabbitmq:3-management

This will start RabbitMQ with:

  • AMQP port: 5672
  • Management UI: http://localhost:15672 (username: guest, password: guest)

Alternatively, the project includes a pre-configured docker-compose.yml file with RabbitMQ. You can use:

docker-compose up -d

4. Configure PM2 Log Rotation

To prevent log files from growing too large and consuming excessive disk space, the project uses PM2's log rotation feature. Here's how to set it up:

  1. Install the PM2 log rotation module:
pm2 install pm2-logrotate
  1. Configure global log rotation settings:
pm2 set pm2-logrotate:max_size 100M        # Maximum size of each log file
pm2 set pm2-logrotate:retain 7             # Number of days to keep logs
pm2 set pm2-logrotate:compress true        # Compress rotated logs
pm2 set pm2-logrotate:dateFormat YYYY-MM-DD_HH-mm-ss  # Date format for rotated logs
pm2 set pm2-logrotate:workerInterval 30    # Check interval in seconds
pm2 set pm2-logrotate:rotateInterval '0 0 * * *'  # Daily rotation at midnight
pm2 set pm2-logrotate:rotateModule true    # Enable module rotation
  1. Verify the configuration:
pm2 conf pm2-logrotate

The log rotation configuration will:

  • Rotate logs when they reach 100MB or at midnight each day
  • Keep logs for 7 days
  • Compress old logs to save space
  • Store logs in the logs directory with timestamps
  • Automatically clean up old logs

5. Install Dependencies & Run the Application

npm install
npm start

Or, for development with auto-reloading:

npm run dev

Docker Deployment

The application supports Docker containerization with flexible deployment options for different services.

1. Build Docker Image

docker build -t webhook-cd .

2. Container Startup Options

The Docker image supports running two different services based on the MAIN_BIN environment variable:

  • MAIN_BIN=api: Runs the webhook API server (./bin/api.js)
  • MAIN_BIN=consumer: Runs the message queue consumer (./bin/consumer.js)

3. Running API Server Container

The API server listens for webhooks and runs on port 8800 by default:

# Run with environment file
docker run --env-file .env -e MAIN_BIN=api -p 8800:8800 webhook-cd

# Or with explicit environment variables
docker run -e MAIN_BIN=api \
  -e LISTEN_HOST=0.0.0.0 \
  -e LISTEN_PORT=8800 \
  -e RABBITMQ_HOST=your_rabbitmq_host \
  -e CODING_USER=your_username \
  -e CODING_TOKEN=your_token \
  -e GITHUB_USER=your_github_username \
  -e GITHUB_TOKEN=your_github_token \
  -p 8800:8800 \
  webhook-cd

4. Running Consumer Container

The consumer processes messages from the RabbitMQ queue:

# Run with environment file
docker run --env-file .env -e MAIN_BIN=consumer webhook-cd

# Or with explicit environment variables
docker run -e MAIN_BIN=consumer \
  -e RABBITMQ_HOST=your_rabbitmq_host \
  -e RABBITMQ_USER=guest \
  -e RABBITMQ_PASS=guest \
  -e CODING_USER=your_username \
  -e CODING_TOKEN=your_token \
  -e GITHUB_USER=your_github_username \
  -e GITHUB_TOKEN=your_github_token \
  webhook-cd

5. Docker Compose Example

Create a docker-compose.override.yml file for complete deployment:

version: "3.8"

services:
  rabbitmq:
    image: rabbitmq:3-management
    ports:
      - "5672:5672"
      - "15672:15672"
    environment:
      RABBITMQ_DEFAULT_USER: guest
      RABBITMQ_DEFAULT_PASS: guest

  webhook-api:
    build: .
    ports:
      - "8800:8800"
    environment:
      - MAIN_BIN=api
    env_file:
      - .env
    depends_on:
      - rabbitmq

  webhook-consumer:
    build: .
    environment:
      - MAIN_BIN=consumer
    env_file:
      - .env
    depends_on:
      - rabbitmq

Then run:

docker-compose -f docker-compose.yml -f docker-compose.override.yml up -d

6. Environment Variables for Docker

When running in Docker, ensure your .env file or environment variables include:

# RabbitMQ Configuration
RABBITMQ_HOST=localhost  # or your RabbitMQ container name
RABBITMQ_PORT=5672
RABBITMQ_USER=guest
RABBITMQ_PASS=guest

# Coding.net Configuration
CODING_USER=your_username
CODING_TOKEN=your_token

# GitHub Configuration
GITHUB_USER=your_username
GITHUB_TOKEN=your_token
GITHUB_WEBHOOK_SECRET=your_webhook_secret

# Application Configuration
WORKSPACE=/app/runtime/repos  # Container path
LISTEN_HOST=0.0.0.0
LISTEN_PORT=8800

7. Volume Mounting for Persistent Data

If you need to persist repository data across container restarts:

docker run --env-file .env \
  -e MAIN_BIN=api \
  -p 8800:8800 \
  -v $(pwd)/runtime:/app/runtime \
  webhook-cd

Usage

1. Configure Webhooks

For Coding.net Projects

For each project in Coding.net that you want to integrate:

  1. Go to your project settings in Coding.net.
  2. Find the "Webhooks" section.
  3. Add a new webhook with the following details:
    • Payload URL: http://<your_server_address>:<PORT>/{platform}/{team}/{project}
      • Replace <your_server_address> with the IP or hostname where Webhook-CD is running.
      • Replace <PORT> with the port Webhook-CD is listening on (e.g., 8800).
      • Replace {platform}, {team}, and {project} with the actual platform identifier (e.g., coding), your team name, and your project name respectively.
    • Content type: application/json
    • Secret Token: (Optional but recommended) If you set a secret token, you will need to modify the application to validate it. The current index.js primarily checks the User-Agent.
    • Events: Select the events you want to trigger the webhook. This tool is primarily designed for "Merge Request" events (Push, Opened, Merged, Closed, Commented).
    • Ensure the webhook is active.

Important: The application currently authenticates Coding.net webhooks by checking if the User-Agent header is Coding.net Hook.

For GitHub Projects

For each GitHub repository that you want to integrate:

  1. Go to your repository settings on GitHub.
  2. Navigate to "Settings" > "Webhooks" > "Add webhook".
  3. Configure the webhook with the following details:
    • Payload URL: http://<your_server_address>:<PORT>/github/{owner}/{repo}
      • Replace <your_server_address> with the IP or hostname where Webhook-CD is running.
      • Replace <PORT> with the port Webhook-CD is listening on (e.g., 8800).
      • Replace {owner} and {repo} with your GitHub username/organization and repository name.
    • Content type: application/json
    • Secret: (Optional but recommended) Use the same value as GITHUB_WEBHOOK_SECRET environment variable for signature verification.
    • Events: Select "Pull requests" to trigger on pull request events (opened, synchronized, closed).
    • Active: Ensure the webhook is active.

Important: GitHub webhooks support signature verification using HMAC-SHA256 when a secret is configured.

2. How Deployments Work

  1. When a configured event (e.g., a merge request is merged) occurs in Coding.net for a monitored repository and branch, Coding.net sends a webhook to your Webhook-CD instance.
  2. Webhook-CD processes the event and triggers the appropriate deployment actions.
  3. The system monitors for deployable events for the configured repositories.
  4. If a deployable event is found, Webhook-CD will attempt to perform Git operations (like git checkout, git reset --hard origin/{branch}, git pull) in the corresponding local directory. Ensure this directory exists and is properly initialized as a Git repository, or is a location where the tool can clone into.

CLI Tool (wcd)

The project includes a CLI tool named wcd (defined in package.json's bin field).

Installation (if not already available via npm install): You might need to link it globally or run it via npx if not installed globally after npm install.

npm link # if you want to use 'wcd' directly
# or run via npx
npx @axiosleo/webhook-cd --help

Available Commands: The CLI allows you to manually trigger the internal event system. This can be useful for testing or re-processing tasks. The commands are defined in the commands/ directory.

  • wcd push <project> <repo> [options]: Simulates a GIT_MR_CREATED event. (Note: The file commands/push.js might have a naming error, referring to itself as 'pop').
  • wcd refresh <project> <repo> [options]: Simulates a GIT_MR_UPDATED event. (Note: The file commands/refresh.js might have a naming error).
  • wcd pop <project> <repo> [options]: Simulates a GIT_MR_CLOSED event.

Common Options for CLI commands:

  • --platform <name>: Platform name (default: coding).
  • -s, --source <branch>: Source branch (default: master).
  • -t, --target <branch>: Target branch (default: master).

Example:

wcd pop myteam myawesomeproject myfrontendapp --source dev-branch

License

This project is licensed under the MIT License. See the LICENSE file for details.

About

No description, website, or topics provided.

Resources

License

Stars

Watchers

Forks

Packages

No packages published