Skip to content

Latest commit

 

History

History
135 lines (107 loc) · 5.44 KB

development.md

File metadata and controls

135 lines (107 loc) · 5.44 KB

Development guide

This document should help you make sense of the codebase and provide guidance on working with it and testing it locally.

Directory structure:

  • migrations
    • sqlx migrations that are the source of truth for database schema
  • src/bors
    • Bors commands and their handlers.
  • src/database
    • Database access layer built on top of sqlx.
  • src/github
    • Communication with the GitHub API and definitions of GitHub webhook messages.

Architecture diagram

The following diagram shows a simplified view on the important state entities of Bors. bors_process handles events generated by webhooks. It uses a shared global state through BorsContext, which holds a shared connection to the database and a command parser. It also has access to a map of repository state. Each repository state contains an API client for that repository, its loaded config, and permissions loaded from the Team API.

---
title: Important entities
---
flowchart
    BorsContext
    repo_client_1["GithubRepositoryClient"]
    repo_client_2["GithubRepositoryClient"]

    repo_state_1["RepositoryState"]
    repo_state_2["RepositoryState"]

    BorsContext --> repo_state_1 --> repo_client_1 --> repo1
    BorsContext --> repo_state_2 --> repo_client_2 --> repo2

    repo_state_1 --> cr1["Config (repo 1)"]
    repo_state_1 --> pr1["Permissions (repo 1)"]

    repo_state_2 --> cr2["Config (repo 2)"]
    repo_state_2 --> pr2["Permissions (repo 2)"]

    BorsContext --> db[(Database)]

    bors_process --> BorsContext
Loading

How to test bors on live repositories

Bors has a cargo test suite that you can run locally, but sometimes nothing beats an actual test on live, GitHub repositories. Sadly, the process is currently quite involved, but it can still be done if needed.

Note: we will eventually configure bors to have a staging environment e.g. at https://github.com/rust-lang/bors-kindergarten, however this is not configured yet.

One-time setup:

  • Create your own GitHub app.
    • Configure its webhook secret.
    • Configure its private key.
    • Give it permissions for Actions (r/w), Checks (r), Commit statuses (r), Contents (r/w), Issues (r/w) and Pull requests (r/w).
    • Subscribe it to webhook events Check suite, Check run, Issue comment, Issues, Pull request, Pull request review, Pull request review comment and Workflow run.
  • Install your GitHub app on some test repository where you want to test bors.
    • Don't forget to configure rust-bors.toml in the root of the repository, and also some CI checks.

Everytime you want to run bors:

  • Run bors locally.
    • Set APP_ID to the ID of the app
    • Set WEBHOOK_SECRET to the webhook secret of the app.
    • Set PRIVATE_KEY to the private key of the app.
  • Set up some globally reachable URL/IP address for your computer, e.g. using ngrok.
    • Configure the webhook URL for your app to point to <address>/github.
  • Try @bors ping on some PR on the test repository :)

Faster compilation

Bors has a lot of dependencies and can be fairly slow to compile. If you want to improve compilation speed, you could use a faster linker, e.g. LLD.

To do that, install it (e.g. using sudo apt install lld) and then put this:

[build]
rustflags = ["-C", "link-arg=-fuse-ld=lld"]

into .cargo/config.toml.

Database

You must have sqlx-cli installed for the following commands to work.

$ cargo install [email protected] --no-default-features --features native-tls,postgres

The database can be set up with the docker-compose file in the root of the repository:

$ docker-compose up -d

Then, set the DATABASE_URL environment variable to the connection string of the database. The content of the variable is can be found in the .env.example file. If an .env file is present, the environment variable listed in it will be picked up automatically by sqlx.

$ export DATABASE_URL=postgres://bors:bors@localhost:5432/bors

Running tests

Having the database up and running, with the DATABASE_URL set, is required to run the tests.

Make sure to also run cargo sqlx migrate run to apply the migrations to the database.

Updating the DB schema

Caution

When adding a new NOT NULL column, always specify the DEFAULT value that will be backfilled during the migration! Otherwise, the migration might break the deployed bors service.

  1. Generate a new migration 
    $ cargo sqlx migrate add <new-migration>
  2. Change the migration manually in migrations/<timestamp>-<new-migration>.sql.
  3. Apply migrations to the Postgre DB. 
    $ cargo sqlx migrate run
  4. Add a test data file to tests/data/migrations/<timestamp>-<new-migration>.sql.
    • The file should contain SQL that inserts some reasonable data into a test database after the migration is applied. The goal is to check that we have a test database with production-like data, so that we can test that applying migrations will not produce errors on a non-empty database.
    • If it doesn't make sense to add any data to the migration (e.g. if the migration only adds an index), put -- Empty to satisfy migration tests into the file.

Generate .sqlx directory

$ cargo sqlx prepare -- --all-targets

After that, you should commit the changes to the .sqlx directory.

Logs in tests

By default, logs are disabled in tests. To enable them, add #[traced_test] on top of the test function.