Switch deployment to docker(-compose).

Builds a ~22MB docker image right now!

Runs the CI tests in docker-compose, deploys with
Heroku container support (deletes Procfile).

Add docker-compose-based development flow.

Author: anp

.dockerignore

@@ -0,0 +1,5 @@
+*
+!src
+!Cargo.*
+!rfcbot.toml
+!rust-toolchain

.travis.yml

@@ -4,43 +4,31 @@ notifications:
 addons:
   apt:
     packages:
-      - libpq-dev
-      - pkg-config
-      - postgresql-11
-  postgresql: "11.2"
+    - libpq-dev
+    - pkg-config
 services:
-  - postgresql
+  - docker
 language: rust
-env: |
-  DATABASE_POOL_SIZE=5 \
-  DATABASE_URL=postgres:///rfcbot \
-  GITHUB_SCRAPE_INTERVAL=6000 \
-  GITHUB_USER_AGENT=rfcbot-rs \
-  GITHUB_WEBHOOK_SECRETS=none \
-  POST_COMMENTS=false \
-  RUST_BACKTRACE=1 \
-  RUST_LOG=error,rfcbot=debug
 before_install:
   - cargo fmt --all -- --check
 install:
-  - sudo systemctl start postgresql
-  - sudo -u postgres createuser --no-password --superuser $USER
-  - cargo install diesel_cli --no-default-features --features postgres -Z install-upgrade
+  - cargo install diesel_cli \
+      --no-default-features --features postgres -Z install-upgrade
+  - curl https://cli-assets.heroku.com/install-ubuntu.sh | sh
 before_script:
-  - diesel setup
-  - diesel migration run
-  - psql -q -d "$DATABASE_URL" < githubuser-backup.pg
+  - heroku container:login
+  - heroku container:pull --app=rfcbot-rs web
 script:
-  - export GITHUB_ACCESS_TOKEN="$GITHUB_ACCESS_TOKEN"
-  - cargo build
-  - cargo test
+  - docker-compose up -d
+  - ./setup-db.sh
+  - docker-compose exec -e RUST_BACKTRACE=1 web cargo test --release --locked
+  - heroku container:push --app=rfcbot-rs web
 before_deploy:
-  - DATABASE_URL=$(heroku config:get DATABASE_URL -a rfcbot-rs) diesel migration run
+  - DATABASE_URL=$(heroku config:get DATABASE_URL --app=rfcbot-rs) \
+    diesel migration run
 deploy:
-  provider: heroku
-  skip_cleanup: true
-  api_key:
-    secure: "HgoF2n6pnzUhZ4oBNRo3gSoxj/o9x7QW0fKaj9sEYDNkTgak9IX3ha0antoYKB7dMw7IvB8oqVAHqldlGlG/kxUDKN8kaYA4O7CZgKP0auykTkmH61FP7MKV0hV0vyFfJLSraClVw/DL5tjYybs/rSBWFAvwttIB7QlcFv5sLIuwe5nHzftFfxba0H1+0oynO55FlmRzFBWydlFVNTsGVRddFHPgG3f2V9YYIjT18MKvn+BcV7WUq6QweB/3RjwTDYzws9mcWt6i3ju1D4nDTDVbtzeLs7Yt+NhnvrDeB+2FWSrV03LCjnnzv2zCUaROU522Vns0ZC/+2b9V3YbcCBJRA8QLBdo12JUCUzrdk8DOqIYKwlW6WbI3DcgRGz9Li7Dw2466+lsPidBfcdxnziTvc+r/2r6dKv3q7nwDpoZEyxnnTumPf50PJZs/lDu99Vo5C2NIdN2O5sqDHd8AJTLtWrY3FdN5MGKqTm0R9rVnObvUkTKbZJrDn39O5MU3mnHINUT+6ZCnNgra7rNn7Ai3tA1RKPB7kIZo41Td8j56iSLANXziCh0MikPfQv4x1scl3c1h5bR8ZqxVqH4IODjDJrJv43+hsbE9xGsmrC4OaqImLUpl/8zi6a4zZ8Na1AiFKVcdLZ9xt40/2yg7ysn53SCpQ5vty/j36clNwAE="
+  provider: script
+  script: heroku container:release --app=rfcbot-rs
 before_cache:
   # don't cache the cargo registry
   - rm -rf "$HOME/.cargo/registry"

CONTRIBUTING.md

@@ -4,22 +4,6 @@
 
 Deployed to https://rfcbot.rs right now.
 
-## ToC
-
-* [Development](#development)
-  * [Configuring environment variables](#configuring-environment-variables)
-  * [Running server processes](#running-server-processes)
-  * [Database Connection](#database-connection)
-* [Configuration](#configuration)
-  * [Rust Version](#rust-version)
-  * [Environment variables](#environment-variables)
-* [Database](#database)
-* [Bootstrapping](#bootstrapping)
-* [Scraping](#scraping)
-* [Deployment](#deployment)
-* [Conduct](#conduct)
-* [License](#license)
-
 ## Development
 
 ### Chat
@@ -28,84 +12,38 @@ There is an `#rfcbot` channel in the `ops` section of the [rust-lang discord ser
 
 ### Rust Version
 
-Rust nightly is required, as rfcbot uses [Rocket](rocket.rs). If you use rustup, this version is controlled by a `rust-toolchain` file at the repository root.
-
-### Heroku CLI
-
-https://devcenter.heroku.com/articles/heroku-cli
-
-### Environment variables
+Rust nightly is required, as rfcbot uses [Rocket](rocket.rs). If you use rustup, this version is managed for you by a `rust-toolchain` file at the repository root.
 
-See config.rs for the environment variables expected. Also, rocket env vars are supported.
-
-You can also create a [local `.env` file](https://github.com/dotenv-rs/dotenv#examples) which will 
-be added to the environment variables the server sees.
-
-### Database dumps
+### Running locally
 
-It can be useful to have a database with some existing data to start from. "Bootstrap" files are available at https://www.dropbox.com/sh/dl4pxj1d49ici1f/AAAzZQxWVqQzVk_zOksn0Rbya?dl=0. They usually are behind several migrations, so you'll still need to run the migrations if you start from one.
-
-See [below](#database) for information about restoring from a bootstrap file.
-
-### Running server processes
-
-There are two daemons to run, one for the front-end development server, and one for the back-end API server and scraper. It's recommended to run these in two separate terminal windows/tabs/sessions.
-
-You may need to run database migrations if the bootstrap SQL file is stale:
-
-```
-$ diesel migration run
-```
+Install [Docker](https://docker.dom) and [docker-compose](https://docs.docker.com/compose) and ensure they're working.
 
-To run the back-end API server and scraper:
+Next, start development services and initialize the database:
 
 ```
-$ cargo run
+cargo install diesel_cli
+docker-compose up -d
+./setup-db.sh
 ```
 
-**NOTE:** The API server process needs to be manually restarted whenever you want to see code changes reflected in their behavior, or whenever you run migrations on the test database. A `Ctrl+C` followed by `Up` and `Enter` usually works if running them through cargo. `cargo watch` is also a nice tool.
-
-### Database connection
-
-If you want to perform any database action, make sure you have a reachable installation of PostgreSQL that is configured with the DATABASE_URL environment variable.
-
-## Configuration
-
-### Environment variables
-
-Note that you can configure the Rocket web server using environment variables like `ROCKET_PORT`, according to the Rocket [configuration guide](https://rocket.rs/guide/configuration/).
+After this you may need to run `docker-compose down`/`docker-compose up -d` for the server process to see the database updates.
 
-* `DATABASE_URL`: postgres database URL
-* `DATABASE_POOL_SIZE`: number of connections to maintain in the pool
-* `GITHUB_ACCESS_TOKEN`: your access token from GitHub. See [this page](https://help.github.com/articles/creating-an-access-token-for-command-line-use/) for more information. You shouldn't need to check any of the boxes for granting scopes when creating it.
-* `GITHUB_USER_AGENT`: the UA string to send to GitHub (they request that you send your GitHub username or the app name you registered for the client ID)
-* `GITHUB_WEBHOOK_SECRETS`: a comma-delimited string of the secrets used for any ingestion webhooks. The webhook handler will attempt to validate any POST'd webhook against each secret until it either finds a matching one or runs out.
-* `RUST_LOG`: the logging configuration for [env_logger](https://crates.io/crates/env_logger). If you're unfamiliar, you can read about it in the documentation linked on crates.io. If it's not defined, logging will default to `info!()` and above.
-* `GITHUB_SCRAPE_INTERVAL`: time (in minutes) to wait in between GitHub scrapes (scraping is disabled if this environment variable is omitted)
-* `POST_COMMENTS`: whether to post RFC bot comments on issues -- either `true` or `false`. Be very careful setting to true when testing -- it will post comments using whatever account is associated with the GitHub API key you provide.
+By default this stores your database files in `target/data/`, so any temporary changes you make to the database will be removed by a `cargo clean` and you'll need to run the above commands again.
 
-## Database
-
-To init, make sure `DATABASE_URL` is set and postgres is running, and:
-
-```
-cargo install diesel_cli
-diesel setup
-diesel print-schema > src/domain/schema.rs
-```
+### Database dumps
 
-If the database setup experiences errors, your user may not have permission to create databases or
-there may be other configuration problems.
+It can be useful to have a database with some existing data to start from. "Bootstrap" files are available at https://www.dropbox.com/sh/dl4pxj1d49ici1f/AAAzZQxWVqQzVk_zOksn0Rbya?dl=0.
 
-That should have the database you've specified ready to receive data. Then you can run some of the bootstrapping commands (see below). Alternatively, you can use `bootstrap.sql` to get a nice starting point for the database (note that this isn't maintained regularly).
+Assuming that you download the most recent file in the above folder and name it `bootstrap.sql`:
 
 ```bash
-psql -d $DB_NAME_HERE -f bootstrap.sql
+# see setup-db.sh for the url to use here
+psql -d $DATABASE_URL -f bootstrap.sql
 ```
 
 ## Deployment
 
-Deployed to Heroku via TravisCI from the master branch.
+Deployed to Heroku via TravisCI from the master branch. See (.travis.yml)[./travis.yml] for an up-to-date listing of the actions.
 
 ## Conduct
 

Cargo.lock

@@ -13,6 +13,7 @@ itertools = "0.8.0"
 lazy_static = "1.2.0"
 log = "0.4.6"
 maplit = "1.0.1"
+openssl = "*" # needs to be included for workaround from https://github.com/sgrif/pq-sys/issues/25
 reqwest = "0.9.12"
 rocket = "0.4.0"
 rocket_contrib = { version = "0.4.0", features = ["json", "handlebars_templates"] }

Cargo.toml

@@ -0,0 +1,31 @@
+###############################################################################
+# builder image
+
+FROM clux/muslrust:stable as builder
+
+WORKDIR /rfcbot
+RUN USER=root cargo init --vcs none
+
+COPY rust-toolchain ./
+RUN rustc --version && rustup target add x86_64-unknown-linux-musl
+
+COPY Cargo.toml Cargo.lock ./
+RUN cargo build --release
+
+COPY . ./
+# cargo apparently uses mtime and docker doesn't modify it, needed to rebuild:
+RUN touch src/main.rs
+RUN cargo build --release --locked
+
+###############################################################################
+# runner image
+
+FROM alpine:latest
+RUN apk --no-cache add ca-certificates
+
+# heroku runs as non-root
+RUN adduser -D notroot
+USER notroot
+
+COPY --from=builder /rfcbot/target/x86_64-unknown-linux-musl/release/rfcbot-rs /usr/local/bin/rfcbot
+CMD ROCKET_PORT=$PORT /usr/local/bin/rfcbot

Dockerfile

@@ -1,3 +1,3 @@
 [print_schema]
-file = "src/schema.rs"
+file = "src/domain/schema.rs"
 with_docs = true

Procfile

@@ -0,0 +1,33 @@
+version: '3.4'
+services:
+  web:
+    build:
+      context: .
+      target: builder
+    command: cargo run --release --locked
+    depends_on:
+      - db
+    ports:
+      - "9003:9003"
+    environment:
+      - DATABASE_POOL_SIZE=5
+      - DATABASE_URL=postgres://postgres:ughfineokifitsfordebugging@db:5432/rfcbot
+      - GITHUB_ACCESS_TOKEN=
+      - GITHUB_SCRAPE_INTERVAL=6000
+      - GITHUB_USER_AGENT=rfcbot-rs
+      - GITHUB_WEBHOOK_SECRETS=none
+      - POST_COMMENTS=false
+      - ROCKET_ENV=dev
+      - ROCKET_PORT=9003
+      - RUST_BACKTRACE=1
+      - RUST_LOG=error,rfcbot=debug
+  db:
+    image: "postgres:9"
+    restart: always
+    ports:
+      - "54320:5432"
+    environment:
+      - POSTGRES_DB=rfcbot
+      - POSTGRES_PASSWORD=ughfineokifitsfordebugging
+    volumes:
+      - ./target/data:/var/lib/postgresql/data

diesel.toml

@@ -0,0 +1,6 @@
+-- This file was automatically created by Diesel to setup helper functions
+-- and other internal bookkeeping. This file is safe to edit, any future
+-- changes will be added to existing projects as new migrations.
+
+DROP FUNCTION IF EXISTS diesel_manage_updated_at(_tbl regclass);
+DROP FUNCTION IF EXISTS diesel_set_updated_at();

docker-compose.yml

@@ -0,0 +1,36 @@
+-- This file was automatically created by Diesel to setup helper functions
+-- and other internal bookkeeping. This file is safe to edit, any future
+-- changes will be added to existing projects as new migrations.
+
+
+
+
+-- Sets up a trigger for the given table to automatically set a column called
+-- `updated_at` whenever the row is modified (unless `updated_at` was included
+-- in the modified columns)
+--
+-- # Example
+--
+-- ```sql
+-- CREATE TABLE users (id SERIAL PRIMARY KEY, updated_at TIMESTAMP NOT NULL DEFAULT NOW());
+--
+-- SELECT diesel_manage_updated_at('users');
+-- ```
+CREATE OR REPLACE FUNCTION diesel_manage_updated_at(_tbl regclass) RETURNS VOID AS $$
+BEGIN
+    EXECUTE format('CREATE TRIGGER set_updated_at BEFORE UPDATE ON %s
+                    FOR EACH ROW EXECUTE PROCEDURE diesel_set_updated_at()', _tbl);
+END;
+$$ LANGUAGE plpgsql;
+
+CREATE OR REPLACE FUNCTION diesel_set_updated_at() RETURNS trigger AS $$
+BEGIN
+    IF (
+        NEW IS DISTINCT FROM OLD AND
+        NEW.updated_at IS NOT DISTINCT FROM OLD.updated_at
+    ) THEN
+        NEW.updated_at := current_timestamp;
+    END IF;
+    RETURN NEW;
+END;
+$$ LANGUAGE plpgsql;

migrations/00000000000000_diesel_initial_setup/down.sql

@@ -0,0 +1,16 @@
+#!/usr/bin/env bash
+
+set -xe
+
+export DATABASE_URL="postgres://postgres:ughfineokifitsfordebugging@localhost:54320/rfcbot"
+
+until psql -d "$DATABASE_URL" -c '\q'; do
+  >&2 echo "Postgres is unavailable - sleeping"
+  sleep 1
+done
+
+>&2 echo "Postgres is up - executing commands"
+
+diesel database setup
+diesel migration run
+psql -q -d "$DATABASE_URL" --file ./githubuser-backup.pg

migrations/00000000000000_diesel_initial_setup/up.sql

@@ -1,4 +1,31 @@
 // Copyright 2016 Adam Perry. Dual-licensed MIT and Apache 2.0 (see LICENSE files for details).
+//! Configuration environment variables for rfcbot.
+//!
+//! Note that you can configure the Rocket web server using environment variables like
+//! `ROCKET_PORT`, according to the Rocket
+//! [configuration guide](https://rocket.rs/guide/configuration/).
+//!
+//! Here are the variables rfcbot expects to see in its environment:
+//!
+//! * `DATABASE_URL`: postgres database URL
+//! * `DATABASE_POOL_SIZE`: number of connections to maintain in the pool
+//! * `GITHUB_ACCESS_TOKEN`: your access token from GitHub. See
+//!   [this page](https://help.github.com/articles/creating-an-access-token-for-command-line-use/)
+//!   for more information. You shouldn't need to check any of the boxes for granting scopes when
+//!   creating it.
+//! * `GITHUB_USER_AGENT`: the UA string to send to GitHub (they request that you send your GitHub
+//!   username or the app name you registered for the client ID)
+//! * `GITHUB_WEBHOOK_SECRETS`: a comma-delimited string of the secrets used for any ingestion
+//!   webhooks. The webhook handler will attempt to validate any POST'd webhook against each secret
+//!   until it either finds a matching one or runs out.
+//! * `RUST_LOG`: the logging configuration for [env_logger](https://crates.io/crates/env_logger).
+//!   If you're unfamiliar, you can read about it in the documentation linked on crates.io. If it's
+//!   not defined, logging will default to `info!()` and above.
+//! * `GITHUB_SCRAPE_INTERVAL`: time (in minutes) to wait in between GitHub scrapes (scraping is
+//!   disabled if this environment variable is omitted)
+//! * `POST_COMMENTS`: whether to post RFC bot comments on issues -- either `true` or `false`. Be
+//!   very careful setting to true when testing -- it will post comments using whatever account is
+//!   associated with the GitHub API key you provide.
 
 use std::collections::BTreeMap;
 use std::env;