Skip to content
This repository has been archived by the owner on Oct 14, 2020. It is now read-only.

Commit

Permalink
updated and fully working cli project
Browse files Browse the repository at this point in the history
  • Loading branch information
@jhoffner
jhoffner committed Apr 15, 2016
1 parent 670d921 commit da575d1
Show file tree
Hide file tree
Showing 233 changed files with 81,575 additions and 1,625 deletions.
8 changes: 3 additions & 5 deletions .dockerignore
View file
Original file line number Diff line number Diff line change
Expand Up @@ -11,11 +11,9 @@ TestFixture.java
.idea/
target/*
.lein-repl-history
example.rb
/setup
/examples
/test
/pull.js
/server.js
/push.js
/README.md
/README.md
node_modules/*
docker/*
1 change: 1 addition & 0 deletions .gitignore
View file
Original file line number Diff line number Diff line change
Expand Up @@ -21,3 +21,4 @@ example.rb
Dockerfile
*.o
*.hi
\#*
681 changes: 19 additions & 662 deletions LICENSE
View file

Large diffs are not rendered by default.

55 changes: 55 additions & 0 deletions Makefile
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
HOSTNAME=codewars
# func was removed due to issues compiling haskell using the latest base image.
# We need to fix that image before we can ever update that image/codebase again :(
#CONTAINERS=dotnet func jvm node python ruby systems
CONTAINERS=dotnet jvm node python ruby systems alt func erlang

ALL_CONTAINERS=${CONTAINERS} base

.PHONY: ${ALL_CONTAINERS} clean docker_rm docker_rmi

all: ${CONTAINERS}

base:
cp docker/$@.docker ./Dockerfile
docker build -t $(HOSTNAME)/$@-runner .

${CONTAINERS}: base
cp docker/$@.docker ./Dockerfile
docker build -t $(HOSTNAME)/$@-runner .

# Push docker containers to registry
push_to_registry:
docker push $(HOSTNAME)/base-runner
echo $(patsubst %, $(HOSTNAME)/%-runner, $(CONTAINERS)) | xargs -n 1 docker push

# Remove docker processes that have exited cleanly
docker_rm_exited:
[ ! -n "$(shell docker ps -a | grep Exit | cut -d ' ' -f 1)" ] || echo $(shell docker ps -a | grep Exit | cut -d ' ' -f 1) | xargs -n 1 docker rm -f

# Kill temporary built images might fail if not exited cleanly
docker_rmi_temporary:
[ ! -n "$(shell docker images --no-trunc | grep none | sed -e 's/\s\s*/\t/g' | cut -f3)" ] || docker images --no-trunc | grep none | sed -e 's/\s\s*/\t/g' | cut -f3 | xargs -n 1 docker rmi -f

# Kill all of the in-flight and exited docker containers
docker_rm:
docker ps -q | xargs docker stop
[ ! -n "$(shell docker ps -a -q)" ] || echo $(shell docker ps -a -q) | xargs -n 1 docker rm -f

# Kill all docker images
docker_rmi: docker_rm
[ ! -n "$(shell docker images -q)" ] || docker images -q | xargs -n 1 docker rmi -f

clean: docker_rm_exited docker_rmi_temporary
for i in $(shell for i in $(HOSTNAME)/base $(patsubst %, $(HOSTNAME)/%-runner, $(CONTAINERS)) ; do docker images | grep $$i | sed -e 's/\s\s*/\t/g' | cut -f1; done) ; do docker rmi -f $$i ; done

deep-clean: docker_rmi

pull:
docker pull codewars/ruby-runner
docker pull codewars/node-runner
docker pull codewars/python-runner
docker pull codewars/dotnet-runner
docker pull codewars/jvm-runner
docker pull codewars/systems-runner
docker pull codewars/func-runner
75 changes: 74 additions & 1 deletion README.md
View file
Original file line number Diff line number Diff line change
@@ -1,2 +1,75 @@
# runner

TODO: update documentation
This project is both a command-line utility and server, used by [Codewars](http://www.codewars.com) to execute small sets of code within various languages, using various testing frameworks.

You can run `node run --help` to view information about which arguments are supported.

## Purpose

The purpose of this project is to provide a low level ability to run 'Kata'. It provides a mechanism for executing different configurations of code using various languages and testing frameworks.

Docker can be utilized in order to sandbox code execution. A server is provided that can accept 'run' requests and execute them within a Docker container.

## Language Support Status

Many languages are currently supported in various states of completeness. This list tries to keep track of each.

**Legend:** `!!!` = Failing Specs, `???` = Status is unknown, `*` = Any

| Language | solutionOnly | testSuite | Codewars | Strive | Docker Image | Examples | Notes |
|--------------|--------------|----------------|--------------|----------------|----------------|--------------|-------------------------------------------------------------------------|
| Bash | ✓ | | Kumite | | * | | |
| C | ✓ | Failing | ? | | systems-runner | | |
| Clojure | ✓ | clojure.test | clojure.test | clojure.test | func-runner | clojure.test | |
| CoffeeScript | ✓ | cw-2, mocha | cw-2 | cw-2, mocha | node-runner | cw-2 | |
| CPP | ✓ | | | | systems-runner | | |
| C# | ✓ | nunit | nunit | nunit | dotnet-runner | nunit | |
| Elixir | ✓ | | | | erlang-runner | | |
| Erlang | ✓ | | | | erlang-runner | | |
| F# | ✓ | | Kumite Only | | dotnet-runner | | |
| Go | ✓ | | Kumite Only | | alt-runner | | |
| Groovy | ✓ | | Kumite Only | | jvm-runner | | |
| Haskell | ✓ | hspec!!! | hspec | hspec | func-runner | hspec | An older version is running on CW & Strive that is fully functional |
| Java | ✓ | junit | Yes | Yes | jvm-runner | junit | |
| JavaScript | ✓ | cw-2, mocha | cw-2 | cw-2, mocha | node-runner | cw-2 | |
| Julia | ✓!!! | Failing | | | | | |
| Lisp | ✓ | | Kumite Only | | func-runner | | |
| Lua | ✓ | | Kumite Only | | alt-runner | | |
| ObjC | ??? | ??? | | | | | |
| OCAML | ✓ | | Kumite Only | | func-runner | | |
| Perl | ✓ | | Kumite Only | | * | | |
| Php | ✓ | | Kumite Only | | alt-runner | | |
| Python 2 | ✓ | cw-2, unittest | cw-2 | cw-2, unittest | python-runner | cw-2 | |
| Python 3 | ✓ | cw-2, unittest | | cw-2, unittest | python-runner | cw-2 | |
| R | ✓ | | | | alt-runner | | |
| Racket | ✓ | | Kumite Only | | func-runner | | |
| Ruby | ✓ | cw-2, rspec | cw-2 | cw-2, rspec | ruby-runner | cw-2 | |
| Rust | ✓ | | | | | | |
| Scala | ✓ | | Kumite Only | | jvm-runner | | |
| Swift | ??? | ??? | | | | | Current contribution designed for OSX, need to move to OS linux version |
| TypeScript | ✓ | mocha | Kumite Only | | node-runner | | TypeScript utilizes `require` instead of concatenating files |


## Setup

You should have Docker installed, if not do that first. Before you can run any of the code
environments you will need to build the proper Docker image. To get started lets work with the
node image.

Run `make base node` to build the base and node images. This will take a few minutes.

Once you image is built, you can create a container to work within it. Doing this means you do not
have to worry about having any of the project dependencies loaded directly on your machine.

Run the following command:

```
docker run -it --rm --entrypoint bash -v $(pwd)/lib:/runner/lib -v $(pwd)/frameworks:/runner/frameworks -v $(pwd)/test:/runner/test codewars/node-runner
```

This will create a new container and send you into the instance with your project's lib and test directories mounted
as volumes. Mounting as a volume allows you to change files on your local machine and have those changes available to you
from within the container.

> We do not mount the entire directory because that would overwrite things such as your node_modules directory. If you need
to update these you should `make node` the image to ensure you are always testing against the correct packages.
19 changes: 19 additions & 0 deletions docker-compose.yml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
version: '2'
services:
base:
build:
context: .
dockerfile: base.docker
image: codewars/base-runner

ruby:
build:
context: .
dockerfile: ruby.docker
image: codewars/ruby-runner

node:
build:
context: .
dockerfile: node.docker
image: codewars/node-runner
57 changes: 4 additions & 53 deletions docker/alt.docker
View file
Original file line number Diff line number Diff line change
@@ -1,32 +1,9 @@
# BUILD-USING: docker build -t codewars/runner-julia .
# TEST-USING: docker run --rm -i -t --name=test-runner-julia --entrypoint=/bin/bash codewars/runner-julia -s
# RUN-USING: docker run --rm --name=runner-julia codewars/runner-julia --help

# Pull base image.
FROM codewars/runner-base

# Install Julia
# Julia is really slow, but v0.3 is okay (see http://stackoverflow.com/a/20566032)
# In the future, don't use nightly builds, use releases
RUN add-apt-repository ppa:staticfloat/julianightlies
RUN add-apt-repository ppa:staticfloat/julia-deps
RUN apt-get update
RUN apt-get -y install julia
# Nightly builds have a noisy OpenBLAS error, workaround
RUN mv /usr/bin/julia /usr/bin/julia-noisy
RUN printf '#!/bin/bash\njulia-noisy "$@" 2> >(grep -v "OpenBLAS : Your OS does not support AVX instructions." 1>&2)' > /usr/bin/julia
RUN chmod a+x /usr/bin/julia

# Install erlang
RUN echo "deb http://packages.erlang-solutions.com/ubuntu trusty contrib" >> /etc/apt/sources.list
RUN curl http://packages.erlang-solutions.com/ubuntu/erlang_solutions.asc | apt-key add -
RUN apt-get update
RUN apt-get -y install erlang-nox erlang-dev
FROM codewars/base-runner

# Install PHP
RUN apt-get -y install php-pear php5-dev libcurl3-openssl-dev


# Install GoLang
WORKDIR /tmp
# http://blog.labix.org/2013/06/15/in-flight-deb-packages-of-go
Expand All @@ -36,12 +13,6 @@ RUN ./godeb install 1.3.1
RUN rm godeb
#RUN apt-get install -y golang

# Install Rust 0.11
# Workaround since the author of this PPA is Scandanavian, with a unicode name, and the tools hate this
RUN add-apt-repository "deb http://ppa.launchpad.net/hansjorg/rust/ubuntu trusty main"
RUN apt-key adv --keyserver keyserver.ubuntu.com --recv-keys BD6B6386
RUN apt-get update
RUN apt-get -y install rust-0.11

# Install Lua
RUN apt-get install -y lua5.2
Expand All @@ -51,30 +22,13 @@ RUN apt-get install -y luarocks
#RUN apt-add-repository ppa:swi-prolog/stable && apt-get update
#RUN apt-get install -y swi-prolog

# Install ZMQ
#RUN luarocks install https://raw.github.com/Neopallium/lua-zmq/master/rockspecs/lua-zmq-scm-1.rockspec
RUN pecl install zmq-beta #php
RUN apt-get install -y libzeromq-perl #perl

# Install Elixir
# Elixir needs a UTF-8 locale
RUN apt-get install -y locales && dpkg-reconfigure locales && locale-gen en_US.UTF-8
WORKDIR /tmp
RUN git clone https://github.com/elixir-lang/elixir.git
WORKDIR /tmp/elixir
RUN git checkout tags/v1.0.0
RUN make install
RUN rm -rf /tmp/elixir



# Install GNU R
RUN apt-get install -y r-base

# ADD cli-runner and install node deps
ADD . /codewars
ADD . /runner

WORKDIR /codewars
WORKDIR /runner
RUN npm install

# Run the test suite to make sure this thing works
Expand All @@ -83,10 +37,7 @@ USER codewarrior
# Set environment variables
ENV USER codewarrior
ENV HOME /home/codewarrior
ENV LANG en_US.UTF-8
ENV LANGUAGE en_US:en
ENV LC_ALL en_US.UTF-8
RUN mocha -t 5000 test/runners/{go,php,erlang,elixir,julia,rust,go,lua,perl,r}_spec.js
RUN mocha -t 3000 test/runners/{php,go,r,lua}_spec.js

#timeout is a fallback in case an error with node
#prevents it from exiting properly
Expand Down
68 changes: 41 additions & 27 deletions docker/base.docker
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,35 @@
# RUN-USING: docker run --rm --name=runner-base codewars/runner-base --help

# Pull base image.
FROM dockerfile/ubuntu
FROM buildpack-deps:trusty

RUN apt-get update -qq

# gpg keys listed at https://github.com/nodejs/node
RUN set -ex \
&& for key in \
9554F04D7259F04124DE6B476D5A82AC7E37093B \
94AE36675C464D64BAFA68DD7434390BDBE9B9C5 \
0034A06D9D9B0064CE8ADF6BF1747F4AD2306D93 \
FD3A5288F042B6850C66B31F09FE44734EB7990E \
71DCFD284A79C3B38668286BC97EC7A07EDE3FC1 \
DD8F2338BAE7501E3DD5AC78C273792F7D83545D \
B9AE9905FFD7803F25714661B63B535A4C206CA9 \
C4F0DFFF4E8C1A8236409D08E73BC641CC11F4C8 \
; do \
gpg --keyserver ha.pool.sks-keyservers.net --recv-keys "$key"; \
done

ENV NPM_CONFIG_LOGLEVEL info
ENV NODE_VERSION 5.8.0

# Install NodeJS
RUN curl -SLO "https://nodejs.org/dist/v$NODE_VERSION/node-v$NODE_VERSION-linux-x64.tar.xz" \
&& curl -SLO "https://nodejs.org/dist/v$NODE_VERSION/SHASUMS256.txt.asc" \
&& gpg --batch --decrypt --output SHASUMS256.txt SHASUMS256.txt.asc \
&& grep " node-v$NODE_VERSION-linux-x64.tar.xz\$" SHASUMS256.txt | sha256sum -c - \
&& tar -xJf "node-v$NODE_VERSION-linux-x64.tar.xz" -C /usr/local --strip-components=1 \
&& rm "node-v$NODE_VERSION-linux-x64.tar.xz" SHASUMS256.txt.asc SHASUMS256.txt

# Set the env variables to non-interactive
ENV DEBIAN_FRONTEND noninteractive
Expand All @@ -21,42 +49,28 @@ CMD ["bash"]

# Append any relevant run args as per the help

RUN apt-get update

ENV NODE_ENV production
ENTRYPOINT ["timeout", "15", "node"]

# Install Node.js (needed to run actual runner)
RUN add-apt-repository ppa:chris-lea/node.js
RUN apt-get update
RUN apt-get install -y nodejs
RUN su codewarrior -c "echo '\n# Node.js\nexport PATH=\"/codewars/node_modules/.bin:$PATH\"' >> ~codewarrior/.bash_profile"

# Install Redis
RUN apt-get install -y redis-server

# Install MongoDB
RUN apt-get install -y mongodb-server && \
mkdir -p /data/db && \
chown codewarrior:codewarrior /data/db

# Install SQLITE
RUN apt-get install -y sqlite libsqlite3-dev

# Install ElasticSearch
RUN wget https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-1.1.1.deb && \
dpkg -i elasticsearch-1.1.1.deb
# Install MongoDB
RUN apt-get install -y mongodb

# Install RabitMQ
RUN apt-get install -y rabbitmq-server
RUN mkdir /.npm-global
RUN npm config set prefix '/.npm-global'
RUN su codewarrior -c "echo 'export PATH=\"/.npm-global/bin:$PATH\"' >> ~/.profile"
ENV PATH /.npm-global/bin:$PATH

# Install ZMQ
RUN apt-get install -y libzmq3-dev
# Install Node testing frameworks & additional frameworks
RUN npm install -g chai mocha

# Install Python Basics
RUN apt-get install -y python-software-properties

# Install Node testing frameworks & additional frameworks
RUN npm -g install chai mocha
RUN apt-get install -y python-minimal

# Install pkg-config for things that need to compile from source
RUN apt-get install -y pkg-config
# Install Git
RUN apt-get -y install git
Loading

0 comments on commit da575d1

Please sign in to comment.