Mixed fixes:

-Readme update
-Test Crashed message and unit test
-Test timeout message and unit test
-Replace \n with <:LF:> and unit test
-C unit test on docker build

Author: pibi

README.md

@@ -3,17 +3,17 @@
 [![Build Status](https://travis-ci.org/Codewars/codewars-runner-cli.svg?branch=master)](https://travis-ci.org/Codewars/codewars-runner-cli)
 
 This project is used by [Codewars](http://www.codewars.com) and [Qualified.io](https://www.qualified.io) to execute small sets of code within various languages, using various testing frameworks.
-Each time code is run, it is executed within a Docker container in order to secure unsafe code execution.  
+Each time code is run, it is executed within a Docker container in order to secure unsafe code execution.
 
 All execution is done within Docker, with a Node CLI app contained within each container
-that manages the code execution for that specific language environment and returns the result via stdout. 
+that manages the code execution for that specific language environment and returns the result via stdout.
 
 ## Contributions
 
 This project has been open-sourced so that the Codewars and Qualified communites can contribute support for new languages and
 frameworks. See the [Language Support Status section](#user-content-language-support-status) for more information regarding which languages are currently supported
 and where their Codewars/Qualified support has been added.
- 
+
 ## Basic Usage
 
 Within each Docker image, there is a copy of the Node executable and a `run` script. That script accepts multiple options
@@ -35,7 +35,7 @@ docker run --rm -it codewars/node-runner bash
 docker-compose run node-runner
 ```
 
-Or you could choose to execute the code outside of Docker by creating a container that will remove itself after it executes: 
+Or you could choose to execute the code outside of Docker by creating a container that will remove itself after it executes:
 
 ```
 # direct Docker call:
@@ -50,9 +50,9 @@ docker-compose run ruby -c "puts 'I ran inside of Docker using Ruby'"
 ### Integrated Test Suites
 The most significant aspect of this project is that integrated test support is built in to many languages. This is how
 Codewars and Qualified work, instead of testing STDOUT of a program, the executed code is tested using traditional
-code testing methods, with unit-tests and language-specific testing frameworks. 
+code testing methods, with unit-tests and language-specific testing frameworks.
 
-Here is a very simple example of running tests using the simplified Codewars testing framework. 
+Here is a very simple example of running tests using the simplified Codewars testing framework.
 
 ```
 # manually running docker
@@ -64,7 +64,7 @@ docker-compose run javascript -c "var a = 1;" -t cw -f "Test.assertEquals(a, 1)"
 
 Which would output `<PASSED::>Test Passed: Value == 1` to STDOUT.
 
-## Language Support Status 
+## Language Support Status
 
 Many languages are currently supported in various states of completeness. This list tries to keep track of each.
 
@@ -74,7 +74,7 @@ Many languages are currently supported in various states of completeness. This l
 |----------------|---------------|--------------|------------------|---------------|----------------|----------------|--------------|-------------------------------------------------------------------------|
 | Assembly (GAS) |               | !!!          |                  |               |                | systems-runner |              | Travis is failing, tests pass locally                                   |
 | Bash           |               | ✓            |                  | Kumite Only   |                | *              |              |                                                                         |
-| C              |               | !!!          | !!!              |               |                | systems-runner |              | Travis is failing, tests pass locally                                   |
+| C              | 99            | ✓            | criterion        |               |                | systems-runner |              |                                                                         |
 | Clojure        | 1.6.0         | ✓            | clojure.test     | clojure.test  | clojure.test   | func-runner    | clojure.test |                                                                         |
 | CoffeeScript   | 1.10.0        | ✓            | cw-2             | cw-2          | cw-2           | node-runner    | cw-2         |                                                                         |
 | C++            | 14            | ✓            | igloo            |               |                | systems-runner |              |                                                                         |
@@ -119,7 +119,7 @@ You should have [Docker](https://www.docker.com/) installed, if not do that firs
 environments you will need to build the proper Docker image. To get started lets work with the
 node image.
 
-Run `make node` to build the base and node images. This will take a few minutes. You can speed up the process by first 
+Run `make node` to build the base and node images. This will take a few minutes. You can speed up the process by first
 downloading the existing images that you intend to work on, which will allow you to only need to build when you make a change.
 
 For example, if you intend to work on the jvm image, you would do this:
@@ -129,7 +129,7 @@ For example, if you intend to work on the jvm image, you would do this:
 docker pull codewars/base-runner
 docker pull codewars/jvm-runner
 
-# now build the jvm image, so that any recent changes that have may not have been pushed yet get added into the image 
+# now build the jvm image, so that any recent changes that have may not have been pushed yet get added into the image
 make jvm
 ```
 
@@ -161,8 +161,8 @@ to update these you should `make {image_you_want_to_update}` the image to ensure
 We mentioned before that you also have the option of using [Docker Compose](https://docs.docker.com/compose/) to run the CLI tool. We have setup the `docker-compose.yml`
 file to provide very useful pre-configured services for making development easier. Instead of having to issue the long command
 mentioned above, you can simply run `docker-compose run node-runner` to bash into a fresh container with your local volumes already mounted.
- 
-All of the docker compose services are setup to mount volumes to make development easier, so that is the recommended way of 
+
+All of the docker compose services are setup to mount volumes to make development easier, so that is the recommended way of
 interacting with the codebase. You should note though that the compose file is unable to build images due to how
 the directory structure is laid out, so you have to first `make {runner_name}` the image before you can run it. Otherwise
 it will pull down the latest copy from Docker Hub.
@@ -181,7 +181,7 @@ docker-compose run typescript_test
 
 ## Test Suite Output Format
 
-A custom and very basic format is used for sending data out of the CLI tool. All formatted data is returned via STDOUT. 
+A custom and very basic format is used for sending data out of the CLI tool. All formatted data is returned via STDOUT.
 If you do nothing but write normal strings to STDOUT, then codewars.com will display each line as you would expect, unformatted.
 
 A small subset of commands is supported that can be used to format output. They are:
@@ -193,8 +193,8 @@ A small subset of commands is supported that can be used to format output. They
 - `<ERROR::>`
 - `<COMPLETEDIN::>`
 
-Prefixing a new line with these commands will cause that line to be formatted. 
-Since each new STDOUT line is considered a new piece of data, if you wish to format multiple lines as one 
+Prefixing a new line with these commands will cause that line to be formatted.
+Since each new STDOUT line is considered a new piece of data, if you wish to format multiple lines as one
 item (such as a multi line "passed" message), then you must replace all \n line feed characters with the `<:LF:>` token.
 
 For example, in Ruby, if you wanted to write a multi-line passed message:
@@ -207,8 +207,8 @@ end
 
 ### Nested Describes
 
-Some test frameworks support nested levels of describes. In order for our output to support multiple levels, 
-you must also use the `<COMPLETEDIN::>` token, which acts as a terminator for the current item. This should be used 
+Some test frameworks support nested levels of describes. In order for our output to support multiple levels,
+you must also use the `<COMPLETEDIN::>` token, which acts as a terminator for the current item. This should be used
 to terminate both `<DESCRIBE::>` and `<IT::>` statements.
 
 The following is a full example of what the output might look like, that supports nested describes:
@@ -218,7 +218,7 @@ The following is a full example of what the output might look like, that support
 <IT::>It should return a string
 <PASSED::>Test Passed
 <COMPLETEDIN::>23
-<IT::>It should return "foo" 
+<IT::>It should return "foo"
 This is some direct output (i.e. console.log("..."))
 <FAILED::>Expected "foo" but instead got ""
 <COMPLETEDIN::>10
@@ -236,21 +236,21 @@ statement, the 2nd completes the nested DESCRIBE and the 3rd completes the top l
 #### <COMPLETEDIN::> Details
 
 The value of COMPLETEDIN should be time spent executing the related statement, in milliseconds. It is not required
-to support time. `<COMPLETEDIN::>` is valid on its own, and in that case it is only used to terminate the current statement. 
+to support time. `<COMPLETEDIN::>` is valid on its own, and in that case it is only used to terminate the current statement.
 
 ### Why the custom format?
 
-Getting different test suites in different languages to all play together with the same format can be tricky. In many cases, 
-customizing the test suite output is very limited (sometimes requiring hacking). Because of this, using formats such as 
-XML and JSON are complicated because its not always possibly to correctly close out the data format when a program raises an exception. 
+Getting different test suites in different languages to all play together with the same format can be tricky. In many cases,
+customizing the test suite output is very limited (sometimes requiring hacking). Because of this, using formats such as
+XML and JSON are complicated because its not always possibly to correctly close out the data format when a program raises an exception.
 
 The format choosen was originally done so that at any point in time the program could exit while still having readable data.
 Other formats, such as TAP (Test Anything Protocol) could also be an option. However another requirement that we had when
 designing the format was to have it be incredibly simple yet flexible, so that Codewars.com could support more than simply
 outputing test results. With the current format there is nothing stopping you from outputing HTML, JS canvas code, etc in order
 to create a rich and even interactive test result output.
-  
-  
+
+
 ## How to add a new language
 
 > Note: These steps all assume you are adding a completely new language to the project. Many languages are currently in an incomplete
@@ -272,22 +272,22 @@ as a part of the test suite.
 
 The runner script is responsible for outputing a `run` method. This method utilizes the shovel helper which will handle
 all of the inter-process communication for you. The shovel config accepts strategies to determine how to handle running
-the code based on the options passed in. There are currently two types of strategies. 
+the code based on the options passed in. There are currently two types of strategies.
 
 - `solutionOnly`: Code is simply executed and its STDOUT returned. There is no test integration.
 - `testIntegration`: Code is executed within a configurable test suite and the test output is returned via STDOUT.
 
 Each of these strategies is passed in a `run` method which is used to ultimately execute the final command.
- 
+
 ## TODO
 
 ### Paradigm Shift
 
-There is currently no way of handling language/package versioning well. This is largely caused by the Node CLI tool having to be 
+There is currently no way of handling language/package versioning well. This is largely caused by the Node CLI tool having to be
 baked in to the Docker container. A more ideal solution would involve keeping the CLI outside of Docker (or within its own sibling container)
 and communicating to language specific containers. This would allow us to easily swap out container versions that have no
 dependencies on the CLI codebase. This would involve having to utilize the `docker cp` command to copy
-files to the docker container in order to be compiled. Upgrading to this newer paradigm is the next big phase of this project. 
+files to the docker container in order to be compiled. Upgrading to this newer paradigm is the next big phase of this project.
 
 ### Ability to send input parameters and return data
 

docker/systems.docker

@@ -54,7 +54,7 @@ USER codewarrior
 ENV USER codewarrior
 ENV HOME /home/codewarrior
 #RUN mocha -t 5000 test/runners/{gas,arm,nasm,cpp,c}_spec.js
-RUN mocha -t 5000 test/runners/cpp_spec.js
+RUN mocha -t 5000 test/runners/{cpp,c}_spec.js
 
 #timeout is a fallback in case an error with node
 #prevents it from exiting properly

frameworks/c/criterion.c

@@ -1,4 +1,5 @@
 #include <stdio.h>
+#include <string.h>
 #include <criterion/criterion.h>
 #include <criterion/hooks.h>
 
@@ -23,17 +24,29 @@ ReportHook(ASSERT)(struct criterion_assert_stats *stats) {
     if (stats->passed) {
       puts("<PASSED::>Test Passed");
     } else {
-      printf("<FAILED::>%s\n",stats->message);
+      printf("<FAILED::>");
+      // print the assertion message, replacing \n with <:LF:>
+      for (const char* p = stats->message; *p != '\0'; ++p) {
+          if (*p == '\n') {
+            printf("<:LF:>");
+          } else {
+            fputc(*p,stdout);
+          }
+      }
+      fputc('\n',stdout);
     }
 }
 
 // when a test crashes unexpectedly
 ReportHook(TEST_CRASH)(struct criterion_test_stats *stats) {
-  puts("<FAILED::>Test crashes");
+  puts("<FAILED::>Test Crashed");
 }
 
 //  after a test ends
 ReportHook(POST_TEST)(struct criterion_test_stats *stats) {
+  if (stats->timed_out) {
+    puts("<FAILED::>Test Timedout");
+  }
   printf("<COMPLETEDIN::>%f\n",stats->elapsed_time*1000);
 }
 

test/runners/c_spec.js

@@ -164,6 +164,56 @@ describe('.run', function() {
                 done();
             });
         });
+        it('should handle a crash', function(done) {
+            runner.run({
+                language: 'c',
+                code: ' ',
+                fixture: `
+                    #include <criterion/criterion.h>
+
+                    Test(misc, crash_dereference) {
+                        int *i = NULL;
+                        *i = 42; // CRASH: dereference i on writing
+                    }
+                    `
+            }, function(buffer) {
+                expect(buffer.stdout).to.contain('<FAILED::>Test Crashed');
+                done();
+            });
+        });
+        it('should handle a timeout', function(done) {
+            runner.run({
+                language: 'c',
+                code: ' ',
+                fixture: `
+                    # include <unistd.h>
+                    #include <criterion/criterion.h>
+
+                    Test(timeout, should_fail_with_a_timeout, .timeout = 1.) {
+                        sleep(10);
+                    }
+                    `
+            }, function(buffer) {
+                expect(buffer.stdout).to.contain('<FAILED::>Test Timedout');
+                done();
+            });
+        });
+        it('should replace message line feeds', function(done) {
+            runner.run({
+                language: 'c',
+                code: ' ',
+                fixture: `
+                    #include <criterion/criterion.h>
+
+                    Test(timeout, simple) {
+                        cr_assert(0, "This message contains \\n 2 \\n line feeds");
+                    }
+                    `
+            }, function(buffer) {
+                expect(buffer.stdout.match(/<:LF:>/g).length).to.equal(2);
+                done();
+            });
+        });
     });
 });
 });