Issue/keras label whitelist (#95)

* keras-pretrained analyser only provides results for labels provided in whitelist * rename examples * allow multiple mtriage processes to run at once * correct examples * update keras_pretrained with thresh * fmt * correct labels, name mtriage containers correct * update readme * update * 💄
forensic-architecture · Jul 22, 2019 · fdd51bf · fdd51bf
1 parent b99b697
commit fdd51bf
Show file tree

Hide file tree

Showing 53 changed files with 307 additions and 226 deletions.
diff --git a/README.md b/README.md
@@ -2,79 +2,54 @@
 
 [![Build Status](https://travis-ci.com/forensic-architecture/mtriage.svg?branch=master)](https://travis-ci.com/forensic-architecture/mtriage)
 
-### NB: currently unstable, in active development, and should not be used in production
+##### select, download, and analyse media 
 
-##### scrape and analyse media on the web
+mtriage is a command-line application that can be used to scrape and analyse 
+media. mtriage is developed by [Forensic Architecture](https://forensic-architecture.org), and is intended for use 
+by open source research agencies, journalists, and activists.
 
-mtriage is a command-line application that scrapes and analyses public domain media. mtriage is developed by [Forensic Architecture](https://forensic-architecture.org), and is intended for use by open source research agencies, journalists, and activists.
-
-mtriage is a framework that orchestrates two different kinds of components:
+mtriage consists of two types of components:
 
 * **selectors**: to search for and download media from various platforms.
-* **analysers**: to derive data from media that has been retrieved by
-    a selector.
+* **analysers**: to derive data from media that has been retrieved by a selector.
+
+Below are the following components that are supported. If you are interested in
+helping us to develop additional selectors and analysers, please consider
+joining [the conversaton on Discord](https://discord.gg/FJ4XsCg).
 
 ### selectors
 * youtube - search and download via the [v3 API](https://developers.google.com/youtube/v3/).
+* local - use media that already exists on your filesystem. 
 
 ### analysers
-* frames - extract one frame for each second from a video.
-* ocr - analyse an image using [Google Cloud Platform](https://cloud.google.com/vision/docs/ocr).
-<!-- * pytorch - run inference with a [PyTorch](https://pytorch.org/) model on an image. -->
+* frames - extract frames from videos as images.
+* yolov3 - detect and classify objects in images using [YoloV3](https://pjreddie.com/darknet/yolo/) trained on [ImageNet](http://www.image-net.org/) classes.
+* keras_pretrained - classify objects in images using [Resnet50 trained on
+    ImageNet](https://resources.wolframcloud.com/NeuralNetRepository/resources/ResNet-50-Trained-on-ImageNet-Competition-Data).
+* ocr - analyse an image using optical character recognition from [Google Cloud Platform](https://cloud.google.com/vision/docs/ocr).
 
-
-## development
+## setup 
 mtriage is currently in active development, and is not yet packaged in any way.
-It uses [Docker](https://www.docker.com/products/docker-desktop) to manage
-dependencies, and is written in Python.
+It uses [Docker](https://www.docker.com/products/docker-desktop) to manage dependencies, which you will need to download to ensure mtriage works as expected. 
 
-### dependencies
 - Docker Desktop (Mac installation [here](https://docs.docker.com/v17.12/docker-for-mac/install/), Ubuntu installation [here](https://docs.docker.com/v17.12/install/linux/docker-ce/ubuntu/)).
-- [docker](https://docs.docker.com/install/) (python library, v3.5.0)
-
-Follow the instructions relevant to your operating system to install Docker CE,
-and then install the python dependency with:
-
-```bash
-python -m pip install -r requirements.txt
-```
-
-(Note that mtriage was developed using Python 3, but you should be able to run it with 2.x as well.)
 
-### configuration setup
-Selectors and analysers often rely on private credentials such as API keys. mtriage deals with these in two ways:
+Follow the instructions relevant to your operating system to install Docker CE.
 
-* **`.env` file at the top level**: contains API keys and other environment variables, which are made available when
-    mtriage is running.
-* **`credentials` folder**: in some cases, components require JSON configs, such as for GCP service accounts. mtriage
-    currently deals with this by adding a path to the credentials file in `.env`, and adding the credential file itself
-    in the `credentials` folder.
+You also need to ensure that a version of [Python](https://www.python.org/downloads/) is installed on your computer.
+Most modern operating systems have a version installed by default. 
 
-The specific configuration steps depend on which components you intend to use. For every component you wish to use, run
-through its setup:
+### additional setup
+Depending on what components you intend to use, there may be additional setup
+required. Ensure to read the documentation for each component you wish to use. 
 
-##### selectors
-* [youtube](docs/config/youtube.md)
-##### analysers
-* frames
-* ocr
-
-### running
-You can run mtriage in a Docker container with:
+## Run 
+Once you have Docker and Python installed, you can run mtriage using one of the
+examples provided. From this folder:
 ```bash
-python run.py develop
+./mtriage run examples/_demo/youtube.yaml 
 ```
 
-Selectors and analysers are currently specified as runtime arguments to the
-entrypoint script, "src/run.py". In "scripts" you can find a series of example
-bash scripts that construct appropriate arguments and execute them.
-
-A more robust interface for passing options is a work in progress.
-
-
-### building locally
-You can build the mtriage image locally via run.py as well:
-```bash
-python run.py build
-
-
+When you first run mtriage, it will download the necessary Docker images to
+your system. The first time you run it, it may take several minutes to get up
+and running. Subsequent uses will be much faster.
diff --git a/examples/chase_10e.yaml → examples/_demo/chase_10e.yaml b/examples/chase_10e.yaml → examples/_demo/chase_10e.yaml
diff --git a/examples/frames_tg_turkey.yaml → examples/_demo/frames.yaml b/examples/frames_tg_turkey.yaml → examples/_demo/frames.yaml
@@ -1,4 +1,4 @@
-folder: "temp/tear_gas_turkey"
+folder: "temp/demo"
 phase: "analyse"
 module: "frames"
 config:

diff --git a/examples/imgnet-ambaz.yaml → examples/_demo/keras_pretrained.yaml b/examples/imgnet-ambaz.yaml → examples/_demo/keras_pretrained.yaml
@@ -1,7 +1,11 @@
-folder: "temp/military_vehicles_ambaz"
+folder: "temp/demo"
 phase: "analyse"
 module: "keras_pretrained"
 config:
   elements_in:
     - "youtube/frames"
   model: "ResNet50"
+  labels:
+    - "tank"
+    - "rifle"
+    - "military uniform"
diff --git a/examples/sel-local.yaml → examples/_demo/local.yaml b/examples/sel-local.yaml → examples/_demo/local.yaml
diff --git a/examples/an-meta.yaml → examples/_demo/meta.yaml b/examples/an-meta.yaml → examples/_demo/meta.yaml
@@ -1,4 +1,4 @@
-folder: "temp/demo_output"
+folder: "temp/demo"
 phase: "analyse"
 module: "meta"
 config:

diff --git a/examples/an-ocr.yaml → examples/_demo/ocr.yaml b/examples/an-ocr.yaml → examples/_demo/ocr.yaml
@@ -1,4 +1,4 @@
-folder: "temp/demo_output"
+folder: "temp/demo"
 phase: "analyse"
 module: "ocr"
 config:

diff --git a/examples/resnet_demo.yaml → examples/_demo/resnet.yaml b/examples/resnet_demo.yaml → examples/_demo/resnet.yaml
@@ -1,4 +1,4 @@
-folder: "temp/demo_output"
+folder: "temp/demo"
 phase: "analyse"
 module: "keras_pretrained"
 config:

diff --git a/examples/an-yolo3.yaml → examples/_demo/yolov3.yaml b/examples/an-yolo3.yaml → examples/_demo/yolov3.yaml
diff --git a/examples/sel-youtube.yaml → examples/_demo/youtube.yaml b/examples/sel-youtube.yaml → examples/_demo/youtube.yaml
diff --git a/examples/ambaz/1.select.yaml b/examples/ambaz/1.select.yaml
@@ -0,0 +1,8 @@
+folder: temp/ambaz
+phase: select
+module: youtube
+config:
+  search_term: ambazonia
+  uploaded_before: 2019-06-01T00:00:00Z
+  uploaded_after: 2018-06-01T00:00:00Z
+  daily: false
diff --git a/examples/ambaz/2.frames.yaml b/examples/ambaz/2.frames.yaml
@@ -0,0 +1,10 @@
+folder: temp/ambaz
+phase: analyse
+module: frames
+config:
+  elements_in:
+    - youtube
+  method: opencv
+  change_threshold: 1e-5
+  sequential: false
+  fps: 1
diff --git a/examples/ambaz/3.resnet.yaml b/examples/ambaz/3.resnet.yaml
@@ -0,0 +1,28 @@
+folder: temp/ambaz
+phase: analyse
+module: keras_pretrained
+config:
+  elements_in:
+    - youtube/frames
+  model: ResNet50
+  labels:
+    - tank
+    - rifle
+    - military_uniform
+    - assault_rifle
+    - assault rifle
+    - assault_gun
+    - revolver
+    - bulletproof_vest
+    - bomb
+    - bomblet
+    - chemical_bomb
+    - pipe_bomb
+    - grenade
+    - munition
+    - gunnery
+    - case_shot
+    - canister
+    - tank_shell
+    - artillery_shell
+    - shrapnel
diff --git a/examples/ambaz/4.rank.yaml b/examples/ambaz/4.rank.yaml
@@ -0,0 +1,6 @@
+folder: temp/ambaz
+phase: analyse
+module: ranking
+config:
+  elements_in:
+    - youtube/keras_pretrained
diff --git a/examples/an-frames.yaml b/examples/an-frames.yaml
diff --git a/examples/an-keras-pretrained.yaml b/examples/an-keras-pretrained.yaml
diff --git a/examples/chase_10e_mexico.yaml b/examples/chase_10e_mexico.yaml
diff --git a/examples/chase_10e_turkey.yaml b/examples/chase_10e_turkey.yaml
diff --git a/examples/chase_10e_venezuela.yaml b/examples/chase_10e_venezuela.yaml
diff --git a/examples/chase_generic.yaml b/examples/chase_generic.yaml
diff --git a/examples/frames_mv_ambaz.yaml b/examples/frames_mv_ambaz.yaml
diff --git a/examples/frames_mv_ukraine.yaml b/examples/frames_mv_ukraine.yaml
diff --git a/examples/frames_tg_mexico.yaml b/examples/frames_tg_mexico.yaml
diff --git a/examples/frames_tg_venezuela.yaml b/examples/frames_tg_venezuela.yaml
diff --git a/examples/imgnet-ukraine.yaml b/examples/imgnet-ukraine.yaml
diff --git a/examples/mexico/1.select.yaml b/examples/mexico/1.select.yaml
@@ -0,0 +1,8 @@
+folder: temp/mexico
+phase: select
+module: youtube
+config:
+  search_term: caravanamigrante
+  uploaded_before: 2019-12-05T00:00:00Z
+  uploaded_after: 2018-11-20T00:00:00Z
+  daily: false
diff --git a/examples/mexico/2.frames.yaml b/examples/mexico/2.frames.yaml
@@ -0,0 +1,10 @@
+folder: temp/mexico
+phase: analyse
+module: frames
+config:
+  elements_in:
+    - youtube
+  method: opencv
+  change_threshold: 1e-5
+  sequential: false
+  fps: 1
diff --git a/examples/mexico/3.fastai1.yaml b/examples/mexico/3.fastai1.yaml
@@ -0,0 +1,11 @@
+folder: temp/mexico
+phase: analyse
+module: fastai_model
+config:
+  elements_in:
+    - youtube/frames
+  path: data/generic-chaser.pkl
+  labels:
+    - other
+    - teargas
+    - chaser_validation_set
diff --git a/examples/mexico/3.fastai2.yaml b/examples/mexico/3.fastai2.yaml
@@ -0,0 +1,11 @@
+folder: temp/mexico
+phase: analyse
+module: fastai_model
+config:
+  elements_in:
+    - youtube/frames
+  path: data/canister_10e.pkl
+  labels:
+    - other
+    - teargas
+    - triplechaser_synth
diff --git a/examples/mexico/4.rank.yaml b/examples/mexico/4.rank.yaml
@@ -0,0 +1,6 @@
+folder: temp/tear_gas_mexico
+phase: analyse
+module: ranking
+config:
+  elements_in:
+    - youtube/fastai_model
diff --git a/examples/military_vehicles_ambaz.yaml b/examples/military_vehicles_ambaz.yaml
diff --git a/examples/military_vehicles_ukraine.yaml b/examples/military_vehicles_ukraine.yaml
diff --git a/examples/ranking.yaml b/examples/ranking.yaml
diff --git a/examples/tear_gas_mexico.yaml b/examples/tear_gas_mexico.yaml