Background Jobs

Heliotrope uses Resque to process background jobs, such as characterizing files with FITS and creating derivatives (thumbnails, web-friendly video, etc.). In production we recommend resque-pool for managing resque workers. In development, you can use the relevant rake tasks. The job queue is stored in Redis.

How to install Redis

If you are using a mac, you can use homebrew to install redis:

brew install redis

If you're using a Vagrant box, redis server is probably already installed and running.
You can check with the following command: redis-cli ping, the server should respond with "pong".

Running background jobs

To start the redis server:

redis-server /usr/local/etc/redis.conf
# (or on ubuntu)
sudo redis-server /etc/redis/redis.conf

To see the status of recent jobs in the browser console:

resque-web -F -N curation_concerns:development

If you want to run resque-web as a daemon, remove the -F flag.

To start worker(s) to run the jobs:

QUEUE=* VERBOSE=1 rake resque:work

Installing FITS

If you haven't yet, you'll need to install FITS or some resque jobs like Characterize will fail. And if one job fails, the whole thing fails.

Download FITS and put it where Heliotrope expects it, /usr/local/bin/fits.sh.

1. Download fits and unzip it somewhere (I did ~/bin/fits-0.10.1/)

2. sudo ln -s /path/to/your/fits-0.10.1/fits.sh /usr/local/bin/fits.sh

3. Change the line in fits.sh from this:

    . "$(dirname $BASH_SOURCE)/fits-env.sh"
   

To this:

    . "/path/to/your/fits-env.sh"