Exception thrown when site is unavailable

[bradseefeld]

When I try to crawl a site that is down, I Spider throws an exception:

TypeError: Cannot read property 'statusCode' of undefined
    at Request._callback (spider/main.js:131:15)
    at spider/node_modules/request/main.js:119:22
    at Request.emit (events.js:87:17)
    at ClientRequest.<anonymous> (spider/node_modules/request/main.js:207:10)
    at ClientRequest.emit (events.js:87:17)
    at Socket.<anonymous> (http.js:1265:11)
    at Socket.emit (events.js:87:17)
    at net.js:619:16
    at EventEmitter._tickCallback (node.js:245:11)

[seanowenhayes]

I have put in a pull request that fixes this issue, feel free to try out the updated code from my repository.