ExtractingDataFromNamUs.ipynb

{
 "metadata": {
  "name": ""
 },
 "nbformat": 3,
 "nbformat_minor": 0,
 "worksheets": [
  {
   "cells": [
    {
     "cell_type": "heading",
     "level": 1,
     "metadata": {},
     "source": [
      "Extracting Data from NamUs"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "<a href=\"http://www.findthemissing.org\">The National Missing and Unidentified Persons System (NamUs)</a> is a missing person database funded by the U.S. Department of Justice. NamUs does not provide an API, but does provide a searchable web interface. \n",
      "\n",
      "In this notebook, I want to show how to retrieve missing persons data from NamUs using Selenium. "
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "import time\n",
      "import json\n",
      "from selenium.webdriver.common.by import By\n",
      "from IPython.display import HTML\n",
      "import browsers #from my project"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 5
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Open a new Firefox browser to the NamUs Missing Person database."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "browser = browsers.Firefox()\n",
      "browser.get(\"https://www.findthemissing.org/en\")"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 6
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Try to search for all missing persons in California by selecting California from the drop down menu and clicking submit."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "elem = browser.wait_until_clickable(\"search_Circumstances.StateLKA\", timeout=60)\n",
      "browser.select_option(elem, \"california\")\n",
      "submit_btn = browser.find_element_by_name(\"commit\")\n",
      "submit_btn.click()\n",
      "#In Ubuntu, I needed an extra click, but Windows I didn't.\n",
      "submit_btn.click()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 7
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "table = browser.wait_until_visible(\"list\", timeout=30)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 8
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Now the Firefox browser should show a table of search results, but instead of parsing the table, we can parse the Ajax request the page uses to populate the table directly."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "browser.get(\"https://www.findthemissing.org/en/ajax/search_results?page=1&rows=100&sidx=DateLKA&sord=desc&_search=false\")\n",
      "dct = json.loads(browser.find_element_by_css_selector(\"body\").text)\n",
      "print json.dumps(dct, sort_keys=True, indent=4, separators=(',', ': '))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "{\n",
        "    \"page\": \"1\",\n",
        "    \"records\": 1230,\n",
        "    \"rows\": [\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"24369\",\n",
        "                \"McCauslin, Victoria A.\",\n",
        "                \"04/15/2014\",\n",
        "                \"San Francisco , CA\",\n",
        "                \"Female\",\n",
        "                \"White\",\n",
        "                19\n",
        "            ],\n",
        "            \"id\": \"24369_0\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"24438\",\n",
        "                \"Cole, Angel\",\n",
        "                \"04/01/2014\",\n",
        "                \"Los Angeles, CA\",\n",
        "                \"Female\",\n",
        "                \"White\",\n",
        "                16\n",
        "            ],\n",
        "            \"id\": \"24438_1\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"24225\",\n",
        "                \"Jackson, Hailey M.\",\n",
        "                \"03/31/2014\",\n",
        "                \"Sacramento, CA\",\n",
        "                \"Female\",\n",
        "                \"White\",\n",
        "                16\n",
        "            ],\n",
        "            \"id\": \"24225_2\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"23789\",\n",
        "                \"Davis, Robert L.\",\n",
        "                \"02/26/2014\",\n",
        "                \"Corte Madera, CA\",\n",
        "                \"Male\",\n",
        "                \"White\",\n",
        "                93\n",
        "            ],\n",
        "            \"id\": \"23789_3\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"23681\",\n",
        "                \"Chesley, John D.\",\n",
        "                \"02/22/2014\",\n",
        "                \"Oroville, CA\",\n",
        "                \"Male\",\n",
        "                \"White\",\n",
        "                81\n",
        "            ],\n",
        "            \"id\": \"23681_4\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"23907\",\n",
        "                \"Tucker, Imani\",\n",
        "                \"02/18/2014\",\n",
        "                \"Los Angeles, CA\",\n",
        "                \"Female\",\n",
        "                \"Black/African American\",\n",
        "                16\n",
        "            ],\n",
        "            \"id\": \"23907_5\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"23619\",\n",
        "                \"Butler, Brian\",\n",
        "                \"02/16/2014\",\n",
        "                \"Pacifica, CA\",\n",
        "                \"Male\",\n",
        "                \"White\",\n",
        "                36\n",
        "            ],\n",
        "            \"id\": \"23619_6\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"24036\",\n",
        "                \"Bertolini , Danielle N.\",\n",
        "                \"02/10/2014\",\n",
        "                \"Fortuna, CA\",\n",
        "                \"Female\",\n",
        "                \"White\",\n",
        "                23\n",
        "            ],\n",
        "            \"id\": \"24036_7\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"24382\",\n",
        "                \"GARCIA JR., ALFRED\",\n",
        "                \"01/20/2014\",\n",
        "                \"LA PUENTE, CA\",\n",
        "                \"Male\",\n",
        "                \"White\",\n",
        "                45\n",
        "            ],\n",
        "            \"id\": \"24382_8\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"23413\",\n",
        "                \"Lopez, Ashley B.\",\n",
        "                \"12/20/2013\",\n",
        "                \"Temecula, CA\",\n",
        "                \"Female\",\n",
        "                \"Other\",\n",
        "                16\n",
        "            ],\n",
        "            \"id\": \"23413_9\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"23153\",\n",
        "                \"Mendoza-Amaya, Luis M.\",\n",
        "                \"12/06/2013\",\n",
        "                \"Montclair, CA\",\n",
        "                \"Male\",\n",
        "                \"Other\",\n",
        "                14\n",
        "            ],\n",
        "            \"id\": \"23153_10\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"23902\",\n",
        "                \"Grethel, Linda\",\n",
        "                \"11/30/2013\",\n",
        "                \"Montebello, CA\",\n",
        "                \"Female\",\n",
        "                \"White\",\n",
        "                61\n",
        "            ],\n",
        "            \"id\": \"23902_11\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"23069\",\n",
        "                \"Blakley, Jackson P.\",\n",
        "                \"11/30/2013\",\n",
        "                \"Willits, CA\",\n",
        "                \"Male\",\n",
        "                \"White\",\n",
        "                49\n",
        "            ],\n",
        "            \"id\": \"23069_12\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"22646\",\n",
        "                \"Anaya, Martha E.\",\n",
        "                \"11/12/2013\",\n",
        "                \"Santa Ana, CA\",\n",
        "                \"Female\",\n",
        "                \"Other\",\n",
        "                27\n",
        "            ],\n",
        "            \"id\": \"22646_13\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"23203\",\n",
        "                \"Vargas, Josephine M.\",\n",
        "                \"10/27/2013\",\n",
        "                \"Santa Ana, CA\",\n",
        "                \"Female\",\n",
        "                \"Other\",\n",
        "                34\n",
        "            ],\n",
        "            \"id\": \"23203_14\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"22669\",\n",
        "                \"Doyle, Donavan W.\",\n",
        "                \"10/09/2013\",\n",
        "                \"Sky Forest, CA\",\n",
        "                \"Male\",\n",
        "                \"White\",\n",
        "                21\n",
        "            ],\n",
        "            \"id\": \"22669_15\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"24215\",\n",
        "                \"Jackson, Kianna R.\",\n",
        "                \"10/03/2013\",\n",
        "                \"Santa Ana, CA\",\n",
        "                \"Female\",\n",
        "                \"White\",\n",
        "                20\n",
        "            ],\n",
        "            \"id\": \"24215_16\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"22343\",\n",
        "                \"Krost, Aloys J.\",\n",
        "                \"10/01/2013\",\n",
        "                \"Lake Arrowhead, CA\",\n",
        "                \"Male\",\n",
        "                \"White\",\n",
        "                62\n",
        "            ],\n",
        "            \"id\": \"22343_17\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"23713\",\n",
        "                \"Griffiths, Judith L.\",\n",
        "                \"10/01/2013\",\n",
        "                \"Escondido, CA\",\n",
        "                \"Female\",\n",
        "                \"White\",\n",
        "                73\n",
        "            ],\n",
        "            \"id\": \"23713_18\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"22480\",\n",
        "                \"PATTON, GARY D.\",\n",
        "                \"09/05/2013\",\n",
        "                \"SAN DIEGO, CA\",\n",
        "                \"Male\",\n",
        "                \"White\",\n",
        "                64\n",
        "            ],\n",
        "            \"id\": \"22480_19\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"21970\",\n",
        "                \"Laspisa, Bryce D.\",\n",
        "                \"08/29/2013\",\n",
        "                \"Castaic Lake, CA\",\n",
        "                \"Male\",\n",
        "                \"White\",\n",
        "                19\n",
        "            ],\n",
        "            \"id\": \"21970_20\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"22266\",\n",
        "                \"Frassinelli, Maria V.\",\n",
        "                \"08/11/2013\",\n",
        "                \"Chino, CA\",\n",
        "                \"Female\",\n",
        "                \"Other\",\n",
        "                28\n",
        "            ],\n",
        "            \"id\": \"22266_21\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"21735\",\n",
        "                \"Shea, Daniel A.\",\n",
        "                \"08/10/2013\",\n",
        "                \"sausalito, CA\",\n",
        "                \"Male\",\n",
        "                \"White\",\n",
        "                59\n",
        "            ],\n",
        "            \"id\": \"21735_22\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"21339\",\n",
        "                \"Everling, Roy A.\",\n",
        "                \"08/02/2013\",\n",
        "                \"Hacienda Heights, CA\",\n",
        "                \"Male\",\n",
        "                \"White\",\n",
        "                19\n",
        "            ],\n",
        "            \"id\": \"21339_23\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"21261\",\n",
        "                \"Greene, Matthew B.\",\n",
        "                \"07/16/2013\",\n",
        "                \"Mammoth Lakes, CA\",\n",
        "                \"Male\",\n",
        "                \"White\",\n",
        "                39\n",
        "            ],\n",
        "            \"id\": \"21261_24\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"23738\",\n",
        "                \"Lenoir, Justin E.\",\n",
        "                \"07/02/2013\",\n",
        "                \"San Diego, CA\",\n",
        "                \"Male\",\n",
        "                \"Black/African American\",\n",
        "                24\n",
        "            ],\n",
        "            \"id\": \"23738_25\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"20966\",\n",
        "                \"Aguilar, Juan J.\",\n",
        "                \"06/27/2013\",\n",
        "                \"Desert Hot Springs, CA\",\n",
        "                \"Male\",\n",
        "                \"Other\",\n",
        "                51\n",
        "            ],\n",
        "            \"id\": \"20966_26\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"21130\",\n",
        "                \"Clark, William M.\",\n",
        "                \"06/25/2013\",\n",
        "                \"Novato, CA\",\n",
        "                \"Male\",\n",
        "                \"White\",\n",
        "                67\n",
        "            ],\n",
        "            \"id\": \"21130_27\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"21431\",\n",
        "                \"Lamberg, Erik S.\",\n",
        "                \"05/26/2013\",\n",
        "                \"Laytonville, CA\",\n",
        "                \"Male\",\n",
        "                \"White\",\n",
        "                51\n",
        "            ],\n",
        "            \"id\": \"21431_28\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"23663\",\n",
        "                \"Lewin, Ashley M.\",\n",
        "                \"05/26/2013\",\n",
        "                \"Littlerock, CA\",\n",
        "                \"Female\",\n",
        "                \"White\",\n",
        "                15\n",
        "            ],\n",
        "            \"id\": \"23663_29\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"20560\",\n",
        "                \"Sidi, Sean I.\",\n",
        "                \"05/21/2013\",\n",
        "                \"San Francisco, CA\",\n",
        "                \"Male\",\n",
        "                \"Other\",\n",
        "                19\n",
        "            ],\n",
        "            \"id\": \"20560_30\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"21346\",\n",
        "                \"Veldhaus, Chad L.\",\n",
        "                \"05/15/2013\",\n",
        "                \"Fairfield, CA\",\n",
        "                \"Male\",\n",
        "                \"White\",\n",
        "                42\n",
        "            ],\n",
        "            \"id\": \"21346_31\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"20573\",\n",
        "                \"Geisler, Steven D.\",\n",
        "                \"05/10/2013\",\n",
        "                \"Phelan, CA\",\n",
        "                \"Male\",\n",
        "                \"White\",\n",
        "                51\n",
        "            ],\n",
        "            \"id\": \"20573_32\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"20902\",\n",
        "                \"Russ, Michelle J.\",\n",
        "                \"05/09/2013\",\n",
        "                \"Lancaster, CA\",\n",
        "                \"Female\",\n",
        "                \"White\",\n",
        "                26\n",
        "            ],\n",
        "            \"id\": \"20902_33\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"20409\",\n",
        "                \"Olson, Cheyenne S.\",\n",
        "                \"05/06/2013\",\n",
        "                \"Eureka, CA\",\n",
        "                \"Female\",\n",
        "                \"White\",\n",
        "                17\n",
        "            ],\n",
        "            \"id\": \"20409_34\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"21150\",\n",
        "                \"Jacobson, Susan \",\n",
        "                \"05/01/2013\",\n",
        "                \"Roseville, CA\",\n",
        "                \"Female\",\n",
        "                \"White\",\n",
        "                59\n",
        "            ],\n",
        "            \"id\": \"21150_35\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"20471\",\n",
        "                \"Evans, Kimberly J.\",\n",
        "                \"04/30/2013\",\n",
        "                \"Encinitas, CA\",\n",
        "                \"Female\",\n",
        "                \"White\",\n",
        "                41\n",
        "            ],\n",
        "            \"id\": \"20471_36\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"22737\",\n",
        "                \"Escobedo, Alonso G.\",\n",
        "                \"04/21/2013\",\n",
        "                \"Simi Valley, CA\",\n",
        "                \"Male\",\n",
        "                \"White\",\n",
        "                23\n",
        "            ],\n",
        "            \"id\": \"22737_37\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"21578\",\n",
        "                \"King, Neal F.\",\n",
        "                \"03/26/2013\",\n",
        "                \"Oroville, CA\",\n",
        "                \"Male\",\n",
        "                \"White\",\n",
        "                24\n",
        "            ],\n",
        "            \"id\": \"21578_38\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"22016\",\n",
        "                \"Martin, Natasha\",\n",
        "                \"03/21/2013\",\n",
        "                \"San Diego, CA\",\n",
        "                \"Female\",\n",
        "                \"Other\",\n",
        "                15\n",
        "            ],\n",
        "            \"id\": \"22016_39\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"22014\",\n",
        "                \"Mendoza, Desteni\",\n",
        "                \"03/17/2013\",\n",
        "                \"Dinuba, CA\",\n",
        "                \"Female\",\n",
        "                \"Other\",\n",
        "                16\n",
        "            ],\n",
        "            \"id\": \"22014_40\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"20311\",\n",
        "                \"Saiz, Moses F.\",\n",
        "                \"03/15/2013\",\n",
        "                \"Yermo, CA\",\n",
        "                \"Male\",\n",
        "                \"Other\",\n",
        "                30\n",
        "            ],\n",
        "            \"id\": \"20311_41\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"23905\",\n",
        "                \"Cabrera, Santos\",\n",
        "                \"03/13/2013\",\n",
        "                \"Los Angeles, CA\",\n",
        "                \"Male\",\n",
        "                \"Black/African American\",\n",
        "                17\n",
        "            ],\n",
        "            \"id\": \"23905_42\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"21928\",\n",
        "                \"Giles, Jasmine\",\n",
        "                \"02/17/2013\",\n",
        "                \"Lincoln, CA\",\n",
        "                \"Female\",\n",
        "                \"Other\",\n",
        "                16\n",
        "            ],\n",
        "            \"id\": \"21928_43\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"24102\",\n",
        "                \"Coggin, Christopher L.\",\n",
        "                \"02/16/2013\",\n",
        "                \"Washington, CA\",\n",
        "                \"Male\",\n",
        "                \"White\",\n",
        "                52\n",
        "            ],\n",
        "            \"id\": \"24102_44\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"21919\",\n",
        "                \"Castillo, Alyssa\",\n",
        "                \"02/15/2013\",\n",
        "                \"Visalia, CA\",\n",
        "                \"Female\",\n",
        "                \"Other\",\n",
        "                14\n",
        "            ],\n",
        "            \"id\": \"21919_45\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"21933\",\n",
        "                \"Caylor, Hailey R.\",\n",
        "                \"02/14/2013\",\n",
        "                \"Hacienda Heights, CA\",\n",
        "                \"Female\",\n",
        "                \"White\",\n",
        "                15\n",
        "            ],\n",
        "            \"id\": \"21933_46\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"21961\",\n",
        "                \"Jauregui, Nathalie\",\n",
        "                \"02/13/2013\",\n",
        "                \"Covina, CA\",\n",
        "                \"Female\",\n",
        "                \"Other\",\n",
        "                16\n",
        "            ],\n",
        "            \"id\": \"21961_47\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"21934\",\n",
        "                \"Dominguez, Dominique\",\n",
        "                \"02/08/2013\",\n",
        "                \"San Dimas, CA\",\n",
        "                \"Female\",\n",
        "                \"Other\",\n",
        "                15\n",
        "            ],\n",
        "            \"id\": \"21934_48\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"19238\",\n",
        "                \"Hughes, Crishtian M.\",\n",
        "                \"02/07/2013\",\n",
        "                \"San Francisco, CA\",\n",
        "                \"Male\",\n",
        "                \"White\",\n",
        "                20\n",
        "            ],\n",
        "            \"id\": \"19238_49\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"19383\",\n",
        "                \"Reyes, Carla R.\",\n",
        "                \"02/03/2013\",\n",
        "                \"Yucca Valley, CA\",\n",
        "                \"Female\",\n",
        "                \"White\",\n",
        "                54\n",
        "            ],\n",
        "            \"id\": \"19383_50\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"19215\",\n",
        "                \"Kyles, Jason L.\",\n",
        "                \"02/02/2013\",\n",
        "                \"Santa Rosa, CA\",\n",
        "                \"Male\",\n",
        "                \"White\",\n",
        "                34\n",
        "            ],\n",
        "            \"id\": \"19215_51\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"21915\",\n",
        "                \"Cunningham, Kristen M.\",\n",
        "                \"01/23/2013\",\n",
        "                \"Santa Rosa, CA\",\n",
        "                \"Female\",\n",
        "                \"White\",\n",
        "                65\n",
        "            ],\n",
        "            \"id\": \"21915_52\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"20244\",\n",
        "                \"Michel, Marta A.\",\n",
        "                \"01/19/2013\",\n",
        "                \"Los Angeles, CA\",\n",
        "                \"Female\",\n",
        "                \"Other\",\n",
        "                49\n",
        "            ],\n",
        "            \"id\": \"20244_53\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"19006\",\n",
        "                \"Garcia, Andrew\",\n",
        "                \"01/18/2013\",\n",
        "                \"Chino, CA\",\n",
        "                \"Male\",\n",
        "                \"Other\",\n",
        "                15\n",
        "            ],\n",
        "            \"id\": \"19006_54\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"21929\",\n",
        "                \"Gaskin, Amanda E.\",\n",
        "                \"01/11/2013\",\n",
        "                \"Los Angeles, CA\",\n",
        "                \"Female\",\n",
        "                \"Black/African American\",\n",
        "                14\n",
        "            ],\n",
        "            \"id\": \"21929_55\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"20095\",\n",
        "                \"Vail, Nazario\",\n",
        "                \"01/05/2013\",\n",
        "                \"San Rafael , CA\",\n",
        "                \"Male\",\n",
        "                \"Other\",\n",
        "                28\n",
        "            ],\n",
        "            \"id\": \"20095_56\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"21485\",\n",
        "                \"Henry, Keyon J.\",\n",
        "                \"12/26/2012\",\n",
        "                \"Montebello, CA\",\n",
        "                \"Male\",\n",
        "                \"Black/African American\",\n",
        "                36\n",
        "            ],\n",
        "            \"id\": \"21485_57\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"20713\",\n",
        "                \"Sardella, Anthony\",\n",
        "                \"12/20/2012\",\n",
        "                \"Concord, CA\",\n",
        "                \"Male\",\n",
        "                \"White\",\n",
        "                22\n",
        "            ],\n",
        "            \"id\": \"20713_58\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"22690\",\n",
        "                \"Jones, Derrick M.\",\n",
        "                \"12/06/2012\",\n",
        "                \"San Rafael, CA\",\n",
        "                \"Male\",\n",
        "                \"Black/African American\",\n",
        "                24\n",
        "            ],\n",
        "            \"id\": \"22690_59\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"20682\",\n",
        "                \"Case, Darren S.\",\n",
        "                \"12/03/2012\",\n",
        "                \"Los Angeles , CA\",\n",
        "                \"Male\",\n",
        "                \"White\",\n",
        "                32\n",
        "            ],\n",
        "            \"id\": \"20682_60\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"21999\",\n",
        "                \"Sepulveda, Emily\",\n",
        "                \"11/26/2012\",\n",
        "                \"Los Angeles, CA\",\n",
        "                \"Female\",\n",
        "                \"Other\",\n",
        "                14\n",
        "            ],\n",
        "            \"id\": \"21999_61\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"19578\",\n",
        "                \"Press, Wendie K.\",\n",
        "                \"11/15/2012\",\n",
        "                \"Colton, CA\",\n",
        "                \"Female\",\n",
        "                \"White\",\n",
        "                60\n",
        "            ],\n",
        "            \"id\": \"19578_62\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"23015\",\n",
        "                \"DiPiazza, Megan S.\",\n",
        "                \"11/11/2012\",\n",
        "                \"Sunland , CA\",\n",
        "                \"Female\",\n",
        "                \"White\",\n",
        "                33\n",
        "            ],\n",
        "            \"id\": \"23015_63\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"19524\",\n",
        "                \"Luna, Darlena L.\",\n",
        "                \"11/02/2012\",\n",
        "                \"Los Angeles, CA\",\n",
        "                \"Female\",\n",
        "                \"Other\",\n",
        "                16\n",
        "            ],\n",
        "            \"id\": \"19524_64\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"21932\",\n",
        "                \"Catalan-Lopez, Nancy\",\n",
        "                \"10/26/2012\",\n",
        "                \"Lemon Grove, CA\",\n",
        "                \"Female\",\n",
        "                \"Other\",\n",
        "                16\n",
        "            ],\n",
        "            \"id\": \"21932_65\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"21943\",\n",
        "                \"Smith, Bijon\",\n",
        "                \"10/25/2012\",\n",
        "                \"Torrance, CA\",\n",
        "                \"Female\",\n",
        "                \"White\",\n",
        "                15\n",
        "            ],\n",
        "            \"id\": \"21943_66\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"21967\",\n",
        "                \"Larranaga, Rosey A.\",\n",
        "                \"10/19/2012\",\n",
        "                \"Los Angeles, CA\",\n",
        "                \"Female\",\n",
        "                \"Other\",\n",
        "                15\n",
        "            ],\n",
        "            \"id\": \"21967_67\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"18159\",\n",
        "                \"Koue Castaneda, Marlane L.\",\n",
        "                \"10/01/2012\",\n",
        "                \"Carlsbad, CA\",\n",
        "                \"Female\",\n",
        "                \"White\",\n",
        "                50\n",
        "            ],\n",
        "            \"id\": \"18159_68\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"21688\",\n",
        "                \"Adams, Duchess S.\",\n",
        "                \"09/30/2012\",\n",
        "                \"Lancaster, CA\",\n",
        "                \"Female\",\n",
        "                \"White\",\n",
        "                10\n",
        "            ],\n",
        "            \"id\": \"21688_69\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"21687\",\n",
        "                \"Adams, Tina V.\",\n",
        "                \"09/30/2012\",\n",
        "                \"Lancaster, CA\",\n",
        "                \"Female\",\n",
        "                \"White\",\n",
        "                11\n",
        "            ],\n",
        "            \"id\": \"21687_70\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"17820\",\n",
        "                \"Tanouye, Patrick K.\",\n",
        "                \"09/27/2012\",\n",
        "                \"San Francisco, CA\",\n",
        "                \"Male\",\n",
        "                \"Asian or Pacific Islander\",\n",
        "                30\n",
        "            ],\n",
        "            \"id\": \"17820_71\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"17554\",\n",
        "                \"Estrada, Manuel E.\",\n",
        "                \"09/18/2012\",\n",
        "                \"Chino, CA\",\n",
        "                \"Male\",\n",
        "                \"White\",\n",
        "                79\n",
        "            ],\n",
        "            \"id\": \"17554_72\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"20092\",\n",
        "                \"Gutierrez, Nayley\",\n",
        "                \"09/06/2012\",\n",
        "                \"Exeter, CA\",\n",
        "                \"Female\",\n",
        "                \"Other\",\n",
        "                13\n",
        "            ],\n",
        "            \"id\": \"20092_73\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"22021\",\n",
        "                \"Padilla, Johnson\",\n",
        "                \"09/05/2012\",\n",
        "                \"Los Angeles, CA\",\n",
        "                \"Male\",\n",
        "                \"Other\",\n",
        "                16\n",
        "            ],\n",
        "            \"id\": \"22021_74\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"18301\",\n",
        "                \"Brooks, Deanna K.\",\n",
        "                \"08/24/2012\",\n",
        "                \"Santa Cruz, CA\",\n",
        "                \"Female\",\n",
        "                \"White\",\n",
        "                70\n",
        "            ],\n",
        "            \"id\": \"18301_75\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"19129\",\n",
        "                \"Cameron, Heather L.\",\n",
        "                \"08/18/2012\",\n",
        "                \"Redding, CA\",\n",
        "                \"Female\",\n",
        "                \"Native American\",\n",
        "                28\n",
        "            ],\n",
        "            \"id\": \"19129_76\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"17399\",\n",
        "                \"Jones, Cort N.\",\n",
        "                \"08/16/2012\",\n",
        "                \"Redding, CA\",\n",
        "                \"Male\",\n",
        "                \"White\",\n",
        "                23\n",
        "            ],\n",
        "            \"id\": \"17399_77\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"22268\",\n",
        "                \"Essayah, Jamal\",\n",
        "                \"07/22/2012\",\n",
        "                \"Fairfax, CA\",\n",
        "                \"Male\",\n",
        "                \"White\",\n",
        "                23\n",
        "            ],\n",
        "            \"id\": \"22268_78\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"19347\",\n",
        "                \"Cubias, Selena\",\n",
        "                \"07/20/2012\",\n",
        "                \"Los Angeles, CA\",\n",
        "                \"Female\",\n",
        "                \"Other\",\n",
        "                17\n",
        "            ],\n",
        "            \"id\": \"19347_79\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"17154\",\n",
        "                \"MOSALLAM, HAMMAM A.\",\n",
        "                \"07/14/2012\",\n",
        "                \" Vallejo, CA\",\n",
        "                \"Male\",\n",
        "                \"White\",\n",
        "                47\n",
        "            ],\n",
        "            \"id\": \"17154_80\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"17880\",\n",
        "                \"Rodriguez Valadez, Javier \",\n",
        "                \"07/08/2012\",\n",
        "                \"Mira Loma, CA\",\n",
        "                \"Male\",\n",
        "                \"Other\",\n",
        "                42\n",
        "            ],\n",
        "            \"id\": \"17880_81\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"15618\",\n",
        "                \"Zhou, Heng Yu\",\n",
        "                \"07/05/2012\",\n",
        "                \"Diamond Bar, CA\",\n",
        "                \"Male\",\n",
        "                \"Asian or Pacific Islander\",\n",
        "                34\n",
        "            ],\n",
        "            \"id\": \"15618_82\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"17814\",\n",
        "                \"Hillburn, Donald K.\",\n",
        "                \"07/03/2012\",\n",
        "                \"Novato, CA\",\n",
        "                \"Male\",\n",
        "                \"White\",\n",
        "                64\n",
        "            ],\n",
        "            \"id\": \"17814_83\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"19346\",\n",
        "                \"Brown-Dawson, Deleaha\",\n",
        "                \"06/15/2012\",\n",
        "                \"Los Angeles, CA\",\n",
        "                \"Female\",\n",
        "                \"Black/African American\",\n",
        "                15\n",
        "            ],\n",
        "            \"id\": \"19346_84\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"17252\",\n",
        "                \"Gillham, Katherine\",\n",
        "                \"06/11/2012\",\n",
        "                \"Big Lagoon, CA\",\n",
        "                \"Female\",\n",
        "                \"Native American\",\n",
        "                40\n",
        "            ],\n",
        "            \"id\": \"17252_85\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"18713\",\n",
        "                \"Flores, Ruth\",\n",
        "                \"05/25/2012\",\n",
        "                \"San Diego, CA\",\n",
        "                \"Female\",\n",
        "                \"Other\",\n",
        "                16\n",
        "            ],\n",
        "            \"id\": \"18713_86\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"18791\",\n",
        "                \"Ramirez, Stephanie H.\",\n",
        "                \"05/20/2012\",\n",
        "                \"Montclair, CA\",\n",
        "                \"Female\",\n",
        "                \"Other\",\n",
        "                14\n",
        "            ],\n",
        "            \"id\": \"18791_87\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"14729\",\n",
        "                \"Smith, Gavin\",\n",
        "                \"05/02/2012\",\n",
        "                \"West Hills, CA\",\n",
        "                \"Male\",\n",
        "                \"White\",\n",
        "                57\n",
        "            ],\n",
        "            \"id\": \"14729_88\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"17120\",\n",
        "                \"Cordova, Joseph P.\",\n",
        "                \"05/01/2012\",\n",
        "                \"San Jose , CA\",\n",
        "                \"Male\",\n",
        "                \"Asian or Pacific Islander\",\n",
        "                33\n",
        "            ],\n",
        "            \"id\": \"17120_89\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"22860\",\n",
        "                \"Gonzales, Samantha L.\",\n",
        "                \"04/27/2012\",\n",
        "                \" Alameda,County , CA\",\n",
        "                \"Female\",\n",
        "                \"White\",\n",
        "                23\n",
        "            ],\n",
        "            \"id\": \"22860_90\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"21636\",\n",
        "                \"Blumenfeld-Jones, Rebecca G.\",\n",
        "                \"04/24/2012\",\n",
        "                \"Eureka, CA\",\n",
        "                \"Female\",\n",
        "                \"White\",\n",
        "                26\n",
        "            ],\n",
        "            \"id\": \"21636_91\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"15369\",\n",
        "                \"Limas, Emmanuel\",\n",
        "                \"04/12/2012\",\n",
        "                \"Orange, CA\",\n",
        "                \"Male\",\n",
        "                \"White\",\n",
        "                0\n",
        "            ],\n",
        "            \"id\": \"15369_92\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"15370\",\n",
        "                \"Limas, Fernando\",\n",
        "                \"04/12/2012\",\n",
        "                \"Orange, CA\",\n",
        "                \"Male\",\n",
        "                \"White\",\n",
        "                1\n",
        "            ],\n",
        "            \"id\": \"15370_93\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"22019\",\n",
        "                \"Moran, Rochell\",\n",
        "                \"03/30/2012\",\n",
        "                \"Los Angeles, CA\",\n",
        "                \"Female\",\n",
        "                \"Other\",\n",
        "                17\n",
        "            ],\n",
        "            \"id\": \"22019_94\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"15222\",\n",
        "                \"KAUFFMAN, KOREY A.\",\n",
        "                \"03/29/2012\",\n",
        "                \"Turlock , CA\",\n",
        "                \"Male\",\n",
        "                \"White\",\n",
        "                26\n",
        "            ],\n",
        "            \"id\": \"15222_95\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"18014\",\n",
        "                \"Perez-Gomez, Ismael\",\n",
        "                \"03/27/2012\",\n",
        "                \"Sinaloa, MX, CA\",\n",
        "                \"Male\",\n",
        "                \"White\",\n",
        "                44\n",
        "            ],\n",
        "            \"id\": \"18014_96\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"18853\",\n",
        "                \"Geren, Timothy S.\",\n",
        "                \"03/22/2012\",\n",
        "                \"Modesto, CA\",\n",
        "                \"Male\",\n",
        "                \"White\",\n",
        "                32\n",
        "            ],\n",
        "            \"id\": \"18853_97\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"14258\",\n",
        "                \"LaMar, Sierra M.\",\n",
        "                \"03/16/2012\",\n",
        "                \"Morgan Hill, CA\",\n",
        "                \"Female\",\n",
        "                \"White\",\n",
        "                15\n",
        "            ],\n",
        "            \"id\": \"14258_98\"\n",
        "        },\n",
        "        {\n",
        "            \"cell\": [\n",
        "                \"14965\",\n",
        "                \"Corona, Andrew F.\",\n",
        "                \"03/12/2012\",\n",
        "                \"Riverside, CA\",\n",
        "                \"Male\",\n",
        "                \"Other\",\n",
        "                42\n",
        "            ],\n",
        "            \"id\": \"14965_99\"\n",
        "        }\n",
        "    ],\n",
        "    \"total\": \"13\"\n",
        "}\n"
       ]
      }
     ],
     "prompt_number": 9
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "How many pages are in the search results?"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "int(dct[\"total\"])"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 10,
       "text": [
        "13"
       ]
      }
     ],
     "prompt_number": 10
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "How many open missing persons cases are there in total for California?"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "int(dct[\"records\"])"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 11,
       "text": [
        "1230"
       ]
      }
     ],
     "prompt_number": 11
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "There is some interesting information here, but we need to look at the detailed results of each case. "
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "person = dct[\"rows\"][0]\n",
      "arr = person[\"id\"].split(\"_\")\n",
      "browser.get(\"https://www.findthemissing.org/en/cases/\" + arr[0] + \"/\" + arr[1])\n",
      "time.sleep(10)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 12
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "print \"First name: \" + browser.find_element_by_xpath(\"//div[@id='case_information']/div/table/tbody/tr[2]/td[2]\").text\n",
      "print \"Middle name: \" + browser.find_element_by_xpath(\"//div[@id='case_information']/div/table/tbody/tr[3]/td[2]\").text\n",
      "print \"Last name: \" + browser.find_element_by_xpath(\"//div[@id='case_information']/div/table/tbody/tr[4]/td[2]\").text"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "First name: Victoria\n",
        "Middle name: Ann"
       ]
      },
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "\n",
        "Last name: McCauslin"
       ]
      },
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "\n"
       ]
      }
     ],
     "prompt_number": 13
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "What does the missing person look like?"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "photo = browser.find_element_by_css_selector(\"dt.photo > img\").get_attribute(\"src\")\n",
      "html = \"\"\"<img src=\"https://www.findthemissing.org/en/photos/thumb/44155\"/>\"\"\"\n",
      "HTML(html)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "html": [
        "<img src=\"https://www.findthemissing.org/en/photos/thumb/44155\"/>"
       ],
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 14,
       "text": [
        "<IPython.core.display.HTML at 0x320f2d0>"
       ]
      }
     ],
     "prompt_number": 14
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Next let's determine the missing person's race. It is important to note that in NamUs, race is White, Black/African American, Asian or Pacific Islander, Native American, Other or Unknown and ethnicity is Hispanic/Latino or Other. Therefore a White Hispanic/Latino has race equal to White and ethnicity equal to Hispanic/Latino. A Non-White Hispanic/Latino has race equal to Other and ethnicity equal to Hispanic/Latino.\n",
      "\n",
      "Look at the person's race. If it is White or Other, then consider if the Ethnicity is relevant."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "race = browser.find_element_by_xpath(\"//div[@id='case_information']/div[2]/table/tbody/tr[3]/td[2]\").text\n",
      "\n",
      "if race == \"White\" or race == \"Other\":\n",
      "    ethnicity = browser.find_element_by_xpath(\"//div[@id='case_information']/div[2]/table/tbody/tr[4]/td[2]\").text\n",
      "    if ethnicity == \"Hispanic/Latino\" and race == \"White\":\n",
      "        race = \"White Hispanic/Latino\"\n",
      "    if ethnicity == \"Hispanic/Latino\" and race == \"Other\":\n",
      "        race = \"Non-White Hispanic/Latino\"\n",
      "print \"Race: \" + race"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "Race: White\n"
       ]
      }
     ],
     "prompt_number": 18
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "What are the circumstances surrounding this person's disappearance? This information is on the Circumstances page. First click on the Circumstances link from the Case Navigation."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "browser.find_element_by_link_text(\"Circumstances\").click()\n",
      "time.sleep(3)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 19
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Then print out the city, state and county information."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "print \"City: \" + browser.find_element_by_css_selector(\"div.column1-unit > table > tbody > tr > td.view_field\").text\n",
      "print \"State: \" + browser.find_element_by_xpath(\"//div[@id='circumstances']/div/table/tbody/tr[2]/td[2]\").text\n",
      "print \"County: \" + browser.find_element_by_xpath(\"//div[@id='circumstances']/div/table/tbody/tr[4]/td[2]\").text"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "City: San Francisco\n",
        "State: California"
       ]
      },
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "\n",
        "County: San Francisco"
       ]
      },
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "\n"
       ]
      }
     ],
     "prompt_number": 20
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The circumstance field may not exist so put it within a try/except block."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "try:\n",
      "    circumstance = browser.find_element_by_id(\"case_Circumstances\").text\n",
      "except NoSuchElementException:\n",
      "    circumstance = \"\"\n",
      "print \"Circumstance: \" + circumstance"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "Circumstance: Tori went on a trip with a male, Brandon Mayberry. She was in regular contact with family/friends. She has not be heard from since April 16, 2014.\n"
       ]
      }
     ],
     "prompt_number": 21
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Gather more physical characteristics on the Physical/Medical page."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "browser.find_element_by_link_text(\"Physical / Medical\").click()\n",
      "time.sleep(3)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 23
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Hair color is relatively straightforward."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "hair_color = browser.find_element_by_xpath(\"//div[@id='physical_characteristics']/div/table/tbody/tr/td[3]\").text\n",
      "print \"Hair color: \" + hair_color"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "Hair color: Sandy\n"
       ]
      }
     ],
     "prompt_number": 24
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Eye color is more tricky because the person may have two different eye colors. "
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "left_eye_color = browser.find_element_by_xpath(\"//div[@id='physical_characteristics']/div/table/tbody/tr[5]/td[3]\").text\t\t\n",
      "right_eye_color = browser.find_element_by_xpath(\"//div[@id='physical_characteristics']/div/table/tbody/tr[6]/td[3]\").text\n",
      "if left_eye_color == right_eye_color:\n",
      "    eye_color = left_eye_color\n",
      "else:\n",
      "    eye_color = \"Multicolor\"\n",
      "print \"Eye color: \" + eye_color"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "Eye color: Hazel\n"
       ]
      }
     ],
     "prompt_number": 25
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Now we understand how to gather information from each missing person's page on NamUs."
     ]
    }
   ],
   "metadata": {}
  }
 ]
}