Add paper clasification ipython nb

.gitignore

@@ -13,6 +13,7 @@
 env
 sftp-config.json
 *.pkl
+!classifier.pkl
 *.sql
 *.bak
 data/phenotypes/snorkel/dicts*

.gitmodules

@@ -3,4 +3,4 @@
 	url = https://github.com/HazyResearch/snorkel.git
 [submodule "snorkel-tables"]
 	path = snorkel-tables
-	url = git@github.com:HazyResearch/snorkel.git
+	url = git@github.com:kuleshov/snorkel.git

README.md

@@ -3,17 +3,29 @@ GWASdb
 
 GWASdb is a machine reading system for discovering associations between genetic mutations and disease from academic papers.
 
+## Results
+
+The machine-curated relations are found in `notebooks/results/associations.tsv`. 
+The five columns are: `pmid`, `rsid`, high-level phenotype, low-level phenotype, p-value. If the latter is `-1`, it means that we were not able to extract the p-value.
+
+In addition, the following files are important:
+
+* `notebooks/results/nb-output`: folder containing the output of each system module
+* `notebooks/util/phenotype.mapping.annotated.tsv`: manually annotated mapping between GWAS Central and GWASdb phenotypes
+* `notebooks/util/phenotype.mapping.gwascat.annotated.tsv`: manually annotated mapping between GWAS Catalog and GWASdb phenotypes
+* `notebooks/util/rels.discovered.annotated.txt`: random subset of 100 previously unreported relations with explanations for why they are correct or not.
+
 ## Requirements
 
 GWASdb is implemented in Python and requires:
 
-* `lxml`, ElementTree
+* `lxml`, `ElementTree`
 * `numpy`
 * `sklearn`
 * `sqlite`
 * `snorkel`
 
-Check out the Snorkel web page for a list of its requirements.
+Check out the [Snorkel repo](https://github.com/kuleshov/snorkel) for a list of its requirements.
 
 ## Installation
 
@@ -46,7 +58,7 @@ In addition, we use hand-curated databases such as GWAS Catalog and GWAS Central
 The first step is to download this data onto your machine. The `data` subfolder contains code for doing this.
 
 ```
-cd data
+cd data/db
 
 # we will store part of the dataset in a sqlite databset
 make init # this will initialize an empty database
@@ -57,7 +69,7 @@ make init # this will initialize an empty database
 make phenotypes
 
 # next, we download the contents of the hand-curated GWAS catalog database 
-make gwas-catalog # the results will go in the sqlite db (in /tmp/gwas.sql by default)
+make gwas-catalog # loads into sqlite db (/tmp/gwas.sql by default); this takes a while
 
 # now, let's download from pubmed all the open-access papers mentioned in the GWAS catalog
 make dl-papers # downloads ~600 papers + their supplementary material!
@@ -70,7 +82,7 @@ This process can be automated by just typing `make`.
 
 ## Information extraction
 
-We demo our system in a series of Jupyter notebooks in the `notebooks` subfolder. 
+We demo our system in a series of Jupyter notebooks in the `notebooks` subfolder. Currently, notebooks 1 and 5 are up, and we are cleaning up the others (see `dev` branch).
 
 1. `phenotype-extraction.ipynb` identifies the phenotypes studied in each paper
 2. `table-pval-extraction.ipynb` extracts mutation ids and their associated p-values
@@ -82,4 +94,4 @@ The result is a second SQLite database containing facts (e.g. mutation/disease r
 
 ## Feedback
 
-Please send feedback to [Volodymyr Kuleshov](http://web.stanford.edu/~kuleshov/).
+Please send feedback to [Volodymyr Kuleshov](http://web.stanford.edu/~kuleshov/).