greenelab · Colin-Cheng · Jul 5, 2018 · Jul 5, 2018 · Jul 5, 2018 · Jul 6, 2018
diff --git a/README.md b/README.md
@@ -33,7 +33,7 @@ experimental validation at
 First, clone the repository and navigate into the top directory:
 
 ```bash
-git clone git@github.com:greenelab/tad_pathways_pipeline.git
+git clone https://github.com/marislab/tad_pathways_pipeline.git
 cd tad_pathways_pipeline
 ```
 
@@ -53,75 +53,50 @@ from an existing GWAS or the custom pipeline example for insight on how to run
 
 ### Examples
 
-We provide three different examples for a TAD pathways analysis pipeline. To run
-each of the analyses:
+We provide an example for a TAD pathways analysis pipeline. To run this example:
 
 ```bash
 source activate tad_pathways
 
-# Example using Bone Mineral Density GWAS
-bash example_pipeline_bmd.sh
-
-# Example using Type 2 Diabetes GWAS
-bash example_pipeline_t2d.sh
-
 # Example using custom input SNPs
 bash example_pipeline_custom.sh
 ```
 
 ### General Usage
 
-There are two ways to implement a TAD_Pathways analysis:
+To perform a `TAD_Pathways` analysis, uses need to spicify 3 inputs:
 
-1. GWAS
-2. Custom
+1. name of the tad cell:
 
-#### GWAS
+   E.g.: 'hESC'
 
-To perform a `TAD_Pathways` analysis on publicly available GWAS results, simply
-browse the `data/gwas_catalog/` directory to select a valid GWAS file. These
-files contain a curation of all significant SNPs mapped to specific traits as
-distributed by the [NHGRI-EBI GWAS Catalog](https://www.ebi.ac.uk/gwas/).
+2. path to the TAD domain file:
 
-Each file in this directory is a tab separated text file of genome-wide
-significant SNPs and their genomic location along with their reported nearest
-gene and associated PUBMED id. For complete information on how these files were
-constructed, refer to https://github.com/greenelab/tad_pathways.
+   The TAD domain file is a 3-column tab-separated bed file. The first column is the chromsome number. The second column is the  start position of the tad. And the third position is the end position of the tad.
 
-Each GWAS has 3 associated files, including files in `data/gwas_catalog/`. The
-other files are located in `data/gwas_tad_snps/` and `data/gwas_tad_genes/`.
-All files are important for performing a `TAD_Pathways` analysis. See the
-GWAS example files for instructions on how to implement the necessary scripts.
+   E.g.: [`hESC_domains_hg19.bed`](hESC_domains_hg19.bed)
 
-#### Custom
+3. path to the SNPs file
 
-To perform a `TAD_Pathways` analysis on a list of custom SNPs, generate a comma
-separated text file. The first row of the text file should have group names and
-subsequent rows should list the rs numbers of interest. There can be many
-columns with variable length rows.
+   The SNPs file is a comma separated text file. The first row of the text file should have group names and
+subsequent rows should list the rs numbers of interest. There can be manycolumns with variable length rows.
 
-E.g.: [`custom_example.csv`](custom_example.csv)
+   E.g.: [`custom_example.csv`](custom_example.csv)
 
-| Group 1 | Group 2 |
-| ------- | ------- |
-| rs12345 | rs67891 |
-| rs19876 | rs54321 |
-| ...     | ...     |
+   | Group 1 | Group 2 |
+   | ------- | ------- |
+   | rs12345 | rs67891 |
+   | rs19876 | rs54321 |
+   | ...     | ...     |
 
 Then, perform the following steps:
 
 ```bash
 source activate tad_pathways
 
-# Map custom SNPs to genomic locations
-Rscript --vanilla scripts/build_snp_list.R \
-        --snp_file "custom_example.csv" \
-        --output_file "mapped_results.tsv"
-
-# Build TAD based genelists for each group
-python scripts/build_custom_TAD_genelist.py \
-       --snp_data_file "mapped_results.tsv" \
-       --output_file "custom_tad_genelist.tsv"
+bash run_pipeline.sh --TAD-Boundary hESC \
+                     --TAD-File hESC_domains_hg19.bed \
+                     --SNP-File custom_example.csv
 ```
 
 The output of these steps are Group specific text files with all genes in TADs

diff --git a/example_pipeline_bmd.sh b/example_pipeline_bmd.sh
diff --git a/example_pipeline_custom.sh b/example_pipeline_custom.sh
@@ -6,6 +6,7 @@
 set -o errexit
 
 # Define filenames
+tad_file='data/hESC_domains_hg19.bed'
 candidate_snp_file='custom_example.csv'
 candidate_snp_location_file='results/custom_example_location.tsv'
 candidate_snp_tad_file='results/custom_example_tad_results.tsv'
@@ -14,6 +15,16 @@ trait='custom'
 evidence_file='results/custom_gene_evidence.csv'
 pathway_p_values_file='gestalt/custom_pvals.tsv'
 
+# Generate index files (maps to TAD identifiers to enable fast lookup)
+# 1000G SNP / genes / repeat elements
+python scripts/generate_index_files.py --TAD-Boundary 'hESC' --TAD-File $tad_file
+
+# Visualize SNPs and Genes in TADs
+# Output histograms and line graphs of SNP/Gene/Repeat locations in TADs
+# and gc content distribution across human and mouse tads
+python scripts/visualize_genomic_elements.py --TAD-Boundary 'hESC' 
+python scripts/visualize_gc_and_divergence.py --TAD-Boundary 'hESC' --TAD-File $tad_file
+
 # Map SNPs to genomic location
 Rscript --vanilla scripts/build_snp_list.R \
         --snp_file $candidate_snp_file \
@@ -22,7 +33,8 @@ Rscript --vanilla scripts/build_snp_list.R \
 # Build a customized genelist based on SNP locations
 python scripts/build_custom_tad_genelist.py \
         --snp_data_file $candidate_snp_location_file \
-        --output_file $candidate_snp_tad_file
+        --output_file $candidate_snp_tad_file \
+        --TAD-Boundary 'hESC'
 
 # Perform WebGestalt pathway analysis and parse results
 Rscript --vanilla scripts/webgestalt_run.R \

diff --git a/example_pipeline_t2d.sh b/example_pipeline_t2d.sh
diff --git a/figures/alu_divergence_hg19_hESC.pdf b/figures/alu_divergence_hg19_hESC.pdf
diff --git a/figures/gc_distribution_hg19_hESC.pdf b/figures/gc_distribution_hg19_hESC.pdf
diff --git a/figures/gene_count_hg19_hESC.pdf b/figures/gene_count_hg19_hESC.pdf
diff --git a/figures/gene_tad_distrib_chromosomes_hg19_hESC.pdf b/figures/gene_tad_distrib_chromosomes_hg19_hESC.pdf
diff --git a/figures/gene_types_hg19_hESC.pdf b/figures/gene_types_hg19_hESC.pdf
diff --git a/figures/repeat_count_hg19_hESC.pdf b/figures/repeat_count_hg19_hESC.pdf
diff --git a/figures/repeat_divergence_hg19_hESC.pdf b/figures/repeat_divergence_hg19_hESC.pdf
diff --git a/figures/repeat_type_all_distrib_hg19_hESC.pdf b/figures/repeat_type_all_distrib_hg19_hESC.pdf
diff --git a/figures/repeat_type_hg19_hESC_.pdf b/figures/repeat_type_hg19_hESC_.pdf
diff --git a/figures/snp_count_hg19_hESC.pdf b/figures/snp_count_hg19_hESC.pdf
diff --git a/figures/snp_tad_distrib_chromosomes_hg19_hESC.pdf b/figures/snp_tad_distrib_chromosomes_hg19_hESC.pdf
diff --git a/figures/snp_tad_distribution_hg19_hESC.pdf b/figures/snp_tad_distribution_hg19_hESC.pdf