Merge pull request #522 from markovmodel/ipython_and_docs

Added new ipython notbooks and tests. Updated Changelog and docs index.

doc/source/CHANGELOG.rst

@@ -1,7 +1,41 @@
 Changelog
 =========
 
-1.2.2 (27-7-2015)
+2.0 (8-31-2015)
+---------------
+2.0 is a major release offering several new features and a major internal
+reorganization of the code.
+
+New features:
+
+- msm: Estimator for Bayesian Markov state models.
+- msm: MSMs can be systematically coarse-grained to few-state models
+- msm: Estimators for discrete Hidden Markov Models (HMMs) and Bayesian Hidden Markov models (BHMMs).
+- msm: SampledModels, e.g. generated from BayesianMSM or BayesianHMM allow statistics
+  (means, variances, confidence intervals) to be computed for all properties of MSMs and HMMs.
+- msm: Generalized Chapman-Kolmogorov test for both MSM and HMM models
+- plots: plotting functions for Chapman-Kolmogorov tests and 2D free energy surfaces.
+
+Documentation:
+
+- Two new application-based ipython notebooks and three new methodological ipython notebooks
+  are provided. All Notebooks and most of the data are provided for download at pyemma.org.
+
+Code architecture:
+
+- Object structure is more clear, general and extensible. We have three main
+  class types: Estimators, Transformers and Models. Estimators (e.g. MaximumLikelihoodMSM)
+  read data and produce a Transformer or a Model. Transformers (e.g. TICA) can be employed in
+  order to transform input data into output data (e.g. dimension reduction). Models
+  (e.g. MSM) can be analyzed in order to compute molecular quantities of interest, such
+  as equilibrium probabilities or transition rates.
+- Code for low-level msm functions (msm.analysis, msm.estimation, msm.generation, msm.flux) has
+  been relocated to the subsidiary package msmtools (github.com/markovmodel/msmtools). msmtools is
+  part of the PyEMMA distribution but can be separately installed without depending on
+  PyEMMA in order to facilitate further method development.
+
+
+1.2.2 (7-27-2015)
 -----------------
 - msm estimation: new fast transition matrix sampler
 - msm estimation: new feature "auto-sparse": automatically decide which datatype 
@@ -20,13 +54,13 @@ Changelog
 - general: made all example codes in documentation work.
 
 
-1.2.1 (28-5-2015)
+1.2.1 (5-28-2015)
 -----------------
 - general: Time consuming algorithms now display progressbars (optional).
 - general: removed scikit-learn dependency (due to new kmeans impl. Thanks @clonker)
 - coordinates package: new and faster implementation of Kmeans (10x faster than scikit-learn).
 - coordinates package: allow metrics to be passed to cluster algorithms.
-- coordinates package: cache trajectory lengths by default -
+- coordinates package: cache trajectory lengths by default
                        (uncached led to 1 pass of reading for non indexed (XTC) formats).
   This avoids re-reading e.g XTC files to determine their lengths.
 - coordinates package: enable passing chunk size to readers and pipelines in API.
@@ -36,10 +70,10 @@ Changelog
   In this case, an extra parameter topfile has to be parsed as well.
 - plots package: added functions to plot flux and msm models.
 - Bugfixes:
-    - [msm.MSM.pcca]: coarse-grained transition matrix corrected
-    - [msm.generation]: stopping states option fixed
-    - [coordinates.NumPyReader]: during gathering of shapes of all files, none
-                                 of them were closed. Thanks @gph82
+
+   - [msm.MSM.pcca]: coarse-grained transition matrix corrected
+   - [msm.generation]: stopping states option fixed
+   - [coordinates.NumPyReader]: during gathering of shapes of all files, none of them were closed.
 
 1.2 (4-14-2015)
 ---------------

doc/source/api/index.rst

@@ -1,10 +1,12 @@
 .. _ref_api:
 
-==========
-PyEMMA API
-==========
+=============
+Documentation
+=============
 
-The API is currently divided into two main parts:
+PyEMMA is a Python library and can be used through python or ipython.
+See Tutorials for examples and getting started.
+The API (Application Programming Interface) is currently divided into two main packages:
 
 .. toctree::
    :maxdepth: 2

doc/source/index.rst

@@ -26,8 +26,6 @@ Installation
    :maxdepth: 2
 
    INSTALL
-   Configuration
-
 
 Documentation
 =============
@@ -45,6 +43,13 @@ Tutorials
 
    ipython
 
+Run-time configuration
+======================
+
+.. toctree::
+   :maxdepth: 2
+
+   Configuration
 
 Development
 ===========

doc/source/ipython.rst

@@ -1,32 +1,52 @@
-==========================
-IPython Notebook Tutorials
-==========================
+=================
+IPython Tutorials
+=================
 
 These IPython (http://ipython.org) notebooks show the usage of the PyEMMA API in
 action and also describe the workflow of Markov model building.
 
-
 You can obtain a copy of all notebooks and most of the used data
 `here <https://github.com/markovmodel/PyEMMA_IPython/archive/devel.zip>`_.
 Note that the trajectory of the D.E. Shaw BPTI simulation trajectory is not included
 in this archive, since we're not permitted to share this data. Thus the corresponding
 notebooks can't be run without obtaining the simulation trajectory independently.
 
-Pentapeptide
-=============================
+Application walkthroughs
+========================
 
 .. toctree::
    :maxdepth: 1
 
-   generated/md2msm_penta_peptide
+   generated/pentapeptide_msm
+
+   generated/MSM_BPTI
+
+   generated/trypsin_benzamidine_hmm
+
+
+By means of application examples, these notebooks give an overview of following methods:
 
+   * Featurization and MD trajectory input
+   * Time-lagged independent component analysis (TICA)
+   * Clustering
+   * Markov state model (MSM) estimation and validation
+   * Computing Metastable states and structures, coarse-grained MSMs
+   * Hidden Markov Models (HMM)
+   * Transition Path Theory (TPT)
 
-BPTI
-=============================
+
+Methods
+=======
+
+In this section we will give you in-depth tutorials on specific methods or concepts.
 
 .. toctree::
    :maxdepth: 1
 
-   generated/MSM_BPTI
+   generated/feature_selection
+
+   generated/model_selection_validation
+
+   generated/tpt