Fix build

Author: rogerdahl

README.md

@@ -150,58 +150,6 @@ We have added some custom functionality to pytest which can be enabled to launch
 
 * Pytest-django forces `settings.DEBUG` to `False` in `pytest_django/plugin.py`. To set `settings.DEBUG`, override it close to where it will be read, e.g., wit `@django.test.override_settings(DEBUG=True)`.
 
-### Creating a new release
-
-The stack should pass all the tests with the most recent available versions of all the dependencies.
-
-    $ cd d1_python
-    $ sudo ./dev_tools/src/d1_dev/pip-update-all.py
-
-The DataONE Python stack specifies fixed versions of all its dependencies. This ensures that a stack deployed to production matches one that passed the tests. As updating the versions in the `setup.py` files manually is time consuming and error prone, a script is included that automates the task. The script updates the version information for the dependencies in the `setup.py` files to match the versions of the currently installed dependencies. Run the script with:
-
-    $ cd d1_python
-    $ src-sync-dependencies.py . <version>
-
-The `<version>` argument specifies what the version will be for the release. E.g., `"2.3.1"`. We keep the version numbers in sync between all of the packages in the d1_python git repository, so only one version string needs to be specified.
-
-Check that there are no package version conflicts:
-
-    $ pip check
-
-Some sample files contain the version tag and must be updated when the version changes.  
-
-    $ pytest --sample-ask
-
-Commit and push the changes, and check the build on Travis.
-
-After successful build, clone a fresh copy, which will be used for building the release packages:
-
-    $ cd ~
-    $ git clone [email protected]:DataONEorg/d1_python.git d1_python_build
-
-Building the release packages from a fresh clone is a simple way of ensuring that only tracked files are released. It is a workaround for the way setuptools works, which is basically that it vacuums up everything that looks like a Python script in anything that looks like a package, which makes it easy to publish local files by accident.
-
-Build and publish the packages:
-
-    cd ~/d1_python_build
-    setup-all.py --root . build sdist bdist_wheel  
-
-
-
-
-#### Update the setup.py files
-
-
-#### Build the new packages
-
-    $ clean-tree.py
-    $ setup-all.py sdist bdist_wheel
-
-#### Push the new packages to PyPI
-
-    TODO
-
-
 ### Setting up the development environment
 
 These instructions are tested on Linux Mint 18 and should also work on close derivatives.
@@ -217,10 +165,6 @@ Download the source from GitHub:
 
     $ git clone https://github.com/DataONEorg/d1_python.git
 
-Set the `D1ROOT` environment variable to the location of `d1_python`. It's handy to set the variable in the startup script for your shell. E.g.,:
-
-    $ export D1ROOT=~/my/dev/d1_python
-
 Add the DataONE packages to the Python path, and install their dependencies:
 
     $ sh $D1ROOT/dev_tools/develop.sh
@@ -231,7 +175,6 @@ Run the following commands, except, change the "createuser" line to:
 
     https://pythonhosted.org/dataone.generic_member_node/setup-local-postgresql.html
 
-
 Run the following commands (all sections), except, change the location for openssl.cnf, so the line that copies it becomes:
 
     $ sudo cp /home/dahl/d1_python/d1_mn_generic/src/deployment/openssl.cnf .
@@ -242,12 +185,58 @@ Run the tests and verify that they all pass:
 
     $ pytest
 
+Set up credentials for working with the DataONE account on PyPI:
+
+Edit `~/.pypirc`:
+
+    [server-login]
+    username: dataone
+    password: <secret>
+
+### Creating a new release
+
+The stack should pass all the tests with the most recent available versions of all the dependencies.
+
+    $ cd d1_python
+    $ sudo ./dev_tools/src/d1_dev/pip-update-all.py
+
+The DataONE Python stack specifies fixed versions of all its dependencies. This ensures that a stack deployed to production matches one that passed the tests. As updating the versions in the `setup.py` files manually is time consuming and error prone, a script is included that automates the task. The script updates the version information for the dependencies in the `setup.py` files to match the versions of the currently installed dependencies. Run the script with:
+
+    $ cd d1_python
+    $ src-sync-dependencies.py . <version>
+
+The `<version>` argument specifies what the version will be for the release. E.g., `"2.3.1"`. We keep the version numbers in sync between all of the packages in the d1_python git repository, so only one version string needs to be specified.
+
+Check that there are no package version conflicts:
+
+    $ pip check
+
+Some sample files contain the version tag and must be updated when the version changes.  
+
+    $ pytest --sample-ask
+
+Commit and push the changes, and check the build on Travis.
+
+After successful build, clone a fresh copy, which will be used for building the release packages:
+
+    $ cd ~
+    $ git clone [email protected]:DataONEorg/d1_python.git d1_python_build
+
+Building the release packages from a fresh clone is a simple way of ensuring that only tracked files are released. It is a workaround for the way setuptools works, which is basically that it vacuums up everything that looks like a Python script in anything that looks like a package, which makes it easy to publish local files by accident.
+
+Build and publish the packages:
+
+    cd ~/d1_python_build
+    setup-all.py --root . bdist_wheel upload
+
 
 ### Building the documentation
 
+When `d1_python` is pushed to GitHub, a signal is sent by GitHub to [ReadTheDocs.org](https://readthedocs.org/), which automatically retrieves the new version of the project from GitHub, builds the documentation and makes it available at
 
-    TODO
+http://dataone-python.readthedocs.io/en/latest/
 
+So it is not absolutely necessary to have a local build environment set up for the documentation, but building locally provides faster feedback when making changes that need to be checked before publishing.  
 
 ### Troubleshooting
 

client_cli/src/setup.py

@@ -28,7 +28,7 @@
 def main():
   setuptools.setup(
     name='dataone.cli',
-    version='2.3.2',
+    version='2.3.3',
     description='Command-Line Interface (CLI) for DataONE',
     author='DataONE Project',
     author_email='[email protected]',
@@ -37,8 +37,8 @@ def main():
     packages=setuptools.find_packages(),
     include_package_data=True,
     install_requires=[
-      'dataone.common == 2.3.2',
-      'dataone.libclient == 2.3.2',
+      'dataone.common == 2.3.3',
+      'dataone.libclient == 2.3.3',
       #
       'requests == 2.18.1',
     ],

client_onedrive/src/setup.py

@@ -32,7 +32,7 @@
 def main():
   setuptools.setup(
     name='dataone.onedrive',
-    version='2.3.2',
+    version='2.3.3',
     description='Filesystem access to the DataONE Workspace',
     author='DataONE Project',
     author_email='[email protected]',
@@ -41,8 +41,8 @@ def main():
     packages=setuptools.find_packages(),
     include_package_data=True,
     install_requires=[
-      'dataone.common == 2.3.2',
-      'dataone.libclient == 2.3.2',
+      'dataone.common == 2.3.3',
+      'dataone.libclient == 2.3.3',
       #
       'fusepy == 2.0.4',
       'pyxb == 1.2.5',

dev_tools/src/d1_dev/src-sync-dependencies.py

@@ -91,7 +91,12 @@ def main():
     except Exception as e:
       logging.error(str(e))
 
-  update_common_version_const(args.d1_version, args.show_diff)
+  update_version_const(
+    'd1_common', ['const.py'], args.d1_version, args.show_diff
+  )
+  update_version_const(
+    'd1_gmn', ['version.py'], args.d1_version, args.show_diff
+  )
 
 
 def update_deps_on_file(args, setup_path, show_diff, d1_version):
@@ -176,23 +181,19 @@ def get_package_version(package_name, d1_version):
     return pkg_resources.get_distribution(package_name).version
 
 
-def update_common_version_const(d1_version, only_diff):
-  const_module_path = get_common_const_module_path()
-  logging.info(
-    'Updating VERSION in d1_common.const. path="{}"'.format(const_module_path)
-  )
-  r = d1_dev.util.redbaron_module_path_to_tree(const_module_path)
+def update_version_const(base_name, path_list, d1_version, only_diff):
+  module_path = get_module_path(base_name, path_list)
+  logging.debug('Updating version in module. path="{}"'.format(module_path))
+  r = d1_dev.util.redbaron_module_path_to_tree(module_path)
   for n in r('AssignmentNode'):
-    if n.target.value == 'VERSION':
+    if n.target.value in ('VERSION', '__version__'):
       n.value.value = "'{}'".format(d1_version)
-      d1_dev.util.update_module_file(
-        r, const_module_path, only_diff, update=True
-      )
+      d1_dev.util.update_module_file(r, module_path, only_diff, update=True)
       break
 
 
-def get_common_const_module_path():
-  return os.path.join(pkgutil.get_loader("d1_common").filename, 'const.py')
+def get_module_path(base_str, path_list):
+  return os.path.join(pkgutil.get_loader(base_str).filename, *path_list)
 
 
 class UpdateException(Exception):

dev_tools/src/setup.py

@@ -29,7 +29,7 @@
 def main():
   setuptools.setup(
     name='dataone.dev',
-    version='2.3.2',
+    version='2.3.3',
     description='DataONE developer tools',
     author='DataONE Project',
     author_email='[email protected]',

gmn/src/d1_gmn/__init__.py

@@ -18,4 +18,5 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
+from .version import __version__ # noqa: F401
 default_app_config = 'd1_gmn.app.startup.GMNStartupChecks'

gmn/src/d1_gmn/app/views/internal.py

@@ -58,7 +58,7 @@ def home(request):
   if request.path.endswith('/'):
     return HttpResponseRedirect(request.path[:-1])
 
-  gmn_version = d1_gmn.app.__version__
+  gmn_version = d1_gmn.__version__
   django_version = ', '.join(map(str, django.VERSION))
 
   n_science_objects = '{:,}'.format(

gmn/src/d1_gmn/version.py

@@ -0,0 +1 @@
+__version__ = '2.3.3'

gmn/src/setup.py

@@ -27,9 +27,11 @@
 
 
 def main():
+  exec (open('d1_gmn/version.py').read())
+  # noinspection PyUnresolvedReferences
   setuptools.setup(
     name='dataone.gmn',
-    version='2.3.2',
+    version='2.3.3',
     description='DataONE Generic Member Node (GMN)',
     author='DataONE Project',
     author_email='[email protected]',
@@ -41,9 +43,9 @@ def main():
       '': ['settings.py'],
     },
     install_requires=[
-      'dataone.cli == 2.3.2',
-      'dataone.common == 2.3.2',
-      'dataone.libclient == 2.3.2',
+      'dataone.cli == 2.3.3',
+      'dataone.common == 2.3.3',
+      'dataone.libclient == 2.3.3',
       #
       'django == 1.11.2',
       'iso8601 == 0.1.11',

lib_client/src/setup.py

@@ -28,7 +28,7 @@
 def main():
   setuptools.setup(
     name='dataone.libclient',
-    version='2.3.2',
+    version='2.3.3',
     description='A DataONE client library for Python',
     author='DataONE Project',
     author_email='[email protected]',
@@ -37,7 +37,7 @@ def main():
     packages=setuptools.find_packages(),
     include_package_data=True,
     install_requires=[
-      'dataone.common == 2.3.2',
+      'dataone.common == 2.3.3',
       #
       'cachecontrol == 0.12.3',
       'pyxb == 1.2.5',