pydoit compatibility for nbflow

[camillescott]

Hello there!

I like to use pydoit as my build system; it supports Python3 and is quite extensible. This PR adds a module to nbflow for pydoit.

The folder doit-example shows how it works. It uses a dodo.py file, which follows a similar pattern to the scons example:

from nbflow.doit import create_build_tasks

def task_nbflow():
    yield from create_build_tasks(['analyses'])

Major kudos for your arch making this really easy to write!

Todos:

• Needs tests ;)
• Works only with Python 3, which conflicts with he scons module and example notebooks. See here for instructions on putting Python 2 and 3 kernels side by side.
• Should possibly be its own package, something like nbflow-doit
• A couple bugs have been fixed which I need to push downstream (more PRs incoming)
• If keeping in same project, update README

Cheers,
Camille

[jhamrick]

Thanks! Let's loop @joestubbs too who was thinking of working on exactly this as well.

[jhamrick]

I would prefer if we actually kept just a single example that can be run multiple ways -- so if we have the SConstruct and dodo.py side-by-side and users can either run scons or doit depending on which system they want to try. Then we also don't need to maintain multiple almost identical examples.

[camillescott]

Good point; done.

[joestubbs]

Thanks for looping me in! I wrote a pydoit.py module with a setup function as we discussed, and the original workflow example builds successfully, but I hadn't added any additional tests. My implementation doesn't add pydoit to the requirements; rather, it assumes the user has installed it themselves and has their own dodo.py file (or task loader) that they can complement with the setup function.

If this would be of use I'm happy to make it available - just let me know.

[ajasja]

Hi!
Just watched the Scipy 2016 talk. Great Idea!
I would also much prefer to use pydoit instead of Scons.
Best,
Ajasja