Architecture Refactoring

[heikoheiko]

Reorganize the existing code into new logical layers with the main scope of increase the overall software quality and surpass architecture limitations that critically mine systematic qualities as scalability, extensibility, evolvability, testability and robustness of the actual solution system.

The current code base grew over multiple proof of concept implementations and is partially a bit messy. See issues: #80 #82 #83
The poc8 feature freeze is a good date to consider a cleanup of the architecture.

The revised architecture should strengthen the predominant characters of the python implementation, which are
* readability (the educational implementation)
* hackability (prototyping)
* reusability (libraries)

Issues touched by this refactoring:
#181 #182 #184 #185 #186

The new architecture should be compatible to python 3 (#132) and pypy (#188).
Q: Do we need py2.x compatibility?

Note the architecture proposed below might change

Libraries

devp2p

Standalone library, which offers a p2p base layer with peer discovery, DHT shaped topology, encryption, authentication, multiplexing, framing, peer info persistence and trust metrics. Can be used by registering sub-protocols.

devp2p  <-wirep2p-> other-nodes

pyethereumlib

Library with everything related to the chain and the eth protocol.

    blockchain
        db
            leveldb
            other db
        blocks
        processblocks
        trie
        transactions
        vm
   tester
   eth_wire_protocol
   miner
   synchronization

The Client

The actual client is split into multiple loosely coupled components, which are run in the same event loop.

__main__
event loop
    dev_p2p   <-wirep2p-> other-nodes
     |-- eth_protocol
     |       blockchain         
     |       synchronization
     |-- whisper_protocol
    API   <-json-rpc->   local clients

Separate Processes

miner

Can also be run in the client event loop.

miner process(es) <-> api

(network probably too slow if we incorporate state data)

cli

cli <-> api

gui / wallet (if any)

Can also be run in the client event loop.

gui <-> api

[heikoheiko]

WIP: https://github.com/heikoheiko/pydevp2p