In this release, DistML supoort following features:
- parameter servers on Spark
- data exchange between workers and parameter servers (DataBus)
- model api for new algorithms
- support typical models: word2vec, sparselr, lda and cnn
We have verified word2vec with millions words, sparselr with billions of weights, lda with thousands of topics, additional, it's more easily to scale to bigger problems with bigger clusters.
Welcome everyone to join this project or evaluate DistML, if any issues, please feel free to contact [email protected].