SETL-0.4.1
Changes:
- Changed benchmark unit of time to seconds (#88)
Fixes:
- The master URL of SparkSession can now be overwritten in local environment (#74)
FileConnector
now lists path correctly for nested directories (#97)
New features:
- Added Mermaid diagram generation to Pipeline (#51)
- Added
showDiagram()
method to Pipeline that prints the Mermaid code and generates the live editor URL 🎩🐰✨ (#52) - Added Codecov report and Scala API doc
- Added
delete
method inJDBCConnector
(#82) - Added
drop
method inDBConnector
(#83) - Added support for both of the following two Spark configuration styles in SETL builder (#86)
setl.config { spark { spark.app.name = "my_app" spark.sql.shuffle.partitions = "1000" } } setl.config_2 { spark.app.name = "my_app" spark.sql.shuffle.partitions = "1000" }
Others:
- Improved test coverage