v0.5.0

Enhancements

• #493 Add EMR Submitter
• #488 Use Spring ConfigurationProperties

Bugfixes

• #494 Use the sparkYarnSinkConfig.executablesFolder property
• #490 Use the shellExecutor.executablesFolder property
• #489 Use auth inmemory config properties
• #457 Job status should be updated immediately if the submission failed

Internal tasks

• #496 Remove generic form from details and sensors form
• #487 Upgrade spring version to v2.5.2
• #483 Use JsonB type for Sensor properties
• #477 Drop deprecated database columns
• #481 Add logging for failed futures

Application properties changes
New

• spark.submitApi. Must be either yarn or emr. Default value: yarn
• spark.emr.clusterId. The Id of the EMR-cluster (e.g. j-2AXXXXXXGAPLF) Mandatory, if spark.submitApi=emr
• spark.emr.filesToDeploy. Optional
• spark.emr.additionalConfs. Optional. This is just a prefix and works like the counterpart sparkYarnSink.additionalConfs
• spark.emr.awsProfile. Optional. Intended for local development
• spark.emr.region. Optional. Intended for local development

Renamed

• kafkaSource.key.deserializer to kafkaSource.properties.key.deserializer
• kafkaSource.value.deserializer to kafkaSource.properties.value.deserializer
• kafkaSource.max.poll.records to kafkaSource.properties.max.poll.records