ENG-10202 create kinesis exporter

.gitignore

@@ -0,0 +1,211 @@
+.gradle
+**/build/
+**/tmp/
+loader.cfg
+
+# Ignore Gradle GUI config
+gradle-app.setting
+*.class
+
+# Generated Eclipse stuff
+.project
+.classpath
+
+# Package Files #
+*.jar
+*.war
+*.ear
+
+# virtual machine crash logs, see http://www.java.com/en/download/help/error_hotspot.xml
+hs_err_pid*
+
+.DS_Store
+.AppleDouble
+.LSOverride
+
+# Icon must end with two \r
+Icon
+
+
+# Thumbnails
+._*
+
+# Files that might appear on external disk
+.Spotlight-V100
+.Trashes
+
+# Directories potentially created on remote AFP share
+.AppleDB
+.AppleDesktop
+Network Trash Folder
+Temporary Items
+.apdisk
+
+*~
+\#*\#
+/.emacs.desktop
+/.emacs.desktop.lock
+*.elc
+auto-save-list
+tramp
+.\#*
+
+# Org-mode
+.org-id-locations
+*_archive
+
+# flymake-mode
+*_flymake.*
+
+# eshell files
+/eshell/history
+/eshell/lastdir
+
+# elpa packages
+/elpa/
+
+# reftex files
+*.rel
+
+# AUCTeX auto folder
+/auto/
+
+# cask packages
+.cask/
+
+# cache files for sublime text
+*.tmlanguage.cache
+*.tmPreferences.cache
+*.stTheme.cache
+
+# workspace files are user-specific
+*.sublime-workspace
+
+# project files should be checked into the repository, unless a significant
+# proportion of contributors will probably not be using SublimeText
+# *.sublime-project
+
+# sftp configuration file
+sftp-config.json
+
+# KDE directory preferences
+.directory
+
+*.pydevproject
+.metadata
+.gradle
+gradle.properties
+bin/
+obj/
+tmp/
+*.tmp
+*.bak
+*.swp
+*~.nib
+local.properties
+.settings/
+.loadpath
+
+# External tool builders
+.externalToolBuilders/
+
+# Locally stored "Eclipse launch configurations"
+*.launch
+
+# CDT-specific
+.cproject
+
+# PDT-specific
+.buildpath
+
+# sbteclipse plugin
+.target
+
+# TeXlipse plugin
+.texlipse
+
+nbproject/private/
+build/
+nbbuild/
+dist/
+nbdist/
+nbactions.xml
+nb-configuration.xml
+
+# It's better to unpack these files and commit the raw source because
+# git has its own built in compression methods.
+*.7z
+*.jar
+*.rar
+*.zip
+*.gz
+*.bzip
+#*.bz2
+*.xz
+*.lzma
+*.cab
+
+#packing-only formats
+*.iso
+*.tar
+
+#package management formats
+*.dmg
+*.xpi
+*.gem
+*.egg
+*.deb
+*.rpm
+*.msi
+*.msm
+*.msp
+
+# Covers JetBrains IDEs: IntelliJ, RubyMine, PhpStorm, AppCode, PyCharm
+
+*.iml
+
+## Directory-based project format:
+.idea/
+# if you remove the above rule, at least ignore the following:
+
+# User-specific stuff:
+# .idea/workspace.xml
+# .idea/tasks.xml
+# .idea/dictionaries
+
+# Sensitive or high-churn files:
+# .idea/dataSources.ids
+# .idea/dataSources.xml
+# .idea/sqlDataSources.xml
+# .idea/dynamic.xml
+# .idea/uiDesigner.xml
+
+# Gradle:
+# .idea/gradle.xml
+# .idea/libraries
+
+# Mongo Explorer plugin:
+# .idea/mongoSettings.xml
+
+## File-based project format:
+*.ipr
+*.iws
+
+## Plugin-specific files:
+
+# IntelliJ
+out/
+
+# mpeltonen/sbt-idea plugin
+.idea_modules/
+
+# JIRA plugin
+atlassian-ide-plugin.xml
+
+# Crashlytics plugin (for Android Studio and IntelliJ)
+com_crashlytics_export_strings.xml
+crashlytics.properties
+crashlytics-build.properties
+
+
+/.recommenders/

LICENSE.md

@@ -0,0 +1,23 @@
+The MIT License (MIT)
+=====================
+
+Copyright © 2015 VoltDB Inc.
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+“Software”), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

README.md

@@ -0,0 +1,97 @@
+# VoltDB Kinesis Firehose Export Conduit
+
+An experimental VoltDB to Kinesis Firehose export conduit [Kinesis Firehose API]
+(http://docs.aws.amazon.com/firehose/latest/dev/what-is-this-service.html). It
+allows export stream writers to push data directly into correspoding Kinesis Firehose stream.
+
+## How to build artifacts and setup Eclipse
+
+* Install Gradle
+
+On a Mac if you have [Homebrew](http://brew.sh/) setup then simply install the gradle bottle
+
+```bash
+brew install gradle
+```
+
+On Linux setup [GVM](http://gvmtool.net/), and install gradle as follows
+
+```bash
+gvm install gradle
+```
+
+* Create `gradle.properties` file and set the `voltdbhome` property
+   to the base directory where your VoltDB is installed
+
+```bash
+echo voltdbhome=/voltdb/home/dirname > gradle.properties
+```
+
+* Invoke gradle to compile artifacts
+
+```bash
+gradle shadowJar
+```
+
+* To setup an eclipse project run gradle as follows
+
+```bash
+gradle cleanEclipse eclipse
+```
+then import it into your eclipse workspace by using File->Import projects menu option
+
+## Configuration
+
+* Copy the built jar from `build/libs` to `lib/extension` under your VoltDB installation directory
+
+* Edit your deployment file and use the following export XML stanza as a template
+
+```xml
+<?xml version="1.0"?>
+<deployment>
+    <cluster hostcount="1" sitesperhost="4" kfactor="0" />
+    <httpd enabled="true">
+        <jsonapi enabled="true" />
+    </httpd>
+    <export>
+        <configuration target="default" enabled="true" type="custom"
+            exportconnectorclass="org.voltdb.exportclient.KinesisFirehoseExportClient">
+            <property name="region">us-east-1</property>
+            <property name="stream.name">streamtest</property>
+            <property name="access.key"></property>
+            <property name="secret.key"></property>
+        </configuration>
+    </export>
+</deployment>
+```
+
+This tells VoltDB to write to the alerts stream and send the content to the Amazon Kinesis Firehose stream
+with the name streamtest. If the client created with the supplied access.key and secret.key have access
+to this stream then this stream will be successfully created. In this example we create the VoltDB export
+with the definition:
+
+```sql
+CREATE STREAM alerts EXPORT TO TARGET default (
+  id integer not null,
+  msg varchar(128),
+  continent varchar(64),
+  country varchar(64)
+);
+```
+
+Then data can be inserted into this export stream using the command:
+
+```sql
+INSERT INTO ALERTS (ID,MSG,CONTINENT,COUNTRY) VALUES (1,'fab-02 inoperable','EU','IT');
+```
+
+## Configuration Properties
+
+- `region` (mandatory) designates the AWS region where the Kinesis Firehose stream is defined
+- `stream.name`  (mandatory) Kinesis Firehose stream name
+- `access.key` (mandatory) user's access key
+- `secret.key` (mandatory) user's secret key
+- `skipinternals` (optional, _default:_ false) flag to skip adding internal metadata to each row
+- `timezone` (optional, _default:_ local timezone) timezone used to format timestamp values
+- `binaryencoding` (optional, _default:_ hex) Specifies whether VARBINARY data is encoded in hexadecimal or BASE64 format.
+- `type` (optional, _default:_ csv) specifies whether export format is csv or tsv.