Skip to content
This repository has been archived by the owner on May 21, 2020. It is now read-only.

Commit

Permalink
Update README
Browse files Browse the repository at this point in the history
  • Loading branch information
aserrallerios committed Apr 9, 2018
1 parent 0312d8b commit 36105a4
Showing 1 changed file with 25 additions and 14 deletions.
39 changes: 25 additions & 14 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,17 +12,26 @@ For more information about KCL please visit the [official documentation](http://
## Installation

```xml
<repository>
<snapshots>
<enabled>false</enabled>
</snapshots>
<id>bintray-<username>-maven</id>
<name>bintray</name>
<url>https://dl.bintray.com/content/aserrallerios/maven</url>
</repository>
...
<dependency>
<groupId>aserrallerios</groupId>
<artifactId>kcl-akka-stream_2.11</artifactId>
<version>0.1</version>
<version>0.3</version>
<type>pom</type>
</dependency>
```

```scala
resolvers += "aserrallerios bintray" at "http://dl.bintray.com/content/aserrallerios/maven"
libraryDependencies += "aserrallerios" %% "kcl-akka-stream" % "0.1"
resolvers += "aserrallerios bintray" at "https://dl.bintray.com/content/aserrallerios/maven"
libraryDependencies += "aserrallerios" %% "kcl-akka-stream" % "0.3"
```

## Usage
Expand Down Expand Up @@ -55,27 +64,29 @@ val workerSourceSettings = KinesisWorkerSourceSettings(
}
```

The Source also needs an `ExecutionContext` to run the Worker's thread and to execute record checkpoints. Then the Source can be created as usual:
The Source also needs an `ExecutionContext` to run the Worker's thread and to commit/checkpoint records. Then the Source can be created as usual:

```scala
implicit val _ =
ExecutionContext.fromExecutorService(Executors.newFixedThreadPool(1000))
KinesisWorker(builder, workerSourceSettings).to(Sink.ignore)
ExecutionContext.fromExecutorService(Executors.newFixedThreadPool(1000))
KinesisWorkerSource(builder, workerSourceSettings).to(Sink.ignore)
```

### Committing records

The KCL Worker Source publishes messages downstream that can be committed in order to mark progression of consumers by shard. This process can be done manually or using the provided checkpointer Flow.
The KCL Worker Source publishes messages downstream that can be committed in order to mark progression of consumers by shard. This process can be done manually or using the provided checkpointer Flow/Sink.

In order to use the Flow you can provide additional settings:
In order to use the Flow/Sink you must provide additional checkpoint settings:

```scala
val checkpointSettings = KinesisWorkerCheckpointSettings(100, 30 seconds)
KinesisWorkerSource(builder, workerSourceSettings)
.via(KinesisWorker.checkpointRecordsFlow(checkpointSettings))
.to(Sink.ignore)
KinesisWorker(builder, workerSourceSettings).to(
KinesisWorker.checkpointRecordsSink(checkpointSettings))
val checkpointSettings = KinesisWorkerCheckpointSettings(100, 30 seconds)

KinesisWorkerSource(builder, workerSourceSettings)
.via(KinesisWorker.checkpointRecordsFlow(checkpointSettings))
.to(Sink.ignore)

KinesisWorkerSource(builder, workerSourceSettings).to(
KinesisWorker.checkpointRecordsSink(checkpointSettings))
```

## License
Expand Down

0 comments on commit 36105a4

Please sign in to comment.