forked from orkes-io/docs
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
1 parent
9bb3dcd
commit f5f3425
Showing
7 changed files
with
199 additions
and
158 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -7,121 +7,97 @@ To find out more about Conductor visit: [https://github.com/Netflix/conductor](h | |
| ## Quick Start | ||
|
|
||
| 1. [Setup conductor-go package](#Setup-conductor-go-package) | ||
| 1. [Write worker as a function](#Write-worker-as-a-function) | ||
| 1. [Run workers](#Run-workers) | ||
| 1. [Configuration](#Configuration) | ||
| 2. [Run workers](#Run-workers) | ||
| 3. [Configuration](#Configuration) | ||
|
|
||
| ### Setup-conductor-go-package | ||
| ### Setup conductor go package | ||
|
|
||
| Create a folder to build your package: | ||
| ```shell | ||
| $ mkdir conductor-go/ | ||
| $ cd conductor-go/ | ||
| mkdir conductor-go/ | ||
| cd conductor-go/ | ||
| ``` | ||
|
|
||
| Create a `go.mod` file inside this folder, with this content: | ||
| ``` | ||
| Create a go.mod file for dependencies | ||
| ```go | ||
| module conductor_test | ||
|
|
||
| go 1.18 | ||
|
|
||
| require ( | ||
| github.com/conductor-sdk/conductor-go v1.0.8 | ||
| github.com/conductor-sdk/conductor-go v1.1.1 | ||
| ) | ||
| ``` | ||
|
|
||
| Now you may be able to create your workers and main function. | ||
|
|
||
| ### Write worker as a function | ||
| You can download [this code](examples/task_execute_function/task_execute_function.go) into the repository folder with: | ||
| ```shell | ||
| $ wget "https://github.com/conductor-sdk/conductor-go/blob/main/examples/task_execute_function/task_execute_function.go" | ||
| ``` | ||
|
|
||
| ### Run workers | ||
| You can download [this code](examples/main/main.go) into the repository folder with: | ||
| ```shell | ||
| $ wget "https://github.com/conductor-sdk/conductor-go/blob/main/examples/main/main.go" | ||
| ``` | ||
| Now, create simple worker implentation | ||
| ```go | ||
| package main | ||
|
|
||
| import ( | ||
| "github.com/conductor-sdk/conductor-go/pkg/http_model" | ||
| "github.com/conductor-sdk/conductor-go/pkg/model" | ||
| "github.com/conductor-sdk/conductor-go/pkg/model/enum/task_result_status" | ||
| "github.com/conductor-sdk/conductor-go/pkg/settings" | ||
| "github.com/conductor-sdk/conductor-go/pkg/worker" | ||
| log "github.com/sirupsen/logrus" | ||
| "os" | ||
| ) | ||
|
|
||
| ### Running Conductor server locally in 2-minute | ||
| More details on how to run Conductor see https://netflix.github.io/conductor/server/ | ||
| func init() { | ||
| log.SetFormatter(&log.JSONFormatter{}) | ||
| log.SetOutput(os.Stdout) | ||
| log.SetLevel(log.DebugLevel) | ||
| } | ||
|
|
||
| func Worker(t *http_model.Task) (taskResult *http_model.TaskResult, err error) { | ||
| taskResult = model.GetTaskResultFromTask(t) | ||
| taskResult.OutputData = map[string]interface{}{ | ||
| "task": "task_1", | ||
| "key3": 3, | ||
| "key4": false, | ||
| } | ||
| taskResult.Status = task_result_status.COMPLETED | ||
| err = nil | ||
| return taskResult, err | ||
| } | ||
|
|
||
| func main() { | ||
| taskRunner := worker.NewTaskRunner( | ||
| settings.NewAuthenticationSettings( | ||
| __KEY__, | ||
| __SECRET__, | ||
| ), | ||
| settings.NewHttpSettings( | ||
| "https://play.orkes.io", | ||
| ), | ||
| ) | ||
|
|
||
| taskRunner.StartWorker( | ||
| "go_task_example", | ||
| Worker, | ||
| 2, | ||
| 10, | ||
| ) | ||
|
|
||
| taskRunner.WaitWorkers() | ||
| } | ||
|
|
||
| Use the script below to download and start the server locally. The server runs in memory and no data saved upon exit. | ||
| ```shell | ||
| export CONDUCTOR_VER=3.5.2 | ||
| export REPO_URL=https://repo1.maven.org/maven2/com/netflix/conductor/conductor-server | ||
| curl $REPO_URL/$CONDUCTOR_VER/conductor-server-$CONDUCTOR_VER-boot.jar \ | ||
| --output conductor-server-$CONDUCTOR_VER-boot.jar; java -jar conductor-server-$CONDUCTOR_VER-boot.jar | ||
| ``` | ||
| ### Execute workers | ||
| ```shell | ||
| go ./main.go | ||
| ``` | ||
|
|
||
| ### Create your first workflow | ||
| Now, let's create a new workflow and see your task worker code in execution! | ||
|
|
||
| Create a new Task Metadata for the worker you just created | ||
|
|
||
| Install dependencies. This will download all the required dependencies | ||
| ```shell | ||
| curl -X 'POST' \ | ||
| 'http://localhost:8080/api/metadata/taskdefs' \ | ||
| -H 'accept: */*' \ | ||
| -H 'Content-Type: application/json' \ | ||
| -d '[{ | ||
| "name": "go_task_example", | ||
| "description": "Go task example", | ||
| "retryCount": 3, | ||
| "retryLogic": "FIXED", | ||
| "retryDelaySeconds": 10, | ||
| "timeoutSeconds": 300, | ||
| "timeoutPolicy": "TIME_OUT_WF", | ||
| "responseTimeoutSeconds": 180, | ||
| "ownerEmail": "[email protected]" | ||
| }]' | ||
| go get | ||
| ``` | ||
| **Note:** | ||
| Replace `KEY` and `SECRET` by obtaining a new key and secret from Orkes Playground as described [Generating Access Keys for Programmatic Access](https://orkes.io/content/docs/getting-started/concepts/access-control#access-keys) | ||
|
|
||
| Create a workflow that uses the task | ||
| ```shell | ||
| curl -X 'POST' \ | ||
| 'http://localhost:8080/api/metadata/workflow' \ | ||
| -H 'accept: */*' \ | ||
| -H 'Content-Type: application/json' \ | ||
| -d '{ | ||
| "name": "workflow_with_go_task_example", | ||
| "description": "Workflow with Go Task example", | ||
| "version": 1, | ||
| "tasks": [ | ||
| { | ||
| "name": "go_task_example", | ||
| "taskReferenceName": "go_task_example_ref_1", | ||
| "inputParameters": {}, | ||
| "type": "SIMPLE" | ||
| } | ||
| ], | ||
| "inputParameters": [], | ||
| "outputParameters": { | ||
| "workerOutput": "${go_task_example_ref_1.output}" | ||
| }, | ||
| "schemaVersion": 2, | ||
| "restartable": true, | ||
| "ownerEmail": "[email protected]", | ||
| "timeoutPolicy": "ALERT_ONLY", | ||
| "timeoutSeconds": 0 | ||
| }' | ||
| ``` | ||
|
|
||
| Start a new workflow execution | ||
| ### Run workers | ||
| Start the workers by running `go run` | ||
| ```shell | ||
| curl -X 'POST' \ | ||
| 'http://localhost:8080/api/workflow/workflow_with_go_task_example?priority=0' \ | ||
| -H 'accept: text/plain' \ | ||
| -H 'Content-Type: application/json' \ | ||
| -d '{}' | ||
| go run main.go | ||
| ``` | ||
|
|
||
|
|
||
| ## Configuration | ||
|
|
||
| ### Authentication settings (optional) | ||
|
|
@@ -131,64 +107,49 @@ Use if your conductor server requires authentication | |
|
|
||
| ```go | ||
| authenticationSettings := settings.NewAuthenticationSettings( | ||
| "keyId", | ||
| "keySecret", | ||
| ), | ||
| ``` | ||
|
|
||
| ### External Storage Settings (optional) | ||
| Use if you would like to upload large payload at an external storage | ||
| You may define max payload size and threshold for uploading, also with a function capable of returning the path where it is stored. | ||
|
|
||
| ```go | ||
| externalStorageSettings := settings.NewExternalStorageSettings( | ||
| 4, // taskOutputPayloadThresholdKB | ||
| 10, // taskOutputMaxPayloadThresholdKB | ||
| external_storage_handler.UploadAndGetPath, // External Storage Handler function | ||
| ), | ||
| ``` | ||
|
|
||
| ### HTTP Settings (optional) | ||
|
|
||
| * baseUrl: Conductor server address. e.g. http://localhost:8000 if running locally | ||
|
|
||
| ```go | ||
| httpSettings := settings.NewHttpSettings( | ||
| "https://play.orkes.io/api", | ||
| externalStorageSettings, | ||
| "keyId", | ||
| "keySecret", | ||
| ) | ||
| ``` | ||
|
|
||
| ### Metrics Settings | ||
| Conductor uses [Prometheus](https://prometheus.io/) to collect metrics. | ||
|
|
||
| * apiEndpoint : Address to serve metrics (e.g. `/metrics`) | ||
| * port : Port to serve metrics (e.g. `2112`) | ||
|
|
||
| With this configuration, you can access metrics via `http://localhost:2112/metrics` after exposing them with: | ||
|
|
||
| ```go | ||
| metricsSettings := settings.NewMetricsSettings( | ||
| "/metrics", | ||
| 2112, | ||
| ) | ||
|
|
||
| go metrics.ProvideMetrics(metricsSettings) | ||
| ``` | ||
|
|
||
| ### Worker Settings | ||
|
|
||
| You can create a new worker by calling `workerOrkestrator.StartWorker` with: | ||
| * taskType : Task definition name (e.g `"go_task_example"`) | ||
| * executeFunction : Task Execution Function (e.g. `example.TaskExecuteFunctionExample1` from `example` folder) | ||
| * parallelGoRoutinesAmount : Amount of Go routines to be executed in parallel for new worker (e.g. `1`, single thread) | ||
| * pollingInterval : Amount of ms to wait between polling for task | ||
| * threadCount : Amount of Go routines to be executed in parallel for new worker (e.g. `1`, single thread) | ||
| * pollIntervalInMillis : Amount of ms to wait between polling for task | ||
|
|
||
| ```go | ||
| workerOrkestrator.StartWorker( | ||
| taskRunner.StartWorker( | ||
| "go_task_example", // task definition name | ||
| task_execute_function.Example1, // task execution function | ||
| 1, // parallel go routines amount | ||
| 5000, // 5000ms | ||
| Worker, // task execution function | ||
| 1, // thread count | ||
| 1000, // polling interval in milli-seconds | ||
| ) | ||
| ``` | ||
| ### Start a workflow using APIs | ||
| ```go | ||
|
|
||
| apiClient := conductor_http_client.NewAPIClient( | ||
| settings.NewAuthenticationSettings( | ||
| KEY, | ||
| SECRET, | ||
| ), | ||
| settings.NewHttpSettings( | ||
| "https://play.orkes.io", | ||
| ), | ||
| ) | ||
|
|
||
| workflowClient := *&conductor_http_client.WorkflowResourceApiService{ | ||
| APIClient: apiClient, | ||
| } | ||
| workflowId, _, _ := workflowClient.StartWorkflow( | ||
| context.Background(), | ||
| map[string]interface{}{}, | ||
| "PopulationMinMax", | ||
| &conductor_http_client.WorkflowResourceApiStartWorkflowOpts{}, | ||
| ) | ||
| log.Info("Workflow Id is ", workflowId) | ||
|
|
||
| ``` | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.