Skip to content

Latest commit

 

History

History

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
 
 
 
 

Performing Expensive Computation in Workflow Code

Tip

Workflow tasks have a default 10-second timeout. Expensive computation blocks the task, causing timeouts and livelocks where the workflow perpetually retries but never makes progress.

Performing heavy computation directly in workflow code -- large data transformations, complex calculations, parsing large files -- can cause the workflow task to exceed its timeout. The server reschedules the task, the worker replays the full history, hits the same expensive computation, times out again, and the cycle repeats. Replay makes this worse: the computation runs on every replay, compounding the cost.

performing_expensive_computation_in_workflow_code/workflow.go

// BAD: expensive computation in workflow code
func MyWorkflowV1(ctx workflow.Context, data []Record) error {
	// TODO: make this look less contrived -- maybe use bcrypt or some other expensive operation as an example? Or even sha2 (this might be more realistic, eg. to compute a checksum for the identity of a resouce being created in a subsequent activity).
	result := expensiveTransformation(data) // Takes 30 seconds
	return workflow.ExecuteActivity(ctx, StoreResult, result).Get(ctx, nil)
}

performing_expensive_computation_in_workflow_code/workflow.go

// TransformActivity moves the expensive computation into an activity.
func TransformActivity(_ context.Context, data []Record) (Result, error) {
	return expensiveTransformation(data), nil
}

// GOOD: move it to an activity
func MyWorkflowV2(ctx workflow.Context, data []Record) error {
	var result Result
	ctx = workflow.WithActivityOptions(ctx, workflow.ActivityOptions{
		StartToCloseTimeout: 10 * time.Minute,
		HeartbeatTimeout:    30 * time.Second,
	})
	if err := workflow.ExecuteActivity(ctx, TransformActivity, data).Get(ctx, &result); err != nil {
		return err
	}
	return workflow.ExecuteActivity(ctx, StoreResult, result).Get(ctx, nil)
}

Activity results are recorded in workflow history so the computation doesn't repeat on replay. For moderate computation (tens of milliseconds), local activities are a lighter-weight option. Truly trivial operations (comparisons, arithmetic, string formatting) are fine in workflow code as long as it is deterministic.

See also: Exceeding the 10-Second Workflow Task Timeout.