[Feature Request] Loss of stack trace information

[mrknmc]

Is your feature request related to a problem? Please describe.

• I would like to include full stack trace information in Sentry - see screenshot below where only ChildWorkflowError has a stack trace and both ActivityError and ApplicationError are missing it.
• I understand the exceptions are passed through the distributed system using the Failure Protobuf message which has a stack_trace field which could be used to populate the Sentry stack trace. This works for getting the stack trace of the ApplicationError in the activity. However, it's impossible to get the stack trace for the ActivityError in the TestChildWorkflow because the failure is just copied from the original failure here:

  sdk-python/temporalio/converter.py

  Lines 826 to 829 in bf747f1

  	# If there is an underlying proto already, just use that
  	if error.failure:
  	failure.CopyFrom(error.failure)
  	return

This is the workflow and activity code I'm using to test this:

@activity.defn(...)
async def test_activity() -> None:
    raise TestException("Test error")


@workflow.defn
class TestChildWorkflow:
    @workflow.run
    async def run(self) -> None:
        # execute_default_activity is our wrapper around workflow.execute_activity
        await execute_default_activity(
            test_activity,
            retry_policy=RetryPolicy(maximum_attempts=1),
        ),


@workflow.defn
class TestParentWorkflow:
    @workflow.run
    async def run(self) -> None:
        # child workflow fails parent workflow
        await workflow.execute_child_workflow(
            TestChildWorkflow.run,
            id=f"test-child-workflow-{int(workflow.time())}",
            task_queue=get_temporal_tasks_queue_name(),
        )

Describe the solution you'd like

• Ability to get the stack trace for every error in the chain. I am happy to contribute this code but I am not sure if there is a strong reason for why it is done this way and whether it can be changed.

[cretz]

Python exceptions don't support easily deserializing tracebacks that come from remote/arbitrary strings (such as the ones we store which can come from any programming language or Python version). Therefore, we make the raw failure available to access this if needed. So for instance, this would get a collection of messages and tracebacks (untested, just typed here in code):

@dataclass
class ExcDetails:
    message: str
    stack_trace: Optional[str]

def populate_chain(exc: Exception, chain: list[ExcDetails]) -> None:
    stack_trace = None
    if isinstance(exc, FailureError) and exc.failure:
        stack_trace = exc.failure.stack_trace
    elif exc.__traceback__:
        stack_trace = "\n".join(traceback.format_tb(exc.__traceback__))
    chain.append(ExcDetails(message=exc.message, stack_trace=stack_trace))
    if exc.__cause__:
        populate_chain(exc.__cause__, chain)

Something like that should work. Or do you have alternative suggestions on how to make arbitrary string stack trace available via error?

[tpetkov-red]

Something like that should work. Or do you have alternative suggestions on how to make arbitrary string stack trace available via error?

Found this issue while trying to solve a similar problem.
Honestly, I'd be happy with the above (or similar) solution plopped into the sdk.
@cretz do you think this can be included into the project?

[cretz]

The SDK already includes stack trace on the errors, it's just via failure.stack_trace instead of __traceback__ because Python has limitations on what can be on __traceback__. This little sample code was just showing how you can call failure.stack_trace in your code, but I don't think we'll include this sample in the SDK proper. You can use a whole new object to collect stack traces if you want like this sample does, though many may just prefer to use the existing Python exceptions and provide a formatter of sorts to whatever is displaying/recording the exception.