[Question]: How to use async FunctionTool in workflows

[akabeera]

Question Validation

• I have searched both the documentation and discord for an answer.

Question

Hello, I have a function calling agent with several tools and I'm converting everything to async. I've heard using workflows can make things easier since it's async by default.

I followed the guide here https://docs.llamaindex.ai/en/stable/examples/workflow/function_calling_agent/ and added one of my tools that's async. But it does not work. My function does not get hit at all. I created a test with a synchronous function using exact doc string and type annotations with dummy data and it works fine.

I did not find any examples of having an async function tool in workflows and the closest thing I've found with async FunctionTools is manually creating Wait events as shown in https://docs.llamaindex.ai/en/stable/examples/workflow/workflows_cookbook/#waiting-for-one-or-more-events.

Does workflows support async FunctionTools?

This is what I'm doing. I have the exact same workflow implementation as https://docs.llamaindex.ai/en/stable/examples/workflow/function_calling_agent/

And this is how I'm consuming it:

async def fetch_data_async(param: str) -> SomeObject:
   # make http call to fetch data
   data = await api_call(param)
   return SomeObject(data)
   

def fetch_data(param: str) -> str:
   return "some dummy data"
   
tools: List[BaseTool] = [
   FunctionTool.from_defaults(fetch_data_async),
]
    
agent = FuncationCallingAgent(llm=llm, tools=tools, timeout=None, verbose=True)
ret = await agent.run(input=message)
print(f"output: {ret}")

Some observations:
#1. When I print out the ret object, I see this message in Assistant Chat content, content='I am currently fetching data. Please wait a moment.', but the function is never hit.

#2. When I tried with fetch_data tool and the exact same prompt, it's works as expected so I assume the LLM is able to chose the correct FunctionTool given my doc_string and function type annotations and the input prompt.

#3. Trying to await directly on the tool function call but it returns the following warning which seem to suggest that whatever in invoking the FunctionTool is always doing it synchronously

tool_output = await tool(**tool_call.tool_kwargs)

returns:

RuntimeWarning: coroutine 'fetch_data_async' was never awaited
  tool_output = await tool(**tool_call.tool_kwargs)

Is there a way to handle async FunctionTools in workflows?

[dosubot]

To handle async FunctionTools in workflows using the LlamaIndex library, you need to ensure that your async functions are properly awaited within the workflow. Here are some steps and considerations to help you resolve the issue:

1. Ensure Proper Awaiting: Make sure that your async functions are awaited correctly within the workflow. The warning you received indicates that the coroutine is not being awaited, which suggests that the workflow might be treating the function call synchronously.

2. Use FunctionTool for Async Functions: When defining your FunctionTool, ensure that you specify the async function correctly. The FunctionTool class can handle async functions if they are set up properly. Here's an example of how to define an async FunctionTool:

   from llama_index.core.tools.function_tool import FunctionTool
   
   async def fetch_data_async(param: str) -> SomeObject:
       # make http call to fetch data
       data = await api_call(param)
       return SomeObject(data)
   
   tool = FunctionTool(async_fn=fetch_data_async, name="fetch_data_async", description="Fetches data asynchronously")

3. Check Workflow Implementation: Verify that your workflow is designed to handle async execution. This includes ensuring that any method or function that calls the FunctionTool is also async and uses await when calling the tool.

4. Review Event Handling: If your workflow uses events to manage execution flow, ensure that these events are compatible with async operations. You might need to adjust how events are processed to ensure async functions are awaited.

5. Test with Examples: Refer to the test cases in the LlamaIndex codebase that demonstrate the correct usage of async FunctionTools within workflows. For example, the test_function_tool_async function in the llama-index-core/tests/tools/test_base.py file shows how to define and test async functions using FunctionTool ^[1].

By following these steps, you should be able to integrate async FunctionTools into your workflows effectively. If the issue persists, double-check the workflow's structure and ensure that all async functions are properly awaited.

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Other}