Importing smolagents Takes Excessive Time

[xianminx]

Bug Report: Importing smolagents Takes Excessive Time

Description

Importing the smolagents module takes an unusually long time, which affects usability, especially for non-transformer-related tools. The long import time seems to be caused by the initialization of heavy dependencies like transformers and torch, even when they are not required for certain tools.

Steps to Reproduce

Run the following script to benchmark the import time:

import time

start_time = time.time()
import smolagents
from smolagents import CodeAgent, DuckDuckGoSearchTool, HfApiModel
end_time = time.time() - start_time
print(f"smolagents import time: {end_time:.2f} seconds")
print("smolagents version:", smolagents.__version__)

Observed Behavior

smolagents import time: 9.52 seconds
smolagents version: 1.1.0

Expected Behavior

The smolagents module should import significantly faster, especially when non-transformer-related tools (e.g., DuckDuckGoSearchTool) are being used. The framework should avoid loading unnecessary dependencies during import or should defer their initialization using lazy loading.

Possible Cause

The long import time appears to be due to the initialization of dependencies like:

• transformers
• torch

These dependencies are loaded even when they are not required for certain tools or agents. As an agent framework, smolagents should minimize the overhead of importing the module by:

1. Not loading dependencies unrelated to the tools in use.
2. Implementing lazy loading for modules like transformers and torch.

Environment

• smolagents version: 1.1.0
• Python version: 3.12
• Operating System: macOS/Linux/Windows

Proposed Solution

• Implement lazy loading for dependencies like transformers and torch.
• Optimize the import path to avoid initializing unrelated components when using non-transformer-related tools.

Impact

The high import time reduces the performance and efficiency of the framework, especially in scenarios where only lightweight tools like DuckDuckGoSearchTool are required.

Additional Context

This issue significantly impacts workflows where smolagents is used as a lightweight agent framework for tools that do not require deep learning components.

[aymeric-roucher]

Thank you @xianminx for proposing this! Could you propose a PR and report the reduction in import time?

[xianminx]

Thank you @xianminx for proposing this! Could you propose a PR and report the reduction in import time?

cool, let me try to hack around for the optimization.

[xianminx]

Hi @aymeric-roucher

I took some time and investigated the issue. It seems to be related to PyTorch itself. I ran the test on an old x86 macOS, and importing torch triggers significant disk I/O. The load time depends on disk speed and bandwidth. On a GitHub Codespace machine, the import time is around 1s, which is acceptable for most cases. The speed mainly depends on hardware and cache. Many people encountered similar issue, and you can check out the ongoing discussion here: PyTorch Issue #97106.

Since smolagents relies on transformers, which depends on torch for local and Hugging Face models, refactoring to lazy load these might be a huge complex restructure. I suggest closing the issue and marking it "Won't fix." What do you think?

[aymeric-roucher]

Hi @xianminx.
Thank you for reporting and looking into the issue!
Indeed importing torch was too long. So I removed that dependancy! Since it's needed only for some classes, now the default import won't use it. Upcoming PR to put this.
I've tested the speed increase with this script:

import time

t0 = time.time()
from smolagents import CodeAgent, HfApiModel
t1 = time.time()

print(f"Total time: {t1 - t0:.2f}")

With this removal of torch, the import time falls from 6.52s to 3.8s! So I'd say this mostly solves the issue. 🥳

Stay tuned for the PR!

[aymeric-roucher]

Noting this is closed with the PR merged!