Release Adaptive Auto-Harness and PolyBench on Hugging Face

[NielsRogge]

Hi @nuuuh 🤗

I'm Niels and work as part of the community science team at Hugging Face. I discovered your work through Hugging Face's daily papers as your paper "Adaptive Auto-Harness" was featured: https://huggingface.co/papers/2606.01770.

The paper page allows the community to discuss your work and find related artifacts. I saw your comment regarding the code release, but noticed the GitHub link provided (https://github.com/A-EVO-Lab/a-evolve/tree/release/adaptive-auto-harness) currently returns a 404 error.

Once you are ready to open-source the project, would you be interested in hosting the code and especially the task stream benchmarks (like the PolyBench mentioned in your comment) on the Hugging Face Hub?

Hosting on Hugging Face will give your work significantly more visibility and enable better discoverability. For the datasets/benchmarks, it would also allow people to use them directly via the datasets library:

from datasets import load_dataset

dataset = load_dataset("your-hf-org-or-username/polybench")

Besides that, there's the dataset viewer which allows people to explore the task trajectories and prompts directly in the browser. We can also link these artifacts back to your paper page so readers can find them easily.

Let me know if you're interested or need any guidance regarding this!

Kind regards,

Niels

[HanqingLu]

Hi Niels,

Thanks for reaching out, the code is available now at https://github.com/A-EVO-Lab/a-evolve/tree/release/adaptive-auto-harness and also merged into mainline. Please check and let us know if it is accessible now.

Regarding hosting the code and especially the task stream benchmarks (like the PolyBench mentioned in your comment) on the Hugging Face Hub. Could you share more details and instructions?

Best,

Henry, A-EVO-lab

[NielsRogge]

Hi Henry,

Great to hear the code is now accessible on GitHub.

Regarding hosting your benchmarks (like PolyBench) on Hugging Face, here are the steps:

1. Create a Hugging Face account at https://huggingface.co if you do not have one yet
2. Upload your dataset using the huggingface_hub library:

from huggingface_hub import HfApi

api = HfApi()
api.create_repo(repo_id="your-username/polybench", repo_type="dataset")
api.upload_folder(
    folder_path="path/to/your/dataset",
    repo_id="your-username/polybench",
    repo_type="dataset"
)

3. Add a README.md file to your dataset repository that includes the paper URL. This will automatically link the dataset to the paper page. You can mention either:
   • The Hugging Face paper URL: https://huggingface.co/papers/2606.01770
   • The arXiv abstract URL: https://arxiv.org/abs/2606.01770
   • The arXiv PDF URL

See the dataset upload guide here: https://huggingface.co/docs/huggingface_hub/guides/upload
For dataset cards, see: https://huggingface.co/docs/hub/en/datasets-cards

Once uploaded, users will be able to load your benchmark directly:

from datasets import load_dataset
dataset = load_dataset("your-username/polybench")

Let me know if you need any help with this process.

Best,
Niels