Question about ALFWorld adaptation for baselines

[FredJiang0324]

Hi, thanks for releasing this impressive work.

I have a question about the Embodied Interactive Tasks(ALFWorld) experiments for baselines such as A-MEM and MemoryOS. Since these methods were originally designed more for dialogue / QA settings, I was wondering how you adapted them to ALFWorld in practice.
In particular:
How did you build their memory bank from the ALFWorld training trajectories? What did you use as the retrieval query during evaluation (objective only, or also observation/history)?
Was the adaptation mainly prompt-level, or did you also modify some code structure / memory pipeline for these baselines?

I’m trying to understand whether the ALFWorld setup for these baselines was mostly a unified prompting interface, or whether some method-specific implementation changes were needed.

Thanks again, the work is very impressive.

[FredJiang0324]

Also do you plan to release any additional code for how baselines were tested on ALFWorld and the evolved MemSkill's skills?

[ViktorAxelsen]

Thank you for your interest in our work and for the thoughtful questions.

You are absolutely right that methods such as A-MEM and MemoryOS were originally designed more for dialogue and QA-style settings. Since we also evaluate on LoCoMo and LongMemEval, we included these baselines in ALFWorld as part of a unified comparison setup for presentation consistency.

For the adaptation, we followed the original methods as closely as possible. We did not modify their core code structure or redesign their memory pipeline. Instead, we only made the minimal adaptations necessary to ensure consistent experimental settings across benchmarks. In this sense, the adaptation was mainly at the prompt level rather than through major method-specific implementation changes.

For the ALFWorld evaluation pipeline, it is almost the same as src/alfworld_env_runner.py in this repo, except that the injected memory is constructed by each baseline’s own memory summarization mechanism. In other words, the surrounding agent-environment interaction pipeline remains largely the same, while the main difference lies in how the memory content is produced and then provided back to the agent.

As for the evolved MemSkill skills, they have already been released both in the paper appendix and in the skills directory of this repository.

Regarding additional code release, we do plan to release more ALFWorld results with other baselines such as AWM and Expel as well. We agree that this would make the comparison more complete and helpful for the community.

Thanks again for the careful reading and for raising these questions.