Skip to content

evaluation scripts#342

Closed
Nyakult wants to merge 77 commits intoMemTensor:feat/evaluationfrom
Nyakult:eval/0910
Closed

evaluation scripts#342
Nyakult wants to merge 77 commits intoMemTensor:feat/evaluationfrom
Nyakult:eval/0910

Conversation

@Nyakult
Copy link
Copy Markdown
Contributor

@Nyakult Nyakult commented Sep 24, 2025

Description

refine locomo/lme eval scripts, add memos-api, memobase, mirix evaluation

Checklist:

  • I have performed a self-review of my own code | 我已自行检查了自己的代码
  • I have commented my code in hard-to-understand areas | 我已在难以理解的地方对代码进行了注释
  • I have added tests that prove my fix is effective or that my feature works | 我已添加测试以证明我的修复有效或功能正常
  • I have created related documentation issue/PR in MemOS-Docs (if applicable) | 我已在 MemOS-Docs 中创建了相关的文档 issue/PR(如果适用)
  • I have linked the issue to this PR (if applicable) | 我已将 issue 链接到此 PR(如果适用)
  • I have mentioned the person who will review this PR | 我已提及将审查此 PR 的人

Nyakult added 30 commits July 25, 2025 14:37
Nyakult and others added 18 commits September 24, 2025 11:36
- 统一设置项目根目录和脚本目录的路径常量- 简化 sys.path 的添加逻辑,提高代码可读性
- 将 custom_instructions 移动到 prompts.py 文件中,便于维护
- 延迟导入 memobase_client 和 MemOSAPI 等模块,减少启动时间
- 调整参数默认值,降低并发工作线程数量以优化性能- 移除未使用的 uuid 模块和 string_to_uuid 函数
- 格式化代码结构,提升代码整洁度和一致性
- 添加 memory process model 和 response model 的环境变量配置
- 优化客户端初始化逻辑,按需导入依赖模块
- 移除冗余的模块导入和未使用的代码
- 更新 memobase 和 memos 客户端初始化方式
- 调整 longmemeval 和 locomo 的客户端获取逻辑
- 清理 memobase_utils 中的测试代码和无用导入
…h_memory_detail 改为 memory_detail_list

- 更新结果文件路径从 locomo 目录到 lme 目录
- 增加默认 top_k 值从20 到30
- 添加 longmemeval 数据集准备说明到 README
- 更新 longmemeval_s 数据集的下载和保存指引
## Description

<!--
Please include a summary of the changes below;
Fill in the issue number that this PR addresses (if applicable);
Fill in the related MemOS-Docs repository issue or PR link (if
applicable);
Mention the person who will review this PR (if you know who it is);
Replace (summary), (issue), (docs-issue-or-pr-link), and (reviewer) with
the appropriate information.

请在下方填写更改的摘要;
填写此 PR 解决的问题编号(如果适用);
填写相关的 MemOS-Docs 仓库 issue 或 PR 链接(如果适用);
提及将审查此 PR 的人(如果您知道是谁);
替换 (summary)、(issue)、(docs-issue-or-pr-link) 和 (reviewer) 为适当的信息。
-->

Summary: 
Added OpenTelemetry support (MemTensor#298, MemTensor#307).
Introduced OpenTelemetry to enhance observability and monitoring for the
system, helping to trace requests and measure system performance.

Chatbot API and reranker filter, fixed Pydantic bug (MemTensor#294, MemTensor#303).
Added a new chatbot API and incorporated a reranker filter to improve
the response quality. Also, fixed issues related to Pydantic validation
errors.

Updated Nebula to version 5.1.1 (MemTensor#311).
Updated to the latest version of Nebula (5.1.1), addressing any known
bugs and ensuring better performance with the new features.

Parallelized recall and searcher processes (MemTensor#337).
Improved the performance of recall and search operations by
parallelizing them, allowing for faster query handling.

Added API client feature (MemTensor#334).
Introduced a new API client to facilitate easier interactions with the
system's API, likely for external integrations.


Fix: 
Fixed bugs in retriever, and added new auth info for Neo4j DB (MemTensor#313).
Addressed bugs in the retriever and improved authentication handling for
the Neo4j database, ensuring more reliable connections and data
retrieval.

Docs Issue/PR: (docs-issue-or-pr-link)

Reviewer: @(reviewer)

## Checklist:

- [x] I have performed a self-review of my own code | 我已自行检查了自己的代码
- [x] I have commented my code in hard-to-understand areas |
我已在难以理解的地方对代码进行了注释
- [ ] I have added tests that prove my fix is effective or that my
feature works | 我已添加测试以证明我的修复有效或功能正常
- [ ] I have created related documentation issue/PR in
[MemOS-Docs](https://github.com/MemTensor/MemOS-Docs) (if applicable) |
我已在 [MemOS-Docs](https://github.com/MemTensor/MemOS-Docs) 中创建了相关的文档
issue/PR(如果适用)
- [ ] I have linked the issue to this PR (if applicable) | 我已将 issue
链接到此 PR(如果适用)
- [ ] I have mentioned the person who will review this PR | 我已提及将审查此 PR
的人
Co-authored-by: harvey_xiang <harvey_xiang22@163.com>
* docker start

* docker start

* update config

* 代码检测

* test_start_api

* test_start_api

* test_start_api

* fix docker start

* update start_api

* update start_api

* update start_api

* 代码检测

---------

Co-authored-by: yjy <yuanjinyun@defencedriver.com>
@Nyakult Nyakult closed this Sep 25, 2025
@Nyakult Nyakult reopened this Sep 25, 2025
@Nyakult Nyakult closed this Sep 25, 2025
@Nyakult Nyakult deleted the eval/0910 branch September 25, 2025 02:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants