enable ci llvm debug by FangRui0 · Pull Request #684 · hw-native-sys/PTOAS

FangRui0 · 2026-05-19T06:25:10Z

这个 commit 包含两部分:
CI / release 的 LLVM 构建分流改动：CI 使用 assertion 版本，release 继续使用 release 版本
assert 版本下暴露出来的 ptoas/ptobc 问题修复

gemini-code-assist · 2026-05-19T06:25:15Z

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

reedhecre · 2026-05-19T06:25:28Z

Codex Review

该评论由 review 机器人自动更新。

PR: enable ci llvm debug #684 enable ci llvm debug
Author: FangRui0
Base/Head: main / codex/ci-llvm-debug
Head SHA: 04da79ef65db
Trigger: PR 有新提交
Generated At: 2026-05-20T10:07:28Z
Previous Head SHA: 26c26bdf227f
Status: completed

Summary

PR #684 引入了 1 个明确的兼容性回归。

Findings

P3 `docker/test_ptoas_cli.sh` 现在强制要求 `PTO_BUILD_DIR`，破坏了原有调用契约 docker/test_ptoas_cli.sh:23

这个脚本在 origin/main 上只要求 PTO_SOURCE_DIR、LLVM_BUILD_DIR 和 PTO_INSTALL_DIR，并且会直接从标准的 in-tree 路径 ${PTO_SOURCE_DIR}/build/tools/ptoas 找到二进制。此 PR 把 PTO_BUILD_DIR 加进了必填环境变量校验里，所以任何仍按旧契约调用该脚本的手工流程或外部自动化，都会在脚本开头直接失败，即使 ptoas 仍然位于原来的默认位置。相比之下，这个 PR 里另外几个辅助脚本都把 PTO_BUILD_DIR 设计成可选项，因此这里缺少向后兼容 fallback 是一个明确的兼容性回归。

Signed-off-by: FangRui <[email protected]>

FangRui0 · 2026-05-19T11:14:46Z

/run a3

reedhecre · 2026-05-19T11:15:04Z

已接收 /run a3，A3 板测器会处理这条请求。

进度页：http://154.9.227.233/ptoas-board-dashboard/#board-a3
当前状态：板测器空闲，这条请求会在本轮轮询启动。

页面会自动刷新，可以直接看当前阶段、排队情况和最近结果。

reedhecre · 2026-05-19T11:41:53Z

A3 板测失败

触发方式：manual
源码提交：a18de1a33fd1
结果汇总：OK 167 / FAIL 45 / SKIP 2
日志：/home/zhongxuan/ptoas-board-monitor/runtime/logs/20260519_191506_manual_pr684.log
手动指令：/run a3
触发人：FangRui0
触发评论：enable ci llvm debug #684 (comment)
失败阶段：board-validation / exit=1

失败用例

qwen3_decode_incore_11 (run, exit=1)
qwen3_decode_incore_9 (run, exit=1)
qwen3_decode_incore_6 (run, exit=1)
qwen3_decode_incore_2 (run, exit=1)
qwen3_decode_incore_13 (run, exit=1)
qwen3_decode_incore_16 (run, exit=1)
qwen3_decode_incore_7 (run, exit=1)
qwen3_decode_incore_5 (run, exit=1)
qwen3_decode_incore_15 (run, exit=1)
qwen3_decode_incore_12 (run, exit=1)
sels (run, exit=1)
movfp (run, exit=1)
ands (run, exit=1)
divs (run, exit=1)
cvt_f32_f32 (run, exit=1)
cvt_f32_f16 (run, exit=1)
tcvt (run, exit=1)
concat (run, exit=1)
test_inject_sync_loop (run, exit=1)
tmatmulk_autosync (run, exit=1)
test_barrier_sync (run, exit=1)
test_inject_sync_intra_pipe_barrier (run, exit=1)
test_set_wait_unified_api (run, exit=1)
test_mem_inject_sync_basic (run, exit=1)
test_intercore_sync_a3 (run, exit=1)
syncHigh (run, exit=1)
matmul (run, exit=1)
add_double_dynamic (run, exit=1)
nested_loop_confliect (run, exit=1)
rar_optimization_test (run, exit=1)
test_intercore_sync_a3_modes (run, exit=1)
test_dynamic_valid_shape (run, exit=1)
test_inject_sync_if_else (run, exit=1)
test_auto_sync_tail_hint (run, exit=1)
sync (run, exit=1)
test_intercore_sync_a3_dyn (run, exit=1)
test_inject_sync_loop_nest (run, exit=1)
compensation_test (run, exit=1)
test_inject_sync_if (run, exit=1)
test_inject_sync_two_event_id (run, exit=1)
rem (run, exit=1)
partmin (run, exit=1)
colexpandadd (run, exit=1)
sqrt (run, exit=1)
bf16_tile (run, exit=1)

reedhecre · 2026-05-19T11:41:54Z

A3 板测失败详情：PR #684

qwen3_decode_incore_11

stage=run info=exit=1

[ERROR] aclrtSetDevice(deviceId) failed: 507033 (/home/zhongxuan/ptoas-board-monitor/runtime/runs/20260519_191506_manual_pr684/npu_validation/Qwen3DecodeA3/qwen3_decode_incore_11/main.cpp:79)
[ERROR] RecentErrMsg: [PID: 2659353] 2026-05-19-19:38:31.274.270 Invalid_Argument(EE1001): The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
        Solution: 1.Check the input parameter range of the function. 2.Check the function invocation relationship.
        TraceBack (most recent call last):
        TsdOpen failed. devId=11, tdt error=1[FUNC:PrintfTsdError][FILE:runtime.cc][LINE:2638]
        Check param failed, dev can not be NULL![FUNC:DeviceRetain][FILE:runtime.cc][LINE:3558]
        Check param failed, dev can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3175]
        Check param failed, ctx can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3206]
        Check param failed, context can not be null.[FUNC:SetDevice][FILE:api_impl.cc][LINE:3319]
        rtSetDevice execution failed, reason=device retain error[FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:65]
        open device 11 failed, runtime result = 507033.[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:148]
        ctx is NULL![FUNC:GetDevErrMsg][FILE:api_impl.cc][LINE:6108]
        The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
[2026-05-19 19:38:31] ERROR: testcase failed (exit 1): qwen3_decode_incore_11

qwen3_decode_incore_9

stage=run info=exit=1

[ERROR] aclrtSetDevice(deviceId) failed: 507033 (/home/zhongxuan/ptoas-board-monitor/runtime/runs/20260519_191506_manual_pr684/npu_validation/Qwen3DecodeA3/qwen3_decode_incore_9/main.cpp:81)
[ERROR] RecentErrMsg: [PID: 2661404] 2026-05-19-19:38:35.056.945 Invalid_Argument(EE1001): The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
        Solution: 1.Check the input parameter range of the function. 2.Check the function invocation relationship.
        TraceBack (most recent call last):
        TsdOpen failed. devId=11, tdt error=1[FUNC:PrintfTsdError][FILE:runtime.cc][LINE:2638]
        Check param failed, dev can not be NULL![FUNC:DeviceRetain][FILE:runtime.cc][LINE:3558]
        Check param failed, dev can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3175]
        Check param failed, ctx can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3206]
        Check param failed, context can not be null.[FUNC:SetDevice][FILE:api_impl.cc][LINE:3319]
        rtSetDevice execution failed, reason=device retain error[FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:65]
        open device 11 failed, runtime result = 507033.[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:148]
        ctx is NULL![FUNC:GetDevErrMsg][FILE:api_impl.cc][LINE:6108]
        The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
[2026-05-19 19:38:35] ERROR: testcase failed (exit 1): qwen3_decode_incore_9

qwen3_decode_incore_6

stage=run info=exit=1

[ERROR] aclrtSetDevice(deviceId) failed: 507033 (/home/zhongxuan/ptoas-board-monitor/runtime/runs/20260519_191506_manual_pr684/npu_validation/Qwen3DecodeA3/qwen3_decode_incore_6/main.cpp:86)
[ERROR] RecentErrMsg: [PID: 2662959] 2026-05-19-19:38:38.599.244 Invalid_Argument(EE1001): The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
        Solution: 1.Check the input parameter range of the function. 2.Check the function invocation relationship.
        TraceBack (most recent call last):
        TsdOpen failed. devId=11, tdt error=1[FUNC:PrintfTsdError][FILE:runtime.cc][LINE:2638]
        Check param failed, dev can not be NULL![FUNC:DeviceRetain][FILE:runtime.cc][LINE:3558]
        Check param failed, dev can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3175]
        Check param failed, ctx can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3206]
        Check param failed, context can not be null.[FUNC:SetDevice][FILE:api_impl.cc][LINE:3319]
        rtSetDevice execution failed, reason=device retain error[FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:65]
        open device 11 failed, runtime result = 507033.[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:148]
        ctx is NULL![FUNC:GetDevErrMsg][FILE:api_impl.cc][LINE:6108]
        The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
[2026-05-19 19:38:39] ERROR: testcase failed (exit 1): qwen3_decode_incore_6

qwen3_decode_incore_2

stage=run info=exit=1

[ERROR] aclrtSetDevice(deviceId) failed: 507033 (/home/zhongxuan/ptoas-board-monitor/runtime/runs/20260519_191506_manual_pr684/npu_validation/Qwen3DecodeA3/qwen3_decode_incore_2/main.cpp:89)
[ERROR] RecentErrMsg: [PID: 2664576] 2026-05-19-19:38:42.568.508 Invalid_Argument(EE1001): The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
        Solution: 1.Check the input parameter range of the function. 2.Check the function invocation relationship.
        TraceBack (most recent call last):
        TsdOpen failed. devId=11, tdt error=1[FUNC:PrintfTsdError][FILE:runtime.cc][LINE:2638]
        Check param failed, dev can not be NULL![FUNC:DeviceRetain][FILE:runtime.cc][LINE:3558]
        Check param failed, dev can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3175]
        Check param failed, ctx can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3206]
        Check param failed, context can not be null.[FUNC:SetDevice][FILE:api_impl.cc][LINE:3319]
        rtSetDevice execution failed, reason=device retain error[FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:65]
        open device 11 failed, runtime result = 507033.[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:148]
        ctx is NULL![FUNC:GetDevErrMsg][FILE:api_impl.cc][LINE:6108]
        The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
[2026-05-19 19:38:43] ERROR: testcase failed (exit 1): qwen3_decode_incore_2

qwen3_decode_incore_13

stage=run info=exit=1

[ERROR] aclrtSetDevice(deviceId) failed: 507033 (/home/zhongxuan/ptoas-board-monitor/runtime/runs/20260519_191506_manual_pr684/npu_validation/Qwen3DecodeA3/qwen3_decode_incore_13/main.cpp:80)
[ERROR] RecentErrMsg: [PID: 2666953] 2026-05-19-19:38:48.309.953 Invalid_Argument(EE1001): The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
        Solution: 1.Check the input parameter range of the function. 2.Check the function invocation relationship.
        TraceBack (most recent call last):
        TsdOpen failed. devId=11, tdt error=1[FUNC:PrintfTsdError][FILE:runtime.cc][LINE:2638]
        Check param failed, dev can not be NULL![FUNC:DeviceRetain][FILE:runtime.cc][LINE:3558]
        Check param failed, dev can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3175]
        Check param failed, ctx can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3206]
        Check param failed, context can not be null.[FUNC:SetDevice][FILE:api_impl.cc][LINE:3319]
        rtSetDevice execution failed, reason=device retain error[FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:65]
        open device 11 failed, runtime result = 507033.[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:148]
        ctx is NULL![FUNC:GetDevErrMsg][FILE:api_impl.cc][LINE:6108]
        The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
[2026-05-19 19:38:48] ERROR: testcase failed (exit 1): qwen3_decode_incore_13

qwen3_decode_incore_16

stage=run info=exit=1

[ERROR] aclrtSetDevice(deviceId) failed: 507033 (/home/zhongxuan/ptoas-board-monitor/runtime/runs/20260519_191506_manual_pr684/npu_validation/Qwen3DecodeA3/qwen3_decode_incore_16/main.cpp:81)
[ERROR] RecentErrMsg: [PID: 2668246] 2026-05-19-19:38:51.429.788 Invalid_Argument(EE1001): The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
        Solution: 1.Check the input parameter range of the function. 2.Check the function invocation relationship.
        TraceBack (most recent call last):
        TsdOpen failed. devId=11, tdt error=1[FUNC:PrintfTsdError][FILE:runtime.cc][LINE:2638]
        Check param failed, dev can not be NULL![FUNC:DeviceRetain][FILE:runtime.cc][LINE:3558]
        Check param failed, dev can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3175]
        Check param failed, ctx can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3206]
        Check param failed, context can not be null.[FUNC:SetDevice][FILE:api_impl.cc][LINE:3319]
        rtSetDevice execution failed, reason=device retain error[FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:65]
        open device 11 failed, runtime result = 507033.[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:148]
        ctx is NULL![FUNC:GetDevErrMsg][FILE:api_impl.cc][LINE:6108]
        The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
[2026-05-19 19:38:52] ERROR: testcase failed (exit 1): qwen3_decode_incore_16

qwen3_decode_incore_7

stage=run info=exit=1

[ERROR] aclrtSetDevice(deviceId) failed: 507033 (/home/zhongxuan/ptoas-board-monitor/runtime/runs/20260519_191506_manual_pr684/npu_validation/Qwen3DecodeA3/qwen3_decode_incore_7/main.cpp:83)
[ERROR] RecentErrMsg: [PID: 2669327] 2026-05-19-19:38:55.354.072 Invalid_Argument(EE1001): The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
        Solution: 1.Check the input parameter range of the function. 2.Check the function invocation relationship.
        TraceBack (most recent call last):
        TsdOpen failed. devId=11, tdt error=1[FUNC:PrintfTsdError][FILE:runtime.cc][LINE:2638]
        Check param failed, dev can not be NULL![FUNC:DeviceRetain][FILE:runtime.cc][LINE:3558]
        Check param failed, dev can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3175]
        Check param failed, ctx can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3206]
        Check param failed, context can not be null.[FUNC:SetDevice][FILE:api_impl.cc][LINE:3319]
        rtSetDevice execution failed, reason=device retain error[FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:65]
        open device 11 failed, runtime result = 507033.[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:148]
        ctx is NULL![FUNC:GetDevErrMsg][FILE:api_impl.cc][LINE:6108]
        The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
[2026-05-19 19:38:55] ERROR: testcase failed (exit 1): qwen3_decode_incore_7

qwen3_decode_incore_5

stage=run info=exit=1

[ERROR] aclrtSetDevice(deviceId) failed: 507033 (/home/zhongxuan/ptoas-board-monitor/runtime/runs/20260519_191506_manual_pr684/npu_validation/Qwen3DecodeA3/qwen3_decode_incore_5/main.cpp:83)
[ERROR] RecentErrMsg: [PID: 2670621] 2026-05-19-19:38:58.817.029 Invalid_Argument(EE1001): The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
        Solution: 1.Check the input parameter range of the function. 2.Check the function invocation relationship.
        TraceBack (most recent call last):
        TsdOpen failed. devId=11, tdt error=1[FUNC:PrintfTsdError][FILE:runtime.cc][LINE:2638]
        Check param failed, dev can not be NULL![FUNC:DeviceRetain][FILE:runtime.cc][LINE:3558]
        Check param failed, dev can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3175]
        Check param failed, ctx can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3206]
        Check param failed, context can not be null.[FUNC:SetDevice][FILE:api_impl.cc][LINE:3319]
        rtSetDevice execution failed, reason=device retain error[FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:65]
        open device 11 failed, runtime result = 507033.[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:148]
        ctx is NULL![FUNC:GetDevErrMsg][FILE:api_impl.cc][LINE:6108]
        The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
[2026-05-19 19:38:59] ERROR: testcase failed (exit 1): qwen3_decode_incore_5

qwen3_decode_incore_15

stage=run info=exit=1

[ERROR] aclrtSetDevice(deviceId) failed: 507033 (/home/zhongxuan/ptoas-board-monitor/runtime/runs/20260519_191506_manual_pr684/npu_validation/Qwen3DecodeA3/qwen3_decode_incore_15/main.cpp:80)
[ERROR] RecentErrMsg: [PID: 2673004] 2026-05-19-19:39:04.345.551 Invalid_Argument(EE1001): The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
        Solution: 1.Check the input parameter range of the function. 2.Check the function invocation relationship.
        TraceBack (most recent call last):
        TsdOpen failed. devId=11, tdt error=1[FUNC:PrintfTsdError][FILE:runtime.cc][LINE:2638]
        Check param failed, dev can not be NULL![FUNC:DeviceRetain][FILE:runtime.cc][LINE:3558]
        Check param failed, dev can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3175]
        Check param failed, ctx can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3206]
        Check param failed, context can not be null.[FUNC:SetDevice][FILE:api_impl.cc][LINE:3319]
        rtSetDevice execution failed, reason=device retain error[FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:65]
        open device 11 failed, runtime result = 507033.[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:148]
        ctx is NULL![FUNC:GetDevErrMsg][FILE:api_impl.cc][LINE:6108]
        The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
[2026-05-19 19:39:04] ERROR: testcase failed (exit 1): qwen3_decode_incore_15

qwen3_decode_incore_12

stage=run info=exit=1

[ERROR] aclrtSetDevice(deviceId) failed: 507033 (/home/zhongxuan/ptoas-board-monitor/runtime/runs/20260519_191506_manual_pr684/npu_validation/Qwen3DecodeA3/qwen3_decode_incore_12/main.cpp:80)
[ERROR] RecentErrMsg: [PID: 2674720] 2026-05-19-19:39:09.606.274 Invalid_Argument(EE1001): The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
        Solution: 1.Check the input parameter range of the function. 2.Check the function invocation relationship.
        TraceBack (most recent call last):
        TsdOpen failed. devId=11, tdt error=1[FUNC:PrintfTsdError][FILE:runtime.cc][LINE:2638]
        Check param failed, dev can not be NULL![FUNC:DeviceRetain][FILE:runtime.cc][LINE:3558]
        Check param failed, dev can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3175]
        Check param failed, ctx can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3206]
        Check param failed, context can not be null.[FUNC:SetDevice][FILE:api_impl.cc][LINE:3319]
        rtSetDevice execution failed, reason=device retain error[FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:65]
        open device 11 failed, runtime result = 507033.[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:148]
        ctx is NULL![FUNC:GetDevErrMsg][FILE:api_impl.cc][LINE:6108]
        The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
[2026-05-19 19:39:10] ERROR: testcase failed (exit 1): qwen3_decode_incore_12

sels

stage=run info=exit=1

[ERROR] aclrtSetDevice(deviceId) failed: 507033 (/home/zhongxuan/ptoas-board-monitor/runtime/runs/20260519_191506_manual_pr684/npu_validation/Sels/sels/main.cpp:79)
[ERROR] RecentErrMsg: [PID: 2675795] 2026-05-19-19:39:13.299.710 Invalid_Argument(EE1001): The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
        Solution: 1.Check the input parameter range of the function. 2.Check the function invocation relationship.
        TraceBack (most recent call last):
        TsdOpen failed. devId=11, tdt error=1[FUNC:PrintfTsdError][FILE:runtime.cc][LINE:2638]
        Check param failed, dev can not be NULL![FUNC:DeviceRetain][FILE:runtime.cc][LINE:3558]
        Check param failed, dev can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3175]
        Check param failed, ctx can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3206]
        Check param failed, context can not be null.[FUNC:SetDevice][FILE:api_impl.cc][LINE:3319]
        rtSetDevice execution failed, reason=device retain error[FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:65]
        open device 11 failed, runtime result = 507033.[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:148]
        ctx is NULL![FUNC:GetDevErrMsg][FILE:api_impl.cc][LINE:6108]
        The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
[2026-05-19 19:39:13] ERROR: testcase failed (exit 1): sels

movfp

stage=run info=exit=1

[ERROR] aclrtSetDevice(deviceId) failed: 507033 (/home/zhongxuan/ptoas-board-monitor/runtime/runs/20260519_191506_manual_pr684/npu_validation/Movfp/movfp/main.cpp:83)
[ERROR] RecentErrMsg: [PID: 2677148] 2026-05-19-19:39:16.819.559 Invalid_Argument(EE1001): The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
        Solution: 1.Check the input parameter range of the function. 2.Check the function invocation relationship.
        TraceBack (most recent call last):
        TsdOpen failed. devId=11, tdt error=1[FUNC:PrintfTsdError][FILE:runtime.cc][LINE:2638]
        Check param failed, dev can not be NULL![FUNC:DeviceRetain][FILE:runtime.cc][LINE:3558]
        Check param failed, dev can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3175]
        Check param failed, ctx can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3206]
        Check param failed, context can not be null.[FUNC:SetDevice][FILE:api_impl.cc][LINE:3319]
        rtSetDevice execution failed, reason=device retain error[FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:65]
        open device 11 failed, runtime result = 507033.[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:148]
        ctx is NULL![FUNC:GetDevErrMsg][FILE:api_impl.cc][LINE:6108]
        The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
[2026-05-19 19:39:17] ERROR: testcase failed (exit 1): movfp

ands

stage=run info=exit=1

[ERROR] aclrtSetDevice(deviceId) failed: 507033 (/home/zhongxuan/ptoas-board-monitor/runtime/runs/20260519_191506_manual_pr684/npu_validation/Ands/ands/main.cpp:75)
[ERROR] RecentErrMsg: [PID: 2678367] 2026-05-19-19:39:20.258.338 Invalid_Argument(EE1001): The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
        Solution: 1.Check the input parameter range of the function. 2.Check the function invocation relationship.
        TraceBack (most recent call last):
        TsdOpen failed. devId=11, tdt error=1[FUNC:PrintfTsdError][FILE:runtime.cc][LINE:2638]
        Check param failed, dev can not be NULL![FUNC:DeviceRetain][FILE:runtime.cc][LINE:3558]
        Check param failed, dev can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3175]
        Check param failed, ctx can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3206]
        Check param failed, context can not be null.[FUNC:SetDevice][FILE:api_impl.cc][LINE:3319]
        rtSetDevice execution failed, reason=device retain error[FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:65]
        open device 11 failed, runtime result = 507033.[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:148]
        ctx is NULL![FUNC:GetDevErrMsg][FILE:api_impl.cc][LINE:6108]
        The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
[2026-05-19 19:39:20] ERROR: testcase failed (exit 1): ands

divs

stage=run info=exit=1

[ERROR] aclrtSetDevice(deviceId) failed: 507033 (/home/zhongxuan/ptoas-board-monitor/runtime/runs/20260519_191506_manual_pr684/npu_validation/Divs/divs/main.cpp:75)
[ERROR] RecentErrMsg: [PID: 2679876] 2026-05-19-19:39:23.374.755 Invalid_Argument(EE1001): The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
        Solution: 1.Check the input parameter range of the function. 2.Check the function invocation relationship.
        TraceBack (most recent call last):
        TsdOpen failed. devId=11, tdt error=1[FUNC:PrintfTsdError][FILE:runtime.cc][LINE:2638]
        Check param failed, dev can not be NULL![FUNC:DeviceRetain][FILE:runtime.cc][LINE:3558]
        Check param failed, dev can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3175]
        Check param failed, ctx can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3206]
        Check param failed, context can not be null.[FUNC:SetDevice][FILE:api_impl.cc][LINE:3319]
        rtSetDevice execution failed, reason=device retain error[FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:65]
        open device 11 failed, runtime result = 507033.[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:148]
        ctx is NULL![FUNC:GetDevErrMsg][FILE:api_impl.cc][LINE:6108]
        The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
[2026-05-19 19:39:23] ERROR: testcase failed (exit 1): divs

cvt_f32_f32

stage=run info=exit=1

[ERROR] aclrtSetDevice(deviceId) failed: 507033 (/home/zhongxuan/ptoas-board-monitor/runtime/runs/20260519_191506_manual_pr684/npu_validation/Cvt/cvt_f32_f32/main.cpp:75)
[ERROR] RecentErrMsg: [PID: 2681375] 2026-05-19-19:39:26.526.597 Invalid_Argument(EE1001): The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
        Solution: 1.Check the input parameter range of the function. 2.Check the function invocation relationship.
        TraceBack (most recent call last):
        TsdOpen failed. devId=11, tdt error=1[FUNC:PrintfTsdError][FILE:runtime.cc][LINE:2638]
        Check param failed, dev can not be NULL![FUNC:DeviceRetain][FILE:runtime.cc][LINE:3558]
        Check param failed, dev can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3175]
        Check param failed, ctx can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3206]
        Check param failed, context can not be null.[FUNC:SetDevice][FILE:api_impl.cc][LINE:3319]
        rtSetDevice execution failed, reason=device retain error[FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:65]
        open device 11 failed, runtime result = 507033.[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:148]
        ctx is NULL![FUNC:GetDevErrMsg][FILE:api_impl.cc][LINE:6108]
        The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
[2026-05-19 19:39:27] ERROR: testcase failed (exit 1): cvt_f32_f32

cvt_f32_f16

stage=run info=exit=1

[ERROR] aclrtSetDevice(deviceId) failed: 507033 (/home/zhongxuan/ptoas-board-monitor/runtime/runs/20260519_191506_manual_pr684/npu_validation/Cvt/cvt_f32_f16/main.cpp:75)
[ERROR] RecentErrMsg: [PID: 2683016] 2026-05-19-19:39:30.474.844 Invalid_Argument(EE1001): The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
        Solution: 1.Check the input parameter range of the function. 2.Check the function invocation relationship.
        TraceBack (most recent call last):
        TsdOpen failed. devId=11, tdt error=1[FUNC:PrintfTsdError][FILE:runtime.cc][LINE:2638]
        Check param failed, dev can not be NULL![FUNC:DeviceRetain][FILE:runtime.cc][LINE:3558]
        Check param failed, dev can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3175]
        Check param failed, ctx can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3206]
        Check param failed, context can not be null.[FUNC:SetDevice][FILE:api_impl.cc][LINE:3319]
        rtSetDevice execution failed, reason=device retain error[FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:65]
        open device 11 failed, runtime result = 507033.[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:148]
        ctx is NULL![FUNC:GetDevErrMsg][FILE:api_impl.cc][LINE:6108]
        The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
[2026-05-19 19:39:31] ERROR: testcase failed (exit 1): cvt_f32_f16

tcvt

stage=run info=exit=1

[ERROR] aclrtSetDevice(deviceId) failed: 507033 (/home/zhongxuan/ptoas-board-monitor/runtime/runs/20260519_191506_manual_pr684/npu_validation/Cvt/tcvt/main.cpp:75)
[ERROR] RecentErrMsg: [PID: 2684258] 2026-05-19-19:39:34.350.987 Invalid_Argument(EE1001): The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
        Solution: 1.Check the input parameter range of the function. 2.Check the function invocation relationship.
        TraceBack (most recent call last):
        TsdOpen failed. devId=11, tdt error=1[FUNC:PrintfTsdError][FILE:runtime.cc][LINE:2638]
        Check param failed, dev can not be NULL![FUNC:DeviceRetain][FILE:runtime.cc][LINE:3558]
        Check param failed, dev can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3175]
        Check param failed, ctx can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3206]
        Check param failed, context can not be null.[FUNC:SetDevice][FILE:api_impl.cc][LINE:3319]
        rtSetDevice execution failed, reason=device retain error[FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:65]
        open device 11 failed, runtime result = 507033.[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:148]
        ctx is NULL![FUNC:GetDevErrMsg][FILE:api_impl.cc][LINE:6108]
        The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
[2026-05-19 19:39:34] ERROR: testcase failed (exit 1): tcvt

concat

stage=run info=exit=1

[ERROR] aclrtSetDevice(deviceId) failed: 507033 (/home/zhongxuan/ptoas-board-monitor/runtime/runs/20260519_191506_manual_pr684/npu_validation/Concat/concat/main.cpp:79)
[ERROR] RecentErrMsg: [PID: 2685120] 2026-05-19-19:39:37.400.380 Invalid_Argument(EE1001): The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
        Solution: 1.Check the input parameter range of the function. 2.Check the function invocation relationship.
        TraceBack (most recent call last):
        TsdOpen failed. devId=11, tdt error=1[FUNC:PrintfTsdError][FILE:runtime.cc][LINE:2638]
        Check param failed, dev can not be NULL![FUNC:DeviceRetain][FILE:runtime.cc][LINE:3558]
        Check param failed, dev can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3175]
        Check param failed, ctx can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3206]
        Check param failed, context can not be null.[FUNC:SetDevice][FILE:api_impl.cc][LINE:3319]
        rtSetDevice execution failed, reason=device retain error[FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:65]
        open device 11 failed, runtime result = 507033.[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:148]
        ctx is NULL![FUNC:GetDevErrMsg][FILE:api_impl.cc][LINE:6108]
        The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
[2026-05-19 19:39:37] ERROR: testcase failed (exit 1): concat

test_inject_sync_loop

stage=run info=exit=1

[ERROR] aclrtSetDevice(deviceId) failed: 507033 (/home/zhongxuan/ptoas-board-monitor/runtime/runs/20260519_191506_manual_pr684/npu_validation/Sync/test_inject_sync_loop/main.cpp:75)
[ERROR] RecentErrMsg: [PID: 2685963] 2026-05-19-19:39:41.719.393 Invalid_Argument(EE1001): The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
        Solution: 1.Check the input parameter range of the function. 2.Check the function invocation relationship.
        TraceBack (most recent call last):
        TsdOpen failed. devId=11, tdt error=1[FUNC:PrintfTsdError][FILE:runtime.cc][LINE:2638]
        Check param failed, dev can not be NULL![FUNC:DeviceRetain][FILE:runtime.cc][LINE:3558]
        Check param failed, dev can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3175]
        Check param failed, ctx can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3206]
        Check param failed, context can not be null.[FUNC:SetDevice][FILE:api_impl.cc][LINE:3319]
        rtSetDevice execution failed, reason=device retain error[FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:65]
        open device 11 failed, runtime result = 507033.[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:148]
        ctx is NULL![FUNC:GetDevErrMsg][FILE:api_impl.cc][LINE:6108]
        The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
[2026-05-19 19:39:42] ERROR: testcase failed (exit 1): test_inject_sync_loop

tmatmulk_autosync

stage=run info=exit=1

[ERROR] aclrtSetDevice(deviceId) failed: 507033 (/home/zhongxuan/ptoas-board-monitor/runtime/runs/20260519_191506_manual_pr684/npu_validation/Sync/tmatmulk_autosync/main.cpp:84)
[ERROR] RecentErrMsg: [PID: 2687828] 2026-05-19-19:39:46.745.468 Invalid_Argument(EE1001): The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
        Solution: 1.Check the input parameter range of the function. 2.Check the function invocation relationship.
        TraceBack (most recent call last):
        TsdOpen failed. devId=11, tdt error=1[FUNC:PrintfTsdError][FILE:runtime.cc][LINE:2638]
        Check param failed, dev can not be NULL![FUNC:DeviceRetain][FILE:runtime.cc][LINE:3558]
        Check param failed, dev can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3175]
        Check param failed, ctx can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3206]
        Check param failed, context can not be null.[FUNC:SetDevice][FILE:api_impl.cc][LINE:3319]
        rtSetDevice execution failed, reason=device retain error[FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:65]
        open device 11 failed, runtime result = 507033.[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:148]
        ctx is NULL![FUNC:GetDevErrMsg][FILE:api_impl.cc][LINE:6108]
        The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
[2026-05-19 19:39:47] ERROR: testcase failed (exit 1): tmatmulk_autosync

test_barrier_sync

stage=run info=exit=1

[ERROR] aclrtSetDevice(deviceId) failed: 507033 (/home/zhongxuan/ptoas-board-monitor/runtime/runs/20260519_191506_manual_pr684/npu_validation/Sync/test_barrier_sync/main.cpp:68)
[ERROR] RecentErrMsg: [PID: 2689864] 2026-05-19-19:39:51.564.096 Invalid_Argument(EE1001): The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
        Solution: 1.Check the input parameter range of the function. 2.Check the function invocation relationship.
        TraceBack (most recent call last):
        TsdOpen failed. devId=11, tdt error=1[FUNC:PrintfTsdError][FILE:runtime.cc][LINE:2638]
        Check param failed, dev can not be NULL![FUNC:DeviceRetain][FILE:runtime.cc][LINE:3558]
        Check param failed, dev can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3175]
        Check param failed, ctx can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3206]
        Check param failed, context can not be null.[FUNC:SetDevice][FILE:api_impl.cc][LINE:3319]
        rtSetDevice execution failed, reason=device retain error[FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:65]
        open device 11 failed, runtime result = 507033.[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:148]
        ctx is NULL![FUNC:GetDevErrMsg][FILE:api_impl.cc][LINE:6108]
        The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
[2026-05-19 19:39:52] ERROR: testcase failed (exit 1): test_barrier_sync

test_inject_sync_intra_pipe_barrier

stage=run info=exit=1

[ERROR] aclrtSetDevice(deviceId) failed: 507033 (/home/zhongxuan/ptoas-board-monitor/runtime/runs/20260519_191506_manual_pr684/npu_validation/Sync/test_inject_sync_intra_pipe_barrier/main.cpp:68)
[ERROR] RecentErrMsg: [PID: 2691348] 2026-05-19-19:39:56.897.995 Invalid_Argument(EE1001): The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
        Solution: 1.Check the input parameter range of the function. 2.Check the function invocation relationship.
        TraceBack (most recent call last):
        TsdOpen failed. devId=11, tdt error=1[FUNC:PrintfTsdError][FILE:runtime.cc][LINE:2638]
        Check param failed, dev can not be NULL![FUNC:DeviceRetain][FILE:runtime.cc][LINE:3558]
        Check param failed, dev can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3175]
        Check param failed, ctx can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3206]
        Check param failed, context can not be null.[FUNC:SetDevice][FILE:api_impl.cc][LINE:3319]
        rtSetDevice execution failed, reason=device retain error[FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:65]
        open device 11 failed, runtime result = 507033.[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:148]
        ctx is NULL![FUNC:GetDevErrMsg][FILE:api_impl.cc][LINE:6108]
        The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
[2026-05-19 19:39:57] ERROR: testcase failed (exit 1): test_inject_sync_intra_pipe_barrier

test_set_wait_unified_api

stage=run info=exit=1

[ERROR] aclrtSetDevice(deviceId) failed: 507033 (/home/zhongxuan/ptoas-board-monitor/runtime/runs/20260519_191506_manual_pr684/npu_validation/Sync/test_set_wait_unified_api/main.cpp:68)
[ERROR] RecentErrMsg: [PID: 2692562] 2026-05-19-19:40:01.398.697 Invalid_Argument(EE1001): The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
        Solution: 1.Check the input parameter range of the function. 2.Check the function invocation relationship.
        TraceBack (most recent call last):
        TsdOpen failed. devId=11, tdt error=1[FUNC:PrintfTsdError][FILE:runtime.cc][LINE:2638]
        Check param failed, dev can not be NULL![FUNC:DeviceRetain][FILE:runtime.cc][LINE:3558]
        Check param failed, dev can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3175]
        Check param failed, ctx can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3206]
        Check param failed, context can not be null.[FUNC:SetDevice][FILE:api_impl.cc][LINE:3319]
        rtSetDevice execution failed, reason=device retain error[FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:65]
        open device 11 failed, runtime result = 507033.[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:148]
        ctx is NULL![FUNC:GetDevErrMsg][FILE:api_impl.cc][LINE:6108]
        The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
[2026-05-19 19:40:01] ERROR: testcase failed (exit 1): test_set_wait_unified_api

test_mem_inject_sync_basic

stage=run info=exit=1

[ERROR] aclrtSetDevice(deviceId) failed: 507033 (/home/zhongxuan/ptoas-board-monitor/runtime/runs/20260519_191506_manual_pr684/npu_validation/Sync/test_mem_inject_sync_basic/main.cpp:76)
[ERROR] RecentErrMsg: [PID: 2694151] 2026-05-19-19:40:05.705.121 Invalid_Argument(EE1001): The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
        Solution: 1.Check the input parameter range of the function. 2.Check the function invocation relationship.
        TraceBack (most recent call last):
        TsdOpen failed. devId=11, tdt error=1[FUNC:PrintfTsdError][FILE:runtime.cc][LINE:2638]
        Check param failed, dev can not be NULL![FUNC:DeviceRetain][FILE:runtime.cc][LINE:3558]
        Check param failed, dev can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3175]
        Check param failed, ctx can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3206]
        Check param failed, context can not be null.[FUNC:SetDevice][FILE:api_impl.cc][LINE:3319]
        rtSetDevice execution failed, reason=device retain error[FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:65]
        open device 11 failed, runtime result = 507033.[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:148]
        ctx is NULL![FUNC:GetDevErrMsg][FILE:api_impl.cc][LINE:6108]
        The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
[2026-05-19 19:40:06] ERROR: testcase failed (exit 1): test_mem_inject_sync_basic

test_intercore_sync_a3

stage=run info=exit=1

[ERROR] aclrtSetDevice(deviceId) failed: 507033 (/home/zhongxuan/ptoas-board-monitor/runtime/runs/20260519_191506_manual_pr684/npu_validation/Sync/test_intercore_sync_a3/main.cpp:76)
[ERROR] RecentErrMsg: [PID: 2695212] 2026-05-19-19:40:10.507.674 Invalid_Argument(EE1001): The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
        Solution: 1.Check the input parameter range of the function. 2.Check the function invocation relationship.
        TraceBack (most recent call last):
        TsdOpen failed. devId=11, tdt error=1[FUNC:PrintfTsdError][FILE:runtime.cc][LINE:2638]
        Check param failed, dev can not be NULL![FUNC:DeviceRetain][FILE:runtime.cc][LINE:3558]
        Check param failed, dev can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3175]
        Check param failed, ctx can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3206]
        Check param failed, context can not be null.[FUNC:SetDevice][FILE:api_impl.cc][LINE:3319]
        rtSetDevice execution failed, reason=device retain error[FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:65]
        open device 11 failed, runtime result = 507033.[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:148]
        ctx is NULL![FUNC:GetDevErrMsg][FILE:api_impl.cc][LINE:6108]
        The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
[2026-05-19 19:40:11] ERROR: testcase failed (exit 1): test_intercore_sync_a3

syncHigh

stage=run info=exit=1

[ERROR] aclrtSetDevice(deviceId) failed: 507033 (/home/zhongxuan/ptoas-board-monitor/runtime/runs/20260519_191506_manual_pr684/npu_validation/Sync/syncHigh/main.cpp:75)
[ERROR] RecentErrMsg: [PID: 2696857] 2026-05-19-19:40:15.958.593 Invalid_Argument(EE1001): The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
        Solution: 1.Check the input parameter range of the function. 2.Check the function invocation relationship.
        TraceBack (most recent call last):
        TsdOpen failed. devId=11, tdt error=1[FUNC:PrintfTsdError][FILE:runtime.cc][LINE:2638]
        Check param failed, dev can not be NULL![FUNC:DeviceRetain][FILE:runtime.cc][LINE:3558]
        Check param failed, dev can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3175]
        Check param failed, ctx can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3206]
        Check param failed, context can not be null.[FUNC:SetDevice][FILE:api_impl.cc][LINE:3319]
        rtSetDevice execution failed, reason=device retain error[FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:65]
        open device 11 failed, runtime result = 507033.[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:148]
        ctx is NULL![FUNC:GetDevErrMsg][FILE:api_impl.cc][LINE:6108]
        The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
[2026-05-19 19:40:17] ERROR: testcase failed (exit 1): syncHigh

matmul

stage=run info=exit=1

[ERROR] aclrtSetDevice(deviceId) failed: 507033 (/home/zhongxuan/ptoas-board-monitor/runtime/runs/20260519_191506_manual_pr684/npu_validation/Sync/matmul/main.cpp:84)
[ERROR] RecentErrMsg: [PID: 2698649] 2026-05-19-19:40:21.149.326 Invalid_Argument(EE1001): The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
        Solution: 1.Check the input parameter range of the function. 2.Check the function invocation relationship.
        TraceBack (most recent call last):
        TsdOpen failed. devId=11, tdt error=1[FUNC:PrintfTsdError][FILE:runtime.cc][LINE:2638]
        Check param failed, dev can not be NULL![FUNC:DeviceRetain][FILE:runtime.cc][LINE:3558]
        Check param failed, dev can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3175]
        Check param failed, ctx can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3206]
        Check param failed, context can not be null.[FUNC:SetDevice][FILE:api_impl.cc][LINE:3319]
        rtSetDevice execution failed, reason=device retain error[FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:65]
        open device 11 failed, runtime result = 507033.[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:148]
        ctx is NULL![FUNC:GetDevErrMsg][FILE:api_impl.cc][LINE:6108]
        The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
[2026-05-19 19:40:21] ERROR: testcase failed (exit 1): matmul

add_double_dynamic

stage=run info=exit=1

[ERROR] aclrtSetDevice(deviceId) failed: 507033 (/home/zhongxuan/ptoas-board-monitor/runtime/runs/20260519_191506_manual_pr684/npu_validation/Sync/add_double_dynamic/main.cpp:80)
[ERROR] RecentErrMsg: [PID: 2701106] 2026-05-19-19:40:25.795.120 Invalid_Argument(EE1001): The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
        Solution: 1.Check the input parameter range of the function. 2.Check the function invocation relationship.
        TraceBack (most recent call last):
        TsdOpen failed. devId=11, tdt error=1[FUNC:PrintfTsdError][FILE:runtime.cc][LINE:2638]
        Check param failed, dev can not be NULL![FUNC:DeviceRetain][FILE:runtime.cc][LINE:3558]
        Check param failed, dev can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3175]
        Check param failed, ctx can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3206]
        Check param failed, context can not be null.[FUNC:SetDevice][FILE:api_impl.cc][LINE:3319]
        rtSetDevice execution failed, reason=device retain error[FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:65]
        open device 11 failed, runtime result = 507033.[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:148]
        ctx is NULL![FUNC:GetDevErrMsg][FILE:api_impl.cc][LINE:6108]
        The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
[2026-05-19 19:40:26] ERROR: testcase failed (exit 1): add_double_dynamic

nested_loop_confliect

stage=run info=exit=1

[ERROR] aclrtSetDevice(deviceId) failed: 507033 (/home/zhongxuan/ptoas-board-monitor/runtime/runs/20260519_191506_manual_pr684/npu_validation/Sync/nested_loop_confliect/main.cpp:75)
[ERROR] RecentErrMsg: [PID: 2702598] 2026-05-19-19:40:30.593.273 Invalid_Argument(EE1001): The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
        Solution: 1.Check the input parameter range of the function. 2.Check the function invocation relationship.
        TraceBack (most recent call last):
        TsdOpen failed. devId=11, tdt error=1[FUNC:PrintfTsdError][FILE:runtime.cc][LINE:2638]
        Check param failed, dev can not be NULL![FUNC:DeviceRetain][FILE:runtime.cc][LINE:3558]
        Check param failed, dev can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3175]
        Check param failed, ctx can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3206]
        Check param failed, context can not be null.[FUNC:SetDevice][FILE:api_impl.cc][LINE:3319]
        rtSetDevice execution failed, reason=device retain error[FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:65]
        open device 11 failed, runtime result = 507033.[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:148]
        ctx is NULL![FUNC:GetDevErrMsg][FILE:api_impl.cc][LINE:6108]
        The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
[2026-05-19 19:40:31] ERROR: testcase failed (exit 1): nested_loop_confliect

rar_optimization_test

stage=run info=exit=1

[ERROR] aclrtSetDevice(deviceId) failed: 507033 (/home/zhongxuan/ptoas-board-monitor/runtime/runs/20260519_191506_manual_pr684/npu_validation/Sync/rar_optimization_test/main.cpp:75)
[ERROR] RecentErrMsg: [PID: 2703842] 2026-05-19-19:40:35.506.507 Invalid_Argument(EE1001): The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
        Solution: 1.Check the input parameter range of the function. 2.Check the function invocation relationship.
        TraceBack (most recent call last):
        TsdOpen failed. devId=11, tdt error=1[FUNC:PrintfTsdError][FILE:runtime.cc][LINE:2638]
        Check param failed, dev can not be NULL![FUNC:DeviceRetain][FILE:runtime.cc][LINE:3558]
        Check param failed, dev can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3175]
        Check param failed, ctx can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3206]
        Check param failed, context can not be null.[FUNC:SetDevice][FILE:api_impl.cc][LINE:3319]
        rtSetDevice execution failed, reason=device retain error[FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:65]
        open device 11 failed, runtime result = 507033.[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:148]
        ctx is NULL![FUNC:GetDevErrMsg][FILE:api_impl.cc][LINE:6108]
        The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
[2026-05-19 19:40:36] ERROR: testcase failed (exit 1): rar_optimization_test

test_intercore_sync_a3_modes

stage=run info=exit=1

[ERROR] aclrtSetDevice(deviceId) failed: 507033 (/home/zhongxuan/ptoas-board-monitor/runtime/runs/20260519_191506_manual_pr684/npu_validation/Sync/test_intercore_sync_a3_modes/main.cpp:72)
[ERROR] RecentErrMsg: [PID: 2705166] 2026-05-19-19:40:40.386.275 Invalid_Argument(EE1001): The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
        Solution: 1.Check the input parameter range of the function. 2.Check the function invocation relationship.
        TraceBack (most recent call last):
        TsdOpen failed. devId=11, tdt error=1[FUNC:PrintfTsdError][FILE:runtime.cc][LINE:2638]
        Check param failed, dev can not be NULL![FUNC:DeviceRetain][FILE:runtime.cc][LINE:3558]
        Check param failed, dev can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3175]
        Check param failed, ctx can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3206]
        Check param failed, context can not be null.[FUNC:SetDevice][FILE:api_impl.cc][LINE:3319]
        rtSetDevice execution failed, reason=device retain error[FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:65]
        open device 11 failed, runtime result = 507033.[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:148]
        ctx is NULL![FUNC:GetDevErrMsg][FILE:api_impl.cc][LINE:6108]
        The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
[2026-05-19 19:40:41] ERROR: testcase failed (exit 1): test_intercore_sync_a3_modes

test_dynamic_valid_shape

stage=run info=exit=1

[ERROR] aclrtSetDevice(deviceId) failed: 507033 (/home/zhongxuan/ptoas-board-monitor/runtime/runs/20260519_191506_manual_pr684/npu_validation/Sync/test_dynamic_valid_shape/main.cpp:76)
[ERROR] RecentErrMsg: [PID: 2706518] 2026-05-19-19:40:45.000.068 Invalid_Argument(EE1001): The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
        Solution: 1.Check the input parameter range of the function. 2.Check the function invocation relationship.
        TraceBack (most recent call last):
        TsdOpen failed. devId=11, tdt error=1[FUNC:PrintfTsdError][FILE:runtime.cc][LINE:2638]
        Check param failed, dev can not be NULL![FUNC:DeviceRetain][FILE:runtime.cc][LINE:3558]
        Check param failed, dev can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3175]
        Check param failed, ctx can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3206]
        Check param failed, context can not be null.[FUNC:SetDevice][FILE:api_impl.cc][LINE:3319]
        rtSetDevice execution failed, reason=device retain error[FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:65]
        open device 11 failed, runtime result = 507033.[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:148]
        ctx is NULL![FUNC:GetDevErrMsg][FILE:api_impl.cc][LINE:6108]
        The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
[2026-05-19 19:40:45] ERROR: testcase failed (exit 1): test_dynamic_valid_shape

test_inject_sync_if_else

stage=run info=exit=1

[ERROR] aclrtSetDevice(deviceId) failed: 507033 (/home/zhongxuan/ptoas-board-monitor/runtime/runs/20260519_191506_manual_pr684/npu_validation/Sync/test_inject_sync_if_else/main.cpp:76)
[ERROR] RecentErrMsg: [PID: 2708016] 2026-05-19-19:40:49.902.188 Invalid_Argument(EE1001): The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
        Solution: 1.Check the input parameter range of the function. 2.Check the function invocation relationship.
        TraceBack (most recent call last):
        TsdOpen failed. devId=11, tdt error=1[FUNC:PrintfTsdError][FILE:runtime.cc][LINE:2638]
        Check param failed, dev can not be NULL![FUNC:DeviceRetain][FILE:runtime.cc][LINE:3558]
        Check param failed, dev can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3175]
        Check param failed, ctx can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3206]
        Check param failed, context can not be null.[FUNC:SetDevice][FILE:api_impl.cc][LINE:3319]
        rtSetDevice execution failed, reason=device retain error[FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:65]
        open device 11 failed, runtime result = 507033.[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:148]
        ctx is NULL![FUNC:GetDevErrMsg][FILE:api_impl.cc][LINE:6108]
        The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
[2026-05-19 19:40:50] ERROR: testcase failed (exit 1): test_inject_sync_if_else

test_auto_sync_tail_hint

stage=run info=exit=1

[ERROR] aclrtSetDevice(deviceId) failed: 507033 (/home/zhongxuan/ptoas-board-monitor/runtime/runs/20260519_191506_manual_pr684/npu_validation/Sync/test_auto_sync_tail_hint/main.cpp:75)
[ERROR] RecentErrMsg: [PID: 2709703] 2026-05-19-19:40:54.765.803 Invalid_Argument(EE1001): The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
        Solution: 1.Check the input parameter range of the function. 2.Check the function invocation relationship.
        TraceBack (most recent call last):
        TsdOpen failed. devId=11, tdt error=1[FUNC:PrintfTsdError][FILE:runtime.cc][LINE:2638]
        Check param failed, dev can not be NULL![FUNC:DeviceRetain][FILE:runtime.cc][LINE:3558]
        Check param failed, dev can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3175]
        Check param failed, ctx can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3206]
        Check param failed, context can not be null.[FUNC:SetDevice][FILE:api_impl.cc][LINE:3319]
        rtSetDevice execution failed, reason=device retain error[FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:65]
        open device 11 failed, runtime result = 507033.[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:148]
        ctx is NULL![FUNC:GetDevErrMsg][FILE:api_impl.cc][LINE:6108]
        The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
[2026-05-19 19:40:55] ERROR: testcase failed (exit 1): test_auto_sync_tail_hint

sync

stage=run info=exit=1

[ERROR] aclrtSetDevice(deviceId) failed: 507033 (/home/zhongxuan/ptoas-board-monitor/runtime/runs/20260519_191506_manual_pr684/npu_validation/Sync/sync/main.cpp:75)
[ERROR] RecentErrMsg: [PID: 2711185] 2026-05-19-19:40:59.609.981 Invalid_Argument(EE1001): The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
        Solution: 1.Check the input parameter range of the function. 2.Check the function invocation relationship.
        TraceBack (most recent call last):
        TsdOpen failed. devId=11, tdt error=1[FUNC:PrintfTsdError][FILE:runtime.cc][LINE:2638]
        Check param failed, dev can not be NULL![FUNC:DeviceRetain][FILE:runtime.cc][LINE:3558]
        Check param failed, dev can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3175]
        Check param failed, ctx can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3206]
        Check param failed, context can not be null.[FUNC:SetDevice][FILE:api_impl.cc][LINE:3319]
        rtSetDevice execution failed, reason=device retain error[FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:65]
        open device 11 failed, runtime result = 507033.[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:148]
        ctx is NULL![FUNC:GetDevErrMsg][FILE:api_impl.cc][LINE:6108]
        The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
[2026-05-19 19:41:00] ERROR: testcase failed (exit 1): sync

test_intercore_sync_a3_dyn

stage=run info=exit=1

[ERROR] aclrtSetDevice(deviceId) failed: 507033 (/home/zhongxuan/ptoas-board-monitor/runtime/runs/20260519_191506_manual_pr684/npu_validation/Sync/test_intercore_sync_a3_dyn/main.cpp:76)
[ERROR] RecentErrMsg: [PID: 2712643] 2026-05-19-19:41:04.720.596 Invalid_Argument(EE1001): The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
        Solution: 1.Check the input parameter range of the function. 2.Check the function invocation relationship.
        TraceBack (most recent call last):
        TsdOpen failed. devId=11, tdt error=1[FUNC:PrintfTsdError][FILE:runtime.cc][LINE:2638]
        Check param failed, dev can not be NULL![FUNC:DeviceRetain][FILE:runtime.cc][LINE:3558]
        Check param failed, dev can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3175]
        Check param failed, ctx can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3206]
        Check param failed, context can not be null.[FUNC:SetDevice][FILE:api_impl.cc][LINE:3319]
        rtSetDevice execution failed, reason=device retain error[FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:65]
        open device 11 failed, runtime result = 507033.[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:148]
        ctx is NULL![FUNC:GetDevErrMsg][FILE:api_impl.cc][LINE:6108]
        The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
[2026-05-19 19:41:05] ERROR: testcase failed (exit 1): test_intercore_sync_a3_dyn

test_inject_sync_loop_nest

stage=run info=exit=1

[ERROR] aclrtSetDevice(deviceId) failed: 507033 (/home/zhongxuan/ptoas-board-monitor/runtime/runs/20260519_191506_manual_pr684/npu_validation/Sync/test_inject_sync_loop_nest/main.cpp:75)
[ERROR] RecentErrMsg: [PID: 2713958] 2026-05-19-19:41:09.646.465 Invalid_Argument(EE1001): The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
        Solution: 1.Check the input parameter range of the function. 2.Check the function invocation relationship.
        TraceBack (most recent call last):
        TsdOpen failed. devId=11, tdt error=1[FUNC:PrintfTsdError][FILE:runtime.cc][LINE:2638]
        Check param failed, dev can not be NULL![FUNC:DeviceRetain][FILE:runtime.cc][LINE:3558]
        Check param failed, dev can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3175]
        Check param failed, ctx can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3206]
        Check param failed, context can not be null.[FUNC:SetDevice][FILE:api_impl.cc][LINE:3319]
        rtSetDevice execution failed, reason=device retain error[FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:65]
        open device 11 failed, runtime result = 507033.[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:148]
        ctx is NULL![FUNC:GetDevErrMsg][FILE:api_impl.cc][LINE:6108]
        The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
[2026-05-19 19:41:10] ERROR: testcase failed (exit 1): test_inject_sync_loop_nest

reedhecre · 2026-05-19T11:41:56Z

A3 板测失败详情：PR #684

compensation_test

stage=run info=exit=1

[ERROR] aclrtSetDevice(deviceId) failed: 507033 (/home/zhongxuan/ptoas-board-monitor/runtime/runs/20260519_191506_manual_pr684/npu_validation/Sync/compensation_test/main.cpp:72)
[ERROR] RecentErrMsg: [PID: 2715375] 2026-05-19-19:41:15.211.158 Invalid_Argument(EE1001): The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
        Solution: 1.Check the input parameter range of the function. 2.Check the function invocation relationship.
        TraceBack (most recent call last):
        TsdOpen failed. devId=11, tdt error=1[FUNC:PrintfTsdError][FILE:runtime.cc][LINE:2638]
        Check param failed, dev can not be NULL![FUNC:DeviceRetain][FILE:runtime.cc][LINE:3558]
        Check param failed, dev can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3175]
        Check param failed, ctx can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3206]
        Check param failed, context can not be null.[FUNC:SetDevice][FILE:api_impl.cc][LINE:3319]
        rtSetDevice execution failed, reason=device retain error[FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:65]
        open device 11 failed, runtime result = 507033.[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:148]
        ctx is NULL![FUNC:GetDevErrMsg][FILE:api_impl.cc][LINE:6108]
        The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
[2026-05-19 19:41:15] ERROR: testcase failed (exit 1): compensation_test

test_inject_sync_if

stage=run info=exit=1

[ERROR] aclrtSetDevice(deviceId) failed: 507033 (/home/zhongxuan/ptoas-board-monitor/runtime/runs/20260519_191506_manual_pr684/npu_validation/Sync/test_inject_sync_if/main.cpp:76)
[ERROR] RecentErrMsg: [PID: 2716793] 2026-05-19-19:41:19.559.148 Invalid_Argument(EE1001): The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
        Solution: 1.Check the input parameter range of the function. 2.Check the function invocation relationship.
        TraceBack (most recent call last):
        TsdOpen failed. devId=11, tdt error=1[FUNC:PrintfTsdError][FILE:runtime.cc][LINE:2638]
        Check param failed, dev can not be NULL![FUNC:DeviceRetain][FILE:runtime.cc][LINE:3558]
        Check param failed, dev can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3175]
        Check param failed, ctx can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3206]
        Check param failed, context can not be null.[FUNC:SetDevice][FILE:api_impl.cc][LINE:3319]
        rtSetDevice execution failed, reason=device retain error[FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:65]
        open device 11 failed, runtime result = 507033.[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:148]
        ctx is NULL![FUNC:GetDevErrMsg][FILE:api_impl.cc][LINE:6108]
        The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
[2026-05-19 19:41:20] ERROR: testcase failed (exit 1): test_inject_sync_if

test_inject_sync_two_event_id

stage=run info=exit=1

[ERROR] aclrtSetDevice(deviceId) failed: 507033 (/home/zhongxuan/ptoas-board-monitor/runtime/runs/20260519_191506_manual_pr684/npu_validation/Sync/test_inject_sync_two_event_id/main.cpp:83)
[ERROR] RecentErrMsg: [PID: 2719040] 2026-05-19-19:41:24.773.130 Invalid_Argument(EE1001): The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
        Solution: 1.Check the input parameter range of the function. 2.Check the function invocation relationship.
        TraceBack (most recent call last):
        TsdOpen failed. devId=11, tdt error=1[FUNC:PrintfTsdError][FILE:runtime.cc][LINE:2638]
        Check param failed, dev can not be NULL![FUNC:DeviceRetain][FILE:runtime.cc][LINE:3558]
        Check param failed, dev can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3175]
        Check param failed, ctx can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3206]
        Check param failed, context can not be null.[FUNC:SetDevice][FILE:api_impl.cc][LINE:3319]
        rtSetDevice execution failed, reason=device retain error[FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:65]
        open device 11 failed, runtime result = 507033.[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:148]
        ctx is NULL![FUNC:GetDevErrMsg][FILE:api_impl.cc][LINE:6108]
        The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
[2026-05-19 19:41:25] ERROR: testcase failed (exit 1): test_inject_sync_two_event_id

rem

stage=run info=exit=1

[ERROR] aclrtSetDevice(deviceId) failed: 507033 (/home/zhongxuan/ptoas-board-monitor/runtime/runs/20260519_191506_manual_pr684/npu_validation/Rem/rem/main.cpp:79)
[ERROR] RecentErrMsg: [PID: 2721063] 2026-05-19-19:41:29.652.098 Invalid_Argument(EE1001): The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
        Solution: 1.Check the input parameter range of the function. 2.Check the function invocation relationship.
        TraceBack (most recent call last):
        TsdOpen failed. devId=11, tdt error=1[FUNC:PrintfTsdError][FILE:runtime.cc][LINE:2638]
        Check param failed, dev can not be NULL![FUNC:DeviceRetain][FILE:runtime.cc][LINE:3558]
        Check param failed, dev can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3175]
        Check param failed, ctx can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3206]
        Check param failed, context can not be null.[FUNC:SetDevice][FILE:api_impl.cc][LINE:3319]
        rtSetDevice execution failed, reason=device retain error[FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:65]
        open device 11 failed, runtime result = 507033.[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:148]
        ctx is NULL![FUNC:GetDevErrMsg][FILE:api_impl.cc][LINE:6108]
        The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
[2026-05-19 19:41:30] ERROR: testcase failed (exit 1): rem

partmin

stage=run info=exit=1

[ERROR] aclrtSetDevice(deviceId) failed: 507033 (/home/zhongxuan/ptoas-board-monitor/runtime/runs/20260519_191506_manual_pr684/npu_validation/Partmin/partmin/main.cpp:79)
[ERROR] RecentErrMsg: [PID: 2722329] 2026-05-19-19:41:34.698.305 Invalid_Argument(EE1001): The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
        Solution: 1.Check the input parameter range of the function. 2.Check the function invocation relationship.
        TraceBack (most recent call last):
        TsdOpen failed. devId=11, tdt error=1[FUNC:PrintfTsdError][FILE:runtime.cc][LINE:2638]
        Check param failed, dev can not be NULL![FUNC:DeviceRetain][FILE:runtime.cc][LINE:3558]
        Check param failed, dev can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3175]
        Check param failed, ctx can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3206]
        Check param failed, context can not be null.[FUNC:SetDevice][FILE:api_impl.cc][LINE:3319]
        rtSetDevice execution failed, reason=device retain error[FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:65]
        open device 11 failed, runtime result = 507033.[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:148]
        ctx is NULL![FUNC:GetDevErrMsg][FILE:api_impl.cc][LINE:6108]
        The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
[2026-05-19 19:41:35] ERROR: testcase failed (exit 1): partmin

colexpandadd

stage=run info=exit=1

[ERROR] aclrtSetDevice(deviceId) failed: 507033 (/home/zhongxuan/ptoas-board-monitor/runtime/runs/20260519_191506_manual_pr684/npu_validation/Colexpandadd/colexpandadd/main.cpp:79)
[ERROR] RecentErrMsg: [PID: 2723615] 2026-05-19-19:41:39.563.839 Invalid_Argument(EE1001): The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
        Solution: 1.Check the input parameter range of the function. 2.Check the function invocation relationship.
        TraceBack (most recent call last):
        TsdOpen failed. devId=11, tdt error=1[FUNC:PrintfTsdError][FILE:runtime.cc][LINE:2638]
        Check param failed, dev can not be NULL![FUNC:DeviceRetain][FILE:runtime.cc][LINE:3558]
        Check param failed, dev can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3175]
        Check param failed, ctx can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3206]
        Check param failed, context can not be null.[FUNC:SetDevice][FILE:api_impl.cc][LINE:3319]
        rtSetDevice execution failed, reason=device retain error[FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:65]
        open device 11 failed, runtime result = 507033.[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:148]
        ctx is NULL![FUNC:GetDevErrMsg][FILE:api_impl.cc][LINE:6108]
        The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
[2026-05-19 19:41:40] ERROR: testcase failed (exit 1): colexpandadd

sqrt

stage=run info=exit=1

[ERROR] aclrtSetDevice(deviceId) failed: 507033 (/home/zhongxuan/ptoas-board-monitor/runtime/runs/20260519_191506_manual_pr684/npu_validation/Sqrt/sqrt/main.cpp:75)
[ERROR] RecentErrMsg: [PID: 2725170] 2026-05-19-19:41:44.609.529 Invalid_Argument(EE1001): The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
        Solution: 1.Check the input parameter range of the function. 2.Check the function invocation relationship.
        TraceBack (most recent call last):
        TsdOpen failed. devId=11, tdt error=1[FUNC:PrintfTsdError][FILE:runtime.cc][LINE:2638]
        Check param failed, dev can not be NULL![FUNC:DeviceRetain][FILE:runtime.cc][LINE:3558]
        Check param failed, dev can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3175]
        Check param failed, ctx can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3206]
        Check param failed, context can not be null.[FUNC:SetDevice][FILE:api_impl.cc][LINE:3319]
        rtSetDevice execution failed, reason=device retain error[FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:65]
        open device 11 failed, runtime result = 507033.[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:148]
        ctx is NULL![FUNC:GetDevErrMsg][FILE:api_impl.cc][LINE:6108]
        The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
[2026-05-19 19:41:45] ERROR: testcase failed (exit 1): sqrt

bf16_tile

stage=run info=exit=1

[ERROR] aclrtSetDevice(deviceId) failed: 507033 (/home/zhongxuan/ptoas-board-monitor/runtime/runs/20260519_191506_manual_pr684/npu_validation/Bf16/bf16_tile/main.cpp:71)
[ERROR] RecentErrMsg: [PID: 2726714] 2026-05-19-19:41:49.916.948 Invalid_Argument(EE1001): The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
        Solution: 1.Check the input parameter range of the function. 2.Check the function invocation relationship.
        TraceBack (most recent call last):
        TsdOpen failed. devId=11, tdt error=1[FUNC:PrintfTsdError][FILE:runtime.cc][LINE:2638]
        Check param failed, dev can not be NULL![FUNC:DeviceRetain][FILE:runtime.cc][LINE:3558]
        Check param failed, dev can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3175]
        Check param failed, ctx can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3206]
        Check param failed, context can not be null.[FUNC:SetDevice][FILE:api_impl.cc][LINE:3319]
        rtSetDevice execution failed, reason=device retain error[FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:65]
        open device 11 failed, runtime result = 507033.[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:148]
        ctx is NULL![FUNC:GetDevErrMsg][FILE:api_impl.cc][LINE:6108]
        The argument is invalid.Reason: rtGetDevMsg execution failed, the context is a null pointer.
[2026-05-19 19:41:50] ERROR: testcase failed (exit 1): bf16_tile
[2026-05-19 19:41:50] === SUMMARY ===
[2026-05-19 19:41:50] OK=167 FAIL=45 SKIP=2
[2026-05-19 19:41:50] RESULTS_TSV=/home/zhongxuan/ptoas-board-monitor/runtime/runs/20260519_191506_manual_pr684/remote_npu_validation_results.tsv
[npu-lock] 已释放设备 11 的锁
=== 任务失败 (exit=1) ===
task-submit wait rc=1
completed (exit=1)

FangRui0 · 2026-05-20T01:28:35Z

/run a3

reedhecre · 2026-05-20T01:29:04Z

已接收 /run a3，A3 板测器会处理这条请求。

进度页：http://154.9.227.233/ptoas-board-dashboard/#board-a3
当前状态：板测器空闲，这条请求会在本轮轮询启动。

页面会自动刷新，可以直接看当前阶段、排队情况和最近结果。

FangRui0 · 2026-05-20T03:43:29Z

/run a3

reedhecre · 2026-05-20T03:43:55Z

已接收 /run a3，A3 板测器会处理这条请求。

进度页：http://154.9.227.233/ptoas-board-dashboard/#board-a3
当前状态：板测器空闲，这条请求会在本轮轮询启动。

页面会自动刷新，可以直接看当前阶段、排队情况和最近结果。

reedhecre · 2026-05-20T04:11:54Z

A3 板测完成（有跳过）

触发方式：manual
源码提交：a18de1a33fd1
结果汇总：OK 212 / FAIL 0 / SKIP 2
日志：/home/zhongxuan/ptoas-board-monitor/runtime/logs/20260520_114405_manual_pr684.log
结果 TSV：/home/zhongxuan/ptoas-board-monitor/runtime/logs/20260520_114405_manual_pr684.tsv
手动指令：/run a3
触发人：FangRui0
触发评论：enable ci llvm debug #684 (comment)

FangRui0 force-pushed the codex/ci-llvm-debug branch from 6fe10d7 to 7874020 Compare May 19, 2026 06:27

Enable assertion builds in CI and fix assert regressions

feeed75

Signed-off-by: FangRui <[email protected]>

FangRui0 force-pushed the codex/ci-llvm-debug branch from 7874020 to feeed75 Compare May 19, 2026 07:40

Fix CI test paths for split build directories

04da79e

FangRui0 force-pushed the codex/ci-llvm-debug branch from 26c26bd to 04da79e Compare May 20, 2026 09:54

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

enable ci llvm debug#684

enable ci llvm debug#684
FangRui0 wants to merge 2 commits into
hw-native-sys:mainfrom
FangRui0:codex/ci-llvm-debug

FangRui0 commented May 19, 2026

Uh oh!

gemini-code-assist Bot commented May 19, 2026

Uh oh!

reedhecre commented May 19, 2026 •

edited

Loading

Uh oh!

FangRui0 commented May 19, 2026

Uh oh!

reedhecre commented May 19, 2026

Uh oh!

reedhecre commented May 19, 2026

Uh oh!

reedhecre commented May 19, 2026

Uh oh!

reedhecre commented May 19, 2026

Uh oh!

FangRui0 commented May 20, 2026

Uh oh!

reedhecre commented May 20, 2026

Uh oh!

FangRui0 commented May 20, 2026

Uh oh!

reedhecre commented May 20, 2026

Uh oh!

reedhecre commented May 20, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

FangRui0 commented May 19, 2026

Uh oh!

gemini-code-assist Bot commented May 19, 2026

Uh oh!

reedhecre commented May 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codex Review

Summary

Findings

Uh oh!

FangRui0 commented May 19, 2026

Uh oh!

reedhecre commented May 19, 2026

Uh oh!

reedhecre commented May 19, 2026

A3 板测失败

失败用例

Uh oh!

reedhecre commented May 19, 2026

A3 板测失败详情：PR #684

Uh oh!

reedhecre commented May 19, 2026

A3 板测失败详情：PR #684

Uh oh!

FangRui0 commented May 20, 2026

Uh oh!

reedhecre commented May 20, 2026

Uh oh!

FangRui0 commented May 20, 2026

Uh oh!

reedhecre commented May 20, 2026

Uh oh!

reedhecre commented May 20, 2026

A3 板测完成（有跳过）

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

reedhecre commented May 19, 2026 •

edited

Loading