Paper review 2026-07-29: 25 candidates

[github-actions]

Reply with commands, one per line:
/approve all · /approve 1,3-5 · /reject 2 · /edit 3 category=world_terminal tags=benchmark (edit implies approve; tags=- clears tags). Valid category keys: see taxonomy.json.

1. Where Is the Cost of Third-Party API Routers in Agentic Software Development?

Donghao Fu, Jingxin Li, Xue Jiang, et al. · arXiv 2026/07 · paper
proposed: software_security · tags: empirical

An empirical study showing third-party API routers sitting between coding agents and LLM providers can silently inject or alter model outputs, causing agents to execute unintended repository-level actions with zero detection by existing client-side safeguards. The authors build SIDEL, a framework for trace recording, replay, injection, and defense evaluation across four coding agents.
_{Empirical security study of the trusted path in agentic software development, i.e. security assurance around code-producing agents (software_security).}

2. Towards LLM-assisted High-Quality Property Generation for Solidity Smart Contracts

Muhammad Wahid, Shahzaib Khan, Mashhood Ali, et al. · arXiv 2026/07 · paper
proposed: software_security · tags: none

Uses LLMs with prompting strategies (zero-shot, few-shot, prompt chaining) to generate property-based testing properties for Solidity smart contracts, evaluated via mutation testing against human-written baselines. Gemini Pro 1.5 with prompt chaining approaches human-level property quality on some contracts.
_{Generates fuzzing properties aimed at uncovering smart-contract vulnerabilities, matching software_security's vulnerability-targeted fuzzing scope.}

3. The Illusion of Secure LLM Code: Closing the Security Gap via Iterative Reprompting

Ishpuneet Singh, Shreyas Mahajan, Gurjot Singh, et al. · arXiv 2026/07 · paper
proposed: software_security · tags: empirical

Empirically evaluates authentication code security from five AI coding assistants under varying prompting strategies via static analysis and penetration testing mapped to NIST SP 800-63B. Finds iterative reprompting is required to close persistent security gaps in LLM-generated code.
_{Concerns secure code generation by AI assistants, matching software_security's 'secure code generation' scope.}

4. ALIBI: Adaptive Agentic Attacks on LLM-Based Vulnerability Detectors via Adversarial Code Comments

Zixuan Wu, Cristina Nita-Rotaru · arXiv 2026/07 · paper
proposed: software_security · tags: none

ALIBI is a coding agent that implements features, deliberately injects vulnerabilities, and crafts adversarial code comments to evade LLM-based vulnerability detectors, achieving over 90% attack success across real-world CVE-derived tasks. It also evaluates defenses like architectural isolation and comment sanitization.
_{An agent manipulates code to attack vulnerability-detection tools, a security-assurance activity on code.}

5. ZIMPAF & RedPhuzz: High-fidelity Web Application Fuzzing via Branch, Language Construct, and Function Call Monitoring

Tennov Simanjuntak, Christoph Csallner · arXiv 2026/07 · paper
proposed: software_security · tags: none

ZIMPAF provides novel multi-granular runtime interpreter instrumentation for web applications, identifying tainted and vulnerable functions, which drives RedPhuzz, a targeted fuzzer using sanitization- and type-aware mutations. RedPhuzz detects all vulnerabilities across benchmark web apps while being substantially faster than its predecessor.
_{A fuzzing system whose purpose is finding vulnerabilities in code, matching the security-assurance/fuzzing activity.}

6. SkillGate: Cost Efficient Runtime Malicious Skill File Detection in Coding Agents

Rui Yang, Michael Fu, Kla Tantithamthavorn, et al. · arXiv 2026/07 · paper
proposed: software_security · tags: none

SkillGate is a cost-efficient security gateway that screens AI coding-agent skill files for malicious content before installation, using a regex-prefilter plus LLM-judge pipeline evaluated on SkillsBench.
_{Detects malicious/backdoor-injecting artifacts in the coding-agent supply chain, a security-assurance task on code (software_security).}

7. Distributing Security Controls Through Harness Engineering

William Robert Gore · arXiv 2026/07 · paper
proposed: software_security · tags: empirical

This paper builds SHarD, a distributable security harness for commercial AI coding agents implementing OS sandboxing, skill scanning, and tool restriction, and empirically tests it against an OWASP Agentic-Applications-derived test suite across four agent configurations.
_{Security-assurance work guarding autonomous coding agents' operation falls under the software_security assurance activity.}

8. CodexGraph: Bridging Large Language Models and Code Repositories via Code Graph Databases

Xiangyan Liu, Bo Lan, Zhiyuan Hu, et al. · arXiv 2024/08 · paper
proposed: software_comprehension · tags: none

CodexGraph integrates LLM agents with graph databases built from code repositories, letting agents construct and execute structural graph queries for precise repository-scale code navigation and context retrieval.
_{The served task is repository-scale code search/navigation, matching software_comprehension's code-search inclusion.}

9. CodeNib: A Multi-View Data System for Serving Repository Context to Coding Agents

Zhongming Yu, Hengjia Yu, Boqin Yuan, et al. · arXiv 2026/07 · paper
proposed: software_comprehension · tags: none

CodeNib builds reusable lexical, dense, and structural context views per repository commit to serve ranked search, symbol navigation, and bounded context to coding agents through one runtime. It shows large latency and token savings versus rebuilding indexes or using grep/read.
_{Serves repository code search and navigation for agents, matching software_comprehension's code search/navigation task.}

10. RepoReasoner: Evaluating Repository-Level Code Reasoning Ability of Long-Context Language Models

Yanlin Wang, Suiquan Wang, Yanli Wang, et al. · arXiv 2026/07 · paper
proposed: software_comprehension · tags: benchmark

Introduces RepoReasoner, a benchmark evaluating long-context LLMs on repository-level code reasoning via output prediction and call-chain prediction tasks built from dynamic pytest tracing. Finds current models struggle with cross-file execution reasoning and dependency understanding.
_{benchmark serving repository-level code understanding/QA, matching software_comprehension}

11. ATLAS: Multi-View Code Representation Tool for C and C++ Source Programs

Jaid Monwar Chowdhury, Ahmad Farhan Shahriar Chowdhury, Humayra Binte Monwar, et al. · arXiv 2025/12 · paper
proposed: software_testing · tags: none

ATLAS is a command-line tool that extracts aligned AST, control-flow, and data-flow graphs from C/C++ source without requiring a full build, serving as the front-end for an LLM-based unit-test generation framework.
_{A resource paper whose downstream task is proactive test generation, routed to software_testing per resource-follows-task rule.}

12. PatchGuru: Patch Oracle Inference from Natural Language Artifacts

Thanh Le-Cong, Bach Le, Toby Murray, et al. · arXiv 2026/02 · paper
proposed: software_testing · tags: none

PatchGuru uses LLMs to infer executable patch oracles from pull-request natural-language artifacts, validating patch behavior via runtime assertions and surfacing previously unknown bugs on real-world Python projects.
_{Generates and executes correctness specifications to validate code changes, an assurance/verification task independent of a single reported defect, fitting software_testing.}

13. Do Coverage and Mutation Scores of LLM-Generated Test Suites Correlate with Their Effectiveness? (Replicability Study)

Junda Zhao, Shurui Zhou, Eldan Cohen · arXiv 2026/07 · paper
proposed: software_testing · tags: empirical

A large-scale replication study re-examining whether coverage and mutation scores of LLM-generated test suites correlate with real bug-detection effectiveness, finding the relationship is highly context-dependent.
_{An empirical study of one activity (test generation evaluation), which per taxonomy routes to that leaf plus the empirical tag rather than the field-wide studies node.}

14. RESTOR: Automated Test Oracle Generation for RESTful APIs via Reinforcement Learning

Xun Zhou, Zhen Dong, Mingyu Ren, et al. · arXiv 2026/07 · paper
proposed: software_testing · tags: none

Restor fine-tunes a lightweight LLM via GRPO to generate executable test-oracle assertions for REST APIs from a single request-response trace, avoiding brittle rule-based templates. Evaluated on industrial ByteDance data, it raises automated test-case adoption from 74.1% to over 96% in production CI/CD.
_{Produces test oracles/assertions for correctness assurance, matching software_testing's proactive test generation scope.}

15. Industrial Practice of LLM-Based Test Case Carving and Assertion Generation (Experience Paper)

Haozhen You, Zhen Dong, Jingjing Wang, et al. · arXiv 2026/07 · paper
proposed: software_testing · tags: empirical

NL2Test is an industrial tool that generates executable API regression tests from natural-language scenario descriptions and recorded traffic, combining LLM-based semantic interpretation with deterministic dependency and assertion validation. Deployed in production for 9 months, it generated over 3,000 test cases with high adoption.
_{The task is proactive test generation for regression testing, matching software_testing; industry deployment makes it an experience/empirical report.}

16. RIDGE: An Autonomous Framework for Validation and Method Discovery in LLM-Generated Option Pricing

Liexin Cheng, Xue Cheng, Shuaiqiang Liu, et al. · arXiv 2026/07 · paper
proposed: software_testing · tags: none

RIDGE is an autonomous framework that validates LLM-generated option pricing code via structured no-arbitrage, stress, and consistency tests, iteratively refining implementations and accumulating diagnostic knowledge across models.
_{Proactive, non-defect-specific correctness verification of generated code fits software_testing rather than a mere finance case study since the framework itself is the code-validation agent.}

17. Specula: Scaling formal specifications for autonomous model checking of system code

Qian Cheng, Saad Mohammad Rafid Pial, Ruize Tang, et al. · arXiv 2026/07 · paper
proposed: software_testing · tags: none

Specula is an autonomous LLM coding-agent system that generates TLA+ formal specifications for large system codebases and uses them for model checking. It found 249 bugs across 48 open-source projects via self-evolving refinement loops.
_{Formal verification and proof-assisted correctness of system code performed by an autonomous coding agent falls under software_testing.}

18. KQFuzz: Knowledge-Guided Fuzzing for Quantum Libraries via Large Language Models

Fuyuan Xia, Qixin Zhang, Chenhao Ying, et al. · arXiv 2026/07 · paper
proposed: software_testing · tags: none

KQFuzz is a knowledge-guided LLM fuzzer that generates and mutates quantum programs to test quantum libraries like Qiskit, PennyLane, and Cirq. It found 13 previously unknown bugs, most confirmed and fixed by developers.
_{Proactive automated fuzzing for general correctness bugs in software libraries fits software_testing.}

19. ABISS: Evaluating Text-to-SQL Systems Through Agent Interaction

Giovanni Sullutrone, Luca Sala, Sania Aftar, et al. · arXiv 2026/07 · paper
proposed: database · tags: benchmark

ABISS introduces a taxonomy of ambiguous/unanswerable Text-to-SQL questions, a multi-agent pipeline to generate realistic benchmark data, and a dynamic multi-turn simulation environment for evaluating Text-to-SQL agents' interaction and clarification abilities.
_{Benchmark for interactive Text-to-SQL agents whose deliverable is SQL queries, routing to database.}

20. What You See Is What You Get: Observation-Aligned Supervision for Chart-to-Code Generation

Tianhao Niu, Qingfu Zhu, Wanxiang Che · arXiv 2026/07 · paper
proposed: graphics · tags: none

Identifies latent-observation mismatches (aggregation, normalization, projection) that make gold plotting code an invalid supervision target for chart-to-code generation, and proposes Observation-Aligned Supervision using visually constrained quantities instead. Improves observable-value recovery for VLMs on ChartMimic and ChartX.
_{Serves chart-from-spec code generation where the visual artifact's code is the deliverable, matching the graphics leaf.}

21. Crystalis: Progressive Nucleation and Semantic Annealing for Coordinated Multi-View Visualization Generation

Dazhen Deng, Zhaoping He, Xin Qian, et al. · arXiv 2026/06 · paper
proposed: graphics · tags: none

Crystalis is a framework that decomposes coordinated multi-view visualizations into a dependency graph of data, visualization, and interaction queries, using progressive nucleation and semantic annealing to reliably generate structurally correct executable visualization code from specifications.
_{The task's purpose is producing chart/visualization code from a specification, matching the graphics artifact leaf.}

22. daVinci-kernel: Co-Evolving Skill Selection, Summarization, and Utilization via RL for GPU Kernel Optimization

Dayuan Fu, Mohan Jiang, Tongyu Wang, et al. · arXiv 2026/06 · paper
proposed: systems · tags: model

daVinci-kernel is an RL framework that jointly trains skill-selection, kernel-generation, and skill-summarization agents sharing one LLM backbone to optimize CUDA/Triton GPU kernels, evaluated on KernelBench. It outperforms prior RL-trained kernel models across benchmark levels.
_{KernelBench-style GPU kernel code generation routes to systems per benchmark-routing rule.}

23. Compiler-Grounded Hierarchical Diagnosis for LLM-Based Triton Kernel Optimization

Dongjie Chen, Ping Zhao, Bohua Zhan, et al. · arXiv 2026/07 · paper
proposed: systems · tags: none

Presents a compiler-grounded hierarchical diagnosis framework that escalates from profiling to IR-level compiler analysis to produce evidence-backed source rewrites for Triton kernels on NPUs. Achieves large geometric-mean speedups on an Ascend-950-derived kernel benchmark.
_{Optimizes accelerator kernel code, analogous to KernelBench precedent which routes to systems.}

24. Efficient LLM-Generated Shuttling Compilers for Complex Trapped-Ion Architectures

Fabian Kreppel, Reza Salkhordeh, Ferdinand Schmidt-Kaler, et al. · arXiv 2026/07 · paper
proposed: systems · tags: none

A frontier LLM (Claude) is used to generate and iteratively refine full Python shuttling compilers for trapped-ion quantum architectures, matching or beating hand-crafted compilers in shuttling-timestep efficiency.
_{The deliverable is compiler/runtime code, which falls under the systems leaf per its inclusion of compiler and runtime code.}

25. Kernel Forge: An Agent Harness for LLM-based Generation and Optimization of CUDA Kernels

Joshua Brodsky, Dhravid Kumar, Savini Kashmira, et al. · arXiv 2026/06 · paper
proposed: systems · tags: none

Kernel Forge is an end-to-end agentic harness that uses MCTS-guided LLM optimization to generate and integrate CUDA kernels directly into unmodified PyTorch models, outperforming eager-mode execution across vision, diffusion, and LLM workloads.
_{The agent produces low-level GPU/systems code (KernelBench-style task), so it routes to the systems leaf per benchmark-routing precedent.}

machine payload (do not edit)

[{"paper": {"id": "2607.23624", "title": "Where Is the Cost of Third-Party API Routers in Agentic Software Development?", "authors": ["Donghao Fu", "Jingxin Li", "Xue Jiang", "Yihong Dong"], "venue": "arXiv 2026/07", "category": "", "published": "2026-07-26", "links": {"paper": "https://arxiv.org/abs/2607.23624", "github": "", "website": ""}}, "category": "software_security", "tags": ["empirical"], "summary": "An empirical study showing third-party API routers sitting between coding agents and LLM providers can silently inject or alter model outputs, causing agents to execute unintended repository-level actions with zero detection by existing client-side safeguards. The authors build SIDEL, a framework for trace recording, replay, injection, and defense evaluation across four coding agents.", "reason": "Empirical security study of the trusted path in agentic software development, i.e. security assurance around code-producing agents (software_security).", "source": "crawl"}, {"paper": {"id": "2607.23308", "title": "Towards LLM-assisted High-Quality Property Generation for Solidity Smart Contracts", "authors": ["Muhammad Wahid", "Shahzaib Khan", "Mashhood Ali", "Muhammad Hassan", "Muhammad Naiman Jalil", "Affan Rauf"], "venue": "arXiv 2026/07", "category": "", "published": "2026-07-25", "links": {"paper": "https://arxiv.org/abs/2607.23308", "github": "", "website": ""}}, "category": "software_security", "tags": [], "summary": "Uses LLMs with prompting strategies (zero-shot, few-shot, prompt chaining) to generate property-based testing properties for Solidity smart contracts, evaluated via mutation testing against human-written baselines. Gemini Pro 1.5 with prompt chaining approaches human-level property quality on some contracts.", "reason": "Generates fuzzing properties aimed at uncovering smart-contract vulnerabilities, matching software_security's vulnerability-targeted fuzzing scope.", "source": "crawl"}, {"paper": {"id": "2607.23710", "title": "The Illusion of Secure LLM Code: Closing the Security Gap via Iterative Reprompting", "authors": ["Ishpuneet Singh", "Shreyas Mahajan", "Gurjot Singh", "Maninder Singh"], "venue": "arXiv 2026/07", "category": "", "published": "2026-07-26", "links": {"paper": "https://arxiv.org/abs/2607.23710", "github": "", "website": ""}}, "category": "software_security", "tags": ["empirical"], "summary": "Empirically evaluates authentication code security from five AI coding assistants under varying prompting strategies via static analysis and penetration testing mapped to NIST SP 800-63B. Finds iterative reprompting is required to close persistent security gaps in LLM-generated code.", "reason": "Concerns secure code generation by AI assistants, matching software_security's 'secure code generation' scope.", "source": "crawl"}, {"paper": {"id": "2607.24964", "title": "ALIBI: Adaptive Agentic Attacks on LLM-Based Vulnerability Detectors via Adversarial Code Comments", "authors": ["Zixuan Wu", "Cristina Nita-Rotaru"], "venue": "arXiv 2026/07", "category": "", "published": "2026-07-27", "links": {"paper": "https://arxiv.org/abs/2607.24964", "github": "", "website": ""}}, "category": "software_security", "tags": [], "summary": "ALIBI is a coding agent that implements features, deliberately injects vulnerabilities, and crafts adversarial code comments to evade LLM-based vulnerability detectors, achieving over 90% attack success across real-world CVE-derived tasks. It also evaluates defenses like architectural isolation and comment sanitization.", "reason": "An agent manipulates code to attack vulnerability-detection tools, a security-assurance activity on code.", "source": "crawl"}, {"paper": {"id": "2607.25012", "title": "ZIMPAF & RedPhuzz: High-fidelity Web Application Fuzzing via Branch, Language Construct, and Function Call Monitoring", "authors": ["Tennov Simanjuntak", "Christoph Csallner"], "venue": "arXiv 2026/07", "category": "", "published": "2026-07-27", "links": {"paper": "https://arxiv.org/abs/2607.25012", "github": "", "website": ""}}, "category": "software_security", "tags": [], "summary": "ZIMPAF provides novel multi-granular runtime interpreter instrumentation for web applications, identifying tainted and vulnerable functions, which drives RedPhuzz, a targeted fuzzer using sanitization- and type-aware mutations. RedPhuzz detects all vulnerabilities across benchmark web apps while being substantially faster than its predecessor.", "reason": "A fuzzing system whose purpose is finding vulnerabilities in code, matching the security-assurance/fuzzing activity.", "source": "crawl"}, {"paper": {"id": "2607.25619", "title": "SkillGate: Cost Efficient Runtime Malicious Skill File Detection in Coding Agents", "authors": ["Rui Yang", "Michael Fu", "Kla Tantithamthavorn", "Chetan Arora", "Joey Chua"], "venue": "arXiv 2026/07", "category": "", "published": "2026-07-28", "links": {"paper": "https://arxiv.org/abs/2607.25619", "github": "", "website": ""}}, "category": "software_security", "tags": [], "summary": "SkillGate is a cost-efficient security gateway that screens AI coding-agent skill files for malicious content before installation, using a regex-prefilter plus LLM-judge pipeline evaluated on SkillsBench.", "reason": "Detects malicious/backdoor-injecting artifacts in the coding-agent supply chain, a security-assurance task on code (software_security).", "source": "crawl"}, {"paper": {"id": "2607.25890", "title": "Distributing Security Controls Through Harness Engineering", "authors": ["William Robert Gore"], "venue": "arXiv 2026/07", "category": "", "published": "2026-07-28", "links": {"paper": "https://arxiv.org/abs/2607.25890", "github": "", "website": ""}}, "category": "software_security", "tags": ["empirical"], "summary": "This paper builds SHarD, a distributable security harness for commercial AI coding agents implementing OS sandboxing, skill scanning, and tool restriction, and empirically tests it against an OWASP Agentic-Applications-derived test suite across four agent configurations.", "reason": "Security-assurance work guarding autonomous coding agents' operation falls under the software_security assurance activity.", "source": "crawl"}, {"paper": {"id": "2408.03910", "title": "CodexGraph: Bridging Large Language Models and Code Repositories via Code Graph Databases", "authors": ["Xiangyan Liu", "Bo Lan", "Zhiyuan Hu", "Yang Liu", "Zhicheng Zhang", "Fei Wang", "Michael Shieh", "Wenmeng Zhou"], "venue": "arXiv 2024/08", "category": "", "published": "2024-08-07", "links": {"paper": "https://arxiv.org/abs/2408.03910", "github": "", "website": ""}}, "category": "software_comprehension", "tags": [], "summary": "CodexGraph integrates LLM agents with graph databases built from code repositories, letting agents construct and execute structural graph queries for precise repository-scale code navigation and context retrieval.", "reason": "The served task is repository-scale code search/navigation, matching software_comprehension's code-search inclusion.", "source": "crawl"}, {"paper": {"id": "2607.25431", "title": "CodeNib: A Multi-View Data System for Serving Repository Context to Coding Agents", "authors": ["Zhongming Yu", "Hengjia Yu", "Boqin Yuan", "Shuting Zhao", "Yizhao Chen", "Aryan Dokania", "Mihir Jagtap", "Jiayu Chang", "Yitong Ma", "Yash Jayswal", "Wentao Ni", "Hejia Zhang", "Zhaoling Chen", "Gangda Deng", "Jishen Zhao"], "venue": "arXiv 2026/07", "category": "", "published": "2026-07-28", "links": {"paper": "https://arxiv.org/abs/2607.25431", "github": "", "website": ""}}, "category": "software_comprehension", "tags": [], "summary": "CodeNib builds reusable lexical, dense, and structural context views per repository commit to serve ranked search, symbol navigation, and bounded context to coding agents through one runtime. It shows large latency and token savings versus rebuilding indexes or using grep/read.", "reason": "Serves repository code search and navigation for agents, matching software_comprehension's code search/navigation task.", "source": "crawl"}, {"paper": {"id": "2607.25996", "title": "RepoReasoner: Evaluating Repository-Level Code Reasoning Ability of Long-Context Language Models", "authors": ["Yanlin Wang", "Suiquan Wang", "Yanli Wang", "Bowen Zhang", "Daya Guo", "Jiachi Chen", "Zibin Zheng"], "venue": "arXiv 2026/07", "category": "", "published": "2026-07-28", "links": {"paper": "https://arxiv.org/abs/2607.25996", "github": "", "website": ""}}, "category": "software_comprehension", "tags": ["benchmark"], "summary": "Introduces RepoReasoner, a benchmark evaluating long-context LLMs on repository-level code reasoning via output prediction and call-chain prediction tasks built from dynamic pytest tracing. Finds current models struggle with cross-file execution reasoning and dependency understanding.", "reason": "benchmark serving repository-level code understanding/QA, matching software_comprehension", "source": "crawl"}, {"paper": {"id": "2512.12507", "title": "ATLAS: Multi-View Code Representation Tool for C and C++ Source Programs", "authors": ["Jaid Monwar Chowdhury", "Ahmad Farhan Shahriar Chowdhury", "Humayra Binte Monwar", "Mahmuda Naznin"], "venue": "arXiv 2025/12", "category": "", "published": "2025-12-14", "links": {"paper": "https://arxiv.org/abs/2512.12507", "github": "", "website": ""}}, "category": "software_testing", "tags": [], "summary": "ATLAS is a command-line tool that extracts aligned AST, control-flow, and data-flow graphs from C/C++ source without requiring a full build, serving as the front-end for an LLM-based unit-test generation framework.", "reason": "A resource paper whose downstream task is proactive test generation, routed to software_testing per resource-follows-task rule.", "source": "crawl"}, {"paper": {"id": "2602.05270", "title": "PatchGuru: Patch Oracle Inference from Natural Language Artifacts", "authors": ["Thanh Le-Cong", "Bach Le", "Toby Murray", "Michael Pradel", "Cristian Cadar"], "venue": "arXiv 2026/02", "category": "", "published": "2026-02-05", "links": {"paper": "https://arxiv.org/abs/2602.05270", "github": "", "website": ""}}, "category": "software_testing", "tags": [], "summary": "PatchGuru uses LLMs to infer executable patch oracles from pull-request natural-language artifacts, validating patch behavior via runtime assertions and surfacing previously unknown bugs on real-world Python projects.", "reason": "Generates and executes correctness specifications to validate code changes, an assurance/verification task independent of a single reported defect, fitting software_testing.", "source": "crawl"}, {"paper": {"id": "2607.22880", "title": "Do Coverage and Mutation Scores of LLM-Generated Test Suites Correlate with Their Effectiveness? (Replicability Study)", "authors": ["Junda Zhao", "Shurui Zhou", "Eldan Cohen"], "venue": "arXiv 2026/07", "category": "", "published": "2026-07-24", "links": {"paper": "https://arxiv.org/abs/2607.22880", "github": "", "website": ""}}, "category": "software_testing", "tags": ["empirical"], "summary": "A large-scale replication study re-examining whether coverage and mutation scores of LLM-generated test suites correlate with real bug-detection effectiveness, finding the relationship is highly context-dependent.", "reason": "An empirical study of one activity (test generation evaluation), which per taxonomy routes to that leaf plus the empirical tag rather than the field-wide studies node.", "source": "crawl"}, {"paper": {"id": "2607.23963", "title": "RESTOR: Automated Test Oracle Generation for RESTful APIs via Reinforcement Learning", "authors": ["Xun Zhou", "Zhen Dong", "Mingyu Ren", "Qiang Li", "JunJie Li", "Sifan Wang", "Xiaolong Yu", "Chaofeng Sha", "Xin Peng"], "venue": "arXiv 2026/07", "category": "", "published": "2026-07-27", "links": {"paper": "https://arxiv.org/abs/2607.23963", "github": "", "website": ""}}, "category": "software_testing", "tags": [], "summary": "Restor fine-tunes a lightweight LLM via GRPO to generate executable test-oracle assertions for REST APIs from a single request-response trace, avoiding brittle rule-based templates. Evaluated on industrial ByteDance data, it raises automated test-case adoption from 74.1% to over 96% in production CI/CD.", "reason": "Produces test oracles/assertions for correctness assurance, matching software_testing's proactive test generation scope.", "source": "crawl"}, {"paper": {"id": "2607.24000", "title": "Industrial Practice of LLM-Based Test Case Carving and Assertion Generation (Experience Paper)", "authors": ["Haozhen You", "Zhen Dong", "Jingjing Wang", "Qiang Li", "Xin Peng"], "venue": "arXiv 2026/07", "category": "", "published": "2026-07-27", "links": {"paper": "https://arxiv.org/abs/2607.24000", "github": "", "website": ""}}, "category": "software_testing", "tags": ["empirical"], "summary": "NL2Test is an industrial tool that generates executable API regression tests from natural-language scenario descriptions and recorded traffic, combining LLM-based semantic interpretation with deterministic dependency and assertion validation. Deployed in production for 9 months, it generated over 3,000 test cases with high adoption.", "reason": "The task is proactive test generation for regression testing, matching software_testing; industry deployment makes it an experience/empirical report.", "source": "crawl"}, {"paper": {"id": "2607.25199", "title": "RIDGE: An Autonomous Framework for Validation and Method Discovery in LLM-Generated Option Pricing", "authors": ["Liexin Cheng", "Xue Cheng", "Shuaiqiang Liu", "Cornelis W. Oosterlee"], "venue": "arXiv 2026/07", "category": "", "published": "2026-07-28", "links": {"paper": "https://arxiv.org/abs/2607.25199", "github": "", "website": ""}}, "category": "software_testing", "tags": [], "summary": "RIDGE is an autonomous framework that validates LLM-generated option pricing code via structured no-arbitrage, stress, and consistency tests, iteratively refining implementations and accumulating diagnostic knowledge across models.", "reason": "Proactive, non-defect-specific correctness verification of generated code fits software_testing rather than a mere finance case study since the framework itself is the code-validation agent.", "source": "crawl"}, {"paper": {"id": "2607.25333", "title": "Specula: Scaling formal specifications for autonomous model checking of system code", "authors": ["Qian Cheng", "Saad Mohammad Rafid Pial", "Ruize Tang", "Yiming Su", "Emilie Ma", "Finn Hackett", "Ivan Beschastnikh", "Yu Huang", "Tianyin Xu"], "venue": "arXiv 2026/07", "category": "", "published": "2026-07-28", "links": {"paper": "https://arxiv.org/abs/2607.25333", "github": "", "website": ""}}, "category": "software_testing", "tags": [], "summary": "Specula is an autonomous LLM coding-agent system that generates TLA+ formal specifications for large system codebases and uses them for model checking. It found 249 bugs across 48 open-source projects via self-evolving refinement loops.", "reason": "Formal verification and proof-assisted correctness of system code performed by an autonomous coding agent falls under software_testing.", "source": "crawl"}, {"paper": {"id": "2607.25647", "title": "KQFuzz: Knowledge-Guided Fuzzing for Quantum Libraries via Large Language Models", "authors": ["Fuyuan Xia", "Qixin Zhang", "Chenhao Ying", "Haojin Zhu", "Shuai Wang", "Yuan Luo", "Pingchuan Ma", "Yuxuan Du"], "venue": "arXiv 2026/07", "category": "", "published": "2026-07-28", "links": {"paper": "https://arxiv.org/abs/2607.25647", "github": "", "website": ""}}, "category": "software_testing", "tags": [], "summary": "KQFuzz is a knowledge-guided LLM fuzzer that generates and mutates quantum programs to test quantum libraries like Qiskit, PennyLane, and Cirq. It found 13 previously unknown bugs, most confirmed and fixed by developers.", "reason": "Proactive automated fuzzing for general correctness bugs in software libraries fits software_testing.", "source": "crawl"}, {"paper": {"id": "2607.23340", "title": "ABISS: Evaluating Text-to-SQL Systems Through Agent Interaction", "authors": ["Giovanni Sullutrone", "Luca Sala", "Sania Aftar", "Georgia Koutrika", "Sonia Bergamaschi"], "venue": "arXiv 2026/07", "category": "", "published": "2026-07-25", "links": {"paper": "https://arxiv.org/abs/2607.23340", "github": "", "website": ""}}, "category": "database", "tags": ["benchmark"], "summary": "ABISS introduces a taxonomy of ambiguous/unanswerable Text-to-SQL questions, a multi-agent pipeline to generate realistic benchmark data, and a dynamic multi-turn simulation environment for evaluating Text-to-SQL agents' interaction and clarification abilities.", "reason": "Benchmark for interactive Text-to-SQL agents whose deliverable is SQL queries, routing to database.", "source": "crawl"}, {"paper": {"id": "2607.04726", "title": "What You See Is What You Get: Observation-Aligned Supervision for Chart-to-Code Generation", "authors": ["Tianhao Niu", "Qingfu Zhu", "Wanxiang Che"], "venue": "arXiv 2026/07", "category": "", "published": "2026-07-06", "links": {"paper": "https://arxiv.org/abs/2607.04726", "github": "", "website": ""}}, "category": "graphics", "tags": [], "summary": "Identifies latent-observation mismatches (aggregation, normalization, projection) that make gold plotting code an invalid supervision target for chart-to-code generation, and proposes Observation-Aligned Supervision using visually constrained quantities instead. Improves observable-value recovery for VLMs on ChartMimic and ChartX.", "reason": "Serves chart-from-spec code generation where the visual artifact's code is the deliverable, matching the graphics leaf.", "source": "crawl"}, {"paper": {"id": "2607.24766", "title": "Crystalis: Progressive Nucleation and Semantic Annealing for Coordinated Multi-View Visualization Generation", "authors": ["Dazhen Deng", "Zhaoping He", "Xin Qian", "Xiaotong Wang", "Zi Ying", "Yingcai Wu"], "venue": "arXiv 2026/06", "category": "", "published": "2026-06-07", "links": {"paper": "https://arxiv.org/abs/2607.24766", "github": "", "website": ""}}, "category": "graphics", "tags": [], "summary": "Crystalis is a framework that decomposes coordinated multi-view visualizations into a dependency graph of data, visualization, and interaction queries, using progressive nucleation and semantic annealing to reliably generate structurally correct executable visualization code from specifications.", "reason": "The task's purpose is producing chart/visualization code from a specification, matching the graphics artifact leaf.", "source": "crawl"}, {"paper": {"id": "2606.16497", "title": "daVinci-kernel: Co-Evolving Skill Selection, Summarization, and Utilization via RL for GPU Kernel Optimization", "authors": ["Dayuan Fu", "Mohan Jiang", "Tongyu Wang", "Dian Yang", "Jiarui Hu", "Liming Liu", "Jinlong Hou", "Pengfei Liu"], "venue": "arXiv 2026/06", "category": "", "published": "2026-06-15", "links": {"paper": "https://arxiv.org/abs/2606.16497", "github": "", "website": ""}}, "category": "systems", "tags": ["model"], "summary": "daVinci-kernel is an RL framework that jointly trains skill-selection, kernel-generation, and skill-summarization agents sharing one LLM backbone to optimize CUDA/Triton GPU kernels, evaluated on KernelBench. It outperforms prior RL-trained kernel models across benchmark levels.", "reason": "KernelBench-style GPU kernel code generation routes to systems per benchmark-routing rule.", "source": "crawl"}, {"paper": {"id": "2607.23089", "title": "Compiler-Grounded Hierarchical Diagnosis for LLM-Based Triton Kernel Optimization", "authors": ["Dongjie Chen", "Ping Zhao", "Bohua Zhan", "Yulong Wang", "Shushu Chen", "Liangjun Feng", "Hao Zhou", "Min Shen", "Linmu Wang", "Weijia Sheng", "Xiangyu Wei", "Weijie Ding", "Jianhui Huang", "Yaoqing Gao"], "venue": "arXiv 2026/07", "category": "", "published": "2026-07-25", "links": {"paper": "https://arxiv.org/abs/2607.23089", "github": "", "website": ""}}, "category": "systems", "tags": [], "summary": "Presents a compiler-grounded hierarchical diagnosis framework that escalates from profiling to IR-level compiler analysis to produce evidence-backed source rewrites for Triton kernels on NPUs. Achieves large geometric-mean speedups on an Ascend-950-derived kernel benchmark.", "reason": "Optimizes accelerator kernel code, analogous to KernelBench precedent which routes to systems.", "source": "crawl"}, {"paper": {"id": "2607.24714", "title": "Efficient LLM-Generated Shuttling Compilers for Complex Trapped-Ion Architectures", "authors": ["Fabian Kreppel", "Reza Salkhordeh", "Ferdinand Schmidt-Kaler", "André Brinkmann"], "venue": "arXiv 2026/07", "category": "", "published": "2026-07-27", "links": {"paper": "https://arxiv.org/abs/2607.24714", "github": "", "website": ""}}, "category": "systems", "tags": [], "summary": "A frontier LLM (Claude) is used to generate and iteratively refine full Python shuttling compilers for trapped-ion quantum architectures, matching or beating hand-crafted compilers in shuttling-timestep efficiency.", "reason": "The deliverable is compiler/runtime code, which falls under the systems leaf per its inclusion of compiler and runtime code.", "source": "crawl"}, {"paper": {"id": "2607.24762", "title": "Kernel Forge: An Agent Harness for LLM-based Generation and Optimization of CUDA Kernels", "authors": ["Joshua Brodsky", "Dhravid Kumar", "Savini Kashmira", "Jayanaka Danatanarayana", "Jason Mars", "Krisztian Flautner", "Lingjia Tang"], "venue": "arXiv 2026/06", "category": "", "published": "2026-06-02", "links": {"paper": "https://arxiv.org/abs/2607.24762", "github": "", "website": ""}}, "category": "systems", "tags": [], "summary": "Kernel Forge is an end-to-end agentic harness that uses MCTS-guided LLM optimization to generate and integrate CUDA kernels directly into unmodified PyTorch models, outperforming eager-mode execution across vision, diffusion, and LLM workloads.", "reason": "The agent produces low-level GPU/systems code (KernelBench-style task), so it routes to the systems leaf per benchmark-routing precedent.", "source": "crawl"}]