Skip to content

Update: card metrics query #420

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ibraheem-abe merged 6 commits into from
Jun 23, 2026
Merged

Update: card metrics query #420

Changes from 3 commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
097b572
Merge pull request #414 from ridgesai/release/0.1.1
camfairchild Jun 9, 2026
4e74430
Merge pull request #417 from ridgesai/release/0.1.2
ibraheem-abe Jun 15, 2026
6ee217a
update eval query
ibraheem-abe Jun 15, 2026
9d6ca59
feat: :sparkles: Add corresponding agent_id for efficiency metrics
jmnmv12 Jun 19, 2026
f26fe30
feat: :sparkles: Remove rank for cancelled agents and transform their…
jmnmv12 Jun 19, 2026
ca18854
test: :white_check_mark: Update tests to comply with schemas changes
jmnmv12 Jun 19, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 7 additions & 9 deletions queries/evaluation_set.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -417,8 +417,10 @@ async def get_evaluation_set_score_stats(conn: DatabaseConnection, set_id: int)
- agents_beating_previous_best: The count of agents in the current set that beat the previous best score.
"""
return await conn.fetchrow(
"""
WITH prev_best AS (
f"""
WITH {_SQL_SET_WINDOW_CTE},
{_sql_agents_in_window_cte("a.agent_id, a.status")},
prev_best AS (
SELECT
MAX(final_score) AS score
FROM
Expand Down Expand Up @@ -471,15 +473,11 @@ async def get_evaluation_set_score_stats(conn: DatabaseConnection, set_id: int)
) :: int AS agents_beating_previous_best
FROM
agent_scores s
JOIN agents_in_window aiw ON aiw.agent_id = s.agent_id
WHERE
s.set_id = $1
AND s.agent_id NOT IN (
SELECT
agent_id
FROM
benchmark_agent_ids
)
AND s.status = 'finished'
AND aiw.status = 'finished'
AND NOT aiw.disqualified
""",
set_id,
)
Expand Down
Loading