Skip to content

Update results.json #321

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
GeorgeTsoukalas merged 1 commit into from
Dec 28, 2025
Merged

Update results.json #321

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions docs/results.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -331,5 +331,15 @@
"size": 0,
"condensed-compute-budget": "Max $300 per problem",
"note": "pass@2 — First run: avg 1834 tool calls, Second run: avg 1218 tool calls, avg commit: 2a4fc44, version: 4.22, problems: 1962 A3, 1962 A4, 1962 A5, 1962 A6, 1962 B1, 1962 B2, 1962 B3, 1962 B5, 1963 A2, 1963 A4, 1963 B1, 1963 B2, 1963 B3, 1963 B5, 1963 B6, 1964 A2, 1964 A4, 1964 A5, 1964 B1, 1964 B2, 1964 B3, 1964 B5, 1964 B6, 1965 A2, 1965 A3, 1965 A4, 1965 A5, 1965 A6, 1965 B1, 1965 B2, 1965 B3, 1965 B4, 1965 B5, 1965 B6, 1966 A1, 1966 A3, 1966 A4, 1966 A5, 1966 A6, 1966 B2, 1966 B3, 1966 B4, 1966 B6, 1967 A1, 1967 A3, 1967 B2, 1967 B3, 1967 B4, 1967 B5, 1968 A1, 1968 A2, 1968 A4, 1968 A5, 1968 B1, 1968 B2, 1968 B5, 1968 B6, 1969 A1, 1969 A2, 1969 A4, 1969 A5, 1969 A6, 1969 B1, 1969 B2, 1969 B3, 1969 B5, 1969 B6, 1970 A1, 1970 A2, 1970 A3, 1970 A4, 1970 B1, 1970 B2, 1970 B3, 1970 B4, 1970 B5, 1971 A1, 1971 A2, 1971 A4, 1971 A5, 1971 A6, 1971 B1, 1971 B2, 1971 B3, 1971 B6, 1972 A1, 1972 A2, 1972 A5, 1972 A6, 1972 B2, 1972 B3, 1972 B4, 1972 B6, 1973 A2, 1973 A3, 1973 A4, 1973 A6, 1973 B1, 1973 B2, 1973 B3, 1973 B4, 1974 A1, 1974 A3, 1974 A4, 1974 A6, 1974 B2, 1974 B3, 1974 B6, 1975 A1, 1975 A2, 1975 A3, 1975 A4, 1975 B1, 1975 B3, 1975 B4, 1975 B5, 1975 B6, 1976 A2, 1976 A3, 1976 A4, 1976 A6, 1976 B2, 1976 B5, 1976 B6, 1977 A1, 1977 A3, 1977 A4, 1977 A5, 1977 B1, 1977 B3, 1977 B5, 1978 A1, 1978 A2, 1978 A4, 1978 A5, 1978 A6, 1978 B3, 1978 B4, 1978 B5, 1978 B6, 1979 A1, 1979 A2, 1979 A3, 1979 A4, 1979 A5, 1979 A6, 1979 B6, 1980 A2, 1980 A3, 1980 A5, 1980 A6, 1980 B1, 1980 B3, 1980 B4, 1980 B5, 1980 B6, 1981 A1, 1981 A5, 1981 B1, 1981 B2, 1981 B3, 1981 B4, 1981 B5, 1982 A2, 1982 A3, 1982 A4, 1982 A5, 1982 B2, 1982 B4, 1982 B5, 1983 A1, 1983 A3, 1983 A4, 1983 A5, 1983 B2, 1983 B4, 1983 B5, 1983 B6, 1984 A2, 1984 A3, 1984 A5, 1984 B1, 1984 B2, 1984 B3, 1985 A1, 1985 A3, 1985 A4, 1985 A5, 1985 A6, 1985 B1, 1985 B2, 1985 B3, 1985 B5, 1985 B6, 1986 A1, 1986 A2, 1986 A3, 1986 A5, 1986 B1, 1986 B2, 1986 B3, 1986 B4, 1986 B5, 1986 B6, 1987 A1, 1987 A2, 1987 B1, 1987 B3, 1987 B4, 1987 B5, 1987 B6, 1988 A1, 1988 A2, 1988 A3, 1988 A5, 1988 A6, 1988 B1, 1988 B2, 1988 B3, 1988 B4, 1988 B6, 1989 A1, 1989 A2, 1989 A3, 1989 A6, 1989 B2, 1989 B3, 1989 B4, 1990 A1, 1990 A2, 1990 A4, 1990 A5, 1990 A6, 1990 B1, 1990 B2, 1990 B3, 1990 B5, 1991 A2, 1991 A3, 1991 A4, 1991 A5, 1991 B1, 1991 B2, 1991 B4, 1991 B5, 1991 B6, 1992 A1, 1992 A2, 1992 A3, 1992 A4, 1992 A5, 1992 B1, 1992 B2, 1992 B3, 1992 B5, 1993 A1, 1993 A2, 1993 A3, 1993 A4, 1993 A5, 1993 B1, 1993 B4, 1993 B5, 1993 B6, 1994 A1, 1994 A3, 1994 A4, 1994 A5, 1994 A6, 1994 B1, 1994 B2, 1994 B3, 1994 B4, 1994 B5, 1994 B6, 1995 A1, 1995 A2, 1995 A3, 1995 A4, 1995 A5, 1995 B1, 1995 B3, 1995 B4, 1995 B6, 1996 A2, 1996 A3, 1996 A4, 1996 A5, 1996 B1, 1996 B2, 1996 B4, 1996 B5, 1997 A3, 1997 A4, 1997 B1, 1997 B2, 1997 B4, 1997 B5, 1998 A2, 1998 A3, 1998 A4, 1998 A5, 1998 B1, 1998 B2, 1998 B4, 1998 B5, 1998 B6, 1999 A1, 1999 A2, 1999 A3, 1999 A4, 1999 A5, 1999 A6, 1999 B2, 1999 B4, 1999 B5, 1999 B6, 2000 A1, 2000 A2, 2000 A4, 2000 A5, 2000 A6, 2000 B1, 2000 B2, 2000 B4, 2000 B5, 2001 A1, 2001 A3, 2001 A5, 2001 B1, 2001 B2, 2001 B3, 2001 B4, 2001 B5, 2001 B6, 2002 A1, 2002 A2, 2002 A3, 2002 A5, 2002 A6, 2002 B3, 2002 B5, 2002 B6, 2003 A1, 2003 A2, 2003 A3, 2003 A4, 2003 B1, 2003 B2, 2003 B3, 2003 B4, 2003 B6, 2004 A1, 2004 A3, 2004 A4, 2004 B1, 2004 B2, 2004 B4, 2004 B5, 2005 A1, 2005 A4, 2005 A5, 2005 B1, 2005 B2, 2005 B3, 2005 B4, 2006 A1, 2006 A3, 2006 A4, 2006 A5, 2006 B1, 2006 B2, 2006 B4, 2006 B5, 2006 B6, 2007 A1, 2007 A4, 2007 A5, 2007 B1, 2007 B2, 2007 B5, 2008 A1, 2008 A3, 2008 A4, 2008 A5, 2008 A6, 2008 B1, 2008 B2, 2008 B3, 2008 B4, 2008 B5, 2009 A1, 2009 A2, 2009 A3, 2009 A4, 2009 A5, 2009 B1, 2009 B2, 2009 B3, 2009 B5, 2009 B6, 2010 A1, 2010 A2, 2010 A4, 2010 A5, 2010 A6, 2010 B1, 2010 B2, 2010 B3, 2010 B4, 2010 B5, 2010 B6, 2011 A2, 2011 A4, 2011 B1, 2011 B2, 2011 B3, 2011 B4, 2011 B5, 2011 B6, 2012 A1, 2012 A2, 2012 A3, 2012 A4, 2012 A5, 2012 B1, 2012 B3, 2012 B5, 2013 A3, 2013 A4, 2013 B1, 2013 B2, 2013 B4, 2014 A1, 2014 A2, 2014 A3, 2014 A4, 2014 A5, 2014 A6, 2014 B1, 2014 B4, 2015 A1, 2015 A2, 2015 A3, 2015 A4, 2015 A6, 2015 B1, 2015 B3, 2016 A1, 2016 A2, 2016 A5, 2016 A6, 2016 B1, 2016 B3, 2017 A1, 2017 A2, 2017 A3, 2017 B1, 2018 A1, 2018 A3, 2018 A5, 2018 A6, 2018 B2, 2018 B3, 2018 B4, 2018 B5, 2018 B6, 2019 A1, 2019 A3, 2019 B2, 2019 B3, 2019 B4, 2019 B6, 2020 A1, 2020 A2, 2020 A3, 2020 B1, 2021 A1, 2021 A2, 2021 A3, 2021 A5, 2021 B2, 2021 B4, 2021 B5, 2022 A1, 2022 A2, 2022 A3, 2022 B1, 2022 B3, 2023 A1, 2023 A2, 2023 A3, 2023 A4, 2023 A6, 2023 B2, 2024 A1, 2024 A2, 2024 A4, 2024 B1, 2024 B3, second pass 1961 A1, 1962 B6, 1964 A3, 1964 A6, 1965 A1, 1966 A2, 1966 B1, 1966 B5, 1967 A2, 1967 A6, 1968 A6, 1971 A3, 1972 A3, 1972 B1, 1972 B5, 1973 A1, 1974 B4, 1975 B2, 1976 B1, 1977 A6, 1978 B2, 1979 B2, 1979 B3, 1981 A3, 1982 A6, 1982 B3, 1983 A6, 1984 A6, 1984 B5, 1986 A6, 1987 A4, 1987 A5, 1987 A6, 1987 B2, 1988 A4, 1988 B5, 1989 B1, 1990 B4, 1992 B4, 1992 B6, 1993 A6, 1993 B3, 1996 A6, 1997 A6, 1997 B3, 1998 A6, 2004 A5, 2004 A6, 2004 B6, 2005 A2, 2005 B5, 2005 B6, 2007 A2, 2007 A3, 2007 B3, 2007 B4, 2007 B6, 2008 B6, 2009 B4, 2011 A3, 2011 A5, 2011 A6, 2012 A6, 2012 B4, 2012 B6, 2013 A2, 2013 B3, 2013 B5, 2014 B2, 2014 B3, 2014 B6, 2015 A5, 2015 B2, 2015 B5, 2015 B6, 2016 A3, 2016 B2, 2016 B4, 2016 B5, 2016 B6, 2017 A4, 2017 B2, 2017 B3, 2017 B4, 2017 B6, 2018 A2, 2018 A4, 2018 B1, 2019 B1, 2020 A5, 2020 A6, 2020 B4, 2020 B5, 2020 B6, 2021 A4, 2021 B3, 2022 B2, 2022 B4, 2022 B5, 2022 B6, 2023 A5, 2023 B1, 2023 B3, 2023 B5, 2023 B6, 2024 A5, 2024 A6, 2024 B4, 2024 B5, 2024 B6, 2025 A1, 2025 A2, 2025 A3, 2025 A4, 2025 A6, 2025 B1, 2025 B2, 2025 B3, 2025 B4, 2025 B5"
},
"Seed-Prover 1.5 (ByteDance)": {
"link": "https://arxiv.org/abs/2512.17260",
"open-data": "NONE",
"num-solved": {
"lean-wsolution": 581
},
"size": 0,
"condensed-compute-budget": "10 H20 days per problem",
"note": "commit: 6df1d4c, version: 4.22, problems: 1962 A1, 1962 A3, 1962 A4, 1962 A5, 1962 A6, 1962 B1, 1962 B2, 1962 B3, 1962 B5, 1963 A2, 1963 A4, 1963 B1, 1963 B2, 1963 B3, 1963 B5, 1963 B6, 1964 A1, 1964 A2, 1964 A4, 1964 A5, 1964 B1, 1964 B2, 1964 B3, 1964 B5, 1964 B6, 1965 A1, 1965 A2, 1965 A3, 1965 A4, 1965 A5, 1965 A6, 1965 B1, 1965 B2, 1965 B3, 1965 B4, 1965 B5, 1965 B6, 1966 A1, 1966 A2, 1966 A3, 1966 A4, 1966 A5, 1966 A6, 1966 B2, 1966 B3, 1966 B4, 1966 B5, 1966 B6, 1967 A1, 1967 A2, 1967 A3, 1967 B2, 1967 B3, 1967 B4, 1967 B5, 1968 A1, 1968 A2, 1968 A3, 1968 A4, 1968 A5, 1968 A6, 1968 B1, 1968 B2, 1968 B5, 1968 B6, 1969 A1, 1969 A2, 1969 A4, 1969 A5, 1969 A6, 1969 B1, 1969 B2, 1969 B3, 1969 B5, 1969 B6, 1970 A1, 1970 A2, 1970 A3, 1970 A4, 1970 B1, 1970 B2, 1970 B3, 1970 B4, 1970 B5, 1971 A1, 1971 A2, 1971 A3, 1971 A4, 1971 A5, 1971 A6, 1971 B1, 1971 B2, 1971 B3, 1971 B6, 1972 A1, 1972 A2, 1972 A5, 1972 A6, 1972 B1, 1972 B2, 1972 B3, 1972 B4, 1972 B5, 1972 B6, 1973 A1, 1973 A2, 1973 A3, 1973 A4, 1973 A6, 1973 B1, 1973 B2, 1973 B3, 1973 B4, 1974 A1, 1974 A3, 1974 A4, 1974 A6, 1974 B2, 1974 B3, 1974 B4, 1974 B5, 1974 B6, 1975 A1, 1975 A2, 1975 A3, 1975 A4, 1975 A5, 1975 B1, 1975 B3, 1975 B4, 1975 B5, 1975 B6, 1976 A2, 1976 A3, 1976 A4, 1976 A6, 1976 B1, 1976 B2, 1976 B5, 1976 B6, 1977 A1, 1977 A3, 1977 A4, 1977 A5, 1977 A6, 1977 B1, 1977 B3, 1977 B5, 1978 A1, 1978 A2, 1978 A4, 1978 A5, 1978 A6, 1978 B2, 1978 B3, 1978 B4, 1978 B5, 1978 B6, 1979 A1, 1979 A2, 1979 A3, 1979 A4, 1979 A5, 1979 A6, 1979 B2, 1979 B3, 1979 B6, 1980 A2, 1980 A3, 1980 A5, 1980 A6, 1980 B1, 1980 B3, 1980 B4, 1980 B5, 1980 B6, 1981 A1, 1981 A3, 1981 A5, 1981 B1, 1981 B2, 1981 B3, 1981 B4, 1981 B5, 1982 A2, 1982 A3, 1982 A4, 1982 A5, 1982 A6, 1982 B2, 1982 B3, 1982 B4, 1982 B5, 1983 A1, 1983 A3, 1983 A4, 1983 A5, 1983 A6, 1983 B2, 1983 B4, 1983 B5, 1983 B6, 1984 A2, 1984 A3, 1984 A5, 1984 A6, 1984 B1, 1984 B2, 1984 B3, 1984 B5, 1985 A1, 1985 A3, 1985 A4, 1985 A5, 1985 A6, 1985 B1, 1985 B2, 1985 B3, 1985 B5, 1985 B6, 1986 A1, 1986 A2, 1986 A3, 1986 A4, 1986 A5, 1986 A6, 1986 B1, 1986 B2, 1986 B3, 1986 B4, 1986 B5, 1986 B6, 1987 A1, 1987 A2, 1987 A4, 1987 A5, 1987 A6, 1987 B1, 1987 B2, 1987 B3, 1987 B4, 1987 B5, 1987 B6, 1988 A1, 1988 A2, 1988 A3, 1988 A4, 1988 A5, 1988 A6, 1988 B1, 1988 B2, 1988 B3, 1988 B4, 1988 B5, 1988 B6, 1989 A1, 1989 A2, 1989 A3, 1989 A6, 1989 B1, 1989 B2, 1989 B3, 1989 B4, 1990 A1, 1990 A2, 1990 A4, 1990 A5, 1990 A6, 1990 B1, 1990 B2, 1990 B3, 1990 B5, 1991 A2, 1991 A3, 1991 A4, 1991 A5, 1991 A6, 1991 B1, 1991 B2, 1991 B4, 1991 B5, 1991 B6, 1992 A1, 1992 A2, 1992 A3, 1992 A4, 1992 A5, 1992 B1, 1992 B2, 1992 B3, 1992 B4, 1992 B5, 1992 B6, 1993 A1, 1993 A2, 1993 A3, 1993 A4, 1993 A5, 1993 A6, 1993 B1, 1993 B3, 1993 B4, 1993 B5, 1994 A1, 1994 A3, 1994 A4, 1994 A5, 1994 A6, 1994 B1, 1994 B2, 1994 B3, 1994 B4, 1994 B5, 1994 B6, 1995 A1, 1995 A2, 1995 A3, 1995 A4, 1995 B1, 1995 B3, 1995 B4, 1996 A2, 1996 A3, 1996 A4, 1996 A5, 1996 A6, 1996 B1, 1996 B2, 1996 B3, 1996 B4, 1996 B5, 1997 A3, 1997 A4, 1997 A6, 1997 B1, 1997 B2, 1997 B3, 1997 B4, 1997 B5, 1998 A2, 1998 A3, 1998 A4, 1998 A5, 1998 A6, 1998 B1, 1998 B2, 1998 B4, 1998 B5, 1998 B6, 1999 A1, 1999 A2, 1999 A3, 1999 A4, 1999 A5, 1999 A6, 1999 B2, 1999 B4, 1999 B5, 1999 B6, 2000 A1, 2000 A2, 2000 A4, 2000 A5, 2000 A6, 2000 B1, 2000 B2, 2000 B4, 2000 B5, 2001 A1, 2001 A3, 2001 A5, 2001 B1, 2001 B2, 2001 B3, 2001 B4, 2001 B5, 2001 B6, 2002 A1, 2002 A2, 2002 A3, 2002 A5, 2002 A6, 2002 B3, 2002 B5, 2002 B6, 2003 A1, 2003 A2, 2003 A3, 2003 A4, 2003 A5, 2003 B1, 2003 B2, 2003 B3, 2003 B4, 2003 B6, 2004 A1, 2004 A3, 2004 A4, 2004 A6, 2004 B1, 2004 B2, 2004 B4, 2004 B5, 2004 B6, 2005 A1, 2005 A4, 2005 A5, 2005 B1, 2005 B2, 2005 B3, 2005 B4, 2005 B5, 2005 B6, 2006 A1, 2006 A4, 2006 A5, 2006 B1, 2006 B2, 2006 B4, 2006 B5, 2006 B6, 2007 A1, 2007 A2, 2007 A3, 2007 A4, 2007 A5, 2007 B1, 2007 B2, 2007 B3, 2007 B5, 2007 B6, 2008 A1, 2008 A4, 2008 A5, 2008 A6, 2008 B1, 2008 B2, 2008 B3, 2008 B4, 2008 B5, 2008 B6, 2009 A1, 2009 A2, 2009 A3, 2009 A4, 2009 A5, 2009 B1, 2009 B2, 2009 B3, 2009 B4, 2009 B5, 2010 A1, 2010 A2, 2010 A3, 2010 A4, 2010 A5, 2010 A6, 2010 B1, 2010 B2, 2010 B3, 2010 B4, 2010 B5, 2010 B6, 2011 A2, 2011 A3, 2011 A4, 2011 A6, 2011 B1, 2011 B2, 2011 B3, 2011 B4, 2011 B5, 2012 A1, 2012 A2, 2012 A3, 2012 A4, 2012 A5, 2012 A6, 2012 B1, 2012 B3, 2012 B4, 2012 B5, 2013 A2, 2013 A3, 2013 A4, 2013 A6, 2013 B1, 2013 B2, 2013 B3, 2013 B4, 2014 A1, 2014 A2, 2014 A3, 2014 A4, 2014 A5, 2014 A6, 2014 B1, 2014 B2, 2014 B3, 2014 B4, 2015 A1, 2015 A2, 2015 A3, 2015 A4, 2015 A5, 2015 A6, 2015 B1, 2015 B3, 2015 B6, 2016 A1, 2016 A2, 2016 A3, 2016 A5, 2016 A6, 2016 B1, 2016 B2, 2016 B3, 2016 B4, 2016 B5, 2016 B6, 2017 A1, 2017 A2, 2017 A3, 2017 A4, 2017 B1, 2017 B2, 2017 B4, 2018 A1, 2018 A2, 2018 A3, 2018 A5, 2018 A6, 2018 B1, 2018 B2, 2018 B3, 2018 B4, 2018 B5, 2018 B6, 2019 A1, 2019 A3, 2019 B2, 2019 B3, 2019 B4, 2019 B6, 2020 A1, 2020 A2, 2020 A3, 2020 A5, 2020 A6, 2020 B1, 2020 B4, 2020 B5, 2021 A1, 2021 A2, 2021 A3, 2021 A5, 2021 B2, 2021 B3, 2021 B4, 2021 B5, 2022 A1, 2022 A2, 2022 A3, 2022 B1, 2022 B2, 2022 B3, 2022 B5, 2023 A1, 2023 A2, 2023 A3, 2023 A4, 2023 A5, 2023 A6, 2023 B2, 2023 B6, 2024 A1, 2024 A2, 2024 A4, 2024 B1, 2024 B3, 2024 B6"
}
}