Try Iris Benchmarks

[Illviljan]

• Closes #xxxx
• Tests added
• User visible changes (including notable bug fixes) are documented in whats-new.rst
• New functions/methods are listed in api.rst

https://github.com/SciTools/workflows
https://github.com/conda/conda-lock
conda lock -f environment.yml -p linux-64

[Illviljan]

Strange errors now. I wonder if parts of it (the version locks for example) needs to be merged and sit in main for a while for performance runs to start working?

[trexfeathers]

This error:

NotImplementedError: No environment setup is known for this commit of Iris.

Is because this block is turning up False. I assume Nox is not appropriate for Xarray. You could write whatever you like in this file though 👍

xarray/benchmarks/asv_delegated.py

Lines 71 to 88 in 3eab7c0

	if noxfile.is_file():
	# Our noxfile originally did not support `--install-only` - you
	# could either run the tests, or run nothing at all. Adding
	# `run_always` to `prepare_venv` enabled environment setup without
	# running tests.
	noxfile_tree = ast.parse(source=noxfile.read_text())
	prep_session = next(
	filter(
	lambda node: getattr(node, "name", "") == "prepare_venv",
	ast.walk(noxfile_tree),
	)
	)
	prep_session_code = ast.unparse(prep_session)
	if (
	"session.run(" not in prep_session_code
	and "session.run_always(" in prep_session_code
	):
	mode = Mode.NOX

[Illviljan]

Yeah it looks to me I need to merge the noxfile and locks to get useful commits that asv can checkout.

·· Running delegated environment management for: delegated-pyDELEGATED at commit: 490f1e09

[trexfeathers]

Yeah it looks to me I need to merge the noxfile and locks to get useful commits that asv can checkout.

·· Running delegated environment management for: delegated-pyDELEGATED at commit: 490f1e09

Yes, when you want it working 'for real' that's probably the most sensible setup. But the structure of asv_delegated should¹ allow you to write whatever custom rules you like for managing environments, so you could do something completely different as a proof-of-concept, just to get it working.

Footnotes

1. 90% sure! ↩