Benchmarking with asv

[VeckoTheGecko]

Todo

• remove example benchmarks
• Add integration test benchmarks (See asv_bench/benchmarks/benchmarks_integration.py for an example.)
  • advection 2d
  • ARGO float example
  • tutorial_nemo_curvilinear.ipynb (See asv_bench/benchmarks/benchmarks_particle_execution.py for an example.)
• Add more detailed timing benchmarks
  • time the execution of 1000 particles for 1 time step
  • time the execution of 1000 particles for 100 time steps

When applicable, split into different phases. We could, e.g., go for a .setup() method for fieldset creating, and a .time_execute() method for the particle execution.

---

This PR introduces benchmarking infrastructure to the project via asv. Benchmarks can be run on a pull request by adding the run-benchmarks label to it. Two environments will be created in the CI runner with the prior and proposed changes, and both suites of benchmarks will be run and compared against each other.

Note that this PR only has example benchmarks for the timebeing until we can discuss benchmarks of interest.

The running of the benchmarks in CI is only one aspect of the benchmarking (ie, only for core parcels functionality). Using asv, we can create different suites of benchmarks (e.g., one for CI, and one for more heavy simulations). The benefit of using asv is everything else that comes out of the box with it, some being:

• being able to run benchmarks easily across several commits, visualising them in a dashboard
• easily create and manage these benchmarking environments
• profiling support to dive into locations of slowdowns
• community support

Changes:

• asv configuration (conf file, benchmarks folder, and CI workflow)
• asv documentation (available via the community page in the maintainer section)

---

I have done some testing of the PR label workflow in VeckoTheGecko#10 . We can only test this for PRs in OceanParcels/parcels when its in master

---

Related to #1712

[VeckoTheGecko]

@erikvansebille On the topic of performance, are you also experiencing it taking something like 10s occasionally to run import parcels?

[erikvansebille]

@erikvansebille On the topic of performance, are you also experiencing it taking something like 10s occasionally to run import parcels?

Yes, I also experience this slow import sometimes. Not sure why...

[VeckoTheGecko]

Setting to draft until we have some actual benchmarks that we can include in this.

[VeckoTheGecko]

From meeting:
We can use tutorial_nemo_curvilinear.ipynb as well

[erikvansebille]

The Argo tutorial at https://docs.oceanparcels.org/en/latest/examples/tutorial_Argofloats.html is also a quite nice simulation for benchmarking, as it has a 'complex' kernel. It took approximately 20s to run on v3.1 in JIT mode, and no takes 50s on my local computer to run in Scipy-mode.

@danliba or @willirath, could you add the Argo tutorial to the benchmark stack?

[willirath]

@danliba I've left a few comments.