Skip to content

Update speed benchmarks #8

Description

@annaveronika

We are happy to contribute here.

We suggest following updates:

  • using newest versions of the libraries
  • using 1000 iterations instead of 500, because when you are using 500 iterations preprocessing might be a bottleneck, which is not what you want to measure. Plus you are usually using GPU-s for large datasets where it's not enough to run for 500
  • using two different aws configurations, one with 8 V100 another without GPU-s for running on CPU. It is cheaper this way, and you don't need to pay for GPU-s when you are not using them
  • run 5 times every train on CPU, because for all the libraries CPU time might differ by up to 30% from run to run. So the benchmark will contain average time and standard deviation for CPU

Are you OK with these changes?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions