Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Benchmarking workflow fix #45

Merged
merged 6 commits into from
Nov 20, 2024
Merged

Benchmarking workflow fix #45

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
fc0c6b1
output dir content in workflow file
ADBond Nov 20, 2024
0481003
remove some stray double-quotes
ADBond Nov 20, 2024
daaa7b2
innovation: checkout the code before attempting to run it :facepalm:
ADBond Nov 20, 2024
61a94a3
include script dir in run command
ADBond Nov 20, 2024
61bb677
some diagnostic info
ADBond Nov 20, 2024
f3c2dad
another data source in workflow
ADBond Nov 20, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 11 additions & 4 deletions .github/workflows/benchmark.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ jobs:
fail-fast: true
matrix:
# add in other data choices once we know things are working okay
data_choice: ["fake_1000"]
data_choice: ["fake_1000", "historical_50k"]
backend_to_use: ["duckdb", "clickhouse"]
name: Benchmark ${{ matrix.backend_to_use }} for data ${{ matrix.data_choice }}
steps:
Expand All @@ -39,17 +39,21 @@ jobs:
- name: Run benchmarking script
run: uv run python benchmarking/benchmark.py ${{ matrix.data_choice }} ${{ matrix.backend_to_use }}

- name: Check directory content
run: ls -alhR benchmarking

- uses: actions/upload-artifact@v4
with:
name: bm_partial_${{ matrix.data_choice }}_${{ matrix.backend_to_use }}
path: benchmarking/output/run_data_${{ matrix.data_choice }}_${{ matrix.backend_to_use }}.json"
path: benchmarking/output/run_data_${{ matrix.data_choice }}_${{ matrix.backend_to_use }}.json
if-no-files-found: error

combine_benchmark_data:
runs-on: ubuntu-latest
needs: get_benchmark_data
name: Combine data from benchmark runs
steps:
- uses: actions/checkout@v4

- name: Setup uv
id: setup-uv
Expand All @@ -71,11 +75,14 @@ jobs:
pattern: 'bm_partial_*'
merge-multiple: true

- name: Check directory content
run: ls -alhR benchmarking

- name: Combine benchmark data
run: uv run python combine_data.py
run: uv run python benchmarking/combine_data.py

- uses: actions/upload-artifact@v4
with:
name: bm_charts
path: benchmarking/output/*.html"
path: benchmarking/output/*.html
if-no-files-found: error
1 change: 1 addition & 0 deletions benchmarking/benchmark.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -181,6 +181,7 @@ def timed_full_run(data: str, config: dict, backend_to_use: str) -> Timer:
timer = timed_full_run(data_choice, config, backend_to_use)

output_data_file = f"benchmarking/output/run_data_{data_choice}_{backend_to_use}.json"
print(f"Writing data to to {output_data_file}") # noqa: T201
with open(output_data_file, "w+") as f:
json.dump(timer.records, f, indent=4)

Expand Down
2 changes: 2 additions & 0 deletions benchmarking/combine_data.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,9 +11,11 @@
continue
if file[0:3] != "run":
continue
print(f"Adding records from file {file}") # noqa: T201
with open(f"{_root}/{file}", "r") as f:
records += json.load(f)

print(f"All run data: {records}") # noqa: T201
with open("benchmarking/output/combined_run_data.json", "w+") as f:
json.dump(records, f, indent=4)

Expand Down