Skip to content

test: ignore some slow tests on miri, speed up one test #950

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
Mar 20, 2025
Merged

test: ignore some slow tests on miri, speed up one test #950

merged 2 commits into from
Mar 20, 2025

Conversation

morrisonlevi
Copy link
Contributor

@morrisonlevi morrisonlevi commented Mar 19, 2025 •
edited
Loading

What does this PR do?

This skips some slow tests on miri.

It also converts a test to use &str instead of String to speed up the miri run for that test (rather than skipping it).

Motivation

The miri job is slow again.

Additional Notes

It's still a bit slow after making this change, but it shaved off a few minutes and these were the "SLOW" tests.

How to test the change?

Just regular test commands are fine.

@morrisonlevi morrisonlevi requested review from a team as code owners March 19, 2025 23:33
@pr-commenter
Copy link

pr-commenter bot commented Mar 19, 2025 •
edited
Loading

Benchmarks

Comparison

Benchmark execution time: 2025-03-20 11:03:04

Comparing candidate commit eebc134 in PR branch levi/slow-miri with baseline commit f60e05a in branch main.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 52 metrics, 2 unstable metrics.

Candidate

Candidate benchmark details

Group 1

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz eebc134 1742467883 levi/slow-miri
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... execution_time 504.433µs 505.746µs ± 0.555µs 505.741µs ± 0.303µs 506.036µs 506.464µs 506.660µs 510.684µs 0.98% 3.412 29.705 0.11% 0.039µs 1 200
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... throughput 1958156.800op/s 1977278.502op/s ± 2162.327op/s 1977295.050op/s ± 1184.975op/s 1978544.009op/s 1980015.113op/s 1981087.110op/s 1982424.485op/s 0.26% -3.347 28.985 0.11% 152.900op/s 1 200
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて execution_time 452.444µs 453.292µs ± 0.323µs 453.289µs ± 0.191µs 453.479µs 453.861µs 454.025µs 454.883µs 0.35% 0.560 2.434 0.07% 0.023µs 1 200
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて throughput 2198368.701op/s 2206084.550op/s ± 1569.259op/s 2206096.515op/s ± 929.800op/s 2207032.749op/s 2208350.713op/s 2209731.631op/s 2210217.793op/s 0.19% -0.552 2.403 0.07% 110.963op/s 1 200
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters execution_time 175.127µs 176.660µs ± 0.383µs 176.688µs ± 0.237µs 176.915µs 177.171µs 177.365µs 177.559µs 0.49% -0.880 1.952 0.22% 0.027µs 1 200
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters throughput 5631946.267op/s 5660611.664op/s ± 12304.268op/s 5659685.330op/s ± 7586.858op/s 5667358.804op/s 5681578.910op/s 5698875.358op/s 5710127.468op/s 0.89% 0.901 2.020 0.22% 870.043op/s 1 200
normalization/normalize_service/normalize_service/[empty string] execution_time 37.532µs 37.642µs ± 0.050µs 37.639µs ± 0.031µs 37.671µs 37.732µs 37.770µs 37.820µs 0.48% 0.513 0.518 0.13% 0.004µs 1 200
normalization/normalize_service/normalize_service/[empty string] throughput 26440764.157op/s 26565892.786op/s ± 35025.551op/s 26568230.346op/s ± 21847.047op/s 26588782.148op/s 26615265.575op/s 26642343.488op/s 26644112.116op/s 0.29% -0.504 0.504 0.13% 2476.680op/s 1 200
normalization/normalize_service/normalize_service/test_ASCII execution_time 48.095µs 48.303µs ± 0.223µs 48.116µs ± 0.018µs 48.549µs 48.631µs 48.671µs 48.769µs 1.36% 0.392 -1.646 0.46% 0.016µs 1 200
normalization/normalize_service/normalize_service/test_ASCII throughput 20504745.722op/s 20703091.688op/s ± 95287.281op/s 20783323.462op/s ± 7774.898op/s 20787551.226op/s 20789640.344op/s 20790920.380op/s 20792362.668op/s 0.04% -0.389 -1.650 0.46% 6737.828op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... execution_time [505.669µs; 505.823µs] or [-0.015%; +0.015%] None None None
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... throughput [1976978.825op/s; 1977578.180op/s] or [-0.015%; +0.015%] None None None
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて execution_time [453.247µs; 453.337µs] or [-0.010%; +0.010%] None None None
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて throughput [2205867.066op/s; 2206302.034op/s] or [-0.010%; +0.010%] None None None
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters execution_time [176.607µs; 176.713µs] or [-0.030%; +0.030%] None None None
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters throughput [5658906.411op/s; 5662316.918op/s] or [-0.030%; +0.030%] None None None
normalization/normalize_service/normalize_service/[empty string] execution_time [37.635µs; 37.649µs] or [-0.018%; +0.018%] None None None
normalization/normalize_service/normalize_service/[empty string] throughput [26561038.581op/s; 26570746.990op/s] or [-0.018%; +0.018%] None None None
normalization/normalize_service/normalize_service/test_ASCII execution_time [48.272µs; 48.334µs] or [-0.064%; +0.064%] None None None
normalization/normalize_service/normalize_service/test_ASCII throughput [20689885.788op/s; 20716297.589op/s] or [-0.064%; +0.064%] None None None

Group 2

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz eebc134 1742467883 levi/slow-miri
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
benching string interning on wordpress profile execution_time 148.267µs 149.267µs ± 0.699µs 149.172µs ± 0.166µs 149.385µs 149.751µs 150.402µs 158.077µs 5.97% 10.146 124.997 0.47% 0.049µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
benching string interning on wordpress profile execution_time [149.170µs; 149.363µs] or [-0.065%; +0.065%] None None None

Group 3

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz eebc134 1742467883 levi/slow-miri
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... execution_time 208.676µs 209.085µs ± 0.158µs 209.083µs ± 0.093µs 209.176µs 209.386µs 209.448µs 209.507µs 0.20% 0.135 0.068 0.08% 0.011µs 1 200
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... throughput 4773114.003op/s 4782736.233op/s ± 3624.254op/s 4782794.376op/s ± 2121.679op/s 4784783.591op/s 4788529.844op/s 4791308.714op/s 4792110.908op/s 0.19% -0.130 0.067 0.08% 256.273op/s 1 200
normalization/normalize_name/normalize_name/bad-name execution_time 18.214µs 18.317µs ± 0.082µs 18.317µs ± 0.032µs 18.345µs 18.387µs 18.410µs 19.257µs 5.13% 7.421 83.241 0.45% 0.006µs 1 200
normalization/normalize_name/normalize_name/bad-name throughput 51929281.918op/s 54595380.708op/s ± 237629.381op/s 54592878.224op/s ± 96108.986op/s 54718631.363op/s 54861702.293op/s 54879097.953op/s 54903683.253op/s 0.57% -7.023 77.372 0.43% 16802.935op/s 1 200
normalization/normalize_name/normalize_name/good execution_time 10.652µs 10.716µs ± 0.038µs 10.717µs ± 0.031µs 10.745µs 10.777µs 10.788µs 10.840µs 1.16% 0.335 -0.466 0.35% 0.003µs 1 200
normalization/normalize_name/normalize_name/good throughput 92246863.961op/s 93320223.078op/s ± 330581.981op/s 93312675.365op/s ± 273008.619op/s 93628525.037op/s 93789098.341op/s 93836310.795op/s 93880439.649op/s 0.61% -0.320 -0.497 0.35% 23375.676op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... execution_time [209.063µs; 209.107µs] or [-0.011%; +0.011%] None None None
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... throughput [4782233.946op/s; 4783238.520op/s] or [-0.011%; +0.011%] None None None
normalization/normalize_name/normalize_name/bad-name execution_time [18.306µs; 18.328µs] or [-0.062%; +0.062%] None None None
normalization/normalize_name/normalize_name/bad-name throughput [54562447.561op/s; 54628313.855op/s] or [-0.060%; +0.060%] None None None
normalization/normalize_name/normalize_name/good execution_time [10.711µs; 10.721µs] or [-0.049%; +0.049%] None None None
normalization/normalize_name/normalize_name/good throughput [93274407.594op/s; 93366038.561op/s] or [-0.049%; +0.049%] None None None

Group 4

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz eebc134 1742467883 levi/slow-miri
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_trace/test_trace execution_time 245.700ns 254.382ns ± 10.398ns 248.881ns ± 2.698ns 258.696ns 280.080ns 283.270ns 285.037ns 14.53% 1.427 0.971 4.08% 0.735ns 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_trace/test_trace execution_time [252.941ns; 255.823ns] or [-0.566%; +0.566%] None None None

Group 5

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz eebc134 1742467883 levi/slow-miri
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
write only interface execution_time 1.204µs 3.152µs ± 1.423µs 2.960µs ± 0.028µs 2.985µs 3.589µs 13.803µs 14.877µs 402.68% 7.385 55.550 45.04% 0.101µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
write only interface execution_time [2.954µs; 3.349µs] or [-6.258%; +6.258%] None None None

Group 6

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz eebc134 1742467883 levi/slow-miri
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
redis/obfuscate_redis_string execution_time 32.273µs 32.912µs ± 0.781µs 32.483µs ± 0.159µs 33.978µs 34.105µs 34.989µs 35.302µs 8.68% 0.988 -0.646 2.37% 0.055µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
redis/obfuscate_redis_string execution_time [32.804µs; 33.021µs] or [-0.329%; +0.329%] None None None

Group 7

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz eebc134 1742467883 levi/slow-miri
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
tags/replace_trace_tags execution_time 2.366µs 2.407µs ± 0.030µs 2.402µs ± 0.012µs 2.415µs 2.444µs 2.600µs 2.612µs 8.72% 4.330 26.040 1.25% 0.002µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
tags/replace_trace_tags execution_time [2.403µs; 2.411µs] or [-0.173%; +0.173%] None None None

Group 8

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz eebc134 1742467883 levi/slow-miri
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
sql/obfuscate_sql_string execution_time 65.392µs 65.598µs ± 0.195µs 65.571µs ± 0.077µs 65.654µs 65.819µs 66.061µs 67.739µs 3.31% 6.866 70.645 0.30% 0.014µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
sql/obfuscate_sql_string execution_time [65.571µs; 65.625µs] or [-0.041%; +0.041%] None None None

Group 9

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz eebc134 1742467883 levi/slow-miri
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
ip_address/quantize_peer_ip_address_benchmark execution_time 4.937µs 5.007µs ± 0.050µs 4.983µs ± 0.029µs 5.066µs 5.082µs 5.088µs 5.090µs 2.13% 0.480 -1.419 0.99% 0.004µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
ip_address/quantize_peer_ip_address_benchmark execution_time [5.000µs; 5.013µs] or [-0.137%; +0.137%] None None None

Group 10

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz eebc134 1742467883 levi/slow-miri
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
concentrator/add_spans_to_concentrator execution_time 6.047ms 6.057ms ± 0.016ms 6.055ms ± 0.003ms 6.058ms 6.063ms 6.090ms 6.232ms 2.92% 9.075 89.794 0.26% 0.001ms 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
concentrator/add_spans_to_concentrator execution_time [6.055ms; 6.059ms] or [-0.036%; +0.036%] None None None

Group 11

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz eebc134 1742467883 levi/slow-miri
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
credit_card/is_card_number/ execution_time 3.894µs 3.916µs ± 0.003µs 3.915µs ± 0.001µs 3.917µs 3.920µs 3.921µs 3.922µs 0.18% -2.174 15.996 0.07% 0.000µs 1 200
credit_card/is_card_number/ throughput 254956458.326op/s 255390111.785op/s ± 186692.309op/s 255404070.231op/s ± 96453.546op/s 255482863.489op/s 255600059.811op/s 255854920.668op/s 256823307.776op/s 0.56% 2.203 16.259 0.07% 13201.140op/s 1 200
credit_card/is_card_number/ 3782-8224-6310-005 execution_time 81.801µs 82.121µs ± 0.212µs 82.056µs ± 0.139µs 82.242µs 82.572µs 82.690µs 82.835µs 0.95% 0.926 0.364 0.26% 0.015µs 1 200
credit_card/is_card_number/ 3782-8224-6310-005 throughput 12072192.901op/s 12177306.627op/s ± 31375.566op/s 12186774.914op/s ± 20586.573op/s 12202246.450op/s 12213677.426op/s 12221437.332op/s 12224768.604op/s 0.31% -0.915 0.332 0.26% 2218.588op/s 1 200
credit_card/is_card_number/ 378282246310005 execution_time 76.249µs 76.862µs ± 0.320µs 76.807µs ± 0.176µs 77.008µs 77.521µs 77.819µs 77.849µs 1.36% 1.083 0.904 0.41% 0.023µs 1 200
credit_card/is_card_number/ 378282246310005 throughput 12845370.082op/s 13010534.420op/s ± 53851.066op/s 13019576.773op/s ± 29822.875op/s 13047679.474op/s 13075039.957op/s 13090255.220op/s 13114881.456op/s 0.73% -1.061 0.852 0.41% 3807.845op/s 1 200
credit_card/is_card_number/37828224631 execution_time 3.893µs 3.915µs ± 0.003µs 3.915µs ± 0.002µs 3.917µs 3.919µs 3.921µs 3.922µs 0.18% -1.795 11.371 0.08% 0.000µs 1 200
credit_card/is_card_number/37828224631 throughput 254975311.803op/s 255442229.764op/s ± 197692.446op/s 255432195.530op/s ± 103444.509op/s 255533685.839op/s 255757252.154op/s 255848437.431op/s 256846182.210op/s 0.55% 1.819 11.564 0.08% 13978.967op/s 1 200
credit_card/is_card_number/378282246310005 execution_time 71.893µs 72.328µs ± 0.325µs 72.300µs ± 0.246µs 72.520µs 72.855µs 73.140µs 73.772µs 2.04% 0.954 1.160 0.45% 0.023µs 1 200
credit_card/is_card_number/378282246310005 throughput 13555329.980op/s 13826247.516op/s ± 61816.784op/s 13831277.913op/s ± 47116.970op/s 13878801.865op/s 13897541.805op/s 13908737.678op/s 13909618.501op/s 0.57% -0.924 1.028 0.45% 4371.107op/s 1 200
credit_card/is_card_number/37828224631000521389798 execution_time 51.827µs 52.131µs ± 0.106µs 52.137µs ± 0.073µs 52.202µs 52.301µs 52.340µs 52.383µs 0.47% -0.200 -0.369 0.20% 0.008µs 1 200
credit_card/is_card_number/37828224631000521389798 throughput 19090259.642op/s 19182489.720op/s ± 39067.956op/s 19180399.556op/s ± 26851.696op/s 19211146.309op/s 19253960.488op/s 19263816.285op/s 19295018.318op/s 0.60% 0.210 -0.364 0.20% 2762.522op/s 1 200
credit_card/is_card_number/x371413321323331 execution_time 6.429µs 6.448µs ± 0.026µs 6.442µs ± 0.003µs 6.445µs 6.483µs 6.569µs 6.649µs 3.21% 4.980 28.253 0.40% 0.002µs 1 200
credit_card/is_card_number/x371413321323331 throughput 150405926.516op/s 155093745.970op/s ± 605874.089op/s 155236903.964op/s ± 63416.906op/s 155289300.572op/s 155373524.307op/s 155490582.570op/s 155548628.928op/s 0.20% -4.916 27.479 0.39% 42841.768op/s 1 200
credit_card/is_card_number_no_luhn/ execution_time 3.896µs 3.914µs ± 0.003µs 3.914µs ± 0.002µs 3.916µs 3.918µs 3.920µs 3.921µs 0.17% -1.628 9.808 0.07% 0.000µs 1 200
credit_card/is_card_number_no_luhn/ throughput 255034051.639op/s 255469378.584op/s ± 177098.875op/s 255475761.337op/s ± 104333.177op/s 255553127.436op/s 255707028.708op/s 255931018.488op/s 256673769.247op/s 0.47% 1.647 9.950 0.07% 12522.782op/s 1 200
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 execution_time 64.040µs 64.257µs ± 0.107µs 64.251µs ± 0.040µs 64.286µs 64.401µs 64.464µs 65.303µs 1.64% 4.848 44.434 0.17% 0.008µs 1 200
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 throughput 15313248.699op/s 15562639.553op/s ± 25672.788op/s 15563888.667op/s ± 9592.032op/s 15574612.214op/s 15591124.215op/s 15604682.484op/s 15615321.754op/s 0.33% -4.735 42.987 0.16% 1815.340op/s 1 200
credit_card/is_card_number_no_luhn/ 378282246310005 execution_time 58.160µs 58.299µs ± 0.165µs 58.270µs ± 0.029µs 58.307µs 58.446µs 58.680µs 59.838µs 2.69% 7.542 64.522 0.28% 0.012µs 1 200
credit_card/is_card_number_no_luhn/ 378282246310005 throughput 16711819.347op/s 17153009.798op/s ± 47632.677op/s 17161636.021op/s ± 8601.948op/s 17169683.063op/s 17175296.515op/s 17187186.783op/s 17193959.762op/s 0.19% -7.461 63.462 0.28% 3368.139op/s 1 200
credit_card/is_card_number_no_luhn/37828224631 execution_time 3.895µs 3.914µs ± 0.004µs 3.914µs ± 0.002µs 3.916µs 3.918µs 3.921µs 3.940µs 0.66% 1.654 15.392 0.10% 0.000µs 1 200
credit_card/is_card_number_no_luhn/37828224631 throughput 253819864.787op/s 255505377.262op/s ± 256591.428op/s 255491453.817op/s ± 123368.370op/s 255626452.021op/s 255848724.378op/s 255933481.711op/s 256754063.874op/s 0.49% -1.610 15.201 0.10% 18143.754op/s 1 200
credit_card/is_card_number_no_luhn/378282246310005 execution_time 54.556µs 54.743µs ± 0.180µs 54.668µs ± 0.030µs 54.742µs 55.150µs 55.339µs 55.527µs 1.57% 2.072 3.803 0.33% 0.013µs 1 200
credit_card/is_card_number_no_luhn/378282246310005 throughput 18009309.928op/s 18267460.591op/s ± 59823.978op/s 18292105.128op/s ± 9975.819op/s 18299861.836op/s 18312092.758op/s 18320238.514op/s 18329925.436op/s 0.21% -2.057 3.717 0.33% 4230.194op/s 1 200
credit_card/is_card_number_no_luhn/37828224631000521389798 execution_time 51.836µs 52.155µs ± 0.100µs 52.173µs ± 0.066µs 52.229µs 52.293µs 52.361µs 52.364µs 0.37% -0.573 0.139 0.19% 0.007µs 1 200
credit_card/is_card_number_no_luhn/37828224631000521389798 throughput 19096909.155op/s 19173539.942op/s ± 36846.517op/s 19167013.570op/s ± 24376.599op/s 19197591.189op/s 19236481.033op/s 19277824.236op/s 19291429.802op/s 0.65% 0.584 0.157 0.19% 2605.442op/s 1 200
credit_card/is_card_number_no_luhn/x371413321323331 execution_time 6.429µs 6.448µs ± 0.016µs 6.443µs ± 0.003µs 6.447µs 6.485µs 6.514µs 6.531µs 1.37% 2.935 8.756 0.25% 0.001µs 1 200
credit_card/is_card_number_no_luhn/x371413321323331 throughput 153118580.826op/s 155083070.699op/s ± 385780.409op/s 155208737.264op/s ± 65273.507op/s 155251965.044op/s 155333837.954op/s 155470007.299op/s 155535745.529op/s 0.21% -2.919 8.650 0.25% 27278.794op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
credit_card/is_card_number/ execution_time [3.915µs; 3.916µs] or [-0.010%; +0.010%] None None None
credit_card/is_card_number/ throughput [255364238.026op/s; 255415985.544op/s] or [-0.010%; +0.010%] None None None
credit_card/is_card_number/ 3782-8224-6310-005 execution_time [82.091µs; 82.150µs] or [-0.036%; +0.036%] None None None
credit_card/is_card_number/ 3782-8224-6310-005 throughput [12172958.275op/s; 12181654.979op/s] or [-0.036%; +0.036%] None None None
credit_card/is_card_number/ 378282246310005 execution_time [76.818µs; 76.906µs] or [-0.058%; +0.058%] None None None
credit_card/is_card_number/ 378282246310005 throughput [13003071.180op/s; 13017997.659op/s] or [-0.057%; +0.057%] None None None
credit_card/is_card_number/37828224631 execution_time [3.914µs; 3.915µs] or [-0.011%; +0.011%] None None None
credit_card/is_card_number/37828224631 throughput [255414831.492op/s; 255469628.035op/s] or [-0.011%; +0.011%] None None None
credit_card/is_card_number/378282246310005 execution_time [72.283µs; 72.373µs] or [-0.062%; +0.062%] None None None
credit_card/is_card_number/378282246310005 throughput [13817680.304op/s; 13834814.728op/s] or [-0.062%; +0.062%] None None None
credit_card/is_card_number/37828224631000521389798 execution_time [52.116µs; 52.146µs] or [-0.028%; +0.028%] None None None
credit_card/is_card_number/37828224631000521389798 throughput [19177075.277op/s; 19187904.163op/s] or [-0.028%; +0.028%] None None None
credit_card/is_card_number/x371413321323331 execution_time [6.444µs; 6.451µs] or [-0.055%; +0.055%] None None None
credit_card/is_card_number/x371413321323331 throughput [155009777.649op/s; 155177714.292op/s] or [-0.054%; +0.054%] None None None
credit_card/is_card_number_no_luhn/ execution_time [3.914µs; 3.915µs] or [-0.010%; +0.010%] None None None
credit_card/is_card_number_no_luhn/ throughput [255444834.383op/s; 255493922.784op/s] or [-0.010%; +0.010%] None None None
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 execution_time [64.242µs; 64.271µs] or [-0.023%; +0.023%] None None None
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 throughput [15559081.551op/s; 15566197.554op/s] or [-0.023%; +0.023%] None None None
credit_card/is_card_number_no_luhn/ 378282246310005 execution_time [58.276µs; 58.322µs] or [-0.039%; +0.039%] None None None
credit_card/is_card_number_no_luhn/ 378282246310005 throughput [17146408.367op/s; 17159611.229op/s] or [-0.038%; +0.038%] None None None
credit_card/is_card_number_no_luhn/37828224631 execution_time [3.913µs; 3.914µs] or [-0.014%; +0.014%] None None None
credit_card/is_card_number_no_luhn/37828224631 throughput [255469816.158op/s; 255540938.366op/s] or [-0.014%; +0.014%] None None None
credit_card/is_card_number_no_luhn/378282246310005 execution_time [54.718µs; 54.768µs] or [-0.046%; +0.046%] None None None
credit_card/is_card_number_no_luhn/378282246310005 throughput [18259169.563op/s; 18275751.619op/s] or [-0.045%; +0.045%] None None None
credit_card/is_card_number_no_luhn/37828224631000521389798 execution_time [52.142µs; 52.169µs] or [-0.027%; +0.027%] None None None
credit_card/is_card_number_no_luhn/37828224631000521389798 throughput [19168433.369op/s; 19178646.515op/s] or [-0.027%; +0.027%] None None None
credit_card/is_card_number_no_luhn/x371413321323331 execution_time [6.446µs; 6.450µs] or [-0.035%; +0.035%] None None None
credit_card/is_card_number_no_luhn/x371413321323331 throughput [155029605.244op/s; 155136536.153op/s] or [-0.034%; +0.034%] None None None

Group 12

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz eebc134 1742467883 levi/slow-miri
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
two way interface execution_time 17.526µs 25.362µs ± 11.571µs 17.695µs ± 0.107µs 34.630µs 44.302µs 49.831µs 109.054µs 516.30% 2.668 13.825 45.51% 0.818µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
two way interface execution_time [23.758µs; 26.965µs] or [-6.323%; +6.323%] None None None

Group 13

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz eebc134 1742467883 levi/slow-miri
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
benching deserializing traces from msgpack to their internal representation execution_time 53.835ms 54.186ms ± 0.250ms 54.128ms ± 0.124ms 54.280ms 54.658ms 54.965ms 55.638ms 2.79% 1.975 6.336 0.46% 0.018ms 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
benching deserializing traces from msgpack to their internal representation execution_time [54.152ms; 54.221ms] or [-0.064%; +0.064%] None None None

Baseline

Omitted due to size.

@codecov-commenter
Copy link

codecov-commenter commented Mar 19, 2025 •
edited
Loading

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 72.85%. Comparing base (f60e05a) to head (eebc134).

Additional details and impacted files 
@@            Coverage Diff             @@
##             main     #950      +/-   ##
==========================================
- Coverage   72.87%   72.85%   -0.02%     
==========================================
  Files         334      334              
  Lines       51019    51019              
==========================================
- Hits        37178    37171       -7     
- Misses      13841    13848       +7
Components Coverage Δ
crashtracker 42.88% <ø> (+0.02%) ⬆️
crashtracker-ffi 6.25% <ø> (ø)
datadog-alloc 98.73% <ø> (ø)
data-pipeline 91.96% <ø> (ø)
data-pipeline-ffi 90.28% <ø> (ø)
ddcommon 81.37% <ø> (ø)
ddcommon-ffi 66.10% <ø> (ø)
ddtelemetry 61.87% <ø> (ø)
ddtelemetry-ffi 22.46% <ø> (ø)
dogstatsd 89.60% <ø> (ø)
dogstatsd-client 82.57% <ø> (ø)
ipc 82.41% <ø> (ø)
profiling 81.89% <ø> (+0.09%) ⬆️
profiling-ffi 70.68% <ø> (ø)
serverless 0.00% <ø> (ø)
sidecar 41.53% <ø> (ø)
sidecar-ffi 5.58% <ø> (ø)
spawn-worker 54.37% <ø> (ø)
tinybytes 91.24% <ø> (ø)
trace-mini-agent 74.66% <ø> (ø)
trace-normalization 98.24% <ø> (ø)
trace-obfuscation 96.00% <ø> (ø)
trace-protobuf 78.13% <ø> (ø)
trace-utils 92.87% <ø> (ø)
🚀 New features to boost your workflow:
  • ❄ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@morrisonlevi morrisonlevi enabled auto-merge (squash) March 20, 2025 03:08
@r1viollet
Copy link
Contributor

Artifact Size Benchmark Report

aarch64-alpine-linux-musl
Artifact Baseline Commit Change
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.so 7.80 MB 7.80 MB 0% (0 B) 👌
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.a 78.56 MB 78.56 MB 0% (0 B) 👌
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.so.debug 24.63 MB 24.63 MB 0% (0 B) 👌
aarch64-unknown-linux-gnu
Artifact Baseline Commit Change
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.a 73.04 MB 73.04 MB 0% (0 B) 👌
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.so.debug 23.27 MB 23.27 MB 0% (0 B) 👌
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.so 7.73 MB 7.73 MB 0% (0 B) 👌
i686-alpine-linux-musl
Artifact Baseline Commit Change
/i686-alpine-linux-musl/lib/libdatadog_profiling.a 67.89 MB 67.89 MB 0% (0 B) 👌
/i686-alpine-linux-musl/lib/libdatadog_profiling.so 8.27 MB 8.27 MB 0% (0 B) 👌
/i686-alpine-linux-musl/lib/libdatadog_profiling.so.debug 23.81 MB 23.81 MB 0% (0 B) 👌
i686-unknown-linux-gnu
Artifact Baseline Commit Change
/i686-unknown-linux-gnu/lib/libdatadog_profiling.a 68.81 MB 68.81 MB 0% (0 B) 👌
/i686-unknown-linux-gnu/lib/libdatadog_profiling.so 8.15 MB 8.15 MB 0% (0 B) 👌
/i686-unknown-linux-gnu/lib/libdatadog_profiling.so.debug 21.43 MB 21.43 MB 0% (0 B) 👌
libdatadog-x64-windows
Artifact Baseline Commit Change
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.dll 17.19 MB 17.19 MB 0% (0 B) 👌
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.lib 54.81 KB 54.81 KB 0% (0 B) 👌
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.pdb 116.94 MB 116.95 MB +0% (+8.00 KB) 👌
/libdatadog-x64-windows/debug/static/datadog_profiling_ffi.lib 699.17 MB 699.17 MB 0% (0 B) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.dll 5.05 MB 5.05 MB 0% (0 B) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.lib 54.81 KB 54.81 KB 0% (0 B) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.pdb 16.28 MB 16.28 MB 0% (0 B) 👌
/libdatadog-x64-windows/release/static/datadog_profiling_ffi.lib 26.96 MB 26.96 MB 0% (0 B) 👌
libdatadog-x86-windows
Artifact Baseline Commit Change
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.dll 14.58 MB 14.58 MB 0% (0 B) 👌
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.lib 55.66 KB 55.66 KB 0% (0 B) 👌
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.pdb 118.97 MB 118.98 MB +0% (+8.00 KB) 👌
/libdatadog-x86-windows/debug/static/datadog_profiling_ffi.lib 690.67 MB 690.67 MB 0% (0 B) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.dll 3.84 MB 3.84 MB 0% (0 B) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.lib 55.66 KB 55.66 KB 0% (0 B) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.pdb 16.96 MB 16.96 MB 0% (0 B) 👌
/libdatadog-x86-windows/release/static/datadog_profiling_ffi.lib 24.96 MB 24.96 MB 0% (0 B) 👌
x86_64-alpine-linux-musl
Artifact Baseline Commit Change
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.a 67.89 MB 67.89 MB 0% (0 B) 👌
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.so 8.27 MB 8.27 MB 0% (0 B) 👌
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.so.debug 23.81 MB 23.81 MB 0% (0 B) 👌
x86_64-unknown-linux-gnu
Artifact Baseline Commit Change
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.a 68.81 MB 68.81 MB 0% (0 B) 👌
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.so 8.15 MB 8.15 MB 0% (0 B) 👌
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.so.debug 21.43 MB 21.43 MB 0% (0 B) 👌

@morrisonlevi morrisonlevi merged commit 0219502 into main Mar 20, 2025
31 checks passed
@morrisonlevi morrisonlevi deleted the levi/slow-miri branch March 20, 2025 11:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@astuyve astuyve astuyve approved these changes

@ekump ekump ekump approved these changes

Assignees
No one assigned
Labels
mini-agent sidecar
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@morrisonlevi @codecov-commenter @r1viollet @astuyve @ekump @ivoanjo