Skip to content

WIP no-std, no-panic profiling core #1062

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
wants to merge 13 commits into
base: main
Choose a base branch
from
Draft

WIP no-std, no-panic profiling core #1062

wants to merge 13 commits into from

Conversation

morrisonlevi
Copy link
Contributor

@morrisonlevi morrisonlevi commented May 16, 2025 •
edited
Loading

What does this PR do?

Oh boy, this does way too much. It's built from bits and pieces of R&D projects and it's an absolute mess. Please don't try to review the code right now.

But it does these things and probably more:

  • Adds VirtualVec which uses the pre-existing VirtualAllocator to build a Vec-like type which is no-std, no-panic. There are tests that it is actually no-panic. They work locally, will be interesting to see if it works in CI.
  • Adds traits NoGrowOps and MayGrowOps for buffer/vec types.
  • Hand-encodes the in-wire protobuf serialization for profiles. This uses the above traits to elide unnecessary bounds checks and to handle allocation failures.
    • Uses prost still to check that these things encode correctly, so prost isn't gone, at least not yet. It may eventually become a dev-only dependency.
  • Creates string tables and "stores" which are currently unused, but provide the ability to deduplicate based on the in-wire protobuf serialization instead of the in-memory type. This is valuable if the results are cached, but more expensive if a lot of deduplicating is going on.

Motivation

There are a few broad motivations:

  • Creating a no-panic core for profiling, which generally also means no-std. Eventually extend this no-panic guarantee out to a subset of the profiling FFI.
  • Handle OOM situations gracefully (part of no-panic, but calling it out because it affects APIs quite a bit).
  • The hand-encoding is more efficient, as prost is quite wasteful. To be honest, protobuf is pretty wasteful because you have to calculate the length of a message (which sometimes means calculating lengths of its parts), and then you actually encode it later, and you essentially calculate it again. Sometimes I've been able to pass through the previous calculation, so we save that. But other times it just has to get calculated again.
  • I think this might shrink our size, which is one reason why I'm opening this as a PR in this crazy state. I want the Erwan bot to tell me if it's true. Update: Yes, but it's a small reduction, nothing interesting.

Additional Notes

Again, please don't review this right now.

How to test the change?

Oh boy, please don't bother yet.

@github-actions github-actions bot added the profiling Relates to the profiling* modules. label May 16, 2025
@codecov-commenter
Copy link

codecov-commenter commented May 16, 2025 •
edited
Loading

Codecov Report

Attention: Patch coverage is 80.41854% with 262 lines in your changes missing coverage. Please review.

Project coverage is 71.23%. Comparing base (f292b76) to head (c2618a2).

Additional details and impacted files 
@@            Coverage Diff             @@
##             main    #1062      +/-   ##
==========================================
- Coverage   71.25%   71.23%   -0.03%     
==========================================
  Files         322      334      +12     
  Lines       49339    50380    +1041     
==========================================
+ Hits        35158    35889     +731     
- Misses      14181    14491     +310
Components Coverage Δ
datadog-crashtracker 42.43% <ø> (ø)
datadog-crashtracker-ffi 6.30% <ø> (ø)
datadog-alloc 84.07% <69.26%> (-14.66%) ⬇️
data-pipeline 90.48% <ø> (-0.34%) ⬇️
data-pipeline-ffi 89.26% <ø> (ø)
ddcommon 78.28% <ø> (ø)
ddcommon-ffi 66.37% <ø> (ø)
ddtelemetry 60.56% <ø> (ø)
ddtelemetry-ffi 21.32% <ø> (ø)
dogstatsd-client 83.26% <ø> (ø)
datadog-ipc 82.68% <ø> (+0.10%) ⬆️
datadog-profiling 77.83% <91.50%> (+0.33%) ⬆️
datadog-profiling-ffi 62.12% <ø> (ø)
datadog-sidecar 41.58% <ø> (-1.06%) ⬇️
datdog-sidecar-ffi 4.67% <ø> (-8.18%) ⬇️
spawn-worker 55.35% <ø> (ø)
tinybytes 90.96% <ø> (ø)
datadog-trace-normalization 98.24% <ø> (ø)
datadog-trace-obfuscation 94.16% <ø> (ø)
datadog-trace-protobuf 77.10% <ø> (ø)
datadog-trace-utils 89.52% <ø> (+0.24%) ⬆️
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@morrisonlevi morrisonlevi force-pushed the levi/datadog-profiling-core branch from b092bca to e373664 Compare May 16, 2025 15:49
@morrisonlevi
fix: compat with Rust 1.78
f5e95c2
@morrisonlevi
style: fix clippy lints
f69d0b8
@morrisonlevi
style: fix clippy::expect-fun-call
adfb19b
@morrisonlevi
ci: skip benches in datadog-profiling-core
d24c976 
It doesn't have any, but it can fail with messages like:
> error: Unrecognized option: 'warm-up-time'
If we don't set benches=false
@pr-commenter
Copy link

pr-commenter bot commented May 16, 2025 •
edited
Loading

Benchmarks

Comparison

Benchmark execution time: 2025-05-21 23:05:01

Comparing candidate commit c2618a2 in PR branch levi/datadog-profiling-core with baseline commit f292b76 in branch main.

Found 12 performance improvements and 0 performance regressions! Performance is the same for 40 metrics, 2 unstable metrics.

scenario:credit_card/is_card_number/x371413321323331

  • 🟩 execution_time [-560.690ns; -541.365ns] or [-8.514%; -8.221%]
  • 🟩 throughput [+13627493.014op/s; +14072974.774op/s] or [+8.973%; +9.266%]

scenario:credit_card/is_card_number_no_luhn/x371413321323331

  • 🟩 execution_time [-559.629ns; -541.211ns] or [-8.499%; -8.219%]
  • 🟩 throughput [+13624968.958op/s; +14049954.641op/s] or [+8.971%; +9.251%]

scenario:normalization/normalize_name/normalize_name/bad-name

  • 🟩 execution_time [-2.291µs; -2.278µs] or [-11.568%; -11.503%]
  • 🟩 throughput [+6566947.321op/s; +6603216.232op/s] or [+13.004%; +13.076%]

scenario:normalization/normalize_name/normalize_name/good

  • 🟩 execution_time [-1.963µs; -1.904µs] or [-16.044%; -15.564%]
  • 🟩 throughput [+15120478.535op/s; +15516504.103op/s] or [+18.493%; +18.978%]

scenario:normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000...

  • 🟩 execution_time [-73.662µs; -72.383µs] or [-13.747%; -13.508%]
  • 🟩 throughput [+291658.561op/s; +297634.289op/s] or [+15.629%; +15.949%]

scenario:normalization/normalize_service/normalize_service/[empty string]

  • 🟩 execution_time [-1.679µs; -1.643µs] or [-4.185%; -4.094%]
  • 🟩 throughput [+1064744.342op/s; +1087720.326op/s] or [+4.272%; +4.364%]

Candidate

Candidate benchmark details

Group 1

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz c2618a2 1747867979 levi/datadog-profiling-core
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
ip_address/quantize_peer_ip_address_benchmark execution_time 4.940µs 5.006µs ± 0.041µs 4.984µs ± 0.022µs 5.043µs 5.078µs 5.080µs 5.083µs 2.00% 0.540 -1.195 0.81% 0.003µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
ip_address/quantize_peer_ip_address_benchmark execution_time [5.000µs; 5.011µs] or [-0.112%; +0.112%] None None None

Group 2

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz c2618a2 1747867979 levi/datadog-profiling-core
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
concentrator/add_spans_to_concentrator execution_time 5.977ms 5.990ms ± 0.006ms 5.989ms ± 0.003ms 5.992ms 5.999ms 6.009ms 6.020ms 0.51% 1.311 4.135 0.09% 0.000ms 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
concentrator/add_spans_to_concentrator execution_time [5.989ms; 5.991ms] or [-0.013%; +0.013%] None None None

Group 3

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz c2618a2 1747867979 levi/datadog-profiling-core
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
two way interface execution_time 17.671µs 25.397µs ± 11.313µs 17.967µs ± 0.169µs 34.973µs 44.999µs 56.089µs 97.614µs 443.31% 2.251 8.983 44.43% 0.800µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
two way interface execution_time [23.829µs; 26.965µs] or [-6.173%; +6.173%] None None None

Group 4

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz c2618a2 1747867979 levi/datadog-profiling-core
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_trace/test_trace execution_time 250.516ns 261.748ns ± 12.718ns 255.889ns ± 2.496ns 263.100ns 291.638ns 296.631ns 301.076ns 17.66% 1.547 1.092 4.85% 0.899ns 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_trace/test_trace execution_time [259.985ns; 263.510ns] or [-0.673%; +0.673%] None None None

Group 5

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz c2618a2 1747867979 levi/datadog-profiling-core
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
redis/obfuscate_redis_string execution_time 31.530µs 32.562µs ± 1.450µs 31.656µs ± 0.066µs 34.301µs 35.178µs 35.339µs 35.385µs 11.78% 0.975 -0.945 4.44% 0.103µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
redis/obfuscate_redis_string execution_time [32.361µs; 32.763µs] or [-0.617%; +0.617%] None None None

Group 6

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz c2618a2 1747867979 levi/datadog-profiling-core
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... execution_time 452.761µs 462.828µs ± 4.585µs 463.149µs ± 3.438µs 466.131µs 469.150µs 469.533µs 469.693µs 1.41% -0.344 -0.879 0.99% 0.324µs 1 200
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... throughput 2129050.013op/s 2160840.339op/s ± 21481.949op/s 2159131.630op/s ± 16057.288op/s 2177370.803op/s 2201281.917op/s 2207257.017op/s 2208671.313op/s 2.29% 0.374 -0.849 0.99% 1519.003op/s 1 200
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて execution_time 450.009µs 450.758µs ± 0.482µs 450.716µs ± 0.225µs 450.941µs 451.375µs 452.259µs 455.010µs 0.95% 3.911 29.775 0.11% 0.034µs 1 200
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて throughput 2197753.654op/s 2218487.109op/s ± 2360.199op/s 2218692.898op/s ± 1106.110op/s 2219794.507op/s 2221088.036op/s 2221862.923op/s 2222177.285op/s 0.16% -3.859 29.165 0.11% 166.891op/s 1 200
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters execution_time 179.358µs 180.384µs ± 0.543µs 180.331µs ± 0.274µs 180.614µs 181.002µs 181.200µs 184.499µs 2.31% 3.656 24.991 0.30% 0.038µs 1 200
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters throughput 5420077.846op/s 5543787.657op/s ± 16515.417op/s 5545344.008op/s ± 8419.303op/s 5552900.058op/s 5563667.957op/s 5567216.779op/s 5575449.636op/s 0.54% -3.535 23.887 0.30% 1167.816op/s 1 200
normalization/normalize_service/normalize_service/[empty string] execution_time 38.290µs 38.461µs ± 0.068µs 38.456µs ± 0.044µs 38.502µs 38.594µs 38.630µs 38.649µs 0.50% 0.326 0.161 0.18% 0.005µs 1 200
normalization/normalize_service/normalize_service/[empty string] throughput 25874032.777op/s 26000427.424op/s ± 46068.604op/s 26003944.766op/s ± 29969.670op/s 26033193.267op/s 26066907.613op/s 26103764.545op/s 26116258.494op/s 0.43% -0.315 0.153 0.18% 3257.542op/s 1 200
normalization/normalize_service/normalize_service/test_ASCII execution_time 47.197µs 47.579µs ± 0.138µs 47.581µs ± 0.100µs 47.678µs 47.814µs 47.872µs 47.912µs 0.69% -0.011 -0.424 0.29% 0.010µs 1 200
normalization/normalize_service/normalize_service/test_ASCII throughput 20871798.612op/s 21017758.148op/s ± 61132.174op/s 21016621.196op/s ± 44214.959op/s 21062672.163op/s 21112836.507op/s 21134235.069op/s 21187953.815op/s 0.82% 0.025 -0.422 0.29% 4322.697op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... execution_time [462.193µs; 463.464µs] or [-0.137%; +0.137%] None None None
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... throughput [2157863.148op/s; 2163817.531op/s] or [-0.138%; +0.138%] None None None
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて execution_time [450.691µs; 450.825µs] or [-0.015%; +0.015%] None None None
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて throughput [2218160.009op/s; 2218814.210op/s] or [-0.015%; +0.015%] None None None
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters execution_time [180.308µs; 180.459µs] or [-0.042%; +0.042%] None None None
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters throughput [5541498.779op/s; 5546076.535op/s] or [-0.041%; +0.041%] None None None
normalization/normalize_service/normalize_service/[empty string] execution_time [38.452µs; 38.470µs] or [-0.025%; +0.025%] None None None
normalization/normalize_service/normalize_service/[empty string] throughput [25994042.759op/s; 26006812.090op/s] or [-0.025%; +0.025%] None None None
normalization/normalize_service/normalize_service/test_ASCII execution_time [47.560µs; 47.598µs] or [-0.040%; +0.040%] None None None
normalization/normalize_service/normalize_service/test_ASCII throughput [21009285.817op/s; 21026230.480op/s] or [-0.040%; +0.040%] None None None

Group 7

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz c2618a2 1747867979 levi/datadog-profiling-core
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... execution_time 218.108µs 219.294µs ± 0.479µs 219.330µs ± 0.366µs 219.677µs 220.030µs 220.261µs 220.290µs 0.44% -0.167 -0.699 0.22% 0.034µs 1 200
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... throughput 4539470.109op/s 4560113.420op/s ± 9963.472op/s 4559342.778op/s ± 7602.395op/s 4567483.757op/s 4577713.305op/s 4579985.529op/s 4584891.159op/s 0.56% 0.176 -0.696 0.22% 704.524op/s 1 200
normalization/normalize_name/normalize_name/bad-name execution_time 17.465µs 17.518µs ± 0.028µs 17.514µs ± 0.015µs 17.530µs 17.569µs 17.596µs 17.647µs 0.76% 1.322 3.133 0.16% 0.002µs 1 200
normalization/normalize_name/normalize_name/bad-name throughput 56665330.309op/s 57085205.972op/s ± 91072.561op/s 57097682.053op/s ± 50179.228op/s 57146183.867op/s 57203357.867op/s 57234617.321op/s 57258191.500op/s 0.28% -1.306 3.061 0.16% 6439.803op/s 1 200
normalization/normalize_name/normalize_name/good execution_time 10.266µs 10.301µs ± 0.019µs 10.297µs ± 0.013µs 10.314µs 10.335µs 10.346µs 10.355µs 0.57% 0.647 -0.323 0.18% 0.001µs 1 200
normalization/normalize_name/normalize_name/good throughput 96571775.682op/s 97080217.995op/s ± 176445.734op/s 97120200.706op/s ± 118949.706op/s 97214273.065op/s 97304816.288op/s 97371857.743op/s 97407256.766op/s 0.30% -0.641 -0.334 0.18% 12476.598op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... execution_time [219.227µs; 219.360µs] or [-0.030%; +0.030%] None None None
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... throughput [4558732.579op/s; 4561494.262op/s] or [-0.030%; +0.030%] None None None
normalization/normalize_name/normalize_name/bad-name execution_time [17.514µs; 17.522µs] or [-0.022%; +0.022%] None None None
normalization/normalize_name/normalize_name/bad-name throughput [57072584.191op/s; 57097827.753op/s] or [-0.022%; +0.022%] None None None
normalization/normalize_name/normalize_name/good execution_time [10.298µs; 10.303µs] or [-0.025%; +0.025%] None None None
normalization/normalize_name/normalize_name/good throughput [97055764.313op/s; 97104671.677op/s] or [-0.025%; +0.025%] None None None

Group 8

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz c2618a2 1747867979 levi/datadog-profiling-core
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
benching string interning on wordpress profile execution_time 148.487µs 149.372µs ± 0.261µs 149.354µs ± 0.117µs 149.466µs 149.854µs 150.157µs 150.763µs 0.94% 1.105 5.521 0.17% 0.018µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
benching string interning on wordpress profile execution_time [149.336µs; 149.408µs] or [-0.024%; +0.024%] None None None

Group 9

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz c2618a2 1747867979 levi/datadog-profiling-core
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
credit_card/is_card_number/ execution_time 3.892µs 3.912µs ± 0.003µs 3.911µs ± 0.001µs 3.913µs 3.915µs 3.919µs 3.922µs 0.27% -1.078 14.327 0.07% 0.000µs 1 200
credit_card/is_card_number/ throughput 254967173.350op/s 255654016.860op/s ± 178223.136op/s 255662596.576op/s ± 86052.825op/s 255747460.348op/s 255874301.577op/s 255931989.809op/s 256950888.314op/s 0.50% 1.110 14.529 0.07% 12602.279op/s 1 200
credit_card/is_card_number/ 3782-8224-6310-005 execution_time 82.570µs 84.082µs ± 0.513µs 84.161µs ± 0.264µs 84.374µs 84.581µs 84.722µs 88.433µs 5.08% 2.519 24.776 0.61% 0.036µs 1 200
credit_card/is_card_number/ 3782-8224-6310-005 throughput 11307986.294op/s 11893598.771op/s ± 71520.774op/s 11882049.273op/s ± 37121.878op/s 11929521.932op/s 12018072.368op/s 12052686.617op/s 12110974.066op/s 1.93% -2.164 21.457 0.60% 5057.282op/s 1 200
credit_card/is_card_number/ 378282246310005 execution_time 78.844µs 79.900µs ± 0.480µs 79.905µs ± 0.283µs 80.173µs 80.480µs 80.626µs 83.902µs 5.00% 2.705 22.446 0.60% 0.034µs 1 200
credit_card/is_card_number/ 378282246310005 throughput 11918598.244op/s 12516054.998op/s ± 74118.580op/s 12514836.538op/s ± 44269.639op/s 12560796.513op/s 12623152.572op/s 12658802.281op/s 12683196.697op/s 1.35% -2.412 19.528 0.59% 5240.975op/s 1 200
credit_card/is_card_number/37828224631 execution_time 3.893µs 3.911µs ± 0.003µs 3.911µs ± 0.001µs 3.912µs 3.916µs 3.918µs 3.922µs 0.28% -0.717 9.976 0.07% 0.000µs 1 200
credit_card/is_card_number/37828224631 throughput 254996490.183op/s 255696318.477op/s ± 180298.720op/s 255698747.078op/s ± 86825.994op/s 255793646.055op/s 255921761.328op/s 256002607.883op/s 256894869.417op/s 0.47% 0.741 10.107 0.07% 12749.045op/s 1 200
credit_card/is_card_number/378282246310005 execution_time 75.975µs 76.935µs ± 0.258µs 76.984µs ± 0.167µs 77.106µs 77.301µs 77.380µs 77.604µs 0.81% -0.726 0.902 0.33% 0.018µs 1 200
credit_card/is_card_number/378282246310005 throughput 12885931.180op/s 12998050.304op/s ± 43723.933op/s 12989746.289op/s ± 28303.137op/s 13023381.499op/s 13077211.270op/s 13125411.435op/s 13162237.641op/s 1.33% 0.750 0.958 0.34% 3091.749op/s 1 200
credit_card/is_card_number/37828224631000521389798 execution_time 51.340µs 51.393µs ± 0.025µs 51.391µs ± 0.016µs 51.407µs 51.436µs 51.477µs 51.485µs 0.18% 0.689 1.106 0.05% 0.002µs 1 200
credit_card/is_card_number/37828224631000521389798 throughput 19423258.705op/s 19457773.688op/s ± 9618.541op/s 19458564.932op/s ± 5872.906op/s 19464233.498op/s 19472112.916op/s 19475385.861op/s 19477851.735op/s 0.10% -0.685 1.096 0.05% 680.134op/s 1 200
credit_card/is_card_number/x371413321323331 execution_time 6.028µs 6.034µs ± 0.003µs 6.034µs ± 0.002µs 6.036µs 6.040µs 6.043µs 6.051µs 0.28% 0.986 3.250 0.05% 0.000µs 1 200
credit_card/is_card_number/x371413321323331 throughput 165266008.998op/s 165722505.283op/s ± 88518.904op/s 165730285.566op/s ± 52265.961op/s 165778863.196op/s 165856693.964op/s 165881863.540op/s 165906246.360op/s 0.11% -0.980 3.219 0.05% 6259.232op/s 1 200
credit_card/is_card_number_no_luhn/ execution_time 3.894µs 3.911µs ± 0.002µs 3.910µs ± 0.001µs 3.912µs 3.914µs 3.917µs 3.918µs 0.21% -0.948 10.123 0.06% 0.000µs 1 200
credit_card/is_card_number_no_luhn/ throughput 255206176.049op/s 255718167.817op/s ± 157301.588op/s 255736762.164op/s ± 88293.059op/s 255816901.929op/s 255892935.602op/s 255978885.032op/s 256787340.314op/s 0.41% 0.969 10.256 0.06% 11122.902op/s 1 200
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 execution_time 65.897µs 66.225µs ± 0.168µs 66.202µs ± 0.117µs 66.341µs 66.520µs 66.639µs 66.982µs 1.18% 0.810 1.397 0.25% 0.012µs 1 200
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 throughput 14929348.821op/s 15100245.346op/s ± 38145.510op/s 15105370.894op/s ± 26660.865op/s 15128964.611op/s 15153198.061op/s 15165511.645op/s 15175265.834op/s 0.46% -0.789 1.319 0.25% 2697.295op/s 1 200
credit_card/is_card_number_no_luhn/ 378282246310005 execution_time 59.387µs 59.512µs ± 0.062µs 59.505µs ± 0.042µs 59.552µs 59.613µs 59.658µs 59.755µs 0.42% 0.502 0.500 0.10% 0.004µs 1 200
credit_card/is_card_number_no_luhn/ 378282246310005 throughput 16734899.567op/s 16803408.493op/s ± 17360.182op/s 16805386.967op/s ± 11865.909op/s 16815820.324op/s 16829072.222op/s 16837732.449op/s 16838648.065op/s 0.20% -0.495 0.484 0.10% 1227.550op/s 1 200
credit_card/is_card_number_no_luhn/37828224631 execution_time 3.893µs 3.910µs ± 0.002µs 3.910µs ± 0.001µs 3.911µs 3.914µs 3.916µs 3.917µs 0.18% -1.809 16.464 0.06% 0.000µs 1 200
credit_card/is_card_number_no_luhn/37828224631 throughput 255288296.564op/s 255728707.983op/s ± 148809.011op/s 255735687.943op/s ± 78406.646op/s 255814272.236op/s 255899615.775op/s 255961017.273op/s 256875875.005op/s 0.45% 1.836 16.688 0.06% 10522.386op/s 1 200
credit_card/is_card_number_no_luhn/378282246310005 execution_time 56.317µs 56.764µs ± 0.166µs 56.763µs ± 0.121µs 56.885µs 57.001µs 57.118µs 57.219µs 0.80% -0.086 -0.349 0.29% 0.012µs 1 200
credit_card/is_card_number_no_luhn/378282246310005 throughput 17476817.545op/s 17617084.083op/s ± 51594.282op/s 17617200.985op/s ± 37618.567op/s 17652575.118op/s 17704574.868op/s 17730361.354op/s 17756527.907op/s 0.79% 0.100 -0.349 0.29% 3648.267op/s 1 200
credit_card/is_card_number_no_luhn/37828224631000521389798 execution_time 51.345µs 51.406µs ± 0.036µs 51.403µs ± 0.021µs 51.424µs 51.475µs 51.518µs 51.558µs 0.30% 1.038 1.813 0.07% 0.003µs 1 200
credit_card/is_card_number_no_luhn/37828224631000521389798 throughput 19395525.999op/s 19453041.337op/s ± 13779.363op/s 19454204.802op/s ± 8031.246op/s 19462195.658op/s 19472266.723op/s 19475236.127op/s 19476100.877op/s 0.11% -1.032 1.793 0.07% 974.348op/s 1 200
credit_card/is_card_number_no_luhn/x371413321323331 execution_time 6.027µs 6.034µs ± 0.004µs 6.034µs ± 0.002µs 6.036µs 6.041µs 6.047µs 6.052µs 0.30% 1.340 2.971 0.06% 0.000µs 1 200
credit_card/is_card_number_no_luhn/x371413321323331 throughput 165237088.297op/s 165720437.874op/s ± 106199.814op/s 165735434.237op/s ± 51944.001op/s 165785960.020op/s 165862681.435op/s 165889709.149op/s 165909145.068op/s 0.10% -1.334 2.946 0.06% 7509.461op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
credit_card/is_card_number/ execution_time [3.911µs; 3.912µs] or [-0.010%; +0.010%] None None None
credit_card/is_card_number/ throughput [255629316.847op/s; 255678716.873op/s] or [-0.010%; +0.010%] None None None
credit_card/is_card_number/ 3782-8224-6310-005 execution_time [84.011µs; 84.153µs] or [-0.085%; +0.085%] None None None
credit_card/is_card_number/ 3782-8224-6310-005 throughput [11883686.679op/s; 11903510.862op/s] or [-0.083%; +0.083%] None None None
credit_card/is_card_number/ 378282246310005 execution_time [79.834µs; 79.967µs] or [-0.083%; +0.083%] None None None
credit_card/is_card_number/ 378282246310005 throughput [12505782.876op/s; 12526327.121op/s] or [-0.082%; +0.082%] None None None
credit_card/is_card_number/37828224631 execution_time [3.911µs; 3.911µs] or [-0.010%; +0.010%] None None None
credit_card/is_card_number/37828224631 throughput [255671330.809op/s; 255721306.146op/s] or [-0.010%; +0.010%] None None None
credit_card/is_card_number/378282246310005 execution_time [76.900µs; 76.971µs] or [-0.047%; +0.047%] None None None
credit_card/is_card_number/378282246310005 throughput [12991990.588op/s; 13004110.021op/s] or [-0.047%; +0.047%] None None None
credit_card/is_card_number/37828224631000521389798 execution_time [51.390µs; 51.397µs] or [-0.007%; +0.007%] None None None
credit_card/is_card_number/37828224631000521389798 throughput [19456440.651op/s; 19459106.725op/s] or [-0.007%; +0.007%] None None None
credit_card/is_card_number/x371413321323331 execution_time [6.034µs; 6.035µs] or [-0.007%; +0.007%] None None None
credit_card/is_card_number/x371413321323331 throughput [165710237.414op/s; 165734773.152op/s] or [-0.007%; +0.007%] None None None
credit_card/is_card_number_no_luhn/ execution_time [3.910µs; 3.911µs] or [-0.009%; +0.009%] None None None
credit_card/is_card_number_no_luhn/ throughput [255696367.330op/s; 255739968.305op/s] or [-0.009%; +0.009%] None None None
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 execution_time [66.201µs; 66.248µs] or [-0.035%; +0.035%] None None None
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 throughput [15094958.745op/s; 15105531.947op/s] or [-0.035%; +0.035%] None None None
credit_card/is_card_number_no_luhn/ 378282246310005 execution_time [59.503µs; 59.520µs] or [-0.014%; +0.014%] None None None
credit_card/is_card_number_no_luhn/ 378282246310005 throughput [16801002.539op/s; 16805814.448op/s] or [-0.014%; +0.014%] None None None
credit_card/is_card_number_no_luhn/37828224631 execution_time [3.910µs; 3.911µs] or [-0.008%; +0.008%] None None None
credit_card/is_card_number_no_luhn/37828224631 throughput [255708084.485op/s; 255749331.480op/s] or [-0.008%; +0.008%] None None None
credit_card/is_card_number_no_luhn/378282246310005 execution_time [56.741µs; 56.787µs] or [-0.041%; +0.041%] None None None
credit_card/is_card_number_no_luhn/378282246310005 throughput [17609933.612op/s; 17624234.555op/s] or [-0.041%; +0.041%] None None None
credit_card/is_card_number_no_luhn/37828224631000521389798 execution_time [51.401µs; 51.411µs] or [-0.010%; +0.010%] None None None
credit_card/is_card_number_no_luhn/37828224631000521389798 throughput [19451131.650op/s; 19454951.024op/s] or [-0.010%; +0.010%] None None None
credit_card/is_card_number_no_luhn/x371413321323331 execution_time [6.034µs; 6.035µs] or [-0.009%; +0.009%] None None None
credit_card/is_card_number_no_luhn/x371413321323331 throughput [165705719.601op/s; 165735156.147op/s] or [-0.009%; +0.009%] None None None

Group 10

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz c2618a2 1747867979 levi/datadog-profiling-core
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
tags/replace_trace_tags execution_time 2.305µs 2.371µs ± 0.017µs 2.371µs ± 0.005µs 2.377µs 2.392µs 2.403µs 2.406µs 1.49% -1.677 4.370 0.71% 0.001µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
tags/replace_trace_tags execution_time [2.368µs; 2.373µs] or [-0.098%; +0.098%] None None None

Group 11

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz c2618a2 1747867979 levi/datadog-profiling-core
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
write only interface execution_time 1.212µs 3.165µs ± 1.425µs 2.965µs ± 0.024µs 2.984µs 3.645µs 13.958µs 14.921µs 403.20% 7.403 55.699 44.92% 0.101µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
write only interface execution_time [2.967µs; 3.362µs] or [-6.241%; +6.241%] None None None

Group 12

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz c2618a2 1747867979 levi/datadog-profiling-core
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
benching deserializing traces from msgpack to their internal representation execution_time 72.023ms 72.263ms ± 0.193ms 72.206ms ± 0.072ms 72.294ms 72.647ms 72.972ms 73.159ms 1.32% 1.983 4.394 0.27% 0.014ms 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
benching deserializing traces from msgpack to their internal representation execution_time [72.236ms; 72.289ms] or [-0.037%; +0.037%] None None None

Group 13

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz c2618a2 1747867979 levi/datadog-profiling-core
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
sql/obfuscate_sql_string execution_time 67.391µs 67.610µs ± 0.200µs 67.592µs ± 0.042µs 67.639µs 67.728µs 68.013µs 70.004µs 3.57% 9.211 103.781 0.29% 0.014µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
sql/obfuscate_sql_string execution_time [67.582µs; 67.638µs] or [-0.041%; +0.041%] None None None

Baseline

Omitted due to size.

@r1viollet
Copy link
Contributor

r1viollet commented May 16, 2025 •
edited
Loading

Artifact Size Benchmark Report

aarch64-alpine-linux-musl
Artifact Baseline Commit Change
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.so.debug 20.31 MB 20.31 MB +.01% (+3.88 KB) 🔍
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.a 66.02 MB 66.05 MB +.04% (+33.69 KB) 🔍
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.so 8.03 MB 8.03 MB +0% (+48 B) 👌
aarch64-unknown-linux-gnu
Artifact Baseline Commit Change
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.a 72.62 MB 72.61 MB -0% (-4.45 KB) 👌
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.so 8.00 MB 8.00 MB --.01% (-944 B) 💪
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.so.debug 22.70 MB 22.69 MB --.04% (-9.53 KB) 💪
libdatadog-x64-windows
Artifact Baseline Commit Change
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.dll 16.71 MB 16.73 MB +.11% (+20.50 KB) 🔍
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.lib 62.31 KB 62.31 KB 0% (0 B) 👌
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.pdb 112.66 MB 112.83 MB +.14% (+168.00 KB) 🔍
/libdatadog-x64-windows/debug/static/datadog_profiling_ffi.lib 631.82 MB 632.43 MB +.09% (+622.39 KB) 🔍
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.dll 4.99 MB 5.00 MB +.09% (+5.00 KB) 🔍
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.lib 62.31 KB 62.31 KB 0% (0 B) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.pdb 15.88 MB 15.87 MB --.04% (-8.00 KB) 💪
/libdatadog-x64-windows/release/static/datadog_profiling_ffi.lib 26.43 MB 26.42 MB -0% (-2.21 KB) 👌
libdatadog-x86-windows
Artifact Baseline Commit Change
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.dll 14.15 MB 14.15 MB +0% (+512 B) 👌
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.lib 63.26 KB 63.26 KB 0% (0 B) 👌
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.pdb 114.51 MB 114.57 MB +.04% (+56.00 KB) 🔍
/libdatadog-x86-windows/debug/static/datadog_profiling_ffi.lib 624.40 MB 624.46 MB +0% (+59.46 KB) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.dll 3.78 MB 3.78 MB +.07% (+3.00 KB) 🔍
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.lib 63.26 KB 63.26 KB 0% (0 B) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.pdb 16.50 MB 16.51 MB +.04% (+8.00 KB) 🔍
/libdatadog-x86-windows/release/static/datadog_profiling_ffi.lib 24.34 MB 24.35 MB +.03% (+9.69 KB) 🔍
x86_64-alpine-linux-musl
Artifact Baseline Commit Change
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.a 57.50 MB 57.52 MB +.02% (+15.82 KB) 🔍
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.so 8.43 MB 8.43 MB +0% (+400 B) 👌
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.so.debug 18.71 MB 18.71 MB -0% (-1.17 KB) 👌
x86_64-unknown-linux-gnu
Artifact Baseline Commit Change
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.a 68.61 MB 68.60 MB --.01% (-8.32 KB) 💪
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.so 8.43 MB 8.42 MB --.05% (-4.38 KB) 💪
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.so.debug 20.90 MB 20.88 MB --.06% (-14.10 KB) 💪

@morrisonlevi morrisonlevi force-pushed the levi/datadog-profiling-core branch from 8b9c49d to 8789730 Compare May 21, 2025 20:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
profiling Relates to the profiling* modules.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@morrisonlevi @codecov-commenter @r1viollet