feat(profiling): zstd as a compression option

[morrisonlevi]

What does this PR do?

Adds zstd compression as on option for upload config.

Motivation

zstd may have a better tradeoff for performance to compressed size.

Here are some benchmarks on compressors for a CPU profile which I cannot share publicly yet. We're working on anonymizing it so we can commit these to the repository. Throughput is in MiB/s. Utility is the compression ratio multiplied by throughput. This is on an Apple M1 Max.

Algo.      Size     Ratio    Throughput     Utility
None    2363410     1.000     11,746.     11,746.
lz4     1215720     1.944        459.23      892.74
zstd-1   778653     3.035        518.39    1,573.31
zstd-2   745217     3.171        350.07    1,110.07
zstd-3   698466     3.384        277.35      938.55
zstd-4   694724     3.402        270.86      921.47

I tested zstd-5 and zstd-6 informally, and compression rations were incrementally better, but you lose throughput steadily. Sticking to something in the level 1-4 range seems better.

And here's from another, also not public set of data:

Algo.      Size     Ratio    Throughput     Utility
None    5804495     1.0000    11.901.     11,901.
lz4     1467466     3.9554       760.52    3,008.1
zstd-1   746324     7.7774       728.48    5,665.6
zstd-2   727673     7.9768       639.62    5,102.1
zstd-3   696252     8.3368       585.82    4,883.9
zstd-4   696241     8.3369       569.99    4,751.9

My personal opinion is that zstd-1 is a safe upgrade. The performance is roughly similar to lz4, but the compression is much better.. Moving to a higher level is something that could be discussed.

Additional Notes

Currently lacks tests, pushed to see if this will cause build issues or significant artifact size bloat.

How to test the change?

Use Zstd instead of On or Lz4, and most things should work the same, except it's compressed differently.

[r1viollet]

Artifact Size Benchmark Report

aarch64-alpine-linux-musl

Artifact	Baseline	Commit	Change
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.so	8.03 MB	8.30 MB	+3.43% (+282.67 KB) ⚠️
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.so.debug	20.31 MB	23.84 MB	+17.38% (+3.53 MB) 🚨
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.a	66.02 MB	77.64 MB	+17.59% (+11.61 MB) 🚨

aarch64-unknown-linux-gnu

Artifact	Baseline	Commit	Change
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.so	8.00 MB	8.28 MB	+3.44% (+282.53 KB) ⚠️
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.so.debug	22.70 MB	26.41 MB	+16.32% (+3.70 MB) 🚨
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.a	72.62 MB	85.16 MB	+17.27% (+12.54 MB) 🚨

libdatadog-x64-windows

Artifact	Baseline	Commit	Change
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.dll	16.71 MB	17.21 MB	+3.00% (+514.50 KB) ⚠️
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.lib	62.31 KB	62.64 KB	+.53% (+340 B) 🔍
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.pdb	112.66 MB	113.67 MB	+.89% (+1.00 MB) 🔍
/libdatadog-x64-windows/debug/static/datadog_profiling_ffi.lib	631.82 MB	634.66 MB	+.44% (+2.84 MB) 🔍
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.dll	4.99 MB	5.34 MB	+7.01% (+359.00 KB) 🚨
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.lib	62.31 KB	62.64 KB	+.53% (+340 B) 🔍
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.pdb	15.88 MB	17.23 MB	+8.50% (+1.35 MB) 🚨
/libdatadog-x64-windows/release/static/datadog_profiling_ffi.lib	26.43 MB	30.92 MB	+17.00% (+4.49 MB) 🚨

libdatadog-x86-windows

Artifact	Baseline	Commit	Change
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.dll	14.15 MB	14.50 MB	+2.47% (+359.00 KB) ⚠️
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.lib	63.26 KB	63.60 KB	+.53% (+344 B) 🔍
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.pdb	114.52 MB	115.48 MB	+.83% (+976.00 KB) 🔍
/libdatadog-x86-windows/debug/static/datadog_profiling_ffi.lib	624.40 MB	627.28 MB	+.46% (+2.88 MB) 🔍
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.dll	3.78 MB	4.07 MB	+7.78% (+301.50 KB) 🚨
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.lib	63.26 KB	63.60 KB	+.53% (+344 B) 🔍
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.pdb	16.49 MB	17.87 MB	+8.33% (+1.37 MB) 🚨
/libdatadog-x86-windows/release/static/datadog_profiling_ffi.lib	24.34 MB	28.54 MB	+17.26% (+4.20 MB) 🚨

x86_64-alpine-linux-musl

Artifact	Baseline	Commit	Change
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.a	57.50 MB	65.34 MB	+13.63% (+7.83 MB) 🚨
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.so	8.43 MB	8.73 MB	+3.55% (+306.79 KB) ⚠️
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.so.debug	18.71 MB	22.01 MB	+17.64% (+3.30 MB) 🚨

x86_64-unknown-linux-gnu

Artifact	Baseline	Commit	Change
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.a	68.61 MB	81.24 MB	+18.41% (+12.63 MB) 🚨
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.so	8.43 MB	8.72 MB	+3.45% (+298.35 KB) ⚠️
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.so.debug	20.90 MB	24.38 MB	+16.65% (+3.48 MB) 🚨

[pr-commenter]

Benchmarks

Comparison

Benchmark execution time: 2025-05-29 17:59:41

Comparing candidate commit b751d09 in PR branch PROF-11832-Add-zstd-to-compression-configuration-to-libdatadog with baseline commit 35933df in branch PROF-11830-Add-upload-compression-configuration-to-libdatadog.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 52 metrics, 2 unstable metrics.

Candidate

Candidate benchmark details

Group 1

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	b751d09	1748540931	PROF-11832-Add-zstd-to-compression-configuration-to-libdatadog

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
benching deserializing traces from msgpack to their internal representation	execution_time	73.684ms	74.571ms ± 0.456ms	74.617ms ± 0.353ms	74.936ms	75.220ms	75.418ms	75.941ms	1.77%	-0.084	-0.766	0.61%	0.032ms	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
benching deserializing traces from msgpack to their internal representation	execution_time	[74.507ms; 74.634ms] or [-0.085%; +0.085%]	None	None	None

Group 2

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	b751d09	1748540931	PROF-11832-Add-zstd-to-compression-configuration-to-libdatadog

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
redis/obfuscate_redis_string	execution_time	31.874µs	32.825µs ± 1.360µs	31.961µs ± 0.053µs	34.728µs	35.139µs	35.579µs	36.809µs	15.17%	0.976	-0.858	4.13%	0.096µs	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
redis/obfuscate_redis_string	execution_time	[32.637µs; 33.014µs] or [-0.574%; +0.574%]	None	None	None

Group 3

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	b751d09	1748540931	PROF-11832-Add-zstd-to-compression-configuration-to-libdatadog

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
ip_address/quantize_peer_ip_address_benchmark	execution_time	4.936µs	5.023µs ± 0.044µs	5.039µs ± 0.030µs	5.057µs	5.088µs	5.092µs	5.095µs	1.11%	-0.305	-1.149	0.88%	0.003µs	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
ip_address/quantize_peer_ip_address_benchmark	execution_time	[5.017µs; 5.029µs] or [-0.123%; +0.123%]	None	None	None

Group 4

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	b751d09	1748540931	PROF-11832-Add-zstd-to-compression-configuration-to-libdatadog

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
tags/replace_trace_tags	execution_time	2.321µs	2.376µs ± 0.023µs	2.372µs ± 0.008µs	2.383µs	2.421µs	2.446µs	2.449µs	3.26%	0.875	2.126	0.96%	0.002µs	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
tags/replace_trace_tags	execution_time	[2.373µs; 2.379µs] or [-0.133%; +0.133%]	None	None	None

Group 5

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	b751d09	1748540931	PROF-11832-Add-zstd-to-compression-configuration-to-libdatadog

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
sql/obfuscate_sql_string	execution_time	65.301µs	65.488µs ± 0.244µs	65.442µs ± 0.051µs	65.503µs	65.742µs	66.095µs	68.260µs	4.31%	8.059	84.073	0.37%	0.017µs	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
sql/obfuscate_sql_string	execution_time	[65.454µs; 65.521µs] or [-0.052%; +0.052%]	None	None	None

Group 6

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	b751d09	1748540931	PROF-11832-Add-zstd-to-compression-configuration-to-libdatadog

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo...	execution_time	217.939µs	218.565µs ± 0.317µs	218.571µs ± 0.254µs	218.797µs	219.082µs	219.301µs	219.550µs	0.45%	0.265	-0.423	0.14%	0.022µs	1	200
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo...	throughput	4554764.010op/s	4575308.905op/s ± 6623.705op/s	4575170.874op/s ± 5308.133op/s	4581389.297op/s	4585267.020op/s	4586850.881op/s	4588447.103op/s	0.29%	-0.258	-0.434	0.14%	468.367op/s	1	200
normalization/normalize_name/normalize_name/bad-name	execution_time	17.519µs	17.626µs ± 0.038µs	17.626µs ± 0.026µs	17.649µs	17.690µs	17.711µs	17.792µs	0.95%	0.417	1.064	0.22%	0.003µs	1	200
normalization/normalize_name/normalize_name/bad-name	throughput	56203867.960op/s	56735271.529op/s ± 123771.571op/s	56735911.514op/s ± 82907.593op/s	56825242.940op/s	56924507.615op/s	56972332.746op/s	57082362.174op/s	0.61%	-0.398	1.015	0.22%	8751.972op/s	1	200
normalization/normalize_name/normalize_name/good	execution_time	10.335µs	10.413µs ± 0.026µs	10.413µs ± 0.015µs	10.430µs	10.455µs	10.468µs	10.503µs	0.87%	-0.137	0.984	0.25%	0.002µs	1	200
normalization/normalize_name/normalize_name/good	throughput	95209092.977op/s	96031505.971op/s ± 242381.936op/s	96033839.168op/s ± 141044.691op/s	96171158.233op/s	96458564.318op/s	96627678.092op/s	96763232.926op/s	0.76%	0.159	0.977	0.25%	17138.991op/s	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo...	execution_time	[218.521µs; 218.609µs] or [-0.020%; +0.020%]	None	None	None
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo...	throughput	[4574390.923op/s; 4576226.886op/s] or [-0.020%; +0.020%]	None	None	None
normalization/normalize_name/normalize_name/bad-name	execution_time	[17.620µs; 17.631µs] or [-0.030%; +0.030%]	None	None	None
normalization/normalize_name/normalize_name/bad-name	throughput	[56718117.979op/s; 56752425.078op/s] or [-0.030%; +0.030%]	None	None	None
normalization/normalize_name/normalize_name/good	execution_time	[10.410µs; 10.417µs] or [-0.035%; +0.035%]	None	None	None
normalization/normalize_name/normalize_name/good	throughput	[95997914.166op/s; 96065097.776op/s] or [-0.035%; +0.035%]	None	None	None

Group 7

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	b751d09	1748540931	PROF-11832-Add-zstd-to-compression-configuration-to-libdatadog

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
write only interface	execution_time	1.208µs	3.152µs ± 1.430µs	2.950µs ± 0.030µs	2.984µs	3.624µs	13.901µs	15.043µs	409.90%	7.421	55.966	45.26%	0.101µs	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
write only interface	execution_time	[2.954µs; 3.350µs] or [-6.289%; +6.289%]	None	None	None

Group 8

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	b751d09	1748540931	PROF-11832-Add-zstd-to-compression-configuration-to-libdatadog

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
credit_card/is_card_number/	execution_time	3.889µs	3.911µs ± 0.003µs	3.911µs ± 0.002µs	3.913µs	3.915µs	3.917µs	3.917µs	0.16%	-1.631	12.458	0.08%	0.000µs	1	200
credit_card/is_card_number/	throughput	255285314.535op/s	255695910.632op/s ± 193633.823op/s	255704820.108op/s ± 111658.012op/s	255805801.286op/s	255946336.832op/s	255988165.140op/s	257110017.044op/s	0.55%	1.658	12.687	0.08%	13691.979op/s	1	200
credit_card/is_card_number/ 3782-8224-6310-005	execution_time	76.715µs	77.553µs ± 0.419µs	77.516µs ± 0.287µs	77.767µs	78.224µs	78.735µs	78.875µs	1.75%	0.765	0.354	0.54%	0.030µs	1	200
credit_card/is_card_number/ 3782-8224-6310-005	throughput	12678298.399op/s	12894735.728op/s ± 69446.324op/s	12900613.425op/s ± 47938.524op/s	12950291.812op/s	12982740.845op/s	13001294.018op/s	13035269.600op/s	1.04%	-0.737	0.287	0.54%	4910.597op/s	1	200
credit_card/is_card_number/ 378282246310005	execution_time	70.833µs	71.356µs ± 0.261µs	71.318µs ± 0.171µs	71.498µs	71.857µs	72.151µs	72.244µs	1.30%	0.845	0.782	0.36%	0.018µs	1	200
credit_card/is_card_number/ 378282246310005	throughput	13841916.090op/s	14014446.849op/s ± 51060.493op/s	14021651.220op/s ± 33658.648op/s	14052635.259op/s	14082684.572op/s	14100389.615op/s	14117696.533op/s	0.68%	-0.823	0.728	0.36%	3610.522op/s	1	200
credit_card/is_card_number/37828224631	execution_time	3.893µs	3.911µs ± 0.003µs	3.910µs ± 0.002µs	3.912µs	3.915µs	3.917µs	3.917µs	0.17%	-0.979	7.204	0.07%	0.000µs	1	200
credit_card/is_card_number/37828224631	throughput	255308769.287op/s	255718617.047op/s ± 177099.766op/s	255735411.903op/s ± 120170.045op/s	255840349.224op/s	255923366.872op/s	256011210.364op/s	256861162.622op/s	0.44%	0.996	7.324	0.07%	12522.845op/s	1	200
credit_card/is_card_number/378282246310005	execution_time	67.106µs	67.864µs ± 0.355µs	67.821µs ± 0.232µs	68.090µs	68.451µs	68.919µs	69.181µs	2.00%	0.567	0.498	0.52%	0.025µs	1	200
credit_card/is_card_number/378282246310005	throughput	14454923.396op/s	14735741.362op/s ± 76828.633op/s	14744683.185op/s ± 50543.476op/s	14790142.379op/s	14849812.061op/s	14882841.243op/s	14901870.678op/s	1.07%	-0.533	0.420	0.52%	5432.605op/s	1	200
credit_card/is_card_number/37828224631000521389798	execution_time	51.740µs	51.808µs ± 0.039µs	51.800µs ± 0.023µs	51.826µs	51.899µs	51.908µs	51.964µs	0.32%	1.080	1.231	0.08%	0.003µs	1	200
credit_card/is_card_number/37828224631000521389798	throughput	19244109.244op/s	19301916.254op/s ± 14651.063op/s	19304934.779op/s ± 8590.841op/s	19312942.566op/s	19320786.708op/s	19324001.485op/s	19327404.337op/s	0.12%	-1.076	1.216	0.08%	1035.987op/s	1	200
credit_card/is_card_number/x371413321323331	execution_time	6.044µs	6.182µs ± 0.057µs	6.183µs ± 0.042µs	6.225µs	6.283µs	6.307µs	6.312µs	2.08%	0.078	-0.422	0.92%	0.004µs	1	200
credit_card/is_card_number/x371413321323331	throughput	158430262.327op/s	161783555.983op/s ± 1495102.913op/s	161721569.855op/s ± 1099754.357op/s	162828335.432op/s	164141856.177op/s	164902754.414op/s	165463704.061op/s	2.31%	-0.034	-0.422	0.92%	105719.741op/s	1	200
credit_card/is_card_number_no_luhn/	execution_time	3.893µs	3.912µs ± 0.003µs	3.911µs ± 0.002µs	3.913µs	3.916µs	3.920µs	3.921µs	0.24%	-0.636	6.683	0.08%	0.000µs	1	200
credit_card/is_card_number_no_luhn/	throughput	255047181.235op/s	255643471.327op/s ± 201895.764op/s	255661497.383op/s ± 115338.067op/s	255768373.283op/s	255903554.644op/s	255986389.908op/s	256894895.226op/s	0.48%	0.656	6.793	0.08%	14276.186op/s	1	200
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005	execution_time	65.947µs	66.245µs ± 0.160µs	66.221µs ± 0.081µs	66.306µs	66.588µs	66.772µs	66.854µs	0.96%	1.425	2.747	0.24%	0.011µs	1	200
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005	throughput	14957971.073op/s	15095596.050op/s ± 36311.805op/s	15100900.014op/s ± 18486.063op/s	15119310.005op/s	15140738.959op/s	15151959.022op/s	15163583.940op/s	0.42%	-1.405	2.682	0.24%	2567.632op/s	1	200
credit_card/is_card_number_no_luhn/ 378282246310005	execution_time	59.395µs	59.556µs ± 0.070µs	59.558µs ± 0.046µs	59.600µs	59.676µs	59.696µs	59.749µs	0.32%	0.183	-0.397	0.12%	0.005µs	1	200
credit_card/is_card_number_no_luhn/ 378282246310005	throughput	16736653.440op/s	16790883.864op/s ± 19714.980op/s	16790463.857op/s ± 13074.744op/s	16806068.251op/s	16821426.376op/s	16829807.137op/s	16836376.675op/s	0.27%	-0.177	-0.401	0.12%	1394.060op/s	1	200
credit_card/is_card_number_no_luhn/37828224631	execution_time	3.892µs	3.911µs ± 0.003µs	3.911µs ± 0.002µs	3.913µs	3.917µs	3.919µs	3.919µs	0.22%	-0.658	6.964	0.08%	0.000µs	1	200
credit_card/is_card_number_no_luhn/37828224631	throughput	255135855.854op/s	255668038.544op/s ± 202675.244op/s	255688724.267op/s ± 108199.499op/s	255791008.954op/s	255930845.932op/s	255993166.575op/s	256940559.142op/s	0.49%	0.679	7.084	0.08%	14331.304op/s	1	200
credit_card/is_card_number_no_luhn/378282246310005	execution_time	56.170µs	56.285µs ± 0.092µs	56.265µs ± 0.046µs	56.317µs	56.428µs	56.662µs	56.768µs	0.89%	2.327	8.127	0.16%	0.007µs	1	200
credit_card/is_card_number_no_luhn/378282246310005	throughput	17615565.034op/s	17766865.055op/s ± 28940.835op/s	17772944.255op/s ± 14528.756op/s	17785583.008op/s	17797156.282op/s	17801070.486op/s	17803122.668op/s	0.17%	-2.304	7.981	0.16%	2046.426op/s	1	200
credit_card/is_card_number_no_luhn/37828224631000521389798	execution_time	51.742µs	51.814µs ± 0.059µs	51.811µs ± 0.023µs	51.831µs	51.869µs	51.896µs	52.312µs	0.97%	5.494	42.874	0.11%	0.004µs	1	200
credit_card/is_card_number_no_luhn/37828224631000521389798	throughput	19115910.803op/s	19299993.981op/s ± 21756.931op/s	19300927.375op/s ± 8745.654op/s	19311639.113op/s	19321228.031op/s	19325334.695op/s	19326740.059op/s	0.13%	-5.444	42.340	0.11%	1538.447op/s	1	200
credit_card/is_card_number_no_luhn/x371413321323331	execution_time	6.035µs	6.182µs ± 0.054µs	6.184µs ± 0.038µs	6.219µs	6.270µs	6.303µs	6.310µs	2.04%	0.051	-0.341	0.88%	0.004µs	1	200
credit_card/is_card_number_no_luhn/x371413321323331	throughput	158466298.515op/s	161785328.994op/s ± 1423437.798op/s	161701672.873op/s ± 986214.807op/s	162775912.471op/s	163968696.911op/s	164903578.280op/s	165712258.183op/s	2.48%	-0.007	-0.336	0.88%	100652.252op/s	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
credit_card/is_card_number/	execution_time	[3.910µs; 3.911µs] or [-0.010%; +0.010%]	None	None	None
credit_card/is_card_number/	throughput	[255669074.846op/s; 255722746.417op/s] or [-0.010%; +0.010%]	None	None	None
credit_card/is_card_number/ 3782-8224-6310-005	execution_time	[77.495µs; 77.611µs] or [-0.075%; +0.075%]	None	None	None
credit_card/is_card_number/ 3782-8224-6310-005	throughput	[12885111.136op/s; 12904360.321op/s] or [-0.075%; +0.075%]	None	None	None
credit_card/is_card_number/ 378282246310005	execution_time	[71.320µs; 71.392µs] or [-0.051%; +0.051%]	None	None	None
credit_card/is_card_number/ 378282246310005	throughput	[14007370.356op/s; 14021523.342op/s] or [-0.050%; +0.050%]	None	None	None
credit_card/is_card_number/37828224631	execution_time	[3.910µs; 3.911µs] or [-0.010%; +0.010%]	None	None	None
credit_card/is_card_number/37828224631	throughput	[255694072.723op/s; 255743161.371op/s] or [-0.010%; +0.010%]	None	None	None
credit_card/is_card_number/378282246310005	execution_time	[67.815µs; 67.913µs] or [-0.072%; +0.072%]	None	None	None
credit_card/is_card_number/378282246310005	throughput	[14725093.653op/s; 14746389.072op/s] or [-0.072%; +0.072%]	None	None	None
credit_card/is_card_number/37828224631000521389798	execution_time	[51.803µs; 51.814µs] or [-0.011%; +0.011%]	None	None	None
credit_card/is_card_number/37828224631000521389798	throughput	[19299885.757op/s; 19303946.750op/s] or [-0.011%; +0.011%]	None	None	None
credit_card/is_card_number/x371413321323331	execution_time	[6.174µs; 6.190µs] or [-0.128%; +0.128%]	None	None	None
credit_card/is_card_number/x371413321323331	throughput	[161576349.099op/s; 161990762.868op/s] or [-0.128%; +0.128%]	None	None	None
credit_card/is_card_number_no_luhn/	execution_time	[3.911µs; 3.912µs] or [-0.011%; +0.011%]	None	None	None
credit_card/is_card_number_no_luhn/	throughput	[255615490.516op/s; 255671452.138op/s] or [-0.011%; +0.011%]	None	None	None
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005	execution_time	[66.223µs; 66.267µs] or [-0.033%; +0.033%]	None	None	None
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005	throughput	[15090563.583op/s; 15100628.517op/s] or [-0.033%; +0.033%]	None	None	None
credit_card/is_card_number_no_luhn/ 378282246310005	execution_time	[59.547µs; 59.566µs] or [-0.016%; +0.016%]	None	None	None
credit_card/is_card_number_no_luhn/ 378282246310005	throughput	[16788151.558op/s; 16793616.171op/s] or [-0.016%; +0.016%]	None	None	None
credit_card/is_card_number_no_luhn/37828224631	execution_time	[3.911µs; 3.912µs] or [-0.011%; +0.011%]	None	None	None
credit_card/is_card_number_no_luhn/37828224631	throughput	[255639949.705op/s; 255696127.384op/s] or [-0.011%; +0.011%]	None	None	None
credit_card/is_card_number_no_luhn/378282246310005	execution_time	[56.272µs; 56.297µs] or [-0.023%; +0.023%]	None	None	None
credit_card/is_card_number_no_luhn/378282246310005	throughput	[17762854.133op/s; 17770875.976op/s] or [-0.023%; +0.023%]	None	None	None
credit_card/is_card_number_no_luhn/37828224631000521389798	execution_time	[51.805µs; 51.822µs] or [-0.016%; +0.016%]	None	None	None
credit_card/is_card_number_no_luhn/37828224631000521389798	throughput	[19296978.679op/s; 19303009.282op/s] or [-0.016%; +0.016%]	None	None	None
credit_card/is_card_number_no_luhn/x371413321323331	execution_time	[6.174µs; 6.189µs] or [-0.122%; +0.122%]	None	None	None
credit_card/is_card_number_no_luhn/x371413321323331	throughput	[161588054.206op/s; 161982603.783op/s] or [-0.122%; +0.122%]	None	None	None

Group 9

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	b751d09	1748540931	PROF-11832-Add-zstd-to-compression-configuration-to-libdatadog

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
two way interface	execution_time	17.549µs	25.113µs ± 11.176µs	17.837µs ± 0.193µs	34.567µs	44.456µs	56.775µs	96.970µs	443.64%	2.305	9.365	44.39%	0.790µs	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
two way interface	execution_time	[23.564µs; 26.662µs] or [-6.167%; +6.167%]	None	None	None

Group 10

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	b751d09	1748540931	PROF-11832-Add-zstd-to-compression-configuration-to-libdatadog

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
normalization/normalize_trace/test_trace	execution_time	246.485ns	255.981ns ± 12.870ns	251.111ns ± 2.944ns	256.049ns	286.614ns	299.075ns	300.606ns	19.71%	2.116	3.560	5.02%	0.910ns	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
normalization/normalize_trace/test_trace	execution_time	[254.198ns; 257.765ns] or [-0.697%; +0.697%]	None	None	None

Group 11

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	b751d09	1748540931	PROF-11832-Add-zstd-to-compression-configuration-to-libdatadog

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
benching string interning on wordpress profile	execution_time	147.987µs	148.807µs ± 0.325µs	148.785µs ± 0.154µs	148.944µs	149.257µs	149.616µs	151.475µs	1.81%	2.946	21.675	0.22%	0.023µs	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
benching string interning on wordpress profile	execution_time	[148.762µs; 148.852µs] or [-0.030%; +0.030%]	None	None	None

Group 12

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	b751d09	1748540931	PROF-11832-Add-zstd-to-compression-configuration-to-libdatadog

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000...	execution_time	437.505µs	438.340µs ± 0.361µs	438.297µs ± 0.221µs	438.576µs	438.910µs	439.186µs	440.074µs	0.41%	0.717	1.845	0.08%	0.026µs	1	200
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000...	throughput	2272343.298op/s	2281333.908op/s ± 1877.422op/s	2281560.412op/s ± 1149.190op/s	2282560.244op/s	2284054.379op/s	2284759.396op/s	2285688.686op/s	0.18%	-0.709	1.812	0.08%	132.754op/s	1	200
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて	execution_time	452.455µs	453.365µs ± 0.435µs	453.311µs ± 0.261µs	453.588µs	454.141µs	454.429µs	455.769µs	0.54%	1.149	3.807	0.10%	0.031µs	1	200
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて	throughput	2194095.141op/s	2205729.284op/s ± 2114.997op/s	2205993.100op/s ± 1269.645op/s	2207152.155op/s	2208572.032op/s	2209586.879op/s	2210165.768op/s	0.19%	-1.136	3.735	0.10%	149.553op/s	1	200
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters	execution_time	179.061µs	179.651µs ± 0.413µs	179.610µs ± 0.173µs	179.813µs	180.073µs	180.429µs	184.158µs	2.53%	6.574	69.008	0.23%	0.029µs	1	200
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters	throughput	5430124.823op/s	5566366.488op/s ± 12612.773op/s	5567613.438op/s ± 5355.380op/s	5572639.888op/s	5579857.385op/s	5583222.131op/s	5584685.213op/s	0.31%	-6.385	66.283	0.23%	891.858op/s	1	200
normalization/normalize_service/normalize_service/[empty string]	execution_time	38.965µs	39.065µs ± 0.049µs	39.062µs ± 0.032µs	39.093µs	39.148µs	39.217µs	39.228µs	0.42%	0.621	0.765	0.13%	0.003µs	1	200
normalization/normalize_service/normalize_service/[empty string]	throughput	25491988.044op/s	25598421.074op/s ± 32282.229op/s	25600293.786op/s ± 20811.880op/s	25621198.096op/s	25646395.298op/s	25658774.170op/s	25664159.691op/s	0.25%	-0.612	0.745	0.13%	2282.698op/s	1	200
normalization/normalize_service/normalize_service/test_ASCII	execution_time	47.009µs	47.118µs ± 0.051µs	47.116µs ± 0.037µs	47.154µs	47.197µs	47.225µs	47.316µs	0.42%	0.303	0.233	0.11%	0.004µs	1	200
normalization/normalize_service/normalize_service/test_ASCII	throughput	21134484.902op/s	21223278.877op/s ± 22753.432op/s	21224172.093op/s ± 16530.709op/s	21239644.709op/s	21257313.741op/s	21268759.257op/s	21272542.621op/s	0.23%	-0.296	0.218	0.11%	1608.911op/s	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000...	execution_time	[438.290µs; 438.390µs] or [-0.011%; +0.011%]	None	None	None
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000...	throughput	[2281073.716op/s; 2281594.101op/s] or [-0.011%; +0.011%]	None	None	None
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて	execution_time	[453.305µs; 453.426µs] or [-0.013%; +0.013%]	None	None	None
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて	throughput	[2205436.166op/s; 2206022.403op/s] or [-0.013%; +0.013%]	None	None	None
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters	execution_time	[179.594µs; 179.709µs] or [-0.032%; +0.032%]	None	None	None
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters	throughput	[5564618.479op/s; 5568114.497op/s] or [-0.031%; +0.031%]	None	None	None
normalization/normalize_service/normalize_service/[empty string]	execution_time	[39.058µs; 39.072µs] or [-0.017%; +0.017%]	None	None	None
normalization/normalize_service/normalize_service/[empty string]	throughput	[25593947.067op/s; 25602895.080op/s] or [-0.017%; +0.017%]	None	None	None
normalization/normalize_service/normalize_service/test_ASCII	execution_time	[47.111µs; 47.125µs] or [-0.015%; +0.015%]	None	None	None
normalization/normalize_service/normalize_service/test_ASCII	throughput	[21220125.470op/s; 21226432.284op/s] or [-0.015%; +0.015%]	None	None	None

Group 13

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	b751d09	1748540931	PROF-11832-Add-zstd-to-compression-configuration-to-libdatadog

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
concentrator/add_spans_to_concentrator	execution_time	5.940ms	5.952ms ± 0.007ms	5.951ms ± 0.003ms	5.954ms	5.962ms	5.972ms	6.009ms	0.98%	3.960	26.494	0.12%	0.001ms	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
concentrator/add_spans_to_concentrator	execution_time	[5.951ms; 5.953ms] or [-0.017%; +0.017%]	None	None	None

Baseline

Omitted due to size.

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 0% with 3 lines in your changes missing coverage. Please review.

Project coverage is 70.91%. Comparing base (35933df) to head (b751d09).

Additional details and impacted files

@@                                        Coverage Diff                                        @@
##           PROF-11830-Add-upload-compression-configuration-to-libdatadog    #1065      +/-   ##
=================================================================================================
- Coverage                                                          70.92%   70.91%   -0.02%     
=================================================================================================
  Files                                                                329      329              
  Lines                                                              49916    49919       +3     
=================================================================================================
- Hits                                                               35403    35399       -4     
- Misses                                                             14513    14520       +7     

Components	Coverage Δ
datadog-crashtracker	42.58% <ø> (+0.02%)	⬆️
datadog-crashtracker-ffi	6.03% <ø> (ø)
datadog-alloc	98.73% <ø> (ø)
data-pipeline	90.61% <ø> (ø)
data-pipeline-ffi	89.26% <ø> (ø)
ddcommon	79.93% <ø> (ø)
ddcommon-ffi	68.34% <ø> (ø)
ddtelemetry	60.56% <ø> (ø)
ddtelemetry-ffi	21.32% <ø> (ø)
dogstatsd-client	83.26% <ø> (ø)
datadog-ipc	82.58% <ø> (-0.11%)	⬇️
datadog-profiling	77.18% <0.00%> (-0.04%)	⬇️
datadog-profiling-ffi	62.71% <ø> (ø)
datadog-sidecar	41.40% <ø> (ø)
datdog-sidecar-ffi	3.21% <ø> (ø)
spawn-worker	55.35% <ø> (ø)
tinybytes	90.96% <ø> (ø)
datadog-trace-normalization	98.24% <ø> (ø)
datadog-trace-obfuscation	94.16% <ø> (ø)
datadog-trace-protobuf	77.10% <ø> (ø)
datadog-trace-utils	89.44% <ø> (ø)

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[felixge]

Is that the level 3 from the upstream zstd C library? If yes, level 3 is good. I'm asking because there are different zstd implementations out there which assign different meanings to different levels.

[morrisonlevi]

Yes, at least as far as I understood it.