Skip to content

Commit 631d24a

Browse files
bwbushbwbush
authored
Experiments for Linear Leois (#457)
* Phase 1 of linear leios experiment - Designed linear leios experiment - Configuration files for LL experiment - Reran experiment but without txs - Aligned linear leios experiments - Finished tuninng transaction demand * Added txref linear leios experiment * Added stracciatella results * Script to combine panels of plots * Updated logbook
1 parent a15e946 commit 631d24a

File tree

141 files changed

+20343
-0
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

141 files changed

+20343
-0
lines changed

‎Logbook.md‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,18 @@
11
# Leios logbook
22

3+
## 2025-07-17
4+
5+
### Linear Leios experiment
6+
7+
A comprehensive set of simulations for the Linear Leios variant was completed, aiming to map out the protocol's performance under varies loads and comparing several versions of the variant to Stracciatella.
8+
9+
- [Slides](analysis/sims/2025w29/summary.pdf)
10+
- Jupyter notebooks
11+
- [Linear Leios with embedded transactions](analysis/sims/2025w29/linear.ipynb)
12+
- [Linear Leios with transaction references](analysis/sims/2025w29/txrefs.ipynb)
13+
- [Linear Leios without transactions](analysis/sims/2025w29/notxs.ipynb)
14+
- [Stracciatella](analysis/sims/2025w29/stracciatella.ipynb)
15+
316
## 2025-07-11
417

518
### Stracciatella experiment

‎analysis/sims/2025w29/combine-plots.sh‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
#!/usr/bin/env nix-shell
2+
#!nix-shell -i bash -p imagemagick poppler
3+
4+
set -e
5+
6+
for d in plots/stracciatella/*.png
7+
do
8+
FILE=$(basename $d)
9+
if [[ -f "plots/notxs/$FILE" ]]
10+
then
11+
magick montage plots/{txrefs,stracciatella,linear,notxs}/$FILE -tile 2x2 -geometry +0+0 plots/$FILE
12+
else
13+
magick montage plots/{txrefs,stracciatella,linear}/$FILE -tile 2x2 -geometry +0+0 plots/$FILE
14+
fi
15+
done
16+
17+
magick plots/{sizes,spatial-efficiency,reach-eb-tx,reach-rb-tx,temporal-efficiency-bar,elapsed-EB,elapsed-RB,elapsed-TX,elapsed-VT,ingress-average-area,ingress-peak-point,cpu-peak-histogram,cpu-mean-histogram}.png content.pdf
18+
19+
pdfunite front-matter.pdf content.pdf plots.pdf

‎analysis/sims/2025w29/combine-results.sh‎

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
#!/usr/bin/env nix-shell
2+
#!nix-shell -i bash -p gnused gzip pigz "rWrapper.override { packages = with rPackages; [ data_table R_utils bit64 ggplot2 magrittr stringr ]; }"
3+
4+
set -e
5+
6+
for d in stracciatella txrefs linear notxs
7+
do
8+
mkdir -p results/$d
9+
for f in lifecycle resources receipts cpus
10+
do
11+
DIR=$(find $d -type f -name $f.csv.gz \( -not -empty \) -printf %h\\n -quit)
12+
HL=$(sed -n -e '1p' "$DIR/case.csv")
13+
HR=$(zcat "$DIR/$f.csv.gz" | sed -n -e '1p')
14+
if [[ "$f" == "lifecycle" || "$f" == "resources" ]]
15+
then
16+
FRACT=1
17+
else
18+
FRACT=1
19+
fi
20+
(
21+
echo "$HL,$HR"
22+
for g in $(find $d -type f -name $f.csv.gz \( -not -empty \) -printf %h\\n)
23+
do
24+
if [ ! -e "$g/stderr" ]
25+
then
26+
echo "Skipping $g because it has no stderr." >> /dev/stderr
27+
elif [ -s "$g/stderr" ]
28+
then
29+
echo "Skipping $g because its stderr is not empty." >> /dev/stderr
30+
else
31+
BL=$(sed -n -e '2p' "$g/case.csv")
32+
zcat "$g/$f.csv.gz" | gawk 'FNR > 1 && rand() <= '"$FRACT"' { print "'"$BL"'" "," $0}'
33+
fi
34+
done
35+
) | pigz -p 3 -9c > results/$d/$f.csv.gz
36+
####R --vanilla << EOI > /dev/null
37+
####require(data.table)
38+
####sampleSize <- $FRACT
39+
####print(sampleSize)
40+
####$f <- fread("results/$d/$f.csv.gz", stringsAsFactors=TRUE)
41+
####save($f, sampleSize, file="results/$d/$f.Rdata", compression_level=9)
42+
####EOI
43+
done
44+
done

‎analysis/sims/2025w29/linear.ipynb‎

Lines changed: 2789 additions & 0 deletions
Large diffs are not rendered by default.

‎analysis/sims/2025w29/linear.list‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
linear/5,500000/run.sh
2+
linear/5,1000000/run.sh
3+
linear/5,2000000/run.sh
4+
linear/5,5000000/run.sh
5+
linear/8,500000/run.sh
6+
linear/8,1000000/run.sh
7+
linear/8,2000000/run.sh
8+
linear/8,5000000/run.sh

‎analysis/sims/2025w29/linear/5,1000000/case.csv‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
Simulator,Stage length,Max EB size
2+
Rust,5 slot/stage,1.0 MB/EB

‎analysis/sims/2025w29/linear/5,1000000/config.yaml‎

Lines changed: 226 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,226 @@
1+
# yaml-language-server: $schema=./config.schema.json
2+
3+
################################################################################
4+
# Simulation Configuration File
5+
################################################################################
6+
#
7+
# This file contains the default configuration for running Leios simulations in
8+
# the Haskell simulation (`simulation/`) and the Rust simulation (`sim-rs/`).
9+
#
10+
################################################################################
11+
# Simulation Configuration
12+
################################################################################
13+
14+
relay-strategy: "request-from-first"
15+
tcp-congestion-control: true
16+
multiplex-mini-protocols: true
17+
simulate-transactions: true
18+
treat-blocks-as-full: false
19+
cleanup-policies: ["cleanup-expired-vote"]
20+
timestamp-resolution-ms: 0.1
21+
22+
################################################################################
23+
# Leios Protocol Configuration
24+
################################################################################
25+
26+
leios-variant: linear
27+
linear-vote-stage-length-slots: 5
28+
linear-diffuse-stage-length-slots: 5
29+
leios-stage-length-slots: 5
30+
leios-stage-active-voting-slots: 1
31+
leios-vote-send-recv-stages: false
32+
leios-late-ib-inclusion: true
33+
leios-header-diffusion-time-ms: 1000.0
34+
leios-mempool-sampling-strategy: ordered-by-id
35+
leios-mempool-aggressive-pruning: true
36+
# TODO: revise default
37+
praos-chain-quality: 100
38+
praos-fallback-enabled: true
39+
40+
################################################################################
41+
# Transaction Configuration
42+
################################################################################
43+
44+
tx-generation-distribution:
45+
distribution: exp
46+
lambda: 0.85
47+
scale: 68.9
48+
tx-size-bytes-distribution:
49+
distribution: log-normal
50+
mu: 6.833
51+
sigma: 1.127
52+
tx-conflict-fraction: 0
53+
tx-overcollateralization-factor-distribution:
54+
distribution: constant
55+
value: 0
56+
tx-validation-cpu-time-ms: 0.065
57+
tx-max-size-bytes: 16384
58+
tx-start-time: 60
59+
tx-stop-time: 660
60+
61+
################################################################################
62+
# Ranking Block Configuration
63+
################################################################################
64+
65+
# 1/leios-stage-length-slots, targeting one RB per pipeline.
66+
# Also 20s is current rate of praos blocks.
67+
rb-generation-probability: 5.0e-2
68+
# Eng. team targets 1kB as worst case upper bound.
69+
# Actual size fairly close.
70+
rb-head-size-bytes: 1024
71+
rb-body-max-size-bytes: 90112
72+
# Note: certificate generation/validation is not included in the
73+
# timings here, see cert-* fields.
74+
rb-generation-cpu-time-ms: 1.0
75+
rb-head-validation-cpu-time-ms: 1.0
76+
77+
# On average, no Txs directly embedded in blocks.
78+
rb-body-legacy-praos-payload-avg-size-bytes: 0
79+
rb-body-legacy-praos-payload-validation-cpu-time-ms-constant: 50.0
80+
# the -per-byte component is meant to be using size as a (bad)
81+
# proxy for the complexity of the Txs included.
82+
rb-body-legacy-praos-payload-validation-cpu-time-ms-per-byte: 0.0005
83+
84+
################################################################################
85+
# Input Block Configuration
86+
################################################################################
87+
88+
ib-generation-probability: 2.0
89+
ib-shards: 1
90+
ib-shard-period-length-slots: 1
91+
ib-shard-group-count: 1
92+
93+
# ProducerId 32
94+
# SlotNo 64
95+
# VRF proof 80
96+
# Body hash 32
97+
# RB Ref 32
98+
# Signature 64
99+
# Total 304
100+
#
101+
# NOTE: using a KES Signature (like for Praos headers)
102+
# would instead more than double the total to 668.
103+
# And even 828 including Op Cert.
104+
ib-head-size-bytes: 304
105+
# 98KB to optimize for 3 TCP round trips
106+
ib-body-avg-size-bytes: 98304
107+
ib-body-max-size-bytes: 262144
108+
# Here we also use praos blocks as ballpark estimate.
109+
# Sec 2.3 Forging, of the benchmark cluster report, lists
110+
# * Slot start to announced: 0.12975s
111+
ib-generation-cpu-time-ms: 130.0
112+
ib-head-validation-cpu-time-ms: 1.0
113+
ib-body-validation-cpu-time-ms-constant: 50.0
114+
ib-body-validation-cpu-time-ms-per-byte: 0.0005
115+
ib-diffusion-strategy: "freshest-first"
116+
117+
# Haskell prototype relay mini-protocol parameters.
118+
ib-diffusion-max-bodies-to-request: 1
119+
ib-diffusion-max-headers-to-request: 100
120+
ib-diffusion-max-window-size: 100
121+
122+
################################################################################
123+
# Endorsement Block Configuration
124+
################################################################################
125+
126+
# We want one per pipeline, but not too many.
127+
eb-generation-probability: 1.0
128+
# ProducerId 32
129+
# SlotNo 64
130+
# VRF proof 80
131+
# Signature 64
132+
# Total 240
133+
#
134+
# See Note about signatures on ib-head-size-bytes.
135+
eb-size-bytes-constant: 240
136+
# IB hash
137+
eb-size-bytes-per-ib: 32
138+
# Collecting the IBs to reference and cryptography are the main tasks.
139+
# A comparable task is maybe mempool snapshotting.
140+
# Sec 2.3 Forging, of the benchmark cluster report, lists
141+
# * Mempool snapshotting: 0.07252s
142+
# 75ms then seems a generous estimate for eb generation.
143+
eb-generation-cpu-time-ms: 75.0
144+
# Validating signature and vrf proof, as in other headers.
145+
eb-validation-cpu-time-ms: 1.0
146+
147+
eb-diffusion-strategy: "peer-order"
148+
149+
# Haskell prototype relay mini-protocol parameters.
150+
eb-diffusion-max-bodies-to-request: 1
151+
eb-diffusion-max-headers-to-request: 100
152+
eb-diffusion-max-window-size: 100
153+
154+
# The maximum age of EBs included in RBs.
155+
# A an EB from slot `s` can only be included in RBs
156+
# up to slot `s+eb-max-age-slots`.
157+
# In short leios we expect votes to diffuse within 3 stages lengths of
158+
# EB generation, we allow for 2 more stage lengths to account for
159+
# variance in the interval within RBs.
160+
eb-max-age-slots: 240
161+
162+
# The maximum age of EBs to be relayed.
163+
# An EB from slot `s` will only be relayed
164+
# up to slot `s+eb-max-age-for-relay-slots`.
165+
eb-max-age-for-relay-slots: 40
166+
167+
# The maximum size of transactions (in bytes) which an EB can reference.
168+
# Only relevant when running with the "full-without-ibs" variant.
169+
eb-referenced-txs-max-size-bytes: 1000000
170+
eb-body-avg-size-bytes: 1000000
171+
172+
################################################################################
173+
# Vote Configuration
174+
################################################################################
175+
176+
# Cryptography related values taken from [vote-spec](crypto-benchmarks.rs/Specification.md)
177+
# using weighted averages of 80% persistent and 20% non-persistent.
178+
179+
# vote-spec#Committe and quorum size
180+
#
181+
# Note: this is used as the expected amount of total weight of
182+
# generated votes in the sims.
183+
vote-generation-probability: 500.0
184+
# vote-spec#"Committe and quorum size"
185+
# 60% of `vote-generation-probability`
186+
vote-threshold: 300
187+
# vote-spec#"Generate vote" 0.8*135e-3 + 0.2*280e-3
188+
vote-generation-cpu-time-ms-constant: 164.0e-3
189+
# No benchmark yet.
190+
vote-generation-cpu-time-ms-per-ib: 0
191+
# vote-spec#"Verify vote" 0.8*670e-3 + 0.2*1.4
192+
vote-validation-cpu-time-ms: 816.0e-3
193+
# The `Vote` structure counted in the -per-eb already identifies slot
194+
# (in Eid) and voter. We can assume a vote bundle is all for the same
195+
# voter and slot, so for non-persistent voters we could factor their
196+
# PoolKeyHash (28bytes) here, but that is for 20% of cases.
197+
# More relevant if EB generation is very high.
198+
vote-bundle-size-bytes-constant: 0
199+
# vote-spec#Votes 0.8*90 + 0.2*164
200+
vote-bundle-size-bytes-per-eb: 105
201+
202+
vote-diffusion-strategy: "peer-order"
203+
204+
# Haskell prototype relay mini-protocol parameters.
205+
vote-diffusion-max-bodies-to-request: 1
206+
vote-diffusion-max-headers-to-request: 100
207+
vote-diffusion-max-window-size: 100
208+
209+
################################################################################
210+
# Certificate Configuration
211+
################################################################################
212+
213+
# vote-spec - certificate size plot.
214+
# Realistic stake distributions need about 7 kilobytes for the certificate.
215+
cert-size-bytes-constant: 7168
216+
cert-size-bytes-per-node: 0
217+
218+
# For certificate timings we have bulk figures for realistic scenarios,
219+
# so we do not attempt to give -per-node (i.e. per-voter) timings.
220+
#
221+
# vote-spec#"Generate certificate"
222+
cert-generation-cpu-time-ms-constant: 90.0
223+
cert-generation-cpu-time-ms-per-node: 0
224+
# vote-spec#"Verify certificate"
225+
cert-validation-cpu-time-ms-constant: 130.0
226+
cert-validation-cpu-time-ms-per-node: 0

‎analysis/sims/2025w29/linear/5,1000000/run.sh‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
../5,2000000/run.sh

0 commit comments

Comments
 (0)