Leios design: Write introduction and overview chapter

[ch1bo]

Adds a chapter that should set the context, what is important, and provide plenty links to past/related work on the network/consensus design of Cardano.

Tagged @coot @pagio and @nfrisby as possible reviewers - you are authors on some of the linked documents and maybe have an opinion here.

[coot]

Linear Leios doesn't utilise the network any better than Praos. It uses bursts of traffic, as Praos does, but with even higher bursts than Praos does - from that perspective, it's strictly worse than Praos. So we need to be a bit more careful here, what about:

Although Linear Leios does not improve the burstiness of the Praos protocol, it will utilise unused bandwidth during long gaps between Praos blocks. For better utilisation of the underlying TCP protocol, a protocol which uses constant high pressure on the network is required (e.g. some form of full Leios).

[ch1bo]

Linear Leios doesn't utilise the network any better than Praos. It uses bursts of traffic, as Praos does, but with even higher bursts than Praos does - from that perspective, it's strictly worse than Praos

Hm, interesting take that Leios would have higher bursts than Praos. I'd like to challenge this viewpoint: In Leios we communicate five things: Ranking blocks which announce/certify EBs, votes that lead to certificates, transactions submitted by users, EBs, any "missing transactions".

A certifying ranking block (~10kB) is about 90% smaller than a full Praos block (90kB); if not certifying ranking block = praos block.

Voting on EBs could be considered bursts of network traffic (in the order of ~50kB per round), as they are created at the same time, however votes are going to be created distributed across the network and this should even it out further than the single source full block one would expect in Praos?

Which leaves us with transaction submission and diffusion of EBs + missing transactions: how much of the transaction submission is impacting network traffic depends on our understanding of the load scenario - whether traffic would be organic, artificial, coming from one or many sources, etc. - it is what it is; now, under any given high demand (otherwise we'd be using only praos), Leios will only result in the overhead of EBs (~40 bytes per tx) and re-submission of a subset of endorsed transactions (the ones which were not diffused originally during submission). This means, in the average case, this is minimal, and in the worst case it boils down to the same burstiness as Praos would have at this load!?

For better utilisation of the underlying TCP protocol, a protocol which uses constant high pressure on the network is required (e.g. some form of full Leios

Isn't this the case already due to transaction submission, no matter how consensus is achieved?

[coot]

In the happy case, sure, but we should really be focused on improving the worst-case scenario, which, with linear Leios, can lead to 12MB of fresh data being downloaded over a short period, with the TCP window closed due to an idle period.

[ch1bo]

I agree with designing for the worst case (that's why I wrote that section in this PR!). The worst case for Leios is not the same as the worst case for Praos though - is it? What is the worst case scenario for EB diffusion?

I see two situations where urgent fetching of big transaction closures is required:

1. A block is (maliciously) produced that announces a full EB with completely unknown transactions
2. A block is produced that certifies a full EB with completely unknown transactions

Case 1. is only possibly affecting high throughput and would only be problematic if a) most stake is acting like this and/or b) this can be caused by a network attacker (e.g. by eclipsing / partition the majority of the network).

Only 2. is on the critical path in Leios and needs to happen within worst case $\Delta_EB$ (to not affect safety). However, in this situation, it is guaranteed that $\tau - \alpha_A$ honest stake has the data available. Now, the one million dollar question is: would we only have a single upstream peer with a closed TCP window to reach that honest stake?

If not, that worst case scenario is strictly less bad as you paint it. Am I missing something / is my thinking flawed?

[ch1bo]

Meanwhile I have refined the wording in this paragraph: d9c975d. Is this better?

[coot]

It can also happen when the internet is partitioned (e.g. a major routing error). This is a genuine scenario which we've encountered at least once before.

would we only have a single upstream peer with a closed TCP window to reach that honest stake?

After a network partition, all connections to the other side will be new, so this will be true for all peers on different sides of the network partition.

In the partition we encountered, Praos recovered pretty well; Linear Leios should recover similarly if we prioritise Praos over Leios well enough.

Btw, my point wasn't that we're not safe, just that we're not utilising the network fundamentally better. I see this as an incentive, actually, to continue towards full Leios.

[coot]

I'd like to know what has prevented the simulation from scaling to a large network. Was it io-sim performance, or something else?

[ch1bo]

@bwbush @nfrisby Do you recall what made the Haskell sim slow?

[coot]

LGTM

[coot]

This paragraph is quite heavy. I think we should add at the end that further R&D is required for solving the challenges of full Leios, so we keep the door open for the future.