gossip: remove vote buffer, send all verified votes to banking_stage

[AshwinSekar]

Problem

Gossip has a buffer that stores the latest vote from each validator, starting up to 3 * 512 slots before our next leader slot, and sends these votes to banking_stage once we have a working_bank.

This buffer is unnecessary, as banking_stage already holds the latest vote per validator in it's own buffer, latest_unprocessed_votes until it is time to produce a block.

Furthermore, 3 * 512 is a somewhat arbitrary number. We can place no lower bound on the block height of the leader's fork, so votes from before 3 * 512 could be included (and not expired) in the next leader block if a validator has been stuck not voting for a while.

The removal of this buffer allows latest_unprocessed_votes to maintain a consistent view of the latest vote per validator for both gossip and tpu votes, and makes reported metrics for vote ingestion more accurate.

Summary of Changes

Remove verified_vote_packets and the buffering in cluster_info_vote_listener. Forward verified votes directly to banking_stage

As explained here #2183 (comment), verified_vote_packets performed an additional check to only send votes that were part of the leader fork to banking_stage. #2465 added this logic to latest_unprocessed_votes so it still applies to these gossip votes, and now to tpu votes as well.

Fixes #2183

[apfitzge]

Code changes look correct for the description you've given.
I added a comment about some behavior change that I'm unsure of but will ultimately defer to consensus team's expertice on whether it's okay to remove that behavior.

[apfitzge]

So I'm a bit curious about the change as it relates to this line.

With this code we used to store up to 1000 votes per validator.
Votes on our fork would get sent to banking stage and the others remain here in case we switch forks.
But now, we bypass this and all just go directly to banking stage.
Since banking stage only stores the most recent vote (to my knowledge) we no longer have this reserve for "in case we switch forks".

[AshwinSekar]

This code was to deal with pre VoteStateUpdate, back when incremental votes were being sent. This business with the IncrementalVotes and FullTowerVote was to deal with the feature activation and gracefully handle a possible rollback if the feature was busted.

Previously when votes where incremental we stored up to 1000 of the last votes, as we wanted to land as many as possible to ensure that the on chain tower matched the local tower. Now we only store one vote. Even in the situation of switching forks, the most recent vote suffices.

Non modded validators do not send incremental votes, so I think it's fine to just store the most recent vote in banking stage. There is an effort I am working on to remove incremental votes from the wire all together, but for now treating them incorrectly is fine.