Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Output sum column of the epoch table can overflow #197

Closed
erikd opened this issue Jul 22, 2020 · 1 comment
Closed

Output sum column of the epoch table can overflow #197

erikd opened this issue Jul 22, 2020 · 1 comment

Comments

@erikd
Copy link
Contributor

erikd commented Jul 22, 2020

On some testnets (i was aware of the benchmarking team hitting this over 3 months ago) it is possible for the sum of the tx outputs for an epoch to overflow the Word64 values. This is related to #163

The best solution for this would be to use Word128 for the output sum field. I do have a WIP branch with this idea, but it needs some work.
@mmahut mmahut mentioned this issue Jul 23, 2020
Closed
@erikd
Copy link
Contributor Author

erikd commented Jul 23, 2020 •
edited
Loading

The output sum field is a Word64 which only allows positive and zero values. However as related in #163 the Persistent library treats Word64 like Int64 which allows positive and negative values.

When the sum of transactions outputs is greater than 102 times max lovelace in the system (basically maxBound :: Int64) then the output sum that is to be written to the database becomes negative and that violates a domain constraint that the sum should be >= 0.

There are two possible solutions to this:

  • Fix Word64 in the upstream Persistent library. I have already started on this but there is a lot of work on this and requires significant coordination with the upstream Persistent authors. The downside of this is that it is still possible to overflow the maxBound :: Word64 .
  • Switch to from Word64 to Word128. This would actually make it close to impossible to overflow and there is simply not enough block space in an epoch to contain enough transactions to overflow this value. I made two quick attempts to do this and I was not happy with either of them. They could be revisited.

erikd added a commit that referenced this issue Jul 23, 2020
@erikd
db: Fix for the outsum column of the epoch table
221c28c 
This column is the sum of transactions outputs for the epoch. On
test networks (never been seen on mainnet) it is possible for this
sum to end up larger than the maximum positive value that can be
stored in a 64 bit signed integer ('Int64'). However, the output sum
was being stored store in a 64 bit unsigned integer ('Word64') but
the Persistent library does not correctly supported 'Word64'. Instead
it "casts" it from 'Word64' to 'Int64'. For values great that the
max positive value of an 'Int64' this results on the value wrapping
to negative and hence being rejected by the SQL domain constraint
which requres 'outsum >= 0'.

This solution switches 'outsum' in the Haskell world from 'Word64'
to 'Word128' which is incredibly unlikley to overflow given the
limitation on number of transactions per block and per epoch. On
the SQL side we use an PostgreSQL type of 'numeric (38, 0)'.

Closes: #197
@erikd erikd closed this as completed in db4c760 Jul 24, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@erikd