Skip to content

Commit

Permalink
Exponential Golomb idea moved to an issue
Browse files Browse the repository at this point in the history
It's just confusing here in the schema example. Created
#3 instead.
  • Loading branch information
tv42 committed Sep 26, 2015
1 parent 0e615f2 commit e6a9332
Show file tree
Hide file tree
Showing 2 changed files with 0 additions and 38 deletions.
36 changes: 0 additions & 36 deletions future.org
Original file line number Diff line number Diff line change
Expand Up @@ -26,42 +26,6 @@
* [#B] chitin schema file formatter :tools:
- be like gofmt
- use tabs
* [#C] denser field length encoding as option
- we could encode field lengths at <1byte per field
- they're not sorted, so can't delta-compress, but could bit pack
- fastpfor et al
- https://github.com/surge/encoding
- http://godoc.org/j4k.co/exp/varintrle
- http://godoc.org/github.com/Yawning/ntru/bitpack
- https://github.com/robskie/bit
- https://github.com/robskie/fibvec
and https://en.wikipedia.org/wiki/Fibonacci_coding
- https://github.com/crazy2be/birdie
- https://github.com/Smerity/govarint
** self-delimiting number sequences
- goal: "Field lengths are densely encoded, typically at less than 1
byte per field."

Elias δ-coding needs

math.floor(math.log(x, 2)) + 2 * math.floor(math.log(1 + math.floor(math.log(x, 2)), 2)) + 1

bits to encode x

https://en.wikipedia.org/wiki/Prefix_code
https://en.wikipedia.org/wiki/Variable-length_code#Uniquely_decodable_codes

- rice/golomb looks promising
https://en.wikipedia.org/wiki/Golomb_coding
https://github.com/dave-andersen/deltagolomb

- varuint is hard to beat
- exponential golomb is smaller for fields predominantly <30
https://github.com/dave-andersen/deltagolomb
https://en.wikipedia.org/wiki/Exponential-Golomb_coding

- everything more complex than varuint is 10x slower
- make benchmarks reproducible
* write advocacy: no wire overhead for putting a Message in a sized container
- e.g. in a key-value database
- that is, no field types etc
Expand Down
2 changes: 0 additions & 2 deletions index.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,8 +48,6 @@ message Person v1 {
wire format: v1
options {
align: 4
# switches field lengths from varuint to exponential golomb coding
field length encoding: exp-golomb
}
slots {
Expand Down

0 comments on commit e6a9332

Please sign in to comment.