|
2 | 2 | [](https://docs.rs/bitcode) |
3 | 3 | [](https://crates.io/crates/bitcode) |
4 | 4 | [](https://github.com/SoftbearStudios/bitcode/actions/workflows/build.yml) |
5 | | -[](https://github.com/rust-secure-code/safety-dance/) |
6 | 5 |
|
7 | | -A bitwise encoder/decoder similar to [bincode](https://github.com/bincode-org/bincode), which attempts to shrink the serialized size without sacrificing speed (as would be the case with compression). |
8 | | - |
9 | | -The format may change between major versions, so we are free to optimize it. |
10 | | - |
11 | | -## Comparison with [bincode](https://github.com/bincode-org/bincode) |
12 | | - |
13 | | -### Features |
14 | | - |
15 | | -- Bitwise serialization |
16 | | -- [Gamma](https://en.wikipedia.org/wiki/Elias_gamma_coding) encoded lengths and enum variant indices |
17 | | - |
18 | | -### Additional features with `#[derive(bitcode::Encode, bitcode::Decode)]` |
19 | | - |
20 | | -- Enums use the fewest possible bits, e.g. an enum with 4 variants uses 2 bits |
21 | | -- Apply attributes to fields/enum variants: |
22 | | - |
23 | | -| Attribute | Type | Result | |
24 | | -|-----------------------------------------------|---------------|------------------------------------------------------------------------------------------------------------| |
25 | | -| `#[bitcode_hint(ascii)]` | String | Uses 7 bits per character | |
26 | | -| `#[bitcode_hint(ascii_lowercase)]` | String | Uses 5 bits per character | |
27 | | -| `#[bitcode_hint(expected_range = "50..100"]` | u8-u64 | Uses log2(range.end - range.start) bits | |
28 | | -| `#[bitcode_hint(expected_range = "0.0..1.0"]` | f32/f64 | Uses ~25 bits for `f32` and ~54 bits for `f64` | |
29 | | -| `#[bitcode_hint(frequency = 123)` | enum variant | Frequent variants use fewer bits (see [Huffman coding](https://en.wikipedia.org/wiki/Huffman_coding)) | |
30 | | -| `#[bitcode_hint(gamma)]` | i8-i64/u8-u64 | Small integers use fewer bits (see [Elias gamma coding](https://en.wikipedia.org/wiki/Elias_gamma_coding)) | |
31 | | -| `#[bitcode(with_serde)]` | T: Serialize | Uses `serde::Serialize` instead of `bitcode::Encode` | |
32 | | - |
33 | | -### Limitations |
34 | | - |
35 | | -- Doesn't support streaming APIs |
36 | | -- Format may change between major versions |
37 | | -- With `feature = "derive"`, types containing themselves must use `#[bitcode(recursive)]` to compile |
38 | | - |
39 | | -## Benchmarks vs. [bincode](https://github.com/bincode-org/bincode) and [postcard](https://github.com/jamesmunns/postcard) |
40 | | - |
41 | | -### Primitives (size in bits) |
42 | | - |
43 | | -| Type | Bitcode (derive) | Bitcode (serde) | Bincode | Bincode (varint) | Postcard | |
44 | | -|---------------------|------------------|-----------------|---------|------------------|----------| |
45 | | -| bool | 1 | 1 | 8 | 8 | 8 | |
46 | | -| u8/i8 | 8 | 8 | 8 | 8 | 8 | |
47 | | -| u16/i16 | 16 | 16 | 16 | 8-24 | 8-24 | |
48 | | -| u32/i32 | 32 | 32 | 32 | 8-40 | 8-40 | |
49 | | -| u64/i64 | 64 | 64 | 64 | 8-72 | 8-80 | |
50 | | -| u128/i128 | 128 | 128 | 128 | 8-136 | 8-152 | |
51 | | -| usize/isize | 64 | 64 | 64 | 8-72 | 8-80 | |
52 | | -| f32 | 32 | 32 | 32 | 32 | 32 | |
53 | | -| f64 | 64 | 64 | 64 | 64 | 64 | |
54 | | -| char | 21 | 21 | 8-32 | 8-32 | 16-40 | |
55 | | -| Option<()> | 1 | 1 | 8 | 8 | 8 | |
56 | | -| Result<(), ()> | 1 | 1-3 | 32 | 8 | 8 | |
57 | | -| enum { A, B, C, D } | 2 | 1-5 | 32 | 8 | 8 | |
58 | | -| Duration | 94 | 96 | 96 | 16-112 | 16-120 | |
59 | | - |
60 | | -<sup>Note: These are defaults, and can be optimized with hints in the case of Bitcode (derive) or custom `impl Serialize` in the case of `serde` serializers.</sup> |
61 | | - |
62 | | -### Values (size in bits) |
63 | | - |
64 | | -| Value | Bitcode (derive) | Bitcode (serde) | Bincode | Bincode (varint) | Postcard | |
65 | | -|---------------------|------------------|-----------------|---------|------------------|----------| |
66 | | -| [true; 4] | 4 | 4 | 32 | 32 | 32 | |
67 | | -| vec![(); 0] | 1 | 1 | 64 | 8 | 8 | |
68 | | -| vec![(); 1] | 3 | 3 | 64 | 8 | 8 | |
69 | | -| vec![(); 256] | 17 | 17 | 64 | 24 | 16 | |
70 | | -| vec![(); 65536] | 33 | 33 | 64 | 40 | 24 | |
71 | | -| "" | 1 | 1 | 64 | 8 | 8 | |
72 | | -| "abcd" | 37 | 37 | 96 | 40 | 40 | |
73 | | -| "abcd1234" | 71 | 71 | 128 | 72 | 72 | |
74 | | - |
75 | | - |
76 | | -### Random [Structs and Enums](https://github.com/SoftbearStudios/bitcode/blob/2a47235eee64f4a7c49ad1841a5b509abd2d0e99/src/benches.rs#L16-L88) (average size and speed) |
77 | | - |
78 | | -| Format | Size (bytes) | Serialize (ns) | Deserialize (ns) | |
79 | | -|------------------------|--------------|----------------|------------------| |
80 | | -| Bitcode (derive) | 6.2 | 14 | 50 | |
81 | | -| Bitcode (serde) | 6.7 | 18 | 59 | |
82 | | -| Bincode | 20.3 | 17 | 61 | |
83 | | -| Bincode (varint) | 10.9 | 26 | 68 | |
84 | | -| Bincode (LZ4) | 9.9 | 58 | 73 | |
85 | | -| Bincode (Deflate Fast) | 8.4 | 336 | 279 | |
86 | | -| Bincode (Deflate Best) | 7.8 | 1990 | 275 | |
87 | | -| Postcard | 10.7 | 21 | 57 | |
88 | | - |
89 | | -### More benchmarks |
90 | | - |
91 | | -[rust_serialization_benchmark](https://david.kolo.ski/rust_serialization_benchmark/) |
92 | | - |
93 | | -## Acknowledgement |
94 | | - |
95 | | -Some test cases were derived from [bincode](https://github.com/bincode-org/bincode) (see comment in `tests.rs`). |
| 6 | +A binary encoder/decoder with the following goals: |
| 7 | +- 🔥 Blazingly fast |
| 8 | +- 🐁 Tiny serialized size |
| 9 | +- 💎 Highly compressible by Deflate/LZ4/Zstd |
| 10 | + |
| 11 | +In contrast, these are non-goals: |
| 12 | +- Stable format across major versions |
| 13 | +- Self describing format |
| 14 | +- Compatibility with languages other than Rust |
| 15 | + |
| 16 | +See [rust_serialization_benchmark](https://github.com/djkoloski/rust_serialization_benchmark) for benchmarks. |
| 17 | + |
| 18 | +## Example |
| 19 | +```rust |
| 20 | +use bitcode::{Encode, Decode}; |
| 21 | + |
| 22 | +#[derive(Encode, Decode, PartialEq, Debug)] |
| 23 | +struct Foo<'a> { |
| 24 | + x: u32, |
| 25 | + y: &'a str, |
| 26 | +} |
| 27 | + |
| 28 | +let original = Foo { |
| 29 | + x: 10, |
| 30 | + y: "abc", |
| 31 | +}; |
| 32 | + |
| 33 | +let encoded: Vec<u8> = bitcode::encode(&original); // No error |
| 34 | +let decoded: Foo<'_> = bitcode::decode(&encoded).unwrap(); |
| 35 | +assert_eq!(original, decoded); |
| 36 | +``` |
| 37 | + |
| 38 | +## Library Example |
| 39 | + |
| 40 | +Add bitcode to libraries without specifying the major version so binary crates can pick the version. |
| 41 | +This is a minimal stable subset of the bitcode API so avoid using any other functionality. |
| 42 | +```toml |
| 43 | +bitcode = { version = "0", features = ["derive"], default-features = false, optional = true } |
| 44 | +``` |
| 45 | +```rust |
| 46 | +#[cfg_attr(feature = "bitcode", derive(bitcode::Encode, bitcode::Decode))] |
| 47 | +pub struct Vec2 { |
| 48 | + x: f32, |
| 49 | + y: f32, |
| 50 | +} |
| 51 | +``` |
| 52 | + |
| 53 | +## Tuple vs Array |
| 54 | +If you have multiple values of the same type: |
| 55 | +- Use a tuple or struct when the values are semantically different: `x: u32, y: u32` |
| 56 | +- Use an array when all values are semantically similar: `pixels: [u8; 16]` |
| 57 | + |
| 58 | +## Implementation Details |
| 59 | +- Heavily inspired by <https://github.com/That3Percent/tree-buf> |
| 60 | +- All instances of each field are grouped together making compression easier |
| 61 | +- Uses smaller integers where possible all the way down to 1 bit |
| 62 | +- Validation is performed up front on typed vectors before deserialization |
| 63 | +- Code is designed to be auto-vectorized by LLVM |
96 | 64 |
|
97 | 65 | ## License |
98 | | - |
99 | 66 | Licensed under either of |
100 | | - |
101 | | - * Apache License, Version 2.0 |
102 | | - ([LICENSE-APACHE](LICENSE-APACHE) or http://www.apache.org/licenses/LICENSE-2.0) |
103 | | - * MIT license |
104 | | - ([LICENSE-MIT](LICENSE-MIT) or http://opensource.org/licenses/MIT) |
| 67 | +* Apache License, Version 2.0 |
| 68 | + ([LICENSE-APACHE](LICENSE-APACHE) or <http://www.apache.org/licenses/LICENSE-2.0>) |
| 69 | +* MIT license |
| 70 | + ([LICENSE-MIT](LICENSE-MIT) or <http://opensource.org/licenses/MIT>) |
105 | 71 |
|
106 | 72 | at your option. |
107 | 73 |
|
108 | 74 | ## Contribution |
109 | | - |
110 | 75 | Unless you explicitly state otherwise, any contribution intentionally submitted |
111 | 76 | for inclusion in the work by you, as defined in the Apache-2.0 license, shall be |
112 | | -dual licensed as above, without any additional terms or conditions. |
| 77 | +dual licensed as above, without any additional terms or conditions. |
0 commit comments