Use default WriteBuffer size

Using a write buffer of 64M was causing flow-indexer to use 7G of ram while indexing. Changing this back to the default of 4M cuts the peak usage in half. I'm still not sure how a 64M buffer ends up turning into 7G of usage, but I think it has something to do with GC and leveldb allocating a 64M buffer for each transaction. This is a partial fix for #13
JustinAzoff · Apr 11, 2018 · 82e7915 · 82e7915
1 parent 52b1dce
commit 82e7915
Show file tree

Hide file tree

Showing 2 changed files with 1 addition and 2 deletions.
diff --git a/cmd/version.go b/cmd/version.go
@@ -7,7 +7,7 @@ import (
 )
 
 const (
-	VERSION = "0.1.8"
+	VERSION = "0.1.9"
 )
 
 var cmdVersion = &cobra.Command{

diff --git a/store/leveldb.go b/store/leveldb.go
@@ -27,7 +27,6 @@ func NewLevelDBStore(filename string) (IpStore, error) {
 	//Options taken from ledisdb
 	opts := &opt.Options{}
 	opts.BlockSize = 32768
-	opts.WriteBuffer = 67108864
 	opts.BlockCacheCapacity = 524288000
 	opts.OpenFilesCacheCapacity = 1024
 	opts.CompactionTableSize = 32 * 1024 * 1024
-Original file line number
+Diff line change
@@ Expand Up / @@ -7,7 +7,7 @@ import ( @@
     )
     const (
-    	VERSION = "0.1.8"
+    	VERSION = "0.1.9"
     )
     var cmdVersion = &cobra.Command{
@@ Expand Down @@