Update READMEs and get a Makefile

README.md

@@ -168,7 +168,8 @@ instance, there are two options:
    the last write respected by the replica and the first write respected by the
    new instance. This gap might be fixed by subsequent read-repairs.
 
-These properties and procedures warrant careful consideration.
+Both processes can be expedited via a [keyspace walker process][roshi-walker].
+Nevertheless, these properties and procedures warrant careful consideration.
 
 ## Considerations
 
@@ -254,13 +255,18 @@ job with a relatively small surface area. From the bottom up...
   REST-ish HTTP interface. It's effectively stateless, and [12-factor][twelve]
   compliant.
 
+- **[roshi-walker][roshi-walker]** walks the keyspace in semirandom order at a
+  defined rate, making Select requests for each key in order to trigger read
+  repairs.
+
 [sorted-set]: http://redis.io/commands#sorted_set
 [shard]: http://github.com/soundcloud/roshi/tree/master/shard
 [cluster]: http://github.com/soundcloud/roshi/tree/master/cluster
 [commutativity]: http://en.wikipedia.org/wiki/Commutative_property
 [farm]: http://github.com/soundcloud/roshi/tree/master/farm
 [roshi-server]: http://github.com/soundcloud/roshi/tree/master/roshi-server
 [twelve]: http://12factor.net
+[roshi-walker]: http://github.com/soundcloud/roshi/tree/master/roshi-walker
 
 ## The big picture
 

roshi-server/Makefile

@@ -10,4 +10,3 @@ clean:
 
 check:
 	@go list -f '{{join .Deps "\n"}}' | xargs go list -f '{{if not .Standard}}{{.ImportPath}} {{.Dir}}{{end}}' | column -t
-

walker/Makefile

@@ -0,0 +1,12 @@
+GOPATH := $(CURDIR)/../vendor:$(GOPATH)
+
+all: build
+
+build:
+	go build
+
+clean:
+	go clean
+
+check:
+	@go list -f '{{join .Deps "\n"}}' | xargs go list -f '{{if not .Standard}}{{.ImportPath}} {{.Dir}}{{end}}' | column -t

walker/README.md

@@ -0,0 +1,49 @@
+# roshi-walker
+
+roshi-walker walks the keyspace of a Roshi farm in semirandom order (random
+order of Redis instances; Redis [SCAN][scan] command on each instance) and a
+user-defined rate. It makes Select request for each key, using the
+[SendAllReadAll read strategy][send-all-read-all] in order to perform complete
+read repair.
+
+[scan]: http://redis.io/commands/scan
+[send-all-read-all]: https://github.com/soundcloud/roshi/tree/master/farm#read-strategies
+
+## Getting and building
+
+roshi-walker uses vendored dependencies and a "blessed build" process to ensure
+stability over time. Users should get and build roshi-walker by cloning this
+repository and running `make` in the roshi-walker subdirectory. A working Go
+toolchain is assumed.
+
+    git clone [email protected]:soundcloud/roshi
+    cd roshi/roshi-walker
+    make
+
+It's also possible to get roshi-walker via `go get`, and/or build it with a
+simple `go build`, with the caveat that it will use your normal GOPATH to
+resolve dependencies, and therefore will enforce no constraints on dependency
+versions, which could introduce bugs or strange behavior.
+
+## Usage
+
+roshi-walker is designed to be used in two situations.
+
+### Walk forever
+
+One or more roshi-walker instances can be started for a given Roshi farm, set
+to walk at a moderate rate and run forever. That will ensure that all keys are
+made consistent, within some upper time bound. That in turn improves data
+durability, especially if normal reads are non-uniform or low volume.
+
+It's recommended to run a few roshi-walker processes in this mode on every
+Roshi farm. Test against your infrastructure to determine an appropriate rate.
+
+### Walk once
+
+roshi-walker supports a **-once** flag, which will walk the entire keyspace
+once and exit. This is useful if a Redis instance has crashed and come back
+online with no data. In this situation, while the farm still returns correct
+data, it is less resilient to further node failure. After the walk is
+complete, the empty instance will be repopulated with relevant data via read
+repair, and the resiliency of the farm is returned to normal levels.

walker/main.go

@@ -80,11 +80,11 @@ func main() {
 	// Set up our rate limiter. Remember: it's per-key, not per-request.
 	throttle := newThrottle(*maxKeysPerSecond)
 
-	// Perform the walk
+	// Perform the walk.
 	dst := farm.New(clusters, len(clusters), farm.SendAllReadAll, farm.AllRepairs, instr)
 	for {
 		src := scan(clusters) // new key set
-		walk(dst, throttle, src, *batchSize, *maxSize, instr)
+		walkOnce(dst, throttle, src, *batchSize, *maxSize, instr)
 		if *once {
 			return
 		}
@@ -137,7 +137,13 @@ func scan(clusters []cluster.Cluster) <-chan string {
 	return c
 }
 
-func walk(dst farm.Selecter, throttle *throttle, src <-chan string, batchSize, maxSize int, instr instrumentation.WalkInstrumentation) {
+func walkOnce(
+	dst farm.Selecter,
+	throttle *throttle,
+	src <-chan string,
+	batchSize, maxSize int,
+	instr instrumentation.WalkInstrumentation,
+) {
 	batch := []string{}
 	waitSelectReset := func() {
 		throttle.wait(int64(len(batch)))