accept :async and :logger options, add a TraceWrapper utility (#155)

* accept :async and :logger options, add a TraceWrapper utility

* fix the README

* check if sampled and yield the block with the custom span

* fix correct mispelling

* add support for the :async option to the HTTP sender

* call #dup on the spans only when running in async mode

CHANGELOG.md

@@ -1,3 +1,8 @@
+# 0.37.0
+* Add an `async` option to the HTTP and SQS senders.
+* Allow the `logger` option to be provided.
+* Add the TraceWrapper utility class.
+
 # 0.36.2
 * Cleanup the gemspec. No code changes.
 

README.md

@@ -8,27 +8,34 @@ Rack and Faraday integration middlewares for Zipkin tracing.
 
 ### Sending traces on incoming requests
 
-Options can be provided via Rails.config for a Rails 3+ app, or can be passed as a hash argument to the Rack plugin.
+Options can be provided as a hash via `Rails.config.zipkin_tracer` for Rails apps or directly to the Rack middleware:
 
 ```ruby
 require 'zipkin-tracer'
-use ZipkinTracer::RackHandler, config # config is optional
+use ZipkinTracer::RackHandler, config
 ```
 
-where `Rails.config.zipkin_tracer` or `config` is a hash that can contain the following keys:
+### Configuration options
 
+#### Common
 * `:service_name` **REQUIRED** - the name of the service being traced. There are two ways to configure this value. Either write the service name in the config file or set the "DOMAIN" environment variable (e.g. 'test-service.example.com' or 'test-service'). The environment variable takes precedence over the config file value.
 * `:sample_rate` (default: 0.1) - the ratio of requests to sample, from 0 to 1
-* `:json_api_host` - hostname with protocol of a zipkin api instance (e.g. `https://zipkin.example.com`) to use the HTTP sender
-* `:zookeeper` - the address of the zookeeper server to use by the Kafka sender
-* `:sqs_queue_name` - the name of the Amazon SQS queue to use the SQS sender
-* `:sqs_region` - the AWS region for the Amazon SQS queue
-* `:log_tracing` - Set to true to log all traces. Only used if traces are not sent to the API or Kafka.
-* `:annotate_plugin` - plugin function which receives the Rack env, the response status, headers, and body to record annotations
-* `:filter_plugin` - plugin function which receives the Rack env and will skip tracing if it returns false
-* `:whitelist_plugin` - plugin function which receives the Rack env and will force sampling if it returns true
 * `:sampled_as_boolean` - When set to true (default but deprecrated), it uses true/false for the `X-B3-Sampled` header. When set to false uses 1/0 which is preferred.
 * `:trace_id_128bit` - When set to true, high 8-bytes will be prepended to trace_id. The upper 4-bytes are epoch seconds and the lower 4-bytes are random. This makes it convertible to Amazon X-Ray trace ID format v1. (See http://docs.aws.amazon.com/elasticloadbalancing/latest/application/load-balancer-request-tracing.html)
+* `:async` - By default senders will flush traces asynchronously. Set to `false` to make that process synchronous. Only supported by the HTTP and SQS senders.
+* `:logger` - The default logger for Rails apps is `Rails.logger`, else it is `STDOUT`. Use this option to pass a custom logger.
+
+#### Sender specific
+* `:json_api_host` - Hostname with protocol of a zipkin api instance (e.g. `https://zipkin.example.com`) to use the HTTP sender
+* `:zookeeper` - The address of the zookeeper server to use by the Kafka sender
+* `:sqs_queue_name` - The name of the Amazon SQS queue to use the SQS sender
+* `:sqs_region` - The AWS region for the Amazon SQS queue (optional)
+* `:log_tracing` - Set to true to log all traces. Only used if traces are not sent to the API or Kafka.
+
+#### Plugins
+* `:annotate_plugin` - Receives the Rack env, the response status, headers, and body to record annotations
+* `:filter_plugin` - Receives the Rack env and will skip tracing if it returns false
+* `:whitelist_plugin` - Receives the Rack env and will force sampling if it returns true
 
 ### Sending traces on outgoing requests with Faraday
 
@@ -210,6 +217,29 @@ For example:
 lambda { |env| KNOWN_DEVICES.include?(env['HTTP_X_DEVICE_ID']) }
 ```
 
+## Utility classes
+
+### TraceWrapper
+
+This class provides a `.wrap_in_custom_span` method which expects a configuration hash, a span name and a block.
+You may also pass a span kind and an Application object using respectively `span_kind:` and `app:` keyword arguments.
+
+The block you pass will be executed in the context of a custom span.
+This is useful when your application doesn't use the rack handler but still needs to generate complete traces, for instance background jobs or lambdas calling remote services.
+
+The following code will create a trace starting with a span of the (default) `SERVER` kind named "custom span" and then a span of the `CLIENT` kind will be added by the Faraday middleware. Afterwards the configured sender will call `flush!`.
+
+```ruby
+TraceWrapper.wrap_in_custom_span(config, "custom span") do |span|
+  conn = Faraday.new(url: remote_service_url) do |builder|
+    builder.use ZipkinTracer::FaradayHandler, config[:service_name]
+    builder.adapter Faraday.default_adapter
+  end
+  conn.get("/")
+end
+```
+
+
 ## Development
 
 This project uses Rspec. Make sure your PRs contain proper tests.

bin/console

@@ -0,0 +1,14 @@
+#!/usr/bin/env ruby
+
+require "bundler/setup"
+require "zipkin-tracer"
+
+# You can add fixtures and/or initialization code here to make experimenting
+# with your gem easier. You can also use a different console, if you like.
+
+# (If you use this, don't forget to add pry to your Gemfile!)
+# require "pry"
+# Pry.start
+
+require "irb"
+IRB.start(__FILE__)

bin/setup

@@ -0,0 +1,8 @@
+#!/usr/bin/env bash
+set -euo pipefail
+IFS=$'\n\t'
+set -vx
+
+bundle install
+
+# Do any other automated setup that you need to do here

lib/zipkin-tracer.rb

@@ -4,11 +4,12 @@
 require 'zipkin-tracer/trace_client'
 require 'zipkin-tracer/trace_container'
 require 'zipkin-tracer/trace_generator'
+require 'zipkin-tracer/trace_wrapper'
 
 begin
   require 'faraday'
   require 'zipkin-tracer/faraday/zipkin-tracer'
-rescue LoadError #Faraday is not available, we do not load our code.
+rescue LoadError # Faraday is not available, we do not load our code.
 end
 
 begin

lib/zipkin-tracer/config.rb

@@ -5,10 +5,9 @@
 module ZipkinTracer
   # Configuration of this gem. It reads the configuration and provides default values
   class Config
-    attr_reader :service_name, :sample_rate,
+    attr_reader :service_name, :sample_rate, :sampled_as_boolean, :trace_id_128bit, :async, :logger,
       :json_api_host, :zookeeper, :kafka_producer, :kafka_topic, :sqs_queue_name, :sqs_region, :log_tracing,
-      :annotate_plugin, :filter_plugin, :whitelist_plugin,
-      :logger, :sampled_as_boolean, :trace_id_128bit
+      :annotate_plugin, :filter_plugin, :whitelist_plugin
 
     def initialize(app, config_hash)
       config = config_hash || Application.config(app)
@@ -25,14 +24,16 @@ def initialize(app, config_hash)
       @sqs_queue_name    = config[:sqs_queue_name]
       @sqs_region        = config[:sqs_region]
       # Percentage of traces which by default this service traces (as float, 1.0 means 100%)
-      @sample_rate       = config[:sample_rate]       || DEFAULTS[:sample_rate]
+      @sample_rate       = config[:sample_rate] || DEFAULTS[:sample_rate]
       # A block of code which can be called to do extra annotations of traces
       @annotate_plugin   = config[:annotate_plugin]   # call for trace annotation
       # A block of code which can be called to skip traces. Skip tracing if returns false
       @filter_plugin     = config[:filter_plugin]
       # A block of code which can be called to force sampling. Forces sampling if returns true
       @whitelist_plugin  = config[:whitelist_plugin]
-      @logger            = Application.logger
+      # be strict about checking `false` to ensure misconfigurations don't lead to accidental synchronous configurations
+      @async             = config[:async] != false
+      @logger            = config[:logger] || Application.logger
       # Was the logger in fact setup by the client?
       @log_tracing       = config[:log_tracing]
       # When set to false, it uses 1/0 in the 'X-B3-Sampled' header, else uses true/false

lib/zipkin-tracer/trace_wrapper.rb

@@ -0,0 +1,22 @@
+module ZipkinTracer
+  class TraceWrapper
+    def self.wrap_in_custom_span(config, span_name, span_kind: Trace::Span::Kind::SERVER, app: nil)
+      raise ArgumentError, "you must provide a block" unless block_given?
+
+      zipkin_config = ZipkinTracer::Config.new(app, config).freeze
+      tracer = ZipkinTracer::TracerFactory.new.tracer(zipkin_config)
+      trace_id = ZipkinTracer::TraceGenerator.new.next_trace_id
+
+      ZipkinTracer::TraceContainer.with_trace_id(trace_id) do
+        if trace_id.sampled?
+          tracer.with_new_span(trace_id, span_name) do |span|
+            span.kind = span_kind
+            yield(span)
+          end
+        else
+          yield(ZipkinTracer::NullSpan.new)
+        end
+      end
+    end
+  end
+end

lib/zipkin-tracer/tracer_factory.rb

@@ -18,7 +18,12 @@ def tracer(config)
           Trace::ZipkinKafkaSender.new(options)
         when :sqs
           require 'zipkin-tracer/zipkin_sqs_sender'
-          options = { logger: config.logger, queue_name: config.sqs_queue_name , region: config.sqs_region }
+          options = {
+            async: config.async,
+            logger: config.logger,
+            queue_name: config.sqs_queue_name,
+            region: config.sqs_region
+          }
           Trace::ZipkinSqsSender.new(options)
         when :logger
           require 'zipkin-tracer/zipkin_logger_sender'

lib/zipkin-tracer/version.rb

@@ -1,3 +1,3 @@
 module ZipkinTracer
-  VERSION = '0.36.2'.freeze
+  VERSION = '0.37.0'.freeze
 end

lib/zipkin-tracer/zipkin_http_sender.rb

@@ -4,7 +4,7 @@
 require 'zipkin-tracer/hostname_resolver'
 
 module Trace
-  class AsyncHttpApiClient
+  class HttpApiClient
     include SuckerPunch::Job
     SPANS_PATH = '/api/v2/spans'
 
@@ -32,13 +32,18 @@ class ZipkinHttpSender < ZipkinSenderBase
     IP_FORMAT = :string
 
     def initialize(options)
-      SuckerPunch.logger = options[:logger]
       @json_api_host = options[:json_api_host]
+      @async = options[:async] != false
+      SuckerPunch.logger = options[:logger]
       super(options)
     end
 
     def flush!
-      AsyncHttpApiClient.perform_async(@json_api_host, spans.dup)
+      if @async
+        HttpApiClient.perform_async(@json_api_host, spans.dup)
+      else
+        HttpApiClient.new.perform(@json_api_host, spans)
+      end
     end
   end
 end

lib/zipkin-tracer/zipkin_sqs_sender.rb

@@ -5,17 +5,18 @@
 require "zipkin-tracer/hostname_resolver"
 
 module Trace
-  class AsyncSqsClient
+  class SqsClient
     include SuckerPunch::Job
 
     def perform(sqs_options, queue_name, spans)
       spans_with_ips =
         ::ZipkinTracer::HostnameResolver.new.spans_with_ips(spans, ZipkinSqsSender::IP_FORMAT).map(&:to_h)
       sqs = Aws::SQS::Client.new(**sqs_options)
       queue_url = sqs.get_queue_url(queue_name: queue_name).queue_url
-      sqs.send_message(queue_url: queue_url, message_body: JSON.generate(spans_with_ips))
+      body = JSON.generate(spans_with_ips)
+      sqs.send_message(queue_url: queue_url, message_body: body)
     rescue Aws::SQS::Errors::NonExistentQueue
-      error_message = "A queue named '#{@queue_name}' does not exist."
+      error_message = "The queue '#{queue_name}' does not exist."
       SuckerPunch.logger.error(error_message)
     rescue => e
       SuckerPunch.logger.error(e)
@@ -28,12 +29,17 @@ class ZipkinSqsSender < ZipkinSenderBase
     def initialize(options)
       @sqs_options = options[:region] ? { region: options[:region] } : {}
       @queue_name = options[:queue_name]
+      @async = options[:async] != false
       SuckerPunch.logger = options[:logger]
       super(options)
     end
 
     def flush!
-      AsyncSqsClient.perform_async(@sqs_options, @queue_name, spans.dup)
+      if @async
+        SqsClient.perform_async(@sqs_options, @queue_name, spans.dup)
+      else
+        SqsClient.new.perform(@sqs_options, @queue_name, spans)
+      end
     end
   end
 end

spec/lib/config_spec.rb

@@ -5,6 +5,7 @@ module ZipkinTracer
     before do
       allow(Application).to receive(:logger).and_return(Logger.new(nil))
     end
+
     [:service_name, :json_api_host,
       :zookeeper, :log_tracing,
       :annotate_plugin, :filter_plugin, :whitelist_plugin].each do |method|
@@ -26,11 +27,34 @@ module ZipkinTracer
       end
     end
 
+    describe 'async' do
+      it 'uses "true" by default' do
+        config = Config.new(nil, {})
+        expect(config.async).to be true
+      end
+
+      it 'uses "false" only if a boolean with a value of "false" is passed' do
+        config = Config.new(nil, async: false)
+        expect(config.async).to be false
+
+        [nil, 0, "", " ", [], {}].each do |value|
+          config = Config.new(nil, async: value)
+          expect(config.async).to be true
+        end
+      end
+    end
+
     describe 'logger' do
       it 'uses the application logger' do
         config = Config.new(nil, {})
         expect(config.logger).to eq(Application.logger)
       end
+
+      it 'uses the logger provided in the configuration' do
+        logger = Logger.new(STDERR)
+        config = Config.new(nil, logger: logger)
+        expect(config.logger).to eq(logger)
+      end
     end
 
     describe 'sampled_as_boolean' do
@@ -94,6 +118,8 @@ module ZipkinTracer
 
       context 'sqs' do
         context 'Aws::SQS is defined' do
+          before { stub_const("Aws::SQS", "fake SQS") unless defined?(Aws::SQS) }
+
           it 'returns :sqs if sqs_queue_name has been set' do
             config = Config.new(nil, sqs_queue_name: 'zipkin-sqs')
             expect(config.adapter).to eq(:sqs)

spec/lib/middleware_shared_examples.rb

@@ -8,6 +8,7 @@
     end
     include_examples 'make requests', false
   end
+
   context 'We are not sampling this request' do
     before do
       Trace.tracer = Trace::NullSender.new
@@ -25,16 +26,16 @@
       example.run
     end
   end
+
   before do
     allow(::Trace).to receive(:default_endpoint).and_return(::Trace::Endpoint.new('127.0.0.1', '80', service_name))
     allow(::Trace::Endpoint).to receive(:host_to_i32).with(hostname).and_return(host_ip)
   end
-   include_examples 'make requests', true
-end
 
+  include_examples 'make requests', true
+end
 
 shared_examples 'make requests' do |expect_to_trace_request|
-
   let(:hostname) { 'service.example.com' }
   let(:host_ip) { 0x11223344 }
   let(:url_path) { '/some/path/here' }
@@ -114,7 +115,6 @@ def expect_tracing
   end
 
   context 'without tracing id' do
-
     it 'expects tracing' do
       if expect_to_trace_request
         expect_tracing
@@ -183,6 +183,5 @@ def expect_tracing
         process('', url)
       end
     end
-
   end
 end