Merge pull request #4069 from DataDog/run-appraisal

Use BUNDLE_GEMFILE directly instead of appraisal binary

Rakefile

@@ -59,7 +59,8 @@ namespace :test do
         command = if appraisal_group.empty?
                     "bundle exec rake #{spec_task}"
                   else
-                    "bundle exec appraisal #{ruby_runtime}-#{appraisal_group} rake #{spec_task}"
+                    gemfile = File.join(File.dirname(__FILE__), 'gemfiles', "#{ruby_runtime}-#{appraisal_group}.gemfile".tr('-', '_'))
+                    "env BUNDLE_GEMFILE=#{gemfile} bundle exec rake #{spec_task}"
                   end
 
         command += "'[#{spec_arguments}]'" if spec_arguments

docs/DevelopmentGuide.md

@@ -105,6 +105,24 @@ TEST_METADATA = {
 }
 ```
 
+**Using appraisal**
+
+`appraisal` command should only be used to update gemfiles in `gemfiles/`
+and install dependencies. It should not be used to run tests, since it does not
+work in all configurations. To run the tests, use:
+
+```sh
+env BUNDLE_GEMFILE=gemfiles/#{ruby_runtime}_#{appraisal_group}.gemfile rake #{spec_task}
+```
+
+Note that the file names use underscores while appraisal group and
+configuration definitions use dashes. The conversion could be performed as
+follows:
+
+```sh
+env BUNDLE_GEMFILE=gemfiles/#{ruby_runtime.tr('-', '_')}_#{appraisal_group.tr('-', '_')}.gemfile rake #{spec_task}
+```
+
 **Working with appraisal groups**
 
 Checkout [Apppraisal](https://github.com/thoughtbot/appraisal) to learn the basics.