support running a tests in batches in a single process

[grosser]

No description provided.

[rajaramaniyer]

Actually as long as we have some option to avoid the command line too long I am fine with either file based batching or command length based batching, which ever is best for the product.

Usage wise I am still not sure how I will use the command line option ..

This is the current call ..

'bundle exec parallel_cucumber'
" -o '--tags @regression_test -p parallel'"
' features/ --first-is-1 --group-by scenarios -n 8'

so after this fix, I should run like this?

'bundle exec parallel_cucumber'
" -o '--tags @regression_test -p parallel' --test-file-limit 100"
' features/ --first-is-1 --group-by scenarios -n 8'

[grosser]

yeah that would be how to use it
can you try using the branch to see if it works as expected ?

gem 'parallel_tests', git: 'https://github.com/grosser/parallel_tests', branch: 'grosser/max-tests'

[rajaramaniyer]

I am getting error when i run bundle update

fatal: unable to access 'https://github.com/grosser/parallel_tests/': SSL certificate problem: unable to get local issuer certificate

can i only update cli.rb and runner.rb and try ? will it work ?

[grosser]

yeah updating these 2 should also works

[rajaramaniyer]

in my cli.rb i see this implementation of run_tests

def run_tests(group, process_number, num_processes, options)
      if group.empty?
        { stdout: '', exit_status: 0, command: nil, seed: nil }
      else
        @runner.run_tests(group, process_number, num_processes, options)
      end
    end

So I am trying it out after adding the new code as an elsif ..

def run_tests(group, process_number, num_processes, options)
      if group.empty?
        { stdout: '', exit_status: 0, command: nil, seed: nil }
      elsif (limit = options[:test_file_limit])
        # TODO: will have some bugs with summarizing results and last process
        results = group.each_slice(limit).map do |slice|
          @runner.run_tests(slice, process_number, num_processes, options)
        end
        result = results[0]
        results[1..].each do |res|
          result[:stdout] = result[:stdout].to_s + res[:stdout].to_s
          result[:exit_status] = [res[:exit_status], result[:exit_status]].max
          # adding all files back in, not using original cmd to show what was actually run
          result[:command] |= res[:command]
        end
        result
      else
        @runner.run_tests(group, process_number, num_processes, options)
      end
    end

[grosser]

and did it work ?

[rajaramaniyer]

Sorry for the late response. I got busy with some other tasks. Today I tested lib/parallel_tests/cli.rb changes.

There are no errors and it completed in two batches.

But I think there are less scenarios in the final report.

Also we have the following in the cucumber.yaml file:

parallel: 
  --format json --publish-quiet --out output/report/json_output/output_regression_<%= ENV['TEST_ENV_NUMBER'].to_s.rjust(2, '0') %>.json  --format ParallelTests::Cucumber::FailuresLogger --out output/cucumber_failures.log

so after the first batch maybe the output file is getting overwritten ?

i will add a timestamp to the output file name and retry the test.

[grosser]

thx!
I'll release it how it is since it's opt-in, and then come back and fix issues if you find any :)

[grosser]

4.8.0

[rajaramaniyer]

so how can we install after this fix with normal 'gem parallel_tests' ?

[grosser]

just bundle update parallel_tests