Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Grosser/cli #983

Merged
merged 2 commits into from
Dec 30, 2024
Merged

Grosser/cli #983

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
2d3cbfc
make cli options have more meaningful arg names + remove unreachable …
grosser Dec 30, 2024
ed47019
fix broken integration test
grosser Dec 30, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 20 additions & 18 deletions Readme.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -238,36 +238,37 @@ Setup for non-rails

Options are:
<!-- copy output from bundle exec ./bin/parallel_test -h -->
-n [PROCESSES] How many processes to use, default: available CPUs
-p, --pattern [PATTERN] run tests matching this regex pattern
--exclude-pattern [PATTERN] exclude tests matching this regex pattern
--group-by [TYPE] group tests by:
-n PROCESSES How many processes to use, default: available CPUs
-p, --pattern PATTERN run tests matching this regex pattern
--exclude-pattern PATTERN exclude tests matching this regex pattern
--group-by TYPE group tests by:
found - order of finding files
steps - number of cucumber/spinach steps
scenarios - individual cucumber scenarios
filesize - by size of the file
runtime - info from runtime log
default - runtime when runtime log is filled otherwise filesize
-m, --multiply-processes [FLOAT] use given number as a multiplier of processes to run
-s, --single [PATTERN] Run all matching files in the same process
-m, --multiply-processes COUNT use given number as a multiplier of processes to run
-s, --single PATTERN Run all matching files in the same process
-i, --isolate Do not run any other tests in the group used by --single(-s)
--isolate-n [PROCESSES] Use 'isolate' singles with number of processes, default: 1
--isolate-n PROCESSES Use 'isolate' singles with number of processes, default: 1
--highest-exit-status Exit with the highest exit status provided by test run(s)
--failure-exit-code [INT] Specify the exit code to use when tests fail
--specify-groups [SPECS] Use 'specify-groups' if you want to specify multiple specs running in multiple
--failure-exit-code INT Specify the exit code to use when tests fail
--specify-groups SPECS Use 'specify-groups' if you want to specify multiple specs running in multiple
processes in a specific formation. Commas indicate specs in the same process,
pipes indicate specs in a new process. Cannot use with --single, --isolate, or
--isolate-n. Ex.
$ parallel_tests -n 3 . --specify-groups '1_spec.rb,2_spec.rb|3_spec.rb'
Process 1 will contain 1_spec.rb and 2_spec.rb
Process 2 will contain 3_spec.rb
Process 3 will contain all other specs
--only-group INT[,INT] Only run the given group numbers.
--only-group GROUP_INDEX[,GROUP_INDEX]
Only run the given group numbers.
Changes `--group-by` default to 'filesize'.
-e, --exec [COMMAND] execute this code parallel and with ENV['TEST_ENV_NUMBER']
-o, --test-options '[OPTIONS]' execute test commands with those options
-t, --type [TYPE] test(default) / rspec / cucumber / spinach
--suffix [PATTERN] override built in test file pattern (should match suffix):
-e, --exec COMMAND execute this code parallel and with ENV['TEST_ENV_NUMBER']
-o, --test-options 'OPTIONS' execute test commands with those options
-t, --type TYPE test(default) / rspec / cucumber / spinach
--suffix PATTERN override built in test file pattern (should match suffix):
'_spec.rb$' - matches rspec files
'_(test|spec).rb$' - matches test or spec files
--serialize-stdout Serialize stdout output, nothing will be written until everything is done
Expand All @@ -276,14 +277,15 @@ Options are:
--combine-stderr Combine stderr into stdout, useful in conjunction with --serialize-stdout
--non-parallel execute same commands but do not in parallel, needs --exec
--no-symlinks Do not traverse symbolic links to find test files
--ignore-tags [PATTERN] When counting steps ignore scenarios with tags that match this pattern
--ignore-tags PATTERN When counting steps ignore scenarios with tags that match this pattern
--nice execute test commands with low priority.
--runtime-log [PATH] Location of previously recorded test runtimes
--allowed-missing [INT] Allowed percentage of missing runtimes (default = 50)
--runtime-log PATH Location of previously recorded test runtimes
--allowed-missing COUNT Allowed percentage of missing runtimes (default = 50)
--allow-duplicates When detecting files to run, allow duplicates
--unknown-runtime [FLOAT] Use given number as unknown runtime (otherwise use average time)
--unknown-runtime SECONDS Use given number as unknown runtime (otherwise use average time)
--first-is-1 Use "1" as TEST_ENV_NUMBER to not reuse the default test environment
--fail-fast Stop all groups when one group fails (best used with --test-options '--fail-fast' if supported
--test-file-limit LIMIT Limit to this number of files per test run by batching (for windows set to ~100 to stay below 8192 max command limit, might have bugs from reusing test-env-number and summarizing partial results)
--verbose Print debug output
--verbose-command Combines options --verbose-process-command and --verbose-rerun-command
--verbose-process-command Print the command that will be executed by each process before it begins
Expand Down
42 changes: 19 additions & 23 deletions lib/parallel_tests/cli.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -109,11 +109,7 @@ def run_tests_in_parallel(num_processes, options)
end

def run_tests(group, process_number, num_processes, options)
if group.empty?
{ stdout: '', exit_status: 0, command: nil, seed: nil }
else
@runner.run_tests(group, process_number, num_processes, options)
end
@runner.run_tests(group, process_number, num_processes, options)
end

def reprint_output(result, lockfile)
Expand Down Expand Up @@ -194,11 +190,11 @@ def parse_options!(argv)

Options are:
BANNER
opts.on("-n [PROCESSES]", Integer, "How many processes to use, default: available CPUs") { |n| options[:count] = n }
opts.on("-p", "--pattern [PATTERN]", "run tests matching this regex pattern") { |pattern| options[:pattern] = /#{pattern}/ }
opts.on("--exclude-pattern", "--exclude-pattern [PATTERN]", "exclude tests matching this regex pattern") { |pattern| options[:exclude_pattern] = /#{pattern}/ }
opts.on("-n PROCESSES", Integer, "How many processes to use, default: available CPUs") { |n| options[:count] = n }
opts.on("-p", "--pattern PATTERN", "run tests matching this regex pattern") { |pattern| options[:pattern] = /#{pattern}/ }
opts.on("--exclude-pattern", "--exclude-pattern PATTERN", "exclude tests matching this regex pattern") { |pattern| options[:exclude_pattern] = /#{pattern}/ }
opts.on(
"--group-by [TYPE]",
"--group-by TYPE",
<<~TEXT.rstrip.split("\n").join("\n#{newline_padding}")
group tests by:
found - order of finding files
Expand All @@ -209,11 +205,11 @@ def parse_options!(argv)
default - runtime when runtime log is filled otherwise filesize
TEXT
) { |type| options[:group_by] = type.to_sym }
opts.on("-m [FLOAT]", "--multiply-processes [FLOAT]", Float, "use given number as a multiplier of processes to run") do |multiply|
opts.on("-m COUNT", "--multiply-processes COUNT", Float, "use given number as a multiplier of processes to run") do |multiply|
options[:multiply] = multiply
end

opts.on("-s [PATTERN]", "--single [PATTERN]", "Run all matching files in the same process") do |pattern|
opts.on("-s PATTERN", "--single PATTERN", "Run all matching files in the same process") do |pattern|
(options[:single_process] ||= []) << /#{pattern}/
end

Expand All @@ -222,7 +218,7 @@ def parse_options!(argv)
end

opts.on(
"--isolate-n [PROCESSES]",
"--isolate-n PROCESSES",
Integer,
"Use 'isolate' singles with number of processes, default: 1"
) { |n| options[:isolate_count] = n }
Expand All @@ -233,13 +229,13 @@ def parse_options!(argv)
) { options[:highest_exit_status] = true }

opts.on(
"--failure-exit-code [INT]",
"--failure-exit-code INT",
Integer,
"Specify the exit code to use when tests fail"
) { |code| options[:failure_exit_code] = code }

opts.on(
"--specify-groups [SPECS]",
"--specify-groups SPECS",
<<~TEXT.rstrip.split("\n").join("\n#{newline_padding}")
Use 'specify-groups' if you want to specify multiple specs running in multiple
processes in a specific formation. Commas indicate specs in the same process,
Expand All @@ -253,24 +249,24 @@ def parse_options!(argv)
) { |groups| options[:specify_groups] = groups }

opts.on(
"--only-group INT[,INT]",
"--only-group GROUP_INDEX[,GROUP_INDEX]",
Array,
<<~TEXT.rstrip.split("\n").join("\n#{newline_padding}")
Only run the given group numbers.
Changes `--group-by` default to 'filesize'.
TEXT
) { |groups| options[:only_group] = groups.map(&:to_i) }

opts.on("-e", "--exec [COMMAND]", "execute this code parallel and with ENV['TEST_ENV_NUMBER']") { |arg| options[:execute] = Shellwords.shellsplit(arg) }
opts.on("-o", "--test-options '[OPTIONS]'", "execute test commands with those options") { |arg| options[:test_options] = Shellwords.shellsplit(arg) }
opts.on("-t", "--type [TYPE]", "test(default) / rspec / cucumber / spinach") do |type|
opts.on("-e", "--exec COMMAND", "execute this code parallel and with ENV['TEST_ENV_NUMBER']") { |arg| options[:execute] = Shellwords.shellsplit(arg) }
opts.on("-o", "--test-options 'OPTIONS'", "execute test commands with those options") { |arg| options[:test_options] = Shellwords.shellsplit(arg) }
opts.on("-t", "--type TYPE", "test(default) / rspec / cucumber / spinach") do |type|
@runner = load_runner(type)
rescue NameError, LoadError => e
puts "Runner for `#{type}` type has not been found! (#{e})"
abort
end
opts.on(
"--suffix [PATTERN]",
"--suffix PATTERN",
<<~TEXT.rstrip.split("\n").join("\n#{newline_padding}")
override built in test file pattern (should match suffix):
'_spec.rb$' - matches rspec files
Expand All @@ -282,12 +278,12 @@ def parse_options!(argv)
opts.on("--combine-stderr", "Combine stderr into stdout, useful in conjunction with --serialize-stdout") { options[:combine_stderr] = true }
opts.on("--non-parallel", "execute same commands but do not in parallel, needs --exec") { options[:non_parallel] = true }
opts.on("--no-symlinks", "Do not traverse symbolic links to find test files") { options[:symlinks] = false }
opts.on('--ignore-tags [PATTERN]', 'When counting steps ignore scenarios with tags that match this pattern') { |arg| options[:ignore_tag_pattern] = arg }
opts.on('--ignore-tags PATTERN', 'When counting steps ignore scenarios with tags that match this pattern') { |arg| options[:ignore_tag_pattern] = arg }
opts.on("--nice", "execute test commands with low priority.") { options[:nice] = true }
opts.on("--runtime-log [PATH]", "Location of previously recorded test runtimes") { |path| options[:runtime_log] = path }
opts.on("--allowed-missing [INT]", Integer, "Allowed percentage of missing runtimes (default = 50)") { |percent| options[:allowed_missing_percent] = percent }
opts.on("--runtime-log PATH", "Location of previously recorded test runtimes") { |path| options[:runtime_log] = path }
opts.on("--allowed-missing COUNT", Integer, "Allowed percentage of missing runtimes (default = 50)") { |percent| options[:allowed_missing_percent] = percent }
opts.on('--allow-duplicates', 'When detecting files to run, allow duplicates') { options[:allow_duplicates] = true }
opts.on("--unknown-runtime [FLOAT]", Float, "Use given number as unknown runtime (otherwise use average time)") { |time| options[:unknown_runtime] = time }
opts.on("--unknown-runtime SECONDS", Float, "Use given number as unknown runtime (otherwise use average time)") { |time| options[:unknown_runtime] = time }
opts.on("--first-is-1", "Use \"1\" as TEST_ENV_NUMBER to not reuse the default test environment") { options[:first_is_1] = true }
opts.on("--fail-fast", "Stop all groups when one group fails (best used with --test-options '--fail-fast' if supported") { options[:fail_fast] = true }
opts.on("--verbose", "Print debug output") { options[:verbose] = true }
Expand Down
2 changes: 2 additions & 0 deletions lib/parallel_tests/rspec/runtime_logger.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,8 @@ def example_group_finished(notification)
super if defined?(super)
end

def seed(*); end

def dump_summary(*); end

def dump_failures(*); end
Expand Down
Loading