flag for setting command line parameters

[dweiller]

I would like to be able to specify a set of command line arguments that get applied to all programs to be benchmarked, rather than have to include them in each executable argument. For example at the moment I tend to run benchmarks like this

poop "zig-out-v1/bin/prog arg1 arg2 arg3 arg4 arg5" "zig-out-v2/bin/prog arg1 arg2 arg3 arg4 arg5" "zig-out-v3/bin/prog arg1 arg2 arg3 arg4 arg5" "zig-out-v4/bin/prog arg1 arg2 arg3 arg4 arg5"

or, if I remember the precise way my shell does expansion I can do this:

poop zig-out-{v1,v2,v3,v4}/bin/prog" arg1 arg2 arg3 arg4 arg5"

(making sure there is no space between prog and the first ").

I propose there is a flag like --arg to accumulate components of argv allowing the above to be written as this:

poop zig-out-{v1,v2,v3,v4}/bin/prog --arg arg1 --arg arg2 --arg arg3 --arg arg4

The same argv splitting that is currently done should be preserved, and then arg1, arg2, arg3, arg4 would be appended to the argv of each command, this would allow to still supply arguments to a specific command like this:

poop zig-out-{v1,v2,v3}/bin/prog "zig-out-v4/bin/prog --shiny-new-option" --arg arg1 --arg arg2 --arg arg3 --arg arg4

Alternatively (or in addition?), there could be an --args flag to supply whitespace separated arguments:

poop zig-out-{v1,v2,v3}/bin/prog "zig-out-v4/bin/prog --shiny-new-option" --args "arg1 arg2 arg3 arg4"

[sharkdp]

In hyperfine, we have an option for parameterized benchmarks which is sort of a more general version of this. Just as another idea:

  -L, --parameter-list <VAR> <VALUES>
          Perform benchmark runs for each value in the comma-separated list
          VALUES. Replaces the string '{VAR}' in each command by the current
          parameter value.
          
          Example:  hyperfine -L compiler gcc,clang '{compiler} -O2 main.cpp'
          
          This performs benchmarks for 'gcc -O2 main.cpp' and 'clang -O2
          main.cpp'.
          
          The option can be specified multiple times to run benchmarks for all
          possible parameter combinations.