diff --git a/docs/markdown/Python-3-module.md b/docs/markdown/Python-3-module.md index 1631b2a90ecf..311671dc5ddc 100644 --- a/docs/markdown/Python-3-module.md +++ b/docs/markdown/Python-3-module.md @@ -9,9 +9,8 @@ This module is deprecated and replaced by the ## find_python This is a cross platform way of finding the Python 3 executable, which -may have a different name on different operating systems. Returns an -[[@external_program]] -object. +may have a different name on different operating systems. Returns a +[[@program]] object. *Added 0.38.0* diff --git a/docs/markdown/Python-module.md b/docs/markdown/Python-module.md index 66081762d795..24febcece94f 100644 --- a/docs/markdown/Python-module.md +++ b/docs/markdown/Python-module.md @@ -76,7 +76,7 @@ Keyword arguments are the following: ## `python_installation` object -The `python_installation` object is an [[@external_program]], with several +The `python_installation` object is an [[@program]], with several added methods. ### Methods @@ -91,7 +91,7 @@ str py_installation.path() *Deprecated in 0.55: use `full_path()` instead* -Works like the path method of `ExternalProgram` objects. Was not provided prior +Works like the path method of `Program` objects. Was not provided prior to 0.50.0 due to a bug. #### `full_path()` @@ -102,7 +102,7 @@ str py_installation.full_path() *(since 0.55.0)* -Works like the `full_path()` method of `ExternalProgram` objects: [[external_program.full_path]] +Works like the `full_path()` method of `Program` objects: [[program.full_path]] #### `extension_module()` diff --git a/docs/markdown/Release-notes-for-0.59.0.md b/docs/markdown/Release-notes-for-0.59.0.md index 5cdffe867cd9..97ff48203e45 100644 --- a/docs/markdown/Release-notes-for-0.59.0.md +++ b/docs/markdown/Release-notes-for-0.59.0.md @@ -197,7 +197,7 @@ executable( The [[@build_tgt]] object now supports the following two functions, to ensure feature compatibility with -[[@external_program]] objects: +[[@program]] objects: - `found()`: Always returns `true`. This function is meant to make executables objects feature compatible with diff --git a/docs/yaml/builtins/meson.yaml b/docs/yaml/builtins/meson.yaml index eac83685bb8b..3e486dac0a30 100644 --- a/docs/yaml/builtins/meson.yaml +++ b/docs/yaml/builtins/meson.yaml @@ -45,7 +45,7 @@ methods: posargs: script_name: - type: str | file | external_program + type: str | file | program description: | The script to execute. @@ -55,7 +55,7 @@ methods: varargs: name: arg - type: str | file | external_program + type: str | file | program since: 0.49.0 description: | Additional arguments @@ -102,7 +102,7 @@ methods: posargs: script_name: - type: str | file | external_program | exe | custom_tgt | custom_idx + type: str | file | program | exe | custom_tgt | custom_idx description: | The script to execute. @@ -113,7 +113,7 @@ methods: varargs: name: arg - type: str | file | external_program | exe | custom_tgt | custom_idx + type: str | file | program | exe | custom_tgt | custom_idx since: 0.49.0 description: | Additional arguments @@ -389,7 +389,7 @@ methods: description: The name of the program to override. program: - type: exe | file | external_program + type: exe | file | program description: The program to set as the override for `progname`. - name: override_dependency diff --git a/docs/yaml/functions/add_test_setup.yaml b/docs/yaml/functions/add_test_setup.yaml index a29b137e2b0b..41e02730b350 100644 --- a/docs/yaml/functions/add_test_setup.yaml +++ b/docs/yaml/functions/add_test_setup.yaml @@ -26,7 +26,7 @@ kwargs: environment juggling. *(Since 0.52.0)* A dictionary is also accepted. exe_wrapper: - type: array[str | external_program] + type: array[str | program] description: The command or script followed by the arguments to it gdb: diff --git a/docs/yaml/functions/benchmark.yaml b/docs/yaml/functions/benchmark.yaml index fe69f28fd6f2..c70ad2ea6e82 100644 --- a/docs/yaml/functions/benchmark.yaml +++ b/docs/yaml/functions/benchmark.yaml @@ -22,13 +22,13 @@ posargs: description: The *unique* test id executable: - type: exe | jar | external_program | file | custom_tgt | custom_idx + type: exe | jar | program | file | custom_tgt | custom_idx description: | The program to execute. *(Since 1.4.0)* A CustomTarget is also accepted. kwargs: args: - type: array[str | file | tgt | external_program] + type: array[str | file | tgt | program] description: Arguments to pass to the executable env: diff --git a/docs/yaml/functions/custom_target.yaml b/docs/yaml/functions/custom_target.yaml index 5836bb764e3e..3ffd5ddee2cb 100644 --- a/docs/yaml/functions/custom_target.yaml +++ b/docs/yaml/functions/custom_target.yaml @@ -124,7 +124,7 @@ kwargs: serializing all targets in this pool. command: - type: array[str | file | exe | external_program] + type: array[str | file | exe | program] description: | Command to run to create outputs from inputs. The command may be strings or the return value of functions that return file-like diff --git a/docs/yaml/functions/find_program.yaml b/docs/yaml/functions/find_program.yaml index 110f5df83aa6..d03497ced7d0 100644 --- a/docs/yaml/functions/find_program.yaml +++ b/docs/yaml/functions/find_program.yaml @@ -1,5 +1,5 @@ name: find_program -returns: external_program +returns: program description: | `program_name` here is a string that can be an executable or script to be searched for in `PATH` or other places inside the project. @@ -45,7 +45,7 @@ description: | *Since 1.2.0* `find_program('meson')` is automatically overridden to the Meson command used to execute the build script. - The returned [[@external_program]] object also has documented methods. + The returned [[@program]] object also has documented methods. posargs: program_name: @@ -73,7 +73,7 @@ kwargs: When `true`, Meson will abort if no program can be found. If `required` is set to `false`, Meson continue even if none of the programs can be found. You can - then use the `.found()` method on the returned [[@external_program]] to check + then use the `.found()` method on the returned [[@program]] to check whether it was found or not. *(since 0.47.0)* The value of a [`feature`](Build-options.md#features) option can also be passed to the `required` keyword argument. diff --git a/docs/yaml/functions/generator.yaml b/docs/yaml/functions/generator.yaml index c505394945b4..e45c5a27e3b7 100644 --- a/docs/yaml/functions/generator.yaml +++ b/docs/yaml/functions/generator.yaml @@ -40,7 +40,7 @@ description: | posargs: exe: - type: exe | external_program + type: exe | program description: Executable for the command to run kwargs: diff --git a/docs/yaml/functions/run_command.yaml b/docs/yaml/functions/run_command.yaml index 1c9cc531ffde..c61b08655597 100644 --- a/docs/yaml/functions/run_command.yaml +++ b/docs/yaml/functions/run_command.yaml @@ -17,7 +17,7 @@ description: | varargs: name: command - type: str | file | external_program + type: str | file | program description: The command to execute during the setup process. kwargs: diff --git a/docs/yaml/functions/run_target.yaml b/docs/yaml/functions/run_target.yaml index bf670d8d2816..0e80944ba014 100644 --- a/docs/yaml/functions/run_target.yaml +++ b/docs/yaml/functions/run_target.yaml @@ -31,7 +31,7 @@ posargs: kwargs: command: - type: array[exe| external_program | custom_tgt | file | str] + type: array[exe| program | custom_tgt | file | str] description: | An array containing the command to run and the arguments to pass to it. Each array element may be a string or a target. For diff --git a/docs/yaml/functions/summary.yaml b/docs/yaml/functions/summary.yaml index b5312821e0a2..44427adc9e16 100644 --- a/docs/yaml/functions/summary.yaml +++ b/docs/yaml/functions/summary.yaml @@ -65,7 +65,7 @@ arg_flattening: false posargs: key_or_dict: - type: str | dict[str | bool | int | dep | external_program | array[str | bool | int | dep | external_program]] + type: str | dict[str | bool | int | dep | program | array[str | bool | int | dep | program]] description: | The name of the new entry, or a dict containing multiple entries. If a dict is passed it is equivalent to calling summary() once for each @@ -74,7 +74,7 @@ posargs: optargs: value: - type: str | bool | int | dep | external_program | array[str | bool | int | dep | external_program] + type: str | bool | int | dep | program | array[str | bool | int | dep | program] description: | The value to print for the `key`. Only valid if `key_or_dict` is a str. diff --git a/docs/yaml/functions/test.yaml b/docs/yaml/functions/test.yaml index d56822e5828a..691bd04ec643 100644 --- a/docs/yaml/functions/test.yaml +++ b/docs/yaml/functions/test.yaml @@ -4,7 +4,7 @@ description: | Defines a test to run with the test harness. Takes two positional arguments, the first is the name of the test and the second is the executable to run. The executable can be an [[@exe]] object returned by - [[executable]] or an [[@external_program]] object] returned by + [[executable]] or an [[@program]] object] returned by [[find_program]]. *(since 0.55.0)* When cross compiling, if an exe_wrapper is needed and diff --git a/docs/yaml/functions/vcs_tag.yaml b/docs/yaml/functions/vcs_tag.yaml index bf223e7ef223..86ce967dfff2 100644 --- a/docs/yaml/functions/vcs_tag.yaml +++ b/docs/yaml/functions/vcs_tag.yaml @@ -22,7 +22,7 @@ description: | kwargs: command: - type: array[exe | external_program | custom_tgt | file | str] + type: array[exe | program | custom_tgt | file | str] description: | The command to execute, see [[custom_target]] for details on how this command must be specified. @@ -32,7 +32,7 @@ kwargs: *(since 0.62.0)* [[@file]] is accepted. - *(since 0.63.0)* [[@custom_tgt]], [[@exe]], and [[@external_program]] are accepted. + *(since 0.63.0)* [[@custom_tgt]], [[@exe]], and [[@program]] are accepted. input: type: str diff --git a/docs/yaml/objects/build_tgt.yaml b/docs/yaml/objects/build_tgt.yaml index 3fed56c43280..198b18aea1f8 100644 --- a/docs/yaml/objects/build_tgt.yaml +++ b/docs/yaml/objects/build_tgt.yaml @@ -53,9 +53,9 @@ methods: deprecated: 0.59.0 description: | Does the exact same as [[build_tgt.full_path]]. **NOTE**: This - function is solely kept for compatibility with [[@external_program]] objects. + function is solely kept for compatibility with [[@program]] objects. It will be removed once the, also deprecated, corresponding `path()` - function in the [[@external_program]] object is removed. + function in the [[@program]] object is removed. - name: private_dir_include returns: inc @@ -74,9 +74,9 @@ methods: since: 0.59.0 description: | Always returns `true`. This function is meant to make executables - objects feature compatible with [[@external_program]] objects. This + objects feature compatible with [[@program]] objects. This simplifies use-cases where an executable is used instead of - an [[@external_program]]. + an [[@program]]. - name: vala_header returns: file diff --git a/docs/yaml/objects/external_program.yaml b/docs/yaml/objects/external_program.yaml index db5d39fab0be..b1b3953152a9 100644 --- a/docs/yaml/objects/external_program.yaml +++ b/docs/yaml/objects/external_program.yaml @@ -1,4 +1,4 @@ -name: external_program +name: program long_name: External program description: Opaque object representing an external program @@ -11,7 +11,7 @@ methods: returns: str deprecated: 0.55.0 description: | - *Deprecated:* Use [[external_program.full_path]] instead. + *Deprecated:* Use [[program.full_path]] instead. Returns a string pointing to the script or executable. diff --git a/mesonbuild/backend/backends.py b/mesonbuild/backend/backends.py index dddcf67d8dc7..f2312a6d8817 100644 --- a/mesonbuild/backend/backends.py +++ b/mesonbuild/backend/backends.py @@ -535,7 +535,7 @@ def determine_swift_dep_dirs(self, target: build.BuildTarget) -> T.List[str]: return result def get_executable_serialisation( - self, cmd: T.Sequence[T.Union[programs.ExternalProgram, build.BuildTarget, build.CustomTarget, File, str]], + self, cmd: T.Sequence[T.Union[programs.Program, build.BuildTarget, build.CustomTarget, build.CustomTargetIndex, File, str]], workdir: T.Optional[str] = None, extra_bdeps: T.Optional[T.List[build.BuildTarget]] = None, capture: T.Optional[str] = None, @@ -548,13 +548,15 @@ def get_executable_serialisation( # XXX: cmd_args either need to be lowered to strings, or need to be checked for non-string arguments, right? exe, *raw_cmd_args = cmd - if isinstance(exe, programs.ExternalProgram): + if isinstance(exe, build.LocalProgram): + exe = exe.program + if isinstance(exe, programs.Program): exe_cmd = exe.get_command() exe_for_machine = exe.for_machine elif isinstance(exe, build.BuildTarget): exe_cmd = [self.get_target_filename_abs(exe)] exe_for_machine = exe.for_machine - elif isinstance(exe, build.CustomTarget): + elif isinstance(exe, (build.CustomTarget, build.CustomTargetIndex)): # The output of a custom target can either be directly runnable # or not, that is, a script, a native binary or a cross compiled # binary when exe wrapper is available and when it is not. @@ -571,9 +573,11 @@ def get_executable_serialisation( cmd_args: T.List[str] = [] for c in raw_cmd_args: - if isinstance(c, programs.ExternalProgram): + if isinstance(c, build.LocalProgram): + c = c.program + if isinstance(c, programs.Program): cmd_args += c.get_command() - elif isinstance(c, (build.BuildTarget, build.CustomTarget)): + elif isinstance(c, (build.BuildTarget, build.CustomTarget, build.CustomTargetIndex)): cmd_args.append(self.get_target_filename_abs(c)) elif isinstance(c, mesonlib.File): cmd_args.append(c.rel_to_builddir(self.environment.source_dir)) @@ -620,8 +624,8 @@ def get_executable_serialisation( exe_wrapper, workdir, extra_paths, capture, feed, tag, verbose, installdir_map) - def as_meson_exe_cmdline(self, exe: T.Union[str, mesonlib.File, build.BuildTarget, build.CustomTarget, programs.ExternalProgram], - cmd_args: T.Sequence[T.Union[str, mesonlib.File, build.BuildTarget, build.CustomTarget, programs.ExternalProgram]], + def as_meson_exe_cmdline(self, exe: T.Union[str, mesonlib.File, build.BuildTarget, build.CustomTarget, build.CustomTargetIndex, programs.Program], + cmd_args: T.Sequence[T.Union[str, mesonlib.File, build.BuildTarget, build.CustomTarget, build.CustomTargetIndex, programs.Program]], workdir: T.Optional[str] = None, extra_bdeps: T.Optional[T.List[build.BuildTarget]] = None, capture: T.Optional[str] = None, @@ -633,7 +637,7 @@ def as_meson_exe_cmdline(self, exe: T.Union[str, mesonlib.File, build.BuildTarge ''' Serialize an executable for running with a generator or a custom target ''' - cmd: T.List[T.Union[str, mesonlib.File, build.BuildTarget, build.CustomTarget, programs.ExternalProgram]] = [] + cmd: T.List[T.Union[str, mesonlib.File, build.BuildTarget, build.CustomTarget, build.CustomTargetIndex, programs.Program]] = [] cmd.append(exe) cmd.extend(cmd_args) es = self.get_executable_serialisation(cmd, workdir, extra_bdeps, capture, feed, env, can_use_rsp_file, verbose=verbose) @@ -693,8 +697,8 @@ def as_meson_exe_cmdline(self, exe: T.Union[str, mesonlib.File, build.BuildTarge ', '.join(reasons) ) - if isinstance(exe, (programs.ExternalProgram, - build.BuildTarget, build.CustomTarget)): + if isinstance(exe, (programs.Program, + build.BuildTarget, build.CustomTarget, build.CustomTargetIndex)): basename = os.path.basename(exe.name) elif isinstance(exe, mesonlib.File): basename = os.path.basename(exe.fname) @@ -1124,7 +1128,7 @@ def extract_dll_paths(cls, target: build.BuildTarget) -> T.Set[str]: return results def determine_windows_extra_paths( - self, target: T.Union[build.BuildTargetTypes, programs.ExternalProgram, mesonlib.File, str], + self, target: T.Union[build.BuildTargetTypes, programs.Program, mesonlib.File, str], extra_bdeps: T.Sequence[build.BuildTargetTypes]) -> T.List[str]: """On Windows there is no such thing as an rpath. @@ -1168,11 +1172,13 @@ def create_test_serialisation(self, tests: T.List['Test']) -> T.List[TestSeriali arr: T.List[TestSerialisation] = [] for t in sorted(tests, key=lambda tst: -1 * tst.priority): exe = t.get_exe() - if isinstance(exe, programs.ExternalProgram): + if isinstance(exe, build.LocalProgram): + exe = exe.program + if isinstance(exe, programs.Program): cmd = exe.get_command() else: cmd = [os.path.join(self.environment.get_build_dir(), self.get_target_filename(exe))] - if isinstance(exe, (build.BuildTarget, programs.ExternalProgram)): + if isinstance(exe, (build.BuildTarget, programs.Program)): test_for_machine = exe.for_machine else: # E.g. an external verifier or simulator program run on a generated executable. @@ -1344,8 +1350,10 @@ def check_clock_skew(self, file_list: T.Iterable[str]) -> None: if delta > 0.001: raise MesonException(f'Clock skew detected. File {absf} has a time stamp {delta:.4f}s in the future.') - def build_target_to_cmd_array(self, bt: T.Union[build.BuildTarget, programs.ExternalProgram]) -> T.List[str]: - if isinstance(bt, build.BuildTarget): + def build_target_to_cmd_array(self, bt: T.Union[build.BuildTarget, build.CustomTarget, build.CustomTargetIndex, programs.Program]) -> T.List[str]: + if isinstance(bt, build.LocalProgram): + bt = bt.program + if isinstance(bt, (build.Target, build.CustomTargetIndex)): arr = [os.path.join(self.environment.get_build_dir(), self.get_target_filename(bt))] else: arr = bt.get_command() @@ -1426,6 +1434,8 @@ def get_custom_target_sources(self, target: build.CustomTarget) -> T.List[str]: ''' srcs: T.List[str] = [] for i in target.get_sources(): + if isinstance(i, build.LocalProgram): + i = i.program if isinstance(i, str): fname = [os.path.join(self.build_to_src, target.subdir, i)] elif isinstance(i, build.BuildTarget): @@ -1436,10 +1446,11 @@ def get_custom_target_sources(self, target: build.CustomTarget) -> T.List[str]: fname = [os.path.join(self.get_target_private_dir(target), p) for p in i.get_outputs()] elif isinstance(i, build.ExtractedObjects): fname = self.determine_ext_objs(i) - elif isinstance(i, programs.ExternalProgram): + elif isinstance(i, programs.Program): + assert isinstance(i, programs.ExternalProgram) assert i.found(), "This shouldn't be possible" - assert i.path is not None, 'for mypy' - fname = [i.path] + assert i.get_path() is not None, 'for mypy' + fname = [i.get_path()] else: fname = [i.rel_to_builddir(self.build_to_src)] if target.absolute_paths: @@ -1505,7 +1516,7 @@ def get_custom_target_dir_include_args( def eval_custom_target_command( self, target: build.CustomTarget, absolute_outputs: bool = False) -> \ - T.Tuple[T.List[str], T.List[str], T.List[str | programs.ExternalProgram]]: + T.Tuple[T.List[str], T.List[str], T.List[str | programs.Program]]: # We want the outputs to be absolute only when using the VS backend # XXX: Maybe allow the vs backend to use relative paths too? source_root = self.build_to_src @@ -1518,7 +1529,7 @@ def eval_custom_target_command( outputs = [os.path.join(outdir, i) for i in target.get_outputs()] inputs = self.get_custom_target_sources(target) # Evaluate the command list - cmd: T.List[str | programs.ExternalProgram] = [] + cmd: T.List[str | programs.Program] = [] for i in target.command: if isinstance(i, build.BuildTarget): cmd += self.build_target_to_cmd_array(i) @@ -1554,7 +1565,7 @@ def eval_custom_target_command( if not target.absolute_paths: pdir = self.get_target_private_dir(target) i = i.replace('@PRIVATE_DIR@', pdir) - elif isinstance(i, programs.ExternalProgram): + elif isinstance(i, programs.Program): # Let it pass and be extended elsewhere pass else: @@ -1951,7 +1962,7 @@ def get_introspection_data(self, target_id: str, target: build.Target) -> T.List compiler += [j] elif isinstance(j, (build.BuildTarget, build.CustomTarget)): compiler += j.get_outputs() - elif isinstance(j, programs.ExternalProgram): + elif isinstance(j, programs.Program): compiler += j.get_command() else: raise RuntimeError(f'Type "{type(j).__name__}" is not supported in get_introspection_data. This is a bug') diff --git a/mesonbuild/backend/ninjabackend.py b/mesonbuild/backend/ninjabackend.py index 464e703e9acf..642621e53a9c 100644 --- a/mesonbuild/backend/ninjabackend.py +++ b/mesonbuild/backend/ninjabackend.py @@ -2792,6 +2792,10 @@ def generate_genlist_for_target(self, genlist: build.GeneratedList, target: buil exe = generator.get_exe() infilelist = genlist.get_inputs() extra_dependencies = self.get_target_depend_files(genlist) + for d in genlist.extra_depends: + # Add a dependency on all the outputs of this target + for output in d.get_outputs(): + extra_dependencies.append(os.path.join(self.get_target_dir(d), output)) for curfile in infilelist: infilename = curfile.rel_to_builddir(self.build_to_src, self.get_target_private_dir(target)) base_args = generator.get_arglist(infilename) @@ -2838,8 +2842,6 @@ def generate_genlist_for_target(self, genlist: build.GeneratedList, target: buil reason = f' (wrapped by meson {reason})' elem.add_item('DESC', f'Generating {what}{reason}') - if isinstance(exe, build.BuildTarget): - elem.add_dep(self.get_target_filename(exe)) elem.add_item('COMMAND', cmdlist) self.add_build(elem) diff --git a/mesonbuild/backend/vs2010backend.py b/mesonbuild/backend/vs2010backend.py index cc7ce1af97d3..3ec9fc30ae42 100644 --- a/mesonbuild/backend/vs2010backend.py +++ b/mesonbuild/backend/vs2010backend.py @@ -381,10 +381,7 @@ def get_target_deps(self, t: T.Dict[T.Any, build.AnyTargetType], recursive=False all_deps[gendep.target.get_id()] = gendep.target else: generator = gendep.get_generator() - gen_exe = generator.get_exe() - if isinstance(gen_exe, build.Executable): - all_deps[gen_exe.get_id()] = gen_exe - for d in itertools.chain(generator.depends, gendep.depends): + for d in itertools.chain(generator.depends, gendep.depends, gendep.extra_depends): if isinstance(d, build.CustomTargetIndex): all_deps[d.get_id()] = d.target elif isinstance(d, build.Target): diff --git a/mesonbuild/build.py b/mesonbuild/build.py index 138f1ae03233..629606709a4a 100644 --- a/mesonbuild/build.py +++ b/mesonbuild/build.py @@ -363,7 +363,6 @@ def __init__(self, environment: Environment): self.stdlibs = PerMachine({}, {}) self.test_setups: T.Dict[str, TestSetup] = {} self.test_setup_default_name = None - self.find_overrides: T.Dict[str, T.Union['OverrideExecutable', programs.ExternalProgram, programs.OverrideProgram]] = {} self.searched_programs: T.Set[str] = set() # The list of all programs that have been searched for. # If we are doing a cross build we need two caches, if we're doing a @@ -1980,7 +1979,7 @@ def __str__(self) -> str: class Generator(HoldableObject): def __init__(self, env: Environment, - exe: T.Union['Executable', programs.ExternalProgram], + exe: programs.Program, arguments: T.List[str], output: T.List[str], # how2dataclass @@ -2002,7 +2001,7 @@ def __repr__(self) -> str: repr_str = "<{0}: {1}>" return repr_str.format(self.__class__.__name__, self.exe) - def get_exe(self) -> T.Union['Executable', programs.ExternalProgram]: + def get_exe(self) -> programs.Program: return self.exe def get_base_outnames(self, inname: str) -> T.List[str]: @@ -2076,7 +2075,7 @@ def __post_init__(self) -> None: self.infilelist: T.List[FileMaybeInTargetPrivateDir] = [] self.outfilelist: T.List[str] = [] self.outmap: T.Dict[FileMaybeInTargetPrivateDir, T.List[str]] = {} - self.extra_depends = [] # XXX: Doesn't seem to be used? + self.extra_depends = [] self.depend_files: T.List[File] = [] if self.extra_args is None: @@ -2085,9 +2084,12 @@ def __post_init__(self) -> None: if self.env is None: self.env: EnvironmentVariables = EnvironmentVariables() - if isinstance(self.generator.exe, programs.ExternalProgram): + if isinstance(self.generator.exe, programs.Program): if not self.generator.exe.found(): raise InvalidArguments('Tried to use not-found external program as generator') + if isinstance(self.generator.exe, LocalProgram): + self.extra_depends.append(self.generator.exe.program) + else: path = self.generator.exe.get_path() if os.path.isabs(path): # Can only add a dependency on an external program which we @@ -2158,8 +2160,6 @@ def __init__( self.implib_name = kwargs.get('implib') # Only linkwithable if using export_dynamic self.is_linkwithable = self.export_dynamic - # Remember that this exe was returned by `find_program()` through an override - self.was_returned_by_find_program = False self.vs_module_defs: T.Optional[File] = None self.process_vs_module_defs_kw(kwargs) @@ -2241,10 +2241,6 @@ def post_init(self) -> None: def get_default_install_dir(self) -> T.Union[T.Tuple[str, str], T.Tuple[None, None]]: return self.environment.get_bindir(), '{bindir}' - def description(self): - '''Human friendly description of the executable''' - return self.name - def type_suffix(self): return "@exe" @@ -2267,21 +2263,6 @@ def get_debug_filename(self) -> T.Optional[str]: def is_linkable_target(self) -> bool: return self.is_linkwithable - def get_command(self) -> 'ImmutableListProtocol[str]': - """Provides compatibility with ExternalProgram. - - Since you can override ExternalProgram instances with Executables. - """ - return self.outputs - - def get_path(self) -> str: - """Provides compatibility with ExternalProgram.""" - return os.path.join(self.subdir, self.filename) - - def found(self) -> bool: - """Provides compatibility with ExternalProgram.""" - return True - class StaticLibrary(BuildTarget): known_kwargs = known_stlib_kwargs @@ -2781,17 +2762,19 @@ class CommandBase: dependencies: T.List[T.Union[BuildTarget, 'CustomTarget']] subproject: str - def flatten_command(self, cmd: T.Sequence[T.Union[str, File, programs.ExternalProgram, BuildTargetTypes]]) -> \ - T.List[T.Union[str, File, BuildTarget, CustomTarget, programs.ExternalProgram]]: + def flatten_command(self, cmd: T.Sequence[T.Union[str, File, programs.Program, BuildTargetTypes]]) -> \ + T.List[T.Union[str, File, BuildTarget, CustomTarget, programs.Program]]: cmd = listify(cmd) final_cmd: T.List[T.Union[str, File, BuildTarget, 'CustomTarget']] = [] for c in cmd: + if isinstance(c, LocalProgram): + c = c.program if isinstance(c, str): final_cmd.append(c) elif isinstance(c, File): self.depend_files.append(c) final_cmd.append(c) - elif isinstance(c, programs.ExternalProgram): + elif isinstance(c, programs.Program): if not c.found(): raise InvalidArguments('Tried to use not-found external program in "command"') path = c.get_path() @@ -2851,10 +2834,10 @@ def __init__(self, environment: Environment, command: T.Sequence[T.Union[ str, BuildTargetTypes, GeneratedList, - programs.ExternalProgram, File]], + programs.Program, File]], sources: T.Sequence[T.Union[ str, File, BuildTargetTypes, ExtractedObjects, - GeneratedList, programs.ExternalProgram]], + GeneratedList, programs.Program]], outputs: T.List[str], *, build_always_stale: bool = False, @@ -2961,7 +2944,7 @@ def get_outputs(self) -> T.List[str]: def get_filename(self) -> str: return self.outputs[0] - def get_sources(self) -> T.List[T.Union[str, File, BuildTarget, GeneratedTypes, ExtractedObjects, programs.ExternalProgram]]: + def get_sources(self) -> T.List[T.Union[str, File, BuildTarget, GeneratedTypes, ExtractedObjects, programs.Program]]: return self.sources def get_generated_lists(self) -> T.List[GeneratedList]: @@ -3112,7 +3095,7 @@ class RunTarget(Target, CommandBase): typename = 'run' def __init__(self, name: str, - command: T.Sequence[T.Union[str, File, BuildTargetTypes, programs.ExternalProgram]], + command: T.Sequence[T.Union[str, File, BuildTargetTypes, programs.Program]], dependencies: T.Sequence[AnyTargetType], subdir: str, subproject: str, @@ -3342,17 +3325,45 @@ def get(self, name: str) -> T.Tuple[T.Union[str, int, bool], T.Optional[str]]: def keys(self) -> T.Iterator[str]: return self.values.keys() -class OverrideExecutable(Executable): - def __init__(self, executable: Executable, version: str): - self._executable = executable - self._version = version +class LocalProgram(programs.Program): + ''' A wrapper for a program that acts as a build dependency + for other targets.''' + def __init__(self, program: T.Union[programs.ExternalProgram, Executable, CustomTarget, CustomTargetIndex], version: str) -> None: + super().__init__() + if isinstance(program, CustomTarget): + if len(program.outputs) != 1: + raise InvalidArguments('CustomTarget used as LocalProgram must have exactly one output.') + self.name = program.name + self.program = program + self.version = version - def __getattr__(self, name: str) -> T.Any: - _executable = object.__getattribute__(self, '_executable') - return getattr(_executable, name) + def found(self) -> bool: + return True def get_version(self, interpreter: T.Optional[Interpreter] = None) -> str: - return self._version + return self.version + + def get_command(self) -> T.List[str]: + if isinstance(self.program, programs.ExternalProgram): + return self.program.get_command() + # Only the backend knows the actual path to the build program. + raise MesonBugException('Cannot call get_command() on program that is a build target.') + + def get_path(self) -> str: + if isinstance(self.program, programs.ExternalProgram): + return self.program.get_path() + # Only the backend knows the actual path to the build program. + raise MesonBugException('Cannot call get_path() on program that is a build target.') + + def description(self) -> str: + if isinstance(self.program, programs.ExternalProgram): + return self.program.description() + if isinstance(self.program, Executable): + return self.program.name + return self.program.get_filename() + + def runnable(self) -> bool: + return isinstance(self.program, programs.ExternalProgram) # A bit poorly named, but this represents plain data files to copy # during install. diff --git a/mesonbuild/interpreter/__init__.py b/mesonbuild/interpreter/__init__.py index 600798f1a5b8..72f0178a5c4d 100644 --- a/mesonbuild/interpreter/__init__.py +++ b/mesonbuild/interpreter/__init__.py @@ -19,7 +19,6 @@ 'SubprojectHolder', 'DependencyHolder', 'GeneratedListHolder', - 'ExternalProgramHolder', 'extract_required_kwarg', 'ArrayHolder', @@ -34,8 +33,7 @@ from .interpreterobjects import (ExecutableHolder, BuildTargetHolder, CustomTargetHolder, CustomTargetIndexHolder, MachineHolder, Test, ConfigurationDataHolder, SubprojectHolder, DependencyHolder, - GeneratedListHolder, ExternalProgramHolder, - extract_required_kwarg) + GeneratedListHolder, extract_required_kwarg) from .primitives import ( ArrayHolder, diff --git a/mesonbuild/interpreter/interpreter.py b/mesonbuild/interpreter/interpreter.py index 56659460a844..ea571a1853b9 100644 --- a/mesonbuild/interpreter/interpreter.py +++ b/mesonbuild/interpreter/interpreter.py @@ -21,7 +21,7 @@ FileMode, MachineChoice, is_parent_path, listify, extract_as_list, has_path_sep, path_is_in_root, PerMachine) from ..options import OptionKey -from ..programs import ExternalProgram, NonExistingExternalProgram +from ..programs import ExternalProgram, NonExistingExternalProgram, Program from ..dependencies import Dependency from ..depfile import DepFile from ..interpreterbase import ContainerTypeInfo, InterpreterBase, KwargInfo, typed_kwargs, typed_pos_args @@ -119,7 +119,6 @@ from ..backend.backends import Backend from ..interpreterbase.baseobjects import InterpreterObject, TYPE_var, TYPE_kwargs from ..options import OptionDict - from ..programs import OverrideProgram from .type_checking import SourcesVarargsType # Input source types passed to Targets @@ -131,7 +130,7 @@ BuildTargetSource = T.Union[mesonlib.FileOrString, build.GeneratedTypes, build.StructuredSources] - ProgramVersionFunc = T.Callable[[T.Union[ExternalProgram, build.Executable, OverrideProgram]], str] + ProgramVersionFunc = T.Callable[[Program], str] TestClass = T.TypeVar('TestClass', bound=Test) @@ -287,6 +286,7 @@ def __init__( self.build_func_dict() self.build_holder_map() self.user_defined_options = user_defined_options + self.find_overrides: T.Dict[str, Program] = {} self.compilers: PerMachine[T.Dict[str, 'compilers.Compiler']] = PerMachine({}, {}) self.parse_project() self._redetect_machines() @@ -419,7 +419,6 @@ def build_holder_map(self) -> None: build.Generator: OBJ.GeneratorHolder, build.GeneratedList: OBJ.GeneratedListHolder, build.ExtractedObjects: OBJ.GeneratedObjectsHolder, - build.OverrideExecutable: OBJ.OverrideExecutableHolder, build.RunTarget: OBJ.RunTargetHolder, build.AliasTarget: OBJ.AliasTargetHolder, build.Headers: OBJ.HeadersHolder, @@ -449,7 +448,7 @@ def build_holder_map(self) -> None: ''' self.bound_holder_map.update({ dependencies.Dependency: OBJ.DependencyHolder, - ExternalProgram: OBJ.ExternalProgramHolder, + Program: OBJ.ProgramHolder, compilers.Compiler: compilerOBJ.CompilerHolder, ModuleObject: OBJ.ModuleObjectHolder, MutableModuleObject: OBJ.MutableModuleObjectHolder, @@ -747,8 +746,8 @@ def validate_arguments(self, args, argcount, arg_types): # better error messages when overridden @typed_pos_args( 'run_command', - (build.Executable, ExternalProgram, compilers.Compiler, mesonlib.File, str), - varargs=(build.Executable, ExternalProgram, compilers.Compiler, mesonlib.File, str)) + (build.Executable, Program, compilers.Compiler, mesonlib.File, str), + varargs=(build.Executable, Program, compilers.Compiler, mesonlib.File, str)) @typed_kwargs( 'run_command', KwargInfo('check', (bool, NoneType), since='0.47.0'), @@ -756,14 +755,21 @@ def validate_arguments(self, args, argcount, arg_types): ENV_KW.evolve(since='0.50.0'), ) def func_run_command(self, node: mparser.BaseNode, - args: T.Tuple[T.Union[build.Executable, ExternalProgram, compilers.Compiler, mesonlib.File, str], - T.List[T.Union[build.Executable, ExternalProgram, compilers.Compiler, mesonlib.File, str]]], + args: T.Tuple[T.Union[build.Executable, Program, compilers.Compiler, mesonlib.File, str], + T.List[T.Union[build.Executable, Program, compilers.Compiler, mesonlib.File, str]]], kwargs: 'kwtypes.RunCommand') -> RunProcess: return self.run_command_impl(args, kwargs) + def _compiled_exe_error(self, cmd: T.Union[Program, build.Executable]) -> T.NoReturn: + descr = cmd.name if isinstance(cmd, build.Executable) else cmd.description() + for name, exe in self.find_overrides.items(): + if cmd == exe: + raise InterpreterException(f'Program {name!r} was overridden with the compiled executable {descr!r} and therefore cannot be used during configuration') + raise InterpreterException(f'Program {descr!r} is a compiled executable and therefore cannot be used during configuration') + def run_command_impl(self, - args: T.Tuple[T.Union[build.Executable, ExternalProgram, compilers.Compiler, mesonlib.File, str], - T.List[T.Union[build.Executable, ExternalProgram, compilers.Compiler, mesonlib.File, str]]], + args: T.Tuple[T.Union[build.Executable, Program, compilers.Compiler, mesonlib.File, str], + T.List[T.Union[build.Executable, Program, compilers.Compiler, mesonlib.File, str]]], kwargs: 'kwtypes.RunCommand', in_builddir: bool = False) -> RunProcess: cmd, cargs = args @@ -777,21 +783,14 @@ def run_command_impl(self, mlog.warning(implicit_check_false_warning, once=True) check = False - overridden_msg = ('Program {!r} was overridden with the compiled ' - 'executable {!r} and therefore cannot be used during ' - 'configuration') expanded_args: T.List[str] = [] if isinstance(cmd, build.Executable): - for name, exe in self.build.find_overrides.items(): - if cmd == exe: - progname = name - break - else: - raise InterpreterException(f'Program {cmd.description()!r} is a compiled executable and therefore cannot be used during configuration') - raise InterpreterException(overridden_msg.format(progname, cmd.description())) - if isinstance(cmd, ExternalProgram): + self._compiled_exe_error(cmd) + elif isinstance(cmd, Program): if not cmd.found(): raise InterpreterException(f'command {cmd.get_name()!r} not found or not executable') + if not cmd.runnable(): + self._compiled_exe_error(cmd) elif isinstance(cmd, compilers.Compiler): expanded_args = cmd.get_exe_args() cmd = cmd.get_exe() @@ -813,7 +812,11 @@ def run_command_impl(self, expanded_args.append(a) elif isinstance(a, mesonlib.File): expanded_args.append(a.absolute_path(srcdir, builddir)) - elif isinstance(a, ExternalProgram): + elif isinstance(a, Program): + if not a.found(): + raise InterpreterException(f'command {cmd.get_name()!r} not found or not executable') + if not a.runnable(): + self._compiled_exe_error(a) expanded_args.append(a.get_path()) elif isinstance(a, compilers.Compiler): FeatureNew.single_use('Compiler object as a variadic argument to `run_command`', '0.61.0', self.subproject, location=self.current_node) @@ -822,7 +825,7 @@ def run_command_impl(self, raise InterpreterException(f'Program {cmd!r} not found or not executable') expanded_args.append(prog.get_path()) else: - raise InterpreterException(overridden_msg.format(a.name, cmd.description())) + self._compiled_exe_error(a) # If any file that was used as an argument to the command # changes, we must re-run the configuration step. @@ -993,6 +996,7 @@ def _do_subproject_meson(self, subp_name: str, subdir: str, subi.holder_map = self.holder_map subi.bound_holder_map = self.bound_holder_map subi.summary = self.summary + subi.find_overrides = self.find_overrides subi.subproject_stack = self.subproject_stack + [subp_name] current_active = self.active_projectname @@ -1585,12 +1589,12 @@ def program_from_system(self, args: T.List[mesonlib.FileOrString], search_dirs: def program_from_overrides(self, command_names: T.List[mesonlib.FileOrString], extra_info: T.List['mlog.TV_Loggable'] - ) -> T.Optional[T.Union[ExternalProgram, OverrideProgram, build.OverrideExecutable]]: + ) -> T.Optional[Program]: for name in command_names: if not isinstance(name, str): continue - if name in self.build.find_overrides: - exe = self.build.find_overrides[name] + if name in self.find_overrides: + exe = self.find_overrides[name] extra_info.append(mlog.blue('(overridden)')) return exe return None @@ -1600,12 +1604,12 @@ def store_name_lookups(self, command_names: T.List[mesonlib.FileOrString]) -> No if isinstance(name, str): self.build.searched_programs.add(name) - def add_find_program_override(self, name: str, exe: T.Union[build.OverrideExecutable, ExternalProgram, 'OverrideProgram']) -> None: + def add_find_program_override(self, name: str, exe: Program) -> None: if name in self.build.searched_programs: raise InterpreterException(f'Tried to override finding of executable "{name}" which has already been found.') - if name in self.build.find_overrides: + if name in self.find_overrides: raise InterpreterException(f'Tried to override executable "{name}" which has already been overridden.') - self.build.find_overrides[name] = exe + self.find_overrides[name] = exe if name == 'pkg-config' and isinstance(exe, ExternalProgram): from ..dependencies.pkgconfig import PkgConfigInterface PkgConfigInterface.set_program_override(exe, MachineChoice.HOST) @@ -1625,7 +1629,7 @@ def find_program_impl(self, args: T.List[mesonlib.FileOrString], search_dirs: T.Optional[T.List[str]] = None, version_arg: T.Optional[str] = '', version_func: T.Optional[ProgramVersionFunc] = None - ) -> T.Union['ExternalProgram', 'build.OverrideExecutable', 'OverrideProgram']: + ) -> Program: args = mesonlib.listify(args) extra_info: T.List[mlog.TV_Loggable] = [] @@ -1645,8 +1649,6 @@ def find_program_impl(self, args: T.List[mesonlib.FileOrString], self.store_name_lookups(args) if not silent: mlog.log('Program', mlog.bold(progobj.name), 'found:', mlog.green('YES'), *extra_info) - if isinstance(progobj, build.Executable): - progobj.was_returned_by_find_program = True return progobj def program_lookup(self, args: T.List[mesonlib.FileOrString], for_machine: MachineChoice, @@ -1657,7 +1659,7 @@ def program_lookup(self, args: T.List[mesonlib.FileOrString], for_machine: Machi version_arg: T.Optional[str], version_func: T.Optional[ProgramVersionFunc], extra_info: T.List[mlog.TV_Loggable] - ) -> T.Optional[T.Union[ExternalProgram, build.Executable, OverrideProgram]]: + ) -> T.Optional[Program]: progobj = self.program_from_overrides(args, extra_info) if progobj: return progobj @@ -1693,7 +1695,7 @@ def program_lookup(self, args: T.List[mesonlib.FileOrString], for_machine: Machi return progobj - def check_program_version(self, progobj: T.Union[ExternalProgram, build.Executable, OverrideProgram], + def check_program_version(self, progobj: Program, wanted: T.Union[str, T.List[str]], version_func: T.Optional[ProgramVersionFunc], extra_info: T.List[mlog.TV_Loggable]) -> bool: @@ -1720,7 +1722,7 @@ def check_program_version(self, progobj: T.Union[ExternalProgram, build.Executab def find_program_fallback(self, fallback: str, args: T.List[mesonlib.FileOrString], default_options: OptionDict, required: bool, extra_info: T.List[mlog.TV_Loggable] - ) -> T.Optional[T.Union[ExternalProgram, build.Executable, OverrideProgram]]: + ) -> T.Optional[Program]: mlog.log('Fallback to subproject', mlog.bold(fallback), 'which provides program', mlog.bold(' '.join(args))) sp_kwargs: kwtypes.DoSubproject = { @@ -1747,7 +1749,7 @@ def find_program_fallback(self, fallback: str, args: T.List[mesonlib.FileOrStrin @disablerIfNotFound def func_find_program(self, node: mparser.BaseNode, args: T.Tuple[T.List[mesonlib.FileOrString]], kwargs: 'kwtypes.FindProgram', - ) -> T.Union['build.Executable', ExternalProgram, 'OverrideProgram']: + ) -> Program: disabled, required, feature = extract_required_kwarg(kwargs, self.subproject) if disabled: mlog.log('Program', mlog.bold(' '.join(args[0])), 'skipped: feature', mlog.bold(feature), 'disabled') @@ -1923,7 +1925,7 @@ def func_build_target(self, node: mparser.BaseNode, # Cannot use the COMMAND_KW because command is allowed to be empty KwargInfo( 'command', - ContainerTypeInfo(list, (str, build.BuildTarget, build.CustomTarget, build.CustomTargetIndex, ExternalProgram, mesonlib.File)), + ContainerTypeInfo(list, (str, build.BuildTarget, build.CustomTarget, build.CustomTargetIndex, Program, mesonlib.File)), listify=True, default=[], ), @@ -1950,7 +1952,7 @@ def func_vcs_tag(self, node: mparser.BaseNode, args: T.List['TYPE_var'], kwargs: if maincmd.found(): vcs_cmd[0] = maincmd else: - FeatureNew.single_use('vcs_tag with custom_tgt, external_program, or exe as the first argument', '0.63.0', self.subproject, location=node) + FeatureNew.single_use('vcs_tag with custom_tgt, program, or exe as the first argument', '0.63.0', self.subproject, location=node) else: vcs = mesonlib.detect_vcs(source_dir) if vcs: @@ -2183,7 +2185,7 @@ def func_alias_target(self, node: mparser.BaseNode, args: T.Tuple[str, T.List[T. self.add_target(name, tg) return tg - @typed_pos_args('generator', (build.Executable, ExternalProgram)) + @typed_pos_args('generator', (build.Executable, Program)) @typed_kwargs( 'generator', KwargInfo('arguments', ContainerTypeInfo(list, str, allow_empty=False), required=True, listify=True), @@ -2193,7 +2195,7 @@ def func_alias_target(self, node: mparser.BaseNode, args: T.Tuple[str, T.List[T. KwargInfo('capture', bool, default=False, since='0.43.0'), ) def func_generator(self, node: mparser.FunctionNode, - args: T.Tuple[T.Union[build.Executable, ExternalProgram]], + args: T.Tuple[T.Union[build.Executable, Program]], kwargs: 'kwtypes.FuncGenerator') -> build.Generator: for rule in kwargs['output']: if '@BASENAME@' not in rule and '@PLAINNAME@' not in rule: @@ -2205,19 +2207,23 @@ def func_generator(self, node: mparser.FunctionNode, if '@OUTPUT@' in o: raise InvalidArguments('Tried to use @OUTPUT@ in a rule with more than one output.') - return build.Generator(self.environment, args[0], **kwargs) + if isinstance(args[0], build.Executable): + exe_prg = build.LocalProgram(args[0], self.project_version) + else: + exe_prg = args[0] + return build.Generator(self.environment, exe_prg, **kwargs) - @typed_pos_args('benchmark', str, (build.Executable, build.Jar, ExternalProgram, mesonlib.File, build.CustomTarget, build.CustomTargetIndex)) + @typed_pos_args('benchmark', str, (build.Executable, build.Jar, Program, mesonlib.File, build.CustomTarget, build.CustomTargetIndex)) @typed_kwargs('benchmark', *TEST_KWS) def func_benchmark(self, node: mparser.BaseNode, - args: T.Tuple[str, T.Union[build.Executable, build.Jar, ExternalProgram, mesonlib.File]], + args: T.Tuple[str, T.Union[build.Executable, build.Jar, Program, mesonlib.File]], kwargs: 'kwtypes.FuncBenchmark') -> None: self.add_test(node, args, kwargs, False) - @typed_pos_args('test', str, (build.Executable, build.Jar, ExternalProgram, mesonlib.File, build.CustomTarget, build.CustomTargetIndex)) + @typed_pos_args('test', str, (build.Executable, build.Jar, Program, mesonlib.File, build.CustomTarget, build.CustomTargetIndex)) @typed_kwargs('test', *TEST_KWS, KwargInfo('is_parallel', bool, default=True)) def func_test(self, node: mparser.BaseNode, - args: T.Tuple[str, T.Union[build.Executable, build.Jar, ExternalProgram, mesonlib.File, build.CustomTarget, build.CustomTargetIndex]], + args: T.Tuple[str, T.Union[build.Executable, build.Jar, Program, mesonlib.File, build.CustomTarget, build.CustomTargetIndex]], kwargs: 'kwtypes.FuncTest') -> None: self.add_test(node, args, kwargs, True) @@ -2231,7 +2237,7 @@ def unpack_env_kwarg(self, kwargs: T.Union[EnvironmentVariables, T.Dict[str, 'TY return ENV_KW.convertor(envlist) def make_test(self, node: mparser.BaseNode, - args: T.Tuple[str, T.Union[build.Executable, build.Jar, ExternalProgram, mesonlib.File, build.CustomTarget, build.CustomTargetIndex]], + args: T.Tuple[str, T.Union[build.Executable, build.Jar, Program, mesonlib.File, build.CustomTarget, build.CustomTargetIndex]], kwargs: 'kwtypes.BaseTest', klass: T.Type[TestClass] = Test) -> TestClass: name = args[0] @@ -2240,12 +2246,15 @@ def make_test(self, node: mparser.BaseNode, location=node) name = name.replace(':', '_') exe = args[1] - if isinstance(exe, ExternalProgram): + if isinstance(exe, Program): if not exe.found(): raise InvalidArguments('Tried to use not-found external program as test exe') + if isinstance(exe, build.LocalProgram): + # This will add exe to 'depends' below + exe = exe.program elif isinstance(exe, mesonlib.File): exe = self.find_program_impl([exe]) - elif isinstance(exe, build.CustomTarget): + if isinstance(exe, (build.Executable, build.CustomTarget)): kwargs.setdefault('depends', []).append(exe) elif isinstance(exe, build.CustomTargetIndex): kwargs.setdefault('depends', []).append(exe.target) @@ -2284,7 +2293,7 @@ def add_test(self, node: mparser.BaseNode, if isinstance(args[1], (build.CustomTarget, build.CustomTargetIndex)): FeatureNew.single_use('test with CustomTarget as command', '1.4.0', self.subproject) if any(isinstance(i, ExternalProgram) for i in kwargs['args']): - FeatureNew.single_use('test with external_program in args', '1.6.0', self.subproject) + FeatureNew.single_use('test with program in args', '1.6.0', self.subproject) t = self.make_test(node, args, kwargs) if is_base_test: @@ -2604,7 +2613,7 @@ def validate_build_subdir(self, build_subdir: str, target: str): KwargInfo('capture', bool, default=False, since='0.41.0'), KwargInfo( 'command', - (ContainerTypeInfo(list, (build.Executable, ExternalProgram, compilers.Compiler, mesonlib.File, str), allow_empty=False), NoneType), + (ContainerTypeInfo(list, (build.Executable, Program, compilers.Compiler, mesonlib.File, str), allow_empty=False), NoneType), listify=True, ), KwargInfo( diff --git a/mesonbuild/interpreter/interpreterobjects.py b/mesonbuild/interpreter/interpreterobjects.py index 3bdbcb93eab8..fe5c1de9dd50 100644 --- a/mesonbuild/interpreter/interpreterobjects.py +++ b/mesonbuild/interpreter/interpreterobjects.py @@ -23,7 +23,7 @@ flatten, resolve_second_level_holders, InterpreterException, InvalidArguments, InvalidCode) from ..interpreter.type_checking import NoneType, ENV_KW, ENV_SEPARATOR_KW, PKGCONFIG_DEFINE_KW from ..dependencies import Dependency, ExternalLibrary, InternalDependency -from ..programs import ExternalProgram +from ..programs import ExternalProgram, Program from ..mesonlib import HoldableObject, listify, Popen_safe import typing as T @@ -223,7 +223,7 @@ def enable_auto_if_method(self, args: T.Tuple[bool], kwargs: TYPE_kwargs) -> opt class RunProcess(MesonInterpreterObject): def __init__(self, - cmd: ExternalProgram, + cmd: Program, args: T.List[str], env: mesonlib.EnvironmentVariables, source_dir: str, @@ -234,13 +234,11 @@ def __init__(self, check: bool = False, capture: bool = True) -> None: super().__init__() - if not isinstance(cmd, ExternalProgram): - raise AssertionError('BUG: RunProcess must be passed an ExternalProgram') self.capture = capture self.returncode, self.stdout, self.stderr = self.run_command(cmd, args, env, source_dir, build_dir, subdir, mesonintrospect, in_builddir, check) def run_command(self, - cmd: ExternalProgram, + cmd: Program, args: T.List[str], env: mesonlib.EnvironmentVariables, source_dir: str, @@ -633,10 +631,10 @@ def as_shared_method(self, args: T.List[TYPE_var], kwargs: InternalDependencyAsK raise InterpreterException('as_shared method is only supported on declare_dependency() objects') return self.held_object.get_as_shared(kwargs['recursive']) -_EXTPROG = T.TypeVar('_EXTPROG', bound=ExternalProgram) +_PROG = T.TypeVar('_PROG', bound=Program) -class _ExternalProgramHolder(ObjectHolder[_EXTPROG]): - def __init__(self, ep: _EXTPROG, interpreter: 'Interpreter') -> None: +class ProgramHolder(ObjectHolder[_PROG]): + def __init__(self, ep: _PROG, interpreter: 'Interpreter') -> None: super().__init__(ep, interpreter) @noPosargs @@ -647,15 +645,15 @@ def found_method(self, args: T.List[TYPE_var], kwargs: TYPE_kwargs) -> bool: @noPosargs @noKwargs - @FeatureDeprecated('ExternalProgram.path', '0.55.0', - 'use ExternalProgram.full_path() instead') + @FeatureDeprecated('Program.path', '0.55.0', + 'use Program.full_path() instead') @InterpreterObject.method('path') def path_method(self, args: T.List[TYPE_var], kwargs: TYPE_kwargs) -> str: return self._full_path() @noPosargs @noKwargs - @FeatureNew('ExternalProgram.full_path', '0.55.0') + @FeatureNew('Program.full_path', '0.55.0') @InterpreterObject.method('full_path') def full_path_method(self, args: T.List[TYPE_var], kwargs: TYPE_kwargs) -> str: return self._full_path() @@ -663,26 +661,34 @@ def full_path_method(self, args: T.List[TYPE_var], kwargs: TYPE_kwargs) -> str: def _full_path(self) -> str: if not self.found(): raise InterpreterException('Unable to get the path of a not-found external program') + if not self.held_object.runnable(): + assert isinstance(self.held_object, build.LocalProgram) and \ + isinstance(self.held_object.program, (build.BuildTarget, build.CustomTargetIndex)) + return self.interpreter.backend.get_target_filename_abs(self.held_object.program) path = self.held_object.get_path() assert path is not None return path @noPosargs @noKwargs - @FeatureNew('ExternalProgram.cmd_array', '1.10.0') + @FeatureNew('Program.cmd_array', '1.10.0') @InterpreterObject.method('cmd_array') def cmd_array_method(self, args: T.List[TYPE_var], kwargs: TYPE_kwargs) -> T.List[str]: if not self.found(): raise InterpreterException('Unable to get the path of a not-found external program') + if not self.held_object.runnable(): + return [self._full_path()] cmd = self.held_object.get_command() assert cmd is not None return cmd @noPosargs @noKwargs - @FeatureNew('ExternalProgram.version', '0.62.0') + @FeatureNew('Program.version', '0.62.0') @InterpreterObject.method('version') def version_method(self, args: T.List[TYPE_var], kwargs: TYPE_kwargs) -> str: + if isinstance(self.held_object, build.LocalProgram) and isinstance(self.held_object.program, build.Executable): + FeatureNew.single_use('Program.version with an executable', '1.9.0', subproject=self.subproject, location=self.current_node) if not self.found(): raise InterpreterException('Unable to get the version of a not-found external program') try: @@ -693,8 +699,6 @@ def version_method(self, args: T.List[TYPE_var], kwargs: TYPE_kwargs) -> str: def found(self) -> bool: return self.held_object.found() -class ExternalProgramHolder(_ExternalProgramHolder[ExternalProgram]): - pass class ExternalLibraryHolder(ObjectHolder[ExternalLibrary]): def __init__(self, el: ExternalLibrary, interpreter: 'Interpreter'): @@ -955,9 +959,8 @@ def is_cross(self) -> bool: @noPosargs @noKwargs @InterpreterObject.method('found') + @FeatureNew('BuildTarget.found', '0.59.0') def found_method(self, args: T.List[TYPE_var], kwargs: TYPE_kwargs) -> bool: - if not (isinstance(self.held_object, build.Executable) and self.held_object.was_returned_by_find_program): - FeatureNew.single_use('BuildTarget.found', '0.59.0', subproject=self.held_object.subproject) return True @noPosargs @@ -1204,11 +1207,3 @@ class StructuredSourcesHolder(ObjectHolder[build.StructuredSources]): def __init__(self, sources: build.StructuredSources, interp: 'Interpreter'): super().__init__(sources, interp) - -class OverrideExecutableHolder(BuildTargetHolder[build.OverrideExecutable]): - @noPosargs - @noKwargs - @FeatureNew('OverrideExecutable.version', '1.9.0') - @InterpreterObject.method('version') - def version_method(self, args: T.List[TYPE_var], kwargs: TYPE_kwargs) -> str: - return self.held_object.get_version(self.interpreter) diff --git a/mesonbuild/interpreter/kwargs.py b/mesonbuild/interpreter/kwargs.py index 4935973d95d4..506bc83580a7 100644 --- a/mesonbuild/interpreter/kwargs.py +++ b/mesonbuild/interpreter/kwargs.py @@ -16,7 +16,7 @@ from ..mesonlib import EnvironmentVariables, MachineChoice, File, FileMode, FileOrString from ..options import OptionKey from ..modules.cmake import CMakeSubprojectOptions -from ..programs import ExternalProgram +from ..programs import Program, ExternalProgram from .type_checking import PkgConfigDefineType, SourcesVarargsType TestArgs = T.Union[str, File, build.Target, ExternalProgram] @@ -171,7 +171,7 @@ class FuncAddLanguages(ExtractRequired): class RunTarget(TypedDict): - command: T.List[T.Union[str, build.BuildTarget, build.CustomTarget, ExternalProgram, File]] + command: T.List[T.Union[str, build.BuildTarget, build.CustomTarget, Program, File]] depends: T.List[T.Union[build.BuildTarget, build.CustomTarget]] env: EnvironmentVariables @@ -183,7 +183,7 @@ class CustomTarget(TypedDict): build_by_default: T.Optional[bool] build_subdir: str capture: bool - command: T.List[T.Union[str, build.BuildTargetTypes, ExternalProgram, File]] + command: T.List[T.Union[str, build.BuildTargetTypes, Program, File]] console: bool depend_files: T.List[FileOrString] depends: T.List[T.Union[build.BuildTarget, build.CustomTarget]] @@ -282,10 +282,10 @@ class ConfigurationDataSet(TypedDict): class VcsTag(TypedDict): - command: T.List[T.Union[str, build.GeneratedTypes, ExternalProgram, File]] + command: T.List[T.Union[str, build.GeneratedTypes, Program, File]] fallback: T.Optional[str] input: T.List[T.Union[str, build.BuildTarget, build.GeneratedTypes, - build.ExtractedObjects, ExternalProgram, File]] + build.ExtractedObjects, Program, File]] output: T.List[str] replace_string: str install: bool diff --git a/mesonbuild/interpreter/mesonmain.py b/mesonbuild/interpreter/mesonmain.py index 067d5ffb3f30..ec2b10b790a6 100644 --- a/mesonbuild/interpreter/mesonmain.py +++ b/mesonbuild/interpreter/mesonmain.py @@ -14,7 +14,7 @@ from ..mesonlib import MachineChoice from ..options import OptionKey -from ..programs import OverrideProgram, ExternalProgram +from ..programs import Program, ExternalProgram from ..interpreter.type_checking import ENV_KW, ENV_METHOD_KW, ENV_SEPARATOR_KW, env_convertor_with_method from ..interpreterbase import (MesonInterpreterObject, FeatureNew, FeatureDeprecated, typed_pos_args, noArgsFlattening, noPosargs, noKwargs, @@ -58,21 +58,23 @@ def __init__(self, build: 'build.Build', interpreter: 'Interpreter'): self.interpreter = interpreter def _find_source_script( - self, name: str, prog: T.Union[str, mesonlib.File, build.Executable, ExternalProgram], - args: T.List[str]) -> 'ExecutableSerialisation': - largs: T.List[T.Union[str, build.Executable, ExternalProgram]] = [] + self, name: str, prog: T.Union[str, mesonlib.File, build.Executable, Program], + args: T.List[str], *, + allow_built_program: bool = False) -> 'ExecutableSerialisation': + largs: T.List[T.Union[str, build.Executable, Program]] = [] - if isinstance(prog, (build.Executable, ExternalProgram)): + if isinstance(prog, (build.Executable, Program)): FeatureNew.single_use(f'Passing executable/found program object to script parameter of {name}', '0.55.0', self.subproject, location=self.current_node) - largs.append(prog) - else: + if not allow_built_program and not (isinstance(prog, Program) and prog.runnable()): + self.interpreter._compiled_exe_error(prog) + elif isinstance(prog, (str, mesonlib.File)): if isinstance(prog, mesonlib.File): FeatureNew.single_use(f'Passing file object to script parameter of {name}', '0.57.0', self.subproject, location=self.current_node) - found = self.interpreter.find_program_impl([prog]) - largs.append(found) + prog = self.interpreter.find_program_impl([prog]) + largs.append(prog) largs.extend(args) es = self.interpreter.backend.get_executable_serialisation(largs, verbose=True) es.subproject = self.interpreter.subproject @@ -82,7 +84,7 @@ def _process_script_args( self, name: str, args: T.Sequence[T.Union[ str, mesonlib.File, build.BuildTarget, build.CustomTarget, build.CustomTargetIndex, - ExternalProgram, + Program, ]]) -> T.List[str]: script_args = [] # T.List[str] new = False @@ -105,7 +107,7 @@ def _process_script_args( else: a.build_by_default = True else: - script_args.extend(a.command) + script_args.extend(a.get_command()) new = True if new: @@ -117,8 +119,8 @@ def _process_script_args( @typed_pos_args( 'meson.add_install_script', - (str, mesonlib.File, build.Executable, ExternalProgram), - varargs=(str, mesonlib.File, build.BuildTarget, build.CustomTarget, build.CustomTargetIndex, ExternalProgram) + (str, mesonlib.File, build.Executable, Program), + varargs=(str, mesonlib.File, build.BuildTarget, build.CustomTarget, build.CustomTargetIndex, Program) ) @typed_kwargs( 'meson.add_install_script', @@ -129,11 +131,11 @@ def _process_script_args( @InterpreterObject.method('add_install_script') def add_install_script_method( self, - args: T.Tuple[T.Union[str, mesonlib.File, build.Executable, ExternalProgram], - T.List[T.Union[str, mesonlib.File, build.BuildTargetTypes, ExternalProgram]]], + args: T.Tuple[T.Union[str, mesonlib.File, build.Executable, Program], + T.List[T.Union[str, mesonlib.File, build.BuildTargetTypes, Program]]], kwargs: 'AddInstallScriptKW') -> None: script_args = self._process_script_args('add_install_script', args[1]) - script = self._find_source_script('add_install_script', args[0], script_args) + script = self._find_source_script('add_install_script', args[0], script_args, allow_built_program=True) script.skip_if_destdir = kwargs['skip_if_destdir'] script.tag = kwargs['install_tag'] script.dry_run = kwargs['dry_run'] @@ -141,15 +143,15 @@ def add_install_script_method( @typed_pos_args( 'meson.add_postconf_script', - (str, mesonlib.File, ExternalProgram), - varargs=(str, mesonlib.File, ExternalProgram) + (str, mesonlib.File, Program), + varargs=(str, mesonlib.File, Program) ) @noKwargs @InterpreterObject.method('add_postconf_script') def add_postconf_script_method( self, - args: T.Tuple[T.Union[str, mesonlib.File, ExternalProgram], - T.List[T.Union[str, mesonlib.File, ExternalProgram]]], + args: T.Tuple[T.Union[str, mesonlib.File, Program], + T.List[T.Union[str, mesonlib.File, Program]]], kwargs: 'TYPE_kwargs') -> None: script_args = self._process_script_args('add_postconf_script', args[1]) script = self._find_source_script('add_postconf_script', args[0], script_args) @@ -157,16 +159,16 @@ def add_postconf_script_method( @typed_pos_args( 'meson.add_dist_script', - (str, mesonlib.File, ExternalProgram), - varargs=(str, mesonlib.File, ExternalProgram) + (str, mesonlib.File, Program), + varargs=(str, mesonlib.File, Program) ) @noKwargs @FeatureNew('meson.add_dist_script', '0.48.0') @InterpreterObject.method('add_dist_script') def add_dist_script_method( self, - args: T.Tuple[T.Union[str, mesonlib.File, ExternalProgram], - T.List[T.Union[str, mesonlib.File, ExternalProgram]]], + args: T.Tuple[T.Union[str, mesonlib.File, Program], + T.List[T.Union[str, mesonlib.File, Program]]], kwargs: 'TYPE_kwargs') -> None: if args[1]: FeatureNew.single_use('Calling "add_dist_script" with multiple arguments', @@ -313,19 +315,20 @@ def install_dependency_manifest_method(self, args: T.Tuple[str], kwargs: 'TYPE_k self.build.dep_manifest_name = args[0] @FeatureNew('meson.override_find_program', '0.46.0') - @typed_pos_args('meson.override_find_program', str, (mesonlib.File, ExternalProgram, build.Executable)) + @typed_pos_args('meson.override_find_program', str, (mesonlib.File, Program, build.Executable)) @noKwargs @InterpreterObject.method('override_find_program') - def override_find_program_method(self, args: T.Tuple[str, T.Union[mesonlib.File, ExternalProgram, build.Executable]], kwargs: 'TYPE_kwargs') -> None: + def override_find_program_method(self, args: T.Tuple[str, T.Union[mesonlib.File, Program, build.Executable]], kwargs: 'TYPE_kwargs') -> None: name, exe = args if isinstance(exe, mesonlib.File): abspath = exe.absolute_path(self.interpreter.environment.source_dir, self.interpreter.environment.build_dir) if not os.path.exists(abspath): raise InterpreterException(f'Tried to override {name} with a file that does not exist.') - exe = OverrideProgram(name, self.interpreter.project_version, command=[abspath]) + prog = ExternalProgram(name, command=[abspath], silent=True) + exe = build.LocalProgram(prog, self.interpreter.project_version) elif isinstance(exe, build.Executable): - exe = build.OverrideExecutable(exe, self.interpreter.project_version) + exe = build.LocalProgram(exe, self.interpreter.project_version) self.interpreter.add_find_program_override(name, exe) @typed_kwargs( diff --git a/mesonbuild/interpreter/type_checking.py b/mesonbuild/interpreter/type_checking.py index 28280a996538..a302b3cd05bd 100644 --- a/mesonbuild/interpreter/type_checking.py +++ b/mesonbuild/interpreter/type_checking.py @@ -16,7 +16,7 @@ from ..interpreterbase.decorators import KwargInfo, ContainerTypeInfo, FeatureBroken, FeatureDeprecated from ..mesonlib import (File, FileMode, MachineChoice, has_path_sep, listify, stringlistify, EnvironmentVariables) -from ..programs import ExternalProgram +from ..programs import Program, ExternalProgram # Helper definition for type checks that are `Optional[T]` NoneType: T.Type[None] = type(None) @@ -285,9 +285,9 @@ def _env_convertor(value: _FullEnvInitValueType) -> EnvironmentVariables: default=[], ) -COMMAND_KW: KwargInfo[T.List[T.Union[str, BuildTargetTypes, ExternalProgram, File]]] = KwargInfo( +COMMAND_KW: KwargInfo[T.List[T.Union[str, BuildTargetTypes, Program, File]]] = KwargInfo( 'command', - ContainerTypeInfo(list, (str, BuildTarget, CustomTarget, CustomTargetIndex, ExternalProgram, File), allow_empty=False), + ContainerTypeInfo(list, (str, BuildTarget, CustomTarget, CustomTargetIndex, Program, File), allow_empty=False), required=True, listify=True, default=[], @@ -514,7 +514,7 @@ def suite_convertor(suite: T.List[str]) -> T.List[str]: ] TEST_KWS: T.List[KwargInfo] = TEST_KWS_NO_ARGS + [ - KwargInfo('args', ContainerTypeInfo(list, (str, File, BuildTarget, CustomTarget, CustomTargetIndex, ExternalProgram)), + KwargInfo('args', ContainerTypeInfo(list, (str, File, BuildTarget, CustomTarget, CustomTargetIndex, Program)), listify=True, default=[]), ] diff --git a/mesonbuild/modules/__init__.py b/mesonbuild/modules/__init__.py index 3ff9368d907f..dca8cad52592 100644 --- a/mesonbuild/modules/__init__.py +++ b/mesonbuild/modules/__init__.py @@ -18,7 +18,7 @@ from ..interpreter import Interpreter from ..interpreter.interpreter import ProgramVersionFunc from ..interpreterbase import TYPE_var, TYPE_kwargs - from ..programs import OverrideProgram + from ..programs import Program from ..dependencies import Dependency from ..options import ElementaryOptionValues @@ -75,14 +75,14 @@ def find_program(self, prog: T.Union[mesonlib.FileOrString, T.List[mesonlib.File required: bool = True, version_func: T.Optional[ProgramVersionFunc] = None, wanted: T.Union[str, T.List[str]] = '', silent: bool = False, - for_machine: MachineChoice = MachineChoice.HOST) -> T.Union[ExternalProgram, build.OverrideExecutable, OverrideProgram]: + for_machine: MachineChoice = MachineChoice.HOST) -> Program: if not isinstance(prog, list): prog = [prog] return self._interpreter.find_program_impl(prog, required=required, version_func=version_func, wanted=wanted, silent=silent, for_machine=for_machine) def find_tool(self, name: str, depname: str, varname: str, required: bool = True, - wanted: T.Optional[str] = None) -> T.Union[build.OverrideExecutable, ExternalProgram, 'OverrideProgram']: + wanted: T.Optional[str] = None) -> Program: # Look in overrides in case it's built as subproject progobj = self._interpreter.program_from_overrides([name], []) if progobj is not None: @@ -118,7 +118,7 @@ def dependency(self, depname: str, native: bool = False, required: bool = True, # implementations of meson functions anyway. return self._interpreter.func_dependency(self.current_node, [depname], kwargs) # type: ignore - def test(self, args: T.Tuple[str, T.Union[build.Executable, build.Jar, 'ExternalProgram', mesonlib.File]], + def test(self, args: T.Tuple[str, T.Union[build.Executable, build.Jar, Program, mesonlib.File]], workdir: T.Optional[str] = None, env: T.Union[T.List[str], T.Dict[str, str], str] = None, depends: T.List[T.Union[build.CustomTarget, build.BuildTarget]] = None) -> None: diff --git a/mesonbuild/modules/_qt.py b/mesonbuild/modules/_qt.py index b75169e4b2b0..f3575106357e 100644 --- a/mesonbuild/modules/_qt.py +++ b/mesonbuild/modules/_qt.py @@ -28,7 +28,7 @@ from ..interpreter import Interpreter from ..interpreter import kwargs from ..mesonlib import FileOrString - from ..programs import ExternalProgram + from ..programs import CommandList, Program from typing_extensions import Literal QtDependencyType = T.Union[QtPkgConfigDependency, QmakeQtDependency] @@ -209,7 +209,7 @@ def __init__(self, interpreter: Interpreter, qt_version: int = 5): self.qt_version = qt_version # It is important that this list does not change order as the order of # the returned ExternalPrograms will change as well - self.tools: T.Dict[str, T.Union[ExternalProgram, build.Executable]] = { + self.tools: T.Dict[str, Program] = { tool: NonExistingExternalProgram(tool) for tool in self._set_of_qt_tools } self.methods.update({ @@ -251,7 +251,7 @@ def gen_bins() -> T.Generator[T.Tuple[str, str], None, None]: arg = ['-v'] # Ensure that the version of qt and each tool are the same - def get_version(p: T.Union[ExternalProgram, build.Executable]) -> str: + def get_version(p: Program) -> str: _, out, err = Popen_safe(p.get_command() + arg) if name == 'lrelease' or not qt_dep.version.startswith('4'): care = out @@ -441,17 +441,18 @@ def _compile_resources_impl(self, state: 'ModuleState', kwargs: 'ResourceCompile # If a name was set generate a single .cpp file from all of the qrc # files, otherwise generate one .cpp file per qrc file. + cmd: CommandList if name: qrc_deps: T.List[File] = [] for s in sources: qrc_deps.extend(self._parse_qrc_deps(state, s)) - + cmd = [self.tools['rcc'], '-name', name, '-o', '@OUTPUT@', *extra_args, '@INPUT@', *DEPFILE_ARGS] res_target = build.CustomTarget( name, state.subdir, state.subproject, state.environment, - self.tools['rcc'].get_command() + ['-name', name, '-o', '@OUTPUT@'] + extra_args + ['@INPUT@'] + DEPFILE_ARGS, + cmd, sources, [f'{name}.cpp'], depend_files=qrc_deps, @@ -467,12 +468,13 @@ def _compile_resources_impl(self, state: 'ModuleState', kwargs: 'ResourceCompile else: basename = os.path.basename(rcc_file.fname) name = f'qt{self.qt_version}-{basename.replace(".", "_")}' + cmd = [self.tools['rcc'], '-name', '@BASENAME@', '-o', '@OUTPUT@', *extra_args, '@INPUT@', *DEPFILE_ARGS] res_target = build.CustomTarget( name, state.subdir, state.subproject, state.environment, - self.tools['rcc'].get_command() + ['-name', '@BASENAME@', '-o', '@OUTPUT@'] + extra_args + ['@INPUT@'] + DEPFILE_ARGS, + cmd, [rcc_file], [f'{name}.cpp'], depend_files=qrc_deps, @@ -727,7 +729,7 @@ def compile_translations(self, state: ModuleState, args: T.Tuple, kwargs: Compil ts = os.path.basename(ts) else: outdir = state.subdir - cmd: T.List[T.Union[ExternalProgram, build.Executable, str]] = [self.tools['lrelease'], '@INPUT@', '-qm', '@OUTPUT@'] + cmd: CommandList = [self.tools['lrelease'], '@INPUT@', '-qm', '@OUTPUT@'] lrelease_target = build.CustomTarget( f'qt{self.qt_version}-compile-{ts}', outdir, @@ -867,12 +869,13 @@ def _moc_json_collect(self, state: ModuleState, kwargs: MocJsonCollectKwArgs) -> input_args.append(f'@INPUT{input_counter}@') input_counter += 1 + cmd: CommandList = [self.tools['moc'], '--collect-json', '-o', '@OUTPUT@', *input_args] return build.CustomTarget( f'moc_collect_json_{target_name}', state.subdir, state.subproject, state.environment, - self.tools['moc'].get_command() + ['--collect-json', '-o', '@OUTPUT@'] + input_args, + cmd, moc_json, [f'{target_name}_json_collect.json'], description=f'Collecting json type information for {target_name}', @@ -912,12 +915,15 @@ def _gen_qml_cachegen(self, state: ModuleState, kwargs: GenQmlCachegenKwArgs) -> ressource_path = os.path.join('/', kwargs['module_prefix'], source_basename) cachegen_inputs.append(ressource_path) + cmd: CommandList = \ + [self.tools['qmlcachegen'], '-o', '@OUTPUT@', '--resource-name', f'qmlcache_{target_name}', + *kwargs['extra_args'], '--resource=@INPUT@', *cachegen_inputs] cacheloader_target = build.CustomTarget( f'cacheloader_{target_name}', state.subdir, state.subproject, state.environment, - self.tools['qmlcachegen'].get_command() + ['-o', '@OUTPUT@'] + ['--resource-name', f'qmlcache_{target_name}'] + kwargs['extra_args'] + ['--resource=@INPUT@'] + cachegen_inputs, + cmd, [kwargs['qml_qrc']], #output name format matters here [f'{target_name}_qmlcache_loader.cpp'], @@ -945,15 +951,15 @@ def _qml_type_registrar(self, state: ModuleState, kwargs: GenQmlTypeRegistrarKwA install_dir: T.List[T.Union[str, Literal[False]]] = [False] install_tag: T.List[T.Union[str, None]] = [None] - cmd = self.tools['qmltyperegistrar'].get_command() + [ + cmd: CommandList = [ + self.tools['qmltyperegistrar'], '--import-name', import_name, '--major-version', major_version, '--minor-version', minor_version, '-o', '@OUTPUT0@', + *kwargs['extra_args'], ] - cmd.extend(kwargs['extra_args']) - if namespace: cmd.extend(['--namespace', namespace]) diff --git a/mesonbuild/modules/codegen.py b/mesonbuild/modules/codegen.py index f37f964c4255..c3fdff279417 100644 --- a/mesonbuild/modules/codegen.py +++ b/mesonbuild/modules/codegen.py @@ -16,23 +16,21 @@ noPosargs, noKwargs, disablerIfNotFound, InterpreterObject ) from ..mesonlib import File, MesonException, Popen_safe, version_compare -from ..programs import ExternalProgram, NonExistingExternalProgram +from ..programs import Program, ExternalProgram, NonExistingExternalProgram from ..utils.core import HoldableObject from .. import mlog if T.TYPE_CHECKING: - from typing_extensions import Literal, TypeAlias, TypedDict + from typing_extensions import Literal, TypedDict from . import ModuleState from .._typing import ImmutableListProtocol - from ..build import Executable from ..interpreter import Interpreter from ..interpreter.kwargs import ExtractRequired from ..interpreterbase import TYPE_var, TYPE_kwargs from ..mesonlib import MachineChoice - from ..programs import OverrideProgram + from ..programs import CommandList - Program: TypeAlias = T.Union[Executable, ExternalProgram, OverrideProgram] LexImpls = Literal['lex', 'flex', 'reflex', 'win_flex'] YaccImpls = Literal['yacc', 'byacc', 'bison', 'win_bison'] @@ -90,9 +88,8 @@ class _CodeGenerator(HoldableObject): program: Program arguments: ImmutableListProtocol[str] = dataclasses.field(default_factory=list) - def command(self) -> T.List[T.Union[Program, str]]: - return (T.cast('T.List[T.Union[Program, str]]', [self.program]) + - T.cast('T.List[T.Union[Program, str]]', self.arguments)) + def command(self) -> CommandList: + return T.cast('CommandList', [self.program]) + T.cast('CommandList', self.arguments) def found(self) -> bool: return self.program.found() diff --git a/mesonbuild/modules/dlang.py b/mesonbuild/modules/dlang.py index 35ce86be81f1..9ef099d0f164 100644 --- a/mesonbuild/modules/dlang.py +++ b/mesonbuild/modules/dlang.py @@ -22,20 +22,18 @@ from typing_extensions import Literal, TypeAlias from . import ModuleState - from ..build import OverrideExecutable from ..interpreter.interpreter import Interpreter from ..interpreterbase.baseobjects import TYPE_kwargs - from ..programs import ExternalProgram, OverrideProgram + from ..programs import Program - _AnyProgram: TypeAlias = T.Union[OverrideExecutable, ExternalProgram, OverrideProgram] _JSONTypes: TypeAlias = T.Union[str, int, bool, None, T.List['_JSONTypes'], T.Dict[str, '_JSONTypes']] class DlangModule(ExtensionModule): - class_dubbin: T.Union[_AnyProgram, Literal[False], None] = None + class_dubbin: T.Union[Program, Literal[False], None] = None init_dub = False - dubbin: T.Union[_AnyProgram, Literal[False], None] + dubbin: T.Union[Program, Literal[False], None] INFO = ModuleInfo('dlang', '0.48.0') @@ -122,7 +120,7 @@ def _call_dubbin(self, args: T.List[str], env: T.Optional[T.Mapping[str, str]] = p, out = Popen_safe(self.dubbin.get_command() + args, env=env)[0:2] return p.returncode, out.strip() - def check_dub(self, state: ModuleState) -> T.Union[_AnyProgram, Literal[False]]: + def check_dub(self, state: ModuleState) -> T.Union[Program, Literal[False]]: dubbin = state.find_program('dub', silent=True) if dubbin.found(): try: diff --git a/mesonbuild/modules/gnome.py b/mesonbuild/modules/gnome.py index 0f522c106ba6..cabf34c5f939 100644 --- a/mesonbuild/modules/gnome.py +++ b/mesonbuild/modules/gnome.py @@ -22,7 +22,7 @@ from .. import interpreter from .. import mesonlib from .. import mlog -from ..build import CustomTarget, CustomTargetIndex, Executable, GeneratedList, InvalidArguments +from ..build import CustomTarget, CustomTargetIndex, Executable, GeneratedList, InvalidArguments, LocalProgram from ..dependencies import Dependency, InternalDependency from ..dependencies.pkgconfig import PkgConfigDependency, PkgConfigInterface from ..interpreter.type_checking import DEPENDS_KW, DEPEND_FILES_KW, ENV_KW, INSTALL_DIR_KW, INSTALL_KW, NoneType, DEPENDENCY_SOURCES_KW, in_set_validator @@ -33,11 +33,10 @@ MachineChoice, MesonException, OrderedSet, Popen_safe, join_args, quote_arg ) from ..options import OptionKey -from ..programs import OverrideProgram from ..scripts.gettext import read_linguas if T.TYPE_CHECKING: - from typing_extensions import Literal, TypeAlias, TypedDict + from typing_extensions import Literal, TypedDict from . import ModuleState from ..build import BuildTarget @@ -45,7 +44,7 @@ from ..interpreter import Interpreter from ..interpreterbase import TYPE_var, TYPE_kwargs from ..mesonlib import FileOrString - from ..programs import ExternalProgram + from ..programs import Program, CommandList, CommandListEntry class PostInstall(TypedDict): glib_compile_schemas: bool @@ -198,8 +197,6 @@ class MkEnums(_MkEnumsCommon): vtail: T.Optional[str] depends: T.List[T.Union[BuildTarget, CustomTarget, CustomTargetIndex]] - ToolType: TypeAlias = T.Union[Executable, ExternalProgram, OverrideProgram] - # Differs from the CustomTarget version in that it straight defaults to True _BUILD_BY_DEFAULT: KwargInfo[bool] = KwargInfo( @@ -256,8 +253,8 @@ class GnomeModule(ExtensionModule): def __init__(self, interpreter: 'Interpreter') -> None: super().__init__(interpreter) self.gir_dep: T.Optional[Dependency] = None - self.giscanner: T.Optional[ToolType] = None - self.gicompiler: T.Optional[ToolType] = None + self.giscanner: T.Optional[Program] = None + self.gicompiler: T.Optional[Program] = None self.install_glib_compile_schemas = False self.install_gio_querymodules: T.List[str] = [] self.install_gtk_update_icon_cache = False @@ -309,7 +306,7 @@ def _print_gdbus_warning() -> None: once=True, fatal=False) @staticmethod - def _find_tool(state: 'ModuleState', tool: str) -> 'ToolType': + def _find_tool(state: 'ModuleState', tool: str) -> Program: tool_map = { 'gio-querymodules': 'gio-2.0', 'glib-compile-schemas': 'gio-2.0', @@ -398,7 +395,7 @@ def compile_resources(self, state: 'ModuleState', args: T.Tuple[str, 'FileOrStri glib_version = self._get_native_glib_version(state) glib_compile_resources = self._find_tool(state, 'glib-compile-resources') - cmd: T.List[T.Union['ToolType', str]] = [glib_compile_resources, '@INPUT@'] + cmd: CommandList = [glib_compile_resources, '@INPUT@'] source_dirs = kwargs['source_dir'] dependencies = kwargs['dependencies'] @@ -496,7 +493,7 @@ def compile_resources(self, state: 'ModuleState', args: T.Tuple[str, 'FileOrStri raise MesonException('GResource header is installed yet export is not enabled') depfile: T.Optional[str] = None - target_cmd: T.List[T.Union['ToolType', str]] + target_cmd: CommandList if not mesonlib.version_compare(glib_version, gresource_dep_needed_version): # This will eventually go out of sync if dependencies are added target_cmd = cmd @@ -789,8 +786,7 @@ def postconf_hook(self, b: build.Build) -> None: if self.devenv is not None: b.devenv.append(self.devenv) - def _get_gir_dep(self, state: 'ModuleState') -> T.Tuple[Dependency, T.Union[Executable, 'ExternalProgram', 'OverrideProgram'], - T.Union[Executable, 'ExternalProgram', 'OverrideProgram']]: + def _get_gir_dep(self, state: 'ModuleState') -> T.Tuple[Dependency, Program, Program]: if not self.gir_dep: self.gir_dep = state.dependency('gobject-introspection-1.0') self.giscanner = self._find_tool(state, 'g-ir-scanner') @@ -810,7 +806,7 @@ def _giscanner_version_compare(self, state: 'ModuleState', cmp: str) -> bool: @functools.lru_cache(maxsize=None) def _gir_has_option(self, option: str) -> bool: exe = self.giscanner - if isinstance(exe, (Executable, OverrideProgram)): + if isinstance(exe, LocalProgram): # Handle overridden g-ir-scanner assert option in {'--extra-library', '--sources-top-dirs'} return True @@ -971,7 +967,7 @@ def _make_gir_target( self, state: 'ModuleState', girfile: str, - scan_command: T.Sequence[T.Union['FileOrString', Executable, ExternalProgram, OverrideProgram]], + scan_command: T.Sequence[T.Union['FileOrString', Executable, Program]], generated_files: T.Sequence[T.Union[str, mesonlib.File, build.GeneratedTypes]], depends: T.Sequence[T.Union['FileOrString', build.BuildTarget, 'build.GeneratedTypes', build.StructuredSources]], env_flags: T.Sequence[str], @@ -1020,7 +1016,7 @@ def _make_gir_target( @staticmethod def _make_typelib_target(state: 'ModuleState', typelib_output: str, - typelib_cmd: T.Sequence[T.Union[str, Executable, ExternalProgram, CustomTarget]], + typelib_cmd: T.Sequence[T.Union[str, CustomTarget, Program]], generated_files: T.Sequence[T.Union[str, mesonlib.File, build.GeneratedTypes]], kwargs: T.Dict[str, T.Any]) -> TypelibTarget: install = kwargs['install_typelib'] @@ -1194,7 +1190,8 @@ def generate_gir(self, state: 'ModuleState', args: T.Tuple[T.List[T.Union[Execut gir_inc_dirs: T.List[str] = [] - scan_command: T.List[T.Union[str, Executable, 'ExternalProgram', 'OverrideProgram']] = [giscanner] + # Executables are passed as indexes other than 0 in this List + scan_command: T.List[T.Union[CommandListEntry, Executable]] = [giscanner] scan_command += ['--quiet'] scan_command += ['--no-libtool'] scan_command += ['--namespace=' + ns, '--nsversion=' + nsversion] @@ -1247,6 +1244,7 @@ def generate_gir(self, state: 'ModuleState', args: T.Tuple[T.List[T.Union[Execut T.cast('T.Dict[str, T.Any]', kwargs)) typelib_output = f'{ns}-{nsversion}.typelib' + typelib_cmd: T.List[T.Union[str, Program, CustomTarget]] typelib_cmd = [gicompiler, scan_target, '--output', '@OUTPUT@'] typelib_cmd += state.get_include_args(gir_inc_dirs, prefix='--includedir=') @@ -1267,7 +1265,7 @@ def compile_schemas(self, state: 'ModuleState', args: T.List['TYPE_var'], kwargs srcdir = os.path.join(state.build_to_src, state.subdir) outdir = state.subdir - cmd: T.List[T.Union['ToolType', str]] = [self._find_tool(state, 'glib-compile-schemas'), '--targetdir', outdir, srcdir] + cmd: CommandList = [self._find_tool(state, 'glib-compile-schemas'), '--targetdir', outdir, srcdir] if state.subdir == '': targetname = 'gsettings-compile' else: @@ -1347,7 +1345,7 @@ def yelp(self, state: 'ModuleState', args: T.Tuple[str, T.List[str]], kwargs: 'Y pot_file = os.path.join('@SOURCE_ROOT@', state.subdir, 'C', project_id + '.pot') pot_sources = [os.path.join('@SOURCE_ROOT@', state.subdir, 'C', s) for s in sources] - pot_args: T.List[T.Union[ExternalProgram, Executable, OverrideProgram, str]] = [itstool, '-o', pot_file] + pot_args: CommandList = [itstool, '-o', pot_file] pot_args.extend(pot_sources) pottarget = build.RunTarget(f'help-{project_id}-pot', pot_args, [], os.path.join(state.subdir, 'C'), state.subproject, @@ -1379,7 +1377,7 @@ def yelp(self, state: 'ModuleState', args: T.Tuple[str, T.List[str]], kwargs: 'Y targets.append(l_data) po_file = l + '.po' - po_args: T.List[T.Union[ExternalProgram, Executable, OverrideProgram, str]] = [ + po_args: CommandList = [ msgmerge, '-q', '-o', os.path.join('@SOURCE_ROOT@', l_subdir, po_file), os.path.join('@SOURCE_ROOT@', l_subdir, po_file), pot_file] @@ -1644,7 +1642,7 @@ def gdbus_codegen(self, state: 'ModuleState', args: T.Tuple[str, T.Optional[T.Un kwargs: 'GdbusCodegen') -> ModuleReturnValue: namebase = args[0] xml_files: T.List[T.Union['FileOrString', build.GeneratedTypes]] = [args[1]] if args[1] else [] - cmd: T.List[T.Union['ToolType', str]] = [self._find_tool(state, 'gdbus-codegen')] + cmd: CommandList = [self._find_tool(state, 'gdbus-codegen')] cmd.extend(kwargs['extra_args']) # Autocleanup supported? @@ -2049,7 +2047,7 @@ def _make_mkenum_impl( install_dir: T.Optional[T.Sequence[T.Union[str, bool]]] = None, depends: T.Optional[T.Sequence[build.BuildTargetTypes]] = None ) -> build.CustomTarget: - real_cmd: T.List[T.Union[str, 'ToolType']] = [self._find_tool(state, 'glib-mkenums')] + real_cmd: CommandList = [self._find_tool(state, 'glib-mkenums')] real_cmd.extend(cmd) _install_dir = install_dir or state.environment.coredata.optstore.get_value_for(OptionKey('includedir')) assert isinstance(_install_dir, str), 'for mypy' @@ -2095,7 +2093,7 @@ def genmarshal(self, state: 'ModuleState', args: T.Tuple[str], kwargs: 'GenMarsh new_genmarshal = mesonlib.version_compare(self._get_native_glib_version(state), '>= 2.53.3') - cmd: T.List[T.Union['ToolType', str]] = [self._find_tool(state, 'glib-genmarshal'), '--quiet'] + cmd: CommandList = [self._find_tool(state, 'glib-genmarshal'), '--quiet'] if kwargs['prefix']: cmd.extend(['--prefix', kwargs['prefix']]) if kwargs['extra_args']: @@ -2242,8 +2240,7 @@ def generate_vapi(self, state: 'ModuleState', args: T.Tuple[str], kwargs: 'Gener build_dir = os.path.join(state.environment.get_build_dir(), state.subdir) source_dir = os.path.join(state.environment.get_source_dir(), state.subdir) pkg_cmd, vapi_depends, vapi_packages, vapi_includes, packages = self._extract_vapi_packages(state, kwargs['packages']) - cmd: T.List[T.Union[ExternalProgram, Executable, OverrideProgram, str]] - cmd = [state.find_program('vapigen'), '--quiet', f'--library={library}', f'--directory={build_dir}'] + cmd: CommandList = [state.find_program('vapigen'), '--quiet', f'--library={library}', f'--directory={build_dir}'] cmd.extend([f'--vapidir={d}' for d in kwargs['vapi_dirs']]) cmd.extend([f'--metadatadir={d}' for d in kwargs['metadata_dirs']]) cmd.extend([f'--girdir={d}' for d in kwargs['gir_dirs']]) diff --git a/mesonbuild/modules/i18n.py b/mesonbuild/modules/i18n.py index 2d8d04d3ec30..d88f68ac8245 100644 --- a/mesonbuild/modules/i18n.py +++ b/mesonbuild/modules/i18n.py @@ -27,12 +27,13 @@ from ..build import Target from ..interpreter import Interpreter from ..interpreterbase import TYPE_var + from ..programs import Program class MergeFile(TypedDict): input: T.List[T.Union[ str, build.BuildTarget, build.CustomTarget, build.CustomTargetIndex, - build.ExtractedObjects, build.GeneratedList, ExternalProgram, + build.ExtractedObjects, build.GeneratedList, Program, mesonlib.File]] output: str build_by_default: bool @@ -57,7 +58,7 @@ class ItsJoinFile(TypedDict): input: T.List[T.Union[ str, build.BuildTarget, build.GeneratedTypes, - build.ExtractedObjects, ExternalProgram, mesonlib.File]] + build.ExtractedObjects, Program, mesonlib.File]] output: str build_by_default: bool install: bool @@ -259,7 +260,7 @@ def __init__(self, interpreter: 'Interpreter'): 'itstool_join': self.itstool_join, 'xgettext': self.xgettext, }) - self.tools: T.Dict[str, T.Optional[T.Union[ExternalProgram, build.Executable]]] = { + self.tools: T.Dict[str, T.Optional[Program]] = { 'itstool': None, 'msgfmt': None, 'msginit': None, @@ -308,7 +309,7 @@ def merge_file(self, state: 'ModuleState', args: T.List['TYPE_var'], kwargs: 'Me ddirs = self._get_data_dirs(state, kwargs['data_dirs']) datadirs = '--datadirs=' + ':'.join(ddirs) if ddirs else None - command: T.List[T.Union[str, build.BuildTargetTypes, ExternalProgram, mesonlib.File]] = [] + command: T.List[T.Union[str, build.BuildTargetTypes, Program, mesonlib.File]] = [] command.extend(state.environment.get_build_command()) command.extend([ '--internal', 'msgfmthelper', @@ -487,7 +488,7 @@ def itstool_join(self, state: 'ModuleState', args: T.List['TYPE_var'], kwargs: ' for target in mo_targets: mo_fnames.append(path.join(target.get_subdir(), target.get_outputs()[0])) - command: T.List[T.Union[str, build.BuildTargetTypes, ExternalProgram, mesonlib.File]] = [] + command: T.List[T.Union[str, build.BuildTargetTypes, Program, mesonlib.File]] = [] command.extend(state.environment.get_build_command()) itstool_cmd = self.tools['itstool'].get_command() diff --git a/mesonbuild/modules/icestorm.py b/mesonbuild/modules/icestorm.py index 18bf0e2022c8..c89acd38b08b 100644 --- a/mesonbuild/modules/icestorm.py +++ b/mesonbuild/modules/icestorm.py @@ -16,7 +16,7 @@ from . import ModuleState from ..interpreter import Interpreter - from ..programs import ExternalProgram + from ..programs import Program class ProjectKwargs(TypedDict): @@ -29,7 +29,7 @@ class IceStormModule(ExtensionModule): def __init__(self, interpreter: Interpreter) -> None: super().__init__(interpreter) - self.tools: T.Dict[str, T.Union[ExternalProgram, build.Executable]] = {} + self.tools: T.Dict[str, Program] = {} self.methods.update({ 'project': self.project, }) diff --git a/mesonbuild/modules/python.py b/mesonbuild/modules/python.py index 99602c05ad4c..c6dabdf48040 100644 --- a/mesonbuild/modules/python.py +++ b/mesonbuild/modules/python.py @@ -15,7 +15,7 @@ from ..dependencies.detect import get_dep_identifier, find_external_dependency from ..dependencies.python import BasicPythonExternalProgram, python_factory, _PythonDependencyBase from ..interpreter import extract_required_kwarg, primitives as P_OBJ -from ..interpreter.interpreterobjects import _ExternalProgramHolder +from ..interpreter.interpreterobjects import ProgramHolder from ..interpreter.type_checking import NoneType, DEPENDENCY_KWS, PRESERVE_PATH_KW, SHARED_MOD_KWS from ..interpreterbase import ( noPosargs, noKwargs, permittedKwargs, ContainerTypeInfo, @@ -109,9 +109,9 @@ def _get_path(self, state: T.Optional['ModuleState'], key: str) -> str: _LIMITED_API_KW = KwargInfo('limited_api', str, default='', since='1.3.0') _DEFAULTABLE_SUBDIR_KW = KwargInfo('subdir', (str, NoneType)) -class PythonInstallation(_ExternalProgramHolder['PythonExternalProgram']): +class PythonInstallation(ProgramHolder['PythonExternalProgram']): def __init__(self, python: 'PythonExternalProgram', interpreter: 'Interpreter'): - _ExternalProgramHolder.__init__(self, python, interpreter) + ProgramHolder.__init__(self, python, interpreter) info = python.info prefix = self.interpreter.environment.coredata.optstore.get_value_for(OptionKey('prefix')) assert isinstance(prefix, str), 'for mypy' diff --git a/mesonbuild/modules/rust.py b/mesonbuild/modules/rust.py index a8fcc86a06b6..6d6a74648ff3 100644 --- a/mesonbuild/modules/rust.py +++ b/mesonbuild/modules/rust.py @@ -33,7 +33,7 @@ from ..interpreter import kwargs as _kwargs from ..interpreter.interpreter import SourceInputs, SourceOutputs from ..interpreter.interpreterobjects import Test - from ..programs import OverrideProgram + from ..programs import Program from ..interpreter.type_checking import SourcesVarargsType from typing_extensions import TypedDict, Literal @@ -91,7 +91,7 @@ class RustModule(ExtensionModule): def __init__(self, interpreter: Interpreter) -> None: super().__init__(interpreter) - self._bindgen_bin: T.Optional[T.Union[ExternalProgram, Executable, OverrideProgram]] = None + self._bindgen_bin: T.Optional[Program] = None if 'rust' in interpreter.compilers.host: rustc = T.cast('RustCompiler', interpreter.compilers.host['rust']) self._bindgen_rust_target = 'nightly' if rustc.is_nightly else rustc.version @@ -375,10 +375,7 @@ def bindgen(self, state: ModuleState, args: T.List, kwargs: FuncBindgen) -> Modu if self._bindgen_bin is None: self._bindgen_bin = state.find_program('bindgen', wanted=kwargs['bindgen_version']) if self._bindgen_rust_target is not None: - # ExternalCommand.command's type is bonkers - _, _, err = mesonlib.Popen_safe( - T.cast('T.List[str]', self._bindgen_bin.get_command()) + - ['--rust-target', self._bindgen_rust_target]) + _, _, err = mesonlib.Popen_safe(self._bindgen_bin.get_command() + ['--rust-target', self._bindgen_rust_target]) # < 0.71: Sometimes this is "invalid Rust target" and # sometimes "invalid # rust target" # >= 0.71: error: invalid value '...' for '--rust-target ': "..." is not a valid Rust target, accepted values are of the form ... @@ -386,9 +383,7 @@ def bindgen(self, state: ModuleState, args: T.List, kwargs: FuncBindgen) -> Modu if 'Got an invalid' in err or 'is not a valid Rust target' in err: self._bindgen_rust_target = None - # TODO: Executable needs to learn about get_version - if isinstance(self._bindgen_bin, ExternalProgram): - self._bindgen_set_std = mesonlib.version_compare(self._bindgen_bin.get_version(), '>= 0.71') + self._bindgen_set_std = mesonlib.version_compare(self._bindgen_bin.get_version(), '>= 0.71') name: str if isinstance(header, File): diff --git a/mesonbuild/modules/wayland.py b/mesonbuild/modules/wayland.py index 94c6f819db5e..e77a2b13b309 100644 --- a/mesonbuild/modules/wayland.py +++ b/mesonbuild/modules/wayland.py @@ -15,10 +15,9 @@ from typing_extensions import Literal, TypedDict from . import ModuleState - from ..build import Executable from ..dependencies import Dependency from ..interpreter import Interpreter - from ..programs import ExternalProgram + from ..programs import CommandList, Program from ..mesonlib import FileOrString class ScanXML(TypedDict): @@ -42,7 +41,7 @@ def __init__(self, interpreter: Interpreter) -> None: self.protocols_dep: T.Optional[Dependency] = None self.pkgdatadir: T.Optional[str] = None - self.scanner_bin: T.Optional[T.Union[ExternalProgram, Executable]] = None + self.scanner_bin: T.Optional[Program] = None self.methods.update({ 'scan_xml': self.scan_xml, @@ -89,7 +88,7 @@ def scan_xml(self, state: ModuleState, args: T.Tuple[T.List[FileOrString]], kwar targets.append(code) for side in sides: - command = [self.scanner_bin, f'{side}-header', '@INPUT@', '@OUTPUT@'] + command: CommandList = [self.scanner_bin, f'{side}-header', '@INPUT@', '@OUTPUT@'] if kwargs['include_core_only']: command.append('--include-core-only') diff --git a/mesonbuild/modules/windows.py b/mesonbuild/modules/windows.py index 3250c072ca9a..aa1dc33f1025 100644 --- a/mesonbuild/modules/windows.py +++ b/mesonbuild/modules/windows.py @@ -23,6 +23,7 @@ from ..compilers import Compiler from ..interpreter import Interpreter from ..interpreter.interpreter import SourceOutputs + from ..programs import CommandList from typing_extensions import TypedDict @@ -32,13 +33,6 @@ class CompileResources(TypedDict): include_directories: T.List[T.Union[str, build.IncludeDirs]] args: T.List[str] - class RcKwargs(TypedDict): - output: str - input: T.List[T.Union[mesonlib.FileOrString, build.CustomTargetIndex]] - depfile: T.Optional[str] - depend_files: T.List[mesonlib.FileOrString] - depends: T.List[T.Union[build.BuildTarget, build.CustomTarget]] - command: T.List[T.Union[str, ExternalProgram]] class ResourceCompilerType(enum.Enum): windres = 1 @@ -208,7 +202,7 @@ def get_names() -> T.Iterable[T.Tuple[str, str, T.Union[str, mesonlib.File, buil name = name.replace('/', '_').replace('\\', '_').replace(':', '_') name_formatted = name_formatted.replace('/', '_').replace('\\', '_').replace(':', '_') output = f'{name}_@BASENAME@.{suffix}' - command: T.List[T.Union[str, ExternalProgram]] = [] + command: CommandList = [] command.append(rescomp) command.extend(res_args) depfile: T.Optional[str] = None diff --git a/mesonbuild/programs.py b/mesonbuild/programs.py index a1cfc06e98a2..fa6db548986c 100644 --- a/mesonbuild/programs.py +++ b/mesonbuild/programs.py @@ -13,17 +13,53 @@ import re import typing as T from pathlib import Path +from abc import ABCMeta, abstractmethod from . import mesonlib from . import mlog from .mesonlib import MachineChoice, OrderedSet if T.TYPE_CHECKING: + from typing_extensions import TypeAlias from .environment import Environment from .interpreter import Interpreter + CommandListEntry: TypeAlias = T.Union[str, 'Program'] + CommandList: TypeAlias = T.List[CommandListEntry] -class ExternalProgram(mesonlib.HoldableObject): + +class Program(mesonlib.HoldableObject, metaclass=ABCMeta): + ''' A base class for LocalProgram and ExternalProgram.''' + + name: str + for_machine = MachineChoice.BUILD + + @abstractmethod + def found(self) -> bool: + pass + + @abstractmethod + def get_version(self, interpreter: T.Optional[Interpreter] = None) -> str: + pass + + @abstractmethod + def get_command(self) -> T.List[str]: + pass + + @abstractmethod + def get_path(self) -> T.Optional[str]: + pass + + @abstractmethod + def runnable(self) -> bool: + pass + + @abstractmethod + def description(self) -> str: + '''Human friendly description of the command''' + + +class ExternalProgram(Program): """A program that is found on the system. :param name: The name of the program @@ -34,7 +70,6 @@ class ExternalProgram(mesonlib.HoldableObject): :param exclude_paths: A list of directories to exclude when searching in PATH""" windows_exts = ('exe', 'msc', 'com', 'bat', 'cmd') - for_machine = MachineChoice.BUILD def __init__(self, name: str, command: T.Optional[T.List[str]] = None, silent: bool = False, search_dirs: T.Optional[T.List[T.Optional[str]]] = None, @@ -333,6 +368,9 @@ def _search(self, name: str, search_dirs: T.List[T.Optional[str]], exclude_paths # all executables whether in PATH or with an absolute path return [command] + def runnable(self) -> bool: + return self.found() + def found(self) -> bool: return self.command[0] is not None @@ -362,17 +400,6 @@ def found(self) -> bool: return False -class OverrideProgram(ExternalProgram): - - """A script overriding a program.""" - - def __init__(self, name: str, version: str, command: T.Optional[T.List[str]] = None, - silent: bool = False, search_dirs: T.Optional[T.List[T.Optional[str]]] = None, - exclude_paths: T.Optional[T.List[str]] = None): - super().__init__(name, command=command, silent=silent, - search_dirs=search_dirs, exclude_paths=exclude_paths) - self.cached_version = version - def find_external_program(env: 'Environment', for_machine: MachineChoice, name: str, display_name: str, default_names: T.List[str], allow_default_for_cross: bool = True, diff --git a/mesonbuild/utils/universal.py b/mesonbuild/utils/universal.py index 467d4f5d54c6..4c4cd8355ec0 100644 --- a/mesonbuild/utils/universal.py +++ b/mesonbuild/utils/universal.py @@ -1808,7 +1808,7 @@ def Popen_safe_logged(args: T.List[str], msg: str = 'Called', **kwargs: T.Any) - return p, o, e -def iter_regexin_iter(regexiter: T.Iterable[str], initer: T.Iterable[str | programs.ExternalProgram]) -> T.Optional[str]: +def iter_regexin_iter(regexiter: T.Iterable[str], initer: T.Iterable[str | programs.Program]) -> T.Optional[str]: ''' Takes each regular expression in @regexiter and tries to search for it in every item in @initer. If there is a match, returns that match. @@ -1824,7 +1824,7 @@ def iter_regexin_iter(regexiter: T.Iterable[str], initer: T.Iterable[str | progr return None -def _substitute_values_check_errors(command: T.List[str | programs.ExternalProgram], values: T.Dict[str, T.Union[str, T.List[str]]]) -> None: +def _substitute_values_check_errors(command: T.List[str | programs.Program], values: T.Dict[str, T.Union[str, T.List[str]]]) -> None: # Error checking inregex: T.List[str] = ['@INPUT([0-9]+)?@', '@PLAINNAME@', '@BASENAME@'] outregex: T.List[str] = ['@OUTPUT([0-9]+)?@', '@OUTDIR@'] @@ -1864,7 +1864,7 @@ def _substitute_values_check_errors(command: T.List[str | programs.ExternalProgr raise MesonException(m.format(match2.group(), len(values['@OUTPUT@']))) -def substitute_values(command: T.List[str | programs.ExternalProgram], values: T.Dict[str, T.Union[str, T.List[str]]]) -> T.List[str | programs.ExternalProgram]: +def substitute_values(command: T.List[str | programs.Program], values: T.Dict[str, T.Union[str, T.List[str]]]) -> T.List[str | programs.Program]: ''' Substitute the template strings in the @values dict into the list of strings @command and return a new list. For a full list of the templates, @@ -1891,7 +1891,7 @@ def replace(m: T.Match[str]) -> str: _substitute_values_check_errors(command, values) # Substitution - outcmd: T.List[str | programs.ExternalProgram] = [] + outcmd: T.List[str | programs.Program] = [] rx_keys = [re.escape(key) for key in values if key not in ('@INPUT@', '@OUTPUT@')] value_rx = re.compile('|'.join(rx_keys)) if rx_keys else None for vv in command: diff --git a/test cases/failing/111 run_target in test/test.json b/test cases/failing/111 run_target in test/test.json index c0ec0e6ce946..278b168ae4e3 100644 --- a/test cases/failing/111 run_target in test/test.json +++ b/test cases/failing/111 run_target in test/test.json @@ -1,7 +1,7 @@ { "stdout": [ { - "line": "test cases/failing/111 run_target in test/meson.build:7:0: ERROR: test keyword argument 'args' was of type array[RunTarget] but should have been array[str | File | BuildTarget | CustomTarget | CustomTargetIndex | ExternalProgram]" + "line": "test cases/failing/111 run_target in test/meson.build:7:0: ERROR: test keyword argument 'args' was of type array[RunTarget] but should have been array[str | File | BuildTarget | CustomTarget | CustomTargetIndex | Program]" } ] } diff --git a/test cases/failing/112 run_target in add_install_script/test.json b/test cases/failing/112 run_target in add_install_script/test.json index 7753c315b541..22908659e4c0 100644 --- a/test cases/failing/112 run_target in add_install_script/test.json +++ b/test cases/failing/112 run_target in add_install_script/test.json @@ -1,7 +1,7 @@ { "stdout": [ { - "line": "test cases/failing/112 run_target in add_install_script/meson.build:7:6: ERROR: meson.add_install_script argument 2 was of type \"RunTarget\" but should have been one of: \"str\", \"File\", \"BuildTarget\", \"CustomTarget\", \"CustomTargetIndex\", \"ExternalProgram\"" + "line": "test cases/failing/112 run_target in add_install_script/meson.build:7:6: ERROR: meson.add_install_script argument 2 was of type \"RunTarget\" but should have been one of: \"str\", \"File\", \"BuildTarget\", \"CustomTarget\", \"CustomTargetIndex\", \"Program\"" } ] }