🧑‍🌾 `ros2service.ros2service.test.test_cli.test_cli` failing consistently in Kilted

[Crola1702]

Generated by Generative AI

Part of it (using Claude Sonnet 4)

Operating System:

Ubuntu 24.04

ROS version or commit hash:

Kilted (but also flaky on Rolling)

RMW implementation (if applicable):

No response

RMW Configuration (if applicable):

No response

Client library (if applicable):

No response

'ros2 doctor --report' output

No response

Steps to reproduce issue

1. Run a build in https://build.ros2.org/job/Kci__nightly-debug_ubuntu_noble_amd64/
2. See test regression fail

Expected behavior

Test should not fail

Actual behavior

Test is failing

Additional information

Reference build: https://build.ros2.org/job/Kci__nightly-debug_ubuntu_noble_amd64/22/

Test regression: ros2service.ros2service.test.test_cli.test_cli

This is happening on cyclonedds, connextdds and fastrtps.

There are 8 expected services but 57 available services.

The tests expect only the intended test services and their associated parameter services, but the environment contains additional nodes.

This might be a problem of a contamination of the envrionment by problems with test isolation.

Test flakiness:

job_name	last_fail	first_fail	build_count	failure_count	failure_percentage
Jci__nightly-fastrtps-dynamic_ubuntu_noble_amd64	2025-07-09	2025-07-09	4	1	25.0
Rci__nightly-fastrtps_ubuntu_noble_amd64	2025-07-08	2025-07-08	12	1	8.33
Kci__nightly-release_ubuntu_noble_amd64	2025-07-08	2025-06-26	5	5	100.0
Kci__nightly-debug_ubuntu_noble_amd64	2025-07-07	2025-06-25	4	4	100.0
Kci__nightly-connext_ubuntu_noble_amd64	2025-07-07	2025-06-25	5	5	100.0
Rci__nightly-release_ubuntu_noble_amd64	2025-07-04	2025-06-29	12	3	25.0
nightly_linux_release	2025-06-25	2025-06-25	14	1	7.14
nightly_linux_debug	2025-06-25	2025-06-25	14	1	7.14
nightly_linux-aarch64_repeated	2025-06-25	2025-06-25	14	1	7.14
nightly_linux-aarch64_release	2025-06-25	2025-06-25	14	1	7.14
nightly_linux-aarch64_debug	2025-06-25	2025-06-25	13	1	7.69

Log output

=========================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================
FAIL: test_cli.TestROS2ServiceCLI.test_list_count[rmw_cyclonedds_cpp]
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/ws/install_isolated/launch_testing/lib/python3.12/site-packages/launch_testing/markers.py", line 61, in _wrapper
    return func(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/ws/src/ros2/ros2cli/ros2service/test/test_cli.py", line 210, in test_list_count
    assert int(output_lines[0]) == self.builtin_service_count + 1
AssertionError: assert 57 == (7 + 1)
 +  where 57 = int('57')
 +  and   7 = <test_cli.TestROS2ServiceCLI testMethod=test_list_count[rmw_cyclonedds_cpp]>.builtin_service_count


=========================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================
FAIL: test_cli.TestROS2ServiceCLI.test_list_hidden[rmw_cyclonedds_cpp]
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/ws/install_isolated/launch_testing/lib/python3.12/site-packages/launch_testing/markers.py", line 61, in _wrapper
    return func(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/ws/src/ros2/ros2cli/ros2service/test/test_cli.py", line 178, in test_list_hidden
    assert launch_testing.tools.expect_output(
AssertionError: assert False
 +  where False = <function expect_output at 0x7b81efc5c220>(expected_lines=<itertools.chain object at 0x7b81e5e46050>, text='/array_repeater/describe_parameters\n/array_repeater/get_parameter_types\n/array_repeater/get_parameters\n/array_repe...st_parameters\n/unbounded_sequences_repeater/set_parameters\n/unbounded_sequences_repeater/set_parameters_atomically\n', strict=True)
 +    where <function expect_output at 0x7b81efc5c220> = <module 'launch_testing.tools' from '/tmp/ws/install_isolated/launch_testing/lib/python3.12/site-packages/launch_testing/tools/__init__.py'>.expect_output
 +      where <module 'launch_testing.tools' from '/tmp/ws/install_isolated/launch_testing/lib/python3.12/site-packages/launch_testing/tools/__init__.py'> = launch_testing.tools
 +    and   <itertools.chain object at 0x7b81e5e46050> = <class 'itertools.chain'>(['/my_ns/_echo'], ['/my_ns/_hidden_echo_server/describe_parameters', '/my_ns/_hidden_echo_server/get_parameter_types', '/my_ns/_hidden_e.../get_type_description', '/my_ns/_hidden_echo_server/list_parameters', '/my_ns/_hidden_echo_server/set_parameters', ...], ['/my_ns/echo'], ['/my_ns/echo_server/describe_parameters', '/my_ns/echo_server/get_parameter_types', '/my_ns/echo_server/get_parameter...y_ns/echo_server/get_type_description', '/my_ns/echo_server/list_parameters', '/my_ns/echo_server/set_parameters', ...])
 +      where <class 'itertools.chain'> = itertools.chain
 +      and   ['/my_ns/_hidden_echo_server/describe_parameters', '/my_ns/_hidden_echo_server/get_parameter_types', '/my_ns/_hidden_e.../get_type_description', '/my_ns/_hidden_echo_server/list_parameters', '/my_ns/_hidden_echo_server/set_parameters', ...] = <bound method TestROS2ServiceCLI.get_expected_builtin_services of <class 'test_cli.TestROS2ServiceCLI'>>('/my_ns/_hidden_echo_server')
 +        where <bound method TestROS2ServiceCLI.get_expected_builtin_services of <class 'test_cli.TestROS2ServiceCLI'>> = <test_cli.TestROS2ServiceCLI testMethod=test_list_hidden[rmw_cyclonedds_cpp]>.get_expected_builtin_services
 +      and   ['/my_ns/echo_server/describe_parameters', '/my_ns/echo_server/get_parameter_types', '/my_ns/echo_server/get_parameter...y_ns/echo_server/get_type_description', '/my_ns/echo_server/list_parameters', '/my_ns/echo_server/set_parameters', ...] = <bound method TestROS2ServiceCLI.get_expected_builtin_services of <class 'test_cli.TestROS2ServiceCLI'>>('/my_ns/echo_server')
 +        where <bound method TestROS2ServiceCLI.get_expected_builtin_services of <class 'test_cli.TestROS2ServiceCLI'>> = <test_cli.TestROS2ServiceCLI testMethod=test_list_hidden[rmw_cyclonedds_cpp]>.get_expected_builtin_services
 +    and   '/array_repeater/describe_parameters\n/array_repeater/get_parameter_types\n/array_repeater/get_parameters\n/array_repe...st_parameters\n/unbounded_sequences_repeater/set_parameters\n/unbounded_sequences_repeater/set_parameters_atomically\n' = <launch_testing.tools.process.ProcessProxy object at 0x7b81e5e46e70>.output


=========================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================
FAIL: test_cli.TestROS2ServiceCLI.test_list_services[rmw_cyclonedds_cpp]
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/ws/install_isolated/launch_testing/lib/python3.12/site-packages/launch_testing/markers.py", line 61, in _wrapper
    return func(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/ws/src/ros2/ros2cli/ros2service/test/test_cli.py", line 162, in test_list_services
    assert launch_testing.tools.expect_output(
AssertionError: assert False
 +  where False = <function expect_output at 0x7b81efc5c220>(expected_lines=<itertools.chain object at 0x7b81e5e4e6b0>, text='/array_repeater/describe_parameters\n/array_repeater/get_parameter_types\n/array_repeater/get_parameters\n/array_repe...st_parameters\n/unbounded_sequences_repeater/set_parameters\n/unbounded_sequences_repeater/set_parameters_atomically\n', strict=True)
 +    where <function expect_output at 0x7b81efc5c220> = <module 'launch_testing.tools' from '/tmp/ws/install_isolated/launch_testing/lib/python3.12/site-packages/launch_testing/tools/__init__.py'>.expect_output
 +      where <module 'launch_testing.tools' from '/tmp/ws/install_isolated/launch_testing/lib/python3.12/site-packages/launch_testing/tools/__init__.py'> = launch_testing.tools
 +    and   <itertools.chain object at 0x7b81e5e4e6b0> = <class 'itertools.chain'>(['/my_ns/echo'], ['/my_ns/echo_server/describe_parameters', '/my_ns/echo_server/get_parameter_types', '/my_ns/echo_server/get_parameter...y_ns/echo_server/get_type_description', '/my_ns/echo_server/list_parameters', '/my_ns/echo_server/set_parameters', ...])
 +      where <class 'itertools.chain'> = itertools.chain
 +      and   ['/my_ns/echo_server/describe_parameters', '/my_ns/echo_server/get_parameter_types', '/my_ns/echo_server/get_parameter...y_ns/echo_server/get_type_description', '/my_ns/echo_server/list_parameters', '/my_ns/echo_server/set_parameters', ...] = <bound method TestROS2ServiceCLI.get_expected_builtin_services of <class 'test_cli.TestROS2ServiceCLI'>>('/my_ns/echo_server')
 +        where <bound method TestROS2ServiceCLI.get_expected_builtin_services of <class 'test_cli.TestROS2ServiceCLI'>> = <test_cli.TestROS2ServiceCLI testMethod=test_list_services[rmw_cyclonedds_cpp]>.get_expected_builtin_services
 +    and   '/array_repeater/describe_parameters\n/array_repeater/get_parameter_types\n/array_repeater/get_parameters\n/array_repe...st_parameters\n/unbounded_sequences_repeater/set_parameters\n/unbounded_sequences_repeater/set_parameters_atomically\n' = <launch_testing.tools.process.ProcessProxy object at 0x7b81e5e4f4a0>.output


=========================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================
FAIL: test_cli.TestROS2ServiceCLI.test_list_with_types[rmw_cyclonedds_cpp]
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/ws/install_isolated/launch_testing/lib/python3.12/site-packages/launch_testing/markers.py", line 61, in _wrapper
    return func(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/ws/src/ros2/ros2cli/ros2service/test/test_cli.py", line 194, in test_list_with_types
    assert launch_testing.tools.expect_output(
AssertionError: assert False
 +  where False = <function expect_output at 0x7b81efc5c220>(expected_lines=<itertools.chain object at 0x7b81e5e69f90>, text='/array_repeater/describe_parameters [rcl_interfaces/srv/DescribeParameters]\n/array_repeater/get_parameter_types [rcl...SetParameters]\n/unbounded_sequences_repeater/set_parameters_atomically [rcl_interfaces/srv/SetParametersAtomically]\n', strict=True)
 +    where <function expect_output at 0x7b81efc5c220> = <module 'launch_testing.tools' from '/tmp/ws/install_isolated/launch_testing/lib/python3.12/site-packages/launch_testing/tools/__init__.py'>.expect_output
 +      where <module 'launch_testing.tools' from '/tmp/ws/install_isolated/launch_testing/lib/python3.12/site-packages/launch_testing/tools/__init__.py'> = launch_testing.tools
 +    and   <itertools.chain object at 0x7b81e5e69f90> = <class 'itertools.chain'>(['/my_ns/echo [test_msgs/srv/BasicTypes]'], ['/my_ns/echo_server/describe_parameters [rcl_interfaces/srv/DescribeParameters]', '/my_ns/echo_server/get_parameter_t...ters [rcl_interfaces/srv/ListParameters]', '/my_ns/echo_server/set_parameters [rcl_interfaces/srv/SetParameters]', ...])
 +      where <class 'itertools.chain'> = itertools.chain
 +      and   ['/my_ns/echo_server/describe_parameters [rcl_interfaces/srv/DescribeParameters]', '/my_ns/echo_server/get_parameter_t...ters [rcl_interfaces/srv/ListParameters]', '/my_ns/echo_server/set_parameters [rcl_interfaces/srv/SetParameters]', ...] = <bound method TestROS2ServiceCLI.get_expected_builtin_services of <class 'test_cli.TestROS2ServiceCLI'>>('/my_ns/echo_server', with_types=True)
 +        where <bound method TestROS2ServiceCLI.get_expected_builtin_services of <class 'test_cli.TestROS2ServiceCLI'>> = <test_cli.TestROS2ServiceCLI testMethod=test_list_with_types[rmw_cyclonedds_cpp]>.get_expected_builtin_services
 +    and   '/array_repeater/describe_parameters [rcl_interfaces/srv/DescribeParameters]\n/array_repeater/get_parameter_types [rcl...SetParameters]\n/unbounded_sequences_repeater/set_parameters_atomically [rcl_interfaces/srv/SetParametersAtomically]\n' = <launch_testing.tools.process.ProcessProxy object at 0x7b81e5e6adb0>.output


=========================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================
FAIL: test_cli.TestROS2ServiceCLI.test_list_count[rmw_connextdds]
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/ws/install_isolated/launch_testing/lib/python3.12/site-packages/launch_testing/markers.py", line 61, in _wrapper
    return func(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/ws/src/ros2/ros2cli/ros2service/test/test_cli.py", line 210, in test_list_count
    assert int(output_lines[0]) == self.builtin_service_count + 1
AssertionError: assert 57 == (7 + 1)
 +  where 57 = int('57')
 +  and   7 = <test_cli.TestROS2ServiceCLI testMethod=test_list_count[rmw_connextdds]>.builtin_service_count


=========================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================
FAIL: test_cli.TestROS2ServiceCLI.test_list_hidden[rmw_connextdds]
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/ws/install_isolated/launch_testing/lib/python3.12/site-packages/launch_testing/markers.py", line 61, in _wrapper
    return func(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/ws/src/ros2/ros2cli/ros2service/test/test_cli.py", line 178, in test_list_hidden
    assert launch_testing.tools.expect_output(
AssertionError: assert False
 +  where False = <function expect_output at 0x7b81efc5c220>(expected_lines=<itertools.chain object at 0x7b81e5ce2860>, text='/array_repeater/describe_parameters\n/array_repeater/get_parameter_types\n/array_repeater/get_parameters\n/array_repe...st_parameters\n/unbounded_sequences_repeater/set_parameters\n/unbounded_sequences_repeater/set_parameters_atomically\n', strict=True)
 +    where <function expect_output at 0x7b81efc5c220> = <module 'launch_testing.tools' from '/tmp/ws/install_isolated/launch_testing/lib/python3.12/site-packages/launch_testing/tools/__init__.py'>.expect_output
 +      where <module 'launch_testing.tools' from '/tmp/ws/install_isolated/launch_testing/lib/python3.12/site-packages/launch_testing/tools/__init__.py'> = launch_testing.tools
 +    and   <itertools.chain object at 0x7b81e5ce2860> = <class 'itertools.chain'>(['/my_ns/_echo'], ['/my_ns/_hidden_echo_server/describe_parameters', '/my_ns/_hidden_echo_server/get_parameter_types', '/my_ns/_hidden_e.../get_type_description', '/my_ns/_hidden_echo_server/list_parameters', '/my_ns/_hidden_echo_server/set_parameters', ...], ['/my_ns/echo'], ['/my_ns/echo_server/describe_parameters', '/my_ns/echo_server/get_parameter_types', '/my_ns/echo_server/get_parameter...y_ns/echo_server/get_type_description', '/my_ns/echo_server/list_parameters', '/my_ns/echo_server/set_parameters', ...])
 +      where <class 'itertools.chain'> = itertools.chain
 +      and   ['/my_ns/_hidden_echo_server/describe_parameters', '/my_ns/_hidden_echo_server/get_parameter_types', '/my_ns/_hidden_e.../get_type_description', '/my_ns/_hidden_echo_server/list_parameters', '/my_ns/_hidden_echo_server/set_parameters', ...] = <bound method TestROS2ServiceCLI.get_expected_builtin_services of <class 'test_cli.TestROS2ServiceCLI'>>('/my_ns/_hidden_echo_server')
 +        where <bound method TestROS2ServiceCLI.get_expected_builtin_services of <class 'test_cli.TestROS2ServiceCLI'>> = <test_cli.TestROS2ServiceCLI testMethod=test_list_hidden[rmw_connextdds]>.get_expected_builtin_services
 +      and   ['/my_ns/echo_server/describe_parameters', '/my_ns/echo_server/get_parameter_types', '/my_ns/echo_server/get_parameter...y_ns/echo_server/get_type_description', '/my_ns/echo_server/list_parameters', '/my_ns/echo_server/set_parameters', ...] = <bound method TestROS2ServiceCLI.get_expected_builtin_services of <class 'test_cli.TestROS2ServiceCLI'>>('/my_ns/echo_server')
 +        where <bound method TestROS2ServiceCLI.get_expected_builtin_services of <class 'test_cli.TestROS2ServiceCLI'>> = <test_cli.TestROS2ServiceCLI testMethod=test_list_hidden[rmw_connextdds]>.get_expected_builtin_services
 +    and   '/array_repeater/describe_parameters\n/array_repeater/get_parameter_types\n/array_repeater/get_parameters\n/array_repe...st_parameters\n/unbounded_sequences_repeater/set_parameters\n/unbounded_sequences_repeater/set_parameters_atomically\n' = <launch_testing.tools.process.ProcessProxy object at 0x7b81e5ce27e0>.output


=========================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================
FAIL: test_cli.TestROS2ServiceCLI.test_list_services[rmw_connextdds]
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/ws/install_isolated/launch_testing/lib/python3.12/site-packages/launch_testing/markers.py", line 61, in _wrapper
    return func(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/ws/src/ros2/ros2cli/ros2service/test/test_cli.py", line 162, in test_list_services
    assert launch_testing.tools.expect_output(
AssertionError: assert False
 +  where False = <function expect_output at 0x7b81efc5c220>(expected_lines=<itertools.chain object at 0x7b81e5cfc550>, text='/array_repeater/describe_parameters\n/array_repeater/get_parameter_types\n/array_repeater/get_parameters\n/array_repe...st_parameters\n/unbounded_sequences_repeater/set_parameters\n/unbounded_sequences_repeater/set_parameters_atomically\n', strict=True)
 +    where <function expect_output at 0x7b81efc5c220> = <module 'launch_testing.tools' from '/tmp/ws/install_isolated/launch_testing/lib/python3.12/site-packages/launch_testing/tools/__init__.py'>.expect_output
 +      where <module 'launch_testing.tools' from '/tmp/ws/install_isolated/launch_testing/lib/python3.12/site-packages/launch_testing/tools/__init__.py'> = launch_testing.tools
 +    and   <itertools.chain object at 0x7b81e5cfc550> = <class 'itertools.chain'>(['/my_ns/echo'], ['/my_ns/echo_server/describe_parameters', '/my_ns/echo_server/get_parameter_types', '/my_ns/echo_server/get_parameter...y_ns/echo_server/get_type_description', '/my_ns/echo_server/list_parameters', '/my_ns/echo_server/set_parameters', ...])
 +      where <class 'itertools.chain'> = itertools.chain
 +      and   ['/my_ns/echo_server/describe_parameters', '/my_ns/echo_server/get_parameter_types', '/my_ns/echo_server/get_parameter...y_ns/echo_server/get_type_description', '/my_ns/echo_server/list_parameters', '/my_ns/echo_server/set_parameters', ...] = <bound method TestROS2ServiceCLI.get_expected_builtin_services of <class 'test_cli.TestROS2ServiceCLI'>>('/my_ns/echo_server')
 +        where <bound method TestROS2ServiceCLI.get_expected_builtin_services of <class 'test_cli.TestROS2ServiceCLI'>> = <test_cli.TestROS2ServiceCLI testMethod=test_list_services[rmw_connextdds]>.get_expected_builtin_services
 +    and   '/array_repeater/describe_parameters\n/array_repeater/get_parameter_types\n/array_repeater/get_parameters\n/array_repe...st_parameters\n/unbounded_sequences_repeater/set_parameters\n/unbounded_sequences_repeater/set_parameters_atomically\n' = <launch_testing.tools.process.ProcessProxy object at 0x7b81e5cfee10>.output


=========================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================
FAIL: test_cli.TestROS2ServiceCLI.test_list_with_types[rmw_connextdds]
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/ws/install_isolated/launch_testing/lib/python3.12/site-packages/launch_testing/markers.py", line 61, in _wrapper
    return func(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/ws/src/ros2/ros2cli/ros2service/test/test_cli.py", line 194, in test_list_with_types
    assert launch_testing.tools.expect_output(
AssertionError: assert False
 +  where False = <function expect_output at 0x7b81efc5c220>(expected_lines=<itertools.chain object at 0x7b81e5d0b4c0>, text='/array_repeater/describe_parameters [rcl_interfaces/srv/DescribeParameters]\n/array_repeater/get_parameter_types [rcl...SetParameters]\n/unbounded_sequences_repeater/set_parameters_atomically [rcl_interfaces/srv/SetParametersAtomically]\n', strict=True)
 +    where <function expect_output at 0x7b81efc5c220> = <module 'launch_testing.tools' from '/tmp/ws/install_isolated/launch_testing/lib/python3.12/site-packages/launch_testing/tools/__init__.py'>.expect_output
 +      where <module 'launch_testing.tools' from '/tmp/ws/install_isolated/launch_testing/lib/python3.12/site-packages/launch_testing/tools/__init__.py'> = launch_testing.tools
 +    and   <itertools.chain object at 0x7b81e5d0b4c0> = <class 'itertools.chain'>(['/my_ns/echo [test_msgs/srv/BasicTypes]'], ['/my_ns/echo_server/describe_parameters [rcl_interfaces/srv/DescribeParameters]', '/my_ns/echo_server/get_parameter_t...ters [rcl_interfaces/srv/ListParameters]', '/my_ns/echo_server/set_parameters [rcl_interfaces/srv/SetParameters]', ...])
 +      where <class 'itertools.chain'> = itertools.chain
 +      and   ['/my_ns/echo_server/describe_parameters [rcl_interfaces/srv/DescribeParameters]', '/my_ns/echo_server/get_parameter_t...ters [rcl_interfaces/srv/ListParameters]', '/my_ns/echo_server/set_parameters [rcl_interfaces/srv/SetParameters]', ...] = <bound method TestROS2ServiceCLI.get_expected_builtin_services of <class 'test_cli.TestROS2ServiceCLI'>>('/my_ns/echo_server', with_types=True)
 +        where <bound method TestROS2ServiceCLI.get_expected_builtin_services of <class 'test_cli.TestROS2ServiceCLI'>> = <test_cli.TestROS2ServiceCLI testMethod=test_list_with_types[rmw_connextdds]>.get_expected_builtin_services
 +    and   '/array_repeater/describe_parameters [rcl_interfaces/srv/DescribeParameters]\n/array_repeater/get_parameter_types [rcl...SetParameters]\n/unbounded_sequences_repeater/set_parameters_atomically [rcl_interfaces/srv/SetParametersAtomically]\n' = <launch_testing.tools.process.ProcessProxy object at 0x7b81e5d0a750>.output


=========================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================
FAIL: test_cli.TestROS2ServiceCLI.test_list_count[rmw_fastrtps_cpp]
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/ws/install_isolated/launch_testing/lib/python3.12/site-packages/launch_testing/markers.py", line 61, in _wrapper
    return func(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/ws/src/ros2/ros2cli/ros2service/test/test_cli.py", line 210, in test_list_count
    assert int(output_lines[0]) == self.builtin_service_count + 1
AssertionError: assert 57 == (7 + 1)
 +  where 57 = int('57')
 +  and   7 = <test_cli.TestROS2ServiceCLI testMethod=test_list_count[rmw_fastrtps_cpp]>.builtin_service_count


=========================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================
FAIL: test_cli.TestROS2ServiceCLI.test_list_hidden[rmw_fastrtps_cpp]
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/ws/install_isolated/launch_testing/lib/python3.12/site-packages/launch_testing/markers.py", line 61, in _wrapper
    return func(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/ws/src/ros2/ros2cli/ros2service/test/test_cli.py", line 178, in test_list_hidden
    assert launch_testing.tools.expect_output(
AssertionError: assert False
 +  where False = <function expect_output at 0x7b81efc5c220>(expected_lines=<itertools.chain object at 0x7b81e5d755d0>, text='/array_repeater/describe_parameters\n/array_repeater/get_parameter_types\n/array_repeater/get_parameters\n/array_repe...st_parameters\n/unbounded_sequences_repeater/set_parameters\n/unbounded_sequences_repeater/set_parameters_atomically\n', strict=True)
 +    where <function expect_output at 0x7b81efc5c220> = <module 'launch_testing.tools' from '/tmp/ws/install_isolated/launch_testing/lib/python3.12/site-packages/launch_testing/tools/__init__.py'>.expect_output
 +      where <module 'launch_testing.tools' from '/tmp/ws/install_isolated/launch_testing/lib/python3.12/site-packages/launch_testing/tools/__init__.py'> = launch_testing.tools
 +    and   <itertools.chain object at 0x7b81e5d755d0> = <class 'itertools.chain'>(['/my_ns/_echo'], ['/my_ns/_hidden_echo_server/describe_parameters', '/my_ns/_hidden_echo_server/get_parameter_types', '/my_ns/_hidden_e.../get_type_description', '/my_ns/_hidden_echo_server/list_parameters', '/my_ns/_hidden_echo_server/set_parameters', ...], ['/my_ns/echo'], ['/my_ns/echo_server/describe_parameters', '/my_ns/echo_server/get_parameter_types', '/my_ns/echo_server/get_parameter...y_ns/echo_server/get_type_description', '/my_ns/echo_server/list_parameters', '/my_ns/echo_server/set_parameters', ...])
 +      where <class 'itertools.chain'> = itertools.chain
 +      and   ['/my_ns/_hidden_echo_server/describe_parameters', '/my_ns/_hidden_echo_server/get_parameter_types', '/my_ns/_hidden_e.../get_type_description', '/my_ns/_hidden_echo_server/list_parameters', '/my_ns/_hidden_echo_server/set_parameters', ...] = <bound method TestROS2ServiceCLI.get_expected_builtin_services of <class 'test_cli.TestROS2ServiceCLI'>>('/my_ns/_hidden_echo_server')
 +        where <bound method TestROS2ServiceCLI.get_expected_builtin_services of <class 'test_cli.TestROS2ServiceCLI'>> = <test_cli.TestROS2ServiceCLI testMethod=test_list_hidden[rmw_fastrtps_cpp]>.get_expected_builtin_services
 +      and   ['/my_ns/echo_server/describe_parameters', '/my_ns/echo_server/get_parameter_types', '/my_ns/echo_server/get_parameter...y_ns/echo_server/get_type_description', '/my_ns/echo_server/list_parameters', '/my_ns/echo_server/set_parameters', ...] = <bound method TestROS2ServiceCLI.get_expected_builtin_services of <class 'test_cli.TestROS2ServiceCLI'>>('/my_ns/echo_server')
 +        where <bound method TestROS2ServiceCLI.get_expected_builtin_services of <class 'test_cli.TestROS2ServiceCLI'>> = <test_cli.TestROS2ServiceCLI testMethod=test_list_hidden[rmw_fastrtps_cpp]>.get_expected_builtin_services
 +    and   '/array_repeater/describe_parameters\n/array_repeater/get_parameter_types\n/array_repeater/get_parameters\n/array_repe...st_parameters\n/unbounded_sequences_repeater/set_parameters\n/unbounded_sequences_repeater/set_parameters_atomically\n' = <launch_testing.tools.process.ProcessProxy object at 0x7b81e5d75af0>.output


=========================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================
FAIL: test_cli.TestROS2ServiceCLI.test_list_services[rmw_fastrtps_cpp]
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/ws/install_isolated/launch_testing/lib/python3.12/site-packages/launch_testing/markers.py", line 61, in _wrapper
    return func(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/ws/src/ros2/ros2cli/ros2service/test/test_cli.py", line 162, in test_list_services
    assert launch_testing.tools.expect_output(
AssertionError: assert False
 +  where False = <function expect_output at 0x7b81efc5c220>(expected_lines=<itertools.chain object at 0x7b81e5b9e1d0>, text='/array_repeater/describe_parameters\n/array_repeater/get_parameter_types\n/array_repeater/get_parameters\n/array_repe...st_parameters\n/unbounded_sequences_repeater/set_parameters\n/unbounded_sequences_repeater/set_parameters_atomically\n', strict=True)
 +    where <function expect_output at 0x7b81efc5c220> = <module 'launch_testing.tools' from '/tmp/ws/install_isolated/launch_testing/lib/python3.12/site-packages/launch_testing/tools/__init__.py'>.expect_output
 +      where <module 'launch_testing.tools' from '/tmp/ws/install_isolated/launch_testing/lib/python3.12/site-packages/launch_testing/tools/__init__.py'> = launch_testing.tools
 +    and   <itertools.chain object at 0x7b81e5b9e1d0> = <class 'itertools.chain'>(['/my_ns/echo'], ['/my_ns/echo_server/describe_parameters', '/my_ns/echo_server/get_parameter_types', '/my_ns/echo_server/get_parameter...y_ns/echo_server/get_type_description', '/my_ns/echo_server/list_parameters', '/my_ns/echo_server/set_parameters', ...])
 +      where <class 'itertools.chain'> = itertools.chain
 +      and   ['/my_ns/echo_server/describe_parameters', '/my_ns/echo_server/get_parameter_types', '/my_ns/echo_server/get_parameter...y_ns/echo_server/get_type_description', '/my_ns/echo_server/list_parameters', '/my_ns/echo_server/set_parameters', ...] = <bound method TestROS2ServiceCLI.get_expected_builtin_services of <class 'test_cli.TestROS2ServiceCLI'>>('/my_ns/echo_server')
 +        where <bound method TestROS2ServiceCLI.get_expected_builtin_services of <class 'test_cli.TestROS2ServiceCLI'>> = <test_cli.TestROS2ServiceCLI testMethod=test_list_services[rmw_fastrtps_cpp]>.get_expected_builtin_services
 +    and   '/array_repeater/describe_parameters\n/array_repeater/get_parameter_types\n/array_repeater/get_parameters\n/array_repe...st_parameters\n/unbounded_sequences_repeater/set_parameters\n/unbounded_sequences_repeater/set_parameters_atomically\n' = <launch_testing.tools.process.ProcessProxy object at 0x7b81e5b9ca70>.output


=========================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================
FAIL: test_cli.TestROS2ServiceCLI.test_list_with_types[rmw_fastrtps_cpp]
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/ws/install_isolated/launch_testing/lib/python3.12/site-packages/launch_testing/markers.py", line 61, in _wrapper
    return func(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/ws/src/ros2/ros2cli/ros2service/test/test_cli.py", line 194, in test_list_with_types
    assert launch_testing.tools.expect_output(
AssertionError: assert False
 +  where False = <function expect_output at 0x7b81efc5c220>(expected_lines=<itertools.chain object at 0x7b81e5bb2140>, text='/array_repeater/describe_parameters [rcl_interfaces/srv/DescribeParameters]\n/array_repeater/get_parameter_types [rcl...SetParameters]\n/unbounded_sequences_repeater/set_parameters_atomically [rcl_interfaces/srv/SetParametersAtomically]\n', strict=True)
 +    where <function expect_output at 0x7b81efc5c220> = <module 'launch_testing.tools' from '/tmp/ws/install_isolated/launch_testing/lib/python3.12/site-packages/launch_testing/tools/__init__.py'>.expect_output
 +      where <module 'launch_testing.tools' from '/tmp/ws/install_isolated/launch_testing/lib/python3.12/site-packages/launch_testing/tools/__init__.py'> = launch_testing.tools
 +    and   <itertools.chain object at 0x7b81e5bb2140> = <class 'itertools.chain'>(['/my_ns/echo [test_msgs/srv/BasicTypes]'], ['/my_ns/echo_server/describe_parameters [rcl_interfaces/srv/DescribeParameters]', '/my_ns/echo_server/get_parameter_t...ters [rcl_interfaces/srv/ListParameters]', '/my_ns/echo_server/set_parameters [rcl_interfaces/srv/SetParameters]', ...])
 +      where <class 'itertools.chain'> = itertools.chain
 +      and   ['/my_ns/echo_server/describe_parameters [rcl_interfaces/srv/DescribeParameters]', '/my_ns/echo_server/get_parameter_t...ters [rcl_interfaces/srv/ListParameters]', '/my_ns/echo_server/set_parameters [rcl_interfaces/srv/SetParameters]', ...] = <bound method TestROS2ServiceCLI.get_expected_builtin_services of <class 'test_cli.TestROS2ServiceCLI'>>('/my_ns/echo_server', with_types=True)
 +        where <bound method TestROS2ServiceCLI.get_expected_builtin_services of <class 'test_cli.TestROS2ServiceCLI'>> = <test_cli.TestROS2ServiceCLI testMethod=test_list_with_types[rmw_fastrtps_cpp]>.get_expected_builtin_services
 +    and   '/array_repeater/describe_parameters [rcl_interfaces/srv/DescribeParameters]\n/array_repeater/get_parameter_types [rcl...SetParameters]\n/unbounded_sequences_repeater/set_parameters_atomically [rcl_interfaces/srv/SetParametersAtomically]\n' = <launch_testing.tools.process.ProcessProxy object at 0x7b81e5bb2060>.output

[Blast545]

🧑‍🌾 Why is this failing in Kilted and not in Rolling? @Crola1702 Can you investigate what changed that made this disappear in Rolling? Since it's a consistent one, it should be easier to track what happened and backport it.

It seems it was happening in Rolling for
https://build.ros2.org/job/Rci__nightly-debug_ubuntu_noble_amd64/418/#showFailuresLink
And it's not happening for
https://build.ros2.org/job/Rci__nightly-debug_ubuntu_noble_amd64/421/#showFailuresLink

PTAL

[Crola1702]

Last consistent failures in Rolling were:

• https://build.ros2.org/job/Rci__nightly-connext_ubuntu_noble_amd64/431 (25 days ago)
• https://build.ros2.org/job/Rci__nightly-debug_ubuntu_noble_amd64/418/ (27 days ago)
• https://build.ros2.org/job/Rci__nightly-release_ubuntu_noble_amd64/428/ (26 days ago)

In those dates we merged: #1056 and #1055. Latter one was backported into Kilted.

After those consistent failures we've had:

• https://build.ros2.org/view/Rci/job/Rci__nightly-fastrtps_ubuntu_noble_amd64/441/testReport/junit/ros2service.ros2service.test/test_cli/test_cli/
• https://build.ros2.org/view/Rci/job/Rci__nightly-release_ubuntu_noble_amd64/447/testReport/junit/ros2service.ros2service.test/test_cli/test_cli/

[fujitatomoya]

@Blast545 @Crola1702 for rolling, i think #1098 does the work.

[fujitatomoya]

Oh sorry i think i am wrong, the flaky failures disappeared much earlier than #1098, my bad.

[fujitatomoya]

hmmm, i cannot reproduce this issue with kilted source build...

root@tomoyafujita-B760M-Pro-RS-D4:~/ros2_ws/kilted_ws# for i in {1..10}; do
> colcon test --packages-select ros2service --pytest-args test/test_cli.py || break
> done
Starting >>> ros2service
[Processing: ros2service]
Finished <<< ros2service [32.8s]
...<snip>

as it reported, there seems to be some nodes running that are NOT supposed to be running during the test.

probably backporting rmw isolation features could resolve this flaky results?

[Blast545]

🧑‍🌾 Those were flaky when there was Kilted was still rolling. I took a bit of time now to track if there was a clue as to when this stopped happening. I used the clue you said about this only happening when other node was still running and tried to check when the rolling jobs had a drop in the number of tests failing.

It seems to me that backporting #1062 should do the trick. I will run a couple of tests for that.

[ros-discourse]

This issue has been mentioned on Open Robotics Discourse. There might be relevant details there:

https://discourse.openrobotics.org/t/ros-pmc-minutes-for-september-30-2025/50379/1

[Crola1702]

Flakiness in the last 15 days:

job_name	last_fail	first_fail	build_count	failure_count	failure_percentage
Kci__nightly-release_ubuntu_noble_amd64	2025-10-20	2025-10-11	5	4	80.0
Kci__nightly-debug_ubuntu_noble_amd64	2025-10-19	2025-10-10	3	3	100.0
Kci__nightly-connext_ubuntu_noble_amd64	2025-10-19	2025-10-07	5	5	100.0
nightly_linux_repeated	2025-10-16	2025-10-16	14	1	7.14
nightly_linux-aarch64_repeated	2025-10-11	2025-10-11	14	1	7.14
nightly_linux-rhel_repeated	2025-10-09	2025-10-09	15	1	6.67

Friendly ping @cottsay