Cannot run zephyrproject-rtos/example-application Twister tests

[IvanVnucec]

I'm trying Twister testing in the Zephyr example-application.
I'm running twister by calling west twister -G --board-root boards/ --testsuite-root ./tests/ and this is what I got:

INFO    - Using Ninja..
INFO    - Zephyr version: zephyr-v3.3.0-2661-ga86ad87d84da
INFO    - Using 'zephyr' toolchain.
INFO    - Selecting default platforms per test case
INFO    - Building initial testsuite list...
INFO    - Writing JSON report /__w/example-application/example-application/example-application/twister-out/testplan.json
ZEPHYR_BASE unset, using "/__w/example-application/example-application/zephyr"
INFO    - JOBS: 2
INFO    - Adding tasks to the queue...
INFO    - Added initial list of jobs to queue
INFO    - Total complete:    1/   4  25%  skipped:    0, failed:    0, error:    0
INFO    - Total complete:    2/   4  50%  skipped:    0, failed:    0, error:    0
INFO    - Total complete:    3/   4  75%  skipped:    0, failed:    0, error:    0
INFO    - Total complete:    4/   4  100%  skipped:    0, failed:    0, error:    0
INFO    - 2 test scenarios (4 test instances) selected, 0 configurations skipped (0 by static filter, 0 at runtime).
INFO    - 4 of 4 test configurations passed (100.00%), 0 failed, 0 errored, 0 skipped with 0 warnings in 52.04 seconds
INFO    - In total 4 test cases were executed, 0 skipped on 2 out of total 559 platforms (0.36%)
INFO    - 2 test configurations executed on platforms, 2 test configurations were only built.
INFO    - Saving reports...
INFO    - Writing JSON report /__w/example-application/example-application/example-application/twister-out/twister.json
INFO    - Writing xunit report /__w/example-application/example-application/example-application/twister-out/twister.xml...
INFO    - Writing xunit report /__w/example-application/example-application/example-application/twister-out/twister_report.xml...
INFO    - Run completed

Please focus on the 2 test configurations executed on platforms, 2 test configurations were only built line where it says that the 2 test configurations were only built.
To confirm this, I've went in tests\lib\custom_lib\src\main.c and added zassert_equal(1, 0, ""); line that must fail in one of the tests and the test results in my-manifest-repo\twister-out\twister.json shows "status":"passed" on every testsuite.

My question is why the tests are not ran, even on qemu_cortex_m0 platform defined in tests\lib\custom_lib\testcase.yaml? And how do I run the tests?

[IvanVnucec]

I saw the few reasons listed that would prevent the Twister from running the tests listed here:

    The test is marked as build_only: true in its .yaml configuration file.
    The test configuration has defined a harness but you don’t have it or haven’t set it up.
    The target device is not connected and not available for flashing
    You or some higher level automation invoked twister with --build-only.

but none of the above reasons are true in my case. I even have qemu installed and added to path but it isn't ran either.

[73jn]

Hello,
I'm facing the same problem as you. Have you been able to solve it?

[IvanVnucec]

I can't remember. Sorry. I think not...

[73jn]

I've just re-installed the environment but this time on WSL and it works by building and running with native_posix. Surprisingly, with qemu it seems that twister can't launch the application.

[IvanVnucec]

Can you post the logs when the twister can't launch the application in qemu?