tests/run-tests.py: Run tests-with-regex-output as normal tests.
Some checks failed
JavaScript code lint and formatting with Biome / eslint (push) Has been cancelled
Check code formatting / code-formatting (push) Has been cancelled
Check spelling with codespell / codespell (push) Has been cancelled
Build docs / build (push) Has been cancelled
Check examples / embedding (push) Has been cancelled
Package mpremote / build (push) Has been cancelled
.mpy file format and tools / test (push) Has been cancelled
Build ports metadata / build (push) Has been cancelled
alif port / build_alif (alif_ae3_build) (push) Has been cancelled
cc3200 port / build (push) Has been cancelled
esp32 port / build_idf (esp32_build_c2_c6) (push) Has been cancelled
esp32 port / build_idf (esp32_build_cmod_spiram_s2) (push) Has been cancelled
esp32 port / build_idf (esp32_build_s3_c3) (push) Has been cancelled
esp8266 port / build (push) Has been cancelled
mimxrt port / build (push) Has been cancelled
nrf port / build (push) Has been cancelled
powerpc port / build (push) Has been cancelled
qemu port / build_and_test_arm (bigendian) (push) Has been cancelled
qemu port / build_and_test_arm (sabrelite) (push) Has been cancelled
qemu port / build_and_test_arm (thumb) (push) Has been cancelled
qemu port / build_and_test_rv32 (push) Has been cancelled
renesas-ra port / build_renesas_ra_board (push) Has been cancelled
rp2 port / build (push) Has been cancelled
samd port / build (push) Has been cancelled
stm32 port / build_stm32 (stm32_misc_build) (push) Has been cancelled
stm32 port / build_stm32 (stm32_nucleo_build) (push) Has been cancelled
stm32 port / build_stm32 (stm32_pyb_build) (push) Has been cancelled
unix port / minimal (push) Has been cancelled
unix port / reproducible (push) Has been cancelled
unix port / standard (push) Has been cancelled
unix port / standard_v2 (push) Has been cancelled
unix port / coverage (push) Has been cancelled
unix port / coverage_32bit (push) Has been cancelled
unix port / nanbox (push) Has been cancelled
unix port / longlong (push) Has been cancelled
unix port / float (push) Has been cancelled
unix port / gil_enabled (push) Has been cancelled
unix port / stackless_clang (push) Has been cancelled
unix port / float_clang (push) Has been cancelled
unix port / settrace_stackless (push) Has been cancelled
unix port / macos (push) Has been cancelled
unix port / qemu_mips (push) Has been cancelled
unix port / qemu_arm (push) Has been cancelled
unix port / qemu_riscv64 (push) Has been cancelled
unix port / sanitize_address (push) Has been cancelled
unix port / sanitize_undefined (push) Has been cancelled
webassembly port / build (push) Has been cancelled
windows port / build-vs (Debug, x64, dev, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Debug, x64, dev, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Debug, x86, dev, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Debug, x86, dev, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Release, x64, dev, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Release, x64, dev, 2019, [16, 17)) (push) Has been cancelled
windows port / build-vs (Release, x64, dev, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Release, x64, standard, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Release, x64, standard, 2019, [16, 17)) (push) Has been cancelled
windows port / build-vs (Release, x64, standard, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Release, x86, dev, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Release, x86, dev, 2019, [16, 17)) (push) Has been cancelled
windows port / build-vs (Release, x86, dev, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Release, x86, standard, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Release, x86, standard, 2019, [16, 17)) (push) Has been cancelled
windows port / build-vs (Release, x86, standard, 2022, [17, 18)) (push) Has been cancelled
windows port / build-mingw (i686, mingw32, dev) (push) Has been cancelled
windows port / build-mingw (i686, mingw32, standard) (push) Has been cancelled
windows port / build-mingw (x86_64, mingw64, dev) (push) Has been cancelled
windows port / build-mingw (x86_64, mingw64, standard) (push) Has been cancelled
windows port / cross-build-on-linux (push) Has been cancelled
zephyr port / build (push) Has been cancelled
Python code lint and formatting with ruff / ruff (push) Has been cancelled

Some tests (currently given by the `special_tests` list) have output which
must be mached via a regex, because it can change from run to run (eg the
address of an object is printed).  These tests are currently classified as
`is_special` in the test runner, which means they get special treatment.
In particular they don't set the emitter as specified by `args.emit`.  That
means these tests do not run via .mpy or using the native emitter, even if
those options are given on the command line.

This commit fixes that by considering `is_special` as different to
`tests_with_regex_output`.  The former is used for things like target
feature detection (which are not really tests) and when extra command line
options need to be passed to the unix micropython executable.  The latter
(now called `tests_with_regex_output`) are specifically for tests that have
output to be matched via regex.

The `thread_exc2.py` test now needs to be excluded when running using the
native emitter, because the native emitter doesn't print traceback info.
And the `sys_settrace_cov.py` test needs to be excluded because set-trace
output is different with the native emitter.

Signed-off-by: Damien George <damien@micropython.org>
This commit is contained in:
Damien George
2025-07-26 19:11:36 +10:00
parent c35427cbdb
commit 1401fdb8b3

View File

@@ -127,6 +127,7 @@ emitter_tests_to_skip = {
"micropython/emg_exc.py",
"micropython/heapalloc_traceback.py",
"micropython/opt_level_lineno.py",
"thread/thread_exc2.py",
# These require stack-allocated slice optimisation.
"micropython/heapalloc_slice.py",
# These require running the scheduler.
@@ -136,6 +137,7 @@ emitter_tests_to_skip = {
# These require sys.exc_info().
"misc/sys_exc_info.py",
# These require sys.settrace().
"misc/sys_settrace_cov.py",
"misc/sys_settrace_features.py",
"misc/sys_settrace_generator.py",
"misc/sys_settrace_loop.py",
@@ -392,7 +394,7 @@ def run_script_on_remote_target(pyb, args, test_file, is_special):
return had_crash, output_mupy
special_tests = [
tests_with_regex_output = [
base_path(file)
for file in (
"micropython/meminfo.py",
@@ -409,10 +411,7 @@ def run_micropython(pyb, args, test_file, test_file_abspath, is_special=False):
had_crash = False
if pyb is None:
# run on PC
if (
test_file_abspath.startswith((base_path("cmdline/"), base_path("feature_check/")))
or test_file_abspath in special_tests
):
if test_file_abspath.startswith((base_path("cmdline/"), base_path("feature_check/"))):
# special handling for tests of the unix cmdline program
is_special = True
@@ -544,7 +543,7 @@ def run_micropython(pyb, args, test_file, test_file_abspath, is_special=False):
if is_special and not had_crash and b"\nSKIP\n" in output_mupy:
return b"SKIP\n"
if is_special or test_file_abspath in special_tests:
if is_special or test_file_abspath in tests_with_regex_output:
# convert parts of the output that are not stable across runs
with open(test_file + ".exp", "rb") as f:
lines_exp = []