Commit Graph

3 Commits

Author SHA1 Message Date
Damien George
6436f8b93f tests/run-tests.py: Abort test run if enter_raw_repl fails many times.
Prior to this change, if a test crashed a board to the point that it was no
longer responsive but the serial connection was still alive and able to
read/write (eg due to an infinite loop with ctrl-C unavailable), it would
take (30 + 10 * number-of-remaining-tests) seconds to finish the test run.
That can be 10 minutes or more.

The change here aborts the test run if the target does not respond 4 times
to when trying to enter the raw REPL at the start of a test.  Now, it only
takes a maximum of (30 + 4 * 5) = 50 seconds to detect a crashed state and
abort the entire test run.

Can test this by adding the following to a test:

    import time, micropython
    micropython.kbd_intr(-1)
    while 1:
        time.sleep(1)

For example, add the above to the end of `tests/extmod/random_extra.py` and
test via:

    $ ./run-tests.py -t a0 extmod/[r-z]*.py
    $ ./run-natmodtests.py -t a0 extmod/[r-z]*.py

Previously the `run-tests.py` run would take around 16 minutes.  Now it
takes under a minute.

Signed-off-by: Damien George <damien@micropython.org>
2026-02-04 20:17:11 +11:00
Damien George
f624200b31 tests/test_utils.py: Abort test if serial port fails at start.
All test runners (except `run-multitests.py`) will now abort with error
code 2 if the target is a serial port and could not be detected, ie
`enter_raw_repl` failed at the detection stage.

Signed-off-by: Damien George <damien@micropython.org>
2026-02-04 20:17:11 +11:00
Damien George
96bce47762 tests: Factor out common helper functions to separate Python module.
Some checks failed
JavaScript code lint and formatting with Biome / eslint (push) Has been cancelled
Check code formatting / code-formatting (push) Has been cancelled
Check spelling with codespell / codespell (push) Has been cancelled
Build docs / build (push) Has been cancelled
Check examples / embedding (push) Has been cancelled
Package mpremote / build (push) Has been cancelled
.mpy file format and tools / test (push) Has been cancelled
Build ports metadata / build (push) Has been cancelled
alif port / build_alif (alif_ae3_build) (push) Has been cancelled
cc3200 port / build (push) Has been cancelled
esp32 port / build_idf (esp32_build_c2_c5_c6) (push) Has been cancelled
esp32 port / build_idf (esp32_build_cmod_spiram_s2) (push) Has been cancelled
esp32 port / build_idf (esp32_build_p4) (push) Has been cancelled
esp32 port / build_idf (esp32_build_s3_c3) (push) Has been cancelled
esp8266 port / build (push) Has been cancelled
mimxrt port / build (push) Has been cancelled
nrf port / build (push) Has been cancelled
powerpc port / build (push) Has been cancelled
qemu port / build_and_test_arm (bigendian) (push) Has been cancelled
qemu port / build_and_test_arm (sabrelite) (push) Has been cancelled
qemu port / build_and_test_arm (thumb_hardfp) (push) Has been cancelled
qemu port / build_and_test_arm (thumb_softfp) (push) Has been cancelled
qemu port / build_and_test_rv32 (push) Has been cancelled
qemu port / build_and_test_rv64 (push) Has been cancelled
renesas-ra port / build_renesas_ra_board (push) Has been cancelled
rp2 port / build (push) Has been cancelled
samd port / build (push) Has been cancelled
stm32 port / build_stm32 (stm32_misc_build) (push) Has been cancelled
stm32 port / build_stm32 (stm32_nucleo_build) (push) Has been cancelled
stm32 port / build_stm32 (stm32_pyb_build) (push) Has been cancelled
unix port / minimal (push) Has been cancelled
unix port / reproducible (push) Has been cancelled
unix port / standard (push) Has been cancelled
unix port / standard_v2 (push) Has been cancelled
unix port / coverage (push) Has been cancelled
unix port / coverage_32bit (push) Has been cancelled
unix port / nanbox (push) Has been cancelled
unix port / longlong (push) Has been cancelled
unix port / float (push) Has been cancelled
unix port / gil_enabled (push) Has been cancelled
unix port / stackless_clang (push) Has been cancelled
unix port / float_clang (push) Has been cancelled
unix port / settrace_stackless (push) Has been cancelled
unix port / repr_b (push) Has been cancelled
unix port / macos (push) Has been cancelled
unix port / qemu_mips (push) Has been cancelled
unix port / qemu_arm (push) Has been cancelled
unix port / qemu_riscv64 (push) Has been cancelled
unix port / sanitize_address (push) Has been cancelled
unix port / sanitize_undefined (push) Has been cancelled
webassembly port / build (push) Has been cancelled
windows port / build-vs (Debug, true, x64, dev, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Debug, true, x86, dev, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Debug, x64, dev, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Debug, x86, dev, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Release, true, x64, dev, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Release, true, x64, dev, 2019, [16, 17)) (push) Has been cancelled
windows port / build-vs (Release, true, x64, standard, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Release, true, x64, standard, 2019, [16, 17)) (push) Has been cancelled
windows port / build-vs (Release, true, x86, dev, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Release, true, x86, dev, 2019, [16, 17)) (push) Has been cancelled
windows port / build-vs (Release, true, x86, standard, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Release, true, x86, standard, 2019, [16, 17)) (push) Has been cancelled
windows port / build-vs (Release, x64, dev, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Release, x64, standard, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Release, x86, dev, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Release, x86, standard, 2022, [17, 18)) (push) Has been cancelled
windows port / build-mingw (i686, mingw32, dev) (push) Has been cancelled
windows port / build-mingw (i686, mingw32, standard) (push) Has been cancelled
windows port / build-mingw (x86_64, mingw64, dev) (push) Has been cancelled
windows port / build-mingw (x86_64, mingw64, standard) (push) Has been cancelled
windows port / cross-build-on-linux (push) Has been cancelled
zephyr port / build (push) Has been cancelled
Python code lint and formatting with ruff / ruff (push) Has been cancelled
The test runners have evolved over time and become more and more complex.
In particular `tests/run-tests.py` is rather large now.  The test runners
also duplicate some functionality amongst themselves.

As a start to improving this situation, this commit factors out the helper
functions from `run-tests.py` into a new `test_utils.py` file, and uses
that new module in all test runners.

There should be no functional change here.

Signed-off-by: Damien George <damien@micropython.org>
2026-01-30 17:03:08 +11:00