Commit Graph

10 Commits

Author SHA1 Message Date
Anson Mansfield 8a1f050771 tests/run-internalbench.py: Add option for running against CPython.
Occasionally, it's useful to be able to compare MicroPython's performance
figures to CPython's. This change adds the ability to run the internalbench
test runner with `--test-instance=cpython` in order to execute the same
test routines against CPython and produce a benchmark performance report in
the same format as MicroPython.

Signed-off-by: Anson Mansfield <amansfield@mantaro.com>
2026-03-03 23:25:36 +11:00
Damien George 96bce47762 tests: Factor out common helper functions to separate Python module.
JavaScript code lint and formatting with Biome / eslint (push) Has been cancelled
Check code formatting / code-formatting (push) Has been cancelled
Check spelling with codespell / codespell (push) Has been cancelled
Build docs / build (push) Has been cancelled
Check examples / embedding (push) Has been cancelled
Package mpremote / build (push) Has been cancelled
.mpy file format and tools / test (push) Has been cancelled
Build ports metadata / build (push) Has been cancelled
alif port / build_alif (alif_ae3_build) (push) Has been cancelled
cc3200 port / build (push) Has been cancelled
esp32 port / build_idf (esp32_build_c2_c5_c6) (push) Has been cancelled
esp32 port / build_idf (esp32_build_cmod_spiram_s2) (push) Has been cancelled
esp32 port / build_idf (esp32_build_p4) (push) Has been cancelled
esp32 port / build_idf (esp32_build_s3_c3) (push) Has been cancelled
esp8266 port / build (push) Has been cancelled
mimxrt port / build (push) Has been cancelled
nrf port / build (push) Has been cancelled
powerpc port / build (push) Has been cancelled
qemu port / build_and_test_arm (bigendian) (push) Has been cancelled
qemu port / build_and_test_arm (sabrelite) (push) Has been cancelled
qemu port / build_and_test_arm (thumb_hardfp) (push) Has been cancelled
qemu port / build_and_test_arm (thumb_softfp) (push) Has been cancelled
qemu port / build_and_test_rv32 (push) Has been cancelled
qemu port / build_and_test_rv64 (push) Has been cancelled
renesas-ra port / build_renesas_ra_board (push) Has been cancelled
rp2 port / build (push) Has been cancelled
samd port / build (push) Has been cancelled
stm32 port / build_stm32 (stm32_misc_build) (push) Has been cancelled
stm32 port / build_stm32 (stm32_nucleo_build) (push) Has been cancelled
stm32 port / build_stm32 (stm32_pyb_build) (push) Has been cancelled
unix port / minimal (push) Has been cancelled
unix port / reproducible (push) Has been cancelled
unix port / standard (push) Has been cancelled
unix port / standard_v2 (push) Has been cancelled
unix port / coverage (push) Has been cancelled
unix port / coverage_32bit (push) Has been cancelled
unix port / nanbox (push) Has been cancelled
unix port / longlong (push) Has been cancelled
unix port / float (push) Has been cancelled
unix port / gil_enabled (push) Has been cancelled
unix port / stackless_clang (push) Has been cancelled
unix port / float_clang (push) Has been cancelled
unix port / settrace_stackless (push) Has been cancelled
unix port / repr_b (push) Has been cancelled
unix port / macos (push) Has been cancelled
unix port / qemu_mips (push) Has been cancelled
unix port / qemu_arm (push) Has been cancelled
unix port / qemu_riscv64 (push) Has been cancelled
unix port / sanitize_address (push) Has been cancelled
unix port / sanitize_undefined (push) Has been cancelled
webassembly port / build (push) Has been cancelled
windows port / build-vs (Debug, true, x64, dev, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Debug, true, x86, dev, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Debug, x64, dev, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Debug, x86, dev, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Release, true, x64, dev, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Release, true, x64, dev, 2019, [16, 17)) (push) Has been cancelled
windows port / build-vs (Release, true, x64, standard, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Release, true, x64, standard, 2019, [16, 17)) (push) Has been cancelled
windows port / build-vs (Release, true, x86, dev, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Release, true, x86, dev, 2019, [16, 17)) (push) Has been cancelled
windows port / build-vs (Release, true, x86, standard, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Release, true, x86, standard, 2019, [16, 17)) (push) Has been cancelled
windows port / build-vs (Release, x64, dev, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Release, x64, standard, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Release, x86, dev, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Release, x86, standard, 2022, [17, 18)) (push) Has been cancelled
windows port / build-mingw (i686, mingw32, dev) (push) Has been cancelled
windows port / build-mingw (i686, mingw32, standard) (push) Has been cancelled
windows port / build-mingw (x86_64, mingw64, dev) (push) Has been cancelled
windows port / build-mingw (x86_64, mingw64, standard) (push) Has been cancelled
windows port / cross-build-on-linux (push) Has been cancelled
zephyr port / build (push) Has been cancelled
Python code lint and formatting with ruff / ruff (push) Has been cancelled
The test runners have evolved over time and become more and more complex.
In particular `tests/run-tests.py` is rather large now.  The test runners
also duplicate some functionality amongst themselves.

As a start to improving this situation, this commit factors out the helper
functions from `run-tests.py` into a new `test_utils.py` file, and uses
that new module in all test runners.

There should be no functional change here.

Signed-off-by: Damien George <damien@micropython.org>
2026-01-30 17:03:08 +11:00
Anson Mansfield 88cb6bc818 tests/run-internalbench.py: Allow running internalbench on hardware.
JavaScript code lint and formatting with Biome / eslint (push) Has been cancelled
Check code formatting / code-formatting (push) Has been cancelled
Check spelling with codespell / codespell (push) Has been cancelled
Build docs / build (push) Has been cancelled
Check examples / embedding (push) Has been cancelled
Package mpremote / build (push) Has been cancelled
.mpy file format and tools / test (push) Has been cancelled
Build ports metadata / build (push) Has been cancelled
alif port / build_alif (alif_ae3_build) (push) Has been cancelled
cc3200 port / build (push) Has been cancelled
esp32 port / build_idf (esp32_build_cmod_spiram_s2) (push) Has been cancelled
esp32 port / build_idf (esp32_build_s3_c3) (push) Has been cancelled
esp8266 port / build (push) Has been cancelled
mimxrt port / build (push) Has been cancelled
nrf port / build (push) Has been cancelled
powerpc port / build (push) Has been cancelled
qemu port / build_and_test_arm (bigendian) (push) Has been cancelled
qemu port / build_and_test_arm (sabrelite) (push) Has been cancelled
qemu port / build_and_test_arm (thumb) (push) Has been cancelled
qemu port / build_and_test_rv32 (push) Has been cancelled
renesas-ra port / build_renesas_ra_board (push) Has been cancelled
rp2 port / build (push) Has been cancelled
samd port / build (push) Has been cancelled
stm32 port / build_stm32 (stm32_misc_build) (push) Has been cancelled
stm32 port / build_stm32 (stm32_nucleo_build) (push) Has been cancelled
stm32 port / build_stm32 (stm32_pyb_build) (push) Has been cancelled
unix port / minimal (push) Has been cancelled
unix port / reproducible (push) Has been cancelled
unix port / standard (push) Has been cancelled
unix port / standard_v2 (push) Has been cancelled
unix port / coverage (push) Has been cancelled
unix port / coverage_32bit (push) Has been cancelled
unix port / nanbox (push) Has been cancelled
unix port / longlong (push) Has been cancelled
unix port / float (push) Has been cancelled
unix port / gil_enabled (push) Has been cancelled
unix port / stackless_clang (push) Has been cancelled
unix port / float_clang (push) Has been cancelled
unix port / settrace_stackless (push) Has been cancelled
unix port / macos (push) Has been cancelled
unix port / qemu_mips (push) Has been cancelled
unix port / qemu_arm (push) Has been cancelled
unix port / qemu_riscv64 (push) Has been cancelled
unix port / sanitize_address (push) Has been cancelled
unix port / sanitize_undefined (push) Has been cancelled
webassembly port / build (push) Has been cancelled
windows port / build-vs (Debug, x64, dev, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Debug, x64, dev, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Debug, x86, dev, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Debug, x86, dev, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Release, x64, dev, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Release, x64, dev, 2019, [16, 17)) (push) Has been cancelled
windows port / build-vs (Release, x64, dev, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Release, x64, standard, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Release, x64, standard, 2019, [16, 17)) (push) Has been cancelled
windows port / build-vs (Release, x64, standard, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Release, x86, dev, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Release, x86, dev, 2019, [16, 17)) (push) Has been cancelled
windows port / build-vs (Release, x86, dev, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Release, x86, standard, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Release, x86, standard, 2019, [16, 17)) (push) Has been cancelled
windows port / build-vs (Release, x86, standard, 2022, [17, 18)) (push) Has been cancelled
windows port / build-mingw (i686, mingw32, dev) (push) Has been cancelled
windows port / build-mingw (i686, mingw32, standard) (push) Has been cancelled
windows port / build-mingw (x86_64, mingw64, dev) (push) Has been cancelled
windows port / build-mingw (x86_64, mingw64, standard) (push) Has been cancelled
windows port / cross-build-on-linux (push) Has been cancelled
zephyr port / build (push) Has been cancelled
Python code lint and formatting with ruff / ruff (push) Has been cancelled
Signed-off-by: Anson Mansfield <amansfield@mantaro.com>
2025-07-29 11:49:10 +10:00
Angus Gratton b461d218d1 tests/run-internalbench.py: Remove old CPython reference.
At one point it was possible to internal_bench CPython vs MicroPython, but
seemingly not any more.

Signed-off-by: Angus Gratton <angus@redyak.com.au>
2023-09-29 15:41:41 +10:00
Jim Mussared 8b27482692 top: Update Python formatting to black "2023 stable style".
See https://black.readthedocs.io/en/stable/the_black_code_style/index.html

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-02-02 12:51:03 +11:00
stijn 9c7ff87643 all: Keep msvc build output in build/ directories.
This follow the change made for Makefile-based projects in b2e82402.
2022-12-13 17:18:53 +11:00
Damien George cbc9f944c4 tests,tools: Update path to unix micropython executable.
These were missed by 47c84286e8

Signed-off-by: Damien George <damien@micropython.org>
2022-08-18 11:47:58 +10:00
Damien George 6129b8e401 tests: Rename run-tests to run-tests.py for consistency.
Signed-off-by: Damien George <damien@micropython.org>
2021-03-12 19:56:09 +11:00
David Lechner 3dc324d3f1 tests: Format all Python code with black, except tests in basics subdir.
This adds the Python files in the tests/ directory to be formatted with
./tools/codeformat.py.  The basics/ subdirectory is excluded for now so we
aren't changing too much at once.

In a few places `# fmt: off`/`# fmt: on` was used where the code had
special formatting for readability or where the test was actually testing
the specific formatting.
2020-03-30 13:21:58 +11:00
Damien George d86fb670e6 tests: Rename "bench" tests to "internal_bench" and run-internalbench.py
To emphasise these benchmark tests compare the internal performance of
features amongst themselves, rather than absolute performance testing.
2019-06-28 16:28:59 +10:00