138 Commits

Author SHA1 Message Date
Alessandro Gatti
1df86516e0 py/asmrv32: Reserve a flag for the Zcmp RV32 CPU extension.
This commit performs the necessary changes to handle an additional RV32
CPU extension flag, for the Zcmp extension in this case.

The changes are not limited to RV32-only code, as other parts of the
tooling need to be modified for this: the testing framework has to be
made aware that an extra bit can be set in sys.implementation._mpy and
needs to know how it is called, and "mpy-cross" must be able to actually
set that flag bit in the first place via the appropriate command line
argument.

Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
2025-12-19 17:06:53 +11:00
Alessandro Gatti
cb9d8fcc31 tests/run-tests.py: Discover RV32 extension inlineasm tests.
This commit extends the test runner to automatically discover inline
assembler tests for known RV32 extensions, and checks whether to add the
discovered tests to the enabled tests list.

Automatic discovery requires that inline assembler tests for RV32
extensions follow a specific pattern both for filenames and for the
tests' output in case of success.  A valid RV32 extension test must
have:

  * A code fragment that checks for support of the extension on the
    running target in "/tests/feature_check", called
    "inlineasm_rv32_<extensionname>.py" that should print the string
    "rv32_<extensionname>" if the extension is supported
  * A matching expected result file in "/tests/feature_check" called
    "inlineasm_rv32_<extensionname>.py.exp" that must contain the string
    "rv32_<extensionname>" (without quotes)
  * A regular MicroPython test file in "/tests/inlineasm/rv32" called
    "asm_ext_<extensionname>.py"

For example, to test the Zba extension, there must be a file called
"/tests/feature_check/inlineasm_rv32_zba.py" that should print the
string "rv32_zba" if the extension is supported, together with a file
called "/test/feature_check/inlineasm_rv32_zba.py.exp" that contains the
string "rv32_zba" in it, and finally there must be a regular MicroPython
test file called "/tests/inlineasm/rv32/asm_ext_zba.py".

Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
2025-12-19 17:06:53 +11:00
Alessandro Gatti
750a366f20 tests/run-tests.py: Skip list sort stress test for ESP8266.
This commit marks the "stress/list_sort.py" test to be skipped when
running on ESP8266.

The test just takes too long without yielding to the OS whilst doing the
sort, causing the internal software watchdog to kick in and reboot the
board.

Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
2025-12-17 16:05:18 +11:00
Andrew Leech
e067d96c8b tests/run-tests.py: Add general newline normalization function.
Add a general normalize_newlines() function that handles newline variations
(\\r\\r\\n, \\r\\n) to \\n while preserving literal \\r characters that are
part of test content.

This provides a robust solution for cross-platform test compatibility,
particularly addressing PTY double-newline issues that can occur with some
terminal implementations.

The function is applied to all test output before comparison, eliminating
platform-specific newline issues.

Includes a unit test to verify the normalization behavior.

Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>
2025-11-22 00:06:59 +11:00
Ihor Nehrutsa
094437686f tests/extmod_hardware/machine_encoder.py: Use target_wiring for encoder.
Signed-off-by: Ihor Nehrutsa <Ihor.Nehrutsa@gmail.com>
2025-11-19 16:10:12 +11:00
Alessandro Gatti
849c67bcdc tests/run-tests.py: Pass auto-detected architecture flags to mpy-cross.
This commit lets "run-tests.py" use the encoded architecture flags
provided by the interpreter when invoking "mpy-cross".

If architecture flags are detected, they're mapped into the necessary
strings needed by "mpy-cross"'s "-march-flags" argument, so that tests
will always use all available extensions reported by the target.

Currently this is limited to the RV32 platform, as it is the only one
that is making use of this facility as of now.  This also lets the QEMU
port remove forced arguments to "mpy-cross" when running the test suite.

Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
2025-10-24 19:13:15 +02:00
Alessandro Gatti
430837996b py/modsys: Add architecture flags to MicroPython metadata.
This commit adds the currently supported architecture flags value as the
upper part of "sys.implementation._mpy".

This had the side effect of perturbing quite a bit of testing
infrastructure and invalidating documentation related to MPY files.  To
make the test suite run successfully and keep the documentation in sync
the following changes have been made:

* The target info feature check file now isolates eventual architecture
  flags and adds them as a separate field
* The test runner now picks up the new architecture flags field, reports
  it to STDOUT if needed and stores it for future uses
* Relevant test files for MPY files import code had to be updated to
  mask out the architecture flags bits in order to perform correctly
* MPY file format documentation was updated to show how to mask off and
  properly display the architecture flags information.

This works out of the box if the flag bits can fit in a smallint value
once merged with the MPY file header value.

Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
2025-10-24 19:13:15 +02:00
Damien George
ea2fdebce2 tests/run-tests.py: Allow port:<dev> specification in run-multitests.py.
Some checks failed
JavaScript code lint and formatting with Biome / eslint (push) Has been cancelled
Check code formatting / code-formatting (push) Has been cancelled
Check spelling with codespell / codespell (push) Has been cancelled
Build docs / build (push) Has been cancelled
Check examples / embedding (push) Has been cancelled
Package mpremote / build (push) Has been cancelled
.mpy file format and tools / test (push) Has been cancelled
Build ports metadata / build (push) Has been cancelled
alif port / build_alif (alif_ae3_build) (push) Has been cancelled
cc3200 port / build (push) Has been cancelled
esp32 port / build_idf (esp32_build_c2_c6) (push) Has been cancelled
esp32 port / build_idf (esp32_build_cmod_spiram_s2) (push) Has been cancelled
esp32 port / build_idf (esp32_build_s3_c3) (push) Has been cancelled
esp8266 port / build (push) Has been cancelled
mimxrt port / build (push) Has been cancelled
nrf port / build (push) Has been cancelled
powerpc port / build (push) Has been cancelled
qemu port / build_and_test_arm (bigendian) (push) Has been cancelled
qemu port / build_and_test_arm (sabrelite) (push) Has been cancelled
qemu port / build_and_test_arm (thumb_hardfp) (push) Has been cancelled
qemu port / build_and_test_arm (thumb_softfp) (push) Has been cancelled
qemu port / build_and_test_rv32 (push) Has been cancelled
qemu port / build_and_test_rv64 (push) Has been cancelled
renesas-ra port / build_renesas_ra_board (push) Has been cancelled
rp2 port / build (push) Has been cancelled
samd port / build (push) Has been cancelled
stm32 port / build_stm32 (stm32_misc_build) (push) Has been cancelled
stm32 port / build_stm32 (stm32_nucleo_build) (push) Has been cancelled
stm32 port / build_stm32 (stm32_pyb_build) (push) Has been cancelled
unix port / minimal (push) Has been cancelled
unix port / reproducible (push) Has been cancelled
unix port / standard (push) Has been cancelled
unix port / standard_v2 (push) Has been cancelled
unix port / coverage (push) Has been cancelled
unix port / coverage_32bit (push) Has been cancelled
unix port / nanbox (push) Has been cancelled
unix port / longlong (push) Has been cancelled
unix port / float (push) Has been cancelled
unix port / gil_enabled (push) Has been cancelled
unix port / stackless_clang (push) Has been cancelled
unix port / float_clang (push) Has been cancelled
unix port / settrace_stackless (push) Has been cancelled
unix port / repr_b (push) Has been cancelled
unix port / macos (push) Has been cancelled
unix port / qemu_mips (push) Has been cancelled
unix port / qemu_arm (push) Has been cancelled
unix port / qemu_riscv64 (push) Has been cancelled
unix port / sanitize_address (push) Has been cancelled
unix port / sanitize_undefined (push) Has been cancelled
webassembly port / build (push) Has been cancelled
windows port / build-vs (Debug, true, x64, dev, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Debug, true, x86, dev, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Debug, x64, dev, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Debug, x86, dev, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Release, true, x64, dev, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Release, true, x64, dev, 2019, [16, 17)) (push) Has been cancelled
windows port / build-vs (Release, true, x64, standard, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Release, true, x64, standard, 2019, [16, 17)) (push) Has been cancelled
windows port / build-vs (Release, true, x86, dev, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Release, true, x86, dev, 2019, [16, 17)) (push) Has been cancelled
windows port / build-vs (Release, true, x86, standard, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Release, true, x86, standard, 2019, [16, 17)) (push) Has been cancelled
windows port / build-vs (Release, x64, dev, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Release, x64, standard, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Release, x86, dev, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Release, x86, standard, 2022, [17, 18)) (push) Has been cancelled
windows port / build-mingw (i686, mingw32, dev) (push) Has been cancelled
windows port / build-mingw (i686, mingw32, standard) (push) Has been cancelled
windows port / build-mingw (x86_64, mingw64, dev) (push) Has been cancelled
windows port / build-mingw (x86_64, mingw64, standard) (push) Has been cancelled
windows port / cross-build-on-linux (push) Has been cancelled
zephyr port / build (push) Has been cancelled
Python code lint and formatting with ruff / ruff (push) Has been cancelled
This moves the `port:<dev>` check for the test instance to the
`convert_device_shortcut_to_real_device()` helper function, which means
that `run-multitests.py` can use this specification.

Signed-off-by: Damien George <damien@micropython.org>
2025-10-20 13:59:31 +11:00
Damien George
c5c03e74ab tests/net_inet/tls_text_errors.py: Add exp file to run on axTLS.
axTLS will print a warning, and the added .exp file will skip that line.

Signed-off-by: Damien George <damien@micropython.org>
2025-10-20 12:06:39 +11:00
Damien George
60edfeeead tests/run-tests.py: Raise OSError on invalid file open for injected FS.
This makes sure that `open()` raises `OSError(ENOENT)` if the test script
attempts to open a file that does not exist.

For example, the test `tests/ports/esp32/check_err_str.py` attempts to do
this and this test currently fails when run with `--via-mpy` because the
injected filesystem lets a file by any name be opened.

The fix here tries to be as minimal as possible, so includes a size
optimisation for `stat()` to use a fixed 10-tuple.  It also fixes the
`OSError` argument, which should be positive.

Signed-off-by: Damien George <damien@micropython.org>
2025-10-20 11:18:59 +11:00
Damien George
1a96f72b5b tests/run-tests.py: Update list of tests that use float.
Signed-off-by: Damien George <damien@micropython.org>
2025-10-02 00:37:29 +10:00
Damien George
110bff9422 tests/run-tests.py: Don't include cmdline,io tests for minimal targets.
The unix minimal variant cannot run these tests because:
- it doesn't support -v -v debug printing
- it doesn't have `open()`

Signed-off-by: Damien George <damien@micropython.org>
2025-10-02 00:37:19 +10:00
Damien George
c176fe8702 tests/run-tests.py: Skip more tests that need slice.
Signed-off-by: Damien George <damien@micropython.org>
2025-10-01 23:59:05 +10:00
Damien George
72be75f477 tests/run-tests.py: Improve skip detection for tests using slice.
This skips some additional tests that use slice, and no longer skips
`basics/string_format_modulo.py` which doesn't actually use slice (but
`float/string_format_modulo.py` does).

Signed-off-by: Damien George <damien@micropython.org>
2025-10-01 23:59:05 +10:00
Damien George
4d905ef552 tests/run-tests.py: Skip certain tests when using --via-mpy.
These tests cannot pass when using `--via-mpy`.

Signed-off-by: Damien George <damien@micropython.org>
2025-10-01 11:14:46 +10:00
Alessandro Gatti
40dbf77415 py/emitinlinerv32: Add Zba opcodes to the inline assembler.
This commit adds support for Zba opcodes to the RV32 inline assembler.

Three new opcodes were added, SH1ADD, SH2ADD, and SH3ADD, which performs
a scaled addition (by 1, 2, or 3 bits respectively).  At the moment only
qemu's VIRT_RV32 and rp2's RPI_PICO2/RPI_PICO2_W ports support these
opcodes (the latter only when using the RISCV variant).

Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
2025-09-19 15:52:20 +02:00
Damien George
5bafb0bf68 tests: Convert all machine.UART tests to use target_wiring.
All the existing `machine.UART` tests in extmod and extmod_hardware are
converted to use the new `target_wiring` scheme, which removes a lot of
duplicated board-specific settings.

All the existing boards that were supported by these UART tests now have
their own `target_wiring` file.  Some configurations are board specific (eg
NUCLEO_WB55) and others are port specific.

Signed-off-by: Damien George <damien@micropython.org>
2025-09-08 11:52:49 +10:00
Damien George
3285e52127 tests/run-tests.py: Add support for board-specific target_wiring config.
There are currently a few "hardware" tests that need external board
connections to be made for them to work, such as bridging a pair of pins.
For example, all tests in `tests/extmod_hardware` need some kind of
connection.

Along with the physical connections -- which are different for each board
-- there needs to be corresponding Python settings, eg which UART instance
to use and which pins for TX/RX.

These settings are currently hard-coded in each test file. That has a few
problems:
- settings are repeated, eg all the UART tests have pretty much the same
  settings code duplicated across them
- changing the settings means changing many files
- adding a new board means adding a lot of code
- the tests get bigger and bigger with each new board that they support,
  meaning they may not fit on targets with a small amount of RAM (that's
  already the case with `tests/extmod_hardware/machine_pwm.py`)
- if you have a custom board you have to manually edit the test to add your
  own settings

This commit aims to solve all the above problems by splitting the board-
specific settings out into separate files, one for each board (or port).
They are placed in the `tests/target_wiring/` directory.  The
`run-tests.py` test runner then loads the appropriate configuration for the
target that is being tested, sends it to the board so it's available as
`import target_wiring` (without needing a filesystem), and then executes
the test.  Note tht only tests that need `target_wiring` have it loaded.

This commit adds support for this `target_wiring` scheme.

Signed-off-by: Damien George <damien@micropython.org>
2025-09-08 11:24:32 +10:00
Damien George
4a60d230c3 tests/run-tests.py: Detect target sys.implementation._build if possible.
That's useful to know the exact board that's being tested.

Signed-off-by: Damien George <damien@micropython.org>
2025-09-08 11:24:32 +10:00
Damien George
7a8ae72640 tests/run-tests.py: Factor code for device shortcuts to a function.
So it can be reused by other test runners.

Signed-off-by: Damien George <damien@micropython.org>
2025-08-28 23:11:30 +10:00
Alessandro Gatti
c33a02fe6d tests/run-tests.py: Enable Arm inlineasm FPU tests if possible.
Some checks failed
JavaScript code lint and formatting with Biome / eslint (push) Has been cancelled
Check code formatting / code-formatting (push) Has been cancelled
Check spelling with codespell / codespell (push) Has been cancelled
Build docs / build (push) Has been cancelled
Check examples / embedding (push) Has been cancelled
Package mpremote / build (push) Has been cancelled
.mpy file format and tools / test (push) Has been cancelled
Build ports metadata / build (push) Has been cancelled
alif port / build_alif (alif_ae3_build) (push) Has been cancelled
cc3200 port / build (push) Has been cancelled
esp32 port / build_idf (esp32_build_c2_c6) (push) Has been cancelled
esp32 port / build_idf (esp32_build_cmod_spiram_s2) (push) Has been cancelled
esp32 port / build_idf (esp32_build_s3_c3) (push) Has been cancelled
esp8266 port / build (push) Has been cancelled
mimxrt port / build (push) Has been cancelled
nrf port / build (push) Has been cancelled
powerpc port / build (push) Has been cancelled
qemu port / build_and_test_arm (bigendian) (push) Has been cancelled
qemu port / build_and_test_arm (sabrelite) (push) Has been cancelled
qemu port / build_and_test_arm (thumb_hardfp) (push) Has been cancelled
qemu port / build_and_test_arm (thumb_softfp) (push) Has been cancelled
qemu port / build_and_test_rv32 (push) Has been cancelled
renesas-ra port / build_renesas_ra_board (push) Has been cancelled
rp2 port / build (push) Has been cancelled
samd port / build (push) Has been cancelled
stm32 port / build_stm32 (stm32_misc_build) (push) Has been cancelled
stm32 port / build_stm32 (stm32_nucleo_build) (push) Has been cancelled
stm32 port / build_stm32 (stm32_pyb_build) (push) Has been cancelled
unix port / minimal (push) Has been cancelled
unix port / reproducible (push) Has been cancelled
unix port / standard (push) Has been cancelled
unix port / standard_v2 (push) Has been cancelled
unix port / coverage (push) Has been cancelled
unix port / coverage_32bit (push) Has been cancelled
unix port / nanbox (push) Has been cancelled
unix port / longlong (push) Has been cancelled
unix port / float (push) Has been cancelled
unix port / gil_enabled (push) Has been cancelled
unix port / stackless_clang (push) Has been cancelled
unix port / float_clang (push) Has been cancelled
unix port / settrace_stackless (push) Has been cancelled
unix port / macos (push) Has been cancelled
unix port / qemu_mips (push) Has been cancelled
unix port / qemu_arm (push) Has been cancelled
unix port / qemu_riscv64 (push) Has been cancelled
unix port / sanitize_address (push) Has been cancelled
unix port / sanitize_undefined (push) Has been cancelled
webassembly port / build (push) Has been cancelled
windows port / build-vs (Debug, x64, dev, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Debug, x64, dev, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Debug, x86, dev, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Debug, x86, dev, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Release, x64, dev, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Release, x64, dev, 2019, [16, 17)) (push) Has been cancelled
windows port / build-vs (Release, x64, dev, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Release, x64, standard, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Release, x64, standard, 2019, [16, 17)) (push) Has been cancelled
windows port / build-vs (Release, x64, standard, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Release, x86, dev, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Release, x86, dev, 2019, [16, 17)) (push) Has been cancelled
windows port / build-vs (Release, x86, dev, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Release, x86, standard, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Release, x86, standard, 2019, [16, 17)) (push) Has been cancelled
windows port / build-vs (Release, x86, standard, 2022, [17, 18)) (push) Has been cancelled
windows port / build-mingw (i686, mingw32, dev) (push) Has been cancelled
windows port / build-mingw (i686, mingw32, standard) (push) Has been cancelled
windows port / build-mingw (x86_64, mingw64, dev) (push) Has been cancelled
windows port / build-mingw (x86_64, mingw64, standard) (push) Has been cancelled
windows port / cross-build-on-linux (push) Has been cancelled
zephyr port / build (push) Has been cancelled
Python code lint and formatting with ruff / ruff (push) Has been cancelled
This commits lifts the unconditional restriction on inline assembler FPU
tests for the Qemu platform, and makes said restriction conditional to
the lack of an available floating point unit on the running platform.

The Qemu platform supported only emulated machines that could target up
to a Cortex-M3, so an ArmV7-M target that had no support for floating
point.  With the addition of MPS2_AN500 to the list of emulated targets
the range was extended to cover up to Cortex-M7, so a floating point
unit may possibly be available and thus able to run the FPU inlineasm
tests.

For that, the test runner was changed to detect the running architecture
when checking the target capabilities; if the target reports its
MicroPython architecture to be either "armv7emsp" or "armv7emdp"
(providing single-precision and double-precision floating point unit
support respectively) then the FPU-only inline tests are not put into
the blocked tests list.

Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
2025-08-26 12:27:21 +10:00
Damien George
a279c64046 tests: Add .native.exp output files for tests that differ with native.
Signed-off-by: Damien George <damien@micropython.org>
2025-08-15 00:36:27 +10:00
Damien George
3c72c3a1e6 tests/micropython/opt_level_lineno.py: Force test func to use bytecode.
So that the test can run the same on all targets when used with the native
emitter.

Signed-off-by: Damien George <damien@micropython.org>
2025-08-15 00:36:27 +10:00
Damien George
0f5f6484a2 tests/run-tests.py: Add support for .native.exp expected output files.
There are currently a few tests that are excluded when using the native
emitter because they test printing of exception tracebacks, which includes
line numbers.  And the native emitter doesn't store line numbers, so gets
these tests wrong.

But we'd still like to run these tests using the native emitter, because
they test useful things even if the line number info is not in the
traceback (eg that threads which crash print out their exception).

This commit adds support for native-specific .exp files, which are of the
form `<test>.py.native.exp`.  If such an .exp file exists then it take
precedence over any normal `<test>.py.exp` file.

(Actually, the implementation here is general enough that it also supports
`<test>.py.bytecode.exp` as well, if bytecode ever needs a specific exp
file.)

Signed-off-by: Damien George <damien@micropython.org>
2025-08-15 00:36:27 +10:00
Damien George
54e6cfc6e3 tests/basics: Skip tests of io module individually using SKIP.
Instead of using a feature check.  This is more consistent with how other
optional modules are skipped.

Signed-off-by: Damien George <damien@micropython.org>
2025-08-15 00:23:16 +10:00
Damien George
e2744ce679 tests/run-tests.py: Autodetect if the target has unicode support.
The unicode tests are now run on all targets that enable unicode.  And
other unicode tests (namely `extmod/json_loads.py`) are now properly
skipped if the target doesn't have unicode support.

Signed-off-by: Damien George <damien@micropython.org>
2025-08-15 00:23:16 +10:00
Damien George
1db71f9e55 tests/run-tests.py: Generalise addition of port specific test directory.
Signed-off-by: Damien George <damien@micropython.org>
2025-08-15 00:23:16 +10:00
Damien George
6565827955 tests/run-tests.py: Always include stress/ tests directory in tests.
Ports that now run the stress tests, that didn't prior to this commit are:
cc3200, esp8266, minimal, nrf, renesas-ra, samd, qemu, webassembly.

Signed-off-by: Damien George <damien@micropython.org>
2025-08-15 00:23:16 +10:00
Damien George
f493075d88 tests/run-tests.py: Automatically include float tests when possible.
This simplifies the code by removing the explicit addition of the "float/"
test directory for certain targets.  It also means the tests won't be added
incorrectly, eg on a unix build without float.

Signed-off-by: Damien George <damien@micropython.org>
2025-08-15 00:23:16 +10:00
Damien George
1401fdb8b3 tests/run-tests.py: Run tests-with-regex-output as normal tests.
Some checks failed
JavaScript code lint and formatting with Biome / eslint (push) Has been cancelled
Check code formatting / code-formatting (push) Has been cancelled
Check spelling with codespell / codespell (push) Has been cancelled
Build docs / build (push) Has been cancelled
Check examples / embedding (push) Has been cancelled
Package mpremote / build (push) Has been cancelled
.mpy file format and tools / test (push) Has been cancelled
Build ports metadata / build (push) Has been cancelled
alif port / build_alif (alif_ae3_build) (push) Has been cancelled
cc3200 port / build (push) Has been cancelled
esp32 port / build_idf (esp32_build_c2_c6) (push) Has been cancelled
esp32 port / build_idf (esp32_build_cmod_spiram_s2) (push) Has been cancelled
esp32 port / build_idf (esp32_build_s3_c3) (push) Has been cancelled
esp8266 port / build (push) Has been cancelled
mimxrt port / build (push) Has been cancelled
nrf port / build (push) Has been cancelled
powerpc port / build (push) Has been cancelled
qemu port / build_and_test_arm (bigendian) (push) Has been cancelled
qemu port / build_and_test_arm (sabrelite) (push) Has been cancelled
qemu port / build_and_test_arm (thumb) (push) Has been cancelled
qemu port / build_and_test_rv32 (push) Has been cancelled
renesas-ra port / build_renesas_ra_board (push) Has been cancelled
rp2 port / build (push) Has been cancelled
samd port / build (push) Has been cancelled
stm32 port / build_stm32 (stm32_misc_build) (push) Has been cancelled
stm32 port / build_stm32 (stm32_nucleo_build) (push) Has been cancelled
stm32 port / build_stm32 (stm32_pyb_build) (push) Has been cancelled
unix port / minimal (push) Has been cancelled
unix port / reproducible (push) Has been cancelled
unix port / standard (push) Has been cancelled
unix port / standard_v2 (push) Has been cancelled
unix port / coverage (push) Has been cancelled
unix port / coverage_32bit (push) Has been cancelled
unix port / nanbox (push) Has been cancelled
unix port / longlong (push) Has been cancelled
unix port / float (push) Has been cancelled
unix port / gil_enabled (push) Has been cancelled
unix port / stackless_clang (push) Has been cancelled
unix port / float_clang (push) Has been cancelled
unix port / settrace_stackless (push) Has been cancelled
unix port / macos (push) Has been cancelled
unix port / qemu_mips (push) Has been cancelled
unix port / qemu_arm (push) Has been cancelled
unix port / qemu_riscv64 (push) Has been cancelled
unix port / sanitize_address (push) Has been cancelled
unix port / sanitize_undefined (push) Has been cancelled
webassembly port / build (push) Has been cancelled
windows port / build-vs (Debug, x64, dev, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Debug, x64, dev, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Debug, x86, dev, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Debug, x86, dev, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Release, x64, dev, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Release, x64, dev, 2019, [16, 17)) (push) Has been cancelled
windows port / build-vs (Release, x64, dev, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Release, x64, standard, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Release, x64, standard, 2019, [16, 17)) (push) Has been cancelled
windows port / build-vs (Release, x64, standard, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Release, x86, dev, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Release, x86, dev, 2019, [16, 17)) (push) Has been cancelled
windows port / build-vs (Release, x86, dev, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Release, x86, standard, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Release, x86, standard, 2019, [16, 17)) (push) Has been cancelled
windows port / build-vs (Release, x86, standard, 2022, [17, 18)) (push) Has been cancelled
windows port / build-mingw (i686, mingw32, dev) (push) Has been cancelled
windows port / build-mingw (i686, mingw32, standard) (push) Has been cancelled
windows port / build-mingw (x86_64, mingw64, dev) (push) Has been cancelled
windows port / build-mingw (x86_64, mingw64, standard) (push) Has been cancelled
windows port / cross-build-on-linux (push) Has been cancelled
zephyr port / build (push) Has been cancelled
Python code lint and formatting with ruff / ruff (push) Has been cancelled
Some tests (currently given by the `special_tests` list) have output which
must be mached via a regex, because it can change from run to run (eg the
address of an object is printed).  These tests are currently classified as
`is_special` in the test runner, which means they get special treatment.
In particular they don't set the emitter as specified by `args.emit`.  That
means these tests do not run via .mpy or using the native emitter, even if
those options are given on the command line.

This commit fixes that by considering `is_special` as different to
`tests_with_regex_output`.  The former is used for things like target
feature detection (which are not really tests) and when extra command line
options need to be passed to the unix micropython executable.  The latter
(now called `tests_with_regex_output`) are specifically for tests that have
output to be matched via regex.

The `thread_exc2.py` test now needs to be excluded when running using the
native emitter, because the native emitter doesn't print traceback info.
And the `sys_settrace_cov.py` test needs to be excluded because set-trace
output is different with the native emitter.

Signed-off-by: Damien George <damien@micropython.org>
2025-08-10 11:40:54 +10:00
Damien George
c35427cbdb tests/run-tests.py: Move tests to skip with native emitter to a list.
This makes `run-tests.py` a little more organised, by putting all the
tests-to-skip-when-using-the-native-emitter in a dedicated list.

This should make it easier to maintain the list, and understand why a test
is there.

Signed-off-by: Damien George <damien@micropython.org>
2025-08-10 11:25:38 +10:00
Yoctopuce dev
dbbaa959c8 py/formatfloat: Improve accuracy of float formatting code.
Some checks failed
JavaScript code lint and formatting with Biome / eslint (push) Has been cancelled
Check code formatting / code-formatting (push) Has been cancelled
Check spelling with codespell / codespell (push) Has been cancelled
Build docs / build (push) Has been cancelled
Check examples / embedding (push) Has been cancelled
Package mpremote / build (push) Has been cancelled
.mpy file format and tools / test (push) Has been cancelled
Build ports metadata / build (push) Has been cancelled
alif port / build_alif (alif_ae3_build) (push) Has been cancelled
cc3200 port / build (push) Has been cancelled
esp32 port / build_idf (esp32_build_cmod_spiram_s2) (push) Has been cancelled
esp32 port / build_idf (esp32_build_s3_c3) (push) Has been cancelled
esp8266 port / build (push) Has been cancelled
mimxrt port / build (push) Has been cancelled
nrf port / build (push) Has been cancelled
powerpc port / build (push) Has been cancelled
qemu port / build_and_test_arm (bigendian) (push) Has been cancelled
qemu port / build_and_test_arm (sabrelite) (push) Has been cancelled
qemu port / build_and_test_arm (thumb) (push) Has been cancelled
qemu port / build_and_test_rv32 (push) Has been cancelled
renesas-ra port / build_renesas_ra_board (push) Has been cancelled
rp2 port / build (push) Has been cancelled
samd port / build (push) Has been cancelled
stm32 port / build_stm32 (stm32_misc_build) (push) Has been cancelled
stm32 port / build_stm32 (stm32_nucleo_build) (push) Has been cancelled
stm32 port / build_stm32 (stm32_pyb_build) (push) Has been cancelled
unix port / minimal (push) Has been cancelled
unix port / reproducible (push) Has been cancelled
unix port / standard (push) Has been cancelled
unix port / standard_v2 (push) Has been cancelled
unix port / coverage (push) Has been cancelled
unix port / coverage_32bit (push) Has been cancelled
unix port / nanbox (push) Has been cancelled
unix port / longlong (push) Has been cancelled
unix port / float (push) Has been cancelled
unix port / gil_enabled (push) Has been cancelled
unix port / stackless_clang (push) Has been cancelled
unix port / float_clang (push) Has been cancelled
unix port / settrace_stackless (push) Has been cancelled
unix port / macos (push) Has been cancelled
unix port / qemu_mips (push) Has been cancelled
unix port / qemu_arm (push) Has been cancelled
unix port / qemu_riscv64 (push) Has been cancelled
unix port / sanitize_address (push) Has been cancelled
unix port / sanitize_undefined (push) Has been cancelled
webassembly port / build (push) Has been cancelled
windows port / build-vs (Debug, x64, dev, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Debug, x64, dev, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Debug, x86, dev, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Debug, x86, dev, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Release, x64, dev, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Release, x64, dev, 2019, [16, 17)) (push) Has been cancelled
windows port / build-vs (Release, x64, dev, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Release, x64, standard, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Release, x64, standard, 2019, [16, 17)) (push) Has been cancelled
windows port / build-vs (Release, x64, standard, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Release, x86, dev, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Release, x86, dev, 2019, [16, 17)) (push) Has been cancelled
windows port / build-vs (Release, x86, dev, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Release, x86, standard, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Release, x86, standard, 2019, [16, 17)) (push) Has been cancelled
windows port / build-vs (Release, x86, standard, 2022, [17, 18)) (push) Has been cancelled
windows port / build-mingw (i686, mingw32, dev) (push) Has been cancelled
windows port / build-mingw (i686, mingw32, standard) (push) Has been cancelled
windows port / build-mingw (x86_64, mingw64, dev) (push) Has been cancelled
windows port / build-mingw (x86_64, mingw64, standard) (push) Has been cancelled
windows port / cross-build-on-linux (push) Has been cancelled
zephyr port / build (push) Has been cancelled
Python code lint and formatting with ruff / ruff (push) Has been cancelled
Following discussions in PR #16666, this commit updates the float
formatting code to improve the `repr` reversibility, i.e. the percentage of
valid floating point numbers that do parse back to the same number when
formatted by `repr` (in CPython it's 100%).

This new code offers a choice of 3 float conversion methods, depending on
the desired tradeoff between code size and conversion precision:

- BASIC method is the smallest code footprint

- APPROX method uses an iterative method to approximate the exact
  representation, which is a bit slower but but does not have a big impact
  on code size.  It provides `repr` reversibility on >99.8% of the cases in
  double precision, and on >98.5% in single precision (except with REPR_C,
  where reversibility is 100% as the last two bits are not taken into
  account).

- EXACT method uses higher-precision floats during conversion, which
  provides perfect results but has a higher impact on code size.  It is
  faster than APPROX method, and faster than the CPython equivalent
  implementation.  It is however not available on all compilers when using
  FLOAT_IMPL_DOUBLE.

Here is the table comparing the impact of the three conversion methods on
code footprint on PYBV10 (using single-precision floats) and reversibility
rate for both single-precision and double-precision floats.  The table
includes current situation as a baseline for the comparison:

              PYBV10  REPR_C   FLOAT  DOUBLE
    current = 364688   12.9%   27.6%   37.9%
    basic   = 364812   85.6%   60.5%   85.7%
    approx  = 365080  100.0%   98.5%   99.8%
    exact   = 366408  100.0%  100.0%  100.0%

Signed-off-by: Yoctopuce dev <dev@yoctopuce.com>
2025-08-01 00:47:33 +10:00
Anson Mansfield
88cb6bc818 tests/run-internalbench.py: Allow running internalbench on hardware.
Some checks failed
JavaScript code lint and formatting with Biome / eslint (push) Has been cancelled
Check code formatting / code-formatting (push) Has been cancelled
Check spelling with codespell / codespell (push) Has been cancelled
Build docs / build (push) Has been cancelled
Check examples / embedding (push) Has been cancelled
Package mpremote / build (push) Has been cancelled
.mpy file format and tools / test (push) Has been cancelled
Build ports metadata / build (push) Has been cancelled
alif port / build_alif (alif_ae3_build) (push) Has been cancelled
cc3200 port / build (push) Has been cancelled
esp32 port / build_idf (esp32_build_cmod_spiram_s2) (push) Has been cancelled
esp32 port / build_idf (esp32_build_s3_c3) (push) Has been cancelled
esp8266 port / build (push) Has been cancelled
mimxrt port / build (push) Has been cancelled
nrf port / build (push) Has been cancelled
powerpc port / build (push) Has been cancelled
qemu port / build_and_test_arm (bigendian) (push) Has been cancelled
qemu port / build_and_test_arm (sabrelite) (push) Has been cancelled
qemu port / build_and_test_arm (thumb) (push) Has been cancelled
qemu port / build_and_test_rv32 (push) Has been cancelled
renesas-ra port / build_renesas_ra_board (push) Has been cancelled
rp2 port / build (push) Has been cancelled
samd port / build (push) Has been cancelled
stm32 port / build_stm32 (stm32_misc_build) (push) Has been cancelled
stm32 port / build_stm32 (stm32_nucleo_build) (push) Has been cancelled
stm32 port / build_stm32 (stm32_pyb_build) (push) Has been cancelled
unix port / minimal (push) Has been cancelled
unix port / reproducible (push) Has been cancelled
unix port / standard (push) Has been cancelled
unix port / standard_v2 (push) Has been cancelled
unix port / coverage (push) Has been cancelled
unix port / coverage_32bit (push) Has been cancelled
unix port / nanbox (push) Has been cancelled
unix port / longlong (push) Has been cancelled
unix port / float (push) Has been cancelled
unix port / gil_enabled (push) Has been cancelled
unix port / stackless_clang (push) Has been cancelled
unix port / float_clang (push) Has been cancelled
unix port / settrace_stackless (push) Has been cancelled
unix port / macos (push) Has been cancelled
unix port / qemu_mips (push) Has been cancelled
unix port / qemu_arm (push) Has been cancelled
unix port / qemu_riscv64 (push) Has been cancelled
unix port / sanitize_address (push) Has been cancelled
unix port / sanitize_undefined (push) Has been cancelled
webassembly port / build (push) Has been cancelled
windows port / build-vs (Debug, x64, dev, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Debug, x64, dev, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Debug, x86, dev, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Debug, x86, dev, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Release, x64, dev, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Release, x64, dev, 2019, [16, 17)) (push) Has been cancelled
windows port / build-vs (Release, x64, dev, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Release, x64, standard, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Release, x64, standard, 2019, [16, 17)) (push) Has been cancelled
windows port / build-vs (Release, x64, standard, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Release, x86, dev, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Release, x86, dev, 2019, [16, 17)) (push) Has been cancelled
windows port / build-vs (Release, x86, dev, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Release, x86, standard, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Release, x86, standard, 2019, [16, 17)) (push) Has been cancelled
windows port / build-vs (Release, x86, standard, 2022, [17, 18)) (push) Has been cancelled
windows port / build-mingw (i686, mingw32, dev) (push) Has been cancelled
windows port / build-mingw (i686, mingw32, standard) (push) Has been cancelled
windows port / build-mingw (x86_64, mingw64, dev) (push) Has been cancelled
windows port / build-mingw (x86_64, mingw64, standard) (push) Has been cancelled
windows port / cross-build-on-linux (push) Has been cancelled
zephyr port / build (push) Has been cancelled
Python code lint and formatting with ruff / ruff (push) Has been cancelled
Signed-off-by: Anson Mansfield <amansfield@mantaro.com>
2025-07-29 11:49:10 +10:00
Damien George
7729e80fdd all: Go back to using default ruff quote style.
Commit dc2fcfcc55 seems to have accidentally
changed the ruff quote style to "preserve", instead of keeping it at the
default which is "double".

Put it back to the default and update relevant .py files with this rule.

Signed-off-by: Damien George <damien@micropython.org>
2025-07-24 12:48:18 +10:00
Yoctopuce dev
d6876e2273 py/obj: Fix REPR_C bias toward zero.
Current implementation of REPR_C works by clearing the two lower bits of
the mantissa to zero.  As this happens after each floating point operation,
this tends to bias floating point numbers towards zero, causing decimals
like .9997 instead of rounded numbers.  This is visible in test cases
involving repeated computations, such as `tests/misc/rge_sm.py` for
instance.

The suggested fix fills in the missing bits by copying the previous two
bits.  Although this cannot recreate missing information, it fixes the bias
by inserting plausible values for the lost bits, at a relatively low cost.

Some float tests involving irrational numbers have to be softened in case
of REPR_C, as the 30 bits are not always enough to fulfill the expectations
of the original test, and the change may randomly affect the last digits.
Such cases have been made explicit by testing for REPR_C or by adding a
clear comment.

The perf_test fft code was also missing a call to round() before casting a
log_2 operation to int, which was causing a failure due to a last-decimal
change.

Signed-off-by: Yoctopuce dev <dev@yoctopuce.com>
2025-07-24 11:07:30 +10:00
Damien George
941b7e35ac tests/run-tests.py: Use TEST_TIMEOUT as timeout for bare-metal tests.
This parameter is already used for PC-based tests (eg unix and webassembly
ports), and it makes sense for it to be used for bare-metal ports as well.
That way the timeout is configurable for all targets.

Because this increases the default timeout from 10s to 30s, this fixes some
long-running tests that would previously fail due to a timeout such as
`thread/stress_aes.py` on ESP32.

Signed-off-by: Damien George <damien@micropython.org>
2025-07-23 12:42:44 +10:00
Damien George
167c888df9 tests/run-tests.py: Detect threading and automatically run thread tests.
When detecting the target platform, also check if it has threading and
whether the GIL is enabled or not (using the new attribute
`sys.implementation._thread`).  If threading is available, add the thread
tests to the set of tests to run (unless the set of tests is explicitly
given).

With this change, the unix port no longer needs to explicitly run the set
of thread tests, so that line has been removed from the Makefile.

This change will make sure thread tests are run with other testing
combinations.  In particular, thread tests are now run:
- on the unix port with the native emitter
- on macOS builds
- on unix qemu, the architectures MIPS, ARM and RISCV-64

Signed-off-by: Damien George <damien@micropython.org>
2025-07-23 11:06:09 +10:00
Angus Gratton
2d8d64059f tests: Add specific tests for "long long" 64-bit bigints.
These will run on all ports which support them, but importantly
they'll also run on ports that don't support arbitrary precision
but do support 64-bit long ints.

Includes some test workarounds to account for things which will overflow
once "long long" big integers overflow (added in follow-up commit):

- uctypes_array_load_store test was failing already, now won't parse.
- all the ffi_int tests contain 64-bit unsigned values, that won't parse
  as long long.

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <angus@redyak.com.au>
2025-07-18 00:10:44 +10:00
Jim Mussared
5e9189d6d1 py/vm: Avoid heap-allocating slices when subscripting built-ins.
Some checks failed
JavaScript code lint and formatting with Biome / eslint (push) Has been cancelled
Check code formatting / code-formatting (push) Has been cancelled
Check spelling with codespell / codespell (push) Has been cancelled
Build docs / build (push) Has been cancelled
Check examples / embedding (push) Has been cancelled
Package mpremote / build (push) Has been cancelled
.mpy file format and tools / test (push) Has been cancelled
Build ports metadata / build (push) Has been cancelled
alif port / build_alif (alif_ae3_build) (push) Has been cancelled
cc3200 port / build (push) Has been cancelled
esp32 port / build_idf (esp32_build_cmod_spiram_s2) (push) Has been cancelled
esp32 port / build_idf (esp32_build_s3_c3) (push) Has been cancelled
esp8266 port / build (push) Has been cancelled
mimxrt port / build (push) Has been cancelled
nrf port / build (push) Has been cancelled
powerpc port / build (push) Has been cancelled
qemu port / build_and_test_arm (bigendian) (push) Has been cancelled
qemu port / build_and_test_arm (sabrelite) (push) Has been cancelled
qemu port / build_and_test_arm (thumb) (push) Has been cancelled
qemu port / build_and_test_rv32 (push) Has been cancelled
renesas-ra port / build_renesas_ra_board (push) Has been cancelled
rp2 port / build (push) Has been cancelled
samd port / build (push) Has been cancelled
stm32 port / build_stm32 (stm32_misc_build) (push) Has been cancelled
stm32 port / build_stm32 (stm32_nucleo_build) (push) Has been cancelled
stm32 port / build_stm32 (stm32_pyb_build) (push) Has been cancelled
unix port / minimal (push) Has been cancelled
unix port / reproducible (push) Has been cancelled
unix port / standard (push) Has been cancelled
unix port / standard_v2 (push) Has been cancelled
unix port / coverage (push) Has been cancelled
unix port / coverage_32bit (push) Has been cancelled
unix port / nanbox (push) Has been cancelled
unix port / float (push) Has been cancelled
unix port / stackless_clang (push) Has been cancelled
unix port / float_clang (push) Has been cancelled
unix port / settrace_stackless (push) Has been cancelled
unix port / macos (push) Has been cancelled
unix port / qemu_mips (push) Has been cancelled
unix port / qemu_arm (push) Has been cancelled
unix port / qemu_riscv64 (push) Has been cancelled
unix port / sanitize_address (push) Has been cancelled
unix port / sanitize_undefined (push) Has been cancelled
webassembly port / build (push) Has been cancelled
windows port / build-vs (Debug, x64, dev, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Debug, x64, dev, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Debug, x86, dev, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Debug, x86, dev, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Release, x64, dev, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Release, x64, dev, 2019, [16, 17)) (push) Has been cancelled
windows port / build-vs (Release, x64, dev, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Release, x64, standard, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Release, x64, standard, 2019, [16, 17)) (push) Has been cancelled
windows port / build-vs (Release, x64, standard, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Release, x86, dev, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Release, x86, dev, 2019, [16, 17)) (push) Has been cancelled
windows port / build-vs (Release, x86, dev, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Release, x86, standard, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Release, x86, standard, 2019, [16, 17)) (push) Has been cancelled
windows port / build-vs (Release, x86, standard, 2022, [17, 18)) (push) Has been cancelled
windows port / build-mingw (i686, mingw32, dev) (push) Has been cancelled
windows port / build-mingw (i686, mingw32, standard) (push) Has been cancelled
windows port / build-mingw (x86_64, mingw64, dev) (push) Has been cancelled
windows port / build-mingw (x86_64, mingw64, standard) (push) Has been cancelled
windows port / cross-build-on-linux (push) Has been cancelled
zephyr port / build (push) Has been cancelled
Python code lint and formatting with ruff / ruff (push) Has been cancelled
This commit adds a fast-path optimisation for when a BUILD_SLICE is
immediately followed by a LOAD/STORE_SUBSCR for a native type, to avoid
needing to allocate the slice on the heap.

In some cases (e.g. `a[1:3] = x`) this can result in no allocations at all.

We can't do this for instance types because the get/set/delattr
implementation may keep a reference to the slice.

Adds more tests to the basic slice tests to ensure that a stack-allocated
slice never makes it to Python, and also a heapalloc test that verifies
(when using bytecode) that assigning to a slice is no-alloc.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Signed-off-by: Damien George <damien@micropython.org>
2025-07-16 00:12:47 +10:00
Damien George
8f8f853982 tests/run-tests.py: Consider tests ending in _async.py as async tests.
Some checks failed
JavaScript code lint and formatting with Biome / eslint (push) Has been cancelled
Check code formatting / code-formatting (push) Has been cancelled
Check spelling with codespell / codespell (push) Has been cancelled
Build docs / build (push) Has been cancelled
Check examples / embedding (push) Has been cancelled
Package mpremote / build (push) Has been cancelled
.mpy file format and tools / test (push) Has been cancelled
Build ports metadata / build (push) Has been cancelled
alif port / build_alif (alif_ae3_build) (push) Has been cancelled
cc3200 port / build (push) Has been cancelled
esp32 port / build_idf (esp32_build_cmod_spiram_s2) (push) Has been cancelled
esp32 port / build_idf (esp32_build_s3_c3) (push) Has been cancelled
esp8266 port / build (push) Has been cancelled
mimxrt port / build (push) Has been cancelled
nrf port / build (push) Has been cancelled
powerpc port / build (push) Has been cancelled
qemu port / build_and_test_arm (bigendian) (push) Has been cancelled
qemu port / build_and_test_arm (sabrelite) (push) Has been cancelled
qemu port / build_and_test_arm (thumb) (push) Has been cancelled
qemu port / build_and_test_rv32 (push) Has been cancelled
renesas-ra port / build_renesas_ra_board (push) Has been cancelled
rp2 port / build (push) Has been cancelled
samd port / build (push) Has been cancelled
stm32 port / build_stm32 (stm32_misc_build) (push) Has been cancelled
stm32 port / build_stm32 (stm32_nucleo_build) (push) Has been cancelled
stm32 port / build_stm32 (stm32_pyb_build) (push) Has been cancelled
unix port / minimal (push) Has been cancelled
unix port / reproducible (push) Has been cancelled
unix port / standard (push) Has been cancelled
unix port / standard_v2 (push) Has been cancelled
unix port / coverage (push) Has been cancelled
unix port / coverage_32bit (push) Has been cancelled
unix port / nanbox (push) Has been cancelled
unix port / float (push) Has been cancelled
unix port / stackless_clang (push) Has been cancelled
unix port / float_clang (push) Has been cancelled
unix port / settrace_stackless (push) Has been cancelled
unix port / macos (push) Has been cancelled
unix port / qemu_mips (push) Has been cancelled
unix port / qemu_arm (push) Has been cancelled
unix port / qemu_riscv64 (push) Has been cancelled
unix port / sanitize_address (push) Has been cancelled
unix port / sanitize_undefined (push) Has been cancelled
webassembly port / build (push) Has been cancelled
windows port / build-vs (Debug, x64, dev, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Debug, x64, dev, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Debug, x86, dev, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Debug, x86, dev, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Release, x64, dev, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Release, x64, dev, 2019, [16, 17)) (push) Has been cancelled
windows port / build-vs (Release, x64, dev, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Release, x64, standard, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Release, x64, standard, 2019, [16, 17)) (push) Has been cancelled
windows port / build-vs (Release, x64, standard, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Release, x86, dev, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Release, x86, dev, 2019, [16, 17)) (push) Has been cancelled
windows port / build-vs (Release, x86, dev, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Release, x86, standard, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Release, x86, standard, 2019, [16, 17)) (push) Has been cancelled
windows port / build-vs (Release, x86, standard, 2022, [17, 18)) (push) Has been cancelled
windows port / build-mingw (i686, mingw32, dev) (push) Has been cancelled
windows port / build-mingw (i686, mingw32, standard) (push) Has been cancelled
windows port / build-mingw (x86_64, mingw64, dev) (push) Has been cancelled
windows port / build-mingw (x86_64, mingw64, standard) (push) Has been cancelled
windows port / cross-build-on-linux (push) Has been cancelled
zephyr port / build (push) Has been cancelled
Python code lint and formatting with ruff / ruff (push) Has been cancelled
The test `micropython/ringio_async.py` is a test that requires async
keyword support, and will fail with SyntaxError on targets that don't
support async/await.  Really it should be skipped on such targets, and this
commit makes sure that's the case.

Signed-off-by: Damien George <damien@micropython.org>
2025-07-12 23:48:17 +10:00
Jeff Epler
a8c2b917e2 tools/ci.sh: Increase test timeout to 60s in coverage jobs.
The additional overhead of the settrace profiler means that the
`aes_stress.py` test was running too slowly on GitHub CI.  Double the
timeout to 60 seconds.

Signed-off-by: Jeff Epler <jepler@gmail.com>
2025-07-06 00:08:13 +10:00
Jeff Epler
ff8c4e5943 tests/misc: Improve test coverage of py/profile.c.
Signed-off-by: Jeff Epler <jepler@gmail.com>
2025-07-06 00:05:36 +10:00
Alessandro Gatti
6fd7422a8e tests/run-tests.py: Allow injected code customisation.
This commit introduces a mechanism to customise the code that is
injected to the board when performing a test file upload and execution.

A new argument, --begin", is added so regular Python code can be
inserted in the injected fragment between the module file creation and
the effective file import.  This is needed for running larger tests
(usually ones that have been pre-compiled with
"--via-mpy --emit native") on ESP8266, as that board does not have
enough memory to fit certain blocks of code unless additional
configuration is performed.

Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
2025-07-01 15:43:17 +10:00
Andrew Leech
5e965618be tests/run-tests.py: Add support for ctrl keys in REPL tests.
This allows having {\xDD} in tests, which will be expanded to the given
hex character.

Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>
2025-06-19 17:23:39 +10:00
Damien George
2bc5af6945 tests/run-tests.py: Factor out helper function to create test report.
This commit factors existing code in `run-tests.py` into a new helper
function `create_test_report()`.  That function prints out a summary of the
test run (eg number of tests passed, number failed, number skipped) and
creates the corresponding `_results.json` file.

This is done so `create_test_report()` can be reused by the other test
runners.

The `test_count` counter is now gone, and instead the number of passed plus
number of failed tests is used as an equivalent count.

For consistency this commit makes a minor change to the printed output of
`run-tests.py`: instead of printing a shorthand name for tests that failed
or skipped, it now prints the full name.  Eg what was previously printed as
`attrtuple2` is now printed as `basics/attrtuple2.py`.  This makes the
output a little longer (when there are failed/skipped tests) but helps to
disambiguate the test name, eg which directory it's in.

Signed-off-by: Damien George <damien@micropython.org>
2025-06-12 21:47:20 +10:00
Alessandro Gatti
c1c73d966e tests/run-tests.py: Unconditionally enable native tests if asked.
This commit lets the test runner enumerate and run native tests if the
feature check fails but native tests were explicitly requested from the
command line.

The old behaviour would disable native tests anyway if the feature check
failed, however this hid a bug in the x86 native emitter that would be
triggered even during the feature check.  That meant the test suite
would pass on x86 even with a broken emitter, as those tests would have
been skipped anyway.

Now, if the user asks for native code it will get native code out of the
runner no matter what.

Co-authored-by: Damien George <damien@micropython.org>
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
2025-06-10 11:29:02 +10:00
Damien George
5b340b12b8 tests/run-tests.py: Remove filename arg from prepare_script_for_target.
It's no longer used.

Signed-off-by: Damien George <damien@micropython.org>
2025-06-05 15:18:30 +10:00
Damien George
229104558f tests/run-tests.py: Automatically skip tests that are too large.
Some tests are just too big for targets that don't have much heap memory,
eg `tests/extmod/vfs_rom.py`.  Other tests are too large because the target
doesn't have enough IRAM for native code, eg esp8266 running
`tests/micropython/viper_args.py`.

Previously, such tests were explicitly skipped on targets known to have
little memory, eg esp8266.  But this doesn't scale to multiple targets, nor
to more and more tests which are too large.

This commit addresses that by adding logic to the test runner so it can
automatically skip tests when they don't fit in the target's memory.  It
does this by prepending a `print('START TEST')` to every test, and if a
`MemoryError` occurs before that line is printed then the test was too big.
This works for standard tests, tests that go via .mpy files, and tests that
run in native emitter mode via .mpy files.

For tests that are too big, it prints `lrge  <test name>` on the output,
and at the end prints them on a separate line of skipped tests so they can
be distinguished.  They are also distinguished in the `_result.json` file
as a skipped test with reason "too large".

Signed-off-by: Damien George <damien@micropython.org>
2025-06-05 15:15:31 +10:00
Damien George
4dff9cbf1a tests/run-tests.py: Change _results.json to have a combined result list.
The `_results.json` output of `run-tests.py` was recently changed in
7a55cb6b36 to add a list of passed and
skipped tests.

The way this was done turned out to be not general enough, because we want
to add another type of result, namely tests that are skipped because they
are too large.

Instead of having separate lists in `_results.json` for each kind of result
(pass, fail, skip, skip too large, etc), this commit changes the output
form of `_results.json` so that it stores a single list of 3-tuples of all
tests that were run:

    [(test_name, result, reason), ...]

That's more general and allows adding a reason for skipped and failed
tests.  At the moment this reason is just an empty string, but can be
improved in the future.

Signed-off-by: Damien George <damien@micropython.org>
2025-06-03 10:07:29 +10:00
Damien George
7a55cb6b36 tests/run-tests.py: Add list of passed/skipped tests to _result.json.
Some checks failed
JavaScript code lint and formatting with Biome / eslint (push) Has been cancelled
Check code formatting / code-formatting (push) Has been cancelled
Check spelling with codespell / codespell (push) Has been cancelled
Build docs / build (push) Has been cancelled
Check examples / embedding (push) Has been cancelled
Package mpremote / build (push) Has been cancelled
.mpy file format and tools / test (push) Has been cancelled
Build ports metadata / build (push) Has been cancelled
alif port / build_alif (alif_ae3_build) (push) Has been cancelled
cc3200 port / build (push) Has been cancelled
esp32 port / build_idf (esp32_build_cmod_spiram_s2) (push) Has been cancelled
esp32 port / build_idf (esp32_build_s3_c3) (push) Has been cancelled
esp8266 port / build (push) Has been cancelled
mimxrt port / build (push) Has been cancelled
nrf port / build (push) Has been cancelled
powerpc port / build (push) Has been cancelled
qemu port / build_and_test_arm (push) Has been cancelled
qemu port / build_and_test_rv32 (push) Has been cancelled
renesas-ra port / build_renesas_ra_board (push) Has been cancelled
rp2 port / build (push) Has been cancelled
samd port / build (push) Has been cancelled
stm32 port / build_stm32 (stm32_misc_build) (push) Has been cancelled
stm32 port / build_stm32 (stm32_nucleo_build) (push) Has been cancelled
stm32 port / build_stm32 (stm32_pyb_build) (push) Has been cancelled
unix port / minimal (push) Has been cancelled
unix port / reproducible (push) Has been cancelled
unix port / standard (push) Has been cancelled
unix port / standard_v2 (push) Has been cancelled
unix port / coverage (push) Has been cancelled
unix port / coverage_32bit (push) Has been cancelled
unix port / nanbox (push) Has been cancelled
unix port / float (push) Has been cancelled
unix port / stackless_clang (push) Has been cancelled
unix port / float_clang (push) Has been cancelled
unix port / settrace (push) Has been cancelled
unix port / settrace_stackless (push) Has been cancelled
unix port / macos (push) Has been cancelled
unix port / qemu_mips (push) Has been cancelled
unix port / qemu_arm (push) Has been cancelled
unix port / qemu_riscv64 (push) Has been cancelled
webassembly port / build (push) Has been cancelled
windows port / build-vs (Debug, x64, windows-2022, dev, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Debug, x64, windows-latest, dev, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Debug, x86, windows-2022, dev, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Debug, x86, windows-latest, dev, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Release, x64, windows-2019, dev, 2019, [16, 17)) (push) Has been cancelled
windows port / build-vs (Release, x64, windows-2019, standard, 2019, [16, 17)) (push) Has been cancelled
windows port / build-vs (Release, x64, windows-2022, dev, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Release, x64, windows-2022, standard, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Release, x64, windows-latest, dev, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Release, x64, windows-latest, standard, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Release, x86, windows-2019, dev, 2019, [16, 17)) (push) Has been cancelled
windows port / build-vs (Release, x86, windows-2019, standard, 2019, [16, 17)) (push) Has been cancelled
windows port / build-vs (Release, x86, windows-2022, dev, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Release, x86, windows-2022, standard, 2022, [17, 18)) (push) Has been cancelled
windows port / build-vs (Release, x86, windows-latest, dev, 2017, [15, 16)) (push) Has been cancelled
windows port / build-vs (Release, x86, windows-latest, standard, 2017, [15, 16)) (push) Has been cancelled
windows port / build-mingw (i686, mingw32, dev) (push) Has been cancelled
windows port / build-mingw (i686, mingw32, standard) (push) Has been cancelled
windows port / build-mingw (x86_64, mingw64, dev) (push) Has been cancelled
windows port / build-mingw (x86_64, mingw64, standard) (push) Has been cancelled
windows port / cross-build-on-linux (push) Has been cancelled
zephyr port / build (push) Has been cancelled
Python code lint and formatting with ruff / ruff (push) Has been cancelled
The output `_result.json` file generated by `run-tests.py` currently
contains a list of failed tests.  This commit adds to the output a list of
passed and skipped tests, and so now provides full information about which
tests were run and what their results were.

Signed-off-by: Damien George <damien@micropython.org>
2025-05-18 00:31:48 +10:00