790 Commits

Author	SHA1	Message	Date
Damien George	7e9d19e06b	github/workflows: Stop using ubuntu-20.04. ... For GitHub Actions, ubuntu-20.04 is deprecated and will be removed by 1st April 2025. See announcement at https://github.com/actions/runner-images/issues/11101 This commit changes actions that use ubuntu-20.04 to a newer image. Signed-off-by: Damien George <damien@micropython.org>	2025-02-25 15:11:43 +11:00
Glenn Moloney	2992e34956	tools/mpremote: Add support for relative urls in package.json files. ... URLs in `package.json` may now be specified relative to the base URL of the `package.json` file. Relative URLs wil work for `package.json` files installed from the web as well as local file paths. Docs: update `docs/reference/packages.rst` to add documentation for: - Installing packages from local filesystems (PR #12476); and - Using relative URLs in the `package.json` file (PR #12477); - Update the packaging example to encourage relative URLs as the default in `package.json`. Add `tools/mpremote/tests/test_mip_local_install.sh` to test the installation of a package from local files using relative URLs in the `package.json`. Signed-off-by: Glenn Moloney <glenn.moloney@gmail.com>	2025-02-24 13:29:40 +11:00
Angus Gratton	1034b17558	tools/ci.sh: Build the W5100S_EVB_PICO board with no threads. ... Serves as a build test for a config we don't otherwise support. Signed-off-by: Angus Gratton <angus@redyak.com.au>	2025-02-20 00:57:45 +11:00
rufusclark	1a67d720c7	tools/pyboard.py: Make get_time use machine.RTC instead of pyb.RTC. ... The current code evaluates `pyb.RTC().datetime()` resulting in a remote side exception, as `pyb` is not defined on most ports (only stm32). The code should evaluate `machine.RTC().datetime()` and hence return the current time. Signed-off-by: rufusclark <50201718+rufusclark@users.noreply.github.com> Signed-off-by: Damien George <damien@micropython.org>	2025-02-12 12:53:46 +11:00
Glenn Moloney	11c9656fad	tools/mpremote: Support mip install from package.json on local fs. ... Add support for `mpremote mip install package.json` where `package.json` is a json file on the local filesystem. Without this, package json files can only be loaded from http, https, github or gitlab URLs. This is useful for testing `package.json` files for pacages in development and for constructing one's own `package.json` files for Python packages which are not yet available for installation using mip. Signed-off-by: Glenn Moloney <glenn.moloney@gmail.com>	2025-02-10 15:59:58 +11:00
iabdalkader	752c1672af	tools/ci.sh: Build MIMXRT1060_EVK with MSC enabled as part of mimxrt CI. ... Signed-off-by: iabdalkader <i.abdalkader@gmail.com>	2025-02-10 12:53:45 +11:00
robert-hh	e574f68820	mimxrt: Add support for a UF2 bootloader. ... Allowing to use e.g. the Adafruit bootloaders with MicroPython. The .uf2 file is created in addition to the .bin and .hex files allowing to use the latter ones without the bootloader for debugging and testing. Changes: - Set the location of the ISR Vector and .text segment to 0x6000C000 and 0x6000C400. - Reserve an area at the start of ITCM for a copy of the interrupt vector table and copy the table on reset to this place. - Extend `machine.bootloader()` by setting the magic number to enable the bootloader on reset. - Create a .uf2 file which skips the segments below 0x6000C000. The bootloader has to be installed as a preparation step using the board specific methods, but then the firmware's .uf2 file version can be installed using the bootloader. The bootloader can be invoked with: - double reset - calling machine.bootloader() - Using the touch1200 method Double reset is hard to achieve on MIMXRT boards, since there is no clean reset pin. Some MIMXRT boards provide it by switching the power. Some boards are excluded from the .uf2 build: - MIMXRT1050_EVK: The uf2 bootloader is built for the QSPI version of the board. MicroPython supports the Hyperflash version. - MIMXRT1176_EVK: No support for this board yet, but it should be possible. Signed-off-by: robert-hh <robert@hammelrath.com>	2025-02-10 11:28:39 +11:00
Alessandro Gatti	f594c6f66e	tools/ci.sh: Add natmod tests for QEMU/Arm. ... This commit adds the natmod tests for the MPS2_AN385 board running inside QEMU to the CI pipeline. Now natmod tests capabilities are equal between the Arm and RV32 platforms for the QEMU port. Signed-off-by: Alessandro Gatti <a.gatti@frob.it>	2025-02-07 17:37:04 +11:00
Hans Maerki	03fe9c55ea	tools/mpremote: Introduce timeout_overall for read_until(). ... And use it in `enter_raw_repl()`. This prevents waiting forever for a serial device that does not respond to the Ctrl-C/Ctrl-D/etc commands and is constantly outputting data. Signed-off-by: Hans Maerki <buhtig.hans.maerki@ergoinfo.ch>	2025-01-29 10:45:08 +11:00
Hans Maerki	0d46e45a1f	tools/mpremote: Avoid initial blocking read in read_until(). ... If the target does not return any data then `read_until()` will block indefinitely. Fix this by making the initial read part of the general read look, which always checks `inWaiting() > 0` before reading from the serial device. Also added the UART timeout to the constructor. This is not currently used but may be used as an additional safeguard. Signed-off-by: Hans Maerki <buhtig.hans.maerki@ergoinfo.ch>	2025-01-27 10:20:40 +11:00
Angus Gratton	d89e71e6c0	tools/autobuild,esp32: Template the generation of esp32 port deploy.md. ... Allows two source files (ports/esp32/boards/deploy.md and deploy_nativeusb.md for boards with only native USB) for all esp32 installation steps, with templated chip name and flash offset inserted via string formatting. The new files add more text to explain the esptool.py port auto-detection, remove the unnecessary -z feature (already enabled by default), and add a bit of troubleshooting and port detection info. This work was funded through GitHub Sponsors. Signed-off-by: Angus Gratton <angus@redyak.com.au>	2025-01-17 16:03:45 +11:00
Alessandro Gatti	0950f65ac4	tools/pyboard.py: Wait a bit before accessing the PTY serial port. ... Some PTY targets, namely `NETDUINO2` and `MICROBIT` under Qemu, take a bit more time to present a REPL than usual. The pyboard tool is a bit too impatient and would bail out before any of those targets had a chance to respond to the raw REPL request. Co-authored-by: Damien George <damien@micropython.org> Signed-off-by: Alessandro Gatti <a.gatti@frob.it>	2025-01-16 12:10:28 +11:00
Damien George	d533c9067a	tools/autobuild: Don't allow a board to change its ID. ... All board IDs are now the board directory name. Signed-off-by: Damien George <damien@micropython.org>	2025-01-15 16:48:10 +11:00
Damien George	495ce91cae	tools/ci.sh: Run test_full for qemu port CI. ... Signed-off-by: Damien George <damien@micropython.org>	2025-01-06 17:57:48 +11:00
Alessandro Gatti	4729a89504	tools/verifygitlog.py: Show invalid commit subjects in quotes. ... If a commit subject line has any trailing whitespace it won't match the repository validation rules, and the line will show up as part of the relevant error message. However, since there's no quotation marks around the offending text, the trailing whitespace may go unnoticed, and given that the commit message is then discarded when the commit operation is retried this can get fairly annoying. This commit simply modifies the error output for invalid subject lines to add quotation marks around the offending text, so trailing whitespace is much easier to see. Signed-off-by: Alessandro Gatti <a.gatti@frob.it>	2024-12-23 12:55:20 +11:00
Alessandro Gatti	7ca6e5eb68	qemu: Add test_natmod target for RV32 and use as part of CI pipeline. ... This commit brings the natmod tests in the CI build process for the RV32 platform. Not all example natmods are tested at the moment, as `features` requires soft-float support, and `btree` needs thread-local storage support in `mpy_ld.py` when built with the CI's toolchain. Co-authored-by: Damien George <damien@micropython.org> Signed-off-by: Alessandro Gatti <a.gatti@frob.it>	2024-12-23 10:04:19 +11:00
Alessandro Gatti	6760e00817	tools/mpy_ld.py: Add native modules support for RV32 code. ... This commit adds support for RV32IMC native modules, as in embedding native code into a self-contained MPY module and and make its exported functions available to the MicroPython environment. Signed-off-by: Alessandro Gatti <a.gatti@frob.it>	2024-12-23 10:02:20 +11:00
Angus Gratton	a9945fc528	ci: Add caching of ccache for Zephyr. ... Similar to the ESP32 builds, but needs additional step to pass the ccache directory through to the Zephyr container. This work was funded through GitHub Sponsors. Signed-off-by: Angus Gratton <angus@redyak.com.au>	2024-12-18 15:11:01 +11:00
Angus Gratton	db4b095644	ci: Pull the Zephyr CI docker image from GitHub container reg. ... This image is 15GB so in theory this may be faster, although in testing the improvement is either non-existent or marginal. This work was funded through GitHub Sponsors. Signed-off-by: Angus Gratton <angus@redyak.com.au>	2024-12-18 15:11:01 +11:00
Angus Gratton	92a5ea51b4	ci: Cache Zephyr workspace installation. ... Can save several minutes downloading the Zephyr docker image and/or cloning repo from GitHub. Cache keyed on the Zephyr version, which AFAIK is the only determinant for the workspace contents. This work was funded through GitHub Sponsors. Signed-off-by: Angus Gratton <angus@redyak.com.au>	2024-12-18 15:11:01 +11:00
Damien George	3203e950fc	tools/boardgen.py: Provide macro defns for number of cpu/board pins. ... So a port can use them if needed to exclude the Pin.cpu/Pin.board objects. Signed-off-by: Damien George <damien@micropython.org>	2024-12-17 23:53:24 +11:00
Damien George	ee8d8b3448	tools/ci.sh: Re-enable vfs_posix tests on unix qemu MIPS CI. ... These work now that _FILE_OFFSET_BITS=64. Signed-off-by: Damien George <damien@micropython.org>	2024-12-16 11:58:43 +11:00
Angus Gratton	d49c1e836c	tools/ci.sh: Remove explicit macOS pkg-config install. ... Reasons to remove this: - GitHub's macOS runners install this package by default nowadays. - Brew renamed this package to 'pkgconf' so installing the old name on top of the new package name has started failing. This work was funded through GitHub Sponsors. Signed-off-by: Angus Gratton <angus@redyak.com.au>	2024-11-30 11:24:36 +11:00
Damien George	4fd5b72a8b	tools/mpremote: Support trailing slash on dest for non-recursive copy. ... This fixes a regression in db59e55fe7: prior to that commit `mpremote` supported trailing slashes on the destination of a normal (non-recursive) copy. Add back support for that, with the semantics that a trailing slash requires the destination to be an existing directory. Also add a test for this. Signed-off-by: Damien George <damien@micropython.org>	2024-11-13 11:51:35 +11:00
Damien George	3b6024a699	tools/mpremote: Add test for forced copy. ... Signed-off-by: Damien George <damien@micropython.org>	2024-11-13 11:51:35 +11:00
Damien George	76e6c6345c	tools/mpremote: Make sure stdout and stderr output appear in order. ... mpremote error messages now go to stderr, so make sure stdout is flushed before printing them. Also update the test runner to capture error messages. Signed-off-by: Damien George <damien@micropython.org>	2024-11-13 11:51:35 +11:00
Damien George	8978102f35	tests/run-tests.py: Change --target/--device options to --test-instance. ... Previously to this commit, running the test suite on a bare-metal board required specifying the target (really platform) and device, eg: $ ./run-tests.py --target pyboard --device /dev/ttyACM1 That's quite a lot to type, and you also need to know what the target platform is, when a lot of the time you either don't care or it doesn't matter. This commit makes it easier to run the tests by replacing both of these options with a single `--test-instance` (`-t` for short) option. That option specifies the executable/port/device to test. Then the target platform is automatically detected. The `--test-instance` can be passed: - "unix" (the default) to use the unix version of MicroPython - "webassembly" to test the webassembly port - anything else is considered a port/device to pass to Pyboard There are also some shortcuts to specify a port/device, following `mpremote`: - a<n> is short for /dev/ttyACM<n> - u<n> is short for /dev/ttyUSB<n> - c<n> is short for COM<n> For example: $ ./run-tests.py -t a1 Note that the default test instance is "unix" and so this commit does not change the standard way to run tests on the unix port, by just doing `./run-tests.py`. As part of this change, the platform (and it's native architecture if it supports importing native .mpy files) is show at the start of the test run. Signed-off-by: Damien George <damien@micropython.org>	2024-11-04 12:47:47 +11:00
Angus Gratton	787c424cfc	tools/ci.sh: Fix reference commit for code size comparison. ... Previously the code size comparison was between the merge base (i.e. where the PR branched), and the generated merge commit into master. If the PR branch was older than current master, this meant the size comparison could incorrectly include changes already merged on master but missing from the PR branch. This commit changes it to compare the generated merge commit against current master, i.e. the size impact if this PR was to be merged. This commit also disables running the code size check on "push", it now only runs on pull_request events. This work was funded through GitHub Sponsors. Signed-off-by: Angus Gratton <angus@redyak.com.au>	2024-10-31 23:19:55 +11:00
Angus Gratton	9591b0a53c	tools/ci.sh: Fix commit msg checking when PR branch HEAD behind master. ... Fixes the problem noted at https://github.com/micropython/micropython/pull/15547#issuecomment-2434479702 which is that, because default CI HEAD for a PR is a (generated) merge commit into the master branch's current HEAD, then if the PR branch isn't fully rebased then the commit check runs against commits from master as well! Also drops running this check on push, the pull_request event is triggered by default on open and update ("synchronized" event), which probably covers the cases where this check should run. This work was funded through GitHub Sponsors. Signed-off-by: Angus Gratton <angus@redyak.com.au>	2024-10-31 23:16:25 +11:00
Glenn Moloney	9ea8d2a031	tools/mpremote: Fix UnboundLocalError in Transport.fs_writefile(). ... The variable `written` was being used before it was defined in the `fs_writefile()` method of the Transport class. This was causing an `UnboundLocalError` to be raised when the `progress_callback` was not provided. Fixes issue #16084. Signed-off-by: Glenn Moloney <glenn.moloney@gmail.com>	2024-10-28 12:23:10 +11:00
Damien George	f9cebe676e	tools/ci.sh: Add RPI_PICO2 to CI. ... Signed-off-by: Damien George <damien@micropython.org>	2024-10-15 12:21:10 +11:00
Damien George	d92694c3e8	tools: Only issue a single Ctrl-C when entering raw REPL. ... A long time ago when there was only the `stm` port, Ctrl-C would trigger a preemptive NLR jump to break out of running code. Then in commit 124df6f8d0 a more general approach to asynchronous `KeyboardInterrupt` exceptions was implemented, and `stmhal` supported both approaches, with the general (soft) interrupt taking priority. Then in commit bc1488a05f `pyboard.py` was updated with a corresponding change to make it issue a double Ctrl-C to break out of any existing code when entering the raw REPL (two Ctrl-C characters were sent in order to more reliably trigger the preemptive NLR jump). No other port has preemptive NLR jumps and so a double Ctrl-C doesn't really behave any differently to a single Ctrl-C: with USB CDC the double Ctrl-C would most likely be in the same USB packet and so processed in the same low-level USB callback, so it's just setting the keyboard interrupt flag twice in a row. The VM/runtime then just sees one keyboard interrupt and acts as though only one Ctrl-C was sent. This commit changes the double Ctrl-C to a single Ctrl-C in `pyboard.py` and `mpremote`. That keeps things as simple as they need to be. Signed-off-by: Damien George <damien@micropython.org>	2024-10-15 11:40:08 +11:00
Damien George	a25b6b9c65	tools/mpremote: Add option to force copy. ... This adds a -f/--force option to the "cp" command, which forces unconditional copies, in particular does not check the hash. Signed-off-by: Damien George <damien@micropython.org>	2024-10-09 16:39:06 +11:00
Jim Mussared	6461ffd9d1	tools/mpremote: Add initial regression tests for mpremote. ... These tests are specifically for the command-line interface and cover: - resume/soft-reset/connect/disconnect - mount - fs cp,touch,mkdir,cat,sha256sum,rm,rmdir - eval/exec/run This work was funded through GitHub Sponsors. Signed-off-by: Jim Mussared <jim.mussared@gmail.com> Signed-off-by: Damien George <damien@micropython.org>	2024-10-09 16:39:00 +11:00
Jim Mussared	dd6f78f014	tools/mpremote: Improve error output. ... Makes the filesystem command give standard error messages rather than just printing the exception from the device. Makes the distinction between CommandError and TransportError clearer. This work was funded through GitHub Sponsors. Signed-off-by: Jim Mussared <jim.mussared@gmail.com>	2024-10-09 16:38:08 +11:00
Jim Mussared	6f8157d880	tools/mpremote: Add hashing ability and use for recursive copy. ... Changes in this commit: - Adds transport API `fs_hashfile` to compute the hash of a file with given algorithm. - Adds commands `mpremote <...>sum file` to compute and print hashes of various algorithms. - Adds shortcut `mpremote sha256sum file`. - Uses the hash computation to improve speed of recursive file copy to avoid copying a file where the target is identical. For recursive copy, if possible it will use the board's support (e.g. built-in hashlib or hashlib from micropython-lib), but will fall back to downloading the file and using the local implementation. This work was funded through GitHub Sponsors. Signed-off-by: Jim Mussared <jim.mussared@gmail.com> Signed-off-by: Damien George <damien@micropython.org>	2024-10-09 16:35:46 +11:00
Jim Mussared	db59e55fe7	tools/mpremote: Make filesystem commands use transport API. ... This introduces a Python filesystem API on `Transport` that is implemented entirely with eval/exec provided by the underlying transport subclass. Updates existing mpremote filesystem commands (and `edit) to use this API. Also re-implements recursive `cp` to allow arbitrary source / destination. This work was funded through GitHub Sponsors. Signed-off-by: Jim Mussared <jim.mussared@gmail.com> Signed-off-by: Damien George <damien@micropython.org>	2024-10-09 16:12:30 +11:00
Jim Mussared	1091021995	tools/mpremote: Make eval parse by default. ... This is a step towards making the transport expose a Python API rather than functions that mostly print to stdout. Most use cases of `transport.eval()` are to get some state back from the device, so have it return as a value directly by default. Updates uses of `transport.eval()` to remove the parse argument where it now isn't needed, make the `rtc` command use eval/exec, and update the `mip` command to use eval's parsing. This work was funded through GitHub Sponsors. Signed-off-by: Jim Mussared <jim.mussared@gmail.com>	2024-10-09 15:56:17 +11:00
Maureen Helm	545d4efb55	zephyr: Upgrade to Zephyr v3.7.0. ... Updates the Zephyr port build instructions and CI to use the latest Zephyr release tag. Tested on frdm_k64f. Signed-off-by: Maureen Helm <maureen.helm@analog.com>	2024-10-02 07:49:53 +10:00
Maureen Helm	fa7b4c48bc	tools/ci.sh: Upgrade Zephyr docker image 0.26.13 and SDK 0.16.8. ... Upgrades CI to use the latest versions of the Zephyr docker image and Zephyr SDK. Signed-off-by: Maureen Helm <maureen.helm@analog.com>	2024-10-02 07:49:53 +10:00
Alessandro Gatti	3e085c3aa6	tools/ci.sh: Report code size for VIRT_RV32 as well. ... This commit adds the `VIRT_RV32` board to the list of targets for calculating code size changes as part of the CI pipeline. Signed-off-by: Alessandro Gatti <a.gatti@frob.it>	2024-09-23 13:03:26 +10:00
Alessandro Gatti	58e7689142	tools/metrics.py: Add VIRT_RV32 to the code size metrics. ... This commit adds the Qemu-based RISC-V 32 bits `VIRT_RV32` board to the list of ports/boards to be built for measuring code size changes. Signed-off-by: Alessandro Gatti <a.gatti@frob.it>	2024-09-23 13:03:26 +10:00
Alessandro Gatti	bb3c711c04	tools/ci.sh: Let RV64 use a source-built version of libffi. ... This commit lets the RV64 port use the version of libffi that is bundled as a submodule in the MicroPython source tree, as the packaged libffi library coming from Ubuntu's RISC-V repository trashes foreign function call results on exit. Signed-off-by: Alessandro Gatti <a.gatti@frob.it>	2024-09-23 11:47:18 +10:00
Damien George	8b35f2c7fa	tools/mpy_ld.py: Support jumping more than 2k on armv6m architectures. ... Native .mpy files targetting armv6m (eg RP2040) cannot currently have more than about 2kiB of native code (between the start of the file and the init function). This commit fixes that by using bigger jumps to jump to the init function. Signed-off-by: Damien George <damien@micropython.org>	2024-09-19 14:52:58 +10:00
Alessandro Gatti	46d8db81d3	tools/ci.sh: Clean up the Unix port's Arm target. ... The Unix port's Arm target CI steps have been updated to be more in line with the other targets (the MicroPython binary doesn't need an environment variable to be set in order to run now). Signed-off-by: Alessandro Gatti <a.gatti@frob.it>	2024-09-19 13:49:12 +10:00
Alessandro Gatti	52d5f39881	tools/ci.sh: Add missing FFI helper for CI RV64 Unix builds. ... The FFI helper definition was accidentally omitted when committing the necessary shell code for building RV64 Unix builds in the CI environment. Signed-off-by: Alessandro Gatti <a.gatti@frob.it>	2024-09-19 13:49:12 +10:00
Alessandro Gatti	7b5738ad86	tools/ci.sh: Clean up the Unix port's MIPS target. ... The Unix port's MIPS target CI steps have been updated to be more in line with the other targets (the MicroPython binary now runs as a dynamic executable), and the test exceptions for ffi have been lifted. Signed-off-by: Alessandro Gatti <a.gatti@frob.it>	2024-09-19 13:49:12 +10:00
Damien George	067ef81cd0	all: Remove tinytest component. ... With the recent qemu (d9a0fdda9a and 0426934969) and zephyr (05cad7b56f) changes to how their tests are run, two things became unused: - The tinytest framework, which embedded a set of tests and their expected output within firmware, so these tests could be run stand-alone. - The `--write-exp` and `--list-tests` options to `tests/run-tests.py`, which were needed primarily to generated the expected test output for tinytest (also the associated `tests/run-tests-exp.py/.sh` scripts are now unused). This commit removes the tinytest component and all its helper code. This eliminates a maintenance burden. Signed-off-by: Damien George <damien@micropython.org>	2024-09-19 13:31:36 +10:00
Alessandro Gatti	a831c788f7	tools/mpy_ld.py: Ignore R_XTENSA_ASM_EXPAND relocation entries. ... As reported in #14430 the Xtensa compiler can add R_XTENSA_ASM_EXPAND relocation relaxation entries in object files, and they were not supported by mpy_ld. This commit adds handling for that entry, doing nothing with it, as it is only of real use for an optimising linker. Signed-off-by: Alessandro Gatti <a.gatti@frob.it>	2024-09-06 17:10:07 +10:00
Damien George	659113825d	qemu: Rename qemu-arm port to qemu. ... Because this port now supports multiple architectures. Signed-off-by: Damien George <damien@micropython.org>	2024-09-06 11:08:39 +10:00