This commit adds the `VIRT_RV32` board to the list of targets for
calculating code size changes as part of the CI pipeline.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit adds the Qemu-based RISC-V 32 bits `VIRT_RV32` board to the
list of ports/boards to be built for measuring code size changes.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit lets the RV64 port use the version of libffi that is bundled
as a submodule in the MicroPython source tree, as the packaged libffi
library coming from Ubuntu's RISC-V repository trashes foreign function
call results on exit.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
Native .mpy files targetting armv6m (eg RP2040) cannot currently have more
than about 2kiB of native code (between the start of the file and the init
function).
This commit fixes that by using bigger jumps to jump to the init function.
Signed-off-by: Damien George <damien@micropython.org>
The Unix port's Arm target CI steps have been updated to be more in
line with the other targets (the MicroPython binary doesn't need an
environment variable to be set in order to run now).
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
The FFI helper definition was accidentally omitted when committing the
necessary shell code for building RV64 Unix builds in the CI
environment.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
The Unix port's MIPS target CI steps have been updated to be more in
line with the other targets (the MicroPython binary now runs as a
dynamic executable), and the test exceptions for ffi have been lifted.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
With the recent qemu (d9a0fdda9a and
0426934969) and zephyr
(05cad7b56f) changes to how their tests are
run, two things became unused:
- The tinytest framework, which embedded a set of tests and their expected
output within firmware, so these tests could be run stand-alone.
- The `--write-exp` and `--list-tests` options to `tests/run-tests.py`,
which were needed primarily to generated the expected test output for
tinytest (also the associated `tests/run-tests-exp.py/.sh` scripts are
now unused).
This commit removes the tinytest component and all its helper code. This
eliminates a maintenance burden.
Signed-off-by: Damien George <damien@micropython.org>
As reported in #14430 the Xtensa compiler can add R_XTENSA_ASM_EXPAND
relocation relaxation entries in object files, and they were not
supported by mpy_ld.
This commit adds handling for that entry, doing nothing with it, as it
is only of real use for an optimising linker.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
Currently both the qemu-arm and qemu-riscv ports share a lot of code and
functionality. This commit merges the qemu-riscv port into the qemu-arm
port. The only real differences between the two are the toolchains used to
build the code, and the initialisation/startup framework. Everything else
is pretty much the same, so this brings the following benefits:
- less code duplication
- less burden on maintenance
- generalised qemu port, could in the future support other architectures
A new board `VIRT_RV32` has been added to the qemu-arm port which is the
existing RISC-V board from the qemu-riscv port. To build it:
$ make BOARD=VIRT_RV32 repl
To cleanly separate the code for the different architectures, startup code
has been moved to ports/qemu-arm/mcu/<arch>/.
Signed-off-by: Damien George <damien@micropython.org>
Currently, the qemu-arm (and qemu-riscv) port has two build modes:
- a simple test that executes a Python string; and
- a full test that uses tinytest to embed all tests within the firmware,
then executes that and captures the output.
This is very different to all the other ports. A difficulty with using
tinytest is that with the large number of tests the firmware overflows its
virtual flash size. It's also hard to run tests via .mpy files and with
the native emitter. Being different to the other ports also means an extra
burden on maintenance.
This commit reworks the qemu-arm port so that it has a single build target
that creates a standard firmware which has a REPL. When run under
qemu-system-arm, the REPL acts like any other bare-metal port, complete
with soft reset (use machine.reset() to turn it off and exit
qemu-system-arm).
This approach gives many benefits:
- allows playing with a REPL without hardware;
- allows running the test suite as it would on a bare-metal board, by
making qemu-system-arm redirect the UART serial of the virtual device to
a /dev/pts/xx file, and then running run-tests.py against that serial
device;
- skipping tests is now done via the logic in `run-tests.py` and no longer
needs multiple places to define which tests to skip
(`tools/tinytest-codegen.py`, `ports/qemu-arm/tests_profile.txt` and also
`tests/run-tests.py`);
- allows testing/using mpremote with the qemu-arm port.
Eventually the qemu-riscv port would have a similar change.
Prior to this commit the test results were:
743 tests ok. (121 skipped)
With this commit the test results are:
753 tests performed (22673 individual testcases)
753 tests passed
138 tests skipped
More tests are skipped because more are included in the run. But overall
more tests pass.
Signed-off-by: Damien George <damien@micropython.org>
With this change, the zephyr port is tested against the standard test suite
via the following scheme:
- the zephyr port is built with the `qemu_cortex_m3` board and the
`prj_minimal.conf` configuration
- `qemu-system-arm` runs `zephyr.elf`
- the zephyr console is redirected to a pts/pty
- `tests/run-tests.py` is run in bare-metal mode against the pts/pty device
This allows testing the zephyr port as though it were a physical board
attached over a serial port.
Signed-off-by: Damien George <damien@micropython.org>
The pts line printed by qemu-system-arm goes to stdout, not stderr.
Redirect stderr to stdout in case other tools do print to stderr.
Signed-off-by: Damien George <damien@micropython.org>
IDF 3 builds are very old now (it seems like the last successful builds are
from 2021), and the current IDF 5 is stable. So remove IDF 3 variants.
Signed-off-by: Matt Trentini <matt.trentini@gmail.com>
macos-11.0 is no longer available.
With this change in the macos version, some tests which previously failed
now pass, and some different tests now fail. Exclude those that fail from
the CI until they can be fixed properly.
Signed-off-by: Damien George <damien@micropython.org>
This is a fix for an algorithmic error in mpremote mip, that throws an
error due to a '\n' used in the concatenation and split when there is more
than one lib path in `sys.path`.
Signed-off-by: Jos Verlinde <Jos.Verlinde@microsoft.com>
Allow inclusion of large integer constants in frozen files using long-long
representation (mpy-cross option -mlongint-impl=longlong).
Signed-off-by: Yoctopuce <dev@yoctopuce.com>
Prior to this fix the current working path in the remote VFS would always
be prepended to the requested path to get the full path, even if the
requested path was already absolute, ie starting with "/".
So `os.chdir("/remote/dir1")` would set the working path to "/dir1/", and
a subsequent call with an absolute path like `os.listdir("/remote/dir2")`
would try to list the directory "/dir1/dir2/".
Fixes issue #15308.
Signed-off-by: Damien George <damien@micropython.org>
MPY files can now hold generated RV32IMC native code. This can be
accomplished by passing the `-march=rv32imc` flag to mpy-cross.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
When no usable manifest directives are found (as opposed to no manifest
being set in the makefile), non-compiling code was generated for the
empty frozen constants pool block.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This tool is used to compute size differences in the firmware (eg as part
of CI), but it doesn't currently check any firmware that has bare-metal
lwIP/networking, making it hard to see how firmware size changes when
networking related changes are made.
So, change the board selection for the rp2 port to RPI_PICO_W. Changes in
size to standard RPI_PICO firmware will be very similar to other bare-metal
boards like PYBV10.
Signed-off-by: Damien George <damien@micropython.org>
The `run-natmodtests.py` script now properly excludes tests that don't use
the corresponding native module.
Signed-off-by: Damien George <damien@micropython.org>
Also define `mp_type_bytearray`. These all help to write native modules.
Signed-off-by: Brian Pugh <bnp117@gmail.com>
Signed-off-by: Damien George <damien@micropython.org>
This is required because the .mpy native ABI was changed by the
introduction of `mp_proto_fun_t`, see commits:
- 416465d81e
- 5e3006f117
- e2ff00e811
And three `mp_binary` functions were added to `mp_fun_table` in
commit d2276f0d41.
Signed-off-by: Damien George <damien@micropython.org>
These are needed to read/write array.array objects, which is useful in
native code to provide fast extensions that work with big arrays of data.
Signed-off-by: Damien George <damien@micropython.org>
It destroys a few manual alignments, but these seem minor compared to
the benefit of automated code style consistency.
Signed-off-by: Christian Walther <cwalther@gmx.ch>
The documentation for `freeze()` says that:
- If `script` is `None`, all files in `path` will be frozen.
- If `script` is an iterable then `freeze()` is called on all items of the
iterable.
This commit makes sure this behaviour is followed when an empty tuple/list
is passed in for `script` (previously an empty tuple/list froze all files).
Fixes issue #14125.
Signed-off-by: Damien George <damien@micropython.org>
The argument to MP_BC_MAKE_FUNCTION (raw code index) was being encoded as a
byte instead of a variable unsigned int. That meant that if there were
more than 128 merged mpy files the encoding would be invalid.
Fix that by using `mp_encode_uint(idx)` to encode the raw code index. And
also use `Opcode` constants for the opcode values to make it easier to
understand the code.
Signed-off-by: Damien George <damien@micropython.org>
The STATIC macro was introduced a very long time ago in commit
d5df6cd44a. The original reason for this was
to have the option to define it to nothing so that all static functions
become global functions and therefore visible to certain debug tools, so
one could do function size comparison and other things.
This STATIC feature is rarely (if ever) used. And with the use of LTO and
heavy inline optimisation, analysing the size of individual functions when
they are not static is not a good representation of the size of code when
fully optimised.
So the macro does not have much use and it's simpler to just remove it.
Then you know exactly what it's doing. For example, newcomers don't have
to learn what the STATIC macro is and why it exists. Reading the code is
also less "loud" with a lowercase static.
One other minor point in favour of removing it, is that it stops bugs with
`STATIC inline`, which should always be `static inline`.
Methodology for this commit was:
1) git ls-files | egrep '\.[ch]$' | \
xargs sed -Ei "s/(^| )STATIC($| )/\1static\2/"
2) Do some manual cleanup in the diff by searching for the word STATIC in
comments and changing those back.
3) "git-grep STATIC docs/", manually fixed those cases.
4) "rg -t python STATIC", manually fixed codegen lines that used STATIC.
This work was funded through GitHub Sponsors.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
Follow up to 35dd959133, allows explicitly
adding the unix-ffi library path from the command line.
This option is needed when building unix-ffi manifests in micropython-lib
CI.
This work was funded through GitHub Sponsors.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
Allows bytecode itself to be used instead of an mp_raw_code_t in the simple
and common cases of a bytecode function without any children.
This can be used to further reduce frozen code size, and has the potential
to optimise other areas like importing.
Signed-off-by: Damien George <damien@micropython.org>
To simplify their access and reduce code size.
The `scope_flags` member is only ever used to determine if a function is a
generator or not, so make it reflect that fact as a bool type.
Signed-off-by: Damien George <damien@micropython.org>
The `asm_n_pos_args` and `asm_type_sig` members of `mp_raw_code_t` are only
used for raw codes of type MP_CODE_NATIVE_ASM, which are rare, for example
in frozen code. So using a truncated `mp_raw_code_t` in these cases helps
to reduce frozen code size on targets that have MICROPY_EMIT_INLINE_ASM
enabled.
With this, change in firmware size of RPI_PICO builds is -648.
Signed-off-by: Damien George <damien@micropython.org>
The mp_raw_code_t struct has been reordered and some members resized. The
`n_pos_args` member is renamed to `asm_n_pos_args`, and `type_sig` renamed
to `asm_type_sig` to indicate that these are used only for the inline-asm
emitters. These two members are also grouped together in the struct.
The justifications for resizing the members are:
- `fun_data_len` can be 32-bits without issue
- `n_children` is already limited to 16-bits by
`mp_emit_common_t::ct_cur_child`
- `scope_flags` is already limited to 16-bits by `scope_t::scope_flags`
- `prelude_offset` is already limited to 16-bits by the argument to
`mp_emit_glue_assign_native()`
- it's reasonable to limit the maximim number of inline-asm arguments to 12
(24 bits for `asm_type_sig` divided by 2)
This change helps to reduce frozen code size (and in some cases RAM usage)
in the following cases:
- 64-bit targets
- builds with MICROPY_PY_SYS_SETTRACE enabled
- builds with MICROPY_EMIT_MACHINE_CODE enabled but MICROPY_EMIT_INLINE_ASM
disabled
With this change, unix 64-bit builds are -4080 bytes in size. Bare-metal
ports like rp2 are unchanged (because mp_raw_code_t is still 32 bytes on
those 32-bit targets).
Signed-off-by: Damien George <damien@micropython.org>
It's rare to freeze .mpy files without specifying a qstr header from a
firmware build, but it can be useful for testing, eg
`mpy-tool.py -f test.mpy`. Fix this case so static qstrs are properly
excluded from the frozen qstr list.
Signed-off-by: Damien George <damien@micropython.org>
This commit changes how library packages are searched for when a manifest
file is loaded: there is now simply a list of library paths that is
searched in order for the given package. This list defaults to the
main directories in micropython-lib, but can be added to -- either appended
or prepended -- by using `add_library()`.
In particular the way unix-ffi library packages are searched has changed,
because the `unix_ffi` argument to `require()` is now removed. Instead, if
a build wants to include packages from micropython-lib/unix-ffi, then it
must explicitly add this to the list of paths to search using:
add_library("unix-ffi", "$(MPY_LIB_DIR)/unix-ffi")
Work done in collaboration with Jim Mussared.
Signed-off-by: Damien George <damien@micropython.org>
