The aim of this commit is to clarify the command line options available.
While they are available as well as in the CLI with --help, it's useful to
document them and provide a few examples.
Signed-off-by: Jos Verlinde <jos_verlinde@hotmail.com>
This commit lets "tools/mpy_ld.py" store architecture flags in generated
MPY files if explicitly requested, like "mpy-cross" does.
To achieve this, a new command-line option ("--arch-flags") was added to
receive the architecture flags value, accepting the same arguments'
format as "mpy-cross", and performing the same input validation.
The rest of the MPY toolchain was also modified to let the user pass the
arch flags to standard native module makefiles. Given that there's
already a well-established "ARCH" argument, "ARCH_FLAGS" was chosen to
pass the optional flags to "mpy_ld.py".
Finally, documentation was updated to mention the new variable.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit adds the ability to compile native modules for the RV64
platform, using "rv64imc" as its architecture name (eg.
"make ARCH=rv64imc" should build a RV64 natmod).
The rest of 64-bits relocations needed to build a native module are now
implemented, and all sample native modules build without errors or
warnings. The same Picolibc caveats on RV32 also apply on RV64, thus
the documentation was updated accordingly.
RV64 native modules are also built as part of the CI process, but not
yet executed as the QEMU port is not yet able to load and run them.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit adds custom command completion functions for both the zsh
and fish shell.
The behaviour for those new completions follow the existing completion
for the bash shell, including the way to generate the completion alias
(with appropriately named command line switches).
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
Assuming proper C99 language support, we can select "the int type as big as
a pointer" (most of the time) or "the 64-bit int type" (nanboxing with
REPR_D), and then define everything else automatically.
This simplifies port configuration files. And the types can still be
overridden if needed.
Signed-off-by: Jeff Epler <jepler@unpythonic.net>
Python 2.7 has been EOL since January 2020.
Ubuntu oldoldlts (Focal Fossa, 20.04) has Python 3.8. Debian oldoldstable
(Buster, from 2019) has Python 3.7. RHEL 8 (from 2019) has Python 3.6.
It's easier than ever to install a modern Python using uv.
Given this, it seems like a fine idea to drop Python 2.7 support.
Even though the build is not tested on Python as old as 3.3, I left
comments stating that "3.3+" is the baseline Python version. However, it
might make sense to bump this to e.g., 3.10, the oldest Python 3 version
used during CI. Or, using uv or another method actually test on the oldest
Python interpreter that is desirable to support (uv goes back to Python 3.7
easily; in October 2025, the oldest supported Python interpreter version
will be 3.10)
Signed-off-by: Jeff Epler <jepler@gmail.com>
This commit adds some documentation on what are the limitations of using
Picolibc as a standard C library for native modules.
This also contains a reference to the "errno" issue when building
natmods on RV32 that the PR this commit is part of, as it is not obvious
how to approach this issue when encountered for the first time.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit lets mpy_ld.py resolve symbols not only from the object
files involved in the linking process, or from compiler-supplied static
libraries, but also from a list of symbols referenced by an absolute
address (usually provided by the system's ROM).
This is needed for ESP8266 targets as some C stdlib functions are
provided by the MCU's own ROM code to reduce the final code footprint,
and therefore those functions' implementation was removed from the
compiler's support libraries. This means that unless `LINK_RUNTIME` is
set (which lets tooling look at more libraries to resolve symbols) the
build process will fail as tooling is unaware of the ROM symbols'
existence. With this change, fixed-address symbols can be exposed to
the symbol resolution step when performing natmod linking.
If there are symbols coming in from a fixed-address symbols list and
internal code or external libraries, the fixed-address symbol address
will take precedence in all cases.
Although this is - in theory - also working for the whole range of ESP32
MCUs, testing is currently limited to Xtensa processors and the example
natmods' makefiles only make use of this commit's changes for the
ESP8266 target.
Natmod builds can set the MPY_EXTERN_SYM_FILE variable pointing to a
linkerscript file containing a series of symbols (weak or strong) at a
fixed address; these symbols will then be used by the MicroPython
linker when packaging the natmod. If a different natmod build method is
used (eg. custom CMake scripts), `tools/mpy_ld.py` can now accept a
command line parameter called `--externs` (or its short variant `-e`)
that contains the path of a linkerscript file with the fixed-address
symbols to use when performing the linking process.
The linkerscript file parser can handle a very limited subset of
binutils's linkerscript syntax, namely just block comments, strong
symbols, and weak symbols. Each symbol must be in its own line for the
parser to succeed, empty lines or comment blocks are skipped. For an
example of what this parser was meant to handle, you can look at
`ports/esp8266/boards/eagle.rom.addr.v6.ld` and follow its format.
The natmod developer documentation is also updated to reflect the new
command line argument accepted by `mpy_ld.py` and the use cases for the
changes introduced by this commit.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit introduces an additional symbol resolution mechanism to the
natmod linking process. This allows the build scripts to look for required
symbols into selected libraries that are provided by the compiler
installation (libgcc and libm at the moment).
For example, using soft-float code in natmods, whilst technically possible,
was not an easy process and required some additional work to pull it off.
With this addition all the manual (and error-prone) operations have been
automated and folded into `tools/mpy_ld.py`.
Both newlib and picolibc toolchains are supported, albeit the latter may
require a bit of extra configuration depending on the environment the build
process runs on. Picolibc's soft-float functions aren't in libm - in fact
the shipped libm is nothing but a stub - but they are inside libc. This is
usually not a problem as these changes cater for that configuration quirk,
but on certain compilers the include paths used to find libraries in may
not be updated to take Picolibc's library directory into account. The bare
metal RISC-V compiler shipped with the CI OS image (GCC 10.2.0 on Ubuntu
22.04LTS) happens to exhibit this very problem.
To work around that for CI builds, the Picolibc libraries' path is
hardcoded in the Makefile directives used by the linker, but this can be
changed by setting the PICOLIBC_ROOT environment library when building
natmods.
Signed-off-by: Volodymyr Shymanskyy <vshymanskyi@gmail.com>
Co-authored-by: Alessandro Gatti <a.gatti@frob.it>
This commit adds support for RV32IMC native modules, as in embedding native
code into a self-contained MPY module and and make its exported functions
available to the MicroPython environment.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
Previously to this commit, running the test suite on a bare-metal board
required specifying the target (really platform) and device, eg:
$ ./run-tests.py --target pyboard --device /dev/ttyACM1
That's quite a lot to type, and you also need to know what the target
platform is, when a lot of the time you either don't care or it doesn't
matter.
This commit makes it easier to run the tests by replacing both of these
options with a single `--test-instance` (`-t` for short) option. That
option specifies the executable/port/device to test. Then the target
platform is automatically detected.
The `--test-instance` can be passed:
- "unix" (the default) to use the unix version of MicroPython
- "webassembly" to test the webassembly port
- anything else is considered a port/device to pass to Pyboard
There are also some shortcuts to specify a port/device, following
`mpremote`:
- a<n> is short for /dev/ttyACM<n>
- u<n> is short for /dev/ttyUSB<n>
- c<n> is short for COM<n>
For example:
$ ./run-tests.py -t a1
Note that the default test instance is "unix" and so this commit does not
change the standard way to run tests on the unix port, by just doing
`./run-tests.py`.
As part of this change, the platform (and it's native architecture if it
supports importing native .mpy files) is show at the start of the test run.
Signed-off-by: Damien George <damien@micropython.org>
Introduce SRC_USERMOD_LIB_ASM to allow users to include assembly files as
part of their user modules. It could be used to include optimized
functions or outputs of other programming languages.
Signed-off-by: George Hopkins <george-hopkins@null.net>
The STATIC macro was introduced a very long time ago in commit
d5df6cd44a. The original reason for this was
to have the option to define it to nothing so that all static functions
become global functions and therefore visible to certain debug tools, so
one could do function size comparison and other things.
This STATIC feature is rarely (if ever) used. And with the use of LTO and
heavy inline optimisation, analysing the size of individual functions when
they are not static is not a good representation of the size of code when
fully optimised.
So the macro does not have much use and it's simpler to just remove it.
Then you know exactly what it's doing. For example, newcomers don't have
to learn what the STATIC macro is and why it exists. Reading the code is
also less "loud" with a lowercase static.
One other minor point in favour of removing it, is that it stops bugs with
`STATIC inline`, which should always be `static inline`.
Methodology for this commit was:
1) git ls-files | egrep '\.[ch]$' | \
xargs sed -Ei "s/(^| )STATIC($| )/\1static\2/"
2) Do some manual cleanup in the diff by searching for the word STATIC in
comments and changing those back.
3) "git-grep STATIC docs/", manually fixed those cases.
4) "rg -t python STATIC", manually fixed codegen lines that used STATIC.
This work was funded through GitHub Sponsors.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
Allows bytecode itself to be used instead of an mp_raw_code_t in the simple
and common cases of a bytecode function without any children.
This can be used to further reduce frozen code size, and has the potential
to optimise other areas like importing.
Signed-off-by: Damien George <damien@micropython.org>
When building for a specific board this must be specified in make
submodules. I.e. make BOARD=STM32F769DISC submodules.
Signed-off-by: Rene Straub <rene@see5.ch>
In order to keep "import umodule" working, the existing mechanism is
replaced with a simple fallback to drop the "u".
This makes importing of built-ins no longer touch the filesystem, which
makes a typical built-in import take ~0.15ms rather than 3-5ms.
(Weak links were added in c14a81662c)
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This change makes it so the compiler and persistent code loader take a
mp_compiled_module_t* as their last argument, instead of returning this
struct. This eliminates a duplicate context variable for all callers of
these functions (because the context is now stored in the
mp_compiled_module_t by the caller), and also eliminates any confusion
about which context to use after the mp_compile_to_raw_code or
mp_raw_code_load function returns (because there is now only one context,
that stored in mp_compiled_module_t.context).
Reduces code size by 16 bytes on ARM Cortex-based ports.
Signed-off-by: Damien George <damien@micropython.org>
This shows how ports can add their own custom types/classes.
It is part of the unix coverage build, so we can use it for tests too.
Signed-off-by: Laurens Valk <laurens@pybricks.com>
Removes the need for the port to add anything to OBJS or SRC_QSTR.
Also makes it possible for user-C-modules to differentiate between code
that should be processed for QSTR vs other files (e.g. helpers and
libraries).
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Updates all README.md and docs, and manifests to `require("mip")`.
Also extend and improve the documentation on freezing and packaging.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
The separate A and RM toolchains have been discontinued and replaced
by a single toolchain. This updates the links to the RM toolchain to
the new toolchain.
Signed-off-by: David Lechner <david@pybricks.com>
This uses MP_REGISTER_ROOT_POINTER() to register the readline_history root
pointer array used by shared/readline.c and removes the registration from
all mpconfigport.h files.
This also required adding a new MICROPY_READLINE_HISTORY_SIZE config option
since not all ports used the same sized array.
Signed-off-by: David Lechner <david@pybricks.com>
This separates extmod source files from `py.mk`. Previously, `py.mk`
assumed that every consumer of the py/ directory also wanted to include
extmod/. However, this is not the case. For example, building mpy-cross
uses py/ but doesn't need extmod/.
This commit moves all extmod-specific items from `py.mk` to `extmod.mk` and
explicitly includes `extmod.mk` in ports that use it.
Signed-off-by: David Lechner <david@pybricks.com>
It's no longer needed because this macro is now processed after
preprocessing the source code via cpp (in the qstr extraction stage), which
means unused MP_REGISTER_MODULE's are filtered out by the preprocessor.
Signed-off-by: Damien George <damien@micropython.org>
The examples/natmod features0 and features1 examples now build and run on
ARMv6-M platforms. More complicated examples are not yet supported because
the compiler emits references to built-in functions like __aeabi_uidiv.
Signed-off-by: Damien George <damien@micropython.org>
If MICROPY_PY_SYS_PATH_ARGV_DEFAULTS is enabled (which it is by default)
then sys.path and sys.argv will be initialised and populated with default
values. This keeps all bare-metal ports aligned.
Signed-off-by: Damien George <damien@micropython.org>
Anywhere a module is mentioned, use its "non-u" name for consistency.
The "import module" vs "import umodule" is something of a FAQ, and this
commit intends to help clear that up. As a first approximation MicroPython
is Python, and so imports should work the same as Python and use the same
name, to a first approximation. The u-version of a module is a detail that
can be learned later on, when the user wants to understand more and have
finer control over importing.
Existing Python code should just work, as much as it is possible to do that
within the constraints of embedded systems, and the MicroPython
documentation should match the idiomatic way to write Python code.
With universal weak links for modules (via MICROPY_MODULE_WEAK_LINKS) users
can consistently use "import foo" across all ports (with the exception of
the minimal ports). And the ability to override/extend via "foo.py"
continues to work well.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Make and CMake builds are slightly different and these changes help make it
clear what to do in each case.
Signed-off-by: Damien George <damien@micropython.org>
It's a bit of a pitfall with user C modules that including them in the
build does not automatically enable them. This commit changes the docs and
examples for user C modules to encourage writers of user C modules to
enable them unconditionally. This makes things simpler and covers most use
cases.
See discussion in issue #6960, and also #7086.
Signed-off-by: Damien George <damien@micropython.org>
Documents the micropython.cmake file required to make user C modules
compatible with the CMake build system.
Signed-off-by: Phil Howard <phil@pimoroni.com>
Add most formatting-only commits to this file so that when used with
git blame, these commits are excluded and the output shows only the
interesting bits.
Jos Verlinde
Alessandro Gatti
Jeff Epler
Damien George
Jeff Epler
Volodymyr Shymanskyy
George Hopkins
Matt Trentini
Phil Howard
Angus Gratton
Jos Verlinde
Elecia White
Rene Straub
Jim Mussared
marble
Laurens Valk
David Lechner
Cem Eliguzel
Mike Causer
stijn