This commit makes use of the new native code loading changes to provide
a test bed for a target with the native compiler framework available but
without no code loading capabilities.
mpy-cross already fully disables native code loading, however those
changes do not really propagate anywhere outside `py/persistentcode`.
Recent changes in the code loading framework actually touch the core
code in a few more places, but no default CI target is configured to run
with the new native code loading defines switched off.
Given that mpy-cross doesn't really need to load code anyway, it is a
good target for setting up a configuration that goes a bit deeper when
it comes to disabling code loading. Since mpy-cross is built several
times during the CI process, it can be repurposed as an early warning
system for issues related to these new changes. Normal operation should
not be affected in any way.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit performs the necessary changes to handle an additional RV32
CPU extension flag, for the Zcmp extension in this case.
The changes are not limited to RV32-only code, as other parts of the
tooling need to be modified for this: the testing framework has to be
made aware that an extra bit can be set in sys.implementation._mpy and
needs to know how it is called, and "mpy-cross" must be able to actually
set that flag bit in the first place via the appropriate command line
argument.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit extends "mpy-cross"'s parsing of the architecture flags
value command line, allowing raw integer values as well as flag strings.
Integers can be represented as either decimal, binary, or hexadecimal
values, using the usual C language prefixes to mark non-base 10 values.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit introduces the MPY architecture flags checking code specific
for the RV32 target, currently checking for the only additional
extension that is supported by the runtime: Zba.
The warnings inside "mpy-cross" have also been removed since now there
is a way to reject incompatible MPY files at runtime.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit extends the MPY file format in a backwards-compatible way to
store an encoded form of architecture-specific flags that have been
specified in the "mpy-cross" command line, or that have been explicitly
set as part of a native emitter configuration.
The file format changes are as follows:
* The features byte, previously containing the target native
architecture and the minor file format version, now claims bit 6 as a
flag indicating the presence of an encoded architecture flags integer
* If architecture flags need to be stored, they are placed right after
the MPY file header.
This means that properly-written MPY parsers, if encountering a MPY file
containing encoded architecture flags, should raise an error since no
architecture identifiers have been defined that make use of bits 6 and
7 in the referenced header byte. This should give enough guarantees of
backwards compatibility when this feature is used (improper parsers were
subjected to breakage anyway).
The encoded architecture flags could have been placed at the end, but:
* Having them right after the header makes the architecture
compatibility checks occur before having read the whole file in memory
(which still happens on certain platforms as the reader may be backed
by a memory buffer), and prevents eventual memory allocations that do
not take place if the module is rejected early
* Properly-written MPY file parsers should have checked the upper two
bits of the flags byte to be actually zero according to the format
specification available right before this change, so no assumptions
should have been made on the exact order of the chunks for an
unexpected format.
The meaning of the architecture flags value is backend-specific, with
the only common characteristic of being a variable-encoded unsigned
integer for the time being.
The changes made to the file format effectively limit the number of
possible target architectures to 16, of which 13 are already claimed.
There aren't that many new architectures planned to be supported for the
lifetime of the current MPY file format, so this change still leaves
space for architecture updates if needed.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
Assuming proper C99 language support, we can select "the int type as big as
a pointer" (most of the time) or "the 64-bit int type" (nanboxing with
REPR_D), and then define everything else automatically.
This simplifies port configuration files. And the types can still be
overridden if needed.
Signed-off-by: Jeff Epler <jepler@unpythonic.net>
This commit adds a new command line switch to inform the RV32 emitter to
use Zba opcodes in its output.
A new implementation-specific option was introduced, called
"-march-flags", that will contain a list of additional
architecture-specific flags to pass to the chosen native emitter
implementation.
At the moment only the RV32 emitter can make use of this command line
facility: if the architecture flags string equals to "zba"
(case-sensitive), then the native emitter will emit Zba opcodes if it
has a chance to do so.
At the moment there is no check on whether additional architecture flags
using to build a MPY file are compatible with the target the output code
is run on, so use this with caution.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
MPY files can now hold data to be run on RV64IMC. This can be
accomplished by passing the `-march=rv64imc` flag to mpy-cross.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
Add a new MICROPY_COMP_CONST_FLOAT feature, enabled by in mpy-cross and
when compiling with MICROPY_CONFIG_ROM_LEVEL_CORE_FEATURES. The new
feature leverages the code of MICROPY_COMP_CONST_FOLDING to support folding
of floating point constants.
If MICROPY_COMP_MODULE_CONST is defined as well, math module constants are
made available at compile time. For example:
_DEG_TO_GRADIANT = const(math.pi / 180)
_INVALID_VALUE = const(math.nan)
A few corner cases had to be handled:
- The float const folding code should not fold expressions resulting into
complex results, as the mpy parser for complex immediates has
limitations.
- The constant generation code must distinguish between -0.0 and 0.0, which
are different even if C consider them as ==.
This change removes previous limitations on the use of `const()`
expressions that would result in floating point number, so the test cases
of micropython/const_error have to be updated.
Additional test cases have been added to cover the new repr() code (from a
previous commit). A few other simple test cases have been added to handle
the use of floats in `const()` expressions, but the float folding code
itself is also tested when running general float test cases, as float
expressions often get resolved at compile-time (with this change).
Signed-off-by: Yoctopuce dev <dev@yoctopuce.com>
Currently, mpy-cross will crash if called as:
mpy-cross -X emit=native foo.py
because it tries to use the native emitter with no target architecture set.
Fix that by checking that an architecture is set when `-X emit=native` or
`-X emit=viper` is used.
Signed-off-by: Damien George <damien@micropython.org>
This commit renames the NORETURN macro, indicating to the compiler
that a function does not return, into MP_NORETURN to maintain the same
naming convention of other similar macros.
To maintain compaitiblity with existing code NORETURN is aliased to
MP_NORETURN, but it is also deprecated for MicroPython v2.
This changeset was created using a similar process to
decf8e6a8b ("all: Remove the "STATIC"
macro and just use "static" instead."), with no documentation or python
scripts to change to reflect the new macro name.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit adds support for writing inline assembler functions when
targeting a RV32IMC processor.
Given that this takes up a bit of rodata space due to its large
instruction decoding table and its extensive error messages, it is
enabled by default only on offline targets such as mpy-cross and the
qemu port.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This emitter prints out pseudo-machine instructions, instead of the usual
output of the native emitter. It can be enabled on any port via
`MICROPY_EMIT_NATIVE_DEBUG` (make sure other native emitters are disabled)
but the easiest way to use it is with mpy-cross:
$ mpy-cross -march=debug file.py
Signed-off-by: Damien George <damien@micropython.org>
MPY files can now hold generated RV32IMC native code. This can be
accomplished by passing the `-march=rv32imc` flag to mpy-cross.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
The STATIC macro was introduced a very long time ago in commit
d5df6cd44a. The original reason for this was
to have the option to define it to nothing so that all static functions
become global functions and therefore visible to certain debug tools, so
one could do function size comparison and other things.
This STATIC feature is rarely (if ever) used. And with the use of LTO and
heavy inline optimisation, analysing the size of individual functions when
they are not static is not a good representation of the size of code when
fully optimised.
So the macro does not have much use and it's simpler to just remove it.
Then you know exactly what it's doing. For example, newcomers don't have
to learn what the STATIC macro is and why it exists. Reading the code is
also less "loud" with a lowercase static.
One other minor point in favour of removing it, is that it stops bugs with
`STATIC inline`, which should always be `static inline`.
Methodology for this commit was:
1) git ls-files | egrep '\.[ch]$' | \
xargs sed -Ei "s/(^| )STATIC($| )/\1static\2/"
2) Do some manual cleanup in the diff by searching for the word STATIC in
comments and changing those back.
3) "git-grep STATIC docs/", manually fixed those cases.
4) "rg -t python STATIC", manually fixed codegen lines that used STATIC.
This work was funded through GitHub Sponsors.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
If a non-string buffer was passed to execfile, then it would be passed
as a non-null-terminated char* to mp_lexer_new_from_file.
This changes mp_lexer_new_from_file to take a qstr instead (as in almost
all cases a qstr will be created from this input anyway to set the
`__file__` attribute on the module).
This now makes execfile require a string (not generic buffer) argument,
which is probably a good fix to make anyway.
Fixes issue #12522.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This way, a bare `-` is never interpreted as an option, even before
`--`. Filenames starting with `-` still need to be put after `--`.
Signed-off-by: Armin Brauns <armin.brauns@embedded-solutions.at>
Unless -o is given, output defaults to stdout unless a source file is
given (in which case the source file name is used to derive an output
file name).
Signed-off-by: Armin Brauns <armin.brauns@embedded-solutions.at>
Code in tools/mpy-tool.py and py/frozenmod.c relies on the source file
path encoded in a .mpy file to have forward slashes (e.g. by searching
for '/__init__.py'). Enforce that when creating these files, thereby
fixing import of .mpy files and frozen modules not working before
because they could have backslashes when built with the windows port.
This change makes it so the compiler and persistent code loader take a
mp_compiled_module_t* as their last argument, instead of returning this
struct. This eliminates a duplicate context variable for all callers of
these functions (because the context is now stored in the
mp_compiled_module_t by the caller), and also eliminates any confusion
about which context to use after the mp_compile_to_raw_code or
mp_raw_code_load function returns (because there is now only one context,
that stored in mp_compiled_module_t.context).
Reduces code size by 16 bytes on ARM Cortex-based ports.
Signed-off-by: Damien George <damien@micropython.org>
If `-march` isn't set then it means the user hasn't thought about it, or in
the case of freezing, MPY_CROSS_FLAGS isn't set. It's almost certainly
going to lead to problems, as there's no reason why the host architecture
is likely to be the right choice.
Compiling regular Python code is unaffected, but if `@native`/`@viper` is
used, the compiler will raise `SyntaxError: invalid arch`.
For situations where you explicitly want to use the host architecture (e.g.
for running tests on the unix port), added -march=host that keeps the old
behavior.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This makes it so that all a port needs to do is set the relevant variables
and "include extmod.mk" and doesn't need to worry about adding anything to
OBJ, CFLAGS, SRC_QSTR, etc.
Make all extmod variables (src, flags, etc) private to extmod.mk.
Also move common/shared, extmod-related fragments (e.g. wiznet, cyw43,
bluetooth) into extmod.mk.
Now that SRC_MOD, CFLAGS_MOD, CXXFLAGS_MOD are unused by both extmod.mk
(and user-C-modules in a previous commit), remove all uses of them from
port makefiles.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Needed to be updated to use build/mpy-cross.
Also fixes some other issues in the Python wrapper:
- Rename find_mpy_cross_binary to _find_mpy_cross_binary
- Fix passing of -march arg.
- Decode stdout from subprocess.
- Print stdout from mpy-cross in __main__.py.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
The intent is to allow us to make breaking changes to the native ABI (e.g.
changes to dynruntime.h) without needing the bytecode version to increment.
With this commit the two bits previously used for the feature flags (but
now unused as of .mpy version 6) encode a sub-version. A bytecode-only
.mpy file can be loaded as long as MPY_VERSION matches, but a native .mpy
(i.e. one with an arch set) must also match MPY_SUB_VERSION. This allows 3
additional updates to the native ABI per bytecode revision.
The sub-version is set to 1 because the previous commits that changed the
layout of mp_obj_type_t have changed the native ABI.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Signed-off-by: Damien George <damien@micropython.org>
Rather than invoking mpy-cross directly via system/subprocess in our build
tools and other tools, this provides a Python interface for it.
Based on https://gitlab.com/alelec/mpy_cross (with the intention of
eventually replacing that as the "official" pypi distribution once setup.py
etc are added).
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This allows mpy-cross to dynamically select whether ARMv7-M float
instructions are supported in @micropython.asm_thumb functions.
Signed-off-by: Damien George <damien@micropython.org>
This follows on from a5324a1074 and allows
mpy-cross to dynamically select whether ARMv7-M instructions are supported
in @micropython.asm_thumb functions.
The config option MICROPY_EMIT_INLINE_THUMB_ARMV7M is no longer needed, it
is now controlled by MICROPY_EMIT_THUMB_ARMV7M.
Signed-off-by: Damien George <damien@micropython.org>
The following changes are made:
- If MICROPY_VFS is enabled then mp_vfs_import_stat and mp_vfs_open are
automatically used for mp_import_stat and mp_builtin_open respectively.
- If MICROPY_PY_IO is enabled then "open" is automatically included in the
set of builtins, and points to mp_builtin_open_obj.
This helps to clean up and simplify the most common port configuration.
Signed-off-by: Damien George <damien@micropython.org>
Prior to this commit, even with unicode disabled .py and .mpy files could
contain unicode characters, eg by entering them directly in a string as
utf-8 encoded.
The only thing the compiler disallowed (with unicode disabled) was using
\uxxxx and \Uxxxxxxxx notation to specify a character within a string with
value >= 0x100; that would give a SyntaxError.
With this change mpy-cross will now accept \u and \U notation to insert a
character with value >= 0x100 into a string (because the -mno-unicode
option is now gone, there's no way to forbid this). The runtime will
happily work with strings with such characters, just like it already works
with strings with characters that were utf-8 encoded directly.
This change simplifies things because there are no longer any feature
flags in .mpy files, and any bytecode .mpy will now run on any target.
Signed-off-by: Damien George <damien@micropython.org>
The xmlns attribute is required for older msbuild version (e.g. for
VS2015). Add it where needed, and reorder the attributes so all
files look the same.
Damien George
Alessandro Gatti
Jeff Epler
Yoctopuce dev
Angus Gratton
Jim Mussared
Armin Brauns
stijn
David Lechner
Chris Waggoner
David Yang