This commit introduces a test that should check whether viper load or
store operations won't clobber either the buffer address or the index
value being used.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit adds support to the `marshal` module to be able to dump
functions that have child functions. For example:
import marshal
def f():
def child():
return 1
return child
marshal.dumps(f.__code__)
It also covers the case of marshalling functions that use list
comprehensions, because a list comprehension uses a child function.
This is made possible by the newly enhanced
`mp_raw_code_save_fun_to_bytes()` that can now handle nested functions.
Unmarshalling via `marshal.loads()` already supports nested functions
because it uses the standard `mp_raw_code_load_mem()` function which is
used to import mpy files (and hence can handle all possibilities).
Signed-off-by: Damien George <damien@micropython.org>
This variant of `schedule.py` explicitly calls a zero sleep. The existing
variant is kept to ensure the scheduler is called between bytecodes.
Signed-off-by: Jeff Epler <jepler@unpythonic.net>
Add a general normalize_newlines() function that handles newline variations
(\\r\\r\\n, \\r\\n) to \\n while preserving literal \\r characters that are
part of test content.
This provides a robust solution for cross-platform test compatibility,
particularly addressing PTY double-newline issues that can occur with some
terminal implementations.
The function is applied to all test output before comparison, eliminating
platform-specific newline issues.
Includes a unit test to verify the normalization behavior.
Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>
This commit extends the "micropython/import_mpy_native_gc" test with a
pre-built serialised MPY file built for RV32.
Before this commit, the test was skipped on said platform due to the
lack of a known-good MPY file to test the import procedure against.
Now the test is executed as part of the general CI test checks.
The MPY file's Makefile script was also updated to cater for modern
Linux environments where the required Python interpreter's command name
isn't "python", but "python3" instead (this occurs when using the past
two LTS Ubuntu releases for example). This is the same kind of change
made in 31a008c6e2.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit adds the currently supported architecture flags value as the
upper part of "sys.implementation._mpy".
This had the side effect of perturbing quite a bit of testing
infrastructure and invalidating documentation related to MPY files. To
make the test suite run successfully and keep the documentation in sync
the following changes have been made:
* The target info feature check file now isolates eventual architecture
flags and adds them as a separate field
* The test runner now picks up the new architecture flags field, reports
it to STDOUT if needed and stores it for future uses
* Relevant test files for MPY files import code had to be updated to
mask out the architecture flags bits in order to perform correctly
* MPY file format documentation was updated to show how to mask off and
properly display the architecture flags information.
This works out of the box if the flag bits can fit in a smallint value
once merged with the MPY file header value.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
With the aim of getting consistency, and removing the need to learn an
additional term, replace uses of uPy/uPython with MPy/MicroPython.
Rule of thumb was to use "MPy" abbreviation where "CPy" is used nearby, but
the full word MicroPython otherwise.
This work was funded through GitHub Sponsors.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
That is, an object whose type defines the protocol slot.
Note that due to protocol confusion, a variant of the original crasher that
returned e.g., a machine.Pin instance could still lead to a crash (#17852).
Fixes issue #17841.
Signed-off-by: Jeff Epler <jepler@gmail.com>
Signed-off-by: Jeff Epler <jepler@unpythonic.net>
This commit changes the "viper_ptr*_store_boundary" tests to make them
fail more gracefully in low memory conditions.
The original version of the tests compiled viper code blocks on the fly
when it needed them, making them fail at runtime on some boards that do
not come with enough memory for this test. This clashes with
"run-tests.py"'s ability to look for a particular signature to mark
tests as skipped due to not enough memory.
Now compiled code blocks are generated at the beginning of the test
inside an appropriate exception handler. In case of a memory error when
pre-compiling a code block, the running test exits reporting a low
memory condition to the test runner. This allows to have clean test
runs on all platforms when it comes to viper pointer tests.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit fixes the encoding of conditional branch opcodes emitted for
ARMv7-M targets, when the emitter decides to use the T3 encoding for
said operation.
Fields J1 and J2 are now present in the generated opcode word, along
with correcting some minor issues in bitmasks and shifts computation.
This fixes#17940.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
ringbuffer.size must be at least 2, and is a 16-bit quantity.
This fixes several cases including the one the fuzzer discovered, which
would lead to a fatal signal when accessing the object.
Fixes issue #17847.
Signed-off-by: Jeff Epler <jepler@gmail.com>
This commit provides an implementation for viper boundary tests that can
work even without big int support.
Since it uses a fixed-size buffer to hold values to work with, this
should work on any platform as long as its integers are at least 32 bits
wide, regardless its configuration on how big integers can get.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
When running the viper boundary tests, assert that the offset stores don't
clobber the base register, which is saved and temporarily modified on some
architectures.
Signed-off-by: Chris Webb <chris@arachsys.com>
Add a new MICROPY_COMP_CONST_FLOAT feature, enabled by in mpy-cross and
when compiling with MICROPY_CONFIG_ROM_LEVEL_CORE_FEATURES. The new
feature leverages the code of MICROPY_COMP_CONST_FOLDING to support folding
of floating point constants.
If MICROPY_COMP_MODULE_CONST is defined as well, math module constants are
made available at compile time. For example:
_DEG_TO_GRADIANT = const(math.pi / 180)
_INVALID_VALUE = const(math.nan)
A few corner cases had to be handled:
- The float const folding code should not fold expressions resulting into
complex results, as the mpy parser for complex immediates has
limitations.
- The constant generation code must distinguish between -0.0 and 0.0, which
are different even if C consider them as ==.
This change removes previous limitations on the use of `const()`
expressions that would result in floating point number, so the test cases
of micropython/const_error have to be updated.
Additional test cases have been added to cover the new repr() code (from a
previous commit). A few other simple test cases have been added to handle
the use of floats in `const()` expressions, but the float folding code
itself is also tested when running general float test cases, as float
expressions often get resolved at compile-time (with this change).
Signed-off-by: Yoctopuce dev <dev@yoctopuce.com>
These tests all depend on generating arbitrarily long (>64-bit) integers.
It would be possible to have these tests work in this case I think, as the
results are always masked to shorter values. But quite fiddly. So just
rename them so they are automatically skipped if the target doesn't have
big int support.
This work was funded through GitHub Sponsors.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
This commit adds a fast-path optimisation for when a BUILD_SLICE is
immediately followed by a LOAD/STORE_SUBSCR for a native type, to avoid
needing to allocate the slice on the heap.
In some cases (e.g. `a[1:3] = x`) this can result in no allocations at all.
We can't do this for instance types because the get/set/delattr
implementation may keep a reference to the slice.
Adds more tests to the basic slice tests to ensure that a stack-allocated
slice never makes it to Python, and also a heapalloc test that verifies
(when using bytecode) that assigning to a slice is no-alloc.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Signed-off-by: Damien George <damien@micropython.org>
The test runner expects `print("SKIP")` to be followed by
`raise SystemExit`. Otherwise it waits for 10 seconds for the target to
do a soft reset before timing out and continuing.
Signed-off-by: Damien George <damien@micropython.org>
This commit reworks the Viper pointer boundary tests in order to make
them more accurate and easier to extend.
The tests are now easier to reason about in their output, using easier
to read values, and bit thresholds are now more configurable. If a new
conditional code sequence is introduced, adding a new bit threshold is
just a matter of adding a value into a tuple at the beginning of the
relevant test file.
Load tests have also been made more accurate, with better function
templates to test register-indexed operations.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit adds a series of test cases to exercise the Viper code
generator load/store emitting capabilities on certain boundary
conditions.
The new test cases check whether the emitted load/store code performs
correctly when dealing with specific memory offsets, which trigger
specific code generation sequences on different architectures.
Right now the cases are for unsigned offsets whose bitmasks span up to
5, 8, and 12 bits (respectively Arm/Thumb, Xtensa, RV32).
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This allows retrieving the code object of a function using
`function.__code__`, and then reconstructing a function from a code object
using `FunctionType(code_object)`.
This feature is controlled by `MICROPY_PY_FUNCTION_ATTRS_CODE` and is
enabled at the full-features level.
Signed-off-by: Damien George <damien@micropython.org>
A return value of 0 from Python-level `ioctl()` means success, but if
that's returned unconditionally it means that the method supports all
ioctl calls, which is not true. Returning 0 without doing anything can
potentially lead to a crash, eg for MP_STREAM_SEEK which requires returning
a value in the passed-in struct pointer.
This commit makes it so that all `ioctl()` methods respond only to
MP_STREAM_CLOSE, ie they return -1 (indicating error) for all other ioctl
calls.
Signed-off-by: Damien George <damien@micropython.org>
This commit adds a new `RingIO` type which exposes the internal ring-buffer
code for general use in Python programs. It has the stream interface
making it similar to `StringIO` and `BytesIO`, except `RingIO` has a fixed
buffer size and is automatically safe when reads and writes are in
different threads or an IRQ.
This new type is enabled at the "extra features" ROM level.
Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>
Also define `mp_type_bytearray`. These all help to write native modules.
Signed-off-by: Brian Pugh <bnp117@gmail.com>
Signed-off-by: Damien George <damien@micropython.org>
This is required because the .mpy native ABI was changed by the
introduction of `mp_proto_fun_t`, see commits:
- 416465d81e
- 5e3006f117
- e2ff00e811
And three `mp_binary` functions were added to `mp_fun_table` in
commit d2276f0d41.
Signed-off-by: Damien George <damien@micropython.org>
If a non-string buffer was passed to execfile, then it would be passed
as a non-null-terminated char* to mp_lexer_new_from_file.
This changes mp_lexer_new_from_file to take a qstr instead (as in almost
all cases a qstr will be created from this input anyway to set the
`__file__` attribute on the module).
This now makes execfile require a string (not generic buffer) argument,
which is probably a good fix to make anyway.
Fixes issue #12522.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Because mpy_ld.py doesn't know the target object representation, it emits
instances of `MP_OBJ_NEW_QSTR(MP_QSTR_Foo)` as const string objects, rather
than qstrs. However this doesn't work for map keys (e.g. for a locals dict)
because the map has all_keys_are_qstrs flag is set (and also auto-complete
requires the map keys to be qstrs).
Instead, emit them as regular qstrs, and make a functioning MP_OBJ_NEW_QSTR
function available (via `native_to_obj`, also used for e.g. making
integers).
Remove the code from mpy_ld.py to emit qstrs as constant strings, but leave
behind the scaffold to emit constant objects in case we want to do use this
in the future.
Strictly this should be a .mpy sub-version bump, even though the function
table isn't changing, it does lead to a change in behavior for a new .mpy
running against old MicroPython. `mp_native_to_obj` will incorrectly return
the qstr value directly as an `mp_obj_t`, leading to unexpected results.
But given that it's broken at the moment, it seems unlikely that anyone is
relying on this, so it's not work the other downsides of a sub-version bump
(i.e. breaking pure-Python modules that use @native). The opposite case of
running an old .mpy on new MicroPython is unchanged, and remains broken in
exactly the same way.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
So it can run on targets with low memory, eg esp8266.
Also enable the viper_4args() sub-test, which is now supported.
Signed-off-by: Damien George <damien@micropython.org>
This prevents a very subtle bug caused by writing e.g. `bytearray('\xfd')`
which gives you `(0xc3, 0xbd)`.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
The intent is to allow us to make breaking changes to the native ABI (e.g.
changes to dynruntime.h) without needing the bytecode version to increment.
With this commit the two bits previously used for the feature flags (but
now unused as of .mpy version 6) encode a sub-version. A bytecode-only
.mpy file can be loaded as long as MPY_VERSION matches, but a native .mpy
(i.e. one with an arch set) must also match MPY_SUB_VERSION. This allows 3
additional updates to the native ABI per bytecode revision.
The sub-version is set to 1 because the previous commits that changed the
layout of mp_obj_type_t have changed the native ABI.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Signed-off-by: Damien George <damien@micropython.org>
Support for architecture-specific qstr linking was removed in
d4d53e9e11, where native code was changed to
access qstr values via qstr_table. The only remaining use for the special
qstr link table in persistentcode.c is to support native module written in
C, linked via mpy_ld.py. But native modules can also use the standard
module-level qstr_table (and obj_table) which was introduced in the .mpy
file reworking in f2040bfc7e.
This commit removes the remaining native qstr liking support in
persistentcode.c's load_raw_code function, and adds two new relocation
options for constants.qstr_table and constants.obj_table. mpy_ld.py is
updated to use these relocations options instead of the native qstr link
table.
Signed-off-by: Damien George <damien@micropython.org>
And make it so this test can run on any target.
LED and time testing has been removed from this test, that can now be
tested using: ./run-tests.py --via-mpy --emit native.
Signed-off-by: Damien George <damien@micropython.org>
