4382 Commits

Author	SHA1	Message	Date
Alessandro Gatti	3044233ea3	py/misc: Add a popcount(uint32_t) implementation. ... This makes the existing popcount(uint32_t) implementation found in the RV32 emitter available to the rest of the codebase. This version of popcount will use intrinsic or builtin implementations if they are available, falling back to a generic implementation if that is not the case. Signed-off-by: Alessandro Gatti <a.gatti@frob.it>	2025-01-01 10:44:53 +01:00
Damien George	97518064cd	py/persistentcode: Add support for loading .mpy files from a ROM reader. ... This adds an optimisation for loading .mpy files from a reader that points to ROM. In such a case qstr, str and bytes data, along with bytecode, are all referenced in-place in ROM. Signed-off-by: Damien George <damien@micropython.org>	2024-12-23 13:04:55 +11:00
Damien George	f870e8d2d4	py/reader: Provide mp_reader_try_read_rom() function. ... This allows accessing data directly in ROM if the reader supports it. Signed-off-by: Damien George <damien@micropython.org>	2024-12-23 13:04:54 +11:00
Damien George	a1c32101ac	py/qstr: Add qstr_from_strn_static() helper function. ... Allows an interned string to reference static/ROM data, instead of allocating it on the GC heap. Signed-off-by: Damien George <damien@micropython.org>	2024-12-23 13:04:54 +11:00
Damien George	50637ff239	extmod/vfs_rom: Add VfsRom filesystem object. ... This commit defines a new ROMFS filesystem for storing read-only files that can be memory mapped, and a new VfsRom driver. Files opened from this filesystem support the buffer protocol. This allows naturally getting the memory-mapped address of the file using: - memoryview(file) - uctypes.addressof(file) Furthermore, if these files are .mpy files then their content can be referenced in-place when importing. Such imports take up a lot less RAM than importing from a normal filesystem. This is essentially dynamically frozen .mpy files, building on the revamped v6 .mpy file format. Signed-off-by: Damien George <damien@micropython.org>	2024-12-23 13:04:54 +11:00
Alessandro Gatti	4bf087b272	py/dynruntime.mk: Delete compiled module file on clean. ... This commit adds the compiled native module file to the list of files to remove when `make clean` is issued in a native module source directory. Signed-off-by: Alessandro Gatti <a.gatti@frob.it>	2024-12-23 10:03:02 +11:00
Alessandro Gatti	6760e00817	tools/mpy_ld.py: Add native modules support for RV32 code. ... This commit adds support for RV32IMC native modules, as in embedding native code into a self-contained MPY module and and make its exported functions available to the MicroPython environment. Signed-off-by: Alessandro Gatti <a.gatti@frob.it>	2024-12-23 10:02:20 +11:00
Damien George	8b6bd43eab	extmod/vfs: Guard mutating fs functions with MICROPY_VFS_WRITABLE. ... Enabled by default. Useful for ports that need the VFS but don't have any writable filesystems. Signed-off-by: Damien George <damien@micropython.org>	2024-12-20 21:58:50 +11:00
Andrew Leech	fdd606dd53	py/mkrules.mk: Use partial clone for submodules if available. ... MicroPython relies on a number of submodules for third party and chip vendor libraries. Users need to check these out before building their desired ports and Github Actions CI here needs to clone them all multiple times for every build. Many of these are getting significantly larger over time, slowing down usage and consuming more disk space. Newer versions of git have features to avoid pulling all historic / blob data which can have a significant impact of total data use. This commit uses a standard feature of git to do a partial clone, with automatic fallback to previous behavior on error. Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>	2024-12-10 17:17:17 +11:00
Alessandro Gatti	2a8f6047ff	py/emitglue: Fix clear cache builtin warning on Clang for AArch32. ... This commit fixes a warning occurring on Clang when calling `__builtin___clear_cache` with non-void pointers for its start and end memory area locations. The code now uses a char pointer for the end location, and it still builds without warnings on GCC. Signed-off-by: Alessandro Gatti <a.gatti@frob.it>	2024-12-10 12:04:14 +11:00
stijn	0f7d68043f	py/misc: Fix msvc and C++ compatibility. ... Use an explicit cast to suppress the implicit conversion which started popping up in recent compiler versions (and wasn't there yet in 07bf3179). Signed-off-by: stijn <stijn@ignitron.net>	2024-12-10 11:30:59 +11:00
Angus Gratton	160a4812cd	esp32: Pass V=1 or BUILD_VERBOSE through to idf.py when building. ... Allows verbose build to work the same on esp32 port as other ports. To minimise copy/paste, split the BUILD_VERBOSE section of mkenv.mk out to its own verbose.mk and include this in the port Makefile. Signed-off-by: Angus Gratton <angus@redyak.com.au>	2024-12-10 11:28:38 +11:00
Angus Gratton	3b3b48892f	py/objfloat: Workaround non-constant NAN definition on Windows MSVC. ... Recent MSVC versions have changed the definition of NAN to a non-constant expression! This is a bug, C standard says it should be a constant. Good explanation and workaround at: https://stackoverflow.com/a/79199887 This work was funded through GitHub Sponsors. Signed-off-by: Angus Gratton <angus@redyak.com.au>	2024-11-28 23:11:24 +11:00
Andrew Leech	78d017fc4e	py/usermod.cmake: If USER_C_MODULES is a folder add micropython.cmake. ... This mirrors how it works when using a Makefile. Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>	2024-11-20 17:38:39 +11:00
Andrew Leech	dccd206f4c	py/usermod.cmake: Add check that any specified USER_C_MODULES exists. ... Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>	2024-11-20 17:34:41 +11:00
Andrew Leech	b65e89107c	py/py.mk: Add check that any specified USER_C_MODULES folder exists. ... Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>	2024-11-20 17:34:41 +11:00
Jan Sturm	7e1098befe	py/objdeque: Fix buffer overflow in deque_subscr. ... In `deque_subscr()`, if `index_val` equals `self->alloc`, the index correction `index_val -= self->alloc` does not execute, leading to an out-of-bounds access in `self->items[index_val]`. The fix in this commit ensures that the index correction is applied whenever `index_val >= self->alloc`, preventing access beyond the allocated buffer size. Signed-off-by: Jan Sturm <jansturm92@googlemail.com>	2024-11-04 11:21:56 +11:00
Alessandro Gatti	d34b15ac6f	pic16bit: Make it build with recent XC16 versions. ... The PIC16 port didn't catch up with the other ports, so it required a bit of work to make it build with the latest version of XC16. Signed-off-by: Alessandro Gatti <a.gatti@frob.it>	2024-10-30 17:21:30 +11:00
Damien George	44dc6eb30d	all: Bump version to 1.25.0-preview. ... Signed-off-by: Damien George <damien@micropython.org>	2024-10-28 11:08:24 +11:00
Damien George	f212bbe837	all: Bump version to 1.24.0. ... Signed-off-by: Damien George <damien@micropython.org>	2024-10-26 01:43:23 +11:00
iabdalkader	2644f577f1	extmod/modtls_mbedtls: Add a thread-global ptr for current SSL context. ... This is necessary for mbedTLS callbacks that do not carry any user state, so those callbacks can be customised per SSL context. Signed-off-by: iabdalkader <i.abdalkader@gmail.com>	2024-10-25 01:12:36 +11:00
Damien George	1b89c503db	py/objtype: Don't delegate lookup of descriptor methods to __getattr__. ... When descriptors are enabled, lookup of the `__get__`, `__set__` and `__delete__` descriptor methods should not be delegated to `__getattr__`. That follows CPython behaviour. Signed-off-by: Damien George <damien@micropython.org>	2024-10-16 15:03:30 +11:00
Phil Howard	525fce7170	py/usermod.cmake: Check target exists in usermod_gather_sources. ... Check a target exists before accessing properties. Otherwise usermod_gather_sources would recurse into garbage property names and break. Signed-off-by: Phil Howard <phil@gadgetoid.com>	2024-10-15 11:51:56 +11:00
stijn	338df1ae35	py/objtype: Allow passing keyword arguments to native base __init__. ... Allowing passing keyword arguments to a native base's __init__, i.e. `make_new` in the C code. Previously only positional arguments were allowed. The main trade-off in this commit is that every call to the native base's `make_new` is now going to be preceded by a call to `mp_map_init_fixed_table` even though most of what that does is unused and instead it merely serves as a way to pass the number of keyword arguments. Fixes issue #15465. Signed-off-by: stijn <stijn@ignitron.net>	2024-10-07 11:37:52 +11:00
Alessandro Gatti	ca220b5fc5	py/nlrrv64: Add RISC-V RV64I NLR implementation. ... Add custom NLR support for 64 bits RISC-V RV64I targets. Signed-off-by: Alessandro Gatti <a.gatti@frob.it>	2024-10-03 12:06:19 +10:00
Alessandro Gatti	17d8234458	py/parse: Remove old esp32 compiler workaround. ... The ESP32 port contains a workaround to avoid having a certain function in `py/parse.c` being generated incorrectly. The compiler in question is not part of any currently supported version of ESP-IDF anymore, and the problem inside the compiler (well, assembler in this case) has been corrected a few years ago. This commit removes all traces of that workaround from the source tree. Signed-off-by: Alessandro Gatti <a.gatti@frob.it>	2024-09-27 00:03:16 +10:00
Alessandro Gatti	43b05afdf5	py/mpz: Skip separators when running out of digits to print. ... This commit fixes the addition of a stray separator before the number when printing an MPZ-backed integer and the first group is three digits long. This fixes #8984. Signed-off-by: Alessandro Gatti <a.gatti@frob.it>	2024-09-26 23:39:06 +10:00
Damien George	5b22bde044	py/persistentcode: Explicitly track native BSS/rodata when needed. ... Signed-off-by: Damien George <damien@micropython.org>	2024-09-26 22:15:28 +10:00
Andrew Leech	7e14680a83	py/objringio: Add micropython.RingIO() interface for general use. ... This commit adds a new `RingIO` type which exposes the internal ring-buffer code for general use in Python programs. It has the stream interface making it similar to `StringIO` and `BytesIO`, except `RingIO` has a fixed buffer size and is automatically safe when reads and writes are in different threads or an IRQ. This new type is enabled at the "extra features" ROM level. Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>	2024-09-19 18:00:44 +10:00
Angus Gratton	52a593cdb1	py/scheduler: Only run callbacks on the main thread if GIL is disabled. ... Otherwise it's very difficult to reason about thread safety in a scheduler callback, as it can run at any time on any thread - including racing against any bytecode operation on any thread. This work was funded through GitHub Sponsors. Signed-off-by: Angus Gratton <angus@redyak.com.au>	2024-09-19 13:17:01 +10:00
Amirreza Hamzavi	cb7e99098e	py/objint: Make byteorder argument optional in int.from_bytes() method. ... This was made optional in CPython 3.11. Signed-off-by: Amirreza Hamzavi <amirrezahamzavi2000@gmail.com>	2024-09-02 14:15:45 +10:00
Amirreza Hamzavi	0b432b3306	py/objint: Make length argument optional in int.to_bytes() method. ... This was made optional in CPython 3.11. Signed-off-by: Amirreza Hamzavi <amirrezahamzavi2000@gmail.com>	2024-09-02 14:14:36 +10:00
Amirreza Hamzavi	80c5e76483	py/objint: Make byteorder argument optional in int.to_bytes() method. ... This was made optional in CPython 3.11. Signed-off-by: Amirreza Hamzavi <amirrezahamzavi2000@gmail.com>	2024-09-02 14:14:22 +10:00
Christian Walther	0b7f6e1d3d	py/mkrules.mk: Fix 'make submodules' when building out-of-tree. ... When MicroPython is used as a submodule and built from the containing project, e.g. for the embed port, `make submodules` fails because it goes looking for the sub-sub-module paths in the outer repository instead of in the micropython repository. Fix this by invoking git inside the micropython submodule. Signed-off-by: Christian Walther <cwalther@gmx.ch>	2024-08-26 12:30:24 +10:00
Alessandro Gatti	7d8b2d89cc	py/asmrv32: Use REG_TEMP2 whenever possible. ... The RV32 emitter used an additional temporary register, as certain code sequences required extra storage. This commit removes its usage in all but one case, using REG_TEMP2 instead. Signed-off-by: Alessandro Gatti <a.gatti@frob.it>	2024-08-19 15:53:50 +10:00
Alessandro Gatti	da0e027fa5	py/asmrv32: Emit C.LW opcodes only when necessary. ... The RV32 emitter sometimes generated short load opcodes even when it was not supposed to. This commit fixes an off-by-one error in its offset eligibility range calculation and corrects one case of offset calculation, operating on the raw label index number rather than its effective offset in the stack (C.LW assumes all loads are word-aligned). Signed-off-by: Alessandro Gatti <a.gatti@frob.it>	2024-08-19 15:53:50 +10:00
Alessandro Gatti	326e1149ec	py/asmrv32: Fix short/long jumps scheduling. ... The RV32 emitter always scheduled short jumps even outside the emit compiler pass. Running the full test suite through the native emitter instead of just the tests that depend on the emitter at runtime (as in, `micropython/native_*` and `micropython/viper_* tests`) uncovered more places where the invalid behaviour was still present. Signed-off-by: Alessandro Gatti <a.gatti@frob.it>	2024-08-19 15:53:50 +10:00
Glenn Moloney	6367099f83	py/objstr: Skip whitespace in bytes.fromhex(). ... Skip whitespace characters between pairs of hex numbers. This makes `bytes.fromhex()` compatible with cpython. Includes simple test in `tests/basic/builtin_str_hex.py`. Signed-off-by: Glenn Moloney <glenn.moloney@gmail.com>	2024-08-19 14:18:34 +10:00
Angus Gratton	86f2c285eb	py: Add new cstack API for stack checking, with limit margin macro. ... Currently the stack limit margin is hard-coded in each port's call to `mp_stack_set_limit()`, but on threaded ports it's fiddlier and can lead to bugs (such as incorrect thread stack margin on esp32). This commit provides a new API to initialise the C Stack in one function call, with a config macro to set the margin. Where possible the new call is inlined to reduce code size in thread-free ports. Intended replacement for `MP_TASK_STACK_LIMIT_MARGIN` on esp32. The previous `stackctrl.h` API is still present and unmodified apart from a deprecation comment. However it's not available when the `MICROPY_PREVIEW_VERSION_2` macro is set. This work was funded through GitHub Sponsors. Signed-off-by: Angus Gratton <angus@redyak.com.au>	2024-08-14 12:55:45 +10:00
Angus Gratton	6c870dc5ec	py/obj: Remove the legacy object API for version 2. ... These were changed in v1.11 (2019). Prepare to remove the compatibility macros as part of V2 changes. This work was funded through GitHub Sponsors. Signed-off-by: Angus Gratton <angus@redyak.com.au>	2024-08-14 10:33:52 +10:00
Angus Gratton	b0c89377d0	py/modmath: Add option to work around -inf bug in a port's tgamma. ... This is needed for a workaround on esp32 port (in child commit), which produces incorrect results otherwise. This work was funded through GitHub Sponsors. Signed-off-by: Angus Gratton <angus@redyak.com.au>	2024-08-07 14:10:46 +10:00
Damien George	afba3e0540	py/emitnative: Fix case of clobbered REG_TEMP0 when loading const obj. ... The `emit_load_reg_with_object()` helper function will clobber `REG_TEMP0`. This is currently OK on architectures where `REG_RET` and `REG_TEMP0` are the same (all architectures except RV32), because all callers of `emit_load_reg_with_object()` use either `REG_RET` or `REG_TEMP0` as the destination register. But on RV32 these registers are different and so when `REG_RET` is the destination, `REG_TEMP0` is clobbered, leading to incorrectly generated machine code. This commit fixes the issue simply by using `REG_TEMP0` as the destination register for all uses of `emit_load_reg_with_object()`, and adds a comment to make sure the caller of this function is careful. Signed-off-by: Damien George <damien@micropython.org>	2024-08-07 12:25:21 +10:00
George Hopkins	066243ea74	py/py.mk: Add SRC_USERMOD_LIB_ASM to include assembly files. ... Introduce SRC_USERMOD_LIB_ASM to allow users to include assembly files as part of their user modules. It could be used to include optimized functions or outputs of other programming languages. Signed-off-by: George Hopkins <george-hopkins@null.net>	2024-08-01 12:01:18 +10:00
Damien George	233f5ce661	py/runtime: Fix self arg passed to classmethod when accessed via super. ... Thanks to @AJMansfield for the original test case. Signed-off-by: Damien George <damien@micropython.org>	2024-07-25 13:07:42 +10:00
stijn	07bf3179f6	py/misc: Fix msvc and C++ compatibility. ... Use explicit casts to suppress warnings about implicit conversions, add a workaround for constant expression conditional, and make functions static inline (as is done in the rest of the codebase) to suppress 'warning C4505: unreferenced function with internal linkage has been removed'. (Follow up to fix commit 908ab1ceca) Signed-off-by: stijn <stijn@ignitron.net>	2024-07-25 12:55:23 +10:00
stijn	093d0c0a17	py/objtype: Validate super() arguments. ... This fixes various null dereferencing and out-of-bounds access because super_attr assumes the held obj is effectively an object of the held type, which is now verified. Fixes issue #12830. Signed-off-by: stijn <stijn@ignitron.net>	2024-07-25 12:27:33 +10:00
Laurens Valk	9ca668f881	py/objtype: Avoid crash on calling members of uninitialized native type. ... When subclassing a native type, calling native members in `__init__` before `super().__init__()` has been called could cause a crash. In this situation, `self` in `mp_convert_member_lookup` is the `native_base_init_wrapper_obj`. The check added in this commit ensures that an `AttributeError` is raised before this happens, which is consistent with other failed lookups. Also fix a typo in a related comment. Signed-off-by: Laurens Valk <laurens@pybricks.com>	2024-07-25 12:01:43 +10:00
Damien George	5f3ecc29f8	extmod/modmachine: Use sys.exit as implementation of machine.soft_reset. ... It does the same thing, raising `SystemExit`. Signed-off-by: Damien George <damien@micropython.org>	2024-07-20 12:18:07 +10:00
Damien George	5147dc5de5	py/gc: Remove commented-out functions. ... These are old, unused, and most of them no longer compile. The `gc_test()` function is superseded by the test suite. Signed-off-by: Damien George <damien@micropython.org>	2024-07-19 10:36:30 +10:00
Damien George	1548132979	py/sequence: Remove unused len argument from mp_seq_extract_slice. ... Also put this function inside the `MICROPY_PY_BUILTINS_SLICE` guard, because it's only usable when that option is enabled. Signed-off-by: Damien George <damien@micropython.org>	2024-07-18 12:51:29 +10:00