124 Commits

Author	SHA1	Message	Date
Andrew Leech	dc0b2c7f57	py/malloc: Add m_tracked_realloc. ... Add realloc support to the tracked allocation API, using an alloc-copy-free approach for simplicity and thread safety. This function compliments the existing `m_tracked_calloc()` and `m_tracked_free()` functions. Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>	2026-04-08 17:01:10 +10:00
stijn	dc91fa6548	py/misc: Fix truncating cast to char. ... Explicitly specify the wanted cast otherwise this results in warning C4244: '=': conversion from 'unichar' to 'char', possible loss of data when building with msvc with most warnings enabled. Signed-off-by: stijn <stijn@ignitron.net>	2026-02-07 18:16:54 +11:00
Damien George	e07bad9866	py/misc: Remove unused CHECKBUF macros. ... These macros were added in e11b17c25f for use with `sys.path` and building import path names, as well as being used by the unix port in 625d08a93e. But the macro usage for importing was promptly replaced with vstrs in e09ffa1400, and eventually for the unix port in 520f35632d, over 9.5 years ago. So, remove these unused macros. Signed-off-by: Damien George <damien@micropython.org>	2026-02-05 10:42:50 +11:00
Damien George	5f59f399b7	py/vstr: Add vstr_ins_strn helper function. ... This is now an easy function to define as inline, so it does not impact code size unless it's used. Signed-off-by: Damien George <damien@micropython.org>	2026-02-04 23:19:09 +11:00
Damien George	60c37a22d6	py/vstr: Make vstr_ins_byte/char functions inline to reduce code size. ... This saves about 4 bytes on ARM Cortex-M, and about 50-60 bytes on x86-64. It also allows the upcoming `vstr_ins_strn()` function to be inline as well, and have less of a code-size impact when used. Signed-off-by: Damien George <damien@micropython.org>	2026-02-04 23:19:09 +11:00
Alessandro Gatti	3980a0c01f	py/misc: Add byte-swapping macros. ... This commit adds byte-swapping macros for 16-, 32-, and 64-bit values. Modern compilers are usually able to detect manual byte/endian swaps and will generate optimised code sequences if they know how to do that. However, in certain situations it may be helpful to let the compiler generate an optimised sequence if it doesn't recognise the pattern. Most compilers provide built-in byte swap operations for 16, 32, and 64 bit values, accessed via "__builtin_bswap<size>", but if those are not available a fallback implementation is provided. Signed-off-by: Alessandro Gatti <a.gatti@frob.it>	2026-01-04 12:13:34 +11:00
Jeff Epler	8b8cd82560	py/misc: Remove unused mp_check function. ... This is unused since 007f127a61 "all: Simplify mp_int_t/mp_uint_t definition". Signed-off-by: Jeff Epler <jepler@unpythonic.net>	2025-12-18 11:35:38 +11:00
Jeff Epler	007f127a61	all: Simplify mp_int_t/mp_uint_t definition. ... Assuming proper C99 language support, we can select "the int type as big as a pointer" (most of the time) or "the 64-bit int type" (nanboxing with REPR_D), and then define everything else automatically. This simplifies port configuration files. And the types can still be overridden if needed. Signed-off-by: Jeff Epler <jepler@unpythonic.net>	2025-10-23 15:12:28 +11:00
Jeff Epler	33cf1ab3cc	py/misc: Don't warn about a GNU extension for static assert macro. ... This warning was enabled by default on clang 17.0.0 on macOS 26. Disable it, because we want to make these checks at compile-time even if it requires an extension. Fixes issue #18116. Signed-off-by: Jeff Epler <jepler@unpythonic.net>	2025-10-04 15:51:23 +10:00
Jeff Epler	099991f3ef	py/misc: Use _Static_assert for MP_STATIC_ASSERT where possible. ... Or use C++ `static_assert` when that's available. For the same reasons that C++ has trouble with "nonconstexpr" static assertions, `_Static_assert` rejects such expression as well. So, fall back to the old sizeof-array based implementation in that case. When `_Static_assert` can be used, the diagnostic quality is improved: ../py/objint.c: In function ‘mp_obj_int_make_new’: ../py/misc.h:67:32: error: static assertion failed: "37 == 42" ../py/objint.c:45:5: note: in expansion of macro ‘MP_STATIC_ASSERT’ As compared to a diagnostic about ../py/misc.h:71:50: error: size of unnamed array is negative Testing on godbolt indicated that this actually works back to gcc 4.5, but it's easier to use GNUC >= 5 as the test; hypothetical users of 4.5, 4.6, or 4.7 will just get slightly worse diagnostics. See related issue #18116. Signed-off-by: Jeff Epler <jepler@unpythonic.net>	2025-10-04 15:47:51 +10:00
Jeff Epler	7d50952c2b	all: Use "static inline" consistently in function definitions. ... Signed-off-by: Jeff Epler <jepler@unpythonic.net>	2025-10-02 16:30:52 +10:00
Jeff Epler	a809132921	py: Add MICROPY_USE_GCC_MUL_OVERFLOW_INTRINSIC. ... Most MCUs apart from Cortex-M0 with Thumb 1 have an instruction for computing the "high part" of a multiplication (e.g., the upper 32 bits of a 32x32 multiply). When they do, gcc uses this to implement a small and fast overflow check using the __builtin_mul_overflow intrinsic, which is preferable to the guard division method previously used in smallint.c. However, in contrast to the previous mp_small_int_mul_overflow routine, which checks that the result fits not only within mp_int_t but is SMALL_INT_FITS(), __builtin_mul_overflow only checks for overflow of the C type. As a result, a slight change in the code flow is needed for MP_BINARY_OP_MULTIPLY. Other sites using mp_small_int_mul_overflow already had the result value flow through to a SMALL_INT_FITS check so they didn't need any additional changes. Do similarly for the _ll and _ull multiply overflows checks. Signed-off-by: Jeff Epler <jepler@gmail.com>	2025-10-01 09:23:05 +10:00
Angus Gratton	3faf229853	py/misc: Add a way to detect sanitizer builds. ... Clang and gcc>=14 can use __has_feature() to detect if a sanitizer is enabled, but older GCC has no mechanism - need to set a macro explicitly for this to be recognised. Necessary for increasing some resource limits in sanitizer builds. Important not to use to avoid real issues! This work was funded through GitHub Sponsors. Signed-off-by: Angus Gratton <angus@redyak.com.au>	2025-08-19 10:33:40 +10:00
Angus Gratton	744270ac1b	py/misc: Add explicit dependency on py/mpconfig.h. ... Macros in misc.h depend on values defined by including mpconfig.h. This work was funded through GitHub Sponsors. Signed-off-by: Angus Gratton <angus@redyak.com.au>	2025-08-13 10:13:08 +10:00
Yoctopuce dev	dbbaa959c8	py/formatfloat: Improve accuracy of float formatting code. ... Following discussions in PR #16666, this commit updates the float formatting code to improve the `repr` reversibility, i.e. the percentage of valid floating point numbers that do parse back to the same number when formatted by `repr` (in CPython it's 100%). This new code offers a choice of 3 float conversion methods, depending on the desired tradeoff between code size and conversion precision: - BASIC method is the smallest code footprint - APPROX method uses an iterative method to approximate the exact representation, which is a bit slower but but does not have a big impact on code size. It provides `repr` reversibility on >99.8% of the cases in double precision, and on >98.5% in single precision (except with REPR_C, where reversibility is 100% as the last two bits are not taken into account). - EXACT method uses higher-precision floats during conversion, which provides perfect results but has a higher impact on code size. It is faster than APPROX method, and faster than the CPython equivalent implementation. It is however not available on all compilers when using FLOAT_IMPL_DOUBLE. Here is the table comparing the impact of the three conversion methods on code footprint on PYBV10 (using single-precision floats) and reversibility rate for both single-precision and double-precision floats. The table includes current situation as a baseline for the comparison: PYBV10 REPR_C FLOAT DOUBLE current = 364688 12.9% 27.6% 37.9% basic = 364812 85.6% 60.5% 85.7% approx = 365080 100.0% 98.5% 99.8% exact = 366408 100.0% 100.0% 100.0% Signed-off-by: Yoctopuce dev <dev@yoctopuce.com>	2025-08-01 00:47:33 +10:00
Angus Gratton	516aa02104	py/objint_longlong: Add arithmetic overflow checks. ... Long long big integer support now raises an exception on overflow rather than returning an undefined result. Also adds an error when shifting by a negative value. The new arithmetic checks are added in the misc.h header. This work was funded through GitHub Sponsors. Signed-off-by: Angus Gratton <angus@redyak.com.au>	2025-07-18 00:12:05 +10:00
Alessandro Gatti	703d5acbad	py/misc: Introduce macros to check values' bits size. ... This commit adds two macros that lets check whether a given value can fit an arbitrary number of bits, either as a signed or as an unsigned number. The native emitter code backends perform a lot of bit size checks to see if a particular code sequence can be emitted instead of a generic one, and each platform backend has its own ad-hoc macros (usually one per bit count and signedness). With these macros there's a single way to perform those checks, plus there's no more chance for off-by-one mask length errors when dealing with signed numbers. Signed-off-by: Alessandro Gatti <a.gatti@frob.it>	2025-07-01 15:34:29 +10:00
Damien George	6fee099cae	py/misc: Fix fallback implementation of mp_popcount. ... Tested using gcc 7.3.1 which does not have the popcount built-in and uses this fallback version. Without the fix, mpy-cross produces mpy files with corrupt RISC-V machine code. With the fix, mpy-cross output is correct. Signed-off-by: Damien George <damien@micropython.org>	2025-06-25 11:36:28 +10:00
Alessandro Gatti	f47e214cdc	all: Rename the "NORETURN" macro to "MP_NORETURN". ... This commit renames the NORETURN macro, indicating to the compiler that a function does not return, into MP_NORETURN to maintain the same naming convention of other similar macros. To maintain compaitiblity with existing code NORETURN is aliased to MP_NORETURN, but it is also deprecated for MicroPython v2. This changeset was created using a similar process to decf8e6a8b ("all: Remove the "STATIC" macro and just use "static" instead."), with no documentation or python scripts to change to reflect the new macro name. Signed-off-by: Alessandro Gatti <a.gatti@frob.it>	2025-05-13 10:36:47 +10:00
Alessandro Gatti	3044233ea3	py/misc: Add a popcount(uint32_t) implementation. ... This makes the existing popcount(uint32_t) implementation found in the RV32 emitter available to the rest of the codebase. This version of popcount will use intrinsic or builtin implementations if they are available, falling back to a generic implementation if that is not the case. Signed-off-by: Alessandro Gatti <a.gatti@frob.it>	2025-01-01 10:44:53 +01:00
stijn	0f7d68043f	py/misc: Fix msvc and C++ compatibility. ... Use an explicit cast to suppress the implicit conversion which started popping up in recent compiler versions (and wasn't there yet in 07bf3179). Signed-off-by: stijn <stijn@ignitron.net>	2024-12-10 11:30:59 +11:00
Alessandro Gatti	d34b15ac6f	pic16bit: Make it build with recent XC16 versions. ... The PIC16 port didn't catch up with the other ports, so it required a bit of work to make it build with the latest version of XC16. Signed-off-by: Alessandro Gatti <a.gatti@frob.it>	2024-10-30 17:21:30 +11:00
stijn	07bf3179f6	py/misc: Fix msvc and C++ compatibility. ... Use explicit casts to suppress warnings about implicit conversions, add a workaround for constant expression conditional, and make functions static inline (as is done in the rest of the codebase) to suppress 'warning C4505: unreferenced function with internal linkage has been removed'. (Follow up to fix commit 908ab1ceca) Signed-off-by: stijn <stijn@ignitron.net>	2024-07-25 12:55:23 +10:00
Angus Gratton	908ab1ceca	py/objint: Fix int.to_bytes() buffer size checks. ... Fixes and improvements to `int.to_bytes()` are: - No longer overflows if byte size is 0 (closes #13041). - Raises OverflowError in any case where number won't fit into byte length (now matches CPython, previously MicroPython would return a truncated bytes object). - Document that `micropython int.to_bytes()` doesn't implement the optional signed kwarg, but will behave as if `signed=True` when the integer is negative (this is the current behaviour). Add tests for this also. Requires changes for small ints, MPZ large ints, and "long long" large ints. Adds a new set of unit tests for ints between 32 and 64 bits to increase coverage of "long long" large ints, which are otherwise untested. Tested on unix port (64 bit small ints, MPZ long ints) and Zephyr STM32WB board (32 bit small ints, long long large ints). This work was funded through GitHub Sponsors. Signed-off-by: Angus Gratton <angus@redyak.com.au>	2024-06-24 14:07:00 +10:00
Angus Gratton	d933210d96	py/misc: Move mp_clz and mp_ctz intrinsics into misc.h. ... Signed-off-by: Angus Gratton <angus@redyak.com.au>	2024-06-24 14:06:54 +10:00
Damien George	9716171966	py/misc: Remove m_new_obj[_var]_with_finaliser macros. ... They are no longer used. The new `mp_obj_malloc_with_finaliser()` macros should be used instead, which force the setting of the `base.type` field. And there's always `m_malloc_with_finaliser()` if needed. Signed-off-by: Damien George <damien@micropython.org>	2024-02-20 10:33:09 +11:00
stijn	a968888f69	py/obj: Fix mp_obj_is_type compilation with C++. ... Fixes issue #12951. Signed-off-by: stijn <stijn@ignitron.net>	2023-11-17 14:31:42 +11:00
Jim Mussared	b6a9778484	py/misc: Change sizeof to offsetof for variable-length alloc. ... This fixes the case where e.g. struct foo_t { mp_obj_t x; uint16_t y; char buf[]; }; will have `sizeof(struct foo_t)==8`, but `offsetof(struct foo_t, buf)==6`. When computing the size to allocate for `m_new_obj_var` we need to use offsetof to avoid over-allocating. This is important especially when it might cause it to spill over into another GC block. This work was funded through GitHub Sponsors. Signed-off-by: Jim Mussared <jim.mussared@gmail.com>	2023-11-03 16:03:18 +11:00
David Lechner	ab317a0d66	py/misc: Remove use of bitfield from vstr_t. ... Since there is only one flag, we don't need to use a bitfield in vstr_t. Compilers emit extra instructions to access a bitfield, so this should reduce the binary size a small amount. Signed-off-by: David Lechner <david@pybricks.com>	2022-10-12 00:33:09 +11:00
Jim Mussared	165388e4eb	py/objtype: Optimise slot RAM usage for instance types. ... In all cases other than where you have a native base with a protocol, it now fits into 4 GC blocks (like it did before the slots representation). Signed-off-by: Jim Mussared <jim.mussared@gmail.com>	2022-09-19 19:06:13 +10:00
Dan Ellis	6f4d424f46	py/formatfloat: Use pow(10, e) instead of pos/neg_pow lookup tables. ... Rework the conversion of floats to decimal strings so it aligns precisely with the conversion of strings to floats in parsenum.c. This is to avoid rendering 1eX as 9.99999eX-1 etc. This is achieved by removing the power- of-10 tables and using pow() to compute the exponent directly, and that's done efficiently by first estimating the power-of-10 exponent from the power-of-2 exponent in the floating-point representation. Code size is reduced by roughly 100 to 200 bytes by this commit. Signed-off-by: Dan Ellis <dan.ellis@gmail.com>	2022-08-12 23:53:34 +10:00
stijn	1f16d682da	py/misc: Fix msvc compilation with compressed error messages.	2022-07-18 23:25:12 +10:00
Yonatan Goldschmidt	6670281472	py/misc: Add MP_STATIC_ASSERT_NOT_MSC(). ... To be used in cases where the condition of the assert does not compile under msvc. Signed-off-by: Yonatan Goldschmidt <yon.goldschmidt@gmail.com>	2022-07-18 11:11:00 +10:00
Damien George	fca5701f74	py/malloc: Introduce m_tracked_calloc, m_tracked_free functions. ... Enabled by MICROPY_TRACKED_ALLOC. Signed-off-by: Damien George <damien@micropython.org>	2022-05-05 10:31:50 +10:00
Jim Mussared	e64cda5295	stm32: Add implementation of machine.bitstream. ... Hand-written version for M0, and cycle-counter version for everything else. Signed-off-by: Jim Mussared <jim.mussared@gmail.com>	2021-08-19 22:50:32 +10:00
Damien George	d4b706c4d0	py: Add option to compile without any error messages at all. ... This introduces a new option, MICROPY_ERROR_REPORTING_NONE, which completely disables all error messages. To be used in cases where MicroPython needs to fit in very limited systems. Signed-off-by: Damien George <damien@micropython.org>	2021-04-27 23:51:52 +10:00
Jim Mussared	0e8af2b370	extmod/modbluetooth: Add API for L2CAP channels. ... Also known as L2CAP "connection oriented channels". This provides a socket-like data transfer mechanism for BLE. Currently only implemented for NimBLE on STM32 / Unix. Signed-off-by: Jim Mussared <jim.mussared@gmail.com>	2020-11-24 01:07:17 +11:00
stijn	e498a8bd13	py: Workaround clang error when building misc.h with C++ compiler.	2020-10-29 15:28:14 +11:00
Damien George	6164c7e666	py/misc.h: Add missing semi-colon in mp_float_union_t for big-endian. ... Fixes issue #6161. Signed-off-by: Damien George <damien@micropython.org>	2020-06-22 13:42:24 +10:00
Damien George	2725a79192	py: Always give noop defines when MICROPY_ROM_TEXT_COMPRESSION disabled. ... This commit provides a typedef for mp_rom_error_text_t, and a macro define for MP_COMPRESSED_ROM_TEXT, when MICROPY_ROM_TEXT_COMPRESSION is disabled. This simplifies the configuration (it no longer has a special case for MICROPY_ENABLE_DYNRUNTIME) and makes it work for other cases that don't use compression (eg examples/embedding). This commit also ensures MICROPY_ROM_TEXT_COMPRESSION is defined during qstr processing.	2020-04-14 22:13:11 +10:00
Jim Mussared	154b4eb354	py: Implement "common word" compression scheme for error messages. ... The idea here is that there's a moderate amount of ROM used up by exception text. Obviously we try to keep the messages short, and the code can enable terse errors, but it still adds up. Listed below is the total string data size for various ports: bare-arm 2860 minimal 2876 stm32 8926 (PYBV11) cc3200 3751 esp32 5721 This commit implements compression of these strings. It takes advantage of the fact that these strings are all 7-bit ascii and extracts the top 128 frequently used words from the messages and stores them packed (dropping their null-terminator), then uses (0x80 | index) inside strings to refer to these common words. Spaces are automatically added around words, saving more bytes. This happens transparently in the build process, mirroring the steps that are used to generate the QSTR data. The MP_COMPRESSED_ROM_TEXT macro wraps any literal string that should compressed, and it's automatically decompressed in mp_decompress_rom_string. There are many schemes that could be used for the compression, and some are included in py/makecompresseddata.py for reference (space, Huffman, ngram, common word). Results showed that the common-word compression gets better results. This is before counting the increased cost of the Huffman decoder. This might be slightly counter-intuitive, but this data is extremely repetitive at a word-level, and the byte-level entropy coder can't quite exploit that as efficiently. Ideally one would combine both approaches, but for now the common-word approach is the one that is used. For additional comparison, the size of the raw data compressed with gzip and zlib is calculated, as a sort of proxy for a lower entropy bound. With this scheme we come within 15% on stm32, and 30% on bare-arm (i.e. we use x% more bytes than the data compressed with gzip -- not counting the code overhead of a decoder, and how this would be hypothetically implemented). The feature is disabled by default and can be enabled by setting MICROPY_ROM_TEXT_COMPRESSION at the Makefile-level.	2020-04-05 14:20:57 +10:00
Damien George	69661f3343	all: Reformat C and Python source code with tools/codeformat.py. ... This is run with uncrustify 0.70.1, and black 19.10b0.	2020-02-28 10:33:03 +11:00
Damien George	ce39c958ef	py: Factor out definition of mp_float_union_t to one location.	2020-02-18 13:04:36 +11:00
Yonatan Goldschmidt	df5c3bd976	py/unicode: Add unichar_isalnum().	2020-01-12 13:03:57 +11:00
Damien George	c0a1de3c21	py/misc.h: Rename _MP_STRINGIFY to not use leading underscore in ident. ... Macro identifiers with a leading underscore are reserved.	2019-05-09 17:11:33 +10:00
Damien George	43d08d6dd6	py/misc.h: Add MP_STATIC_ASSERT macro to do static assertions.	2018-05-18 23:31:00 +10:00
Damien George	d4b55eff44	py/misc.h: Remove unused count_lead_ones() inline function. ... This function was never used for unicode/utf8 handling code, or anything else, so remove it to keep things clean.	2018-03-13 13:23:30 +11:00
Damien George	19aee9438a	py/unicode: Clean up utf8 funcs and provide non-utf8 inline versions. ... This patch provides inline versions of the utf8 helper functions for the case when unicode is disabled (MICROPY_PY_BUILTINS_STR_UNICODE set to 0). This saves code size. The unichar_charlen function is also renamed to utf8_charlen to match the other utf8 helper functions, and the signature of this function is adjusted for consistency (const char* -> const byte*, mp_uint_t -> size_t).	2018-02-14 18:19:22 +11:00
Paul Sokolovsky	75d3c046da	py/misc.h: Add m_new_obj_var_with_finaliser(). ... Similar to existing m_new_obj_with_finaliser().	2017-12-04 11:05:49 +02:00
Damien George	ca21aed0a1	py: Make m_malloc_fail() have void return type, since it doesn't return.	2017-08-31 17:00:14 +10:00