481 Commits

Author	SHA1	Message	Date
Andrew Leech	820485358d	unix/coverage: Add coverage test for mp_sched_schedule_node. ... Test modified to reschedule itself based on a flag setting. Without the change in the parent commit, this test executes the callback indefinitely and hangs but with the change it runs only once each time mp_handle_pending() is called. Modified-by: Angus Gratton <angus@redyak.com.au> Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>	2025-06-04 11:31:12 +10:00
Andrew Leech	d5f2fc239a	extmod/modbluetooth: Add timeout to deinit. ... If the BLE radio stops responding before deinit is called the function can get stuck waiting for an event that is never received, particularly if the radio is external or on a separate core. This commit adds a timeout, similar to the timeout already used in the init function. Updated for nimble, btstack, esp32 and zephyr bindings. Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>	2025-06-04 02:21:32 +10:00
Angus Gratton	0b224048ef	unix/modsocket: Expose MSG_PEEK flag for recv & recvfrom. ... This work was funded through GitHub Sponsors. Signed-off-by: Angus Gratton <angus@redyak.com.au>	2025-06-03 12:52:06 +10:00
stijn	9a377801dc	unix/coveragecpp: Verify struct-initializing macros' C++-compatibility. ... Add code using all relevant macros to make sure they initialize structs correctly. Signed-off-by: stijn <stijn@ignitron.net>	2025-04-24 15:55:06 +02:00
Alessandro Gatti	38a3873310	unix/mpthreadport: Work around lack of thread cancellation on Android. ... This commit fixes thread-related compilation issues under Android using Termux as its runtime environment. On Android's libc (Bionic) thread cancellation is not implemented, but the Unix port uses that mechanism to provide asynchronous thread termination. In this commit there is a workaround for that, by adding a new signal handler to each newly created thread, whose callback simply exits the thread. Threads are then sent the new signal rather than being explicitly cancelled, which in turn trigger the signal handler to stop the thread execution at the next possible occasion. This makes the cancellation behaviour differ slightly on Android, as threads are probably going to linger a little bit more since the method introduced in this commit is equivalent to setting PTHREAD_CANCEL_DEFERRED as the thread cancellation type. On the other hand there are no guarantees of immediate cancellation using PTHREAD_CANCEL_ASYNCHRONOUS either. This fixes the pthread-related issues reported in #16259. Signed-off-by: Alessandro Gatti <a.gatti@frob.it>	2025-04-22 10:19:20 +10:00
David Yang	6025b78d01	unix/main: Remove PATH_MAX from realpath. ... POSIX.1-2008 ensures realpath() give a dynamically allocated buffer if NULL is passed (which is also true for ports/windows/realpath.c), avoiding an explicit call to malloc() and use of PATH_MAX, which may be undefined on some systems. Signed-off-by: David Yang <mmyangfl@gmail.com>	2025-04-22 00:20:47 +08:00
Yoctopuce dev	52ca826880	unix/variants: Enable os.uname() in coverage build for tests. ... In order to provide test coverage for the previous commit, `os.uname()` support is added to the unix coverage build. Signed-off-by: Yoctopuce dev <dev@yoctopuce.com>	2025-04-21 17:38:42 +10:00
Damien George	be0fce9429	unix/main: Add coverage test for mounting ROMFS filesystem at startup. ... Signed-off-by: Damien George <damien@micropython.org>	2025-03-06 12:52:35 +11:00
Damien George	89e6c58c80	extmod/modvfs: Add vfs.rom_ioctl function and its ioctl constants. ... This is a generic interface to allow querying and modifying the read-only memory area of a device, if it has such an area. Signed-off-by: Damien George <damien@micropython.org>	2025-03-06 12:52:35 +11:00
Angus Gratton	fd0e529a47	unix: Add recursive mutex support. ... Allows refactoring the existing thread_mutex atomic section support to use the new recursive mutex type. This work was funded through GitHub Sponsors. Signed-off-by: Angus Gratton <angus@redyak.com.au>	2025-02-03 15:02:02 +11:00
Angus Gratton	d642cce27a	unix: Use the bare metal mbedTLS config in the coverage buiid. ... This allows coverage to test MicroPython-specific features such as the tracked alloc cleanups added in the parent commit. This work was funded through GitHub Sponsors. Signed-off-by: Angus Gratton <angus@redyak.com.au>	2025-02-03 15:02:02 +11:00
Damien George	d9378c9287	unix: Enable VfsRom on standard and coverage variants. ... Signed-off-by: Damien George <damien@micropython.org>	2024-12-23 13:04:55 +11:00
Damien George	8e4c809694	unix,windows: Force _FILE_OFFSET_BITS=64 to fix 32-bit file ABI. ... On 64-bit systems this should have no effect. On 32-bit systems it will force 64-bit file sizes and fixes directory listing on certain 32-bit ports. This option should work on pretty much all 32-bit systems these days. Signed-off-by: Damien George <damien@micropython.org>	2024-12-16 11:56:03 +11:00
Damien George	838f21298a	unix/main: Add --version command-line option. ... CPython also has this option. Signed-off-by: Damien George <damien@micropython.org>	2024-10-15 14:44:10 +11:00
Damien George	70a01550c9	unix/alloc: Remove unused MICROPY_FORCE_PLAT_ALLOC_EXEC option. ... The `MICROPY_FORCE_PLAT_ALLOC_EXEC` config option was made obsolete by commit 9796625457, so remove it. Signed-off-by: Damien George <damien@micropython.org>	2024-10-03 11:09:42 +10:00
Damien George	36fe9bbf00	unix/alloc: Don't trace executable memory for GC pointers. ... This tracing was originally needed for three reasons: 1. To trace object/raw-code pointers embedded in native code, eg from @micropython.native functions. 2. To trace pointers to BSS/rodata from relocated viper code. 3. For libffi tracing. The tracing in point 1 was made unnecessary long ago in commit 7d4b6cc868 (see also commit bbccb0f630 which removed scanning of native code on the esp8266 port). Since the previous commit, point 2 is no longer needed. For point 3, this was made unnecessary in the recent commit 9796625457. Signed-off-by: Damien George <damien@micropython.org>	2024-09-26 22:36:55 +10:00
Alessandro Gatti	8f465dfd10	unix/modffi: Restrict uint32_t values to 32 bits. ... This commit clears the upper 32 bits of returned `uint32_t` values, which are handled as `unsigned int`s by the MicroPython runtime and thus could be extended to 64 bits on some platforms. RV64 holds 32-bit values as signed integers when held in registers, but the code handling the FFI unsigned int case did not take this into account. That introduced test failures when a 32-bit value had its most significant bit set, as when performing the value extension from 32 to 64 bits, the upper half of the value would be filled with ones. On 32 bit platforms this change should be converted to a no-op, and on other 64 bit platforms that aren't RISC-V it shouldn't hurt as the value being manipulated is expected to only hold valid bits in its lower half. Signed-off-by: Alessandro Gatti <a.gatti@frob.it>	2024-09-23 11:47:18 +10:00
Alessandro Gatti	9796625457	unix/modffi: Clean up FFI closures memory management. ... This commit removes custom FFI closures alloc/free functions, in favour of using the tracked allocation facility to allocate memory for FFI callback objects. This stems from linking issues in the Arm port when updating LibFFI to the latest stable version, as the overridden alloc/free functions didn't replace LibFFI's (unlike in other ports). The original code did no effective cleanup for allocated callback objects, so there is no real impact when switching allocation strategy. The tracked allocation feature used to be enabled only if the Bluetooth stack integration was enabled. This commit also enables tracked allocation support if FFI support is enabled. Co-authored-by: Damien George <damien@micropython.org> Signed-off-by: Alessandro Gatti <a.gatti@frob.it>	2024-09-23 11:47:18 +10:00
Alessandro Gatti	2b5feb9121	lib/libffi: Update libffi to 3.4.6. ... Libffi is updated to the latest stable version (v3.4.6) as the version of libffi that was linked by the repository predates the introduction of RISC-V processors (both 32 and 64 bits). This is necessary as the packaged version of libffi used by the CI for RISC-V 64 turned out to simply not work correctly, whilst a source-built version of libffi worked just fine. Signed-off-by: Alessandro Gatti <a.gatti@frob.it>	2024-09-23 11:47:18 +10:00
Andrew Leech	7e14680a83	py/objringio: Add micropython.RingIO() interface for general use. ... This commit adds a new `RingIO` type which exposes the internal ring-buffer code for general use in Python programs. It has the stream interface making it similar to `StringIO` and `BytesIO`, except `RingIO` has a fixed buffer size and is automatically safe when reads and writes are in different threads or an IRQ. This new type is enabled at the "extra features" ROM level. Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>	2024-09-19 18:00:44 +10:00
Michael Sawyer	b05983ff6d	unix/modffi: Fix signed integer cast in return_ffi_value. ... Casting an ffi_arg to a signed int may truncate the value. E.g., when the ffi_arg is 64-bit and the signed int is 32-bit. Also, casting an ffi_arg to a larger signed type will not sign extend the value. E.g., when the ffi_arg is 32-bit and the larger signed type is int64_t. If the value is signed, it should be cast to ffi_sarg, which is the same size as ffi_arg. Signed-off-by: Michael Sawyer <mjfsawyer@gmail.com>	2024-09-19 15:00:06 +10:00
Amirreza Hamzavi	f1bdac3752	unix/README: Fix typo in build dependencies. ... Signed-off-by: Amirreza Hamzavi <amirrezahamzavi2000@gmail.com>	2024-09-10 11:30:22 +10:00
Angus Gratton	e370999e37	unix: Add a description of COPT in the README. ... This work was funded through GitHub Sponsors. Signed-off-by: Angus Gratton <angus@redyak.com.au>	2024-09-06 17:07:48 +10:00
Angus Gratton	d04974d8d0	unix: Expand the build steps in the README. ... - Present the default build dependencies in one place at the top, and make a separate section about building standalone. - Add steps for the "minimal" variant as well. - Document that building standalone requires autoconf and libtool. - Allow MICROPY_STANDALONE to be set as an environment variable. Fixes issue #11313. This work was funded through GitHub Sponsors. Signed-off-by: Angus Gratton <angus@redyak.com.au>	2024-09-06 17:07:35 +10:00
stijn	a8d1c25a1b	unix/coveragecpp: Include all API headers in the C++ code. ... Make the CI builds compile the public API as C++ to catch accidental introductions of incompatible code. Signed-off-by: stijn <stijn@ignitron.net>	2024-08-26 14:11:40 +10:00
Angus Gratton	a6fa85d8f9	unix: Switch stack limit check to new cstack API. ... Necessary to pass CI when testing the V2 preview APIs. Also adds an extra coverage test for the legacy stackctrl API, to maintain coverage and check for any regression. This work was funded through GitHub Sponsors. Signed-off-by: Angus Gratton <angus@redyak.com.au>	2024-08-14 12:57:27 +10:00
Alessandro Gatti	55b2720687	shared/runtime/gchelper: Add RISC-V RV64I native gchelper. ... Add native gchelper support for 64 bits RISC-V RV64I targets. Now that RV64 is under CI, this also enables platform-specific ghelper in the Unix port. Also changes the data type holding the register contents to something more appropriate, so in the remote eventuality somebody wants to use this with RV128 all they have to do is update the `__riscv_xlen` check. Signed-off-by: Alessandro Gatti <a.gatti@frob.it>	2024-08-07 16:23:21 +10:00
Alessandro Gatti	44527ada5f	unix/main: Fix GCC builds for RISC-V 64 bits. ... This contains a workaround to silence a possibly incorrect warning when building the Unix port with GCC targeting RISC-V 64 bits. Fixes issue #12838. Signed-off-by: Alessandro Gatti <a.gatti@frob.it>	2024-07-23 15:51:50 +10:00
Damien George	8159dcc276	extmod/modos: Include os.sep entry if MICROPY_VFS is enabled. ... This simplifies configuration by removing the `MICROPY_PY_OS_SEP` option and instead including `os.sep` if `MICROPY_VFS` is enabled. That matches the configuration of all existing ports that enabled `os.sep` (they also had `MICROPY_VFS` enabled), and brings consistency to other ports. Fixes issue #15116. Signed-off-by: Damien George <damien@micropython.org>	2024-07-20 12:32:31 +10:00
Damien George	69c25ea865	shared/runtime/pyexec: Make a raised SystemExit always do a forced exit. ... The current situation with SystemExit and soft reset is the following: - `sys.exit()` follows CPython and just raises `SystemExit`. - On the unix port, raising `SystemExit` quits the application/MicroPython, whether at the REPL or in code (this follows CPython behaviour). - On bare-metal ports, raising `SystemExit` at the REPL does nothing, raising it in code will stop the code and drop into the REPL. - `machine.soft_reset()` raises `SystemExit` but with a special flag set, and bare-metal targets check this flag when it propagates to the top-level and do a soft reset when they receive it. The original idea here was that a bare-metal target can't "quit" like the unix port can, and so dropping to the REPL was considered the same as "quit". But this bare-metal behaviour is arguably inconsistent with unix, and "quit" should mean terminate everything, including REPL access. This commit changes the behaviour to the following, which is more consistent: - Raising `SystemExit` on a bare-metal port will do a soft reset (unless the exception is caught by the application). - `machine.soft_reset()` is now equivalent to `sys.exit()`. - unix port behaviour remains unchanged. Tested running the test suite on an stm32 board and everything still passes, in particular tests that skip by raising `SystemExit` still correctly skip. Signed-off-by: Damien George <damien@micropython.org>	2024-07-20 12:13:14 +10:00
Damien George	b4b4d161c2	unix/mpbtstackport_usb: Remove thread detached attribute. ... As in the previous commit, the thread is later joined so can't be detached. Signed-off-by: Damien George <damien@micropython.org>	2024-07-17 15:02:57 +10:00
Leo Chung	220088fff6	unix/mpbthciport: Remove thread detached attribute. ... A detached thread is not joinable, and the behavior maybe undefined. Signed-off-by: Leo Chung <gewalalb@gmail.com>	2024-07-17 15:01:04 +10:00
Lennart	55e75c4ad4	unix/modtermios: Add more baudrate options. ... This adds some more baudrate option as they are available in the termios.h header - up to a point that seems reasonable in an embedded context. Signed-off-by: Lennart Schierling <Lennart@binarylabs.dev>	2024-07-15 11:27:23 +10:00
Jon Foster	92484d8822	all: Use new mp_obj_new_str_from_cstr() function. ... Use new function mp_obj_new_str_from_cstr() where appropriate. It simplifies the code, and makes it smaller too. Signed-off-by: Jon Foster <jon@jon-foster.co.uk>	2024-07-04 15:55:03 +10:00
Jim Mussared	557d31ed2c	py/objint: Try to convert big-int back to small-int after binary op. ... Before this change, long/mpz ints propagated into all future calculations, even if their value could fit in a small-int object. With this change, the result of a big-int binary op will now be converted to a small-int object if the value fits in a small-int. For example, a relatively common operation like `x = a * b // c` where a,b,c all small ints would always result in a long/mpz int, even if it didn't need to, and then this would impact all future calculations with x. This adds +24 bytes on PYBV11 but avoids heap allocations and potential surprises (e.g. `big-big` is now a small `0`, and can safely be accessed with MP_OBJ_SMALL_INT_VALUE). Performance tests are unchanged on PYBV10, except for `bm_pidigits.py` which makes heavy use of big-ints and gains about 8% in speed. Unix coverage tests have been updated to cover mpz code that is now unreachable by normal Python code (removing the unreachable code would lead to some surprising gaps in the internal C functions and the functionality may be needed in the future, so it is kept because it has minimal overhead). This work was funded through GitHub Sponsors. Signed-off-by: Jim Mussared <jim.mussared@gmail.com>	2024-07-01 13:52:59 +10:00
Damien George	b80607ecaf	unix/variants: Don't use native _Float16 type. ... Using it increases code size by about 2k. Signed-off-by: Damien George <damien@micropython.org>	2024-03-20 14:13:49 +11:00
Angus Gratton	decf8e6a8b	all: Remove the "STATIC" macro and just use "static" instead. ... The STATIC macro was introduced a very long time ago in commit d5df6cd44a. The original reason for this was to have the option to define it to nothing so that all static functions become global functions and therefore visible to certain debug tools, so one could do function size comparison and other things. This STATIC feature is rarely (if ever) used. And with the use of LTO and heavy inline optimisation, analysing the size of individual functions when they are not static is not a good representation of the size of code when fully optimised. So the macro does not have much use and it's simpler to just remove it. Then you know exactly what it's doing. For example, newcomers don't have to learn what the STATIC macro is and why it exists. Reading the code is also less "loud" with a lowercase static. One other minor point in favour of removing it, is that it stops bugs with `STATIC inline`, which should always be `static inline`. Methodology for this commit was: 1) git ls-files | egrep '\.[ch]$' | \ xargs sed -Ei "s/(^| )STATIC($| )/\1static\2/" 2) Do some manual cleanup in the diff by searching for the word STATIC in comments and changing those back. 3) "git-grep STATIC docs/", manually fixed those cases. 4) "rg -t python STATIC", manually fixed codegen lines that used STATIC. This work was funded through GitHub Sponsors. Signed-off-by: Angus Gratton <angus@redyak.com.au>	2024-03-07 14:20:42 +11:00
Damien George	2423493774	py/obj: Change sizeof to offsetof in mp_obj_malloc_var macro. ... Following b6a9778484, to properly calculate the size of the variable-length allocation. Signed-off-by: Damien George <damien@micropython.org>	2024-02-19 23:40:54 +11:00
Damien George	5a3dd8c791	tests/ports/unix: Add coverage test for frozen functions and generators. ... Signed-off-by: Damien George <damien@micropython.org>	2024-02-16 14:17:01 +11:00
YAMAMOTO Takashi	809d113dbc	unix: Don't include system headers when features are disabled. ... Because the target system may not have these headers at all. Signed-off-by: YAMAMOTO Takashi <yamamoto@midokura.com>	2024-02-15 16:54:17 +11:00
YAMAMOTO Takashi	f9704ce36e	unix/input: Flush the prompt after writing it to stdout. ... Depending on your setup, stdout might be buffered/line-buffered. Signed-off-by: YAMAMOTO Takashi <yamamoto@midokura.com>	2024-02-15 16:45:00 +11:00
Damien George	8cbae12d0d	unix/variants: Prefer unix-ffi packages when loading the manifest. ... Signed-off-by: Damien George <damien@micropython.org>	2024-02-08 17:16:10 +11:00
Felix Dörre	b802f0f8ab	extmod/modtls: Move the native ssl module to tls. ... The current `ssl` module has quite a few differences to the CPython implementation. This change moves the MicroPython variant to a new `tls` module and provides a wrapper module for `ssl` (in micropython-lib). Users who only rely on implemented comparible behavior can continue to use `ssl`, while users that rely on non-compatible behavior should switch to `tls`. Then we can make the facade in `ssl` more strictly adhere to CPython. Signed-off-by: Felix Dörre <felix@dogcraft.de>	2024-02-07 12:58:52 +11:00
Carlosgg	f3d1495fd3	all: Update bindings, ports and tests for mbedtls v3.5.1. ... Changes include: - Some mbedtls source files renamed or deprecated. - Our `mbedtls_config.h` files are renamed to `mbedtls_config_port.h`, so they don't clash with mbedtls's new default configuration file named `mbedtls_config.h`. - MBEDTLS_TLS_DEFAULT_ALLOW_SHA1_IN_KEY_EXCHANGE is deprecated. - MBEDTLS_HAVE_TIME now requires an `mbedtls_ms_time` function to be defined but it's only used for TLSv1.3 (currently not enabled in MicroPython so there is a lazy implementation, i.e. seconds * 1000). - `tests/multi_net/ssl_data.py` is removed (due to deprecation of MBEDTLS_TLS_DEFAULT_ALLOW_SHA1_IN_KEY_EXCHANGE), there are the existing `ssl_cert_rsa.py` and `sslcontext_server_client.py` tests which do very similar, simple SSL data transfer. - Tests now use an EC key by default (they are smaller and faster), and the RSA key has been regenerated due to the old PKCS encoding used by openssl rsa command, see https://stackoverflow.com/questions/40822328/openssl-rsa-key-pem-and-der-conversion-does-not-match (and `tests/README.md` has been updated accordingly). Signed-off-by: Carlos Gil <carlosgilglez@gmail.com>	2024-01-30 11:08:46 +11:00
Jim Mussared	d4190815a3	py/mpconfig: Disable qstr hashing at minimum feature level. ... This will apply to bare-arm and minimal, as well as the minimal unix variant. Change the default to MICROPY_QSTR_BYTES_IN_HASH=1 for the CORE,BASIC levels, 2 for >=EXTRA. Removes explicit setting of MICROPY_QSTR_BYTES_IN_HASH==1 in ports that don't set the feature level (because 1 is implied by the default level, CORE). Applies to cc3200, pic16bt, powerpc. Removes explicit setting for nRF (which sets feature level). Also for samd, which sets CORE for d21 and FULL for d51. This means that d21 is unchanged with MICROPY_QSTR_BYTES_IN_HASH==1, but d51 now moves from 1 to 2 (roughly adds 1kiB). The only remaining port which explicitly set bytes-in-hash is rp2 because it's high-flash (hence CORE level) but lowish-SRAM, so it's worthwhile saving the RAM for runtime qstrs. This work was funded through GitHub Sponsors. Signed-off-by: Jim Mussared <jim.mussared@gmail.com>	2024-01-25 16:38:17 +11:00
Damien George	0640ff3b97	ports: Move MICROPY_INTERNAL_WFE definition to mphalport.h. ... It belongs here because the default value is defined in py/mphal.h. Signed-off-by: Damien George <damien@micropython.org>	2024-01-05 15:49:42 +11:00
Maarten van der Schrieck	3bca93b2d0	ports: Fix sys.stdout.buffer.write() return value. ... MicroPython code may rely on the return value of sys.stdout.buffer.write() to reflect the number of bytes actually written. While in most scenarios a write() operation is successful, there are cases where it fails, leading to data loss. This problem arises because, currently, write() merely returns the number of bytes it was supposed to write, without indication of failure. One scenario where write() might fail, is where USB is used and the receiving end doesn't read quickly enough to empty the receive buffer. In that case, write() on the MicroPython side can timeout, resulting in the loss of data without any indication, a behavior observed notably in communication between a Pi Pico as a client and a Linux host using the ACM driver. A complex issue arises with mp_hal_stdout_tx_strn() when it involves multiple outputs, such as USB, dupterm and hardware UART. The challenge is in handling cases where writing to one output is successful, but another fails, either fully or partially. This patch implements the following solution: mp_hal_stdout_tx_strn() attempts to write len bytes to all of the possible destinations for that data, and returns the minimum successful write length. The implementation of this is complicated by several factors: - multiple outputs may be enabled or disabled at compiled time - multiple outputs may be enabled or disabled at runtime - mp_os_dupterm_tx_strn() is one such output, optionally containing multiple additional outputs - each of these outputs may or may not be able to report success - each of these outputs may or may not be able to report partial writes As a result, there's no single strategy that fits all ports, necessitating unique logic for each instance of mp_hal_stdout_tx_strn(). Note that addressing sys.stdout.write() is more complex due to its data modification process ("cooked" output), and it remains unchanged in this patch. Developers who are concerned about accurate return values from write operations should use sys.stdout.buffer.write(). This patch might disrupt some existing code, but it's also expected to resolve issues, considering that the peculiar return value behavior of sys.stdout.buffer.write() is not well-documented and likely not widely known. Therefore, it's improbable that much existing code relies on the previous behavior. Signed-off-by: Maarten van der Schrieck <maarten@thingsconnected.nl>	2023-12-22 10:32:46 +11:00
Angus Gratton	2c828a8815	unix: Update port to use the new event functions. ... Necessary to get coverage of the new event functions. Deletes the case that called usleep(delay) for mp_hal_delay_ms(), it seems like this wouldn't have ever happened anyhow (MICROPY_EVENT_POOL_HOOK is always defined for the unix port). This work was funded through GitHub Sponsors. Signed-off-by: Angus Gratton <angus@redyak.com.au>	2023-12-08 13:17:15 +11:00
Damien George	b5449b0f09	extmod/mbedtls: Enable certificate time/date validation by default. ... All ports using this common configuration already enable time/date validation, so this commit is a no-op change. Signed-off-by: Damien George <damien@micropython.org>	2023-12-01 15:08:11 +11:00
Carlosgg	20ffbe1a41	unix/mbedtls: Enable mbedtls cert time validation. ... To match other ports. Signed-off-by: Carlos Gil <carlosgilglez@gmail.com>	2023-12-01 15:08:11 +11:00