1422 Commits

Author	SHA1	Message	Date
Andrew Leech	6515cd05f1	extmod/vfs_lfsx: Allow overriding the LFS2 on-disk version format. ... Back in LFS2 version 2.6 they updated the on-disk version from 2.0 to 2.1 which broke back compatibility (aka older versions could no long read new version disk format), see https://github.com/littlefs-project/littlefs/releases/tag/v2.6.0 Then in LFS2 v2.7 an optional `config->disk_version` was added to force the library to use an older disk format instead, see: https://github.com/littlefs-project/littlefs/releases/tag/v2.7.0 This commit simply exposes `config->disk_version` as a compile time option if LFS2_MULTIVERSION is set, otherwise there is no change in behavior. This is Useful for compatibility with older LFS versions. Note: LFS2_MULTIVERSION needs to be defined at the make / CFLAGS level, setting it in mpconfigboard.h doesn't work as it's not included in the `lfs2.c` file in any way. Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>	2025-07-23 15:51:08 +10:00
Angus Gratton	28082d1d25	extmod/mbedtls: Undefine ARRAY_SIZE if defined by platform. ... This is an annoying regression caused by including mpconfig.h in 36922df - the mimxrt platform headers define ARRAY_SIZE and mbedtls also defines in some source files, using a different parameter name which is a warning in gcc. Technically mimxrt SDK is to blame here, but as this isn't a named warning in gcc the only way to work around it in the mimxrt port would be to disable all warnings when building this particular mbedTLS source file. This work was funded through GitHub Sponsors. Signed-off-by: Angus Gratton <angus@redyak.com.au>	2025-07-23 15:47:17 +10:00
Angus Gratton	9b7d85227e	extmod/mbedtls: Implement recommended DTLS features, make optional. ... - DTLS spec recommends HelloVerify and Anti Replay protection be enabled, and these are enabled in the default mbedTLS config. Implement them here. - To help compensate for the possible increase in code size, add a MICROPY_PY_SSL_DTLS build config macro that's enabled for EXTRA and above by default. This allows bare metal mbedTLS ports to use DTLS with HelloVerify support. This work was funded through GitHub Sponsors. Signed-off-by: Angus Gratton <angus@redyak.com.au>	2025-07-23 15:47:16 +10:00
Angus Gratton	41e0ec96cb	extmod/mbedtls: Implement DTLS HelloVerify cookie support. ... This is already enabled in the ESP-IDF mbedTLS config, so provide an implementation of the cookie store functions. This allows DTLS connections between two esp32 boards. The session cookie store is a very simple dictionary associated with the SSLContext. To work, the server needs to reuse the same SSLContext (but cookies are never cleaned up, so a server with a high number of clients should recycle the context periodically.) Server code still needs to handle the MBEDTLS_ERR_SSL_HELLO_VERIFY_REQUIRED error by waiting for the next UDP packet from the client. Signed-off-by: Angus Gratton <angus@redyak.com.au>	2025-07-23 15:47:16 +10:00
Damien George	cf490ed346	extmod/network_lwip: Add sys_untimeout_all_with_arg helper function. ... Really lwIP should provide this, to deregister all callbacks on the given netif. Signed-off-by: Damien George <damien@micropython.org>	2025-07-17 13:38:58 +10:00
Yoctopuce dev	df05caea6c	shared/timeutils: Standardize supported date range on all platforms. ... This is code makes sure that time functions work properly on a reasonable date range, on all platforms, regardless of the epoch. The suggested minimum range is 1970 to 2099. In order to reduce code footprint, code to support far away dates is only enabled specified by the port. New types are defined to identify timestamps. The implementation with the smallest code footprint is when support timerange is limited to 1970-2099 and Epoch is 1970. This makes it possible to use 32 bit unsigned integers for all timestamps. On ARM4F, adding support for dates up to year 3000 adds 460 bytes of code. Supporting dates back to 1600 adds another 44 bytes of code. Signed-off-by: Yoctopuce dev <dev@yoctopuce.com>	2025-07-09 11:54:21 +10:00
Andrew Leech	2f04381aeb	extmod/modlwip: Fix crash when calling recv on listening socket. ... Add check to prevent calling recv on a socket in the listening state. This prevents a crash/hard fault when user code mistakenly tries to recv on the listening socket instead of on the accepted connection. Add corresponding test case to demonstrate the bug. Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>	2025-06-26 12:46:45 +10:00
Damien George	398da22492	esp8266/modmachine: Use common machine_time_pulse_us implementation. ... Testing shows that for frequencies which the esp8266 can handle -- up to about 1kHz -- `machine.time_pulse_us()` now gives more accurate results. Prior to this commit it would measure on average about 1us lower, but now the average is much closer to the true value. For example a pulse that is 1000us long, it would measure between 998 and 1000us. Now it measures between 999us and 1001us. Signed-off-by: Damien George <damien@micropython.org>	2025-06-13 16:27:35 +10:00
Damien George	ef21ade602	extmod/machine_pulse: Optimise time_pulse_us for code size. ... This implementation is based on the esp8266 custom implementation, and further optimised for size and accuracy. Testing on PYBD_SF2 and RPI_PICO2_W shows that it is at least as good as the original implementation in performance. Signed-off-by: Damien George <damien@micropython.org>	2025-06-13 16:27:35 +10:00
Jeff Epler	745bec9ce3	extmod/modre: Use specific error message if regex is too complex. ... If the error reporting mode is at least "normal", report a failure due to a complex regex with a different message. Fixes issue #17150. Signed-off-by: Jeff Epler <jepler@gmail.com>	2025-06-10 15:14:56 +10:00
Andrew Leech	fffaf8a41f	extmod/modnetwork: Consolidate definition of common drivers. ... Most extmod network drivers were being defined on a per-port basis, duplicating code and making enabling a driver on a new port harder. This consolidates extmod driver declarations and removes the existing per-port definitions of them. This commit has been verified to be a no-op in terms of firmware change. Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>	2025-06-10 11:20:38 +10:00
Damien George	b15348415e	extmod/modframebuf: Add support for blit'ing read-only data. ... Currently the `FrameBuffer.blit(buf, x, y)` method requires the `buf` argument to be another `FrameBuffer`, which is quite restrictive because it doesn't allow blit'ing read-only memory/data. This commit extends `blit()` to allow the `buf` argument to be a tuple or list of the form: (buffer, width, height, format[, stride]) where `buffer` can be anything with the buffer protocol and may be read-only, eg `bytes`. Also, the palette argument to `blit()` may be of the same form. The form of this tuple/list was chosen to be the same as the signature of the `FrameBuffer` constructor (that saves quite a bit of code size doing it that way). Signed-off-by: Damien George <damien@micropython.org>	2025-06-04 02:40:45 +10:00
Andrew Leech	d5f2fc239a	extmod/modbluetooth: Add timeout to deinit. ... If the BLE radio stops responding before deinit is called the function can get stuck waiting for an event that is never received, particularly if the radio is external or on a separate core. This commit adds a timeout, similar to the timeout already used in the init function. Updated for nimble, btstack, esp32 and zephyr bindings. Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>	2025-06-04 02:21:32 +10:00
Angus Gratton	e8447768ef	extmod/modlwip: Add optional flags argument for recv and recvfrom. ... Implements MSG_PEEK and MSG_DONTWAIT. This work was funded through GitHub Sponsors. Signed-off-by: Angus Gratton <angus@redyak.com.au>	2025-06-03 12:52:02 +10:00
Angus Gratton	186caf9f03	extmod/network_cyw43: Disconnect STA if making inactive. ... esp32 port will disconnect if active(0) is called on a STA interface, but rp2 port stays associated without this change. This work was funded through GitHub Sponsors. Signed-off-by: Angus Gratton <angus@redyak.com.au>	2025-05-20 23:14:41 +10:00
Alessandro Gatti	9ef16b466b	extmod/modjson: Detect unterminated composite entities. ... This commit makes the JSON parser raise an exception when handling objects or arrays whose declaration is incomplete, as in missing the closing marker (brace or bracket) and if the missing marker would have been the last non-whitespace character in the incoming string. Since CPython's JSON parser would raise an exception in such a case, unlike MicroPython's, this commit aligns MicroPython's behaviour with CPython. This commit fixes issue #17141. Signed-off-by: Alessandro Gatti <a.gatti@frob.it>	2025-05-19 02:09:40 +02:00
Damien George	62d26bfc15	extmod/vfs_lfsx: Fix errno value raised from chdir. ... OSError errno values should be positive. Signed-off-by: Damien George <damien@micropython.org>	2025-05-15 13:09:49 +10:00
Daniël van de Giessen	3b1e22c669	esp32/network_ppp: Restructure to match extmod/network_ppp_lwip. ... The ESP32 PPP implementation predates the generic implementation in extmod. The new extmod implementation has a few advantages such as a better deinitialisation procedure (the ESP32 implemementation would not clean up properly and cause crashes if recreated) and using the UART IRQ functionality instead of running a task to read data from the UART. This change restructures the ESP implementation to be much closer to the new extmod version, while also bringing a few tiny improvements from the ESP32 version to the extmod version. The diff between extmod/network_ppp_lwip.c and ports/esp32/network_ppp.c is now a small set of easy to review ESP32 port-specific changes. Signed-off-by: Daniël van de Giessen <daniel@dvdgiessen.nl>	2025-05-15 11:56:14 +10:00
Alessandro Gatti	f47e214cdc	all: Rename the "NORETURN" macro to "MP_NORETURN". ... This commit renames the NORETURN macro, indicating to the compiler that a function does not return, into MP_NORETURN to maintain the same naming convention of other similar macros. To maintain compaitiblity with existing code NORETURN is aliased to MP_NORETURN, but it is also deprecated for MicroPython v2. This changeset was created using a similar process to decf8e6a8b ("all: Remove the "STATIC" macro and just use "static" instead."), with no documentation or python scripts to change to reflect the new macro name. Signed-off-by: Alessandro Gatti <a.gatti@frob.it>	2025-05-13 10:36:47 +10:00
Damien George	26e978e7bc	extmod/modlwip: Implement a queue of incoming UDP/raw packets. ... The bare-metal lwIP socket interface is currently quite limited when used for UDP streams, because it only allows one outstanding incoming UDP packet. If one UDP packet is waiting to be socket.recv'd and another one comes along, then the second one is simply dropped. This commit implements a queue for incoming UDP and raw packets. The queue depth is fixed at compile time, and is currently 4. This allows better use of UDP connections, eg more efficient. It also makes DTLS work better which sometimes has a queue of UDP packets (eg during the connection phase). Signed-off-by: Damien George <damien@micropython.org>	2025-05-12 14:17:44 +10:00
Angus Gratton	7d5aba0523	extmod/moductypes: Refactor string literal as array initializer. ... Avoids the new Wunterminated-string-literal when compiled with gcc 15.1. Also split out the duplicate string to a top-level array (probably the duplicate string literal was interned, so unlikely to have any impact.) This work was funded through GitHub Sponsors. Signed-off-by: Angus Gratton <angus@redyak.com.au>	2025-05-09 18:31:40 +10:00
Angus Gratton	d00eab4a30	rp2,extmod/cyw43: Move the LWIP responder fix into common CYW43 config. ... This means the fix from dd1465e7 will also apply to stm32 and mimxrt ports that use CYW43. This work was funded through GitHub Sponsors. Signed-off-by: Angus Gratton <angus@redyak.com.au>	2025-05-08 15:32:05 +10:00
Angus Gratton	b2cda6c604	extmod,alif,mimxrt,rp2,stm32: Create common cyw43 driver config header. ... This is only a surface level refactor, some deeper refactoring would be possible with (for example) the SDIO interface in mimxrt and stm32, or the BTHCI interface which is is similar on supported ports. But sticking to cases where the macros are the same across all ports. This work was funded through GitHub Sponsors. Signed-off-by: Angus Gratton <angus@redyak.com.au>	2025-05-08 15:32:05 +10:00
Daniël van de Giessen	2b29b1b8f9	lib/littlefs: Reuse existing CRC32 function to save space. ... Getting this to work required fixing a small issue in `lfs2_util.h`, which has been submitted upstream. Signed-off-by: Daniël van de Giessen <daniel@dvdgiessen.nl>	2025-05-07 17:03:28 +10:00
Yoctopuce dev	bdb7e036d2	extmod/asyncio: Fix early exit of asyncio scheduler. ... This commit fixes three open issues related to the asyncio scheduler exiting prematurely when the main task queue is empty, in cases where CPython would not exit (for example, because the main task is not done because it's on a different queue). In the first case, the scheduler exits because running a task via `run_until_complete` did not schedule any dependent tasks. In the other two cases, the scheduler exits because the tasks are queued in an event queue. Tests have been added which reproduce the original issues. These test cases document the unauthorized use of `Event.set()` from a soft IRQ, and are skipped in unsupported environments (webassembly and native emitter). Fixes issues #16759, #16569 and #16318. Signed-off-by: Yoctopuce dev <dev@yoctopuce.com>	2025-05-07 14:56:47 +10:00
iabdalkader	193460d18f	drivers/esp-hosted: Rename Bluetooth HCI backend driver. ... Rename `bthci` to `bthci_uart` for consistency with the CYW43 driver and to distinguish it from HCI backends that use a different transport. Signed-off-by: iabdalkader <i.abdalkader@gmail.com>	2025-04-22 12:30:18 +10:00
Andrew Leech	569d472bc7	extmod/modbluetooth: Use newer mp_map_slot_is_filled function. ... Required in MICROPY_PREVIEW_VERSION_2. Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>	2025-04-22 11:29:43 +10:00
Andrew Leech	b6dbc47664	extmod/machine_usb_device: Add exception text wrappers. ... Required in MICROPY_PREVIEW_VERSION_2. Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>	2025-04-22 11:29:32 +10:00
Damien George	a9384c71c5	extmod/extmod.mk: Switch from drivers/cyw43/cywbt to lib/cyw43-drivers. ... The cyw43-driver now provides the Bluetooth initialisation code, making `drivers/cyw43/cywbt.c` obsolete. To use the new code a port must enable the `CYW43_ENABLE_BLUETOOTH_OVER_UART` option. Some ports have yet to migrate to the new code, so in the meantime they can explicitly add the old source to their source list and continue to use it without change. Signed-off-by: Damien George <damien@micropython.org>	2025-04-08 23:52:16 +10:00
Damien George	0ee160e7c0	extmod/extmod.mk: Add cyw43_spi.c to list of sources. ... This file is part of the updated cyw43-driver. It will only be used if `CYW43_USE_SPI` is enabled. Signed-off-by: Damien George <damien@micropython.org>	2025-04-08 23:52:12 +10:00
Angus Gratton	cccac2cc01	rp2,esp32,extmod: Implement UPDATE_SUBMODULES in CMake. ... Rather than having Make calling CMake to generate a list of submodules and then run a Make target (which is complex and prone to masking other errors), implement the submodule update logic in CMake itself. Internal CMake-side changes are that GIT_SUBMODULES is now a CMake list, and the trigger variable name is changed from ECHO_SUBMODULES to UPDATE_SUBMODULES. The run is otherwise 100% a normal CMake run now, so most of the other special casing can be removed. Signed-off-by: Angus Gratton <angus@redyak.com.au>	2025-03-27 17:51:12 +11:00
Damien George	e4051a1ca6	extmod/vfs_rom: Implement minimal VfsRom.getcwd() method. ... This is needed if you chdir to a ROMFS and want to query your current directory. Prior to this change, using `os.getcwd()` when in a ROMFS would raise: AttributeError: 'VfsRom' object has no attribute 'getcwd' Signed-off-by: Damien George <damien@micropython.org>	2025-03-27 17:04:12 +11:00
Anson Mansfield	1487a13079	extmod/vfs: Return mount table from no-args vfs.mount call. ... This extends the existing `vfs.mount()` function to accept zero arguments, in which case it returns a list of tuples of mounted filesystem objects and their mount location. Signed-off-by: Anson Mansfield <amansfield@mantaro.com>	2025-03-27 16:37:04 +11:00
Anson Mansfield	458a8f2e15	extmod/vfs: Refactor mp_vfs_mount to enable no-args mount overload. ... Signed-off-by: Anson Mansfield <amansfield@mantaro.com>	2025-03-27 16:37:01 +11:00
Damien George	fa42487e45	extmod/moddeflate: Keep DeflateIO state consistent on window alloc fail. ... Allocation of a large compression window may fail, and in that case keep the `DeflateIO` state consistent so its other methods (such as `close()`) still work. Consistency is kept by only updating the `self->write` member if the window allocation succeeds. Thanks to @jimmo for finding the bug. Signed-off-by: Damien George <damien@micropython.org>	2025-03-27 11:58:59 +11:00
Damien George	dafff1fd0e	extmod/network_cyw43: Add WPA3 security constants. ... These are now supported by cyw43-driver. Signed-off-by: Damien George <damien@micropython.org>	2025-03-12 13:17:02 +11:00
Damien George	d4b8ca2ffc	extmod/vfs: Add mp_vfs_mount_romfs_protected() helper. ... This function will attempt to create a `VfsRom` instance and mount it at location "/rom" in the filesystem. Signed-off-by: Damien George <damien@micropython.org>	2025-03-06 12:52:35 +11:00
Damien George	89e6c58c80	extmod/modvfs: Add vfs.rom_ioctl function and its ioctl constants. ... This is a generic interface to allow querying and modifying the read-only memory area of a device, if it has such an area. Signed-off-by: Damien George <damien@micropython.org>	2025-03-06 12:52:35 +11:00
Damien George	14ba32bb20	extmod/vfs_rom: Add bounds checking for all filesystem accesses. ... Testing with ROMFS shows that it is relatively easy to end up with a corrupt filesystem on the device -- eg due to the ROMFS deploy process stopping half way through -- which could lead to hard crashes. Notably, there can be boot loops trying to mount a corrupt filesystem, crashes when importing modules like `os` that first scan the filesystem for `os.py`, and crashing when deploying a new ROMFS in certain cases because the old one is removed while still mounted. The main problem is that `mp_decode_uint()` has an loop that keeps going as long as it reads 0xff byte values, which can happen in the case of erased and unwritten flash. This commit adds full bounds checking in the new `mp_decode_uint_checked()` function, and that makes all ROMFS filesystem accesses robust. Signed-off-by: Damien George <damien@micropython.org>	2025-02-26 16:11:19 +11:00
Christian Clauss	5e206fdeb5	all: Upgrade codespell to v2.4.1. ... This commit upgrades from codespell==2.2.6 to the current codespell==2.4.1, adding emac to the ignore-words-list. Signed-off-by: Christian Clauss <cclauss@me.com>	2025-02-25 16:11:33 +11:00
Keenan Johnson	321b30ca56	extmod/modtls_mbedtls: Wire in support for DTLS. ... This commit enables support for DTLS, i.e. TLS over datagram transport protocols like UDP. While support for DTLS is absent in CPython, it is worth supporting it in MicroPython because it is the basis of the ubiquitous CoAP protocol, used in many IoT projects. To select DTLS, a new set of "protocols" are added to SSLContext: - ssl.PROTOCOL_DTLS_CLIENT - ssl.PROTOCOL_DTLS_SERVER If one of these is set, the library assumes that the underlying socket is a datagram-like socket (i.e. UDP or similar). Our own timer callbacks are implemented because the out of the box implementation relies on `gettimeofday()`. This new DTLS feature is enabled on all ports that use mbedTLS. This commit is an update to a previous PR #10062. Addresses issue #5270 which requested DTLS support. Signed-off-by: Keenan Johnson <keenan.johnson@gmail.com>	2025-02-14 12:55:25 +11:00
Thomas Watson	aef6705a32	extmod/lwip-include: Increase number of lwIP timers when mDNS enabled. ... Despite the code comments claiming one is sufficient, the mDNS application is capable of using up to twelve timers. Three per IP protocol are started at once in `mdns_start_multicast_timeouts_ipvX`, then another two per protocol can be started in `mdns_handle_question`. Further timers can be started for two additional callbacks. Having certain timers, such as `MDNS_MULTICAST_TIMEOUT`, fail to start due to none being free will break mDNS forever as the app will never realize it's safe to transmit a packet. Therefore, this commit goes somewhat overkill and allocates the maximal amount of timers; it's uncertain if all can run simultaneously, or how many callback timers are needed. Each timer struct is 16 bytes on standard 32 bit builds. Plus, say, 8 bytes of allocater overhead, that's 288 more bytes of RAM used which shouldn't be too horrible. Users who don't need mDNS can manually disable it to recover the RAM if necessary. This fixes mDNS on W5500_EVB_PICO (among other boards). Before, mDNS would work for a bit after connection until the host's cache expired a minute or two later. Then the board would never respond to further queries. With this patch, all works well. Signed-off-by: Thomas Watson <twatson52@icloud.com>	2025-02-14 12:32:00 +11:00
Damien George	30acb16ad3	extmod/vfs_rom: Remove ability to create VfsRom from an address. ... It's not necessary to support this, which allows an arbitrary memory address to be specified and potentially allows invalid memory accesses. Requiring an object with the buffer protocol is safer, and also means that the length of the region is always specified. Signed-off-by: Damien George <damien@micropython.org>	2025-02-11 17:56:13 +11:00
Damien George	c3a18d74eb	extmod/modmarshal: Add new marshal module. ... This commit implements a small subset of the CPython `marshal` module. It implements `marshal.dumps()` and `marshal.loads()`, but only supports (un)marshalling code objects at this stage. The semantics match CPython, except that the actual marshalled bytes is not compatible with CPython's marshalled bytes. The module is enabled at the everything level (only on the unix coverage build at this stage). Signed-off-by: Damien George <damien@micropython.org>	2025-02-11 16:54:20 +11:00
Angus Gratton	97f444bfa0	extmod/mbedtls: Try GC before failing to setup socket on esp32, unix. ... On mbedTLS ports with non-baremetal configs (mostly esp32, technically also unix port), mbedTLS memory is allocated from the libc heap. This means an old SSL socket may be holding large SSL buffers and preventing a new SSL socket from being allocated. As a workaround, trigger a GC pass and retry before failing outright. This was originally implemented as a global mbedTLS calloc function, but there is complexity around the possibility of C user modules calling into mbedTLS without holding the GIL. It would be interesting to try making a generic version for any malloc which fails, but this would require checking for a Python thread and probably making the GIL recursive. This work was funded through GitHub Sponsors. Signed-off-by: Angus Gratton <angus@redyak.com.au>	2025-02-03 15:02:02 +11:00
Damien George	abb13b1e1e	extmod/lwip-include: Factor common lwIP config into lwipopts_common.h. ... This lwIP configuration file has options that are common to all ports, and the ports are updated to use this file. This change is a no-op, the lwIP configuration remains the same for the four ports using this common file. This reduces code duplication, keeps the ports in sync, and makes it easier to update the configuration for all ports at once. Signed-off-by: Damien George <damien@micropython.org>	2025-01-29 12:36:27 +11:00
Jared Hancock	f29bd5a65e	extmod/modlwip: Fix incorrect peer address for IPv6. ... For IPv6 connections, the peer address was previously defined as only the first four bytes of the IP address. For IPv6 addresses, this resulted in an incorrect IPv4 address. For instance, receiving a packet via `::recvfrom` from `'fe80::87:e7ff:fe48:629a'` is returned as having a peer address of `'254.128.0.0'` Signed-off-by: Jared Hancock <jared.hancock@centeredsolutions.com>	2025-01-29 10:49:34 +11:00
Glenn Strauss	f2cd1a3db6	lib/mbedtls: Update to mbedtls v3.6.2. ... Signed-off-by: Glenn Strauss <gstrauss@gluelogic.com>	2025-01-17 10:10:57 +11:00
Yoctopuce	61995b53c3	extmod/moddeflate: Add missing size_t cast. ... To prevent compiler warnings. Signed-off-by: Yoctopuce <dev@yoctopuce.com>	2025-01-02 13:33:39 +11:00
Yoctopuce	b323b42770	extmod/modsocket: Add missing static in private function definitions. ... Signed-off-by: Yoctopuce <dev@yoctopuce.com>	2025-01-02 13:33:39 +11:00