453 Commits

Author	SHA1	Message	Date
Damien George	fa42487e45	extmod/moddeflate: Keep DeflateIO state consistent on window alloc fail. ... Allocation of a large compression window may fail, and in that case keep the `DeflateIO` state consistent so its other methods (such as `close()`) still work. Consistency is kept by only updating the `self->write` member if the window allocation succeeds. Thanks to @jimmo for finding the bug. Signed-off-by: Damien George <damien@micropython.org>	2025-03-27 11:58:59 +11:00
Damien George	14ba32bb20	extmod/vfs_rom: Add bounds checking for all filesystem accesses. ... Testing with ROMFS shows that it is relatively easy to end up with a corrupt filesystem on the device -- eg due to the ROMFS deploy process stopping half way through -- which could lead to hard crashes. Notably, there can be boot loops trying to mount a corrupt filesystem, crashes when importing modules like `os` that first scan the filesystem for `os.py`, and crashing when deploying a new ROMFS in certain cases because the old one is removed while still mounted. The main problem is that `mp_decode_uint()` has an loop that keeps going as long as it reads 0xff byte values, which can happen in the case of erased and unwritten flash. This commit adds full bounds checking in the new `mp_decode_uint_checked()` function, and that makes all ROMFS filesystem accesses robust. Signed-off-by: Damien George <damien@micropython.org>	2025-02-26 16:11:19 +11:00
Keenan Johnson	321b30ca56	extmod/modtls_mbedtls: Wire in support for DTLS. ... This commit enables support for DTLS, i.e. TLS over datagram transport protocols like UDP. While support for DTLS is absent in CPython, it is worth supporting it in MicroPython because it is the basis of the ubiquitous CoAP protocol, used in many IoT projects. To select DTLS, a new set of "protocols" are added to SSLContext: - ssl.PROTOCOL_DTLS_CLIENT - ssl.PROTOCOL_DTLS_SERVER If one of these is set, the library assumes that the underlying socket is a datagram-like socket (i.e. UDP or similar). Our own timer callbacks are implemented because the out of the box implementation relies on `gettimeofday()`. This new DTLS feature is enabled on all ports that use mbedTLS. This commit is an update to a previous PR #10062. Addresses issue #5270 which requested DTLS support. Signed-off-by: Keenan Johnson <keenan.johnson@gmail.com>	2025-02-14 12:55:25 +11:00
Damien George	30acb16ad3	extmod/vfs_rom: Remove ability to create VfsRom from an address. ... It's not necessary to support this, which allows an arbitrary memory address to be specified and potentially allows invalid memory accesses. Requiring an object with the buffer protocol is safer, and also means that the length of the region is always specified. Signed-off-by: Damien George <damien@micropython.org>	2025-02-11 17:56:13 +11:00
Damien George	c3a18d74eb	extmod/modmarshal: Add new marshal module. ... This commit implements a small subset of the CPython `marshal` module. It implements `marshal.dumps()` and `marshal.loads()`, but only supports (un)marshalling code objects at this stage. The semantics match CPython, except that the actual marshalled bytes is not compatible with CPython's marshalled bytes. The module is enabled at the everything level (only on the unix coverage build at this stage). Signed-off-by: Damien George <damien@micropython.org>	2025-02-11 16:54:20 +11:00
Angus Gratton	195bf05115	tests: Add a test for SSL socket memory leaks. ... Test is for an issue reported on the micropython-lib Discord as effecting the rp2 port umqtt.simple interface when reconnecting with TLS, however it's a more generic problem. Currently this test fails on RPI_PICO_W and ESP32_GENERIC_C3 (and no doubt others). Fixes are in the subsequent commits. This work was funded through GitHub Sponsors. Signed-off-by: Angus Gratton <angus@redyak.com.au>	2025-02-03 15:02:02 +11:00
Alessandro Gatti	e73cf71a24	tests/extmod/re_sub.py: Fix test execution on Python 3.13. ... This commit fixes a test failure for `extmod/re_sub.py` where the code, whilst being correct, would not make the test pass due to a newer Python version than expected. On Python 3.13, running `tests/extmod/re_sub.py` would yield a deprecation warning about `re.sub` not providing the match count as a keyword parameter. This warning would be embedded in the expected test result and thus the test would always fail. Co-authored-by: stijn <stijn@ignitron.net> Signed-off-by: Alessandro Gatti <a.gatti@frob.it>	2025-01-15 07:46:09 +11:00
Damien George	fca8ea6c39	tests/extmod/vfs_rom.py: Import errno for test. ... It's needed by the test. This previously passed because the compiler (actually parser) optimises away errno constants. Signed-off-by: Damien George <damien@micropython.org>	2025-01-06 17:55:11 +11:00
Damien George	c73204128e	tests/extmod: Add VfsRom test. ... Provides full coverage of the VfsRom driver. Signed-off-by: Damien George <damien@micropython.org>	2024-12-23 13:04:55 +11:00
Damien George	d10cda66dc	tests/extmod: Add test for uctypes.addressof function. ... Signed-off-by: Damien George <damien@micropython.org>	2024-12-18 11:01:23 +11:00
Damien George	2c80d36998	tests/extmod: Convert machine1.py test to use unittest. ... Signed-off-by: Damien George <damien@micropython.org>	2024-12-06 13:48:23 +11:00
Angus Gratton	eb80b04944	tests/extmod: Workaround CPython warning in asyncio_new_event_loop test. ... This started failing in CI on the mingw build, after CPython updated to 3.12.7. The test prints two warnings during interpreter shutdown of "Task was destroyed but it is pending!". This didn't happen on other CPython builds, and I think that's because of finalizer order in CPython interpreter shutdown but not certain (the loop finalizer calls loop.close() if not already closed). Adding explicit calls to loop.close() causes the warning to be printed on every run with CPython 3.12.7 on Linux. Next, added the workaround exception handler to swallow this exception as MicroPython doesn't produce an equivalent. This work was funded through GitHub Sponsors. Signed-off-by: Angus Gratton <angus@redyak.com.au>	2024-11-28 23:12:34 +11:00
Corran Webster	e70048cf59	extmod/modframebuf: Fix 0 radius bug in FrameBuffer.ellipse. ... This fixes a bug in FrameBuffer.ellipse where it goes into an infinite loop if both radii are 0. This fixes the bug with a simple pre-check to see if both radii are 0, and in that case sets a single pixel at the center. This is consistent with the behaviour of the method when called with just one of the radii set to 0, where it will draw a horizontal or vertical line of 1 pixel width. The pixel is set with setpixel_checked so it should handle out-of-bounds drawing correctly. This fix also includes three new tests: one for the default behaviour, one for drawing out-of-bounds, and one for when the sector mask is 0. Fixes issue #16053. Signed-off-by: Corran Webster <cwebster@unital.dev>	2024-11-28 22:56:44 +11:00
Damien George	09ea901317	tests/extmod: Add test to compare time_ns with time. ... They should be close together. Signed-off-by: Damien George <damien@micropython.org>	2024-10-24 23:24:09 +11:00
Damien George	1ec0c9b886	tests/extmod: Use time_ns instead of time in lfs mtime test. ... Because VfsLfs2 uses time_ns to create timestamps for files, and for the test to give consistent results it also needs to use this same function. Signed-off-by: Damien George <damien@micropython.org>	2024-10-24 23:24:09 +11:00
Damien George	ca6aed7649	tests/extmod: Fix access of RTC class in machine.RTC test. ... This previously passed on some targets that automatically import the `machine` module in `boot.py`. Signed-off-by: Damien George <damien@micropython.org>	2024-10-24 16:29:43 +11:00
Damien George	6d98280fda	tests/extmod: Add a simple test for machine.RTC. ... Tests at least that the datetime can be set and get correctly. Signed-off-by: Damien George <damien@micropython.org>	2024-10-24 11:43:03 +11:00
Damien George	3f54e5dff2	tests/extmod: Support esp32,mimxrt,stm32,samd ports in UART TX test. ... Getting this test running on stm32- and mimxrt-based boards requires adding a small delay after constructing the UART so that the initial idle frame has time to be transmitted before the test starts. Also, the timing margin needs to account for an additional 1-bit worth of time on some MCUs. Thanks to @robert-hh for the esp32, mimxrt and samd settings. Signed-off-by: Damien George <damien@micropython.org>	2024-10-22 22:21:17 +11:00
Damien George	0646177997	tests/extmod: Make invalid-blockdev test work consistently on all ports. ... Some ports (eg stm32) configure the FAT driver differently (eg with multi-partition support) and that leads to a slightly different sequence of block reads, compared to other configurations (eg rp2). Comment out the printing in `readblocks()` so the tests are deterministic (the printing is still useful for debugging). Signed-off-by: Damien George <damien@micropython.org>	2024-10-22 14:32:51 +11:00
Damien George	1e3b24238e	tests/extmod: Adjust ssl/tls tests to run on targets with axTLS. ... Signed-off-by: Damien George <damien@micropython.org>	2024-10-22 14:31:13 +11:00
Damien George	7d442373af	tests/extmod: Config SPI test for esp8266 and skip SoftTimer test. ... Signed-off-by: Damien George <damien@micropython.org>	2024-10-22 14:31:13 +11:00
Corran Webster	d1574de3b6	extmod/modframebuf: Fix FrameBuffer size check for stride corner-cases. ... This is a fix for issue #15944, and handles corner cases in the FrameBuffer code when using stride values where the last line's stride may extend past the end of the underlying buffer. This commit includes extra tests for these corner cases. For example a GS8 format FrameBuffer with a width of 8, height of 2 and stride of 10 should be able to fit into a buffer of size 18 (10 bytes for the first horizontal line, and 8 bytes for the second -- the full 10 bytes are not needed). Similarly a 1 by 9 FrameBuffer in MONO_VLSB format with a stride of 10 should be able to fit into a buffer of length 11 (10 bytes for the first 8 lines, and then one byte for the 9th line. Being able to do this is particularly important when cropping the corner of an existing FrameBuffer, either to copy a sprite or to clip drawing. Signed-off-by: Corran Webster <cwebster@unital.dev>	2024-10-22 13:05:31 +11:00
Damien George	97af1001ae	rp2/machine_uart: Make it so TX is done only when no longer busy. ... Prior to this commit, when flushing a UART on the rp2 port, it returns just before the last character is sent out the wire. Fix this by waiting until the BUSY flag is cleared. This also fixes the behaviour of `UART.txdone()` to return `True` only when the last byte has gone out. Updated docs and tests to match. The test now checks that UART TX time is very close to the expected time (prior, it was just testing that the TX time was less than the expected time). Signed-off-by: Damien George <damien@micropython.org>	2024-10-22 10:17:05 +11:00
Angus Gratton	05ac69329d	esp32: Fix hang in taskYIELD() on riscv CPUs when IRQs disabled. ... Regression introduced in 337742f. The hang occurs because the esp32 port was calling "from ISR" port-layer functions to set/clear the interrupt mask. FreeRTOS kernel therefore doesn't know the CPU is in a critical section. In taskYIELD() the riscv port layer blocks after yielding until it knows the yield has happened, and would block indefinitely if IRQs are disabled (until INT WDT triggers). Moving to the "public" portENTER_CRITICAL/portEXIT_CRITICAL API means that FreeRTOS knows we're in a critical section and can react accordingly. Adds a regression test for this case (should be safe to run on all ports). On single core CPUs, this should result in almost exactly the same behaviour apart from fixing this case. On dual core CPUs, we now have cross-CPU mutual exclusion for atomic sections. This also shouldn't change anything, mostly because all the code which enters an atomic section runs on the same CPU. If it does change something, it will be to fix a thread safety bug. There is some risk that this change triggers a FreeRTOS crash where there is a call to a blocking FreeRTOS API with interrupts disabled. Previously this code might have worked, but was probably thread unsafe and would have hung in some circumstances. This work was funded through GitHub Sponsors. Signed-off-by: Angus Gratton <angus@redyak.com.au>	2024-10-10 10:59:51 +11:00
Damien George	197becbdcc	tests: Tweak machine SPI and UART tests to work with esp32c6. ... Signed-off-by: Damien George <damien@micropython.org>	2024-10-09 23:34:27 +11:00
Angus Gratton	f4ab9d9247	extmod/vfs_blockdev: Implement common helper for read and write. ... - Code size saving as all of these functions are very similar. - Resolves the "TODO" of the plain read and write functions not propagating errors. An error in the underlying block device now causes VFatFs to return EIO, for example. This work was funded through GitHub Sponsors. Signed-off-by: Angus Gratton <angus@redyak.com.au>	2024-09-26 22:08:48 +10:00
Angus Gratton	4f6d4b2b49	extmod/vfs_blockdev: Check block device function positive results. ... A positive result here can result in eventual memory corruption as littlefs expects the result of a cache read/write function to be 0 or a negative integer for an error. Closes #13046 This work was funded through GitHub Sponsors. Signed-off-by: Angus Gratton <angus@redyak.com.au>	2024-09-26 22:08:48 +10:00
robert-hh	e23fdb1f77	tests/extmod/machine_uart_irq_txidle.py: Simplify the test script. ... Now that no minimal delay time is required for SAMD devices. Signed-off-by: robert-hh <robert@hammelrath.com>	2024-09-06 17:43:49 +10:00
Damien George	b8513e6137	tests/extmod: Add test for machine.UART.IRQ_TXIDLE. ... The test checks whether the message created by the IRQ handler appears about at the end of the data sent by UART. Supported MCUs resp. boards: - RP2040 - Teensy 4.x - Adafruit ItsyBitsy M0 - Adafruit ItsyBitsy M4 - NRF52 (Arduino Nano Connect 33 BLE) Signed-off-by: Damien George <damien@micropython.org>	2024-08-29 16:48:09 +10:00
Angus Gratton	fb4ae1eeec	test/extmod: Fix machine_spi_rate test on ESP32-C3. ... Update to the test added in 1e98c4cb75, changes the SPI pins for ESP32-C3 (IO 18 and 19 are the native USB pins). Signed-off-by: Angus Gratton <angus@redyak.com.au>	2024-08-14 15:03:27 +10:00
Peter Züger	1473ed4c64	tests/extmod/ssl_keycert.py: Add test for PKCS8 formatted DER key. ... Signed-off-by: Peter Züger <zueger.peter@icloud.com>	2024-08-13 17:32:32 +10:00
Angus Gratton	1e98c4cb75	tests/extmod: Add machine_spi_rate test. ... Based on machine_i2s_rate, allows testing basic SPI functionality and timings. Implemented and confirmed working for rp2, esp32, and pyboard. This work was funded through GitHub Sponsors. Signed-off-by: Angus Gratton <angus@redyak.com.au>	2024-08-07 15:07:13 +10:00
Damien George	a4b3825bd4	tests/extmod: Rename machine_timer exp file to machine_soft_timer. ... This was missed in 9ba04cc756 Signed-off-by: Damien George <damien@micropython.org>	2024-08-07 15:02:52 +10:00
Angus Gratton	6d05424754	tests/extmod: Add esp32 support to the machine_i2s_rate test. ... This work was funded through GitHub Sponsors. Signed-off-by: Angus Gratton <angus@redyak.com.au>	2024-08-02 10:31:19 +10:00
Angus Gratton	9ba04cc756	tests/extmod: Skip soft machine.Timer test on esp32 port. ... Also rename the test to reflect that it's a soft timer test. Signed-off-by: Angus Gratton <angus@redyak.com.au>	2024-08-02 10:27:53 +10:00
stijn	444d7bacbe	extmod/moductypes: Validate the descriptor tuple. ... Fixes various null dereferencing, out-of-bounds memory accesses and `assert(0)` failures in the case of an invalid `uctypes` descriptor. By design `uctypes` can crash because it accesses arbitrary memory, but at least describing the descriptor layout should be forced to be correct and not crash. Fixes issue #12702. Signed-off-by: stijn <stijn@ignitron.net>	2024-07-22 14:40:45 +10:00
Michael Vornovitsky	6db91dfefb	extmod/modbtree: Add checks for already-closed database. ... Fixes use-after-free when accessing the database after it is closed with `btree_close`. `btree_close` always succeeds when called with an already-closed database. The new test checks that operations that access the underlying database (get, set, flush, seq) fail with a `ValueError` when the btree is already closed. It also checks that closing and printing the btree succeed when the btree is already closed. Fixes issue #12543. Signed-off-by: Michael Vornovitsky <michaelvornovitskiy@outlook.com>	2024-07-22 10:42:29 +10:00
Damien George	fce3cea244	tests/extmod: Make get_event_loop tests compatible with CPython 3.12. ... Follow up to 2e852522b1: instead of having .exp files for the get_event_loop tests, tweak them so they are compatible with CPython 3.12. This requires calling `asyncio.set_event_loop()` so there is an active event loop and `asyncio.get_event_loop()` succeeds without a warning. Signed-off-by: Damien George <damien@micropython.org>	2024-07-18 11:08:48 +10:00
Sylvain Zimmer	838794ebcc	extmod/mbedtls: Enable GCM and ECDHE-RSA in common mbedtls config. ... Enable support for cipher suites like TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, as suggested in https://github.com/micropython/micropython/issues/14204#issuecomment-2024366349 and https://github.com/micropython/micropython/issues/10485#issuecomment-1396426824 Tests have been run on the top 500 domains from moz.com. Without this patch, 155 out of 500 fail to connect because of TLS issues. This patch fixes them all. And it seems all existing mbedtls flags are needed to get good coverage of those top 500 domains. The `ssl_poll.py` test has the cipher bits increased from 512 to 1024 in its test key/cert so that it can work with ECDHE-RSA which is now the chosen cipher. Signed-off-by: Sylvain Zimmer <sylvain@sylvainzimmer.com> Signed-off-by: Damien George <damien@micropython.org>	2024-07-02 13:19:17 +10:00
Damien George	2e852522b1	tests/extmod: Add .exp test files for asyncio.get_event_loop tests. ... And use `asyncio.new_event_loop()` where possible. This change is needed because CPython 3.12 deprecated the `get_event_loop()` function. Signed-off-by: Damien George <damien@micropython.org>	2024-05-27 13:56:55 +10:00
Damien George	1ea06b99d8	tests/extmod: Fix regex strings to be of r"" type. ... Otherwise escape characters like \s and \W won't work correctly. Signed-off-by: Damien George <damien@micropython.org>	2024-05-27 13:56:55 +10:00
Damien George	8692d2602a	extmod/asyncio: Make current_task raise exception when there is no task. ... Matches CPython behaviour. Fixes issue #11530. Signed-off-by: Damien George <damien@micropython.org>	2024-02-28 15:48:51 +11:00
Felix Dörre	8547a78275	extmod/modwebsocket: Fix websocket to send correct close frame. ... When the websocket closes currently, it does not send a proper "close"-frame, but rather encodes the 0x8800-sequence inside a binary packet, which is wrong. The close packet is a different kind of websocket frame, according to https://www.rfc-editor.org/rfc/rfc6455. This change resolves an error in Firefox when the websocket closes. Signed-off-by: Felix Dörre <felix@dogcraft.de>	2024-02-26 12:20:40 +11:00
Damien George	b87bbaeb43	tests: Use vfs module instead of os. ... Signed-off-by: Damien George <damien@micropython.org>	2024-02-07 13:25:09 +11:00
Felix Dörre	b802f0f8ab	extmod/modtls: Move the native ssl module to tls. ... The current `ssl` module has quite a few differences to the CPython implementation. This change moves the MicroPython variant to a new `tls` module and provides a wrapper module for `ssl` (in micropython-lib). Users who only rely on implemented comparible behavior can continue to use `ssl`, while users that rely on non-compatible behavior should switch to `tls`. Then we can make the facade in `ssl` more strictly adhere to CPython. Signed-off-by: Felix Dörre <felix@dogcraft.de>	2024-02-07 12:58:52 +11:00
Damien George	d5b96813dc	extmod/modssl_mbedtls: Fix cipher iteration in SSLContext.get_ciphers. ... Prior to this commit it would skip every second cipher returned from mbedtls. The corresponding test is also updated and now passes on esp32, rp2, stm32 and unix. Signed-off-by: Damien George <damien@micropython.org>	2024-01-29 15:11:46 +11:00
Damien George	40687451bb	tests/extmod/framebuf_polygon.py: Replace sys.stdout.write with print. ... So the test doesn't depend on the `sys` module. Signed-off-by: Damien George <damien@micropython.org>	2024-01-29 13:07:25 +11:00
Damien George	6bb446b7ff	tests/extmod: Remove asyncio .exp files that match CPython output. ... These were added back in commit c4935f3049 because the tests required CPython 3.8, which was quite new at the time. But CPython 3.8 was released over 4 years ago (October 2019) and the CI test runners, and developers, have this (or a more recent) CPython version. Removing the .exp files also helps keep MicroPython semantics the same as CPython. The asyncio_fair.py test it adjusted slightly to have more deterministic timing and output. Signed-off-by: Damien George <damien@micropython.org>	2024-01-24 11:20:06 +11:00
Damien George	c3ca3612d1	tests/extmod/asyncio_wait_task.py: Add test for raise and delayed wait. ... This case was fixed in 2ecbad4e91, which stored the exception in the task object. Signed-off-by: Damien George <damien@micropython.org>	2024-01-23 11:13:35 +11:00
Damien George	2ecbad4e91	extmod/asyncio: Support gather of tasks that finish early. ... Adds support to asyncio.gather() for the case that one or more (or all) sub-tasks finish and/or raise an exception before the gather starts. Signed-off-by: Damien George <damien@micropython.org>	2024-01-22 12:03:21 +11:00