2.5 can be represented correctly in object representation C, but 2.3 cannot
(it is slightly truncated).
Signed-off-by: Damien George <damien@micropython.org>
Commit 0abf6f830c removed _boot.py from the
manifest for the GENERIC_512K board because the build does not include a
filesystem. But the main code expects _boot.py to be there and prints an
error if it's not. So add a custom _boot.py, which just sets the
gc.threshold().
Signed-off-by: Damien George <damien@micropython.org>
The 72x40 OLED requires selecting the internal IREF, as opposed to the
default external IREF. This is an undocumented feature in the SSD1306
datasheet, but is present in the SSD1315 datasheet. It's possible the
72x40 OLED is actually using the newer SSD1315 controller. Sending the
IREF select command to SSD1306 displays has no effect on them, so it's
added to the init_display() instead of wrapping in an "if width = 72".
Also tested on a 128x64 OLED using the SSD1315 controller (smaller ribbon
cable) and the proposed change has no effect on the display, as the module
comes with the correct current limiting resistor. Internal and external
IREF work the same.
Fixes issue #7281.
This adds a wlan.config(reconnects=N) option to set the number of reconnect
attempts that will be made if the WLAN connection goes down. The default
is N=-1 (infinite retries, current behavior). Setting
wlan.config(reconnects=0) will disable the reconnect attempts.
A nice side effect of reconnects=0 is that wlan.status() will report the
disconnect reason now. See related issue #5326.
Ethernet-PHYs from ESP-IDF (LAN8720, IP101, RTL8201, DP83848) are now
supported in IDF v4.1 and above. PHY_KSZ8041 is only for ESP-IDF 4.3 and
above. ESP32S2 is not supported.
Signed-off-by: Tobias Eydam <eydam-prototyping@outlook.com>
The comments in NimBLE for ble_gattc_notify_custom() state that "This
function consumes the supplied mbuf regardless of the outcome.". And
inspection of NimBLE code shows that this is the case. So the comment can
be removed.
Signed-off-by: Damien George <damien@micropython.org>
This commit fixes a problem with a race between cancellation of task A and
completion of task B, when A waits on B. If task B completes just before
task A is cancelled then the cancellation of A does not work. Instead,
the CancelledError meant to cancel A gets passed through to B (that's
expected behaviour) but B handles it as a "Task exception wasn't retrieved"
scenario, printing out such a message (this is because finished tasks point
their "coro" attribute to themselves to indicate they are done, and
implement the throw() method, but that method inadvertently catches the
CancelledError). The correct behaviour is for B to bounce that
CancelledError back out.
This bug is mainly seen when wait_for() is used, and in that context the
symptoms are:
- occurs when using wait_for(T, S), if the task T being waited on finishes
at exactly the same time as the wait-for timeout S expires
- task T will have run to completion
- the "Task exception wasn't retrieved message" is printed with
"<class 'CancelledError'>" as the error (ie no traceback)
- the wait_for(T, S) call never returns (it's never put back on the
uasyncio run queue) and all tasks waiting on this are blocked forever
from running
- uasyncio otherwise continues to function and other tasks continue to be
scheduled as normal
The fix here reworks the "waiting" attribute of Task to be called "state"
and uses it to indicate whether a task is: running and not awaited on,
running and awaited on, finished and not awaited on, or finished and
awaited on. This means the task does not need to point "coro" to itself to
indicate finished, and also allows removal of the throw() method.
A benefit of this is that "Task exception wasn't retrieved" messages can go
back to being able to print the name of the coroutine function.
Fixes issue #7386.
Signed-off-by: Damien George <damien@micropython.org>
The implementation uses the LPUARTx devices. Up to 8 UARTs can be used,
given that the pins are accessible. E.g. 8 on Teensy 4.1, 5 on
MIMXRT1020_EVK.
For Tennsy 4.0 and 4.1 the UART numbers are as printed on the pinout 1..N.
The MIMXRT10xx-EVK boards have only one UART named, which gets the number
1. All other UART are assigned to different Pins:
MIMXRT1010-EVK:
D0/D1 UART 1
D6/D7 UART 2
A0/D4 UART 3
MIMXRT1020-EVK:
D0/D1 UART 1
D6/D9 UART 2
D10/D12 UART 3
D14/D15 UART 4
A0/A1 UART 5
MIMXRT1050-EVK, MIMXRT1060-EVK, MIMXRT1064-EVK:
D0/D1 UART 1
D7/D6 UART 2
D8/D9 UART 3
A1/A0 UART 4
Now a ctrl-C will not stop mpremote, rather this character will be passed
through to the attached device.
The mpremote version is also increased to 0.0.5.
Signed-off-by: Damien George <damien@micropython.org>
Using just the list of available ports, instead of a hard-coded list of
possible ports, means that all ports will be available for auto connection.
And the order that they will be attempted in will match what's printed by
"mpremote connect list" (and will be the same as before, trying ACMx before
USBx). Auto-connect will also now work on Mac, and will allow all COM
ports on Windows.
Signed-off-by: Damien George <damien@micropython.org>
With docs and a multi-test using TCP server/client.
This method is a MicroPython extension, although there is discussion of
adding it to CPython: https://bugs.python.org/issue41305
Signed-off-by: Mike Teachman <mike.teachman@gmail.com>
This fixes error: cast to smaller integer type 'int' from 'pthread_t'.
pthread_t is defined as long, not as int.
Signed-off-by: Pavol Rusnak <pavol@rusnak.io>
The rtc_set_datetime() from pico-sdk will validate the values in the
datetime_t structure and refuse to set the time if they aren't valid. It
makes sense to raise an exception if this happens instead of failing
silently which might be confusing (as an example, see:
https://github.com/micropython/micropython/pull/6928#issuecomment-860166044
).
The supplied value for microseconds in datetime() will be treated as a
starting value for the reported microseconds. Due to internal processing
in setting the time, there is an offset about 1 ms.
This change moves the datetime tuple format back to the one used by all the
other ports:
(year, month, day, weekday, hour, minute, second, microsecond)
Weekday is a number between 0 and 6, with 0 assigned to Monday. It has to
be provided when setting the RTC with datetime(), but will be ignored on
entry and calculated when needed.
The weekday() method was removed, since that is now again a part of the
datetime tuple.
The now() method was updated so it continues to return a tuple that matches
CPython's datetime module.
Initial support for machine.RTC on rp2 port. It only supports datetime()
method and nothing else. The method gets/returns a tuple of 8 items, just
like esp32 port, for example, but the usec parameter is ignored as the RP2
RTC only works up to seconds precision.
The Pico RTC isn't very useful as the time is lost during reset and there
seems to be no way to easily power up just the RTC clock with a low current
voltage, but still there seems to be use-cases for that, see issues #6831,
and a Thonny issue #1592. It was also requested for inclusion on v1.15
roadmap on #6832.
Signed-off-by: Krzysztof Adamski <k@japko.eu>
Changes introduced are:
- the application offset is now loaded from the partition table instead of
being hard-coded to 0x10000
- maximum size of all sections is computed using the partition table
- an error is generated if any section overflows its allocated space
- remaining bytes are printed for each section
Signed-off-by: Damien George <damien@micropython.org>
This commit adds a few math functions to the source list in the Makefile,
and implements the log2f function, so that ulab can be compiled on the nrf
boards. It also addresses part of #5162.
This fix prevents server.wait_closed() from raising an AttributeError when
trying to access server.task. This can happen if it is called immediately
after start_server().
This commit fixes the following problems converting to/from Python integers
and ffi types:
- integers of 8 and 16 bits not working on big endian
- integers of 64 bits not working on 32 bits architectures
- unsigned returns were converted to signed Python integers
Fixes issue #7269.
Currently only advertising and scanning are supported, using the ring
buffer for events (ie not synchronous events at this stage).
The ble_gap_advertise.py multi-test passes (tested on a nucleo_wb55rg
board).
Signed-off-by: Damien George <damien@micropython.org>
Zephyr's default value for CONFIG_NET_SOCKETS_POSIX_NAMES was changed
from false to true between Zephyr v2.5.0 and v2.6.0. This caused
conflicts in MicroPython, which uses the zsock_ prefixed functions, so
disable it.
Signed-off-by: Maureen Helm <maureen.helm@nxp.com>
Zephyr's Kconfig symbols and defaults for SDHC/SDMMC disk drivers and
the disk access subsystem were reworked between Zephyr v2.5.0 and
v2.6.0. Update MicroPython accordingly.
Signed-off-by: Maureen Helm <maureen.helm@nxp.com>
Following on from ef16834887, this adds a
coverage build and running of the test suite on an ARM 32-bit Linux-based
architecture.
Signed-off-by: Damien George <damien@micropython.org>
Prior to this commit, cache flushing for ARM native code was done only in
the assembler code asm_thumb_end_pass()/asm_arm_end_pass(), at the last
pass of the assembler. But this misses flushing the cache when loading
native code from an .mpy file, ie in persistentcode.c.
The change here makes sure the cache is always flushed/cleaned/invalidated
when assigning native code on ARM architectures.
This problem was found running tests/micropython/import_mpy_native_gc.py on
the mimxrt port.
Signed-off-by: Damien George <damien@micropython.org>
Initial version, using the LP RTC clock. It provides setting the date and
time with rtc.init() or rtc.datetime(), and reading the date and time with
rtc.datetime() or rtc.now(). The method weekday() reports the weekday of
the current date. It starts with 0 for Monday.
The tuple order for datetime() and now() matches the CPython sequence:
(year, month, day, hour, minute, second, microsecond, TZ). TZ is ignored
and reported as None. Microsecond is provided at a best effort.
If a battery is not supplied, the default boot date/time is 1970/1/1 0:0:0.
With a battery, the clock continues to run even when the board is not
powered. The clock is quite precise. If not, using rtc.calibration() may
help.
It supports three hardware timer channels based on the PIT timers of the
MIMXRT MCU. The timer id's are 0, 1 and 2. On soft reboot all active
timers will be stopped via finalisers.
This is required since the Teensy Halfkay loader attempts to erase all of
the flash but fails to do so, at least in my tests. Formatting brings it
back to a known state.
This commit adds full support for a filesystem on all boards, with a block
device object mimxrt.Flash() and uos.VfsLfs2 enabled.
Main changes are:
- Refactoring of linker scripts to accomodate reserved area for VFS. VFS
will take up most of the available flash. 1M is reserved for code. 9K is
reserved for flash configuration, interrupts, etc.
- Addition of _boot.py with filesystem init code, called from main.c.
- Definition of the mimxrt module with a Flash class in modmimxrt.[ch].
- Implementation of a flash driver class in mimxrt_flash.c. All flashing
related functions are stored in ITCM RAM.
- Addition of the uos module with filesystem functions.
- Implementation of uos.urandom() for the sake of completeness of the uos
module.
It uses sample code from CircuitPython supplied under MIT license, which
uses the NXP SDK example code.
Done in collaboration with Philipp Ebensberger aka @alphaFred who
contributed the essential part to enable writing to flash while code is
executing, among other things.
Adds support for NeoPixels on GPIO32 and GPIO33 on ESP32. Otherwise,
NeoPixels wired to GPIO32/33 wll silently fail without any hints to the
user.
With thanks to @robert-hh.
Fixes issue #7221.
ATOM is a very small ESP32 development board produced by M5Stack, with a
size of 24mm * 24mm, with peripherals such as WS2812, IR, button, MPU6886
(Only Matrix), and 8 GPIO extensions. It also has a plastic shell.
The random module's getrandbits() method didn't give a proper error message
when calling it with a value that was outside of the range of 1-32, which
can lead to confusion using this function (which under CPython can accept
numbers larger than 32). Now instead of simply giving a ValueError it
gives an error message that states that the number of bits is constrained.
Also, since the random module's functions getrandbits() and randint()
differ from CPython, tests have been added to describe these differences.
For getrandbits the relevant documentation is shown and added to the docs.
The same is given for randint method so that the information is more easily
found.
Finally, since the int object lacks the bit_length() method there is a test
for that method also to include within the docs, showing the difference to
CPython.
asan considers that memcmp(p, q, N) is permitted to access N bytes at each
of p and q, even for values of p and q that have a difference earlier.
Accessing additional values is frequently done in practice, reading 4 or
more bytes from each input at a time for efficiency, so when completing
"non_exist<TAB>" in the repl, this causes a diagnostic:
==16938==ERROR: AddressSanitizer: global-buffer-overflow on
address 0x555555cd8dc8 at pc 0x7ffff726457b bp 0x7fffffffda20 sp 0x7fff
READ of size 9 at 0x555555cd8dc8 thread T0
#0 0x7ffff726457a (/usr/lib/x86_64-linux-gnu/libasan.so.5+0xb857a)
#1 0x555555b0e82a in mp_repl_autocomplete ../../py/repl.c:301
#2 0x555555c89585 in readline_process_char ../../lib/mp-readline/re
#3 0x555555c8ac6e in readline ../../lib/mp-readline/readline.c:513
#4 0x555555b8dcbd in do_repl /home/jepler/src/micropython/ports/uni
#5 0x555555b90859 in main_ /home/jepler/src/micropython/ports/unix/
#6 0x555555b90a3a in main /home/jepler/src/micropython/ports/unix/m
#7 0x7ffff619a09a in __libc_start_main ../csu/libc-start.c:308
#8 0x55555595fd69 in _start (/home/jepler/src/micropython/ports/uni
0x555555cd8dc8 is located 0 bytes to the right of global variable
'import_str' defined in '../../py/repl.c:285:23' (0x555555cd8dc0) of
size 8
'import_str' is ascii string 'import '
Signed-off-by: Jeff Epler <jepler@gmail.com>
With GCC 11 there is now a warning about array bounds of OTP-mac, due to
the OTP being a literal address.
Signed-off-by: Damien George <damien@micropython.org>
If digest is called then the hash object is put in a "final" state and
calling update() or digest() again will raise a ValueError (instead of
silently producing the wrong result).
See issue #4119.
Signed-off-by: Damien George <damien@micropython.org>
The 512k build does not have a filesystem so there is no reason to include
the filesystem-related modules. This commit provides a custom manifest.py
for this board which no longer includes: _boot.py, flashbdev.py,
inisetup.py, upip.py, upip_utarfile.py. This cuts the build down by about
9k of flash.
Signed-off-by: Damien George <damien@micropython.org>
This adds a coverage build and running of the test suite on a MIPS 32-bit
big endian architecture. It uses the feature of qemu to execute foreign
code as though it were native to the system (using qemu user mode). The
code compiled for MIPS will run under the qemu VM, but all syscalls made by
this code go to the host (Linux) system.
See related #7268 and #7273.
Signed-off-by: Damien George <damien@micropython.org>
- modified pin type from pin_obj_t to machine_pin_obj_t
- created machine_pin.c
- implemented basic version of make-pins.py to genertate pins.c/.h files
automatically; the only alternate function currently supported is GPIO
- added af.csv files for all supported MCUs
- replaced pins.c/pins.h files with pin.csv for all boards
- implemented on/off/high/low/value/init methods
- Implemented IN/OUT/OPEN_DRAIN modes
- modified LDFLAGS for DEBUG build to get usefull .elf file for debugging
Signed-off-by: Philipp Ebensberger
As the new default behaviour, this allows PyDFU to be used with all
devices, not just the ones matching a specific set of VID/PID values. But
it's still possible to specify VID/PID if needed to narrow down the
selection of the USB device.
Signed-off-by: Tobias Thyrrestrup <tt@LEGO.com>
Leaving the bootloader from an IRQ (eg USB or I2C IRQ) will not work if
MBOOT_LEAVE_BOOTLOADER_VIA_RESET is disabled, ie if mboot jumps directly to
the application. This is because the CPU will still be in IRQ state when
the application starts and IRQs of lower priority will be blocked.
Fix this by setting a flag when the bootloader should finish, and exit the
bootloader always from the main (top level) thread.
This also improves the USB behaviour of mboot: it no longer abruptly
disconnects when the manifest command is sent.
Signed-off-by: Damien George <damien@micropython.org>
RX and CTS are the input pins and pull-ups are enabled so they don't cause
a problem if left unconnected. But the output pins don't need a pull up
(they were originally all configured with pull up in commit
8f7491a109).
If needed, the pull-ups can be disabled in Python using machine.Pin after
the UART is constructed.
See issue #4369.
Signed-off-by: Damien George <damien@micropython.org>
The DMA driver will turn off DMA if it hasn't been used for an amount of
time (to save power). The SDIO driver for cyw43 WLAN was not informing the
DMA driver that it was using DMA and there was a chance that the DMA would
turn off in the middle of an SDIO DMA transfer. The symptoms of this would
be printing of SDIO error messages and a failure to communicate with the
cyw43 WLAN module.
This commit fixes this issue by changing the SDIO driver to use the
dma_nohal_XXX API to initialise and start the DMA.
Signed-off-by: Damien George <damien@micropython.org>
SysTick cannot wake the CPU from WFI/WFE so a hardware timer is needed to
keep track of ticks/delay (similar to the nrf port).
Fixes issue #7234.
Signed-off-by: Damien George <damien@micropython.org>
The proper way to do this is to test for __APPLE__ and __MACH__, where
__APPLE__ tests for an Apple OS and __MACH__ tests that it is based on CMU
Mach. Using both tests ensures that just Darwin (Apple's open source base
for MacOS, iOS, etc.) is recognized. __APPLE__ by itself will test for any
Apple OS, which can include older OS 7-9 and any future Apple OS. __MACH__
tests for any OS based on CMU Mach, including Darwin and GNU Hurd.
Fixes#7232.
MicroPython does not store any reference from a function object to the
module it was defined in, but there is a way to use function.__globals__ to
indirectly get the module.
See issue #7259.
Signed-off-by: Damien George <damien@micropython.org>
This significantly reduces the time taken to run the test suite (on the
unix port). Use `-j1` to disable this feature.
Signed-off-by: Jeff Epler <jepler@gmail.com>
Array equality is defined as each element being equal but to keep
code size down MicroPython implements a binary comparison. This
can only be used correctly for elements with the same binary layout
though so turn it into an NotImplementedError when comparing types
for which the binary comparison yielded incorrect results: types
with different sizes, and floating point numbers because nan != nan.
A board can now customise mboot with:
- MBOOT_LED1, MBOOT_LED2, MBOOT_LED3, MBOOT_LED4: if it needs to have
different LEDs for mboot compared to the application
- MBOOT_BOARD_LED_INIT: if it needs a fully customised LED init function
- MBOOT_BOARD_LED_STATE: if it needs a fully customised LED state-setting
function
- MBOOT_BOARD_GET_RESET_MODE: if it needs a fully customised function to
get the reset mode
With full customisation, the only requirement is a single LED to show the
status of the bootloader (idle, erasing, flashing, etc), which can be
configured to do nothing if needed.
Signed-off-by: Damien George <damien@micropython.org>
It is enabled by default to get the standard behaviour of doing a reset
after it is finished, but can be disabled by a board to jump straight to
the application (likely the board needs to use MBOOT_BOARD_CLEANUP to make
this work).
The application is passed a reset mode of BOARDCTRL_RESET_MODE_BOOTLOADER
if the bootloader was active and entered via a jump.
Signed-off-by: Damien George <damien@micropython.org>
This new logic is equivalent to the old logic when the only possibilities
for reset_mode are NORMAL, SAFE_MODE and FILESYSTEM, which is the standard
case. But the new logic also allows other reset_mode values (eg
BOOTLOADER) to run boot.py and main.py.
Signed-off-by: Damien George <damien@micropython.org>
When disabled the bootloader is entered via a direct jump. When enabled
the bootloader is entered via a system reset then a jump. It's enabled by
default to retain the existing behaviour, which is the recommended way.
Signed-off-by: Damien George <damien@micropython.org>
Improvements made:
- PSRAM support for S2
- partition definition for 16MiB flash
- correct ADC and DAC pins
- correct GPIO and IRQ pins
- S3 components in CMakeLists
Based on original commit made by Seon Rozenblum aka @UnexpectedMaker.
Signed-off-by: Damien George <damien@micropython.org>
The RP2040 has 2 cores and supports running at most 2 Python threads (the
main one plus another), and will raise OSError if a thread cannot be
created because core1 is already in use. This commit adjusts some thread
tests to be robust against such OSError's. These tests now pass on rp2
boards.
Signed-off-by: Damien George <damien@micropython.org>
This commit makes gc_lock_depth have one counter per thread, instead of one
global counter. This makes threads properly independent with respect to
the GC, in particular threads can now independently lock the GC for
themselves without locking it for other threads. It also means a given
thread can run a hard IRQ without temporarily locking the GC for all other
threads and potentially making them have MemoryError exceptions at random
locations (this really only occurs on MCUs with multiple cores and no GIL,
eg on the rp2 port).
The commit also removes protection of the GC lock/unlock functions, which
is no longer needed when the counter is per thread (and this also fixes the
cas where a hard IRQ calling gc_lock() may stall waiting for the mutex).
It also puts the check for `gc_lock_depth > 0` outside the GC mutex in
gc_alloc, gc_realloc and gc_free, to potentially prevent a hard IRQ from
waiting on a mutex if it does attempt to allocate heap memory (and putting
the check outside the GC mutex is now safe now that there is a
gc_lock_depth per thread).
Signed-off-by: Damien George <damien@micropython.org>
Any code running on core1 should be stopped on soft-reset (the GC heap is
reset so if code continues to run on core1 it will see corrupt memory).
Signed-off-by: Damien George <damien@micropython.org>
So a lock can be acquired on one Python thread and then released on
another. A test for this is added.
Signed-off-by: Damien George <damien@micropython.org>
Because vPortCleanUpTCB runs on the FreeRTOS idle task and cannot execute
any VM or runtime related code like freeing memory.
Signed-off-by: Damien George <damien@micropython.org>
All the method signatures from rp2_pio.c and friends have been taken and
converted to RST format, then explanatory notes added for each signature.
Signed-off-by: Tim Radvan <tim@tjvr.org>
uctypes.FLOAT32 has a special value representation and
uctypes_struct_scalar_size() should be used instead of GET_SCALAR_SIZE().
Signed-off-by: Damien George <damien@micropython.org>
This fixes a bug where double arguments on a 32-bit architecture would not
be passed correctly because they only had 4 bytes of storage (not 8). It
also fixes a compiler warning/error in return_ffi_value on certian
architectures: array subscript 'double[0]' is partly outside array bounds
of 'ffi_arg[1]' {aka 'long unsigned int[1]'}.
Fixes issue #7064.
Signed-off-by: Damien George <damien@micropython.org>
This adds to the ESP8266 tutorial instructions explaining which pins to
pull low to enter programming mode.
Commit made originally by @ARF1 in #2910.
Signed-off-by: Damien George <damien@micropython.org>
The number shown in the USB id is now the same as that returned by
machine.unique_id(). All 8 bytes are inserted as hex into the USB id. A
usb id at /dev/serial/by-id then looks like:
usb-MicroPython_Board_in_FS_mode_e469b03567342f37-if00
Doing "import <tab>" will now complete/list built-in modules.
Originally at adafruit#4548 and adafruit#4608
Signed-off-by: Artyom Skrobov <tyomitch@gmail.com>
Anything beginning with "_" will now only be tab-completed if there is
already a partial match for such an entry. In other words, entering
foo.<tab> will no longer complete/list anything beginning with "_".
Originally at adafruit#1850
Signed-off-by: Kathryn Lingel <kathryn@lingel.net>
This adds an initial specification of the machine.PWM class, to provide a
way to generate PWM output that is portable across the different ports.
Such functionality may already be available in one way or another (eg
through a Timer object), but because configuring PWM via a Timer is very
port-specific, and because it's a common thing to do, it's beneficial to
have a top-level construct for it.
The specification in this commit aims to provide core functionality in a
minimal way. It also somewhat matches most existing ad-hoc implementations
of machine.PWM.
See discussion in #2283 and #4237.
Signed-off-by: Damien George <damien@micropython.org>
These commented-out lines of code have been unused for a long time, so
remove them to avoid confusion as to why they are there.
mp_obj_dict_free() never existed, this line was converted from
mp_map_deinit() and commented out as soon as it was added. The call to
mp_map_deinit(mp_loaded_modules_map) was commented in
1a1d11fa32.
Fixes issue #3507.
Signed-off-by: Damien George <damien@micropython.org>
It's now possible to specify a device serial port using shorcuts like:
$ ./run-multitests.py -i pyb:a0 -i pyb:u1 multi_bluetooth/*.py
Signed-off-by: Damien George <damien@micropython.org>
This board does not work with CONFIG_NETWORKING enabled. And
CONFIG_CONSOLE_SUBSYS is enabled so that ctrl-C works.
Signed-off-by: Damien George <damien@micropython.org>
And ctrl-C can now interrupt a time.sleep call. This uses Zephyr's k_poll
API to wait efficiently for an event signal, and an optional semaphore.
Signed-off-by: Damien George <damien@micropython.org>
This helper is added to properly set a pending exception, to mirror
mp_sched_schedule(), which schedules a function.
Signed-off-by: Damien George <damien@micropython.org>
If MICROPY_ENABLE_SCHEDULER is enabled then MP_STATE_VM(sched_state) must
be updated after handling the pending exception, which is done by the
mp_handle_pending() function.
Signed-off-by: Damien George <damien@micropython.org>
If MICROPY_ENABLE_SCHEDULER is enabled then MP_STATE_VM(sched_state) must
be updated after handling the pending exception, which is done by the
mp_handle_pending() function.
Signed-off-by: Damien George <damien@micropython.org>
This can now be selected by setting MICROPY_HW_SDIO_SDMMC, which defaults
to 1, ie SDMMC1. The pins can also be selected and default to the standard
C8/C9/C10/C11/C12/D2.
This can now be selected by setting MICROPY_HW_SDCARD_SDMMC, which defaults
to 1, ie SDMMC1. This commit also renames the SD pin configuration macros
from MICROPY_HW_SDMMC2_xxx to MICROPY_HW_SDCARD_xxx, as well as renaming
MICROPY_HW_SDMMC_BUS_WIDTH to MICROPY_HW_SDCARD_BUS_WIDTH.
Signed-off-by: Damien George <damien@micropython.org>
A board can now define MBOOT_TEXT0_ADDR to place mboot at a location other
than 0x08000000. This can be useful if, for example, there is already a
different bootloader on the device.
Signed-off-by: Damien George <damien@micropython.org>
A board can now define MBOOT_LD_FILES (at the Makefile-level) to specify
custom linker scripts. And stm32_generic.ld has been split into 2 pieces
so one or the other can be reused (usually stm32_sections.ld wolud be
reused by a board, and stm32_memory.ld redefined).
Signed-off-by: Damien George <damien@micropython.org>
A board can now use BUILDING_MBOOT at the Makefile-level to do things
conditional on building mboot, for example add source files to SRC_C.
Signed-off-by: Damien George <damien@micropython.org>
Commit 1e297c8898 introduced a bug where the
very first reset-mode state on the LEDs was not shown, because prior to
that commit the first reset-mode state was the same as the initial LED
state (green on, others off) and update_reset_mode() was called after
setting this initial LED state.
This is fixed in this commit by changing the update_reset_mode() loop so
that it displays the current reset mode before doing the delay.
Signed-off-by: Damien George <damien@micropython.org>
And use the same boardctrl.h header for both the application and mboot so
these constants are consistent.
Signed-off-by: Damien George <damien@micropython.org>
This adds support for making static (ie not on the Python GC heap) soft
timers. This can be useful for a board to define a custom background
handler, or eventually for BLE/network processing to use instead of systick
slots; it will be more efficient using soft timer for this.
The main issue with using the existing code for static soft timers is that
it would combine heap allocated and statically allocated soft_timer_entry_t
instances in the same pairing-heap data structure. This would prevent the
GC from tracing some of the heap allocated entries (because the GC won't
follow pointers outside the heap).
This commit makes it so that soft timer entries are explicitly marked,
instead of relying on implicit marking by having the root of the pairing
heap in the root pointer section. Also, on soft reset only the heap-
allocated soft timers are deleted from the pairing heap, leaving the
statically allocated ones.
Signed-off-by: Damien George <damien@micropython.org>
This commit re-enables the command-line make option "FROZEN_MANIFEST". The
boards/*/mpconfigboard.cmake will now use the command-line FROZEN_MANIFEST
value if supplied.
Usage: make FROZEN_MANIFEST=~/foo/my-manifest.py
This introduces a new option, MICROPY_ERROR_REPORTING_NONE, which
completely disables all error messages. To be used in cases where
MicroPython needs to fit in very limited systems.
Signed-off-by: Damien George <damien@micropython.org>
Since version 21.4b0, Black now processes one-line docstrings by stripping
leading and trailing spaces, and adding a padding space when needed to
break up """"; see https://github.com/psf/black/pull/1740
This commit makes the Python code in this repository conform to this rule.
So this driver works on faster MCUs (that run this loop fast) with older,
slower SD cards.
Fixes issue #7129.
Signed-off-by: Damien George <damien@micropython.org>
Because "find_package(Python3 ...)" requires at least this version of
CMake. And other features like GREATER_EQUAL and COMMAND_EXPAND_LISTS need
at least CMake 3.7 and 3.8 respectively.
Signed-off-by: Damien George <damien@micropython.org>
This is now the default, but can be overridden with CLI `--no-exclusive`,
or constructing `Pyboard(..., exclusive=False)`.
Signed-off-by: Damien George <damien@micropython.org>
This commit adds the errno attribute to exceptions, so code can retrieve
errno codes from an OSError using exc.errno.
The implementation here simply lets `errno` (and the existing `value`)
attributes work on any exception instance (they both alias args[0]). This
is for efficiency and to keep code size down. The pros and cons of this
are:
Pros:
- more compatible with CPython, less difference to document and learn
- OSError().errno will correctly return None, whereas the current way of
doing it via OSError().args[0] will raise an IndexError
- it reduces code size on most bare-metal ports (because they already have
the errno qstr)
- for Python code that uses exc.errno the generated bytecode is 2 bytes
smaller and more efficient to execute (compared with exc.args[0]); so
bytecode loaded to RAM saves 2 bytes RAM for each use of this attribute,
and bytecode that is frozen saves 2 bytes flash/ROM for each use
- it's easier/shorter to type, and saves 2 bytes of space in .py files that
use it (for each use)
Cons:
- increases code size by 4-8 bytes on minimal ports that don't already have
the `errno` qstr
- all exceptions now have .errno and .value attributes (a cpydiff test is
added to address this)
See also #2407.
Signed-off-by: Damien George <damien@micropython.org>
Improvements are:
- Default period is 1000ms with callback disabled.
- if period is not specified then it's not updated (previously, if period
was not specified then it was set to -1 and running the timer callback as
fast as possible, making the REPL unresponsive).
- Use uint64_t to compute delta_ms, and raise a ValueError if the period is
too large.
- If callback is not specified then it's not updated.
- Specifying None for the callback will disable the timer.
Signed-off-by: Damien George <damien@micropython.org>
The PIO state machines on the RP2040 have 4 word deep TX and RX FIFOs. If
you only need one direction, you can "merge" them into either a single 8
word deep TX or RX FIFO.
We simply add constants to the PIO object, and set the appropriate bits in
`shiftctrl`.
Resolves#6854.
Signed-off-by: Tim Radvan <tim@tjvr.org>
Commit 8a917ad252 added the gpio_reset_pin()
call to make sure that pins that were used as ADC inputs could subsequently
be used as digital IO. But calling gpio_reset_pin() will enable the
pull-up on the pin and so pull it high for a brief period. Instead use
rtc_gpio_deinit() which will just reconfigure the pin as a digital IO and
do nothing else.
Fixes issue #7079 (see also #5771).
Signed-off-by: Damien George <damien@micropython.org>
Commit cb68a5741a broke automatic Python
feature detection when running tests, because some detection relied on a
crash of a feature script returning exactly b"CRASH".
This commit fixes this and improves the situation by testing for the lack
of a known pass result, rather than an exact failure result.
Signed-off-by: Damien George <damien@micropython.org>
For an unconnected TCP socket, poll should return WR|HUP and read should
raise ENOTCONN. This is implemented by this commit and now the following
tests pass on esp32: extmod/usocket_tcp_basic.py,
net_hosted/connect_poll.py.
Signed-off-by: Damien George <damien@micropython.org>
This change allows running the tests in tests/basics/ without any failures
(but some tests are still skipped).
Signed-off-by: Damien George <damien@micropython.org>
This fixes `error: variable 'subpkg_tried' might be clobbered by 'longjmp'
or 'vfork' [-Werror=clobbered]` when compiling on ppc64le and aarch64 (and
possibly other architectures/toolchains).
This function includes the UART prescaler in the calculation (if it has
one, eg on H7 and WB MCUs).
Signed-off-by: Damien George <damien@micropython.org>
This allows configuring the pre-allocated size of sys.modules dict, in
order to prevent unwanted reallocations at run-time (3 sys-modules is
really not quite enough for a larger project).
When building with STATIC undefined (e.g., -DSTATIC=), there are two
instances of mp_type_code that collide at link time: in profile.c and in
builtinevex.c. This patch resolves the collision by renaming one of them.
The STM32WB has a problem when address resolution is enabled: under certain
conditions the MCU can get into a state where it draws an additional 10mA
or so and eventually ends up with a broken BLE RX path in the silicon. A
simple way to reproduce this is to enable address resolution (which is the
default for NimBLE) and start the device advertising. If there is enough
BLE activity in the vicinity then the device will at some point enter the
bad state and, if left long enough, will have permanent BLE RX damage.
STMicroelectronics are aware of this issue. The only known workaround at
this stage is to not enable address resolution, which is implemented by
this commit.
Work done in collaboration with Jim Mussared aka @jimmo.
Signed-off-by: Damien George <damien@micropython.org>
Prior to this commit, if C2 was busy (eg lots of BLE activity) then it may
not have had time to respond to the notification on the IPCC_CH_MM channel
by the time additional memory was available to put on that buffer. In such
a case C1 would modify the free buffer list while C2 was potentially
accessing it, and this would eventually lead to lost memory buffers (or a
corrupt linked list). If all buffers become lost then ACL packets
(asynchronous events) can no longer be delivered from C2 to C1.
This commit fixes this issue by waiting for C2 to indicate that it has
finished using the free buffer list.
Work done in collaboration with Jim Mussared aka @jimmo.
Signed-off-by: Damien George <damien@micropython.org>
This change allows to build firmware for different rp2-based boards,
following how it is done in other ports like stm32 and esp32. So far only
the original Pico and Adafruit Feather RP2040 are added. Board names
should match (sans case) those in pico-sdk/src/boards/include/boards/.
Usage: Pico firmware can be build either using make as previously (it is
the default board) or by `make BOARD=PICO`. Feather is built by `make
BOARD=ADAFRUIT_FEATHER_RP2040`. Only the board name and flash drive size
is set, pin definition is taken from the appropriate pico-sdk board
definition. Firmware is saved in the directory build-BOARD_NAME.
Instantiation and init now support the rxbuf and txbuf keywords for setting
the buffer size. The default size is 256 bytes. The minimum and maximum
sizes are 32 and 32766 respectively.
uart.write() still includes checks for timeout, even if it is very unlikely
to happen due to a) lack of flow control support and b) the minimal timeout
values being longer than the time it needs to send a byte.
StateMachine.restart: Restarts the state machine
StateMachine.rx_fifo: Return the number of RX FIFO items, 0 if empty
StateMachine.tx_fifo: Return the number of TX FIFO items, 0 if empty
restart() seems to be the most useful one, as it resets the state machine
to the initial state without the need to re-initialise/re-create. It also
makes PIO code easier, because then stalling as an error state can be
unlocked.
rx_fifo() is also useful, for MP code to check for data and timeout if no
data arrived. Complex logic is easier handled in Python code than in PIO
code.
tx_fifo() can be useful to check states where data is not processed, and is
mostly for symmetry.
The implementation samples rosc.randombits at a frequency lower than the
oscillator frequency. This gives better random values. In addition, for
an 8-bit value 8 samples are taken and fed through a 8-bit CRC,
distributing the sampling over the byte. The resulting sampling rate is
about 120k/sec.
The RNG does not include testing of error conditions, like the ROSC being
in sync with the sampling or completely failing. Making the interim value
static causes it to perform a little bit better in short sync or drop-out
situations.
The output of uos.urandom() performs well with the NIST800-22 test suite.
In my trial it passed all tests of the sts 2.1.2 test suite. I also ran a
test of the random data with the Common Criteria test suite AIS 31, and it
passed all tests too.
There were a few changes that had broken this example, specifically
2cdf1d25f5 removed file.c from ports/unix.
And (at least for MacOS) mp_state_ctx must be placed in the BSS with
-fno-common so it is visible to the linker.
Signed-off-by: Santeri Paavolainen <santtu@iki.fi>
STM32L476RG MCU of NUCLEO_L476RG board has 6 UART/USART units in total
(USART1, USART2, USART3, UART4, UART5 and LPUART1), but only UART2,
connected to REPL, was defined and available in Python code.
Defined are all 5 remaining UART/USART units including LPUART1.
Signed-off-by: Alexander Ziubin aziubin@googlemail.com
This commit simplifies the customisation of the main MicroPython execution
loop (4 macros are reduced to 2), and allows a board to have full control
over the execution (or not) of boot.py and main.py.
For boards that use the default start-up code, there is no functional
change in this commit.
Signed-off-by: Damien George <damien@micropython.org>
Per CPython everything which comes after the command, module or file
argument is not an option for the interpreter itself. Hence the processing
of options should stop when encountering those, and the remainder be passed
as sys.argv. Note the latter was already the case for a module or file but
not for a command.
This fixes issues like 'micropython myfile.py -h' showing the help and
exiting instead of passing '-h' as sys.argv[1], likewise for
'-X <something>' being treated as a special option no matter where it
occurs on the command line.
Some forum users noticed that `sm.exec()` took longer the more was present
on the flash filesystem connected to the RP2040. They traced this back to
the `array` import inside `asm_pio()`, which is causing MicroPython to scan
the filesystem.
uarray is a built-in module, so importing it shouldn't require scanning the
filesystem.
We avoid moving the import to the top-level in order to keep the namespace
clean; we don't want to accidentally expose `rp2.array`.
The generated regex code is limited in the range of jumps and counts, and
this commit checks all cases which can overflow given the right kind of
input regex, and returns an error in such a case.
This change assumes that the results that overflow an int8_t do not
overflow a platform int.
Closes: #7078
Signed-off-by: Jeff Epler <jepler@gmail.com>
Support for C++ was added in 97960dc7de but
that commit didn't include the C++ exception handling table in the binary
firmware image. This commit fixes that.
Signed-off-by: Damien George <damien@micropython.org>
This is a workaround for errata RP2040-E5, and is needed to make USB more
reliable on certain USB ports.
Signed-off-by: Damien George <damien@micropython.org>
Make and CMake builds are slightly different and these changes help make it
clear what to do in each case.
Signed-off-by: Damien George <damien@micropython.org>
It's a bit of a pitfall with user C modules that including them in the
build does not automatically enable them. This commit changes the docs and
examples for user C modules to encourage writers of user C modules to
enable them unconditionally. This makes things simpler and covers most use
cases.
See discussion in issue #6960, and also #7086.
Signed-off-by: Damien George <damien@micropython.org>
It was noticed that the esp32 port didn't build ulab correctly. The
problem was a multiple defintion of the 'mp_hal_stdout_tx_str' and
'mp_hal_stdout_tx_strn_cooked' functions.
They were defined in stdout_helpers.c but also in the
ports/esp32/mphalport.c.
Fixed by removing stdout_helpers.c from the build.
Signed-off-by: Michael O'Cleirigh <michael.ocleirigh@rivulet.ca>
Support for User C and C++ modules was lost due to upgrading the esp32 to
the latest CMake based IDF from the GNUMakefile build process.
Restore the support for the esp32 port by integrating with the approach
recently added for the rp2 port.
Signed-off-by: Michael O'Cleirigh <michael.ocleirigh@rivulet.ca>
This USB feature is currently not supported. With this flag enabled (and
the feature not implemented) the USB serial will stop working if there is a
delay of more than about 2 seconds between messages, which can occur with
USB autosuspend enabled.
Fixes issue #6866.
Documents the micropython.cmake file required to make user C modules
compatible with the CMake build system.
Signed-off-by: Phil Howard <phil@pimoroni.com>
examples/usercmodule/micropython.cmake:
Root micropython.cmake file is responsible for including modules.
examples/usercmodule/cexample/micropython.cmake:
examples/usercmodule/cppexample/micropython.cmake:
Module micropython.cmake files define the target and link it to usermod.
Signed-off-by: Phil Howard <phil@pimoroni.com>
The parts that are generic are added to py/ so they can be used by other
ports that use CMake.
py/usermod.cmake:
* Creates a usermod target to hang user C/CXX modules from.
* Gathers sources from user C/CXX modules and libs for QSTR scan.
ports/rp2/CMakeLists.txt:
* Includes py/usermod.cmake.
* Links the resulting usermod library to the MicroPython target.
py/mkrules.cmake:
Add cxxflags to qstr.i.last custom command for CXX modules:
* MICROPY_CPP_FLAGS so CXX modules will find includes.
* -DNO_QSTR to fix fatal error missing "genhdr/qstrdefs.generated.h".
Usage:
The rp2 port can be linked against user C modules by running:
make USER_C_MODULES=/path/to/module/micropython.cmake
CMake will print a list of included modules.
Co-authored-by: Graham Sanderson <graham.sanderson@raspberrypi.org>
Co-authored-by: Michael O'Cleirigh <michael.ocleirigh@rivulet.ca>
Signed-off-by: Phil Howard <phil@pimoroni.com>
This commit simplifies and cleans up the bare-arm port, and adds just
enough system and library code to make it execute on an STM32F405 MCU.
The mpconfigport.h configuration is simplified to just specify those
configuration values that are different from the defaults. And the
addition of -fdata-sections and -ffunction-sections means the final
firmware is smaller than it previously was, by about 4200 bytes.
A README is also added.
Signed-off-by: Damien George <damien@micropython.org>
mp_printf should be used to print the prefix because it's also used in
mp_bytecode_print2 (otherwise, depending on the system, different output
streams may be used).
Also print the current thread state when threading is enabled to easily see
which thread executes what opcode.
Signed-off-by: Damien George <damien@micropython.org>
This helps to reduce memory fragmentation, by freeing the heap data as soon
as it is not needed. It also helps the compiler keeps a reference to the
beginning of both arrays, which need to be traceable by the GC (otherwise
some compilers may optimise this reference to something else).
Signed-off-by: Damien George <damien@micropython.org>
The GNU Make dir command uses spaces as item separator so it does not
work for e.g building the STM32 port on Cygwin with a default Arm
installation in "c:/program files (x86)/GNU Arm Embedded Toolchain".
Fix by using POSIX dirname on a quoted path instead.
Add most formatting-only commits to this file so that when used with
git blame, these commits are excluded and the output shows only the
interesting bits.
When UART is used for REPL and the MCU frequency is changed, the UART
has to be re-initialised. Besides that the UART may have to be recreated
after a frequency change, but with USB REPL this is not a problem.
Thanks to @HermannSW for spotting and providing the change.
Using the standard machine.freq().
The safe ranges tested were 10 and 12-270MHz, at which USB REPL still
worked. Requested settings can be checked with the script:
pico-sdk/src/rp2_common/hardware_clocks/scripts/vcocalc.py. At frequencies
like 300MHz the script still signaled OK, but USB did not work any more.
sm.get(buf) was waiting for one item more than the length of the supplied
buffer. Even if this item was not stored, sm_get would block trying to get
an item from the RX fifo.
As part of the fix, the edge case for a zero length buffer was moved up to
the section where the function arguments are handled. In case of a zero
length buffer, sm.get() now returns immediately that buffer.
The bitmasks supplied for initialization of out/set/sideset were only 8 bit
instead of 32. This resulted in an error, that not more than 8 consecutive
pins would get initialized.
Fixes issue #6933.
From a version numbering point of view this is a downgrade (2.17.0 ->
2.16.x). However the latest commit for version 2.17.0 is from March 2019
and no further minor release happened after 2.17.0. This version is EOL.
2.16.x though is still actively maintained as a long term release, hence
security and stability fixes are still being backported, including
compatibility with upcoming compiler releases.
This allows the user to enable wake-up sources using the EWUP bits, on F7
MCUs.
Disabling the wake-up sources while clearing the wake-up flags follows the
reference manual and ST examples.
state.reset_mode is updated by `MICROPY_BOARD_BEFORE_SOFT_RESET_LOOP` but
not passed to `init_flash_fs`, and so factory reset is not executed on
boards that do not have a bootloader. This bug was introduced by
4c3976bbcaFixes#6903.
A corrupt filesystem may lead to a request for a block which is out of
range of the block device limits. Return an error instead of passing the
request down to the lower layer.
Two of the defaults have also changed in this commit:
- MICROPY_HW_RFCORE_BLE_LSE_SOURCE changed from 1 to 0, which configures
the LsSource to be LSE (needed due to errata 2.2.1).
- MICROPY_HW_RFCORE_BLE_VITERBI_MODE changed from 0 to 1, which enables
Viterbi mode, following all the ST examples.
Signed-off-by: Damien George <damien@micropython.org>
Instead of raising a ZeroDivisionError, this tool now just skips any
elements in the DFU file that have zero size.
Signed-off-by: Damien George <damien@micropython.org>
These ports already have uzlib enabled so this additional ubinascii.crc32
function only costs about 90 bytes of flash.
Signed-off-by: Damien George <damien@micropython.org>
This allows a port to specify a custom qstrdefsport.h file, the same as the
QSTR_DEFS variable in a Makefile.
Signed-off-by: Damien George <damien@micropython.org>
Add LPUART1 as a standard UART. No low power features are supported, yet.
LPUART1 is enabled as the next available UART after the standard U(S)ARTs:
STM32WB: LPUART1 = UART(2)
STM32L0: LPUART1 = UART(6)
STM32L4: LPUART1 = UART(6)
STM32H7: LPUART1 = UART(9)
On all ports: LPUART1 = machine.UART('LP1')
LPUART1 is enabled by defining MICROPY_HW_LPUART1_TX and
MICROPY_HW_LPUART1_RX in mpconfigboard.h.
Signed-off-by: Chris Mason <c.mason@inchipdesign.com.au>
Previously, the MICROPY_PY_BLUETOOTH_ENABLE_CENTRAL_MODE macro
controlled enabling both the central mode and the GATT client
functionality (because usually the two go together).
This commits adds a new MICROPY_PY_BLUETOOTH_ENABLE_GATT_CLIENT
macro that separately enables the GATT client functionality.
This defaults to MICROPY_PY_BLUETOOTH_ENABLE_CENTRAL_MODE.
This also fixes a bug in the NimBLE bindings where a notification
or indication would not be received by a peripheral (acting as client)
as gap_event_cb wasn't handling it. Now both central_gap_event_cb
and peripheral_gap_event_cb share the same common handler for these
events.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This commit fixes two issues on the esp32:
- it enables machine.soft_reset() to be called in main.py;
- it enables machine.reset_cause() to correctly identify a soft reset.
The former is useful in that it enables soft resets in applications that
are started at boot time. The support is patterned after the stm32 port.
This commit implements basic NVS support for the esp32. It follows the
pattern of the esp32.Partition class and exposes an NVS object per NVS
namespace. The initial support provided is only for signed 32-bit integers
and binary blobs. It's easy (albeit a bit tedious) to add support for
more types.
See discussions in: #4436, #4707, #6780
This enables -Os for compilation, but still keeps full assertion messages.
With IDF v4.2, -Os changes the GENERIC firmware size from 1512176 down to
1384640, and the GENERIC_SPIRAM firmware is now 1452320 which fits in the
allocated partition.
Signed-off-by: Damien George <damien@micropython.org>
So that mboot can be used to program encrypted/signed firmware to regions
of flash that are not the main application, eg that are the filesystem.
Signed-off-by: Damien George <damien@micropython.org>
The default for these is to enable them, but they can now be disabled
individually by a board configuration.
Signed-off-by: Damien George <damien@micropython.org>
If a board defines USBD_VID then that will be used instead of the default.
And then the board must also define all USBD_PID_xxx values that it needs.
Signed-off-by: Damien George <damien@micropython.org>
Zephyr controllers can be queried for a static address (computed from the
device ID). BlueKitchen already supports this, but make them both use the
same macro to enable the feature.
The following simple usocket example throws an error EINVAL on connect
import usocket
s = usocket.socket()
s.connect(usocket.getaddrinfo('www.micropython.org', 80)[0][-1])
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
OSError: [Errno 22] EINVAL
Fixing the context parameter in calls of net_context_get_family() and
net_context_get_type(), the connect works fine.
Tested on a nucleo_h743zi board.
Updates the zephyr docker image to the latest, v0.11.13. This updates CI
to use zephyr SDK v0.12.2 and GCC v10.2.0 for the zephyr port.
Signed-off-by: Maureen Helm <maureen.helm@nxp.com>
Refactors the zephyr build infrastructure to build MicroPython as a
cmake target, using the recently introduced core cmake rules.
This change makes it possible to build the zephyr port like most other
zephyr applications using west or cmake directly. It simplifies building
with extra cmake arguments, such as specifying an alternate conf file or
adding an Arduino shield. It also enables building the zephyr port
anywhere in the host file system, which will allow regressing across
multiple boards with the zephyr twister script.
Signed-off-by: Maureen Helm <maureen.helm@nxp.com>
Disables frozen source modules in the zephyr port. They are deprecated
in the makefile rules and not implemented in the new cmake rules.
Signed-off-by: Maureen Helm <maureen.helm@nxp.com>
The core cmake rules use custom commands to invoke qstr processing
scripts. For the zephyr port, it's possible that list arguments to these
commands may contain generator expressions, therefore we need to expand
them properly.
Signed-off-by: Maureen Helm <maureen.helm@nxp.com>
This allows customising which features can be enabled in a frozen library.
e.g. `include("path.py", extra_features=True)`
in path.py:
options.defaults(standard_features=True)
if options.standard_features:
# freeze standard modules.
if options.extra_features:
# freeze extra modules.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This is a MicroPython-extension that allows for code running in IRQ
(hard or soft) or scheduler context to sequence asyncio code.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
On error, the handle is only available on err->att_handle rather than
in attr->handle used in the non-error case.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
The underlying OS (the ESP-IDF) uses it's own internal errno codes and so
it's simpler and cleaner to use those rather than trying to convert
everything to the values defined in py/mperrno.h.
It's now replaced by cmake/idf.py. But a convenience Makefile is still
provided with traditional targets like "all" and "deploy".
Signed-off-by: Damien George <damien@micropython.org>
This commit adds support for building the esp32 port with cmake, and in
particular it builds MicroPython as a component within the ESP-IDF. Using
cmake and the ESP-IDF build infrastructure makes it much easier to maintain
the port, especially with the various new ESP32 MCUs and their required
toolchains.
Signed-off-by: Damien George <damien@micropython.org>
This allows changing the baudrate of the UART without reinitialising it
(reinitialising can lead to spurious characters sent on the TX line).
Signed-off-by: Damien George <damien@micropython.org>
On i.MX the SysTick IRQ cannot wake the CPU from a WFI so the CPU was
blocked on WFI waiting for USB data in mp_hal_stdin_rx_chr() even though it
had already arrived (because it may arrive just after calling the check
tud_cdc_available()). This commit fixes this problem by using SEV/WFE to
indicate that there has been a USB event.
The mp_hal_stdout_tx_strn() function is also fixed so that it doesn't
overflow the USB buffers.
Signed-off-by: Damien George <damien@micropython.org>
So that all MicroPython ports that use tinyusb use the same version. Also
requires fewer submodule checkouts when building rp2 along with other ports
that use tinyusb.
Signed-off-by: Damien George <damien@micropython.org>
START_SEC was changed in e0905e85a7.
Also, update the error message to mention how to format the partition at
the REPL, and make the total message shorter to save a bit of flash.
Signed-off-by: Damien George <damien@micropython.org>
For certain operands to mpn_div, the existing code path for
`DIG_SIZE == MPZ_DBL_DIG_SIZE / 2` had a bug in it where borrow could still
overflow in the `(x >= *n || *n - x <= borrow)` branch, ie
`borrow + x - (mpz_dbl_dig_t)*n` overflows the borrow variable. In such
cases the subsequent right-shift of borrow would not bring in the overflow
bit, leading to an error in the result. An example division that had
overflow when MPZ_DIG_SIZE = 16 is `(2 ** 48 - 1) ** 2 // (2 ** 48 - 1)`.
This is fixed in this commit by simplifying the code and handling the low
digits of borrow first, and then the upper bits (to shift down) separately.
There is no longer a distinction between `DIG_SIZE < MPZ_DBL_DIG_SIZE / 2`
and `DIG_SIZE == MPZ_DBL_DIG_SIZE / 2`.
This commit also simplifies the second part of the calculation so that
borrow does not need to be negated (instead the code just works knowing
that borrow is negative and using + instead of - in calculations involving
borrow).
Fixes#6777.
Signed-off-by: Damien George <damien@micropython.org>
In particular the firmware can now be built in a build directory that lives
outside the source tree, and the py/modarray.c file will still be found.
See issue #6837.
Signed-off-by: Damien George <damien@micropython.org>
The "word" referred to by BYTES_PER_WORD is actually the size of mp_obj_t
which is not always the same as the size of a pointer on the target
architecture. So rename this config value to better reflect what it
measures, and also prefix it with MP_.
For uses of BYTES_PER_WORD in setting the stack limit this has been
changed to sizeof(void *), because the stack usually grows with
machine-word sized values (eg an nlr_buf_t has many machine words in it).
Signed-off-by: Damien George <damien@micropython.org>
It's only used in one location, to test if << or >> will overflow when
shifting mp_uint_t. For such a test it's clearer to use sizeof(lhs_val),
which will be valid even if the type of lhs_val changes.
Signed-off-by: Damien George <damien@micropython.org>
To simplify config, there's no need to specify MP_PLAT_PRINT_STRN if it's
the same as the default definition in py/mpconfig.h.
Signed-off-by: Damien George <damien@micropython.org>
Otherwise it resets the ADC peripheral each time a new ADC object is
constructed, which can reset other state that has already been set up.
See issue #6833.
Signed-off-by: Damien George <damien@micropython.org>
Default to just calling python since that is most commonly available: the
official installer or zipfiles from python.org, anaconda, nupkg all result
in python being available but not python3. In other words: the default
used so far is wrong. Note that os.name is 'posix' when running the python
version which comes with Cygwin or MSys2 so they are not affected by this.
However of all possible ways to get Python on Windows, only Cygwin provides
no python command so update the default way for running tests in the
README.
With mboot encrpytion and fsload enabled, the DEBUG build -O0 compiler
settings result in mboot no longer fitting in the 32k sector. This commit
changes this to -Og which also brings it into line with the regular stm32
build.
MCUs with device-only USB peripherals (eg L0, WB) do not implement (at
least not in the ST HAL) the HAL_PCD_DisconnectCallback event. So if a USB
cable is disconnected the USB driver does not deinitialise itself
(usbd_cdc_deinit is not called) and the CDC driver can stay in the
USBD_CDC_CONNECT_STATE_CONNECTED state. Then if the USB was attached to
the REPL, output can become very slow waiting in usbd_cdc_tx_always for
500ms for each character.
The disconnect event is not implemented on these MCUs but the suspend event
is. And in the situation where the USB cable is disconnected the suspend
event is raised because SOF packets are no longer received.
The issue of very slow output on these MCUs is fixed in this commit (really
worked around) by adding a check in usbd_cdc_tx_always to see if the USB
device state is suspended, and, if so, breaking out of the 500ms wait loop.
This should also help all MCUs for a real USB suspend.
A proper fix for MCUs with device-only USB would be to implement or somehow
synthesise the HAL_PCD_DisconnectCallback event.
See issue #6672.
Signed-off-by: Damien George <damien@micropython.org>
PIO state machines can make a conditional jump on the state of a pin: the
`JMP PIN` command. This requires the pin to be configured with
`sm_config_set_jmp_pin`, but until now we didn't have a way of doing that
in MicroPython.
This commit adds a new `jmp_pin=None` argument to `StateMachine`. If it is
not `None` then we try to interpret it as a Pin, and pass its value to
`sm_config_set_jmp_pin`.
Signed-off-by: Tim Radvan <tim@tjvr.org>
Some devices have lower precision than 1ms for time_ns() (eg PYBv1.x has
3.9ms resolution of the RTC) so make the test more lenient for them.
Signed-off-by: Damien George <damien@micropython.org>
With a check for reproducible build date. Invocation of the test suite is
not needed because it's already run in another job.
Signed-off-by: iTitou <moiandme@gmail.com>
This environment variable, if defined during the build process,
indicates a fixed time that should be used in place of "now" when
such a time is explicitely referenced.
This allows for reproducible builds of micropython.
See https://reproducible-builds.org/specs/source-date-epoch/
Signed-off-by: iTitou <moiandme@gmail.com>
This should be enabled when the mp_raw_code_save_file function is needed.
It is enabled for mpy-cross, and a check for defined(__APPLE__) is added to
cover Mac M1 systems.
The upip module is frozen into ports supporting it, and it is included in
the source tree, so there is no need to get it from PyPi. Moreover the
PyPi package referred to is an out-of-date version of upip which is
basically unrelated to our upip.py because the source is taken from a fork
of micropython-lib instead of this repository.
Add "make submodules" to commands when building for the first time.
Otherwise, on a first time build, the submodules have not been checked out
and a lot of `fatal error: nrfx.h: No such file or directory` errors are
printed.
The main rules enforced are:
- At most 72 characters in the subject line, with a ": " in it.
- At most 75 characters per line in the body.
- No "noreply" email addresses.
This was added a long time ago in 75abee206d
when USB host support was added to the stm (now stm32) port, and when this
pyexec code was actually part of the stm port. It's unlikely to work as
intended anymore. If it is needed in the future then generic hook macros
can be added in pyexec.
It practically does the same as qstr_from_str and was only used in one
place, which should actually use the compile-time MP_QSTR_XXX form for
consistency; qstr_from_str is for runtime strings only.
If the _IRQ_L2CAP_RECV handler does the actual consumption of the incoming
data (i.e. via l2cap_recvinto), rather than setting a flag for
non-scheduler-context to handle it later, then two things can happen:
- It can starve the VM (i.e. the scheduled task never terminates). This is
because calling l2cap_recvinto will empty the rx buffer, which will grant
more credits to the channel (an HCI command), meaning more data can
arrive. This means that the loop in hal_uart.c that keeps reading HCI
data from the uart and executing NimBLE events as they are created will
not terminate, preventing other VM code from running.
- There's no flow control (i.e. data will arrive too quickly). The channel
shouldn't be given credits until after we return from scheduler context.
It's preferable that no work is done in scheduler/IRQ context. But to
prevent this being a problem this commit changes l2cap_recvinto so that if
it is called in IRQ context, and the Python handler empties the rx buffer,
then don't grant credits until the Python handler is complete.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Don't clear the IPCC channel flag until we've actually handled the incoming
data, or else the wireless firmware may clobber the IPCC buffer if more
data arrives. This requires masking the IRQ until the data is handled.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This commit adds a new port "rp2" which targets the new Raspberry Pi RP2040
microcontroller.
The build system uses pure cmake (with a small Makefile wrapper for
convenience). The USB driver is TinyUSB, and there is a machine module
with most of the standard classes implemented. Some examples are provided
in the examples/rp2/ directory.
Work done in collaboration with Graham Sanderson.
Signed-off-by: Damien George <damien@micropython.org>
These args are already bounds checked and clipped, and using unsigned ints
can be more efficient. It also eliminates possible issues and compiler
warnings with shifting of signed integers.
Signed-off-by: Damien George <damien@micropython.org>
Adds a new compile-time option MICROPY_EMIT_THUMB_ARMV7M which is enabled
by default (to get existing behaviour) and which should be disabled (set to
0) when building native emitter support (@micropython.native) on ARMv6M
targets.
This returns a reference to the globals dict associated with the function,
ie the global scope that the function was defined in. This attribute is
read-only but the dict itself is modifiable, per CPython behaviour.
Signed-off-by: Damien George <damien@micropython.org>
It's enabled by default to retain the existing behaviour. A board can
disable this option if it manages mounting the filesystem itself, for
example in frozen code.
Signed-off-by: Damien George <damien@micropython.org>
Changes are:
- refactor to use new _create_element function
- support extended version of MOUNT element with block size
- support STATUS element
Signed-off-by: Damien George <damien@micropython.org>
This new element takes the form: (ELEM_TYPE_STATUS, 4, <address>). If this
element is present in the mboot command then mboot will store to the given
address the result of the filesystem firmware update process. The address
can for example be an RTC backup register.
Signed-off-by: Damien George <damien@micropython.org>
Instead it is now passed in as an optional parameter to the ELEM_MOUNT
element, with a compile-time configurable default.
Signed-off-by: Damien George <damien@micropython.org>
The superblock for littlefs is in block 0 and 1, but block 0 may be erased
or partially written, so block 1 must be checked if block 0 does not have a
valid littlefs superblock in it.
Prior to this commit, if block 0 did not contain a valid littlefs
superblock (but block 1 did) then the auto-detection would fail, mounting a
FAT filesystem would also fail, and the system would reformat the flash,
even though it may have contained a valid littlefs filesystem. This is now
fixed.
Signed-off-by: Damien George <damien@micropython.org>
The superblock for littlefs is in block 0 and 1, but block 0 may be erased
or partially written, so block 1 must be checked if block 0 does not have a
valid littlefs superblock in it.
Prior to this commit, the mount of a block device which auto-detected the
filysystem type would fail for littlefs if block 0 did not contain a valid
superblock. That is now fixed.
Signed-off-by: Damien George <damien@micropython.org>
This commit adds many new sections to the existing "Developing and building
MicroPython" chapter to make it all about the internals of MicroPython.
This work was done as part of Google's Season of Docs 2020.
Changes are:
- Use ubuntu-20.04 so that gcc-multilib installs without error.
- Use "fetch-depth: 100" to get history prior to pull request.
Signed-off-by: Damien George <damien@micropython.org>
* Fix a typo in the Makefile that prevented the debug build to be actually
enabled when BTYPE=debug is used.
* Add a missing header in modmachine.c that is used when a debug build is
created.
This commit improves some FTP implementation details for better
compatibility with FTP clients:
* The PWD command now puts quotes around the directory name before
returning it. This fixes BBEdit’s FTP client, which performs a PWD after
each CWD and gets confused if the returned directory path is not
surrounded by quotes.
* The FEAT command is now allowed before logging in. This fixes the lftp
client, which send FEAT first and gets confused (tries to use TLS) if the
server responds with 332.
With MICROPY_FLOAT_IMPL_FLOAT the results of utime.time(), gmtime() and
localtime() change only every 129 seconds. As one consequence
tests/extmod/vfs_lfs_mtime.py will fail on a unix port with LFS support.
With this patch these functions only return floats if
MICROPY_FLOAT_IMPL_DOUBLE is used. Otherwise they return integers.
According to documentation time() has a precision of at least 1 second.
This test runs for 2.5 seconds and calls all utime functions every 100ms.
Then it checks if they returned enough different results. All functions
with sub-second precision will return ~25 results. This test passes with
15 results or more. Functions that do not exist are skipped silently.
This allows sending arbitrary HCI commands and getting the response. The
return value of the function is the status of the command.
This is intended for debugging and not to be a part of the public API, and
must be enabled via mpconfigboard.h. It's currently only implemented for
NimBLE bindings.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
To match the definition of GENERATE_PACK_DFU, so a board can customise the
location/name of this file if needed.
Signed-off-by: Damien George <damien@micropython.org>
To have at least one board configured with MBOOT_ENABLE_PACKING, for CI
testing purposes and demonstration of the feature.
Signed-off-by: Damien George <damien@micropython.org>
This commit adds support to stm32's mboot for signe, encrypted and
compressed DFU updates. It is based on inital work done by Andrew Leech.
The feature is enabled by setting MBOOT_ENABLE_PACKING to 1 in the board's
mpconfigboard.mk file, and by providing a header file in the board folder
(usually called mboot_keys.h) with a set of signing and encryption keys
(which can be generated by mboot_pack_dfu.py). The signing and encryption
is provided by libhydrogen. Compression is provided by uzlib. Enabling
packing costs about 3k of flash.
The included mboot_pack_dfu.py script converts a .dfu file to a .pack.dfu
file which can be subsequently deployed to a board with mboot in packing
mode. This .pack.dfu file is created as follows:
- the firmware from the original .dfu is split into chunks (so the
decryption can fit in RAM)
- each chunk is compressed, encrypted, a header added, then signed
- a special final chunk is added with a signature of the entire firmware
- all chunks are concatenated to make the final .pack.dfu file
The .pack.dfu file can be deployed over USB or from the internal filesystem
on the device (if MBOOT_FSLOAD is enabled).
See #5267 and #5309 for additional discussion.
Signed-off-by: Damien George <damien@micropython.org>
Prior to this fix, the final piece of data in a compressed file may have
been lost when decompressing.
Signed-off-by: Damien George <damien@micropython.org>
This library is a small and easy-to-use cryptographic library which is well
suited to embedded systems.
Signed-off-by: Damien George <damien@micropython.org>
The original logic of reducing a full path to a relative one assumes
"tests/misc" is in the filename which is limited in usage: it never works
for CPython on Windows since that will use a backslash as path separator,
and also won't work when the filename is a path not relative to the tests
directory which happens for example in the common case of running
"./run-tests -d misc".
Fix all cases by printing only the bare filename, which requires them all
to start with sys_settrace_ hence the renaming.
Changes are:
- Remove include of stm32's adc.h because it was recently changed and is
no longer compatible with teensy (and not used anyway).
- Remove define of __disable_irq in mpconfigport.h because it was clashing
with an equivalent definition in core/mk20dx128.h.
- Add -Werror to CFLAGS, and change -std=gnu99 to -std=c99.
Signed-off-by: Damien George <damien@micropython.org>
Mboot builds do not use the external SPI flash in caching mode, and
explicitly disabling it saves RAM and a small bit of flash.
Signed-off-by: Damien George <damien@micropython.org>
This only needs to be enabled if a board uses FAT FS on external SPI flash.
When disabled (and using external SPI flash) 4k of RAM can be saved.
Signed-off-by: Damien George <damien@micropython.org>
When littlefs is enabled extended reading must be supported, and using this
function to read the first block for auto-detection is more efficient (a
smaller read) and does not require a cached SPI-flash read.
Signed-off-by: Damien George <damien@micropython.org>
These functions enable SDRAM data retention in stop mode. Example usage,
in mpconfigboard.h:
#define MICROPY_BOARD_ENTER_STOP sdram_enter_low_power();
#define MICROPY_BOARD_LEAVE_STOP sdram_leave_low_power();
Calculate the bit timing from baudrate if provided, allowing sample point
override. This makes it a lot easier to make CAN work between different
MCUs with different clocks, prescalers etc.
Tested on F4, F7 and H7 Y/V variants.
This commit prevents uos.mount() from raising an AttributeError.
vfs_autodetect() is supposed to return an object that has a "mount" method,
so if no filesystem is found it should raise an OSError(ENODEV) and not
return the bdev itself which has no "mount" method.
Mounting a bdev directly tries to auto-detect the filesystem and if none is
found an OSError(19,) should be raised.
The fourth parameter of readblocks() and writeblocks() must be optional to
support ports with MICROPY_VFS_FAT=1. Otherwise mounting a bdev may fail
because looking for a FATFS will call readblocks() with only 3 parameters.
Since CPython 3.8 the optional "sep" argument to hexlify is officially
supported, so update comments in the code and the docs to reflect this.
Signed-off-by: Damien George <damien@micropython.org>
As a general pattern, required positional arguments that are not named do
not need to be parsed using mp_arg_parse_all().
Signed-off-by: Damien George <damien@micropython.org>
Because the version included in xtensa-lx106-elf-standalone.tar.gz needs
Python 2 (and pyserial for Python 2).
Signed-off-by: Damien George <damien@micropython.org>
This much buffer space is required for CDC data out endpoints to avoid any
buffer overflows when the USB CDC is saturated with data.
Signed-off-by: Damien George <damien@micropython.org>
The nrf52840-mdk-usb-dongle and pca10050 comes with a pre-flashed
bootloader (OpenBootloader).
This commit updates the boards "mpconfigboard.mk" to use DFU as
default flashing method and set the corresponding BOOTLOADER
settings such that nrf52840_open_bootloader_1.2.x.ld linker
script is used.
The default DFU flashing method can be disabled by issuing "DFU=0"
when invoking make. This will lead to "segger" being used as default
flashing tool. When using "DFU=0", the linker scripts will not
compensate for any MBR and Bootloader region being present, and might
overwrite them if they were present.
The commit also removes the custom linker script specific to
nrf52840-mdk-usb-dongle as it now points to a generic.
Updated nrf52840-mdk-usb-dongle's README.md to be more clear on
how to deploy the built firmware.
The port README.md has also been updated. In the list of target
boards a new column has been added to indicate which bootloader
is present on the target board. And for consistency, changed all
examples in the README.md to use "deploy" instead of "flash".
An additional Makefile parameter NRFUTIL_PORT can be set in order
to define the serial port to used for the DFU (Default: /dev/ttyACM0).
The "nrfutil" that is used as flasher towards OpenBootloader is
available for installation through Python "pip".
In case of SD=s140, SoftDevice ID 0xB6 is passed to nrfutil's package
generation which corresponds to SoftDevice s140 v6.1.1.
Add the option for "mpconfigboard.mk" to define whether the
board hosts a bootloader or not. The BOOTLOADER make variable
must be set to the name of the bootloader.
When the BOOTLOADER name is set it is also required to supply
the BOOTLOADER_VERSION_MAJOR and the BOOTLOADER_VERSION_MINOR
from the "mpconfigboards.mk". These will be used to resolve which
bootloader linker script that should be passed to the linker.
The BOOTLOADER section also supplies the C-compiler with
BOOTLOADER_<bootloader name>=<version major><version minor>
as a compiler define. This is for future use in case a bootloader
needs to do modification to the startup files or similar (like
setting the VTOR specific to a version of a bootloader).
Adding variables that can be set from other linker scripts:
- _bootloader_head_size:
Bootloader flash offset in front of the application.
- _bootloader_tail_size:
Bootloader offset from the tail of the flash.
In case the bootloader is located at the end.
- _bootloader_head_ram_size:
Bootloader RAM usage in front of the application.
Updated calculations of application flash and RAM.
Two issues are tackled:
1. The calculation of the correct length to print is fixed to treat the
precision as a maximum length instead as the exact length.
This is done for both qstr (%q) and for regular str (%s).
2. Fix the incorrect use of mp_printf("%.*s") to mp_print_strn().
Because of the fix of above issue, some testcases that would print
an embedded null-byte (^@ in test-output) would now fail.
The bug here is that "%s" was used to print null-bytes. Instead,
mp_print_strn is used to make sure all bytes are outputted and the
exact length is respected.
Test-cases are added for both %s and %q with a combination of precision
and padding specifiers.
The zephyr function net_shell_cmd_iface() was removed in zephyr v1.14.0,
therefore the MicroPython zephyr port did not build with newer zephyr
versions when CONFIG_NET_SHELL=y. Replace with a more general
shell_exec() function that can execute any zephyr shell command. For
example:
>>> zephyr.shell_exec("net")
Subcommands:
allocs :Print network memory allocations.
arp :Print information about IPv4 ARP cache.
conn :Print information about network connections.
dns :Show how DNS is configured.
events :Monitor network management events.
gptp :Print information about gPTP support.
iface :Print information about network interfaces.
ipv6 :Print information about IPv6 specific information and
configuration.
mem :Print information about network memory usage.
nbr :Print neighbor information.
ping :Ping a network host.
pkt :net_pkt information.
ppp :PPP information.
resume :Resume a network interface
route :Show network route.
stacks :Show network stacks information.
stats :Show network statistics.
suspend :Suspend a network interface
tcp :Connect/send/close TCP connection.
vlan :Show VLAN information.
websocket :Print information about WebSocket connections.
>>> zephyr.shell_exec("kernel")
kernel - Kernel commands
Subcommands:
cycles :Kernel cycles.
reboot :Reboot.
stacks :List threads stack usage.
threads :List kernel threads.
uptime :Kernel uptime.
version :Kernel version.
Signed-off-by: Maureen Helm <maureen.helm@nxp.com>
The -Og optimisation level produces a more realistic build, gives a better
debugging experience, and generates smaller code than -O0, allowing debug
builds to fit in flash.
This commit also assigns variables in can.c to prevent warnings when -Og is
used, and builds a board in CI with DEBUG=1 enabled.
Signed-off-by: Damien George <damien@micropython.org>
Allows reserving CAN, I2C, SPI, Timer and UART peripherals. If reserved
the peripheral cannot be accessed from Python.
Signed-off-by: Damien George <damien@micropython.org>
Even though IRQs are disabled this seems to be required on H7 Rev Y,
otherwise Systick interrupt triggers and the MCU leaves the stop mode
immediately.
This commit saves OSCs/PLLs state before STOP mode and restores them on
exit. Some boards use HSI48 for USB for example, others have PLL2/3
enabled, etc.
Rather than dealing with the different int types, just pass them all as a
single array of mp_int_t with n_unsigned (before addr) and n_signed (after
addr).
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This adds `_IRQ_GET_SECRET` and `_IRQ_SET_SECRET` events to allow the BT
stack to request the Python code retrive/store/delete secret key data. The
actual keys and values are opaque to Python and stack-specific.
Only NimBLE is implemented (pending moving btstack to sync events). The
secret store is designed to be compatible with BlueKitchen's TLV store API.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This allows the application to be notified if any of encrypted,
authenticated and bonded state change, as well as the encryption key size.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Enable it on STM32/Unix NimBLE only (pairing/bonding requires synchronous
events and full bindings).
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Instead of returning None/bool from the IRQ, return None/int (where a zero
value means success). This mirrors how the L2CAP_ACCEPT return value
works.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This widens the characteristic/descriptor flags to 16-bit, to allow setting
encryption/authentication requirements.
Sets the required flags for NimBLE and btstack implementations.
The BLE.FLAG_* constants will eventually be deprecated in favour of copy
and paste Python constants (like the IRQs).
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This allows the application to be notified of changes to the connection
interval, connection latency and supervision timeout.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This commit switches the roles of the helper task from a cancellation task
to a runner task, to get the correct semantics for cancellation of
wait_for.
Some uasyncio tests are now disabled for the native emitter due to issues
with native code generation of generators and yield-from.
Fixes#5797.
Signed-off-by: Damien George <damien@micropython.org>
This is added because task.coro==None is no longer the way to detect if a
task is finished. Providing a (CPython compatible) function for this
allows the implementation to be abstracted away.
Signed-off-by: Damien George <damien@micropython.org>
When a tasks raises an exception which is uncaught, and no other task
await's on that task, then an error message is printed (or a user function
called) via a call to Loop.call_exception_handler. In CPython this call is
made when the Task object is freed (eg via reference counting) because it's
at that point that it is known that the exception that was raised will
never be handled.
MicroPython does not have reference counting and the current behaviour is
to deal with uncaught exceptions as early as possible, ie as soon as they
terminate the task. But this can be undesirable because in certain cases
a task can start and raise an exception immediately (before any await is
executed in that task's coro) and before any other task gets a chance to
await on it to catch the exception.
This commit changes the behaviour so that tasks which end due to an
uncaught exception are scheduled one more time for execution, and if they
are not await'ed on by the next scheduling loop, then the exception handler
is called (eg the exception is printed out).
Signed-off-by: Damien George <damien@micropython.org>
This commit adds support to pyboard.py for the new raw REPL paste mode.
Note that this new pyboard.py is fully backwards compatible with old
devices (it detects if the device supports the new raw REPL paste mode).
Signed-off-by: Damien George <damien@micropython.org>
Background: the friendly/normal REPL is intended for human use whereas the
raw REPL is for computer use/automation. Raw REPL is used for things like
pyboard.py script_to_run.py. The normal REPL has built-in flow control
because it echos back the characters. That's not so with raw REPL and flow
control is just implemented by rate limiting the amount of data that goes
in. Currently it's fixed at 256 byte chunks every 10ms. This is sometimes
too fast for slow MCUs or systems with small stdin buffers. It's also too
slow for a lot of higher-end MCUs, ie it could be a lot faster.
This commit adds a new raw REPL mode which includes flow control: the
device will echo back a character after a certain number of bytes are sent
to the host, and the host can use this to regulate the data going out to
the device. The amount of characters is controlled by the device and sent
to the host before communication starts. This flow control allows getting
the maximum speed out of a serial link, regardless of the link or the
device at the other end.
Also, this new raw REPL mode parses and compiles the incoming data as it
comes in. It does this by creating a "stdin reader" object which is then
passed to the lexer. The lexer requests bytes from this "stdin reader"
which retrieves bytes from the host, and does flow control. What this
means is that no memory is used to store the script (in the existing raw
REPL mode the device needs a big buffer to read in the script before it can
pass it on to the lexer/parser/compiler). The only memory needed on the
device is enough to parse and compile.
Finally, it would be possible to extend this new raw REPL to allow bytecode
(.mpy files) to be sent as well as text mode scripts (but that's not done
in this commit).
Some results follow. The test was to send a large 33k script that contains
mostly comments and then prints out the heap, run via pyboard.py large.py.
On PYBD-SF6, prior to this PR:
$ ./pyboard.py large.py
stack: 524 out of 23552
GC: total: 392192, used: 34464, free: 357728
No. of 1-blocks: 12, 2-blocks: 2, max blk sz: 2075, max free sz: 22345
GC memory layout; from 2001a3f0:
00000: h=hhhh=======================================hhBShShh==h=======h
00400: =====hh=B........h==h===========================================
00800: ================================================================
00c00: ================================================================
01000: ================================================================
01400: ================================================================
01800: ================================================================
01c00: ================================================================
02000: ================================================================
02400: ================================================================
02800: ================================================================
02c00: ================================================================
03000: ================================================================
03400: ================================================================
03800: ================================================================
03c00: ================================================================
04000: ================================================================
04400: ================================================================
04800: ================================================================
04c00: ================================================================
05000: ================================================================
05400: ================================================================
05800: ================================================================
05c00: ================================================================
06000: ================================================================
06400: ================================================================
06800: ================================================================
06c00: ================================================================
07000: ================================================================
07400: ================================================================
07800: ================================================================
07c00: ================================================================
08000: ================================================================
08400: ===============================================.....h==.........
(349 lines all free)
(the big blob of used memory is the large script).
Same but with this PR:
$ ./pyboard.py large.py
stack: 524 out of 23552
GC: total: 392192, used: 1296, free: 390896
No. of 1-blocks: 12, 2-blocks: 3, max blk sz: 40, max free sz: 24420
GC memory layout; from 2001a3f0:
00000: h=hhhh=======================================hhBShShh==h=======h
00400: =====hh=h=B......h==.....h==....................................
(381 lines all free)
The only thing in RAM is the compiled script (and some other unrelated
items).
Time to download before this PR: 1438ms, data rate: 230,799 bits/sec.
Time to download with this PR: 119ms, data rate: 2,788,991 bits/sec.
So it's more than 10 times faster, and uses significantly less RAM.
Results are similar on other boards. On an stm32 board that connects via
UART only at 115200 baud, the data rate goes from 80kbit/sec to
113kbit/sec, so gets close to saturating the UART link without loss of
data.
The new raw REPL mode also supports a single ctrl-C to break out of this
flow-control mode, so that a ctrl-C can always get back to a known state.
It's also backwards compatible with the original raw REPL mode, which is
still supported with the same sequence of commands. The new raw REPL
mode is activated by ctrl-E, which gives an error on devices that do not
support the new mode.
Signed-off-by: Damien George <damien@micropython.org>
Travis now limits the amount of free minutes for open-source projects, and
it does not provide enough for this project. So stop using it and instead
use on GitHub Actions.
Signed-off-by: Damien George <damien@micropython.org>
Also known as L2CAP "connection oriented channels". This provides a
socket-like data transfer mechanism for BLE.
Currently only implemented for NimBLE on STM32 / Unix.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Hardware I2C implementations must provide a .init() protocol method if they
want to support reconfiguration. Otherwise the default is that i2c.init()
raises an OSError (currently the case for all ports).
mp_machine_soft_i2c_locals_dict is renamed to mp_machine_i2c_locals_dict to
match the generic SPI bindings.
Fixes issue #6623 (where calling .init() on a HW I2C would crash).
Signed-off-by: Damien George <damien@micropython.org>
This fixes the build for non-STM32WB based boards when the NimBLE submodule
has not been fetched, and also allows STM32WB boards to build with BLE
disabled.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This is needed to moderate concurrent access to the internal flash, as
while an erase/write is in progress execution will stall on the wireless
core due to the bus being locked.
This implements Figure 10 from AN5289 Rev 3.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This commit switches the STM32WB HCI interface (between the two CPUs) to
require the use of MICROPY_PY_BLUETOOTH_USE_SYNC_EVENTS, and as a
consequence to require NimBLE. IPCC RX IRQs now schedule the NimBLE
handler to run via mp_sched_schedule.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This test currently passes on Unix/PYBD, but fails on WB55 because it lacks
synchronisation of the internal flash.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This changes stm32 from using PENDSV to run NimBLE to use the MicroPython
scheduler instead. This allows Python BLE callbacks to be invoked directly
(and therefore synchronously) rather than via the ringbuffer.
The NimBLE UART HCI and event processing now happens in a scheduled task
every 128ms. When RX IRQ idle events arrive, it will also schedule this
task to improve latency.
There is a similar change for the unix port where the background thread now
queues the scheduled task.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This requires that the event handlers are called from non-interrupt context
(i.e. the MicroPython scheduler).
This will allow the BLE stack (e.g. NimBLE) to run from the scheduler
rather than an IRQ like PENDSV, and therefore be able to invoke Python
callbacks directly/synchronously. This allows writing Python BLE handlers
for events that require immediate response such as _IRQ_READ_REQUEST (which
was previous a hard IRQ) and future events relating to pairing/bonding.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Using a semaphore (the previous approach) will only run the UART, whereas
for startup we need to also run the event queue.
This change makes it run the full scheduler hook.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Instead of having the stack indicate a "start", "data"..., "end", pass
through the data in one callback as an array of chunks of data.
This is because the upcoming non-ringbuffer modbluetooth implementation
cannot buffer the data in the ringbuffer and requires instead a single
callback with all the data, to pass to the Python callback.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Use the same `wait_for_event` in all tests that doesn't hold a reference to
the event data tuple and handles repeat events.
Also fix a few misc reliability issues around timeouts and sequencing.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Devices with RTC backup-batteries have been shown (very rarely) to have
incorrect RTC prescaler values. Such incorrect values mean the RTC counts
fast or slow, and will be wrong forever if the power/backup-battery is
always present.
This commit detects such a state at start up (hard reset) and corrects it
by reconfiguring the RTC prescaler values.
Signed-off-by: Damien George <damien@micropython.org>
And rename SRC_HAL -> HAL_SRC_C and SRC_USBDEV -> USBDEV_SRC_C for
consistency with other source variables.
Follow on from 0fff2e03fe
Signed-off-by: Damien George <damien@micropython.org>
Prior to this change machine.mem32['foo'] (or using any other non-integer
subscript) could result in a fault due to 'foo' being interpreted as an
integer. And when writing code it's hard to tell if the fault is due to a
bad subscript type, or an integer subscript that specifies an invalid
memory address.
The type of the object used in the subscript is now tested to be an
integer by using mp_obj_get_int_truncated instead of
mp_obj_int_get_truncated. The performance hit of this change is minimal,
and machine.memX objects are more for convenience than performance (there
are many other ways to read/write memory in a faster way),
Fixes issue #6588.
The file `$(BUILD)/firmware.bin` was used by the target `deploy-stlink` and
`deploy-openocd` but it was generated indirectly by the target
`firmware.dfu`.
As this file could be used to program boards directly by a Mass Storage
copy, it's better to make it explicitly generated.
Additionally, some target are refactored to remove redundancy and be more
explicit on dependencies.
This gives a substantial speedup of the preprocessing step, i.e. the
generation of qstr.i.last. For example on a clean build, making
qstr.i.last:
21s -> 4s on STM32 (WB55)
8.9 -> 1.8s on Unix (dev).
Done in collaboration with @stinos.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Running the update inside the soft-reset loop will mean that (on boards
like PYBD that use a bootloader) the same reset mode is used each
reset loop, eg factory reset occurs each time.
Signed-off-by: Damien George <damien@micropython.org>
Add working example code to provide a starting point for users with files
that they can just copy, and include the modules in the coverage test to
verify the complete user C module build functionality. The cexample module
uses the code originally found in cmodules.rst, which has been updated to
reflect this and partially rewritten with more complete information.
Support building .cpp files and linking them into the micropython
executable in a way similar to how it is done for .c files. The main
incentive here is to enable user C modules to use C++ files (which are put
in SRC_MOD_CXX by py.mk) since the core itself does not utilize C++.
However, to verify build functionality a unix overage test is added. The
esp32 port already has CXXFLAGS so just add the user modules' flags to it.
For the unix port use a copy of the CFLAGS but strip the ones which are not
usable for C++.
Support C++ code in .cpp files by providing CXX counterparts of the
_USERMOD_ flags we have for C already. This merely enables the Makefile of
user C modules to use variables specific to C++ compilation, it is still up
to each port's main Makefile to also include these in the build.
When SCR_QSTR contains C++ files they should be preprocessed with the same
compiler flags (CXXFLAGS) as they will be compiled with, to make sure code
scanned for QSTR occurrences is effectively the code used in the rest of
the build. The 'split SCR_QSTR in .c and .cpp files and process each with
different flags' logic isn't trivial to express in a Makefile and the
existing principle for deciding which files to preprocess was already
rather complicated, so the actual preprocessing is moved into
makeqstrdefs.py completely.
When process_file() is passed a preprocessed C++ file for instance it won't
find any lines containing .c files and the last_fname variable remains
None, so handle that gracefully.
If a port provides MICROPY_PY_URANDOM_SEED_INIT_FUNC as a source of
randomness then this will be used when urandom.seed() is called without
an argument (or with None as the argument) to seed the pRNG.
Other related changes in this commit:
- mod_urandom___init__ is changed to call seed() without arguments, instead
of explicitly passing in the result of MICROPY_PY_URANDOM_SEED_INIT_FUNC.
- mod_urandom___init__ will only ever seed the pRNG once (before it could
seed it again if imported by, eg, random and then urandom).
- The Yasmarang state is moved to the BSS for builds where the state is
guaranteed to be initialised on import of the (u)random module.
Signed-off-by: Damien George <damien@micropython.org>
The same seed will only occur if the board is the same, the RTC has the
same time (eg freshly powered up) and the first call to this function (eg
via an "import random") is done at exactly the same time since reset.
Signed-off-by: Damien George <damien@micropython.org>
For seeding, the RNG function of the ESP-IDF is used, which is told to be a
true RNG, at least when WiFi or Bluetooth is enabled. Seeding on import is
as per CPython. To obtain a reproducible sequence of pseudo-random numbers
one must explicitly seed with a known value.
Prior to this commit, the ADC calibration code was never executing because
ADVREGEN bit was set making the CR register always non-zero.
This commit changes the logic so that ADC calibration is always run when
the ADC is disabled and an ADC channel is initialised. It also uses the LL
API functions to do the calibration, to make sure it is done correctly on
each MCU variant.
Signed-off-by: Damien George <damien@micropython.org>
When threading is enabled without the GIL then there can be races between
the threads accessing the globals dict. Avoid this issue by making sure
all globals variables are allocated before starting the threads.
Signed-off-by: Damien George <damien@micropython.org>
Newer GCC versions are able to warn about switch cases that fall
through. This is usually a sign of a forgotten break statement, but in
the few cases where a fall through is intended we annotate it with this
macro to avoid the warning.
Like Clang, GCC warns about this file, but only with -Woverride-init
which is enabled by -Wextra. Disable the warnings for this file just
like we do for Clang to make -Wextra happy.
When compiling with -Wextra which includes -Wmissing-field-initializers
GCC will warn that the defval field of mp_arg_val_t is not initialized.
This is just a warning as it is defined to be zero initialized, but since
it is a union it makes sense to be explicit about which member we're
going to use, so add the explicit initializers and get rid of the
warning.
On x86 chars are signed, but we're comparing a char to '0' + unsigned int,
which is promoted to an unsigned int. Let's promote the char to unsigned
before doing the comparison to avoid weird corner cases.
The function scope_find_or_add_id used to take a scope_kind_t enum and
save it in an uint8_t. Saving an enum in a uint8_t is fine, but
everywhere this function is called it is not actually given a
scope_kind_t but an anonymous enum instead. Let's give this enum a name
and use that as the argument type.
This doesn't change the generated code, but is a C type mismatch that
unfortunately doesn't show up unless you enable -Wenum-conversion.
If the device is not connected over USB CDC to a host then all output to
the CDC (eg initial boot messages) is written to the CDC TX buffer with
wrapping, so that the most recent data is retained when the USB CDC is
eventually connected (eg so the REPL banner is displayed upon connection).
This commit fixes a bug in this behaviour, which was likely introduced in
e4fcd216e0, where the initial data in the CDC
TX buffer is repeated multiple times on first connection of the device to
the host.
Signed-off-by: Damien George <damien@micropython.org>
This is a generally useful feature and because it's part of the object
model it cannot be added at runtime by some loadable Python code, so enable
it on the standard unix build.
The last argument of TUD_CDC_DESCRIPTOR() is the endpoint size (or
wMaxPacketSize), not the CDC RX buffer size (which can be larger than the
endpoint size).
Signed-off-by: Damien George <damien@micropython.org>
When installing WS firmware, the very first GET_STATE can take several
seconds to respond (especially with the larger binaries like
BLE_stack_full).
Allows stm.rfcore_sys_hci to take an optional timeout, defaulting to
SYS_ACK_TIMEOUT_MS (which is 250ms).
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
The flash can sometimes be in an already-unlocked state, and attempting to
unlock it again will cause an immediate reset. So make _Flash.unlock()
check FLASH_CR_LOCK to get the current state.
Also fix some magic numbers for FLASH_CR_LOCK AND FLASH_CR_STRT.
The machine.reset() could be removed because it no longer crashes now that
the flash unlock is fixed.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This commit adds a script that can be run on-device to install FUS and WS
binaries from the filesystem. Instructions for use are provided in
the rfcore_firmware.py file.
The commit also removes unneeded functionality from the existing rfcore.py
debug script (and renames it rfcore_debug.py).
The new functions provide FUS/WS status, version and SYS HCI commands:
- stm.rfcore_status()
- stm.rfcore_fw_version(fw_id)
- stm.rfcore_sys_hci(ogf, ocf, cmd)
Changes are:
- Fix missing IRQ handler when SDMMC2 is used instead of SDMMC1 with H7
MCUs.
- Removed outdated H7 series compatibility macros.
- Defined common IRQ handler macro for F4 series.
It requires mp_hal_time_ns() to be provided by a port. This function
allows very accurate absolute timestamps.
Enabled on unix, windows, stm32, esp8266 and esp32.
Signed-off-by: Damien George <damien@micropython.org>
With a warning that this way of constructing software I2C/SPI is
deprecated. The check and warning will be removed in a future release.
This should help existing code to migrate to the new SoftI2C/SoftSPI types.
Signed-off-by: Damien George <damien@micropython.org>
Previous commits removed the ability for one I2C/SPI constructor to
construct both software- or hardware-based peripheral instances. Such
construction is now split to explicit soft and non-soft types.
This commit makes both types available in all ports that previously could
create both software and hardware peripherals: machine.I2C and machine.SPI
construct hardware instances, while machine.SoftI2C and machine.SoftSPI
create software instances.
This is a breaking change for use of software-based I2C and SPI. Code that
constructed I2C/SPI peripherals in the following way will need to be
changed:
machine.I2C(-1, ...) -> machine.SoftI2C(...)
machine.I2C(scl=scl, sda=sda) -> machine.SoftI2C(scl=scl, sda=sda)
machine.SPI(-1, ...) -> machine.SoftSPI(...)
machine.SPI(sck=sck, mosi=mosi, miso=miso)
-> machine.SoftSPI(sck=sck, mosi=mosi, miso=miso)
Code which uses machine.I2C and machine.SPI classes to access hardware
peripherals does not need to change.
Signed-off-by: Damien George <damien@micropython.org>
The SoftSPI constructor is now used soley to create SoftSPI instances, it
can no longer delegate to create a hardware-based SPI instance.
Signed-off-by: Damien George <damien@micropython.org>
The SoftI2C constructor is now used soley to create SoftI2C instances, it
can no longer delegate to create a hardware-based I2C instance.
Signed-off-by: Damien George <damien@micropython.org>
Also rename machine_i2c_type to mp_machine_soft_i2c_type. These changes
make it clear that it's a soft-I2C implementation, and match SoftSPI.
Signed-off-by: Damien George <damien@micropython.org>
Some downstream projects may use tags in their repositories for more than
just designating MicroPython releases. In those cases, the
makeversionhdr.py script would end up using a different tag than intended.
So tell `git describe` to only match tags that look like a MicroPython
version tag, such as `v1.12` or `v2.0`.
Zephyr v2.4.0 added a const qualifier to usages of struct device to
allow storing device driver instances exclusively in flash and thereby
reduce ram footprint.
Signed-off-by: Maureen Helm <maureen.helm@nxp.com>
The use of -S ensures that only the CPython standard library is accessible,
which makes tests run the same regardless of any site-packages that are
installed. It also improves start-up time of CPython, reducing the overall
time spent running the test suite.
tests/basics/containment.py is updated to work around issue with old Python
versions not being able to str-format a dict-keys object, which becomes
apparent when -S is used.
Signed-off-by: Damien George <damien@micropython.org>
A read-only memoryview object is a better representation of the data, which
is owned by the ubluetooth module and may change between calls to the
user's irq callback function.
Signed-off-by: Damien George <damien@micropython.org>
Calling the bytes constructor on a bytes object returns the original bytes
object. This saves allocating a new instance, and matches CPython.
Signed-off-by: Iyassou Shimels <s.iyassou@gmail.com>
Make the instructions more complete by documenting all needed steps for
starting from scratch. Also add a section for MSYS2 since the Travis build
uses it as well and it's a good alternative for Cygwin. Remove the mingw32
reference since it's not readily available anymore in most Linux distros
nor compiles successfully.
Prior to this commit, uos.chdir('/') followed by uos.stat('noexist') would
succeed that stat even though the entry did not exist (some other functions
like listdir would have similar issues). This is because, if the current
directory was the root and the path was relative, mp_vfs_lookup_path would
return success for bad paths.
Signed-off-by: Damien George <damien@micropython.org>
The device info table has a different layout when core 2 is in FUS mode.
In particular it's larger than the 32 bytes used when in WS mode and if the
correct amount of space is not allocated then the end of the table may be
overwritten with other data (eg with FUS version 0.5.3). So update the
structure to fix this.
Also update rfcore.py to disable IRQs (which are enabled by rfcore.c), to
not depend on uctypes, and to not require the asm_thumb emitter.
Signed-off-by: Damien George <damien@micropython.org>
And enable this feature on unix, the coverage variant. The .exp test file
is needed so the test can run on CPython versions prior to "@=" operator
support.
Signed-off-by: Damien George <damien@micropython.org>
For time-based functions that work with absolute time there is the need for
an Epoch, to set the zero-point at which the absolute time starts counting.
Such functions include time.time() and filesystem stat return values. And
different ports may use a different Epoch.
To make it clearer what functions use the Epoch (whatever it may be), and
make the ports more consistent with their use of the Epoch, this commit
renames all Epoch related functions to include the word "epoch" in their
name (and remove references to "2000").
Along with this rename, the following things have changed:
- mp_hal_time_ns() is now specified to return the number of nanoseconds
since the Epoch, rather than since 1970 (but since this is an internal
function it doesn't change anything for the user).
- littlefs timestamps on the esp8266 have been fixed (they were previously
off by 30 years in nanoseconds).
Otherwise, there is no functional change made by this commit.
Signed-off-by: Damien George <damien@micropython.org>
To portably get the Epoch. This is simply aliased to localtime() on ports
that are not timezone aware.
Signed-off-by: Damien George <damien@micropython.org>
This commit removes release-specific builds for the esp8266 and makes the
normal build of the GENERIC board more like the release build. This makes
esp8266 like all the other ports, for which there is no difference between
a daily build and a release build, making things less confusing.
Release builds were previously defined by UART_OS=-1 (disable OS messages)
and using manifest_release.py to include more frozen modules.
The changes in this commit are:
- Remove manifest_release.py.
- Add existing modules from manifest_release.py (except example code)
to the GENERIC board's manifest.py file.
- Change UART_OS default to -1 to disable OS messages by default.
Signed-off-by: Damien George <damien@micropython.org>
PPP support was disabled in 96008ff59a -
marked as "unsupported" due to an early IDF v4 release. With the currently
supported IDF v4.x version - 4c81978a - it appears to be working just fine.
This commit changes the default logging level on all esp32 boards to ERROR.
The esp32 port is now stable enough that it makes sense to remove the info
logs to make the output cleaner, and to match other ports. More verbose
logging can always be reenabled via esp.osdebug().
This also fixes issue #6354, error messages from NimBLE: the problem is
that ble.active(True) will cause the IDF's NimBLE port to reset the
"NimBLE" tag back to the default level (which was INFO prior to this
commit). Even if the user had previously called esp.osdebug(None), because
the IDF is setting the "NimBLE" tag back to the default (INFO), the
messages will continue to be shown.
The one quirk is that if the user does want to see the additional logging,
then they must call esp.osdebug(0, 3) after ble.active(True) to undo the
IDF setting the level back to the default (now ERROR). This means that
it's impossible (via Python/esp.osdebug) to see stack-startup logging,
you'd have to recompile with the default level changed back to INFO.
For the following 3 functions, previously the code relied on whether the
arg was passed at all, to make it optional. Now it allows them to be
explicitly `None` to indicate they are not used:
- gatts_notify(..., [data])
- gattc_discover_services(..., [uuid])
- gattc_discover_characteristics(..., [uuid])
Also ensure that the uuid arguments are actually instances of the uuid
type, and fix handling of the 5th arg in gattc_discover_characteristics().
Support freezing modules via manifest.py for consistency with the other
ports. In essence this comes down to calling makemanifest.py and adding
the resulting .c file to the build. Note the file with preprocessed qstrs
has been renamed to match what makemanifest.py expects and which is also
the name all other ports use.
The msvc compiler doesn't accept zero-sized arrays so let the freezing
process generate compatible C code in case no modules are found and the
involved arrays are all empty. This doesn't affect the functionality in
any way because those arrays only get accessed when mp_frozen_mpy_names
contains names, i.e. when modules are actually found.
Prior to this commit, pow(-2, float('nan')) would return (nan+nanj), or
raise an exception on targets that don't support complex numbers. This is
fixed to return simply nan, as CPython does.
Signed-off-by: Damien George <damien@micropython.org>
The mpconfigport.h file is an internal header and should only ever be
included once by mpconfig.h.
Signed-off-by: Damien George <damien@micropython.org>
MP_BC_CALL_FUNCTION will leave the result on the Python stack, so that
result must be discarded by MP_BC_POP_TOP.
Signed-off-by: Damien George <damien@micropython.org>
This allows prototyping rfcore.c improvements from Python.
This was mostly written by @dpgeorge with small modifications to work after
rfcore_init() by @jimmo.
Before this change there was up to a 128ms delay on incoming payloads from
CPU2 as it was polled by SysTick. Now the RX IRQ immediately schedules the
PendSV.
This is required to allow using WS firmware newer than 1.1.1 concurrently
with USB (e.g. USB VCP). It prevents CPU2 from modifying the CLK48 config
on boot.
Tested on WS=1.8 FUS=1.1.
See AN5289 and https://github.com/micropython/micropython/issues/6316
- Split tables and buffers into SRAM2A/2B.
- Use structs rather than word offsets to access tables.
- Use FLASH_IPCCDBA register value rather than option bytes directly.
This allows `ble.active(1)` to fail correctly if the HCI controller is
unavailable.
It also avoids an infine loop in the NimBLE event handler where NimBLE
doesn't correctly detect that the HCI controller is unavailable and keeps
trying to reset.
Furthermore, it fixes an issue where GATT service registrations were left
allocated, which led to a bad realloc if the stack was activated multiple
times.
MicroPython and NimBLE must be on the same core, for synchronisation of the
BLE ringbuf and the MicroPython scheduler. However, in the current IDF
versions (3.3 and 4.0) there are issues (see e.g. #5489) with running
NimBLE on core 1.
This change - pinning both tasks to core 0 - makes it possible to reliably
run the BLE multitests on esp32 boards.
Generally a controller should either have its own public address hardcoded,
or loaded by the driver (e.g. cywbt43).
However, for a controller that has no public address where you still want a
long-term stable address, this allows you to use a static address generated
by the port. Typically on STM32 this will be an LAA, but a board might
override this.
This commit adds support for using Bluetooth on the unix port via a H4
serial interface (distinct from a USB dongle), with both BTstack and NimBLE
Bluetooth stacks.
Note that MICROPY_PY_BLUETOOTH is now disabled for the coverage variant.
Prior to this commit Bluetooth was anyway not being built on Travis because
libusb was not detected. But now that bluetooth works in H4 mode it will
be built, and will lead to a large decrease in coverage because Bluetooth
tests cannot be run on Travis.
Previously the interaction between the different layers of the Bluetooth
stack was different on each port and each stack. This commit defines
common interfaces between them and implements them for cyw43, btstack,
nimble, stm32, unix.
mp_irq_init() is useful when the IRQ object is allocated by the caller.
The mp_irq_methods_t.init method is not used anywhere so has been removed.
Signed-off-by: Damien George <damien@micropython.org>
This is consistent with the other 'micro' modules and allows implementing
additional features in Python via e.g. micropython-lib's sys.
Note this is a breaking change (not backwards compatible) for ports which
do not enable weak links, as "import sys" must now be replaced with
"import usys".
Verifies mtime timestamps on files match the value returned by time.time().
Also update vfs_fat_ramdisk.py so it doesn't check FAT timestamp of the
root, because that may change across runs/ports.
Signed-off-by: Damien George <damien@micropython.org>
By setting MICROPY_EPOCH_IS_1970 a port can opt to use 1970/1/1 as the
Epoch for timestamps returned by stat(). And this setting is enabled on
the unix and windows ports because that's what they use.
Signed-off-by: Damien George <damien@micropython.org>
On 32-bit builds these stat fields will overflow a small-int, so use
mp_obj_new_int_from_uint to construct the int object.
Signed-off-by: Damien George <damien@micropython.org>
On ports like unix where the Epoch is 1970/1/1 and atime/mtime/ctime are in
seconds since the Epoch, this value will overflow a small-int on 32-bit
systems. So far this is only an issue on 32-bit unix builds that use the
VFS layer (eg dev and coverage unix variants) but the fix (using
mp_obj_new_int_from_uint instead of MP_OBJ_NEW_SMALL_INT) is there for all
ports so as to not complicate the code, and because they will need the
range one day.
Also apply a similar fix to other fields in VfsPosix.stat because they may
also be large.
Signed-off-by: Damien George <damien@micropython.org>
gettimeofday returns seconds since 2000/1/1 so needs to be adjusted to
seconds since 1970/1/1 to give the correct return value of mp_hal_time_ns.
Signed-off-by: Damien George <damien@micropython.org>
This commit fixes the cases when a TCP socket is in STATE_NEW,
STATE_LISTENING or STATE_CONNECTING and recv() is called on it. It now
raises ENOTCONN instead of a random error code due to it previously
indexing beyond the start of error_lookup_table[].
Signed-off-by: Damien George <damien@micropython.org>
Updating to Black v20.8b1 there are two changes that affect the code in
this repository:
- If there is a trailing comma in a list (eg [], () or function call) then
that list is now written out with one line per element. So remove such
trailing commas where the list should stay on one line.
- Spaces at the start of """ doc strings are removed.
Signed-off-by: Damien George <damien@micropython.org>
So they can be skipped if __rOP__'s are not supported on the target. Also
fix the typo in the complex_special_methods.py filename.
Signed-off-by: Damien George <damien@micropython.org>
The memory operation functions read_mem() and write_mem() create a
temporary buffer on the local C stack for the address bytes with the size
of 4 bytes. This buffer is filled in a loop from the user supplied address
and address length. If the user supplied 'addrsize' is bigger than 32, the
local buffer is overrun.
Fix this by raising an exception for invalid 'addrsize' values.
Signed-off-by: Michael Buesch <m@bues.ch>
A configurable result directory is advantageous because it enables
using a dedicated location, eventually outside of the source tree,
instead of forcing the output files into a fixed directory which might
also contain other files already. For that reason the default output
directory also has been changed to tests/results/.
Replace some usages of paths relative to the current working directory
with absolute paths relative to the tests directory.
Fixes and resulting changes:
- default values of MICROPYTHON and MPYCROSS are absolute paths and
always correct
- likewise, the correct full paths for tools and extmod directories
are appended to sys.path
- printing/cleaning failures works properly since it expects the .exp
and .out files in the tests directory which is also where they
are written to now, plus no more need for changing directories
This fixes#5872 and allows running custom tests which use run-tests
without having to cd to the tests directory first, and the test output
still is in the tests/ directory instead of the current working directory.
Discovery of tests and all skip test logic based on paths relative to
the current working directory remains unchanged which essentially means
that for running most of MicroPython's own tests, run-tests must still
be ran from within it's directory, so document that.
With sleep(0.2) a multiple of sleep(0.1), the order of task 2 and 3
execution is not well defined, and depends on the precision of the system
clock and how fast the rest of the code runs. So change 0.2 to 0.18 to
make the test more reliable.
Also fix a typo of t3/t4, and cancel t4 at the end.
Signed-off-by: Damien George <damien@micropython.org>
This adds an additional optional parameter to gap_scan() to select active
scanning, where scan responses are returned as well as normal scan results.
This parameter is False by default which retains the existing behaviour.
The READ_REQUEST callback is handled as a hard interrupt (because the BLE
stack needs an immediate response from it so it can continue) and so calls
to Python require extra protection:
- the caller-owned tuple passed into the callback must be separate from the
tuple used by other callback events (which are soft interrupts);
- the GC and scheduler must be locked during callback execution.
This commit adds support for modification time of files on littlefs v2
filesystems, using file attributes. For some background see issue #6114.
Features/properties of this implementation:
- Only supported on littlefs2 (not littlefs1).
- Uses littlefs2's general file attributes to store the timestamp.
- The timestamp is 64-bits and stores nanoseconds since 1970/1/1 (if the
range to the year 2554 is not enough then additional bits can be added to
this timestamp by adding another file attribute).
- mtime is enabled by default but can be disabled in the constructor, eg:
uos.mount(uos.VfsLfs2(bdev, mtime=False), '/flash')
- It's fully backwards compatible, existing littlefs2 filesystems will work
without reformatting and timestamps will be added transparently to
existing files (once they are opened for writing).
- Files without timestamps will open correctly, and stat will just return 0
for their timestamp.
- mtime can be disabled or enabled each mount time and timestamps will only
be updated if mtime is enabled (otherwise they will be untouched).
Signed-off-by: Damien George <damien@micropython.org>
Otherwise a task that continuously awaits on a large negative sleep can
monopolise the scheduler (because its wake time is always less than
everything else in the pairing heap).
Signed-off-by: Damien George <damien@micropython.org>
As per CPython behaviour, compile(stmt, "file", "single") should create
code which prints to stdout (via __repl_print__) the results of any
expressions in stmt.
Signed-off-by: Damien George <damien@micropython.org>
The existing implementation of mkdir() in this file is not sophisticated
enough to work correctly on all operating systems (eg Mac can raise
EISDIR). Using the standard os.makedirs() function handles all cases
correctly.
Signed-off-by: Damien George <damien@micropython.org>
Prior to this commit, pyboard.py used eval() to "parse" file data received
from the board. Using eval() on received data from a device is dangerous,
because a malicious device may inject arbitrary code execution on the PC
that is doing the operation.
Consider the following scenario:
Eve may write a malicious script to Bob's board in his absence. On return
Bob notices that something is wrong with the board, because it doesn't work
as expected anymore. He wants to read out boot.py (or any other file) to
see what is wrong. What he gets is a remote code execution on his PC.
Proof of concept:
Eve:
$ cat boot.py
_print = print
print = lambda *x, **y: _print("os.system('ls /; echo Pwned!')", end="\r\n\x04")
$ ./pyboard.py -f cp boot.py :
cp boot.py :boot.py
Bob:
$ ./pyboard.py -f cp :boot.py /tmp/foo
cp :boot.py /tmp/foo
bin chroot dev home lib32 media opt root sbin sys usr
boot config etc lib lib64 mnt proc run srv tmp var
Pwned!
There's also the possibility that the device is malfunctioning and sends
random and possibly dangerous data back to the PC, to be eval'd.
Fix this problem by using ast.literal_eval() to parse the received bytes,
instead of eval().
Signed-off-by: Michael Buesch <m@bues.ch>
Prior to this commit, if you configure a pin as an output type (I2C in this
example) and then later configure it back as an input, then it will report
the type incorrectly. Example:
>>> import machine
>>> b6 = machine.Pin('B6')
>>> b6
Pin(Pin.cpu.B6, mode=Pin.IN)
>>> machine.I2C(1)
I2C(1, scl=B6, sda=B7, freq=420000)
>>> b6
Pin(Pin.cpu.B6, mode=Pin.ALT_OPEN_DRAIN, pull=Pin.PULL_UP, af=Pin.AF4_I2C1)
>>> b6.init(machine.Pin.IN)
>>> b6
Pin(Pin.cpu.B6, mode=Pin.ALT_OPEN_DRAIN, af=Pin.AF4_I2C1)
With this commit the last print now works:
>>> b6
Pin(Pin.cpu.B6, mode=Pin.IN)
Latest versions of Sphinx (at least 3.1.0) do not need the `*` escaped and
will render the `\` in the output if it is there, so remove it.
Fixes issue #6209.
Adds a job to build the zephyr port in CI using the same docker container
that the zephyr project uses for its own CI.
Always make clean zephyr builds to ensure we don't just rebuild C code, but
we also rebuild Kconfig and dts. This is required when switching between
boards, which have different Kconfigs and device trees.
Uses the tagged zephyr 2.3.0 release.
Signed-off-by: Maureen Helm <maureen.helm@nxp.com>
Include storage/flash_map.h unconditionally so we always have access to the
FLASH_AREA_LABEL_EXISTS macro, even if CONFIG_FLASH_MAP is not defined.
This fixes a build error for the qemu_x86 board:
main.c:108:63: error: missing binary operator before token "("
108 | #elif defined(CONFIG_FLASH_MAP) && FLASH_AREA_LABEL_EXISTS(storage)
| ^
../../py/mkrules.mk:88: recipe for target 'build/genhdr/qstr.i.last' failed
Signed-off-by: Maureen Helm <maureen.helm@nxp.com>
mp_reader_new_file() is used to read in files for importing, either .py or
.mpy files, for the lexer and persistent code loader respectively. In both
cases the file should be opened in raw bytes mode: the lexer handles
unicode characters itself, and .mpy files contain 8-bit bytes by nature.
Before this commit importing was working correctly because, although the
file was opened in text mode, all native filesystem implementations (POSIX,
FAT, LFS) would access the file in raw bytes mode via mp_stream_rw()
calling mp_stream_p_t.read(). So it was only an issue for non-native
filesystems, such as those implemented in Python. For Python-based
filesystem implementations, a call to mp_stream_rw() would go via IOBase
and then to readinto() at the Python level, and readinto() is only defined
on files opened in raw bytes mode.
Signed-off-by: Damien George <damien@micropython.org>
If mpy-cross exits with an error be sure to print that error in a way that
is readable, instead of a long bytes object.
Signed-off-by: Damien George <damien@micropython.org>
On ports where normal heap memory can contain executable code (eg ARM-based
ports such as stm32), native code loaded from an .mpy file may be reclaimed
by the GC because there's no reference to the very start of the native
machine code block that is reachable from root pointers (only pointers to
internal parts of the machine code block are reachable, but that doesn't
help the GC find the memory).
This commit fixes this issue by maintaining an explicit list of root
pointers pointing to native code that is loaded from an .mpy file. This
is not needed for all ports so is selectable by the new configuration
option MICROPY_PERSISTENT_CODE_TRACK_RELOC_CODE. It's enabled by default
if a port does not specify any special functions to allocate or commit
executable memory.
A test is included to test that native code loaded from an .mpy file does
not get reclaimed by the GC.
Fixes#6045.
Signed-off-by: Damien George <damien@micropython.org>
All imports are now tested to see if the test should be skipped,
UserFile.read is removed, and UserFile.readinto is made more efficient.
Signed-off-by: Damien George <damien@micropython.org>
These tests are specific to MicroPython so have a better home in the
micropython/ test subdir, and putting them here allows them to be run by
all targets, not just those that have access to the local filesystem (eg
the unix port).
Signed-off-by: Damien George <damien@micropython.org>
It raises on EOFError instead of an IncompleteReadError (which is what
CPython does). But the latter is derived from EOFError so code compatible
with MicroPython and CPython can be written by catching EOFError (eg see
included test).
Fixes issue #6156.
Signed-off-by: Damien George <damien@micropython.org>
The SCSI driver calls GetCapacity to get the block size and number of
blocks of the underlying block-device/LUN. It caches these values and uses
them later on to verify that reads/writes are within the bounds of the LUN.
But, prior to this commit, there was only one set of cached values for all
LUNs, so the bounds checking for a LUN could use incorrect values, values
from one of the other LUNs that most recently updated the cached values.
This would lead to failed SCSI requests.
This commit fixes this issue by having separate cached values for each LUN.
Signed-off-by: Damien George <damien@micropython.org>
MicroPython's original implementation of __aiter__ was correct for an
earlier (provisional) version of PEP492 (CPython 3.5), where __aiter__ was
an async-def function. But that changed in the final version of PEP492 (in
CPython 3.5.2) where the function was changed to a normal one. See
https://www.python.org/dev/peps/pep-0492/#why-aiter-does-not-return-an-awaitable
See also the note at the end of this subsection in the docs:
https://docs.python.org/3.5/reference/datamodel.html#asynchronous-iterators
And for completeness the BPO: https://bugs.python.org/issue27243
To be consistent with the Python spec as it stands today (and now that
PEP492 is final) this commit changes MicroPython's behaviour to match
CPython: __aiter__ should return an async-iterable object, but is not
itself awaitable.
The relevant tests are updated to match.
See #6267.
Enabling the following features for all targets, except for nrf51
targets compiled to be used with SoftDevice:
- MICROPY_PY_ARRAY_SLICE_ASSIGN
- MICROPY_PY_SYS_STDFILES
- MICROPY_PY_UBINASCII
Splitting mpconfigport.h into multiple device specific
files in order to facilitate variations between devices.
Due to the fact that the devices might have variations in
features and also variations in flash size it makes sense
that some devices offers more functionality than others
without being limited by restricted devices.
For example more micropython features can be activated for
nrf52840 with 1MB flash, compared to nrf51 with 256KB.
Fixing 98e583430f, the semantics of strncpy
require that the remainder of dst be filled with null bytes.
Signed-off-by: Damien George <damien@micropython.org>
This code is imported from musl, to match existing code in libm_dbl.
The file is also added to the build in stm32/Makefile. It's not needed by
the core code but, similar to c5cc64175b,
allows round() to be used by user C modules or board extensions.
Because the argument arrays may overlap, as show by the new tests in this
commit.
Also remove the debugging comments for these macros, add a new comment
about overlapping regions, and separate the macros by blank lines to make
them easier to read.
Fixes issue #6244.
Signed-off-by: Damien George <damien@micropython.org>
Changes in this new library version are:
- Update H7 HAL to v1.6.0.
- Update WB HAL to v1.6.0.
- Add patches to fix F4 ll_uart clock selection for UART9/UART10.
Signed-off-by: Damien George <damien@micropython.org>
A previous commit 3a9d948032 can cause
lock-ups of the RMT driver, so this commit reverses that, adds a loop_en
flag, and explicitly controls the TX interrupt in write_pulses(). This
provides correct looping, non-blocking writes and sensible behaviour for
wait_done().
See also #6167.
The file `mbedtls_errors/mp_mbedtls_errors.c` can be used instead of
`mbedtls/library/error.c` to give shorter error strings, reducing the build
size of the error strings from about 12-16kB down to about 2-5kB.
This commit adds human readable error messages when mbedtls or axtls raise
an exception. Currently often just an EIO error is raised so the user is
lost and can't tell whether it's a cert error, buffer overrun, connecting
to a non-ssl port, etc. The axtls and mbedtls error raising in the ussl
module is modified to raise:
OSError(-err_num, "error string")
For axtls a small error table of strings is added and used for the second
argument of the OSErrer. For mbedtls the code uses mbedtls' built-in
strerror function, and if there is an out of memory condition it just
produces OSError(-err_num). Producing the error string for mbedtls is
conditional on them being included in the mbedtls build, via
MBEDTLS_ERROR_C.
This commit adds the IRQ_GATTS_INDICATE_DONE BLE event which will be raised
with the status of gatts_indicate (unlike notify, indications require
acknowledgement).
An example of its use is added to ble_temperature.py, and to the multitests
in ble_characteristic.py.
Implemented for btstack and nimble bindings, tested in both directions
between unix/btstack and pybd/nimble.
The goal of this commit is to allow using ble.gatts_notify() at any time,
even if the stack is not ready to send the notification right now. It also
addresses the same issue for ble.gatts_indicate() and ble.gattc_write()
(without response). In addition this commit fixes the case where the
buffer passed to write-with-response wasn't copied, meaning it could be
modified by the caller, affecting the in-progress write.
The changes are:
- gatts_notify/indicate will now run in the background if the ACL buffer is
currently full, meaning that notify/indicate can be called at any time.
- gattc_write(mode=0) (no response) will now allow for one outstanding
write.
- gattc_write(mode=1) (with response) will now copy the buffer so that it
can't be modified by the caller while the write is in progress.
All four paths also now track the buffer while the operation is in
progress, which prevents the GC free'ing the buffer while it's still
needed.
It might compile fine with S132 as SoftDevice for nRF52840.
However, there might be different hardware on the SoC which in
turn could make it fail.
SoftDevice S140 is correct BLE stack for nRF52840 SoC and would
provide a more accurate test build.
So that micropython-dev can be used to test VFS code, and inspect and build
filesystem images that are compatible with bare-metal systems.
Signed-off-by: Damien George <damien@micropython.org>
This commit adds time.ticks_ms/us support using RTC1 as the timebase. It
also adds the time.ticks_add/diff helper functions. This feature can be
enabled using MICROPY_PY_TIME_TICKS. If disabled the system uses the
legacy sleep methods and does not have any ticks functions.
In addition support for MICROPY_EVENT_POLL_HOOK was added to the
time.sleep_ms(x) function, making this function more power efficient and
allows support for select.poll/asyncio. To support this, the RTC's CCR0
was used to schedule a ~1msec event to wakeup the CPU.
Some important notes about the RTC timebase:
- Since the granularity of RTC1's ticks are approx 30usec, time.ticks_us is
not perfect, does not have 1us resolution, but is otherwise quite usable.
For tighter measurments the ticker's 1MHz counter should be used.
- time.ticks_ms(x) should *not* be called in an IRQ with higher prio than
the RTC overflow irq (3). If so it introduces a race condition and
possibly leads to wrong tick calculations.
See #6171 and #6202.
When compiling for debug (-O0) the .text segment cannot fit the flash
region when MICROPY_ROM_TEXT_COMPRESSION=1, because the compiler does not
optimise away the large if-else chain used to select the correct compressed
string.
This commit enforces MICROPY_ROM_TEXT_COMPRESSION=0 when compiling for
debug (DEBUG=1).
The storage space of the advertisement name is not declared static, leading
to a random advertisement name. This commit fixes the issue by declaring
it static.
The Bluetooth link gets disconnected when connecting from a PC after 30-40
seconds. This commit adds handling of the data length update request. The
data length parameter pointer is set to NULL in the reply, letting the
SoftDevice automatically set values and use them in the data length update
procedure.
mp_keyboard_interrupt() triggers a compiler error because the function is
implicitly declared. This commit adds "py/runtime.h" to the includes.
Fixes issue #5732.
Changes are:
- The default manifest.py is moved to the variants directory (it's in
"boards" in other ports).
- The coverage variant now uses a custom manifest in its variant directory
to add frzmpy/frzstr.
- The frzmpy/frzstr tests are moved to variants/coverage/.
This reverts commit 4d6f60d428.
This implementation used the timeout as a maximum amount of time needed for
the operation, when actually the spec and other tools suggest that it's the
minumum delay needed between subsequent USB transfers.
Polling mode will cause failures with the mass-erase command due to USB
timeouts, because the USB IRQs are not being serviced. Swiching from
polling to IRQ mode fixes this because the USB IRQs can be serviced between
page erases.
Note that when the flash is being programmed or erased the MCU is halted
and cannot respond to USB IRQs, because mboot runs from flash, as opposed
to the built-in bootloader which is in system ROM. But the maximum delay
in responding to an IRQ is the time taken to erase a single page, about
100ms for large pages, and that is short enough that the USB does not
timeout on the host side.
Recent tests have shown that in the current mboot code IRQ mode is pretty
much the same speed as polling mode (within timing error), code size is
slightly reduced in IRQ mode, and IRQ mode idles at about half of the power
consumption as polling mode.
This is treated more like a "delay before continuing" in the spec and
official tools and does not appear to be really needed. In particular,
downloading firmware is much slower with non-zero timeouts because the host
must pause by the timeout between sending each DFU_GETSTATUS to poll for
download/erase complete.
This commit fixes lookups of class members to make it so that built-in
functions that are used as methods/functions of a class work correctly.
The mp_convert_member_lookup() function is pretty much completely changed
by this commit, but for the most part it's just reorganised and the
indenting changed. The functional changes are:
- staticmethod and classmethod checks moved to later in the if-logic,
because they are less common and so should be checked after the more
common cases.
- The explicit mp_obj_is_type(member, &mp_type_type) check is removed
because it's now subsumed by other, more general tests in this function.
- MP_TYPE_FLAG_BINDS_SELF and MP_TYPE_FLAG_BUILTIN_FUN type flags added to
make the checks in this function much simpler (now they just test this
bit in type->flags).
- An extra check is made for mp_obj_is_instance_type(type) to fix lookup of
built-in functions.
Fixes#1326 and #6198.
Signed-off-by: Damien George <damien@micropython.org>
Supports hard and soft interrupts. In the current implementation, soft
interrupt callbacks will only be called when the VM is executing, ie they
will not be called during a blocking kernel call like k_msleep. And the
behaviour of hard interrupt callbacks will depend on the underlying device,
as well as the amount of ISR stack space.
Soft and hard interrupts tested on frdm_k64f and nucleo_f767zi boards.
Signed-off-by: Damien George <damien@micropython.org>
So it can be unconditionally included in a port's build even if certain
configurations in that port do not use its features, to simplify the
Makefile.
Signed-off-by: Damien George <damien@micropython.org>
The implementation internally uses sector erase to wipe everything except
the sector(s) that mboot lives in (by erasing starting from
APPLICATION_ADDR).
The erase command can take some time (eg an STM32F765 with 2MB of flash
takes 8 to 10 seconds). This time is normally enough to make pydfu.py fail
with a timeout. The DFU standard includes a mechanism for the DFU device
to request a longer timeout as part of the get-status response just before
starting an operation. This timeout functionality has been implemented
here.
Before this commit the USB VCP TX ring-buffer used the basic implementation
where it can only be filled to a maximum of buffer size-1. For a 1024 size
buffer this means the largest packet that can be sent is 1023. Once a
packet of this size is sent the next byte copied in goes to the final byte
in the buffer, so must be sent as a 1 byte packet before the read pointer
can be wrapped around to the beginning. So in large streaming transfers,
watching the USB sniffer you basically get alternating 1023 byte packets
then 1 byte packets.
This commit changes the ring-buffer implementation to a scheme that doesn't
have the full-size limitation, and the USB VCP driver can now achieve a
constant stream of full-sized packets. This scheme introduces a
restriction on the size of the buffer: it must be a power of 2, and the
maximum size is half of the size of the index (in this case the index is
16-bit, so the maximum size would be 32767 bytes rounded to 16384 for a
power-of-2). But this is not a big limitation because the size of the
ring-buffer prior to this commit was restricted to powers of 2 because it
was using a mask-based method to wrap the indices.
For an explanation of the new scheme see
https://www.snellman.net/blog/archive/2016-12-13-ring-buffers/
The RX buffer could likely do with a similar change, though as it's not
read from in chunks like the TX buffer it doesn't present the same issue,
all that's lost is one byte capacity of the buffer.
USB VCP TX throughput is improved by this change, potentially doubling the
speed in certain cases.
This allows complex binary operations to fail gracefully with unsupported
operation rather than raising an exception, so that special methods work
correctly.
Signed-off-by: Damien George <damien@micropython.org>
uint types in viper mode can now be used for all binary operators except
floor-divide and modulo.
Fixes issue #1847 and issue #6177.
Signed-off-by: Damien George <damien@micropython.org>
By passing through the I2C instance to the application callbacks, the
application can implement multiple I2C slave devices on different
peripherals (eg I2C1 and I2C2).
This commit also adds a proper rw argument to i2c_slave_process_addr_match
for F7/H7/WB MCUs, and enables the i2c_slave_process_tx_end callback.
Mboot is also updated for these changes.
Signed-off-by: Damien George <damien@micropython.org>
Mboot now supports FAT, LFS1 and LFS2 filesystems, to load firmware from.
The filesystem needed by the board must be explicitly enabled by the
configuration variables MBOOT_VFS_FAT, MBOOT_VFS_LFS1 and MBOOT_VFS_LFS2.
Boards that previously used FAT implicitly (with MBOOT_FSLOAD enabled) must
now add the following config to mpconfigboard.h:
#define MBOOT_VFS_FAT (1)
Signed-off-by: Damien George <damien@micropython.org>
This commit factors the code for files and streaming to separate source
files (vfs_fat.c and gzstream.c respectively) and introduces an abstract
gzstream interface to make it easier to plug in different filesystems.
Signed-off-by: Damien George <damien@micropython.org>
There's no need to do a directory listing to search for the given firmware
filename, it just takes extra time and code size. Instead this commit
changes it so that the requested firmware file is opened immediately and
will abort if the file couldn't be opened. This also allows to specify
files in a directory.
Signed-off-by: Damien George <damien@micropython.org>
Previously, if FAT was not enabled but LFS1/2 was then MICROPY_PY_IO_FILEIO
would be disabled and file binary-mode was not supported.
Signed-off-by: Damien George <damien@micropython.org>
This adds support for freezing an entire directory while keeping the
directory as part of the import path. For example
freeze("path/to/library", "module")
will recursively freeze all scripts in "path/to/library/module" and have
them importable as "from module import ...".
Signed-off-by: Damien George <damien@micropython.org>
An OrderedDict can now be used for the locals when creating a type
explicitly via type(name, bases, locals).
Signed-off-by: Damien George <damien@micropython.org>
Commit 8675858465 switched to using the CMSIS
provided SystemInit function which sets VTOR to 0x00000000 (previously it
was 0x08000000). A VTOR of 0x00000000 will be correct on some MCUs but not
on others where the built-in bootloader is remapped to this address, via
__HAL_SYSCFG_REMAPMEMORY_SYSTEMFLASH().
To make sure mboot has the correct vector table, this commit explicitly
sets VTOR to the correct value of 0x08000000.
Signed-off-by: Damien George <damien@micropython.org>
There's no need to duplicate this functionality in mboot, the code provided
in stm32lib/CMSIS does the same thing and makes it easier to support other
MCU series.
Signed-off-by: Damien George <damien@micropython.org>
The flash functions in ports/stm32/flash.c are almost identical to those in
ports/stm32/mboot/main.c, so remove the duplicated code in mboot and use
instead the main stm32 code. This also allows supporting other MCU series.
Signed-off-by: Damien George <damien@micropython.org>
This commit makes the low-level flash C functions usable by code other than
flashbdev.c (eg by mboot). Changes in this commit are:
- flash_erase() and flash_write() now return an errno error code, a
negative value on error.
- flash_erase() now automatically locks the flash, as well as unlocking it.
- flash_write() now automatically unlocks the flash, as well as locking it.
- flashbdev.c is modified for the above changes.
Signed-off-by: Damien George <damien@micropython.org>
irq.h is included by py/mphal.h but it's better to be explicit, eg if mboot
uses powerctrlboot.c.
Signed-off-by: Damien George <damien@micropython.org>
The irq.h file now just provides low-level IRQ definitions and priorities.
All Python binding definitions are moved to modmachine.h, with some
renaming of pyb -> machine, and also the machine_idle definition (was
pyb_wfi) is moved to modmachine.c.
The cc3200 and teensy ports are updated to build with these changes.
Signed-off-by: Damien George <damien@micropython.org>
Eventually it would be good to run the full test suite on a big-endian
system, but for now this will help to catch build errors with the
big-endian configuration.
Signed-off-by: Damien George <damien@micropython.org>
Otherwise the RMT will repeat pulses when using loop(True). This repeating
is due to a bug in the IDF which will be fixed in an upcoming release, but
for now the accepted workaround is to swap these calls, which should still
work in the fixed version of the IDF.
Fixes issue #6167.
With only `sp_func_proto_paren = remove` set there are some cases where
uncrustify misses removing a space between the function name and the
opening '('. This sets all of the related options to `force` as well.
There are a maximum of 8 USB endpoints and each has 2 buffer slots
(in/out). This commit add support for up to 8 endpoints and adds FIFO
configuration for USB profiles with 2xVCP on MCUs that have device-only USB
peripherals.
Tested on NUCLEO_WB55 in 2xVCP, 2xVCP+MSC and 2xVCP+MSC+HID mode.
Signed-off-by: Damien George <damien@micropython.org>
The ESP32 RMT peripheral has hardware support for a carrier frequency, and
this commit exposes it to Python with the keyword arguments carrier_freq
and carrier_duty_percent in the constructor. Example usage:
r = esp32.RMT(0, pin=Pin(2), clock_div=80, carrier_freq=38000, carrier_duty_percent=50)
This addition to the grammar was introduced in Python 3.6. It allows
annotating the type of a varilable, like:
x: int = 123
s: str
The implementation in this commit is quite simple and just ignores the
annotation (the int and str bits above). The reason to implement this is
to allow Python 3.6+ code that uses this feature to compile under
MicroPython without change, and for users to use type checkers.
In the future viper could use this syntax as a way to give types to
variables, which is currently done in a bit of an ad-hoc way, eg
x = int(123). And this syntax could potentially be used in the inline
assembler to define labels in an way that's easier to read.
The syntax matches CPython and the semantics are equivalent except that,
unlike CPython, MicroPython allows using := to assign to comprehension
iteration variables, because disallowing this would take a lot of code to
check for it.
The new compile-time option MICROPY_PY_ASSIGN_EXPR selects this feature and
is enabled by default, following MICROPY_PY_ASYNC_AWAIT.
This is the result of running...
uncrustify -c tools/uncrustify.cfg --update-config-with-doc -o tools/uncrustify.cfg
...with some manual fixups to correct places where it changed things it
should not have.
Essentially it just adds new config parameters introduced in v0.71.0
with their default values.
Signed-off-by: David Lechner <david@pybricks.com>
Formatting for `* sizeof` was fixed in uncrustify v0.71, so we no longer
need the fixups for it. Also, there was one file where the updated
uncrustify caught a problem that the regex didn't pick up, which is updated
in this commit.
Signed-off-by: David Lechner <david@pybricks.com>
MicroPython already requires contributors to implicitly sign-off on a set
of points, which are listed in CODECONVENTIONS.md.
This commit adjusts this wording to allow explicit sign-off using the git
"Signed-off-by:" feature. There is no reference made to
https://developercertificate.org/ because the project already has its own
version of this.
Signed-off-by: Damien George <damien@micropython.org>
Zephyr renamed CONFIG_FLOAT to CONFIG_FPU to better reflect its semantics
of enabling the hardware floating point unit (FPU) rather than enabling
toolchain-level floating point support (i.e., software floating point for
FPU-less socs).
Zephyr deprecated and then removed its stack_analyze function because it
was unsafe. Use the new zephyr thread analyzer instead and rename the
MicroPython function to zephyr.thread_analyze() to be more consistent with
the implementation.
Tested on mimxrt1050_evk.
The output now looks like this:
>>> zephyr.thread_analyze()
Thread analyze:
80004ff4 : unused 400 usage 112 / 512 (21 %)
rx_workq : unused 1320 usage 180 / 1500 (12 %)
tx_workq : unused 992 usage 208 / 1200 (17 %)
net_mgmt : unused 656 usage 112 / 768 (14 %)
sysworkq : unused 564 usage 460 / 1024 (44 %)
idle : unused 256 usage 64 / 320 (20 %)
main : unused 2952 usage 1784 / 4736 (37 %)
This patch adds quickref documentation for the change in commit
afd0701bf7. This commit added the ability to
disable the REPL and hence use UART0 for serial communication on the
esp8266, but was not previously documented anywhere.
The text is largely taken from the commit message, with generic information
on using the UART duplicated from the Wipy quickref document.
For example, to run the BLE multitests entirely with the unix port:
env MICROPY_MICROPYTHON=../ports/unix/micropython-dev ./run-multitests.py \
-i micropython,MICROPYBTUSB=01 \
-i micropython,MICROPYBTUSB=02:02 \
multi_bluetooth/ble_*.py
Long ago, prior to 0ef01d0a75, fixed and
ordered maps were the same setting with the "table_is_fixed_array" member
of mp_map_t. But these settings are actually independent, and it is
possible to have is_fixed=1, is_ordered=0 (although this can currently
only be done by tools/cc1). So update the comments to reflect this.
The resulting dict is now marked as read-only (is_fixed=1) to enforce the
fact that changes to this dict will not be reflected in the class instance.
This commit reduces code size by about 20 bytes, and should be more
efficient because it creates a direct copy of the dict rather than
reinserting all elements.
The behavior mirrors the instance object dict attribute where a copy of the
local attributes are provided (unless the dict is read-only, then that dict
itself is returned, as an optimisation). MicroPython does not support
modifying this dict because the changes will not be reflected in the class.
The feature is only enabled if MICROPY_CPYTHON_COMPAT is set, the same as
the instance version.
Sending more than 64 bytes to the USB CDC endpoint in HS mode will lead to
a hard crash. This commit fixes the issue, although there may be a better
fix from upstream TinyUSB in the future.
This enables warnings as errors and fixes all current errors, namely:
- reference to terms in the glossary must now be explicit (:term:)
- method overloads must not be declared as a separate method or must
use :noindex:
- 2 cases where `` should have been used instead of `
With this commit the code should work correctly regardless of the size of
StackType_t (it's actually 1 byte in size for the esp32's custom FreeRTOS).
Fixes issue #6072.
According to Supplement to the Bluetooth Core Specification v8 Part A
1.3.1, to support BR/EDR the code should set the fifth bit (Simultaneous LE
and BR/EDR to Same Device Capable (Controller)) and fourth bit
(Simultaneous LE and BR/EDR to Same Device Capable (Host)) of the flag.
The ring buffer previously used a single unsigned byte field to save the
length, meaning that it would overflow for large characteristic value
responses.
With this commit it now use a 16-bit length instead and has code to
explicitly truncate at UINT16_MAX (although this should be impossible to
achieve in practice).
This commit makes sure that all discovery complete and read/write status
events set the status to zero on success.
The status value will be implementation-dependent on non-success cases.
On btstack there's no status associated with the read result, it comes
through as a separate event. This allows you to detect read failures or
timeouts.
There doesn't appear to be any use for only triggering on specific events,
so it's just easier to number them sequentially. This makes them smaller
values so they take up only 1 byte in the ringbuf, only 1 byte for the
opcode in the bytecode, and makes room for more events.
Also add a couple of new event types that need to be implemented (to avoid
re-numbering later).
And rename _COMPLETE and _STATUS to _DONE for consistency.
In the future the "trigger" keyword argument can be reinstated by requiring
the user to compute the bitmask, eg:
ble.irq(handler, 1 << _IRQ_SCAN_RESULT | 1 << _IRQ_SCAN_DONE)
This commit implements an LED class with rudimentary parts of a pin C API
to support it. The LED class does not yet support setting an intensity.
This LED class is put in the machine module for the time being, until a
better place is found.
One LED is supported on TEENSY40 and MIMXRT1010_EVK boards.
Changes are:
- string0 is no longer built when building for host as the target, because
it'll be provided by the system libc and may in some cases clash with the
system one (eg on OSX).
- mp_int_t/mp_uint_t are defined in terms of intptr_t/uintptr_t to support
both 32-bit and 64-bit builds.
- Configuration values which are the default in py/mpconfig.h are removed
from mpconfigport.h to make the configuration a bit more minimal, eg as
a better starting point for new ports.
This adds a new command line option `-v` to `tools/codeformat.py` to enable
verbose printing of all files that are scanned.
Normally `uncrustify` and `black` are called with the `-q` option so
setting verbose suppresses the `-q` option and on `black` also enables the
`-v` option which makes it print out all file names matching the filter
similar to how `uncrustify` does by default.
The powerpc port can be built with two different UART drivers, so build
both in CI.
The default compiler is now powerpc64le-linux-gnu- so it does not need to
be specified on the command line.
Microwatt may have firmware that places data in r3, which was used to
detect microwatt vs powernv. This breaks the existing probing of the UART
type in this powerpc port.
Instead build only the appropriate UART into the firmware, selected by
passing the option UART=potato or UART=lpc_serial to the Makefile.
A future enhancement would be to parse the device tree and configure
MicroPython based on the settings.
The code previously called rtc_get_reset_reason which is a "raw" reset
cause. The ESP-IDF massages that for the proper reset cause available from
esp_reset_reason.
Fixes issue #5134.
This suppresses the Parsing: <file> as language C lines. This makes
parsing run a bit faster and on CI it makes for less scrolling through logs
(and black already uses the -q option).
Add configuration which otherwise has to be set via the UI so the file is
more self-contained, and remove configuration which is not needed because
it's the same as the default. The major change here is that for a while
now Appveyor has been using Visual Studio 2015 by default while we still
want to support 2013.
Older implementations deal with infinity/negative zero incorrectly. This
commit adds generic fixes that can be enabled by any port that needs them,
along with new tests cases.
Otherwise functions like memset might get optimised to call themselves (eg
with gcc 10). And provide CFLAGS_BUILTIN so these options can be changed
by a port if needed.
Fixes issue #6053.
The linker script was included in the "$^" inputs, causing the build to
fail:
LINK build/firmware.elf
powerpc64le-linux-gnu-ld: error: linker script file 'powerpc.lds' appears multiple times
As a fix the linker script is left as a dependency of the elf, but only the
object files are linked.
The PWM driver uses a double buffer for the PWM timing array, one in
current use and the other one to update when changing duty parameters.
The issue was that once the duty parameters were changed the updated buffer
was applied immediately without synchronising to the start of the PWM
period. By moving the buffer toggling/swapping to the interrupt when the
cycle is done there are no more glitches.
Specifically:
- pca10040: It was already the default.
- microbit: It uses nRF51, has a Cortex-M0, and has additional libraries.
- pca10056: It has USB CDC.
Commit 6cea369b89 updated the TinyUSB
submodule to a version based on nrfx v2.0.0. This commit updates the nrf
port to work with the latest TinyUSB and nrfx v2.0.0.
Because it can confuse older versions of gcc. Instead use the correct
instruction for Thumb vs Thumb-2 (sub vs subs) so the assembler emits the
2-byte instruction.
Related to commit 1aa9ff9141.
Prior to e0905e85a7 it was possible to
disable btree support on build. This patch allows to configure btree
support on make again and also the two new introduced options for FAT and
LFS2 filesystems.
Before this change, any NimBLE error that does not appear in the
ble_hs_err_to_errno_table maps to return code 0, meaning success. If we
miss adding an error code to the table we end up returning success in case
of failure.
Instead, handle the zero case explicitly and default to MP_EIO. This
allows removing the now-redundant MP_EIO entries from the mapping.
This commit allows the user to set/get the GAP device name used by service
0x1800, characteristic 0x2a00. The usage is:
BLE.config(gap_name="myname")
print(BLE.config("gap_name"))
As part of this change the compile-time setting
MICROPY_PY_BLUETOOTH_DEFAULT_NAME is renamed to
MICROPY_PY_BLUETOOTH_DEFAULT_GAP_NAME to emphasise its link to GAP and this
new "gap_name" config value. And the default value of this for the NimBLE
bindings is changed from "PYBD" to "MPY NIMBLE" to be more generic.
This commit fixes the behaviour of socket.getaddrinfo on the ESP32 so it
raises an OSError when the name resolution fails instead of returning a []
or a resolution for 0.0.0.0.
Tests are added (generic and ESP32-specific) to verify behaviour consistent
with CPython, modulo the different types of exceptions per MicroPython
documentation.
The ones that are moved out of iRAM should not need to be there, because
either they call functions in iROM (eg mp_hal_stdout_tx_str), or they are
only ever called from a function in iROM and not from an interrupt (eg
ets_esf_free_bufs).
This frees up about 800 bytes of iRAM.
If the new name start with '/', cur_dir is not prepened any more, so that
the current working directory is respected. And extend the test cases for
rename to cover this functionality.
This change scans for '.', '..' and multiple '/' and normalizes the new
path name. If the resulting path does not exist, an error is raised.
Non-existing interim path elements are ignored if they are removed during
normalization.
This fixes the bug, that stat(filename) would not consider the current
working directory. So if e.g. the cwd is "lib", then stat("main.py") would
return the info for "/main.py" instead of "/lib/main.py".
The zephyr build system supports merging application-level board
configurations, so there is no need to reproduce this functionality in
MicroPython.
If CONF_FILE is not explicitly set, then the zephyr build system looks for
prj.conf in the application directory. Therefore we rename the MicroPython
prj_base.conf to prj.conf.
Furthermore, if the zephyr build system finds boards/$(BOARD).conf in the
application directory, it merges that configuration with prj.conf.
Therefore we rename all the MicroPython board .conf files and move them
into a boards/ directory.
The minimal configuration, prj_minimal.conf, is left in the application
directory because it is used as an explicitly set CONF_FILE in
make-minimal.
On arm64 with CPython:
>>> _thread.stack_size(32*1024)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ValueError: size not valid: 32768 bytes
So increase the CPython value in the test to 512k so it runs on more
systems (on modern Linux the default stack size is usually 8MB).
Constant expression like "2 ** 3" will now be folded, and the special form
"X = const(2 ** 3)" will now compile because the argument to the const is
now a constant.
Fixes issue #5865.
This commit adds several small items to improve the support for OTA
updates on an esp32:
- a partition table for 4MB flash modules that has two OTA partitions ready
to go to do updates
- a GENERIC_OTA board that uses that partition table and that enables
automatic roll-back in the bootloader
- a new esp32.Partition.mark_app_valid_cancel_rollback() class-method to
signal that the boot is successful and should not be rolled back at the
next reset
- an automated test for doing an OTA update
- documentation updates
For ports that have a system malloc which is not garbage collected (eg
unix, esp32), the stream object for the DB must be retained separately to
prevent it from being reclaimed by the MicroPython GC (because the
berkeley-db library uses malloc to allocate the DB structure which stores
the only reference to the stream).
Although in some cases the user code will explicitly retain a reference to
the underlying stream because it needs to call close() on it, this is not
always the case, eg in cases where the DB is intended to live forever.
Fixes issue #5940.
GPIO interrupts can occur when the flash ROM cache is in use and so the
GPIO interrupt handler must be in iRAM. This commit moves the handler to
iRAM, and also moves mp_sched_schedule to iRAM which is called by
pin_intr_handler.
As part of this fix the Pin class can no longer support hard=True in the
Pin.irq() method, because the VM and runtime are too big to put in iRAM.
Fixes#5714.
Explicitly add the repository as upstream and fetch the master commit.
This makes this bare-arm/minimal job more robust when finding the
fork-point of a PR relative to upstream/master (especially for forks of
this repo).
No functionality change is intended with this commit, it just consolidates
the separate implementations of GC helper code to the lib/utils/ directory
as a general set of helper functions useful for any port. This reduces
duplication of code, and makes it easier for future ports or embedders to
get the GC implementation correct.
Ports should now link against gchelper_native.c and either gchelper_m0.s or
gchelper_m3.s (currently only Cortex-M is supported but other architectures
can follow), or use the fallback gchelper_generic.c which will work on
x86/x64/ARM.
The gc_helper_get_sp function from gchelper_m3.s is not really GC related
and was only used by cc3200, so it has been moved to that port and renamed
to cortex_m3_get_sp.
But only when bluetooth is enabled, i.e. if building the dev or coverage
variants, and we have libusb available.
Update travis to match, i.e. specify the variant when doing
`make submodules`.
One can now use `-i micropython` and `-i cpython` to add instances using
the `MICROPYTHON` and `CPYTHON3` variables (which can be overridden by env
vars).
This commit adds full support to the unix port for Bluetooth using the
common extmod/modbluetooth Python bindings. This uses the libusb HCI
transport, which supports many common USB BT adaptors.
This allows user code that inherits from uio.IOBase to return an errno
error code from the user readinto/write function, by returning a negative
value. Eg returning -123 means an errno of 123. This is already how the
custom ioctl works.
This change is made for two reasons:
1. A 3rd-party library (eg berkeley-db-1.xx, axtls) may use the system
provided errno for certain errors, and yet MicroPython stream objects
that it calls will be using the internal mp_stream_errno. So if the
library returns an error it is not known whether the corresponding errno
code is stored in the system errno or mp_stream_errno. Using the system
errno in all cases (eg in the mp_stream_posix_XXX wrappers) fixes this
ambiguity.
2. For systems that have threading the system-provided errno should always
be used because the errno value is thread-local.
For systems that do not have an errno, the new lib/embed/__errno.c file is
provided.
Note: the uncrustify configuration is explicitly set to 'add' instead of
'force' in order not to alter the comments which use extra spaces after //
as a means of indenting text for clarity.
This commit consolidates a number of check_esp_err functions that check
whether an ESP-IDF return code is OK and raises an exception if not. The
exception raised is an OSError with the error code as the first argument
(negative if it's ESP-IDF specific) and the ESP-IDF error string as the
second argument.
This commit also fixes esp32.Partition.set_boot to use check_esp_err, and
uses that function for a unit test.
This commit adds an idf_heap_info(capabilities) method to the esp32 module
which returns info about the ESP-IDF heaps. It's useful to get a bit of a
picture of what's going on when code fails because ESP-IDF can't allocate
memory anymore. Includes documentation and a test.
This is to make the Travis CI size check more robust, by not relying on the
saved firmware from a previous build (which may use a different compiler,
environment, etc) but rather compile both master and the PR and diff them.
This size check now checks both bare-arm and minimal x86-32 builds (before
it just checked minimal Cortex-M build).
Error string compression is not deterministic in certain cases: it depends
on the Python version (whether dicts are ordered by default or not) and
probably also the order files are passed to this script, leading to a
difference in which words are included in the top 128 most common.
The changes in this commit use OrderedDict to keep parsed lines in a known
order, and, when computing how many bytes are saved by a given word, it
uses the word itself to break ties (which would otherwise be "random").
In mboot, the ability to override the USB vendor/product id's was added
back in 5688c9ba09. However, when the main
firmware is turned into a DFU file the default VID/PID are used there.
pydfu.py doesn't care about this but dfu-util does and prevents its use
when the VID/PID don't match.
This commit exposes BOOTLOADER_DFU_USB_VID/PID as make variables, for use
on either command line or mpconfigboard.mk, to set VID/PID in both mboot
and DFU files.
Add -Wdouble-promotion and -Wfloat-conversion for most ports to ban out
implicit floating point conversions, and add extra Travis builds using
MICROPY_FLOAT_IMPL_FLOAT to uncover warnings which weren't found
previously. For the unix port -Wsign-comparison is added as well but only
there since only clang supports this but gcc doesn't.
For combinations of certain versions of glibc and gcc the definition of
fpclassify always takes float as argument instead of adapting itself to
float/double/long double as required by the C99 standard. At the time of
writing this happens for instance for glibc 2.27 with gcc 7.5.0 when
compiled with -Os and glibc 3.0.7 with gcc 9.3.0. When calling fpclassify
with double as argument, as in objint.c, this results in an implicit
narrowing conversion which is not really correct plus results in a warning
when compiled with -Wfloat-conversion. So fix this by spelling out the
logic manually.
When the unix and windows ports use MICROPY_FLOAT_IMPL_FLOAT instead of
MICROPY_FLOAT_IMPL_DOUBLE, the test output has for example
complex(-0.15052, 0.34109) instead of the expected
complex(-0.15051, 0.34109).
Use one decimal place less for the output printing to fix this.
Initially some of these were found building the unix coverage variant on
MacOS because that build uses clang and has -Wdouble-promotion enabled, and
clang performs more vigorous promotion checks than gcc. Additionally the
codebase has been compiled with clang and msvc (the latter with warning
level 3), and with MICROPY_FLOAT_IMPL_FLOAT to find the rest of the
conversions.
Fixes are implemented either as explicit casts, or by using the correct
type, or by using one of the utility functions to handle floating point
casting; these have been moved from nativeglue.c to the public API.
For jobs which run tests multiple times terminate after the first run fails
otherwise the next test run overwrites the previous results, making
--print-failures useless.
Looking at the recent build history the time it takes just to complete the
OSX build is already 12 minutes so make it start early, which brings down
the total build time from about 20 minutes to 14 minutes.
Change mp_uint_t to size_t to match the mp_print_strn_t function prototype.
This fixes a compiler warning when mp_uint_t and size_t are not the same
size.
This commit provides a typedef for mp_rom_error_text_t, and a macro define
for MP_COMPRESSED_ROM_TEXT, when MICROPY_ROM_TEXT_COMPRESSION is disabled.
This simplifies the configuration (it no longer has a special case for
MICROPY_ENABLE_DYNRUNTIME) and makes it work for other cases that don't use
compression (eg examples/embedding). This commit also ensures
MICROPY_ROM_TEXT_COMPRESSION is defined during qstr processing.
Now that error string compression is supported it's more important to have
consistent error string formatting (eg all lowercase English words,
consistent contractions). This commit cleans up some of the strings to
make them more consistent.
This commit adds Loop.new_event_loop() which is used to reset the singleton
event loop. This functionality is put here instead of in Loop.close() to
make it possible to write code that is compatible with CPython.
Because the atomic section starts after checking whether the scheduler
state is pending, it's possible it can become a different state by the time
the atomic section starts.
This is especially likely on ports where MICROPY_BEGIN_ATOMIC_SECTION is
implemented with a mutex (i.e. it might block), but the race exists
regardless, i.e. if a context switch occurs between those two lines.
This macro is used to implement global serialisation, typically by
disabling IRQs. On the unix port, if threading is enabled, use the
existing thread mutex (that protects the thread list structure) for this
purpose. Other places in the code (eg the scheduler) assume this macro
will provide serialisation.
Based on eg 1e6fd9f2b4, it's understood that
the intention for unix builds is that regular builds disable assert, but
the coverage build should set -O0 and enable asserts.
It looks like this didn't work (even before variants were introduced, eg at
v1.11) -- coverage always built with -Os and -DNDEBUG.
This commit makes it possible for variants to have finer-grained control
over COPT flags, and enables assert() and -O0 on coverage builds.
Other variants already match the defaults so they have been updated.
TimeoutError was added back in 077812b2ab for
the cc3200 port. In f522849a4d the cc3200
port enabled use of it in the socket module aliased to socket.timeout. So
it was never added to the builtins. Then it was replaced by
OSError(ETIMEDOUT) in 047af9b10b.
The esp32 port enables this exception, since the very beginning of that
port, but it could never be accessed because it's not in builtins.
It's being removed: 1) to not encourage its use; 2) because there are a lot
of other OSError subclasses which are not defined at all, and having
TimeoutError is a bit inconsistent.
Note that ports can add anything to the builtins via MICROPY_PORT_BUILTINS.
And they can also define their own exceptions using the
MP_DEFINE_EXCEPTION() macro.
In this part of the code there is no way to get the ** operator, so no need
to check for it.
This commit also adds tests for this, and other related, invalid const
operations.
Recent builds are failing with the following error:
Error: pkg-config 0.29.2 is already installed
Assuming this will be the case form now on, we don't have to install
pkgconfig anymore.
The latest version of BTstack has a bug fixed so that it correctly
configures scan parameters if they are set right after activating the
stack. This means that BLE.gap_scan() will correctly set the scanning to
passive and so SCAN_RSP events are not passed through, so we don't need to
explicitly filter them in our bindings.
Making it more specific to use 0x02 for display with an aspect ratio > 2
(resolutions 96x16 and 128x32) and 0x12 for all other sizes as recommended
by @mcauser. Tested with a 64x32 display which did not work before.
The decompression of error-strings is only done if the string is accessed
via printing or via er.args. Tests are added for this feature to ensure
the decompression works.
The idea here is that there's a moderate amount of ROM used up by exception
text. Obviously we try to keep the messages short, and the code can enable
terse errors, but it still adds up. Listed below is the total string data
size for various ports:
bare-arm 2860
minimal 2876
stm32 8926 (PYBV11)
cc3200 3751
esp32 5721
This commit implements compression of these strings. It takes advantage of
the fact that these strings are all 7-bit ascii and extracts the top 128
frequently used words from the messages and stores them packed (dropping
their null-terminator), then uses (0x80 | index) inside strings to refer to
these common words. Spaces are automatically added around words, saving
more bytes. This happens transparently in the build process, mirroring the
steps that are used to generate the QSTR data. The MP_COMPRESSED_ROM_TEXT
macro wraps any literal string that should compressed, and it's
automatically decompressed in mp_decompress_rom_string.
There are many schemes that could be used for the compression, and some are
included in py/makecompresseddata.py for reference (space, Huffman, ngram,
common word). Results showed that the common-word compression gets better
results. This is before counting the increased cost of the Huffman
decoder. This might be slightly counter-intuitive, but this data is
extremely repetitive at a word-level, and the byte-level entropy coder
can't quite exploit that as efficiently. Ideally one would combine both
approaches, but for now the common-word approach is the one that is used.
For additional comparison, the size of the raw data compressed with gzip
and zlib is calculated, as a sort of proxy for a lower entropy bound. With
this scheme we come within 15% on stm32, and 30% on bare-arm (i.e. we use
x% more bytes than the data compressed with gzip -- not counting the code
overhead of a decoder, and how this would be hypothetically implemented).
The feature is disabled by default and can be enabled by setting
MICROPY_ROM_TEXT_COMPRESSION at the Makefile-level.
This commit makes all functions and function wrappers in modubinascii.c
STATIC and conditional on the MICROPY_PY_UBINASCII setting, which will
exclude the file from qstr/ compressed-string searching when ubinascii is
not enabled. The now-unused modubinascii.h header file is also removed.
The cc3200 port is updated accordingly to use this module in its entirety
instead of providing its own top-level definition of ubinascii.
This was originally like this because the cc3200 port has its own ubinascii
module which referenced these methods. The plan appeared to be that the
API might diverge (e.g. hardware crc), but this should be done similar to
I2C/SPI via a port-specific handler, rather than the port having its own
definition of the module. Having a centralised module definition also
enforces consistency of the API among ports.
Instead of compiler-level if-logic. This is necessary to know what error
strings are included in the build at the preprocessor stage, so that string
compression can be implemented.
This commit changes the default filesystem type for esp32 to littlefs v2.
This port already enables both VfsFat and VfsLfs2, so either can be used
for the filesystem, and existing systems that use FAT will still work.
This commit changes the esp8266 boards to use littlefs v2 as the
filesystem, rather than FAT. Since the esp8266 doesn't expose the
filesystem to the PC over USB there's no strong reason to keep it as FAT.
Littlefs is smaller in code size, is more efficient in use of flash to
store data, is resilient over power failure, and using it saves about 4k of
heap RAM, which can now be used for other things.
This is a backwards incompatible change because all existing esp8266 boards
will need to update their filesystem after installing new firmware (eg
backup old files, install firmware, restore files to new filesystem).
As part of this commit the memory layout of the default board (GENERIC) has
changed. It now allocates all 1M of memory-mapped flash to the firmware,
so the filesystem area starts at the 2M point. This is done to allow more
frozen bytecode to be stored in the 1M of memory-mapped flash. This
requires an esp8266 module with 2M or more of flash to work, so a new board
called GENERIC_1M is added which has the old memory-mapping (but still
changed to use littlefs for the filesystem).
In summary there are now 3 esp8266 board definitions:
- GENERIC_512K: for 512k modules, doesn't have a filesystem.
- GENERIC_1M: for 1M modules, 572k for firmware+frozen code, 396k for
filesystem (littlefs).
- GENERIC: for 2M (or greater) modules, 968k for firmware+frozen code,
1M+ for filesystem (littlefs), FAT driver also included in firmware for
use on, eg, external SD cards.
This commit adds support for global exception handling in uasyncio
according to the CPython error handling:
https://docs.python.org/3/library/asyncio-eventloop.html#error-handling-api
This allows a program to receive exceptions from detached tasks and log
them to an appropriate location, instead of them being printed to the REPL.
The implementation preallocates a context dictionary so in case of an
exception there shouldn't be any RAM allocation.
The approach here is compatible with CPython except that in CPython the
exception handler is called once the task that threw an uncaught exception
is freed, whereas in MicroPython the exception handler is called
immediately when the exception is thrown.
Following up to 5e6cee07ab, some systems (eg
FreeBSD 12.0 64-bit) will crash if the stack-overflow margin is too small.
It seems the margin of 8192 bytes (or thereabouts) is always needed. This
commit adds this much margin if the requested stack size is too small.
Fixes issue #5824.
This adds a couple of commands to the run-tests script to print the diffs
of failed tests and also to clean up the .exp and .out files after failed
tests. (And a spelling error is fixed while we are touching nearby code.)
Travis is also updated to use these new commands, including using it for
more builds.
Since automatically formatting tests with black, we have lost one line of
code coverage. This adds an explicit test to ensure we are testing the
case that is no longer covered implicitly.
This adds the Python files in the tests/ directory to be formatted with
./tools/codeformat.py. The basics/ subdirectory is excluded for now so we
aren't changing too much at once.
In a few places `# fmt: off`/`# fmt: on` was used where the code had
special formatting for readability or where the test was actually testing
the specific formatting.
These were found by buiding the unix coverage variant on macOS (so clang
compiler). Mostly, these are fixing implicit cast of float/double to
mp_float_t which is one of those two and one mp_int_t to size_t fix for
good measure.
These are mainly used by the previous version of uasyncio which is now
replaced by a newer version, with built-in C module _uasyncio. Saves about
1300 bytes of flash.
https://www.python.org/dev/peps/pep-0475/
This implements something similar to PEP 475 on the unix port, and for the
VfsPosix class.
There are a few differences from the CPython implementation:
- Since we call mp_handle_pending() between any ENITR's, additional
functions could be called if MICROPY_ENABLE_SCHEDULER is enabled, not
just signal handlers.
- CPython only handles signal on the main thread, so other threads will
raise InterruptedError instead of retrying. On MicroPython,
mp_handle_pending() will currently raise exceptions on any thread.
A new macro MP_HAL_RETRY_SYSCALL is introduced to reduce duplicated code
and ensure that all instances behave the same. This will also allow other
ports that use POSIX-like system calls (and use, eg, VfsPosix) to provide
their own implementation if needed.
The stack size adjustment for detecting stack overflow in threads was not
taking into account that the requested stack size could be <= 8k, in which
case the subtraction would overflow. This is fixed in this commit by
ensuring that the adjustment can't be more than the available size.
This fixes the test tests/thread/thread_stacksize1.py which sometimes
crashes with a segmentation fault because of an uncaught NLR jump, which is
a "maximum recursion depth exceeded" exception.
Suggested-by: @dpgeorge
Implements Task and TaskQueue classes in C, using a pairing-heap data
structure. Using this reduces RAM use of each Task, and improves overall
performance of the uasyncio scheduler.
Includes a test where the (non uasyncio) client does a RST on the
connection, as a simple TCP server/client test where both sides are using
uasyncio, and a test for TCP stream close then write.
This commit adds a completely new implementation of the uasyncio module.
The aim of this version (compared to the original one in micropython-lib)
is to be more compatible with CPython's asyncio module, so that one can
more easily write code that runs under both MicroPython and CPython (and
reuse CPython asyncio libraries, follow CPython asyncio tutorials, etc).
Async code is not easy to write and any knowledge users already have from
CPython asyncio should transfer to uasyncio without effort, and vice versa.
The implementation here attempts to provide good compatibility with
CPython's asyncio while still being "micro" enough to run where MicroPython
runs. This follows the general philosophy of MicroPython itself, to make it
feel like Python.
The main change is to use a Task object for each coroutine. This allows
more flexibility to queue tasks in various places, eg the main run loop,
tasks waiting on events, locks or other tasks. It no longer requires
pre-allocating a fixed queue size for the main run loop.
A pairing heap is used to queue Tasks.
It's currently implemented in pure Python, separated into components with
lazy importing for optional components. In the future parts of this
implementation can be moved to C to improve speed and reduce memory usage.
But the aim is to maintain a pure-Python version as a reference version.
To enable lazy loading of submodules (among other things), which is very
useful for MicroPython libraries that want to have optional subcomponents.
Disabled explicitly on minimal ports.
This function is not used by the core but having it as part of the build
allows it to be used by user C modules, or board extensions. The linker
won't include it in the final firmware if it remains unused.
On CPython, and with pylint, the variables MATCH_ROM and SEARCH_ROM are
undefined. This code works in MicroPython because these variables are
constants and the MicroPython parser/compiler optimises them out. But it
is not valid Python because they are technically undefined within the scope
they are used.
This commit makes the code valid Python code. The const part is removed
completely because these constants are part of the public API and so cannot
be moved to the global scope (where they could still use the MicroPython
const optimisation).
This removes the port-specific definition of MP_PLAT_PRINT_STRN on the
windows port, so that the default mp_hal_stdout_tx_strn_cooked() is always
used. This fixes releasing the GIL during the call to write() (this was
missed in bc3499f010).
Also, mp_hal_dupterm_tx_strn() was defined but never used anywhere so it is
safe to delete it.
This removes the port-specific definition of MP_PLAT_PRINT_STRN on the unix
port. Since fee7e5617f this is no longer a
single function call so we are not really optimising anything over using
the default definition of MP_PLAT_PRINT_STRN which calls
mp_hal_stdout_tx_strn_cooked().
Zephyr v2.2 reworked its gpio api to support linux device tree bindings and
pin logical levels. This commit updates the zephyr port's machine.Pin
class to replace the deprecated gpio api calls with the new supported gpio
api. This resolves several build warnings.
Tested on frdm_k64f and mimxrt1050_evk boards.
The "led" argument is always a pointer to the GPIO port, or'd with the pin
that the LED is on, so testing that it is "1" is unnecessary. The type of
"led" is also changed to uint32_t so it can properly hold a 32-bit pointer.
Updating the LED0 state from systick handler ensures LED0 is always
consistent with its flash rate regardless of other processing going on in
either interrupts or main. This improves the visible stability of the
bootloader, rather than LED0 flashing somewhat randomly at times.
This commit also changes the LED0 flash rate depending on the current state
of DFU, giving slightly more visual feedback on what the device is doing.
Also support MP_STREAM_GET_FILENO ioctl. The stdio flush change was done
previously for the unix port in 3e0b46b9af.
These changes make this POSIX file implementation equivalent to the unix
file implementation.
Fixes UDP non-blocking recv so it returns EAGAIN instead of ETIMEDOUT.
Timeout waiting for incoming data is also improved by replacing 100ms delay
with poll_sockets(), as is done in other parts of this module.
Fixes issue #5759.
Adds support in the zephyr port to execute main.py if the file system is
enabled and the file exists. Existing support for executing a main.py
frozen module is preserved, since pyexec_file_if_exists() works just
like pyexec_frozen_module() if there's no vfs.
Enables the zephyr usb device stack and mass storage class on the
mimxrt1050_evk board. The mass storage class is backed by the sdhc disk
access driver, so it's now possible to browse and modify the contents of
the SD card from a USB host (your PC). This is in preparation to support
writing a main.py script to the SD card, and then executing it after the
next reset.
Adds support in the zephyr port to mount a file system if a block device
(sdhc disk access or flash area) is available. The mount point is either
"/sd" or "/flash" depending on the type of block device.
Tested with an sdhc disk access block device and fatfs on the
mimxrt1050_evk board.
Tested with a flash area block device and littlefs on the reel_board.
This commit adds micropython.heap_locked() which returns the current
lock-depth of the heap, and can be used by Python code to check if the heap
is locked or not. This new function is configured via
MICROPY_PY_MICROPYTHON_HEAP_LOCKED and is disabled by default.
This commit also changes the return value of micropython.heap_unlock() so
it returns the current lock-depth as well.
This is an extremely minimal port to the NXP i.MX RT, in the style of the
SAMD port It's largely based on the TinyUSB mimxrt implementation, using
the NXP SDK. It currently supports the Teensy 4.0 board with a REPL over
the USB-VCP interface.
This commit also adds the NXP SDK submodule (also from TinyUSB) to
lib/nxp_driver.
Note: if you already have the tinyusb submodule initialized recursively you
will need to run the following as the tinyusb sub-submodules have been
rearranged (upstream):
git submodule deinit lib/tinyusb
rm -rf .git/modules/lib/tinyusb
git submodule update --init lib/tinyusb
This eliminates the need for the sizeof regex fixup by rearranging things a
bit. All other bitfields already use the parentheses around expressions
with sizeof, so one case is fixed by following this convention.
VM_MAX_STATE_ON_STACK is the only remaining problem and it can be worked
around by changing the order of the operands.
The double-% was added in 11de8399fe (Jun
2014) when such errors were formatted with printf. But then
55830dd9bf (Dec 2018) changed
mp_obj_new_exception_msg() to not format the message, as discussed
in #3004. So such error strings are no longer formatted and a % is just
that.
This commit changes the BLE _IRQ_SCAN_RESULT data from:
addr_type, addr, connectable, rssi, adv_data
to:
addr_type, addr, adv_type, rssi, adv_data
This allows _IRQ_SCAN_RESULT to handle all scan result types (not just
connectable and non-connectable passive scans), and to distinguish between
them using adv_type which is an integer taking values 0x00-0x04 per the BT
specification.
This is a breaking change to the API, albeit a very minor one: the existing
connectable value was a boolean and True now becomes 0x00, False becomes
0x02.
Documentation is updated and a test added.
Fixes#5738.
This commit ensures that the BLE stack is active before allowing operations
that may otherwise crash if it's not active. It also clarifies the state
better (adding the "stopping" state) and renames mp_bluetooth_is_enabled to
the more self-explanatory mp_bluetooth_is_active.
This commit adds a test runner and initial test scripts which run multiple
Python/MicroPython instances (eg executables, target boards) in parallel.
This is useful for testing, eg, network and Bluetooth functionality.
Each test file has a set of functions called instanceX(), where X ranges
from 0 up to the maximum number of instances that are needed, N-1. Then
run-multitests.py will execute this script on N separate instances (eg
micropython executables, or attached boards via pyboard.py) at the same
time, synchronising their start in the right order, possibly passing IP
address (or other address like bluetooth MAC) from the "server" instance to
the "client" instances so they can connect to each other. It then runs
them to completion, collects the output, and then tests against what
CPython gives (or what's in a provided .py.exp file).
The tests will be run using the standard unix executable for all instances
by default, eg:
$ ./run-multitests.py multi_net/*.py
Or they can be run with a board and unix executable via:
$ ./run-multitests.py --instance pyb:/dev/ttyACM0 --instance exec:micropython multi_net/*.py
This makes a cleaner separation between the: driver, HCI UART and BT stack.
Also updated the naming to be more consistent (mp_bluetooth_hci_*).
Work done in collaboration with Jim Mussared aka @jimmo.
Move extmod/modbluetooth_nimble.* to extmod/nimble. And move common
Makefile lines to extmod/nimble/nimble.mk (which was previously only used
by stm32). This allows (upcoming) btstack to follow a similar structure.
Work done in collaboration with Jim Mussared aka @jimmo.
When using a manifest on Windows the reference to mpy-cross compiler was
missing the .exe file extension, so add it when appropriate.
Also allow the default path to mpy-cross to be overridden by the (optional)
MICROPY_MPYCROSS environment variable, to allow full flexibility on any OS.
sys.stdout.flush() is needed on CPython to flush the output, and the change
in this commit makes such an expression also work on MicroPython (although
MicroPython doesn't actual need to do any flushing).
The CI job will fail if there is any code which does not conform to the
style encoded by tools/codeformat.py. And it will list any changes
required.
uncrustify is built from source because Ubuntu bionic has uncrustify-0.66.1
which is from 2017/11/22 and is missing many options needed here.
This string is recognised by uncrustify, to disable formatting in the
region marked by these comments. This is necessary in the qstrdef*.h files
to prevent modification of the strings within the Q(...). In other places
it is used to prevent excessive reformatting that would make the code less
readable.
This commit adds a tool, codeformat.py, which will reformat C and Python
code to fit a certain style. By default the tool will reformat (almost)
all the original (ie not 3rd-party) .c, .h and .py files in this
repository. Passing filenames on the command-line to codeformat.py will
reformat only those. Reformatting is done in-place.
uncrustify is used for C reformatting, which is available for many
platforms and can be easily built from source, see
https://github.com/uncrustify/uncrustify. The configuration for uncrustify
is also added in this commit and values are chosen to best match the
existing code style. A small post-processing stage on .c and .h files is
done by codeformat.py (after running uncrustify) to fix up some minor
items:
- space inserted after * when used as multiplication with sizeof
- #if/ifdef/ifndef/elif/else/endif are dedented by one level when they are
configuring if-blocks and case-blocks.
For Python code, the formatter used is black, which can be pip-installed;
see https://github.com/psf/black. The defaults are used, except for line-
length which is set at 99 characters to match the "about 100" line-length
limit used in C code.
The formatting tools used and their configuration were chosen to strike a
balance between keeping existing style and not changing too many lines of
code, and enforcing a relatively strict style (especially for Python code).
This should help to keep the code consistent across everything, and reduce
cognitive load when writing new code to match the style.
And rename it to mp_obj_cast_to_native_base() to indicate this. This
allows users of this function to easily support native and native-subclass
objects in the same way (by just passing the object through this function).
Since commit 3aab54bf43 this piece of code is
no longer needed because the top-level function mp_obj_equal_not_equal()
now handles the case of user types, and will never call tuple's binary_op
function with MP_BINARY_OP_EQUAL and a non-tuple on the RHS.
Only the "==" operator was tested by the test suite in for such arguments.
Other comparison operators like "<" take a different path in the code so
need to be tested separately.
If the built-in input() is enabled (which it is by default) then it needs
some form of readline, so supply it with one when MICROPY_USE_READLINE=0.
Fixes issue #5658.
Follow up to recent commit ad7213d3c3, the
name "varg2" is misleading, vlist describes better that the argument is a
va_list. This name also matches CircuitPython, which already has such
helper functions.
This changes the signal used to trigger garbage collection from SIGUSR1 to
SIGRTMIN + 5. SIGUSR1 is quite common compared to SIGRTMIN (measured by
google search results) and is more likely to conflict with libraries that
may use the same signal.
POSIX specifies that there are at least 8 real-time signal so 5 was chosen
as a "random" number to further avoid potential conflict with libraries
that may use SIGRTMIN or SIGRTMAX.
Also, if we ever have a `usignal` module, it would be nice to leave SIGUSR1
and SIGUSR2 free for user programs.
The "random" module no longer uses the hardware RNG (the extmod version of
this module has a pseudo-random number generator), so the config option
MICROPY_PY_RANDOM_HW_RNG is no longer meaningful. This commit replaces it
with MICROPY_HW_ENABLE_RNG, which controls whether the hardware RNG is
included in the build.
The install target is current broken when PROG is used to override the
default executable name. This fixes it by removing the redundant TARGET
variable and uses PROG directly instead.
The install and uninstall targets are also moved to the common unix
Makefile so that all variants can be installed in the same way.
Currently it is not possible to override PREFIX when installing micropython
using the makefile. It is common practice to be able to run something like
this:
$ make install PREFIX=/usr DESTDIR=/tmp/staging
This fixes such usage.
These addresses were initially chosen to match the nRF24 Arduino library
examples but they are byte-reversed. So change them to be on-air
compatible with the Arduino library.
Also, the data sheet for the nRF24 says that RX data pipes 1-5 must share
the same top 32-bits, and must differ only in the LSbyte. The addresses
used here (while correct because they are on TX pipe and RX pipe 0) are
misleading in this sense, because it looks like they were chosen to share
the top 32-bits per the datasheet.
This provides a more consistent C-level API to raise exceptions, ie moving
away from nlr_raise towards mp_raise_XXX. It also reduces code size by a
small amount on some ports.
The default value for MICROPYPATH used in unix/main.c is
"~/.micropython/lib:/usr/lib/micropython" which has 2 problems when used in
the Windows port:
- it has a ':' as path separator but the port uses ';' so the entire string
is effectively discarded since it gets interpreted as a single path which
doesn't exist
- /usr/lib/micropython is not a valid path in a standard Windows
environment
Override the value with a suitable default.
If the exception doesn't need printf-style formatting then calling
mp_raise_msg is more efficient. Also shorten exception messages to match
style in core and other ports.
Both bool and namedtuple will check against other types for equality; int,
float and complex for bool, and tuple for namedtuple. So to make them work
after the recent commit 3aab54bf43 they would
need MP_TYPE_FLAG_NEEDS_FULL_EQ_TEST set. But that makes all bool and
namedtuple equality checks less efficient because mp_obj_equal_not_equal()
could no longer short-cut x==x, and would need to try __ne__. To improve
this, this commit splits the MP_TYPE_FLAG_NEEDS_FULL_EQ_TEST flags into 3
separate flags to give types more fine-grained control over how their
equality behaves. These new flags are then used to fix bool and namedtuple
equality.
Fixes issue #5615 and #5620.
This fix can be demonstrated by the following:
b = bytearray(32)
f = framebuf.FrameBuffer(b, 32, 8, framebuf.MONO_HLSB)
f.pixel(0, 0, 1)
print('MONO_HLSB', hex(b[0]))
b = bytearray(32)
f = framebuf.FrameBuffer(b, 32, 8, framebuf.MONO_HMSB)
f.pixel(0, 0, 1)
print('MONO_HMSB', hex(b[0]))
Outcome:
MONO_HLSB 0x80
MONO_HMSB 0x1
It's not needed. The C integer implicit promotion rules mean that the
uint8_t of the incoming character is promoted to a (signed) int, matching
the type of interrupt_char. Thus the uint8_t incoming character can never
be equal to -1 (the value of interrupt_char that indicate that interruption
is disabled).
The mp_keyboard_interrupt() function does exactly what is needed here, and
using it gets ctrl-C working when MICROPY_ENABLE_SCHEDULER is enabled on
these ports (and MICROPY_ASYNC_KBD_INTR is disabled).
This is a more logical place to clear the KeyboardInterrupt traceback,
right before it is set as a pending exception. The clearing is also
optimised from a function call to a simple store of NULL.
It was originally in IRAM due to the linker script specification, but
since the function moved from lib/utils/interrupt_char.c to py/scheduler.c
it needs to be put back in IRAM.
Functions like mp_keyboard_interrupt() may need to be called from an IRQ
handler and may need to be in a special memory section, so provide a
generic wrapping macro for a port to do this. The macro name is chosen to
be MICROPY_WRAP_<function name in uppercase> so that (in the future with
more wrappers) each function could potentially be handled separately.
This function is tightly coupled to the state and behaviour of the
scheduler, and is a core part of the runtime: to schedule a pending
exception. So move it there.
Pending exceptions would otherwise be handled later on where there may not
be an NLR handler in place.
A similar fix is also made to the unix port's REPL handler.
Fixes issues #4921 and #5488.
Previous behaviour is when this argument is set to "true", in which case
the function will raise any pending exception. Setting it to "false" will
cancel any pending exception.
Enables the littlefs (v1 and v2) filesystems in the zephyr port.
Example usage with the internal flash on the reel_board or the
rv32m1_vega_ri5cy board:
import os
from zephyr import FlashArea
bdev = FlashArea(FlashArea.STORAGE, 4096)
os.VfsLfs2.mkfs(bdev)
os.mount(bdev, '/flash')
with open('/flash/hello.txt','w') as f:
f.write('Hello world')
print(open('/flash/hello.txt').read())
Things get a little trickier with the frdm_k64f due to the micropython
application spilling into the default flash storage partition defined
for this board. The zephyr build system doesn't enforce the flash
partitioning when mcuboot is not enabled (which it is not for
micropython). For now we can demonstrate that the littlefs filesystem
works on frdm_k64f by constructing the FlashArea block device on the
mcuboot scratch partition instead of the storage partition. Do this by
replacing the FlashArea.STORAGE constant above with the value 4.
Introduces a new zephyr.FlashArea class that uses the zephyr flash map
api to implement the uos.AbstractBlockDev protocol. The flash driver is
enabled on the frdm_k64f board, reel_board, and rv32m1_vega_ri5cy board.
The standard and extended block device protocols are both supported,
therefore this class can be used with file systems like littlefs which
require the extended interface.
Enables the fatfs filesystem in the zephyr port.
Example usage with an SD card on the mimxrt1050_evk board:
import zephyr, os
bdev = zephyr.DiskAccess('SDHC')
os.VfsFat.mkfs(bdev)
os.mount(bdev, '/sd')
with open('/sd/hello.txt','w') as f:
f.write('Hello world')
print(open('/sd/hello.txt').read())
Introduces a new zephyr.DiskAccess class that uses the zephyr disk
access api to implement the uos.AbstractBlockDev protocol. This can be
used with any type of zephyr disk access driver, which currently
includes SDHC, RAM, and FLASH implementations. The SDHC driver is
enabled on the mimxrt1050_evk board.
Only the standard block device protocol (without the offset parameter)
can be supported with the zephyr disk access api, therefore this class
cannot be used with file systems like littlefs which require the
extended interface. In the future it may be possible to implement the
extended interface in a new class using the zephyr flash api.
A 'return' statement on module/class level is not correct Python, but
nothing terribly bad happens when it's allowed. So remove the check unless
MICROPY_CPYTHON_COMPAT is on.
This is similar to MicroPython's treatment of 'import *' in functions
(except 'return' has unsurprising behavior if it's allowed).
By simply reordering the enums for pyexec_mode_kind_t it eliminates a data
variable which costs ROM to initialise it. And the minimal build now has
nothing in the data section.
It seems the compiler is smart enough so that the generated code for
if-logic which tests these enum values is unchanged.
When this variable is set to non-empty string it triggers the REPL after a
command/module/file finishes running.
The Python file without the file extension is because the cmdline: parser
in run-test splits on spaces, so we can't use the -c option since
`import os` can't be written without a space.
CPython also has os.environ, which should be used instead of os.getenv()
due to caching in the os.environ mapping. But for MicroPython it makes
sense to only implement the basic underlying methods, ie getenv/putenv/
unsetenv.
This adds a -h option to print the usage help text and adds a new, shorter
error message that is printed when invalid arguments are given. This
behaviour follows CPython (and other tools) more closely.
This commit modifies the usage() function to only print the -v option help
text when MICROPY_DEBUG_PRINTERS is enabled. The -v option requires this
build option to be enabled for it to have any effect.
The usage text is also modified to show the -i and -m options, and also
show that running a command, module or file are mutually exclusive.
This adds support for a MICROPYINSPECT environment variable that works
exactly like PYTHONINSPECT; per CPython docs:
If this is set to a non-empty string it is equivalent to specifying the
-i option.
This variable can also be modified by Python code using os.environ to
force inspect mode on program termination.
Zephyr removed the build target syscall_macros_h_target in commit
f4adf107f31674eb20881531900ff092cc40c07f. Removes reference from
MicroPython to fix build errors in the zephyr port.
This change is not compatible with zephyr v2.1 or earlier. It will be
compatible with Zephyr v2.2 when released.
The SYS_CLOCK_HW_CYCLES_TO_NS macro was deprecated in zephyr commit
8892406c1de21bd5de5877f39099e3663a5f3af1. This commit updates MicroPython
to use the new k_cyc_to_ns_floor64 api and fix build warnings in the zephyr
port.
This change is compatible with Zephyr v2.1 and later.
Zephyr restructured its includes in v2.0 and removed compatibility shims
after two releases in commit 1342dadc365ee22199e51779185899ddf7478686.
Updates include paths in MicroPython accordingly to fix build errors in
the zephyr port.
These changes are compatible with Zephyr v2.0 and later.
Show how to send an HTTP response code and content-type. Without the
response code Safari/iOS will fail. Without the content-type Lynx/Links
will fail.
When stdout is redirected it is useful to have errors printed to stderr
instead of being redirected.
mp_stderr_print() can't be used in these two instances since the
MicroPython runtime is not running so we use fprintf(stderr) instead.
This option makes pyboard.py exit as soon as the script/command is
successfully sent to the device, ie it does not wait for any output. This
can help to avoid hangs if the board is being rebooted with --comman (for
example).
Example usage:
$ python3 ./tools/pyboard.py --device /dev/ttyUSB0 --no-follow \
--command 'import machine; machine.reset()'
The ability to change the host is a frequently requested feature, so
explicitly document how it can be achieved using the existing code.
See issues #2121, #4385, #4622, #5122, #5536.
This commit improves pllvalues.py to generate PLL values for H7 MCUs that
are valid (VCO in and out are in range) and extend for the entire range of
SYSCLK values up to 400MHz (up to 480MHz is currently unsupported).
This commit implements a more complete replication of CPython's behaviour
for equality and inequality testing of objects. This addresses the issues
discussed in #5382 and a few other inconsistencies. Improvements over the
old code include:
- Support for returning non-boolean results from comparisons (as used by
numpy and others).
- Support for non-reflexive equality tests.
- Preferential use of __ne__ methods and MP_BINARY_OP_NOT_EQUAL binary
operators for inequality tests, when available.
- Fallback to op2 == op1 or op2 != op1 when op1 does not implement the
(in)equality operators.
The scheme here makes use of a new flag, MP_TYPE_FLAG_NEEDS_FULL_EQ_TEST,
in the flags word of mp_obj_type_t to indicate if various shortcuts can or
cannot be used when performing equality and inequality tests. Currently
four built-in classes have the flag set: float and complex are
non-reflexive (since nan != nan) while bytearray and frozenszet instances
can equal other builtin class instances (bytes and set respectively). The
flag is also set for any new class defined by the user.
This commit also includes a more comprehensive set of tests for the
behaviour of (in)equality operators implemented in special methods.
This board now has the following 3 build configurations:
- mboot + external QSPI in XIP mode + internal filesystem
- mboot + external QSPI with filesystem (the default)
- no mboot + external QSPI with filesystem
With a SPI flash that has more than 16MB, 32-bit addressing is required
rather than the standard 24-bit. This commit adds support for 32-bit
addressing so that the SPI flash commands (read/write/erase) are selected
automatically depending on the size of the address being used at each
operation.
This modifies the signature of mp_thread_set_state() to use
mp_state_thread_t* instead of void*. This matches the return type of
mp_thread_get_state(), which returns the same value.
`struct _mp_state_thread_t;` had to be moved before
`#include <mpthreadport.h>` since the stm32 port uses it in its
mpthreadport.h file.
PLLM is shared among all PLL blocks on F7 MCUs, and this calculation to
configure PLLSAI to have 48MHz on the P output previously assumed that PLLM
is equal to HSE (eg PLLM=25 for HSE=25MHz). This commit relaxes this
assumption to allow other values of PLLM.
The loop searches backwards for a target, but doesn't stop after finding
the first result, meaning that it'll always end up at the outermost
exception handler.
This previously made the native emitter incompatible with the bytecode
emitter, and mp_resume (and subsequently mp_obj_generator_resume) expects
the bytecode emitter behavior (i.e. throw==NULL).
This commit adds a generator test for throwing into a nested exception, and
one when using yield-from with a pending exception cleanup. Both these
tests currently fail on the native emitter, and are simplified versions of
native test failures from uasyncio in #5332.
It is not safe to enable MICROPY_ASYNC_KBD_INTR and MICROPY_PY_THREAD_GIL
at the same time. This will trigger a compiler error to ensure that it
is not possible to make this mistake.
Addition of GIL EXIT/ENTER pairs are:
- modos: release the GIL during system calls. CPython does this as well.
- moduselect: release the GIL during the poll() syscall. This call can be
blocking, so it is important to allow other threads to run at this time.
- modusocket: release the GIL during system calls. Many of these calls can
be blocking, so it is important to allow other threads to run.
- unix_mphal: release the GIL during the read and write syscalls in
mp_hal_stdin_rx_chr and mp_hal_stdout_tx_strn. If we don't do this
threads are blocked when the REPL or the builtin input function are used.
- file, main, mpconfigport.h: release GIL during syscalls in built-in
functions that could block.
When CFLAGS_EXTRA/LDFLAGS_EXTRA (or anything) is set on the command line of
a make invocation then it will completely override any setting or appending
of these variables in the makefile(s). This means builds like the coverage
variant will have their mpconfigvariant.mk settings overridden. Fix this
by using CFLAGS/LDFLAGS exclusively in the makefile(s), reserving the
CFLAGS_EXTRA/LDFLAGS_EXTRA variables for external command-line use only.
Commit d96cfd13e3 introduced a regression in
testing for bool objects, that such objects were in some cases no longer
recognised and bools, eg when using mp_obj_is_type(o, &mp_type_bool), or
mp_obj_is_integer(o).
This commit fixes that problem by adding mp_obj_is_bool(o). Builds with
MICROPY_OBJ_IMMEDIATE_OBJS enabled check if the object is any of the const
True or False objects. Builds without it use the old method of ->type
checking, which compiles to smaller code (compared with the former
mentioned method).
Fixes#5538.
When threads and the GIL are enabled, then the qstr mutex is not needed.
The qstr_mutex field is never used in this case because of:
#if MICROPY_PY_THREAD && !MICROPY_PY_THREAD_GIL
#define QSTR_ENTER() mp_thread_mutex_lock(&MP_STATE_VM(qstr_mutex), 1)
#define QSTR_EXIT() mp_thread_mutex_unlock(&MP_STATE_VM(qstr_mutex))
#else
#define QSTR_ENTER()
#define QSTR_EXIT()
#endif
So, we can completely remove qstr_mutex everywhere when MICROPY_PY_THREAD
&& !MICROPY_PY_THREAD_GIL.
When threads and the GIL are enabled, then the GC mutex is not needed. The
gc_mutex field is never used in this case because of:
#if MICROPY_PY_THREAD && !MICROPY_PY_THREAD_GIL
#define GC_ENTER() mp_thread_mutex_lock(&MP_STATE_MEM(gc_mutex), 1)
#define GC_EXIT() mp_thread_mutex_unlock(&MP_STATE_MEM(gc_mutex))
#else
#define GC_ENTER()
#define GC_EXIT()
#endif
So, we can completely remove gc_mutex everywhere when MICROPY_PY_THREAD
&& !MICROPY_PY_THREAD_GIL.
Some parts of code have been aligned to increase readability. In general
'' instead of "" were used wherever possible to keep the same convention
for entire file. Import inspect line has been moved to the top according
to hints reported by pep8 tools. A few extra spaces were removed, a few
missing spaces were added. Comments have been updated, mostly in
"read_dfu_file" function. Some other comments have been capitalized and/or
slightly updated. A few docstrings were fixed as well. No real code
changes intended.
Translate common Ctrl-Left/Right/Delete/Backspace to the EMACS-style
sequences (i.e. Alt key based) for forward-word, backward-word, forwad-kill
and backward-kill. Requires MICROPY_REPL_EMACS_WORDS_MOVE to be defined so
the readline implementation interprets these.
Prior to this commit, if the flash filesystem was not formatted then it
would error: "AttributeError: 'FlashBdev' object has no attribute 'mount'".
That is due to it not being able to detect the filesystem on the block
device and just trying to mount the block device directly.
This commit fixes the issue by just catching all exceptions. Also it's not
needed to try the mount if `flashbdev.bdev` is None.
Can be used where mp_obj_int_get_checked() will overflow due to the
sign-bit solely. This returns an mp_uint_t, so it also verifies the given
integer is not negative.
Currently implemented only for mpz configurations.
This function is called often and with immediate objects enabled it has
more cases, so optimise it for speed. With this optimisation the runtime
is now slightly faster with immediate objects enabled than with them
disabled.
This option (enabled by default for object representation A, B, C) makes
None/False/True objects immediate objects, ie they are no longer a concrete
object in ROM but are rather just values, eg None=0x6 for representation A.
Doing this saves a considerable amount of code size, due to these objects
being widely used:
bare-arm: -392 -0.591%
minimal x86: -252 -0.170% [incl +52(data)]
unix x64: -624 -0.125% [incl -128(data)]
unix nanbox: +0 +0.000%
stm32: -1940 -0.510% PYBV10
cc3200: -1216 -0.659%
esp8266: -404 -0.062% GENERIC
esp32: -732 -0.064% GENERIC[incl +48(data)]
nrf: -988 -0.675% pca10040
samd: -564 -0.556% ADAFRUIT_ITSYBITSY_M4_EXPRESS
Thanks go to @Jongy aka Yonatan Goldschmidt for the idea.
This commit adjusts the definition of qstr encoding in all object
representations by taking a single bit from the qstr space and using it to
distinguish between qstrs and a new kind of literal object: immediate
objects. In other words, the qstr space is divided in two pieces, one half
for qstrs and the other half for immediate objects.
There is still enough room for qstr values (29 bits in representation A on
a 32-bit architecture, and 19 bits in representation C) and the new
immediate objects can be used for things like None, False and True.
This moves the MICROPY_PORT_INIT_FUNC hook to the end of mp_init(), just
before MP_THREAD_GIL_ENTER(), so that everything (in particular the GIL
mutex) is intialized before the hook is called. MICROPY_PORT_DEINIT_FUNC
is also moved to be symmetric (but there is no functional change there).
If a port needs to perform initialisation earlier than
MICROPY_PORT_INIT_FUNC then it can do it before calling mp_init().
This commit adds backward-word, backward-kill-word, forward-word,
forward-kill-word sequences for the REPL, with bindings to Alt+F, Alt+B,
Alt+D and Alt+Backspace respectively. It is disabled by default and can be
enabled via MICROPY_REPL_EMACS_WORDS_MOVE.
Further enabling MICROPY_REPL_EMACS_EXTRA_WORDS_MOVE adds extra bindings
for these new sequences: Ctrl+Right, Ctrl+Left and Ctrl+W.
The features are enabled on unix micropython-coverage and micropython-dev.
During readline development, this function may receive bad `pos` values.
It's easier to understand the assert() failing error than to have a "stack
smashing detected" message.
This adds a short paragraph on how to hook readthedocs.org up. The main
goal is to make people aware of the option, to help with contributing to
the documentation.
Invoking "make" will still build the standard "micropython" executable, but
other variants are now build using, eg, "make VARIANT=minimal". This
follows how bare-metal ports specify a particular board, and allows running
any make target (eg clean, test) with any variant.
Convenience targets (eg "make coverage") are provided to retain the old
behaviour, at least for now.
See issue #3043.
Most types are in rodata/ROM, and mp_obj_base_t.type is a constant pointer,
so enforce this const-ness throughout the code base. If a type ever needs
to be modified (eg a user type) then a simple cast can be used.
This change has the following effects:
- Reduces the resolution of the RTC sub-second counter from 30.52us to
122.07us.
- Allows RTC.calibration() to now support positive values (as well as
negative values).
- Reduces VBAT current consumption in standby mode by a small amount.
For general purpose use 122us resolution of the sub-second counter is
good enough, and the benefits of full range calibration and minor reduction
in VBAT consumption are worth the change.
As the mktime documentation for CPython states: "The earliest date for
which it can generate a time is platform-dependent". In particular on
Windows this depends on the timezone so e.g. for UTC+2 the earliest is 2
hours past midnight January 1970. So change the reference to the earliest
possible, for UTC+14.
Make version 4.1 and lower does not allow $call as the main expression on a
line, so assign the result of the $call to a dummy variable.
Fixes issue #5426.
It is possile for `run_feature_check(pyb, args, base_path, 'float.py')` to
return `b'CRASH'`. This causes an unhandled exception in `int()`.
This commit fixes the problem by first testing for `b'CRASH'` before trying
to convert the return value to an integer.
Instances of the slice class are passed to __getitem__() on objects when
the user indexes them with a slice. In practice the majority of the time
(other than passing it on untouched) is to work out what the slice means in
the context of an array dimension of a particular length. Since Python 2.3
there has been a method on the slice class, indices(), that takes a
dimension length and returns the real start, stop and step, accounting for
missing or negative values in the slice spec. This commit implements such
a indices() method on the slice class.
It is configurable at compile-time via MICROPY_PY_BUILTINS_SLICE_INDICES,
disabled by default, enabled on unix, stm32 and esp32 ports.
This commit also adds new tests for slice indices and for slicing unicode
strings.
The existing uos.remove cannot be used to remove directories, instead
uos.rmdir is needed. And also provide uos.rename to get a good set of
filesystem functionality without requiring additional Python-level os
functions (eg using ffi).
In CPython, EnvironmentError and IOError are now aliases of OSError so no
need to have them listed in the code. OverflowError inherits from
ArithmeticError because it's intended to be raised "when the result of an
arithmetic operation is too large to be represented" (per CPython docs),
and MicroPython aims to match the CPython exception hierarchy.
For the 3 ports that already make use of this feature (stm32, nrf and
teensy) this doesn't make any difference, it just allows to disable it from
now on.
For other ports that use pyexec, this decreases code size because the debug
printing code is dead (it can't be enabled) but the compiler can't deduce
that, so code is still emitted.
The qst value is always small enough to fit in 31-bits (even less) and
using a 32-bit shift rather than a 64-bit shift reduces code size by about
300 bytes.
Most stm32 boards can now be built in nan-boxing mode via:
$ make NANBOX=1
Note that if float is enabled then it will be forced to double-precision.
Also, native emitters will be disabled.
A user-defined type that defines __iter__ doesn't need any memory to be
pre-allocated for its iterator (because it can't use such memory). So
optimise for this case by not allocating the iter-buf.
In commit 71a3d6ec3b mp_setup_code_state was
changed from a 5-arg function to a 4-arg function, and at that point 5-arg
calls in native code were no longer needed. See also commit
4f9842ad80.
The struct member "dest" should never be less than "destStart", so their
difference is never negative. Cast as such to make the comparison
explicitly unsigned, ensuring the compiler produces the correct comparison
instruction, and avoiding any compiler warnings.
Allows assigning attributes on class instances that implement their own
__setattr__. Both object.__setattr__ and super(A, b).__setattr__ will work
with this commit.
Move webrepl support code from ports/esp8266/modules into extmod/webrepl
(to be alongside extmod/modwebrepl.c), and use frozen manifests to include
it in the build on esp8266 and esp32.
A small modification is made to webrepl.py to make it work on non-ESP
ports, i.e. don't call dupterm_notify if not available.
This is an alternative to f4ed2df that adds a newline so that the output of
the test starts on a new line and the result of the test is prefixed with
"result: " to distinguish it from the test output.
Suggested-by: @dpgeorge
This reverts commit f4ed2dfa94.
This lets tinytest work as it was originally designed. An alternate
solution for the reverted commit will be implemented in a future commit.
This patch allows executing .mpy files (including native ones) directly on
a target, eg a board over a serial connection. So there's no need to copy
the file to its filesystem to test it.
For example:
$ mpy-cross foo.py
$ pyboard.py foo.mpy
- Corrected pin assignments and checked with CubeMX.
- Added additional I2C and UARTs.
- Added Ethernet interface definitions with lwIP and SSL support (but
Ethernet is currently unsupported on H7 MCUs so not fully enabled).
- Removed remarks on DFU/OCD in mpconfigboard.h because deploy-stlink works
fine too.
- Added more UARTs, I2C, corrected SPI, CAN, etc; verified against CubeMX.
- Adapted pins.csv to remove errors, add omissions, etc. according to
NUCLEO-144 User Manual.
- Changed linker file stm32f767.ld to reflect correct size of the Flash.
- Tested with LAN and SD card.
The Nucleo board does not have an SD card slot but does have the requisite
pins next to each other and labelled, so provide the configuration for
convenience.
This makes the loading of viper-code-with-relocations a bit neater and
easier to understand, by treating the rodata/bss like a special object to
be loaded into the constant table (which is how it behaves).
Because CPython 3.8.0 now produces different output:
- basics/parser.py: CPython does not allow '\\\n' as input.
- import/import_override: CPython imports _io.
We don't want to add a feature flag to .mpy files that indicate float
support because it will get complex and difficult to use. Instead the .mpy
is built using whatever precision it chooses (float or double) and the
native glue API will convert between this choice and what the host runtime
actually uses.
This commit adds a new tool called mpy_ld.py which is essentially a linker
that builds .mpy files directly from .o files. A new header file
(dynruntime.h) and makefile fragment (dynruntime.mk) are also included
which allow building .mpy files from C source code. Such .mpy files can
then be dynamically imported as though they were a normal Python module,
even though they are implemented in C.
Converting .o files directly (rather than pre-linked .elf files) allows the
resulting .mpy to be more efficient because it has more control over the
relocations; for example it can skip PLT indirection. Doing it this way
also allows supporting more architectures, such as Xtensa which has
specific needs for position-independent code and the GOT.
The tool supports targets of x86, x86-64, ARM Thumb and Xtensa (windowed
and non-windowed). BSS, text and rodata sections are supported, with
relocations to all internal sections and symbols, as well as relocations to
some external symbols (defined by dynruntime.h), and linking of qstrs.
Usage:
mpy-tool.py -o merged.mpy --merge mod1.mpy mod2.mpy
The constituent .mpy files are executed sequentially when the merged file
is imported, and they all use the same global namespace.
Implements text, rodata and bss generalised relocations, as well as generic
qstr-object linking. This allows importing dynamic native modules on all
supported architectures in a unified way.
The size of the event ringbuf was previously fixed to compile-time config
value, but it's necessary to sometimes increase this for applications that
have large characteristic buffers to read, or many events at once.
With this commit the size can be set via BLE.config(rxbuf=512), for
example. This also resizes the internal event data buffer which sets the
maximum size of incoming data passed to the event handler.
This allows the user to explicitly select the behaviour of the write to the
remote peripheral. This is needed for peripherals that have
characteristics with WRITE_NO_RESPONSE set (instead of normal WRITE). The
function's signature is now:
BLE.gattc_write(conn_handle, value_handle, data, mode=0)
mode=0 means write without response, while mode=1 means write with
response. The latter was the original behaviour so this commit is a change
in behaviour of this method, and one should specify 1 as the 4th argument
to get back the old behaviour.
In the future there could be more modes supported, such as long writes.
The default protection for the BLE ringbuf is to use
MICROPY_BEGIN_ATOMIC_SECTION, which disables all interrupts. On stm32 it
only needs to disable the lowest priority IRQ, pendsv, because that's the
IRQ level at which the BLE stack is driven.
This removes the limit on data coming in from a BLE.gattc_read() request,
or a notify with payload (coming in to a central). In both cases the data
coming in to the BLE callback is now limited only by the available data in
the ringbuf, whereas before it was capped at (default hard coded) 20 bytes.
Instead of enqueue_irq() inspecting the ringbuf to decide whether to
schedule the IRQ callback (if ringbuf is empty), maintain a flag that knows
if the callback is on the schedule queue or not. This saves about 150
bytes of code (for stm32 builds), and simplifies all uses of enqueue_irq()
and schedule_ringbuf().
qstrs in this file are always included in all builds, even if not used
anywhere. So remove those that are never needed, and make USB names
conditional on having USB enabled.
With the memcpy() call placed last it avoids the effects of registers
clobbering. It's definitely effective in non-inlined functions, but even
here it is still making a small difference. For example, on stm32, this
saves an extra `ldr` instruction to load `o->vstr` after the memcpy()
returns.
The string length being longer than the allowed qstr length can happen in
many locations, for example in the parser with very long variable names.
Without an explicit check that the length is within range (as done in this
patch) the code would exhibit crashes and strange behaviour with truncated
strings.
And return -MP_EIO if calling storage_read_block/storage_write_block fails.
This lines up with the return type and value (negative for error) of the
calls to MICROPY_HW_BDEV_READBLOCKS (and WRITEBLOCKS, and BDEV2 versions).
The pyb.Flash() class can now be used to construct objects which reference
sections of the flash storage, starting at a certain offset and going for a
certain length. Such objects also support the extended block protocol.
The signature for the constructor is: pyb.Flash(start=-1, len=-1).
This commit refactors and generalises the boot-mount routine on stm32 so
that it can mount filesystems of arbitrary type. That is, it no longer
assumes that the filesystem is FAT. It does this by using mp_vfs_mount()
which does auto-detection of the filesystem type.
The address, adv payload and uuid fields of the event are pre-allocated by
modbluetooth, and reused in the IRQ handler. Simplify this and move all
storage into the `mp_obj_bluetooth_ble_t` instance.
This now allows users to hold on to a reference to these instances without
crashes, although they may be overwritten by future events. If they want
to hold onto the values longer term they need to copy them.
Using mp_hal_delay_ms allows the scheduler to run, which might result in
another transmit operation happening, which would bypass the sleep (and
fail). Use mp_hal_delay_us instead.
The compile-time configuration value MICROPY_HW_RTC_USER_MEM_MAX can now be
used to define the amount of memory set aside for RTC.memory(). If this
value is configured to zero then the RTC.memory functionality is not
included in the build.
Remove existing scan result events from the ringbuf if the ringbuf is full
and we're trying to enqueue any other event. This is needed so that events
such as SCAN_COMPLETE are always put on the ringbuf.
The IDF heap is more fragmented with IDF 4 and mbedtls cannot allocate
enough RAM with 16+16kiB for both in and out buffers, so reduce output
buffer size.
Fixes issue #5303.
This commit removes the Makefile-level MICROPY_FATFS config and moves the
MICROPY_VFS_FAT config to the Makefile level to replace it. It also moves
the include of the oofatfs source files in the build from each port to a
central place in extmod/extmod.mk.
For a port to enabled VFS FAT support it should now set MICROPY_VFS_FAT=1
at the level of the Makefile. This will include the relevant oofatfs files
in the build and set MICROPY_VFS_FAT=1 at the C (preprocessor) level.
defindex.html (used by topindex.html) is deprecated, but topindex.html was
already identical other than setting the title, so just inherit directly
from layout.html.
This commit adds support for littlefs (v2) on all esp32 boards.
The original FAT filesystem still works and any board with a preexisting
FAT filesystem will still work as normal. It's possible to switch to
littlefs by reformatting the block device using:
import uos, flashbdev
uos.VfsLfs2.mkfs(flashbdev.bdev)
Then when the board reboots (soft or hard) the new littlefs filesystem will
be mounted. It's possible to switch back to a FAT filesystem by formatting
with uos.VfsFat.mkfs(flashbdev.bdev).
While the new manifest.py style got introduced for freezing python code
into the resulting binary, the old way - where files and modules within
ports/*/modules where baked into the resulting binary - was still
supported via `freeze('$(PORT_DIR)/modules')` within manifest.py.
However behaviour changed for symlinked directories (=modules), as those
links weren't followed anymore.
This commit restores the original behaviour by explicitly following
symlinks within a modules/ directory
This commit adds a sys.implementation.mpy entry when the system supports
importing .mpy files. This entry is a 16-bit integer which encodes two
bytes of information from the header of .mpy files that are supported by
the system being run: the second and third bytes, .mpy version, and flags
and native architecture. This allows determining the supported .mpy file
dynamically by code, and also for the user to find it out by inspecting
this value. It's further possible to dynamically detect if the system
supports importing .mpy files by `hasattr(sys.implementation, 'mpy')`.
Replace the is_running field with a tri-state variable to indicate
running/not-running/pending-exception.
Update tests to cover the various cases.
This allows cancellation in uasyncio even if the coroutine hasn't been
executed yet. Fixes#5242
This wasn't necessary as the wrapped function already has a reference to
its globals. But it had a dual purpose of tracking whether the function
was currently running, so replace it with a bool.
This commit adds an implementation of machine.Timer backed by the soft
timer mechanism. It allows an arbitrary number of timers with 1ms
resolution, with an associated Python callback. The Python-level API
matches existing ports that have a soft timer, and is used as:
from machine import Timer
t = Timer(freq=10, callback=lambda t:print(t))
...
t = Timer(mode=Timer.ONE_SHOT, period=2000, callback=lambda t:print(t))
...
t.deinit()
This commit adds an implementation of a "software timer" with a 1ms
resolution, using SysTick. It allows unlimited number of concurrent
timers (limited only by memory needed for each timer entry). They can be
one-shot or periodic, and associated with a Python callback.
There is a very small overhead added to the SysTick IRQ, which could be
further optimised in the future, eg by patching SysTick_Handler code
dynamically.
POSIX poll should always return POLLERR and POLLHUP in revents, regardless
of whether they were requested in the input events flags.
See issues #4290 and #5172.
POSIX poll should always return POLLERR and POLLHUP in revents, regardless
of whether they were requested in the input events flags.
See issues #4290 and #5172.
runtime0.h is part of the MicroPython ABI so it's simpler if it's
independent of config options, like MICROPY_PY_REVERSE_SPECIAL_METHODS.
What's effectively done here is to move MP_BINARY_OP_DIVMOD and
MP_BINARY_OP_CONTAINS up in the enum, then remove the #if
MICROPY_PY_REVERSE_SPECIAL_METHODS conditional.
Without this change .mpy files would need to have a feature flag for
MICROPY_PY_REVERSE_SPECIAL_METHODS (when embedding native code that uses
this enum).
This commit has no effect when MICROPY_PY_REVERSE_SPECIAL_METHODS is
disabled. With this option enabled this commit reduces code size by about
60 bytes.
- Adds an explicit way to set the size of a value's internal buffer,
replacing `ble.gatts_write(handle, bytes(size))` (although that
still works).
- Add an "append" mode for values, which means that remote writes
will append to the buffer.
The MP_STATE_THREAD(stack_top) is always available so use it instead of
creating a separate variable. This also allows gc_collect() to be used as
an independent function, without real_main() being called.
This commit adds helper functions to call readblocks/writeblocks with a
fourth argument, the byte offset within a block.
Although the mp_vfs_blockdev_t struct has grown here by 2 machine words, in
all current uses of this struct within this repository it still fits within
the same number of GC blocks.
When a SPI bus is initialized with a SPI host that is currently in use the
exception msg incorrectly indicates "SPI device already in use". The
mention of "device" in the exception msg is confusing because the error is
about trying to use a SPI host that is already claimed. A better exception
msg is "SPI host already in use".
For consistency with "umachine". Now that weak links are enabled
by default for built-in modules, this should be a no-op, but allows
extension of the bluetooth module by user code.
Also move registration of ubluetooth to objmodule rather than
port-specific.
This commit implements automatic module weak links for all built-in
modules, by searching for "ufoo" in the built-in module list if "foo"
cannot be found. This means that all modules named "ufoo" are always
available as "foo". Also, a port can no longer add any other weak links,
which makes strict the definition of a weak link.
It saves some code size (about 100-200 bytes) on ports that previously had
lots of weak links.
Some changes from the previous behaviour:
- It doesn't intern the non-u module names (eg "foo" is not interned),
which saves code size, but will mean that "import foo" creates a new qstr
(namely "foo") in RAM (unless the importing module is frozen).
- help('modules') no longer lists non-u module names, only the u-variants;
this reduces duplication in the help listing.
Weak links are effectively the same as having a set of symbolic links on
the filesystem that is searched last. So an "import foo" will search
built-in modules first, then all paths in sys.path, then weak links last,
importing "ufoo" if it exists. Thus a file called "foo.py" somewhere in
sys.path will still have precedence over the weak link of "foo" to "ufoo".
See issues: #1740, #4449, #5229, #5241.
NimBLE doesn't actually copy this data, it requires it to stay live.
Only dereference when we register a new set of services.
Fixes#5226
This will allow incrementally adding services in the future, so
rename `reset` to `append` to make it clearer.
Internally change the representation of UUIDs to LE uint8* to simplify this.
This allows UUIDs to be easily used in BLE payloads (such as advertising).
Ref: #5186
When loading a manifest file, e.g. by include(), it will chdir first to the
directory of that manifest. This means that all file operations within a
manifest are relative to that manifest's location.
As a consequence of this, additional environment variables are needed to
find absolute paths, so the following are added: $(MPY_LIB_DIR),
$(PORT_DIR), $(BOARD_DIR). And rename $(MPY) to $(MPY_DIR) to be
consistent.
Existing manifests are updated to match.
Prior to this commit the systick IRQ priority was set at lowest priority on
F0/L0/WB MCUs, because it was left at the default and never configured.
This commit ensures the priority is configured and sets it to the highest
priority.
Remove the 240MHz CPU config option from sdkconfig.base and create a new
sdkconfig.240mhz file for those boards that want to use 240MHz on boot.
The default CPU frequency is now 160MHz (was 240MHz), to align with the ESP
IDF and support more boards (eg those with D2WD chips).
Fixes issue #5169.
DS1822P sensors behave just like the DS18B20 except for the following:
- it has a different family code: 0x22
- it has only the GND and DQ pins connected, it uses parasitic power from
the data line
Contributed by @nebelgrau77.
This introduces a new build variable FROZEN_MANIFEST which can be set to a
manifest listing (written in Python) that describes the set of files to be
frozen in to the firmware.
This prevents issues with concurrent access to the ringbuf.
MICROPY_BEGIN_ATOMIC_SECTION is only atomic to the same core. We could
address this with a mutex, but it's also not safe to call mp_sched_schedule
across cores.
This avoids a confusing ENOMEM raised from gap_advertise if there is
currently an active connection. This refers to the static connection
buffer pre-allocated by Nimble (nothing to do with MicroPython heap
memory).
Instead of encoding 4 zero bytes as placeholders for the simple_name and
source_file qstrs, and storing the qstrs after the bytecode, store the
qstrs at the location of these 4 bytes. This saves 4 bytes per bytecode
function stored in a .mpy file (for example lcd160cr.mpy drops by 232
bytes, 4x 58 functions). And resulting code size is slightly reduced on
ports that use this feature.
This is to more accurately match the BLE spec, where intervals are
configured in units of channel hop time (625us). When it was
specified in ms, not all "valid" intervals were able to be
specified.
Now that we're also allowing configuration of scan interval, this
commit updates advertising to match.
This adds two additional optional kwargs to `gap_scan()`:
- `interval_us`: How long between scans.
- `window_us`: How long to scan for during a scan.
The default with NimBLE is a 11.25ms window with a 1.28s interval.
Changing these parameters is important for detecting low-frequency
advertisements (e.g. beacons).
Note: these params are in microseconds, not milliseconds in order
to allow the 625us granularity offered by the spec.
This patch add basic building blocks for nrf9P60.
It also includes a secure bootloader which forwards all
possible peripherals that are user selectable to become
non-secure. After configuring Flash, RAM and peripherals
the secure bootloader will jump to the non-secure domain
where MicroPython is placed.
The minimum size of a secure boot has to be a flash
block of 32Kb, hence why the linker scripts are
offsetting the main application this much.
The RAM offset is set to 128K, to allow for later
integration of Nordic Semiconductor's BSD socket
library which reserves the range 0x20010000 - 0x2001FFFF.
Add support for pca10059 with REPL over tinyusb USB CDC.
The board also includes a board specific module that will
recover UICR->REGOUT0 in case this has been erased.
This initial support does not preserve any existing bootloader
on the pca10090 in case this was present, and expects to use all
available flash on the device.
Add nrf-port finyusb driver files. USB CDC can be activated
by board configuration files using the MICROPY_HW_USB_CDC.
Updating BLE driver, Makefile, nrfx-glue and main.c to plug
in the tinyusb stack.
The specific board can be selected with the BOARD makefile variable. This
defaults (if not specified) to BOARD=GENERIC, which is the original default
firmware build. For the 512k target use BOARD=GENERIC_512K.
On other ports (e.g. ESP32) they provide a complete Nimble implementation
(i.e. we don't need to use the code in extmod/nimble). This change
extracts out the bits that we don't need to use in other ports:
- malloc/free/realloc for Nimble memory.
- pendsv poll handler
- depowering the cywbt
Also cleans up the root pointer management.
In which case place the native function prelude in a bytes object, linked
from the const_table of that function. An architecture should define
N_PRELUDE_AS_BYTES_OBJ to 1 before including py/emitnative.c to emit
correct machine code, then enable MICROPY_EMIT_NATIVE_PRELUDE_AS_BYTES_OBJ
so the runtime can correctly handle the prelude being in a bytes object.
Such that args/return regs for the parent are different to args/return regs
for child calls. For an architecture to use this feature it should define
the REG_PARENT_xxx macros before including py/emitnative.c.
Prior to this commit, when unwinding through an active finally the stack
was not being correctly popped/folded, which resulting in the VM crashing
for complicated unwinding of nested finallys.
This should be fixed with this commit, and more tests for return/break/
continue within a finally have been added to exercise this.
STM32F0 has PCLK=48MHz and maximum ADC clock is 14MHz so use PCLK/4=12MHz
to stay within spec of the ADC peripheral. In pyb.ADC set common sampling
time to approx 4uS for internal and external sources. In machine.ADC
reduce sample time to approx 1uS for external source, leave internal at
maximum sampling time.
As of 7d58a197cf, `NULL` should no longer be
here because it's allowed (MP_QSTRnull took its place). This entry was
preventing the use of MP_QSTR_NULL to mean "NULL" (although this is not
currently used).
A blacklist should not be needed because it should be possible to intern
all strings.
Fixes issue #5140.
This check follows CPython's behaviour, because 'import *' always populates
the globals with the imported names, not locals.
Since it's safe to do this (doesn't lead to a crash or undefined behaviour)
the check is only enabled for MICROPY_CPYTHON_COMPAT.
Fixes issue #5121.
This commit adds the option to use HSE or MSI system clock, and LSE or LSI
RTC clock, on L4 MCUs.
Note that prior to this commit the default clocks on an L4 part were MSI
and LSE. The defaults are now MSI and LSI.
In mpconfigboard.h select the clock source via:
#define MICROPY_HW_RTC_USE_LSE (0) or (1)
#define MICROPY_HW_CLK_USE_HSE (0) or (1)
and the PLLSAI1 N,P,Q,R settings:
#define MICROPY_HW_CLK_PLLSAIN (12)
#define MICROPY_HW_CLK_PLLSAIP (RCC_PLLP_DIV7)
#define MICROPY_HW_CLK_PLLSAIQ (RCC_PLLQ_DIV2)
#define MICROPY_HW_CLK_PLLSAIR (RCC_PLLR_DIV2)
The the nrfx driver is aware of chip specific registers, while
the raw HAL abstraction is not. This driver enables use of NVMC
in non-secure domain for nrf9160.
This patch moves the check for MICROPY_PY_MACHINE_TEMP to come
before the inclusion of nrf_temp.h. The nrf_temp.h depends on
the NRF_TEMP_Type which might not be defined for all nRF devices.
This patch compresses the second part of the bytecode prelude which
contains the source file name, function name, source-line-number mapping
and cell closure information. This part of the prelude now begins with a
single varible length unsigned integer which encodes 2 numbers, being the
byte-size of the following 2 sections in the header: the "source info
section" and the "closure section". After decoding this variable unsigned
integer it's possible to skip over one or both of these sections very
easily.
This scheme saves about 2 bytes for most functions compared to the original
format: one in the case that there are no closure cells, and one because
padding was eliminated.
The start of the bytecode prelude contains 6 numbers telling the amount of
stack needed for the Python values and exceptions, and the signature of the
function. Prior to this patch these numbers were all encoded one after the
other (2x variable unsigned integers, then 4x bytes), but using so many
bytes is unnecessary.
An entropy analysis of around 150,000 bytecode functions from the CPython
standard library showed that the optimal Shannon coding would need about
7.1 bits on average to encode these 6 numbers, compared to the existing 48
bits.
This patch attempts to get close to this optimal value by packing the 6
numbers into a single, varible-length unsigned integer via bit-wise
interleaving. The interleaving scheme is chosen to minimise the average
number of bytes needed, and at the same time keep the scheme simple enough
so it can be implemented without too much overhead in code size or speed.
The scheme requires about 10.5 bits on average to store the 6 numbers.
As a result most functions which originally took 6 bytes to encode these 6
numbers now need only 1 byte (in 80% of cases).
For use with F0 MCUs that don't have HSI48. Select the clock source
explicitly in mpconfigboard.h.
On the NUCLEO_F091RC board use HSE bypass when HSE is chosen because the
NUCLEO clock source is STLINK not a crystal.
Before this patch the UART baudrate on F0 MCUs was wrong because the
stm32lib SystemCoreClockUpdate sets SystemCoreClock to 8MHz instead of
48MHz if HSI48 is routed directly to SYSCLK.
The workaround is to use HSI48 -> PREDIV (/2) -> PLL (*2) -> SYSCLK.
Fixes issue #5049.
From the beginning of this project the RAISE_VARARGS opcode was named and
implemented following CPython, where it has an argument (to the opcode)
counting how many args the raise takes:
raise # 0 args (re-raise previous exception)
raise exc # 1 arg
raise exc from exc2 # 2 args (chained raise)
In the bytecode this operation therefore takes 2 bytes, one for
RAISE_VARARGS and one for the number of args.
This patch splits this opcode into 3, where each is now a single byte.
This reduces bytecode size by 1 byte for each use of raise. Every byte
counts! It also has the benefit of reducing code size (on all ports except
nanbox).
To make progress towards MicroPython supporting Python 3.5, adding the
matmul operator is important because it's a really "low level" part of the
language, being a new token and modifications to the grammar.
It doesn't make sense to make it configurable because 1) it would make the
grammar and lexer complicated/messy; 2) no other operators are
configurable; 3) it's not a feature that can be "dynamically plugged in"
via an import.
And matmul can be useful as a general purpose user-defined operator, it
doesn't have to be just for numpy use.
Based on work done by Jim Mussared.
Enabled by default, but disabled when REPL is connected to the VCP (this is
the existing behaviour). Can be configured at run-time with, eg:
pyb.USB_VCP().init(flow=pyb.USB_VCP.RTS | pyb.USB_VCP.CTS)
The new fdcan.c file provides the low-level C interface to the FDCAN
peripheral, and pyb_can.c is updated to support both traditional CAN and
FDCAN, depending on the MCU being compiled for.
Add the project file to the mpy-cross directory, which is also where the
executable ends up, and change the Appveyor settings to build mpy-cross
with both msvc and mingw-w64 and verify this all works by running tests
with --via-mpy.
If this is not set it might default to calls to open() to use text mode
which is usually not wanted, and even wrong and leading to incorrect
results when loading binary .mpy files.
This also means that text files written and read will not have line-ending
translation from \n to \r\n and vice-versa anymore. This shouldn't be much
of a problem though since most tools dealing with text files adapt
automatically to any of the 2 formats.
Reserve sources.props for listing just the MicroPython core and extmod
files, similar to how py.mk lists port-independent source files. This
allows reusing the source list, for instance for building mpy-cross. The
sources for building the executable itself are listed in the corresponding
project file, similar to how the other ports specify the source files in
their Makefile.
Append to PyIncDirs, used to define include directories specific to
MicroPython, instead of just overwriting it so project files importing this
file can define additional directories. And allow defining the target
directory for the executable instead of hardcoding it to the windows
directory. Main reason for this change is that it will allow building
mpy-cross with msvc.
We want the .vcxproj to be just a container with the minimum content for
making it work as a project file for Visual Studio and MSBuild, whereas the
actual build options and actions get placed in separate reusable files.
This was roughly the case already except some compiler options were
overlooked; fix this here: we'll need those common options when adding a
project file for building mpy-cross.
These were probably added to detect more qstrs but as long as the
micropython executable itself doesn't use the same build options the qstrs
would be unused anyway. Furthermore these definitions are for internal use
and get enabled when corresponding MICROPY_EMIT_XXX are defined, in which
case the compiler would warn about symbol redefinitions since they'd be
defined both here and in the source.
This commit adds support for a second supported hash (currently set to the
4.0-beta1 tag). When this hash is detected, the relevant changes are
applied.
This allows to start using v4 features (e.g. BLE with Nimble), and also
start doing testing, while still supporting the original, stable, v3.3 IDF.
Note: this feature is experimental, not well tested, and network.LAN and
network.PPP are currently unsupported.
This option affects py/vm.c and py/gc.c and using -Os gets them compiling a
bit smaller, and small firmware is the aim of these two ports. Also,
having these files compiled with -Os on these ports, and -O3 as the default
on other ports, gives a better understanding of code-size changes when
making changes to these files.
According to the schematic, the SDRAM part on this board is a
MT48LC4M32B2B5-6A, with "Row addressing 4K A[11:0]" (per datasheet). This
commit updates mpconfigboard.h from 13 to 12 to match.
This patch uses the newly-added esp32.Partition class to replace the
existing FlashBdev class. Partition objects implement the block protocol
so can be directly mounted via uos.mount(). This has the following
benefits:
- allows the filesystem partition location and size to be specified in
partitions.csv, and overridden by a particular board
- very easily allows to have multiple filesystems by simply adding extra
entries to partitions.csv
- improves efficiency/speed of filesystem operations because the block
device is implemented fully in C
- opens the possibility to have encrypted flash storage (since Partitions
can be encrypted)
Note that this patch is fully backwards compatible: existing filesystems
remain untouched and work with this new code.
- STM32F407VGT6 (1MB of Flash, 192+4 Kbytes of SRAM)
- 5V (via USB) or Li-Polymer Battery (3.7V) power input
- 2 x LEDs
- 2 x user switches
- 2 x mikroBUS sockets
- 2 x 1x26 mikromedia-compatible headers (52 pins)
https://www.mikroe.com/clicker-2-stm32f4
Mboot currently requires at least three LEDs to display each of the four
states. However, since there are only four possible states, the states can
be displayed via binary counting on only 2 LEDs (if only 2 are available).
The existing patterns are still used for 3 or 4 LEDs.
It was previously not taking into account that the list of pins was sparse,
so using the wrong index. The boards/X/pins.csv was generating the wrong
data for machine.Pin.board.
As part of this fix rename the variables to make it more clear what the
list contains (only board pins).
Prior to this patch mp_opcode_format would calculate the incorrect size of
the MP_BC_UNWIND_JUMP opcode, missing the additional byte. But, because
opcodes below 0x10 are unused and treated as bytes in the .mpy load/save
and freezing code, this bug did not show any symptoms, since nested unwind
jumps would rarely (if ever) reach a depth of 16 (so the extra byte of this
opcode would be between 0x01 and 0x0f and be correctly loaded/saved/frozen
simply as an undefined opcode).
This patch fixes this bug by correctly accounting for the additional byte.
.
With this patch alignment is done relative to the start of the buffer that
is being unpacked, not the raw pointer value, as per CPython.
Fixes issue #3314.
This commit adds support for sys.settrace, allowing to install Python
handlers to trace execution of Python code. The interface follows CPython
as closely as possible. The feature is disabled by default and can be
enabled via MICROPY_PY_SYS_SETTRACE.
Prior to this patch the line number for a lambda would be "line 1" if the
body of the lambda contained only a simple expression (with no line number
stored in the parse node). Now the line number is always reported
correctly.
mp_compile no longer takes an emit_opt argument, rather this setting is now
provided by the global default_emit_opt variable.
Now, when -X emit=native is passed as a command-line option, the emitter
will be set for all compiled modules (included imports), not just the
top-level script.
In the future there could be a way to also set this variable from a script.
Fixes issue #4267.
With this patch exceptions that are re-raised have improved tracebacks
(less confusing, match CPython), and it makes re-raise slightly more
efficient (in time and RAM) because they no longer need to add a traceback.
Also general VM performance is not measurably affected.
Partially fixes issue #2928.
With this patch exception tracebacks that go through a finally are improved
(less confusing, match CPython), and it makes finally's slightly more
efficient (in time and RAM) because they no longer need to add a traceback.
Partially fixes issue #2928.
As per the README.md of the upstream source at
https://github.com/B-Con/crypto-algorithms, this source code was released
into the public domain, so make that explicit in the copyright line in the
header.
It's really an opcode that's not implemented, so use "opcode" instead of
"byte code". And remove the redundant "not implemented" text because that
is already implied by the exception type. There's no need to have a long
error message for an exception that is almost never encountered. Saves
about 20 bytes of code size on most ports.
- Split 'qemu-arm' from 'unix' for generating tests.
- Add frozen module to the qemu-arm test build.
- Add test that reproduces the requirement to half-word align native
function data.
Enabled via MICROPY_PY_URE_DEBUG, disabled by default (but enabled on unix
coverage build). This is a rarely used feature that costs a lot of code
(500-800 bytes flash). Debugging of regular expressions can be done
offline with other tools.
Recent versions of gcc perform optimisations which can lead to the
following code from the MP_NLR_JUMP_HEAD macro being omitted:
top->ret_val = val; \
MP_NLR_RESTORE_PYSTACK(top); \
*_top_ptr = top->prev; \
This is noticeable (at least) in the unix coverage on x86-64 built with gcc
9.1.0. This is because the nlr_jump function is marked as no-return, so
gcc deduces that the above code has no effect.
Adding MP_UNREACHABLE tells the compiler that the asm code may branch
elsewhere, and so it cannot optimise away the code.
As per PEP 485, this function appeared in for Python 3.5. Configured via
MICROPY_PY_MATH_ISCLOSE which is disabled by default, but enabled for the
ports which already have MICROPY_PY_MATH_SPECIAL_FUNCTIONS enabled.
Before this patch I2C transactions using a hardware I2C peripheral on F0/F7
MCUs would not correctly generate the I2C restart condition, and instead
would generate a stop followed by a start. This is because the CR2 AUTOEND
bit was being set before CR2 START when the peripheral already had the I2C
bus from a previous transaction that did not generate a stop.
As a consequence all combined transactions, eg read-then-write for an I2C
memory transfer, generated a stop condition after the first transaction and
didn't generate a stop at the very end (but still released the bus). Some
I2C devices require a repeated start to function correctly.
This patch fixes this by making sure the CR2 AUTOEND bit is set after the
start condition and slave address have been fully transferred out.
Replaces the `SDKCONFIG` makefile variable with `BOARD`. Defaults to
BOARD=GENERIC. spiram can be enabled with `BOARD=GENERIC_SPIRAM`
Add example definition for TINYPICO, currently identical to GENERIC_SPIRAM
but with custom board/SoC names for the uPy banner.
Prior to this patch the amount of free space in an array (including
bytearray) was not being maintained correctly for the case of slice
assignment which changed the size of the array. Under certain cases (as
encoded in the new test) it was possible that the array could grow beyond
its allocated memory block and corrupt the heap.
Fixes issue #4127.
This patch implements a new sys.atexit function which registers a function
that is later executed when the main script ends. It is configurable via
MICROPY_PY_SYS_ATEXIT, disabled by default.
This is not compliant with CPython, rather it can be used to implement a
CPython compatible "atexit" module if desired (similar to how
sys.print_exception can be used to implement functionality of the
"traceback" module).
They are both enabled by default, but can be disabled by defining
MICROPY_HW_ENABLE_MDNS_QUERIES and/or MICROPY_HW_ENABLE_MDNS_RESPONDER to
0. The hostname for the responder is currently taken from
tcpip_adapter_get_hostname() but should eventually be configurable.
This commit adds the connect() method to the PPP interface and requires
that connect() be called after active(1). This is a breaking change for
the PPP API.
With the connect() method it's now possible to pass in authentication
information for PAP/CHAP, eg:
ppp.active(1)
ppp.connect(authmode=ppp.AUTH_PAP, username="user", "password="password")
If no authentication is needed simply call connect() without any
parameters. This will get the original behaviour of calling active(1).
Some SD/MMC breakout boards don't support 4-bit bus mode. This adds a new
macro MICROPY_HW_SDMMC_BUS_WIDTH that allows each board to define the width
of the SD/MMC bus interface used on that board, defaulting to 4 bits.
This patch adds a simple but powerful hook into the import system, in a
CPython compatible way, by allowing to override builtins.__import__.
This does introduce some overhead to all imports but it's minor:
- the dict lookup of __import__ is bypassed if there are no modifications
to the builtins module (which is the case at start up);
- imports are not performance critical, usually done just at the start of a
script;
- compared to how much work is done in an import, looking up a value in a
dict is a relatively small additional piece of work.
JSON requires that keys of objects be strings. CPython will therefore
automatically quote simple types (NoneType, bool, int, float) when they are
used directly as keys in JSON output. To prevent subtle bugs and emit
compliant JSON, MicroPython should at least test for such keys so they
aren't silently let through. Then doing the actual quoting is a similar
cost to raising an exception, so that's what is implemented by this patch.
Fixes issue #4790.
The previous version did not work on MCUs that only had USB device mode
(compared to OTG) because of the handling of NAK. And this previous
handling of NAK had a race condition where a new packet could come in
before USBD_HID_SetNAK was called (since USBD_HID_ReceivePacket clears NAK
as part of its operation). Furthermore, the double buffering of incoming
reports was not working, only one buffer could be used at a time.
This commit rewrites the HID interface code to have a single incoming
buffer, and only calls USBD_HID_ReceivePacket after the user has read the
incoming report (similar to how the VCP does its flow control). As such,
USBD_HID_SetNAK and USBD_HID_ClearNAK are no longer needed.
API functionality from the user's point of view should be unchanged with
this commit.
Use "-f" to select filesystem mode, followed by the command to execute.
Optionally put ":" at the start of a filename to indicate that it's on the
remote device, if it would otherwise be ambiguous.
Examples:
$ pyboard.py -f ls
$ pyboard.py -f cat main.py
$ pyboard.py -f cp :main.py . # get from device
$ pyboard.py -f cp main.py : # put to device
$ pyboard.py -f rm main.py
On this port the GIL is enabled and everything works under the assumption
of the GIL, ie that a given task has exclusive access to the uPy state, and
any ISRs interrupt the current task and therefore the ISR inherits
exclusive access to the uPy state for the duration of its execution.
If the MicroPython tasks are not pinned to a specific core then an ISR may
be executed on a different core to the task, making it possible for the
main task and an ISR to execute in parallel, breaking the assumption of the
GIL.
The easiest and safest fix for this is to pin all MicroPython related code
to the same CPU core, as done by this patch. Then any ISR that accesses
MicroPython state must be registered from a MicroPython task, to ensure it
is invoked on the same core.
See issue #4895.
The C++ standard forbids redefining keywords, like inline and alignof, so
guard these definitions to avoid that, allowing to include the MicroPython
headers by C++ code.
This new series of MCUs is similar to the L4 series with an additional
Cortex-M0 coprocessor. The firmware for the wireless stack must be managed
separately and MicroPython does not currently interface to it. Supported
features so far include: RTC, UART, USB, internal flash filesystem.
Behaviour was changed from stack to queue in
8977c7eb58, and this updates variable names
to match. Also updates other references (docs, error messages).
The new configurations MICROPY_HW_USB_MSC and MICROPY_HW_USB_HID can be
used by a board to enabled or disable MSC and/or HID. They are both
enabled by default.
__clear_cache causes a compile error when using clang. Instead use
__builtin___clear_cache which is available under both gcc and clang.
Also replace tabs with spaces in this section of code (introduced by a
previous commit).
In a non-thread build, using &_ram_end as the top-of-stack is no longer
correct because the stack is not always at the very top end of RAM. See
eg 04c7cdb668 and
3786592097. The correct value to use is
&_estack, which is the value stored in MP_STATE_THREAD(stack_top), and
using the same code for both thread and non-thread builds makes the code
cleaner.
stm32lib now provides system_stm32XXxx.c source files for all MCU variants,
which includes SystemInit and prescaler tables. Since these are quite
standard and don't need to be changed, switch to use them instead of custom
variants, making the start-up code cleaner.
The SystemInit code in stm32lib was checked and is equivalent to what is
removed from the stm32 port in this commit.
Without this you often don't get any DNS server from your network provider.
Additionally, setting your own DNS _does not work_ without this option set
(which could be a bug in the PPP stack).
This is a start to make a more consistent machine.RTC class across ports.
The stm32 pyb.RTC class at least has the datetime() method which behaves
the same as esp8266 and esp32, and with this patch the ntptime.py script
now works with stm32.
The helper function exec_user_callback executes within the context of an
lwIP C callback, and the user (Python) callback to be scheduled may want to
perform further TCP/IP actions, so the latter should be scheduled to run
outside the lwIP context (otherwise it's effectively a "hard IRQ" and such
callbacks have lots of restrictions).
If tcp_write returns ERR_MEM then it's not a fatal error but instead means
the caller should retry the write later on (and this is what lwIP's netconn
API does).
This fixes problems where a TCP send would raise OSError(ENOMEM) in
situations where the TCP/IP stack is under heavy load. See eg issues #1897
and #1971.
If both FS and HS USB peripherals are enabled for a board then the active
one used for the REPL will now be auto-detected, by checking to see if both
the DP and DM lines are actively pulled low. By default the code falls
back to use MICROPY_HW_USB_MAIN_DEV if nothing can be detected.
When going out of memory-mapped mode to do a control transfer to the QSPI
flash, the MPU settings must be changed to forbid access to the memory
mapped region. And any ongoing transfer (eg memory mapped continuous read)
must be aborted.
The Cortex-M7 CPU will do speculative loads from any memory location that
is not explicitly forbidden. This includes the QSPI memory-mapped region
starting at 0x90000000 and with size 256MiB. Speculative loads to this
QSPI region may 1) interfere with the QSPI peripheral registers (eg the
address register) if the QSPI is not in memory-mapped mode; 2) attempt to
access data outside the configured size of the QSPI flash when it is in
memory-mapped mode. Both of these scenarios will lead to issues with the
QSPI peripheral (eg Cortex bus lock up in scenario 2).
To prevent such speculative loads from interfering with the peripheral the
MPU is configured in this commit to restrict access to the QSPI mapped
region: when not memory mapped the entire region is forbidden; when memory
mapped only accesses to the valid flash size are permitted.
This fixes compiling for older architectures (e.g. armv5tej).
According to [1], the limit of R0-R7 for the STR and LDR instructions is
tied to the Thumb instruction set and not any specific processor
architectures.
[1]: http://www.keil.com/support/man/docs/armasm/armasm_dom1361289906890.htm
When compiled with hard float the system should enable FP access when it
starts or else FP instructions lead to a fault. But this minimal port does
not enable (or use) FP and so, to keep it minimal, switch to use soft
floating point. (This became an issue due to the recent commit
34c04d2319 which saves/restores FP registers
in the NLR state.)
misc_aes.py and misc_mandel.py are adapted from sources in this repository.
misc_pystone.py is the standard Python pystone test. misc_raytrace.py is
written from scratch.
This benchmarking test suite is intended to be run on any MicroPython
target. As such all tests are parameterised with N and M: N is the
approximate CPU frequency (in MHz) of the target and M is the approximate
amount of heap memory (in kbytes) available on the target. When running
the benchmark suite these parameters must be specified and then each test
is tuned to run on that target in a reasonable time (<1 second).
The test scripts are not standalone: they require adding some extra code at
the end to run the test with the appropriate parameters. This is done
automatically by the run-perfbench.py script, in such a way that imports
are minimised (so the tests can be run on targets without filesystem
support).
To interface with the benchmarking framework, each test provides a
bm_params dict and a bm_setup function, with the later taking a set of
parameters (chosen based on N, M) and returning a pair of functions, one to
run the test and one to get the results.
When running the test the number of microseconds taken by the test are
recorded. Then this is converted into a benchmark score by inverting it
(so higher number is faster) and normalising it with an appropriate factor
(based roughly on the amount of work done by the test, eg number of
iterations).
Test outputs are also compared against a "truth" value, computed by running
the test with CPython. This provides a basic way of making sure the test
actually ran correctly.
Each test is run multiple times and the results averaged and standard
deviation computed. This is output as a summary of the test.
To make comparisons of performance across different runs the
run-perfbench.py script also includes a diff mode that reads in the output
of two previous runs and computes the difference in performance. Reports
are given as a percentage change in performance with a combined standard
deviation to give an indication if the noise in the benchmarking is less
than the thing that is being measured.
Example invocations for PC, pyboard and esp8266 targets respectively:
$ ./run-perfbench.py 1000 1000
$ ./run-perfbench.py --pyboard 100 100
$ ./run-perfbench.py --pyboard --device /dev/ttyUSB0 50 25
With both MICROPY_PERSISTENT_CODE_SAVE and MICROPY_PERSISTENT_CODE_LOAD
enabled the code fails to compile, due to undeclared 'n_obj'. If
MICROPY_EMIT_NATIVE is disabled there are more errors due to the use of
undefined fields in mp_raw_code_t.
This patch fixes such compilation by avoiding undefined fields.
MICROPY_EMIT_NATIVE was changed to MICROPY_EMIT_MACHINE_CODE in this file
to match the mp_raw_code_t definition.
Change static LED functions to lowercase names, and trim down source code
lines for variants of MICROPY_HW_LED_COUNT. Also rename configuration for
MICROPY_HW_LEDx_LEVEL to MICROPY_HW_LEDx_PULLUP to align with global PULLUP
configuration.
Commit 9e68eec8ea introduced a regression
where the PID of the USB device would be 0xffff if the default value was
used. This commit fixes that by using a signed int type.
This saves time when building on Travis CI: unconditionally fetching all
submodules takes about 40 seconds, but not all are needed for any given
port, so only fetch as necessary.
Entering a bootloader (ST system bootloader, or custom mboot) from software
by directly branching to it is not reliable, and the reliability of it
working can depend on the peripherals that were enabled by the application
code. It's also not possible to branch to a bootloader if the WDT is
enabled (unless the bootloader has specific provisions to feed the WDT).
This patch changes the way a bootloader is entered from software by first
doing a complete system reset, then branching to the desired bootloader
early on in the start-up process. The top two words of RAM (of the stack)
are reserved to store flags indicating that the bootloader should be
entered after a reset.
WIFI_REASON_AUTH_FAIL does not necessarily mean the password is wrong, and
a wrong password may not lead to a WIFI_REASON_AUTH_FAIL error code. So to
improve reliability connecting to a WLAN always reconnect regardless of the
error.
These s16-s21 registers are used by gcc so need to be saved. Future
versions of gcc (beyond v9.1.0), or other compilers, may eventually need
additional registers saved/restored.
See issue #4844.
This updates ESP IDF to use v3.3-beta3. And also adjusts README.md to
point to stable docs which provide a link to download the correct toolchain
for this IDF version, namely 1.22.0-80-g6c4433a-5.2.0
Previously the end of the heap was the start (lowest address) of the stack.
With the changes in this commit these addresses are now independent,
allowing a board to place the heap and stack in separate locations.
With this the user can select multiple logical units to expose over USB MSC
at once, eg: pyb.usb_mode('VCP+MSC', msc=(pyb.Flash(), pyb.SDCard())). The
default behaviour is the original behaviour of just one unit at a time.
Eventually these responses could be filled in by a function to make their
contents dynamic, depending on the attached logical units. But for now
they are fixed, and this patch fixes the MODE SENSE(6) responses so it is
the correct length with the correct header.
SCSI can support multiple logical units over the one interface (in this
case over USBD MSC) and here the MSC code is reworked to support this
feature. At this point only one LU is used and the behaviour is mostly
unchanged from before, except the INQUIRY result is different (it will
report "Flash" for both flash and SD card).
Previously, when linking qstr objects in native code for ARM Thumb, the
index into the machine code was being incremented by 4, not 8. It should
be 8 to account for the size of the two machine instructions movw and movt.
This patch makes sure the index into the machine code is incremented by the
correct amount for all variations of qstr linking.
See issue #4829.
To use it a board should define MICROPY_PY_USSL=1 and MICROPY_SSL_MBEDTLS=1
at the Makefile level. With the provided configuration it adds about 64k
to the build.
Setting MICROPY_PY_USSL and MICROPY_SSL_MBEDTLS at the Makefile-level will
now build mbedTLS from source and include it in the build, with the ussl
module using this TLS library. Extra settings like MBEDTLS_CONFIG_FILE may
need to be provided by a given port.
If a port wants to use its own mbedTLS library then it should not set
MICROPY_SSL_MBEDTLS at the Makefile-level but rather set it at the C level,
and provide the library as part of the build in its own way (see eg esp32
port).
Instead of converting to a small-int at runtime this can be done at compile
time, then we only have a simple comparison during runtime. This reduces
code size on some ports (e.g -4 on qemu-arm, -52 on unix nanbox), and for
others at least doesn't increase code size.
Fixes errors in the tool when 1) linking qstrs in native ARM-M code; 2)
freezing multiple files some of which use native code and some which don't.
Fixes issue #4829.
It doesn't work to tie the polling of an underlying NIC driver (eg to check
the NIC for pending Ethernet frames) with its associated lwIP netif. This
is because most NICs are implemented with IRQs and don't need polling,
because there can be multiple lwIP netif's per NIC driver, and because it
restricts the use of the netif->state variable. Instead the NIC should
have its own specific way of processing incoming Ethernet frame.
This patch removes this generic NIC polling feature, and for the only
driver that uses it (Wiznet5k) replaces it with an explicit call to the
poll function (which could eventually be improved by using a proper
external interrupt).
This adds support for SD cards using the ESP32's built-in hardware SD/MMC
host controller, over either the SDIO bus or SPI. The class is available
as machine.SDCard and using it can be as simple as:
uos.mount(machine.SDCard(), '/sd')
If the board-pin name is left empty then only the cpu-pin name is used, eg
",PA0". If the board-pin name starts with a hyphen then it's available as
a C definition but not in the firmware, eg "-X1,PA0".
The patch solves the problem where multiple Timer objects (e.g. multiple
Timer(0) instances) could initialise multiple handles to the same internal
timer. The list of timers is now maintained not for "active" timers (where
init is called), but for all timers created. The timers are only removed
from the list of timers on soft-reset (machine_timer_deinit_all).
Fixes#4078.
The board config option MICROPY_HW_USB_ENABLE_CDC2 is now changed to
MICROPY_HW_USB_CDC_NUM, and the latter should be defined to the maximum
number of CDC interfaces to support (defaults to 1).
mpy-cross uses MICROPY_DYNAMIC_COMPILER and MICROPY_EMIT_NATIVE but does
not actually need to execute native functions, and does not need
mp_fun_table. This commit makes it so mp_fun_table and all its entries are
not built when MICROPY_DYNAMIC_COMPILER is enabled, significantly reducing
the size of the mpy-cross executable and allowing it to be built on more
machines/OS's.
Initialise the timer. Initialisation must be either by frequency (in Hz)
or by prescaler and period::
@@ -112,7 +112,7 @@ Methods
..method:: Timer.deinit()
Deinitialises the timer.
Disables the callback (and the associated irq).
Disables any channel callbacks (and the associated irq).
@@ -191,7 +191,7 @@ Methods
- Read the encoder value using the timer.counter() method.
- Only works on CH1 and CH2 (and not on CH1N or CH2N)
- The channel number is ignored when setting the encoder mode.
PWM Example::
timer = pyb.Timer(2, freq=1000)
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
