User callbacks allow code to respond to incoming messages without blocking
or polling. User callbacks are optional, and if no callback is registered
the code has no effect.
The mechanism is the same as for TCP: when a connection is accepted or a
TCP packet is received, a user callback is executed.
Fixes issue #3594.
Signed-off-by: Jack Whitham <jack.d.whitham@gmail.com>
Because socket objects have a finaliser they must be created carefully, in
case an exception is raised during the population of their members, eg
invalid input argument or out-of-memory when allocating additional arrays.
Prior to the fix in this commit, the finaliser would crash due to
`incoming.udp_raw.array` being an invalid pointer in the following cases:
- if a SOCK_RAW was created with a proto argument that was not an integer
- if a SOCK_DGRAM or SOCK_RAW was created where the allocation of
`lwip_incoming_packet_t` failed
- if an integer was passed in for the socket type but it was not one of
SOCK_STREAM, SOCK_DGRAM or SOCK_RAW
Furthermore, if the allocation of `lwip_incoming_packet_t` failed then it
may have led to corruption within lwIP when freeing `socket->pcb.raw`
because that PCB was not fully set up with its callbacks.
This commit fixes all of these issues by ensuring:
- `pcb.tcp` and `incoming.udp_raw.array` are initialised to NULL early on
- the proto argument is parsed before allocating the PCB
- the allocation of `lwip_incoming_packet_t` occurs befor allocating the
PCB
- `incoming.udp_raw.array` is checked for NULL in the finaliser code
The corresponding test (which already checked most of these causes of
failure) has been updated to include a previously-uncovered scenario.
Signed-off-by: Damien George <damien@micropython.org>
This commit fixes an issue related to the NimBLE initialisation
procedure in low memory environments on ESP32 boards.
MicroPython uses at least two different NimBLE stacks across the
supported ports, mynewt (imported as an external library), and the one
provided by Espressif in their own SDKs. The problem is that these two
ports differ in the signature for `nimble_port_init(void)`, with mynewt
returning `void`, and Espressif's returning a status code on failure.
On ESP32 boards, allocating almost all the available heap and then
turning on the Bluetooth stack would trigger a failure in the NimBLE
initialisation function that is not handled by the NimBLE integration
code, as there's no expectation of a recoverable condition. Since the
stack initialisation would progress, then uninitialised memory accesses
crash the board.
Since we cannot really modify neither mynewt nor Espressif SDKs, the
next best thing is to provide two conditional initialisation paths
depending on a configuration setting. This would make Espressif ports
recover from a failed initialisation whilst retaining the existing
behaviour on other ports.
This fixes#14293.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
API is different to the original machine.CAN proposal, as numerous
shortcomings were found during initial implementation.
This work was funded through GitHub Sponsors.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
These classes are based on the Quadrature Encoder blocks of the i.MXRT
MCUs. The i.MXRT 102x has two encoders, the other ones four. The i.MXRT
101x does not support this function. It is implemented as two classes,
Encoder and Counter.
The number of pins that can be uses as inputs is limited by the MCU
architecture and the board schematics. The Encoder class supports:
- Defining the module.
- Defining the input pins.
- Defining a pin for an index signal.
- Defining a pin for a reset signal.
- Defining an output pin showing the compare match signal.
- Setting the number of cycles per revolution (min/max).
- Setting the initial value for the position.
- Setting the counting direction.
- Setting a glitch filter.
- Defining callbacks for getting to a specific position, overrun and
underrun (starting the next revolution). These callbacks can be hard
interrupts to ensure short latency.
The encoder counts all phases of a cycle. The span for the position is
2**32, for the revolution is 2**16. The highest input frequency is
CPU-Clock/24. Note that the "phases" argument is emulated at the API
level (the hardware will always count all phases).
The Counter mode counts single pulses on input A of the Encoder. The
configuration supports:
- Defining the module.
- Defining the input pin.
- Defining the counting direction, either fixed or controlled by the level
of an input pin.
- Defining a pin for an index signal.
- Defining an output pin showing the compare match signal.
- Setting the counter value.
- Setting the glitch filter.
- Defining a callback which is called at a certain value.
- Settings for MIMXRT1015. The MIMXRT1015 MCU has only one encoder/counter
unit.
The counting range is 0 - 2**32-1 and a 16 bit overrun counter. The
highest input frequency is CPU-Clock/12.
The implementation of the `.irq()` method uses the common code from
`shared/runtime/mpirq.c`, including the `irq().flags()` and
`irq().trigger()` methods.
Signed-off-by: robert-hh <robert@hammelrath.com>
Fixes rp2 issue where socket.getaddrinfo() could block indefinitely if an
interface goes down and still has a DNS server configured, as the LWIP
timer stops running and can't time out the DNS query.
Adds a regression test under multi_wlan that times out on rp2 without this
fix.
Fixes issue #18797.
This work was funded through GitHub Sponsors.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
Factor out mp_os_urandom() of each port into extmod/modos.c, which then
calls the port-specific function mp_hal_get_random().
Move mp_hal_get_random() to mphalport where suitable. At the
MIMXRT and SAMD it is left in modos.c, since there are different
implementation depending on the MCU family.
At the ALIF, ESP32, CC3200 and RP2 port the file modos.c was removed,
since it was empty after moving mp_hal_get_random().
Tested for the cc3200, esp32, esp8266, mimxrt, nrf, rp2, samd, stm32
and unix ports. Compiled for the alif and the renesas port.
Signed-off-by: robert-hh <robert@hammelrath.com>
Call `mp_event_handle_nowait()` in the VFS reader buffer refill path so
that pending scheduled events (USB task, network poll, etc.) get processed
during long-running import/parse/compile operations.
Without this, importing a large Python module from the filesystem blocks
for too long causing TinyUSB event queue to overflow. For example, on
renesas-ra, running a script that imports iperf3 via mpremote run, asserts,
most likely due to SOF interrupts not getting processing:
queue_event at lib/tinyusb/src/device/usbd.c:382
dcd_event_handler at lib/tinyusb/src/device/usbd.c:1318
dcd_event_sof at lib/tinyusb/src/device/dcd.h:237
dcd_int_handler at tinyusb/src/portable/renesas/rusb2/dcd_rusb2.c:964
<signal handler called>
disk_ioctl at extmod/vfs_fat_diskio.c:125
validate at lib/oofatfs/ff.c:3359
f_read at lib/oofatfs/ff.c:3625
file_obj_read at extmod/vfs_fat_file.c:75
mp_stream_rw at py/stream.c:60
mp_reader_vfs_readbyte at extmod/vfs_reader.c:59
next_char at py/lexer.c:174
mp_lexer_to_next at py/lexer.c:713
mp_parse at py/parse.c:1167
Signed-off-by: iabdalkader <i.abdalkader@gmail.com>
If the return value is set only when certain condition are true,
the compiler sometimes raises an error.
Signed-off-by: robert-hh <robert@hammelrath.com>
Both mbedTLS and axTLS have support for producing more detailed error
strings. However, these are not used if the error is raised in stream
protocol functions (read/write/ioctl).
This commit adds support for more detailed error messages from streams.
Under the hood it's using a new MP_STREAM_RAISE_ERROR ioctl request to pass
the error code back to the stream implementation which can raise a more
detailed error. If the ioctl is not implemented, we fall back to the old
behaviour and raise an OSError with the error code.
Currently the detailed messages are only implemented for TLS sockets since
those already had helper functions for raising detailed exceptions, but can
be easily implemented in any other stream.
Signed-off-by: Daniël van de Giessen <daniel@dvdgiessen.nl>
This commit adds support to the `marshal` module to be able to dump
functions that have child functions. For example:
import marshal
def f():
def child():
return 1
return child
marshal.dumps(f.__code__)
It also covers the case of marshalling functions that use list
comprehensions, because a list comprehension uses a child function.
This is made possible by the newly enhanced
`mp_raw_code_save_fun_to_bytes()` that can now handle nested functions.
Unmarshalling via `marshal.loads()` already supports nested functions
because it uses the standard `mp_raw_code_load_mem()` function which is
used to import mpy files (and hence can handle all possibilities).
Signed-off-by: Damien George <damien@micropython.org>
This commit fixes a long standing bug/deficiency in the lwIP socket code,
whereby it would abandon all incoming TCP data if the remote sent a TCP
RST.
This behaviour it tested by the existing
`tests/multi_net/tcp_client_rst.py` and
`tests/multi_net/asyncio_tcp_client_rst.py` tests, and they both fail on
boards like PYBD_SFx and RPI_PICO_W due to the deficiency.
With the fix here, both of those tests now pass on lwIP targets, along with
all existing socket tests.
Signed-off-by: Damien George <damien@micropython.org>
This allows tests like `asyncio_event_queue.py` to succeed under the native
emitter when poll is enabled.
Signed-off-by: Jeff Epler <jepler@unpythonic.net>
This commit reworks the setup procedure for the OpenAMP trace buffer,
used by the libmetal framework to provide cross-core logging data if
needed.
Before these changes, the buffer was provided by MicroPython, as a fixed
size 128 bytes chunk that was accidentally put into the .rodata section,
making it not usable for its intended purpose.
Now, a buffer placed in .bss with a default size of 128 bytes is
provided by MicroPython unless chosen otherwise. A user-chosen buffer
pointer can be provided to MicroPython using the
MICROPY_PY_OPENAMP_TRACE_BUF preprocessor definition. If what
MicroPython provides by default is fine, the buffer size can be
overridden with a new value for the MICROPY_PY_OPENAMP_TRACE_BUF_LEN
preprocessor definition instead.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
TinyUSB defines TUD_OPT_RHPORT which is the same thing, make
shorter definition RHPORT in the two files which use it.
This work was funded through GitHub Sponsors.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
This fixes several assertion errors that were found in fuzz testing, for
unimplemented portions of the websocket spec. The assertions were either
turned into Python exceptions, or the missing functionality was
implemented.
Split frames are now enabled and work, enabling reception of frames up to
64kB (assuming they are encoded with a 16-bit size field).
Frames with a 64-bit size fields remain unsupported but no longer result in
an assertion error. Instead, Initial reception of such a frame will result
in OSError(EIO) and subsequent operations on the same websocket will fail
because framing has been lost.
Transmitting frames larger than 64kB is unsupported. Attempting to
transmit such a frame will result in OSError(ENOBUFS). Subsequent
operations on the websocket are possible.
Signed-off-by: Jeff Epler <jepler@gmail.com>
With the aim of getting consistency, and removing the need to learn an
additional term, replace uses of uPy/uPython with MPy/MicroPython.
Rule of thumb was to use "MPy" abbreviation where "CPy" is used nearby, but
the full word MicroPython otherwise.
This work was funded through GitHub Sponsors.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
Ensures that the underlying socket is opened with the correct protocol as
parsed by `getaddrinfo()`.
Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>
This catches cases where the return type is not a small int, eg it could be
a string, or even a big integer.
Signed-off-by: Damien George <damien@micropython.org>
That is, an object whose type defines the protocol slot.
Note that due to protocol confusion, a variant of the original crasher that
returned e.g., a machine.Pin instance could still lead to a crash (#17852).
Fixes issue #17841.
Signed-off-by: Jeff Epler <jepler@gmail.com>
Signed-off-by: Jeff Epler <jepler@unpythonic.net>
machine.Timer() has inconsistent behaviour between ports: some run
callbacks in hard IRQ context whereas others schedule them like soft IRQs.
As on the rp2 port, add support to the generic software timer for a hard=
argument to explicitly choose between these, setting the default to False
to match the existing behaviour. This enables hard timer callbacks for
the alif, mimxrt and samd ports.
Signed-off-by: Chris Webb <chris@arachsys.com>
Avoiding the double timeout when used with the UART class.
`stream.readinto1()` returns after the first timeout.
Fixes issue #17611.
Signed-off-by: robert-hh <robert@hammelrath.com>
This adds the ability to expose CPU-specific features/extensions to
scripts when the `platform` module is compiled in, by implementing
`platform.processor()`. Right now this is only available on
bare-metal RV32 and RV64.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
When disconnecting from PPP the modem sends a confirmation. This message
is received, like all messages, through the poll() method. lwIP may then
immediately call our status callback with code PPPERR_USER to indicate
the connection was closed. Our callback then immediately proceeds to
free the PCB. Thus, during each new iteration of the loop in poll() we
must check if we haven't disconnected in the meantime to prevent calling
the pppos_input_tcpip with a PCB that is now NULL.
Signed-off-by: Daniël van de Giessen <daniel@dvdgiessen.nl>
If while a polling operation is active the stream is removed we should
stop polling data from that stream.
This was already the intended behaviour, but implemented incorrectly.
Signed-off-by: Daniël van de Giessen <daniel@dvdgiessen.nl>
The lwIP includes are now port-specific.
The `sys_arch_{,un}protect` functions are not used according to line 34
of `extmod/lwip-include/lwipopts_common.h`.
Neither have been touched in nearly a decade.
Signed-off-by: Thomas Watson <twatson52@icloud.com>
This factors code out of the ports and into the common `time.localtime`
implementation in `extmod/modtime.c`. That helps to reduce code
duplication, prevent errors in implementation, and reduce code size on
some ports (mimxrt and stm32 at least).
Signed-off-by: Damien George <damien@micropython.org>
There's a slight code size increase paid for by using setpixel_checked for
the last pixel of a line, instead of repeating the checks inline.
Signed-off-by: Jeff Epler <jepler@gmail.com>
If mp_int_t is wider than int, then the tests such as `xend < 0` can fail
even when the amount of scrolling requested is out of range. This resulted
in a segmentation fault when attempting an out-of-bounds access to the
framebuffer.
Signed-off-by: Jeff Epler <jepler@gmail.com>
`socket.getaddrinfo()` supports the specification of an address family;
however, the LwIP implementation does not use it. This change allows the
application to specify the address family request in DNS resolution. If
no family is specified, it falls back to the default preference
configured with `network.ipconfig()`.
Signed-off-by: Jared Hancock <jared.hancock@centeredsolutions.com>
Pattern objects have two additional parameters for the ::search and ::match
methods to define the starting and ending position of the subject within
the string to be searched.
This allows for searching a sub-string without creating a slice. However,
one caveat of using the start-pos rather than a slice is that the start
anchor (`^`) remains anchored to the beginning of the text.
Signed-off-by: Jared Hancock <jared@greezybacon.me>
If PPP is still connected, freeing the PCB will fail and thus instead we
should trigger a disconnect and wait for the lwIP callback to actually
free the PCB.
When PPP is not connected we should check if the freeing failed, warn
the user if so, and only mark the connection as inactive if not.
When all this happens during garbage collection the best case is that
the PPP connection is already dead, which means the callback will be
called immediately and cleanup will happen correctly. The worst case is
that the connection is still alive, thus we are unable to free the PCB
(lwIP won't let us) and it remains referenced in the netif_list, meaning
a use-after-free happens later when lwIP traverses that linked list.
This change does not fully prevent that, but it *does* improve how the
PPP.active(False) method on the ESP32 port behaves: It no longer
immediately tries to free (and fails), but instead triggers a disconnect
and lets the cleanup happen correctly through the status callback.
Signed-off-by: Daniël van de Giessen <daniel@dvdgiessen.nl>
The status callback runs on the lwIP tcpip_thread, and thus must use the
non-thread-safe API because the thread-safe API would cause a deadlock.
Signed-off-by: Daniël van de Giessen <daniel@dvdgiessen.nl>
This commit implements a generic I2C target/peripheral/"slave" device,
called `machine.I2CTarget`. It can work in two separate modes:
- A general device with interrupts/events/callbacks for low-level I2C
operations like address match, read request and stop.
- A memory device that allows reading/writing a specific region of memory
(or "registers") on the target I2C device.
To make a memory device is very simple:
from machine import I2CTarget
mem = bytearray(8)
i2c = I2CTarget(addr=67, mem=mem)
That's all that's needed to start the I2C target. From then on it will
respond to any I2C controller on the bus, allowing reads and writes to the
mem bytearray.
It's also possible to register to receive events. For example to be
notified when the memory is read/written:
from machine import I2CTarget
def irq_handler(i2c_target):
flags = i2c_target.irq().flags()
if flags & I2CTarget.IRQ_END_READ:
print("controller read target at addr", i2c_target.memaddr)
if flags & I2CTarget.IRQ_END_WRITE:
print("controller wrote target at addr", i2c_target.memaddr)
mem = bytearray(8)
i2c = I2CTarget(addr=67, mem=mem)
i2c.irq(irq_handler)
Instead of a memory device, an arbitrary I2C device can be implemented
using all the events (see docs).
This is based on the discussion in #3935.
Signed-off-by: Damien George <damien@micropython.org>
Contrary to the docs, mbedtls can return more than just
MBEDTLS_ERR_SSL_ALLOC_FAILED when `mbedtls_ssl_setup()` fails. At least
MBEDTLS_ERR_MD_ALLOC_FAILED was also seen on ESP32_GENERIC, but there
could possibly be other error codes.
To cover all these codes, just check if `ret` is non-0, and in that case
do a `gc_collect()` and retry the init.
Signed-off-by: Damien George <damien@micropython.org>
As timeout is of type `mp_uint_t`, it must be printed with UINT_FMT.
Before, the compiler plugin produced an error in the PYBD_SF6 build, which
is a nanboxing build with 64-bit ints.
Signed-off-by: Jeff Epler <jepler@gmail.com>
Jack Whitham
Damien George
Alessandro Gatti
Angus Gratton
robert-hh
iabdalkader
Daniël van de Giessen
Jeff Epler
Chris Liechti
Jeff Epler
Andrew Leech
Chris Webb
Anson Mansfield
Thomas Watson
Jared Hancock