This reverts commit 3289b9b7a7.
The commit broke building on MINGW because the filename became
micropython.exe.exe. A proper solution to support more Windows build
environments requires more thought and testing.
Per the comment found here
https://github.com/micropython/micropython-esp32/issues/209#issuecomment-339855157,
this patch adds finaliser code to prevent memory leaks from ussl objects,
which is especially useful when memory for a ussl context is allocated
outside the uPy heap. This patch is in-line with the finaliser code found
in many modsocket implementations for various ports.
This feature is configured via MICROPY_PY_USSL_FINALISER and is disabled by
default because there may be issues using it when the ussl state *is*
allocated on the uPy heap, rather than externally.
With inplace methods now disabled by default, it makes sense to enable
reverse methods, as they allow for more useful features, e.g. allow
for datetime module to implement both 2 * HOUR and HOUR * 2 (where
HOUR is e.g. timedelta object).
This allows to configure support for inplace special methods separately,
similar to "normal" and reverse special methods. This is useful, because
inplace methods are "the most optional" ones, for example, if inplace
methods aren't defined, the operation will be executed using normal
methods instead.
As a caveat, __iadd__ and __isub__ are implemented even if
MICROPY_PY_ALL_INPLACE_SPECIAL_METHODS isn't defined. This is similar
to the state of affairs before binary operations refactor, and allows
to run existing tests even if MICROPY_PY_ALL_INPLACE_SPECIAL_METHODS
isn't defined.
If MICROPY_PY_ALL_SPECIAL_METHODS is defined, actually define all special
methods (still subject to gating by e.g. MICROPY_PY_REVERSE_SPECIAL_METHODS).
This adds quite a number of qstr's, so should be used sparingly.
VLAs can be expensive on stack usage due to stack alignment requirements,
and also the fact that extra local variables are needed to track the
dynamic size of the stack. So using fixed-size arrays when possible can
help to reduce code size and stack usage.
In this particular case, the maximum value of n_args in the VLA is 2 and so
it's more efficient to just allocate this array with a fixed size. This
reduces code size by around 30 bytes on Thumb2 and Xtensa archs. It also
reduces total stack usage of the function: on Thumb2 the usage with VLA is
between 40 and 48 bytes, which is reduced to 32; on Xtensa, VLA usage is
between 64 and 80 bytes, reduced to 32; on x86-64 it's at least 88 bytes
reduced to 80.
CPython only supports the server_hostname keyword arg via the SSLContext
object, so use that instead of the top-level ssl.wrap_socket. This allows
the test to run on CPython the same as uPy.
Also add the "Host:" header to correctly make a GET request (for URLs that
are hosted on other servers). This is not strictly needed to test the SSL
connection but helps to debug things when printing the response.
Unix naming is historical, before current conventions were established.
All other ports however have it as "modusocket.c", so rename for
consistency and to avoid confusion.
Update makeqstrdata.py to sort strings starting with "__" to the beginning
of qstr list, so they get low qstr id's, guaranteedly fitting in 8 bits.
Then use this property to further compact op_id => qstr mapping arrays.
Per https://docs.python.org/3/library/sys.html#sys.getsizeof:
getsizeof() calls the object’s __sizeof__ method. Previously, "getsizeof"
was used mostly to save on new qstr, as we don't really support calling
this method on arbitrary objects (so it was used only for reporting).
However, normalize it all now.
Not all compilers/analysers are smart enough to realise that this function
is never called if MICROPY_ERROR_REPORTING is not TERSE, because the logic
in the code uses if statements rather than #if to select whether to call
this function or not (MSC in debug mode is an example of this, but there
are others). So just unconditionally compile this helper function. The
code-base anyway relies on the linker to remove unused functions.
The legacy function pyb.repl_uart() is still provided and retains its
original behaviour (it only accepts a UART object). uos.dupterm() will now
accept any object with write/readinto methods. At the moment there is just
1 dupterm slot.
Without this the board will crash when deactivating a stream that doesn't
have a close() method (eg UART) or that raises an exception within the
method (eg user-defined function).
The W5200 and W5500 can support up to 80MHz so 42MHz (the maximum the
pyboard can do in its standard configuration) should be safe.
Tested to give around 1050000 kbytes/sec TCP download speed on a W5500,
which is about 10% more than with the previous SPI speed of 21MHz.
Which Wiznet chip to use is a compile-time option: MICROPY_PY_WIZNET5K
should be set to either 5200 or 5500 to support either one of these
Ethernet chips. The driver is called network.WIZNET5K in both cases.
Note that this commit introduces a breaking-change at the build level
because previously the valid values for MICROPY_PY_WIZNET5K were 0 and 1
but now they are 0, 5200 and 5500.
This patch implements the basic SPI read/write functions for the W5500
chip. It also allows _WIZCHIP_ to be configured externally to select the
specific Wiznet chip.
The uos.dupterm() signature and behaviour is updated to reflect the latest
enhancements in the docs. It has minor backwards incompatibility in that
it no longer accepts zero arguments.
The dupterm_rx helper function is moved from esp8266 to extmod and
generalised to support multiple dupterm slots.
A port can specify multiple slots by defining the MICROPY_PY_OS_DUPTERM
config macro to an integer, being the number of slots it wants to have;
0 means to disable the dupterm feature altogether.
The unix and esp8266 ports are updated to work with the new interface and
are otherwise unchanged with respect to functionality.
So that a pointer to it can be passed as a pointer to math_generic_1. This
patch also makes the function work for single and double precision floating
point.
This patch changes how most of the plain math functions are implemented:
there are now two generic math wrapper functions that take a pointer to a
math function (like sin, cos) and perform the necessary conversion to and
from MicroPython types. This helps to reduce code size. The generic
functions can also check for math domain errors in a generic way, by
testing if the result is NaN or infinity combined with finite inputs.
The result is that, with this patch, all math functions now have full
domain error checking (even gamma and lgamma) and code size has decreased
for most ports. Code size changes in bytes for those with the math module
are:
unix x64: -432
unix nanbox: -792
stm32: -88
esp8266: +12
Tests are also added to check domain errors are handled correctly.
While this console API improves handling on real hardware boards
(e.g. clipboard paste is much more reliable, as well as programmatic
communication), it vice-versa poses problems under QEMU, apparently
because it doesn't emulate UART interrupt handling faithfully. That
leads to inability to run the testsuite on QEMU at all. To work that
around, we have to suuport both old and new console routines, and use
the old ones under QEMU.
We want to close communication object even if there were exceptions
somewhere in the code. This is important for --device exec:/execpty:
which may otherwise leave processing running in the background.
Ideally, these should be configurable from Python (using network module),
but as that doesn't exist, we better off using Zephyr's native bootstrap
configuration facility.
The poweroff() and poweron() methods are used to do soft power control of
the display, and this patch makes these methods work the same for both I2C
and SPI interfaces.
Printing "(null)" when a NULL string pointer is passed to %s is a debugging
feature and not a feature that's relied upon by the code. So it only needs
to be compiled in when debugging (such as assert) is enabled, and saves
roughy 30 bytes of code when disabled.
This patch also fixes this NULL check to not do the check if the precision
is specified as zero.
Header files that are considered internal to the py core and should not
normally be included directly are:
py/nlr.h - internal nlr configuration and declarations
py/bc0.h - contains bytecode macro definitions
py/runtime0.h - contains basic runtime enums
Instead, the top-level header files to include are one of:
py/obj.h - includes runtime0.h and defines everything to use the
mp_obj_t type
py/runtime.h - includes mpstate.h and hence nlr.h, obj.h, runtime0.h,
and defines everything to use the general runtime support functions
Additional, specific headers (eg py/objlist.h) can be included if needed.
Qstr values fit in 16-bits (and this fact is used elsewhere in the code) so
no need to use more than that for the large lookup tables. The compiler
will anyway give a warning if the qstr values don't fit in 16 bits. Saves
around 80 bytes of code space for Thumb2 archs.
Building mpy-cross: this patch adds .exe to the PROG name when building
executables for host (eg mpy-cross) on Windows. make clean now removes
mpy-cross.exe under Windows.
Building MicroPython: this patch sets MPY_CROSS to mpy-cross.exe or
mpy-cross so they can coexist and use cygwin or WSL without rebuilding
mpy-cross. The dependency in the mpy rule now uses mpy-cross.exe for
Windows and mpy-cross for Linux.
CPython docs explicitly state that the RHS of a set/frozenset binary op
must be a set to prevent user errors. It also preserves commutativity of
the ops, eg: "abc" & set() is a TypeError, and so should be set() & "abc".
This change actually decreases unix (x64) code by 160 bytes; it increases
stm32 by 4 bytes and esp8266 by 28 bytes (but previous patch already
introduced a much large saving).
A lot of set's methods (the mutable ones) are not allowed to operate on a
frozenset, and giving frozenset a separate locals dict with only the
methods that it supports allows to simplify the logic that verifies if
args are a set or a frozenset. Even though the new frozenset locals dict
is relatively large (88 bytes on 32-bit archs) there is a much bigger
saving coming from the removal of a const string for an error message,
along with the removal of some checks for set or frozenset type.
Changes in code size due to this patch are (for ports that changed at all):
unix x64: -56
unix nanbox: -304
stm32: -64
esp8266: -124
cc3200: -40
Apart from the reduced code, frozenset now has better tab-completion
because it only lists the valid methods. And the error message for
accessing an invalid method is now more detailed (it includes the
method name that wasn't found).
This returns a complex number, following CPython behaviour. For ports that
don't have complex numbers enabled this will raise a ValueError which gives
a fail-safe for scripts that were written assuming complex numbers exist.
This adds a new configuration option to print runtime warnings and errors to
stderr. On Unix, CPython prints warnings and unhandled exceptions to stderr,
so the unix port here is configured to use this option.
The unix port already printed unhandled exceptions on the main thread to
stderr. This patch fixes unhandled exceptions on other threads and warnings
(issue #2838) not printing on stderr.
Additionally, a couple tests needed to be fixed to handle this new behavior.
This is done by also capturing stderr when running tests.
It removes the need for a wrapper Python function to dispatch to the
framebuf method which makes each function call a bit faster, roughly 2.5x.
This patch also adds the rest of the framebuf methods to the SSD class.
The timer prescaler is buffered by default, and this patch enables ARPE
which buffers the auto-reload register. With both of these registers
buffered it's now possible to smoothly change the timer's frequency and
have a smoothly varying PWM output.
Allow literal minus in char classes to be in trailing position, e.g. [a-c-].
(Previously, minus was allowed only at the start.)
This increases ARM Thumb2 code size by 8 bytes.
Current users of fixed vstr buffers (building file paths) assume that there
is no overflow and do not check for overflow after building the vstr. This
has the potential to lead to NULL pointer dereferences
(when vstr_null_terminated_str returns NULL because it can't allocate RAM
for the terminating byte) and stat'ing and loading invalid path names (due
to the path being truncated). The safest and simplest thing to do in these
cases is just raise an exception if a write goes beyond the end of a fixed
vstr buffer, which is what this patch does. It also simplifies the vstr
code.
The vstr argument to the calls to vstr_add_len are dynamically allocated
(ie fixed_buf=false) and so vstr_add_len will never return NULL. So
there's no need to check for it. Any out-of-memory errors are raised by
the call to m_renew in vstr_ensure_extra.
The aim of this patch is to rewrite the functions that create exception
instances (mp_obj_exception_make_new and mp_obj_new_exception_msg_varg) so
that they do not call any functions that may raise an exception. Otherwise
it's possible to create infinite recursion with an exception being raised
while trying to create an exception object.
The two main things that are done to accomplish this are:
1. Change mp_obj_new_exception_msg_varg to just format the string, then
call mp_obj_exception_make_new to actually create the exception object.
2. In mp_obj_exception_make_new and mp_obj_new_exception_msg_varg try to
allocate all memory first using functions that don't raise exceptions
If any of the memory allocations fail (return NULL) then degrade
gracefully by trying other options for memory allocation, eg using the
emergency exception buffer.
3. Use a custom printer backend to conservatively format strings: if it
can't allocate memory then it just truncates the string.
As part of this rewrite, raising an exception without a message, like
KeyError(123), will now use the emergency buffer to store the arg and
traceback data if there is no heap memory available.
Memory use with this patch is unchanged. Code size is increased by:
bare-arm: +136
minimal x86: +124
unix x64: +72
unix nanbox: +96
stm32: +88
esp8266: +92
cc3200: +80
This allows user classes to implement __abs__ special method, and saves
code size (104 bytes for x86_64), even though during refactor, an issue
was fixed and few optimizations were made:
* abs() of minimum (negative) small int value is calculated properly.
* objint_longlong and objint_mpz avoid allocating new object is the
argument is already non-negative.
Prior to this patch calling pyb.Timer(id) would always create a new timer
instance, even if there was an existing one. This patch fixes this
behaviour to match other peripherals, like UART, such that constructing a
timer with just the id will retrieve any existing instances.
The patch also refactors the way timers are validated on construction to
simplify and reduce code size.
This patch also removes the empty type "pinbase_type" (which crashes if
accessed) and uses "machine_pinbase_type" instead as the type of the
PinBase singleton.
Given that various ports now require submodules, rewrite the section
to be more generic.
Also, add git submodule update command to other sections for easy user
start.
If, for class X, X.__add__(Y) doesn't exist (or returns NotImplemented),
try Y.__radd__(X) instead.
This patch could be simpler, but requires undoing operand swap and
operation switch to get non-confusing error message in case __radd__
doesn't exist.
connect, send, recv, sendto and recvfrom now release the GIL. accept
already releases the GIL because it calls mp_hal_delay_ms() within its
busy-wait loop.
Previous to this patch the i2c.scan() method would do up to 100 probes per
I2C address, to detect the devices on the bus. This repeated probing was a
relic from when the code was copied from the accelerometer initialisation,
which requires to do repeated probes while waiting for the accelerometer
chip to turn on.
But I2C devices shouldn't need more than 1 probe to detect their presence,
and the generic software I2C implementation uses 1 probe successfully. So
this patch changes the implementation to use 1 probe per address, which
significantly speeds up the scan operation.
This is to allow to place reverse ops immediately after normal ops, so
they can be tested as one range (which is optimization for reverse ops
introduction in the next patch).
Originally, there were grouped in blocks of 5, to make it easier e.g.
to assess and numeric code of each. But now it makes more sense to
group it by semantics/properties, and then split in chunks still,
which usually leads to chunks of ~6 ops.
It starts a dichotomy of mp_binary_op_t values which can't appear in the
bytecode. Another reason to move it is to VALUES of OP_* and OP_INPLACE_*
nicely adjacent. This also will be needed for OP_REVERSE_*, to be soon
introduced.
With MBEDTLS_DEBUG_C disabled the function mbedtls_debug_set_threshold()
doesn't exist. There's also no need to call mbedtls_ssl_conf_dbg() so a
few bytes can be saved on disabling that and not needing the mbedtls_debug
callback.
This patch adds a function utf8_check() to check for a valid UTF-8 encoded
string, and calls it when constructing a str from raw bytes. The feature
is selectable at compile time via MICROPY_PY_BUILTINS_STR_UNICODE_CHECK and
is enabled if unicode is enabled. It costs about 110 bytes on Thumb-2, 150
bytes on Xtensa and 170 bytes on x86-64.
This is to keep the top-level directory clean, to make it clear what is
core and what is a port, and to allow the repository to grow with new ports
in a sustainable way.
There are 2 changes:
- remove early initialisation of LSE and replaced it by LSEDRIVE config
(there is no reason to call HAL_RCC_OscConfig twice).
- add initialisation of the variables PLLSAI1Source and PLLSAI1M as they
are needed in Cube HAL 1.8.1.
IEEE floating point is specified such that a comparison of NaN with itself
returns false, and Python respects these semantics. This patch makes uPy
also have these semantics. The fix has a minor impact on the speed of the
object-equality fast-path, but that seems to be unavoidable and it's much
more important to have correct behaviour (especially in this case where
the wrong answer for nan==nan is silently returned).
These are now returned as "operation not supported" instead of raising
TypeError. In particular, this fixes equality for float vs incompatible
types, which now properly results in False instead of exception. This
also paves the road to support reverse operation (e.g. __radd__) with
float objects.
This is achieved by introducing mp_obj_get_float_maybe(), similar to
existing mp_obj_get_int_maybe().
Prior to this patch, the size of the buffer given to pack_into() was checked
for being too small by using the count of the arguments, not their actual
size. For example, a format spec of '4I' would only check that there was 4
bytes available, not 16; and 'I' would check for 1 byte, not 4.
The pack() function is ok because its buffer is created to be exactly the
correct size.
The fix in this patch calculates the total size of the format spec at the
start of pack_into() and verifies that the buffer is large enough. This
adds some computational overhead, to iterate through the whole format spec.
The alternative is to check during the packing, but that requires extra
code to handle alignment, and the check is anyway not needed for pack().
So to maintain minimal code size the check is done using struct_calcsize.
Prior to this patch, the size of the buffer given to unpack/unpack_from was
checked for being too small by using the count of the arguments, not their
actual size. For example, a format spec of '4I' would only check that
there was 4 bytes available, not 16; and 'I' would check for 1 byte, not 4.
This bug is fixed in this patch by calculating the total size of the format
spec at the start of the unpacking function. This function anyway needs to
calculate the number of items at the start, so calculating the total size
can be done at the same time.
This patch makes a repeat counter behave the same as repeating the
typecode, when there are not enough args. For example:
struct.pack('2I', 1) now behave the same as struct.pack('II', 1).
Similar to the existing testcase, but test that returning both value of
native type and instance of another user class from __new__ lead to
__init__ not being called, for better coverage.
NotImplemented means "try other fallbacks (like calling __rop__
instead of __op__) and if nothing works, raise TypeError". As
MicroPython doesn't implement any fallbacks, signal to raise
TypeError right away.
This upgrades the HAL to the versions:
- F4 V1.16.0
- F7 V1.7.0
- L4 V1.8.1
The main changes were in the SD card driver. The vendor changed the SD
read/write functions to accept block number intead of byte address, so
there is no longer any need for a custom patch for this in stm32lib.
The CardType values also changed, so pyb.SDCard().info() will return
different values for the 3rd element of the tuple, but this function was
never documented.
The unary-op/binary-op enums are already defined, and there are no
arithmetic tricks used with these types, so it makes sense to use the
correct enum type for arguments that take these values. It also reduces
code size quite a bit for nan-boxing builds.
The SPI flash driver now supports using an arbitrary SPI object to
communicate with the flash chip, and in particular can use a hardware SPI
peripheral.
Otherwise, it will silently get incorrect result on other values types,
including CPython tuple form like "foo.png".endswith(("png", "jpg"))
(which MicroPython doesn't support for unbloatedness).
Allows for simpler, smaller and faster code at run time when selecting the
boards frequency, and allows more customisation opportunities for the PLL
values depending on the target MCU.
Changes for F7 are:
- machine.reset_cause() now reports DEEPSLEEP_RESET correctly;
- machine.sleep() is further optimised to reduce power consumption;
- machine.deepsleep() is now implemented and working.
For SEEK_SET, offset should be treated as unsigned, to allow full-width
stream sizes (e.g. 32-bit instead of 31-bit). This is now fully documented
in stream.h. Also, seek symbolic constants are added.
Too big positive, or too big negative offset values could lead to overflow
and address space wraparound and thus access to unrelated areas of memory
(a security issue).
State that this doc describes generic, "core" MicroPython functionality,
any particular port may diverge in both directions, by both omitting
some functionality, and adding more, both cases described outside the
generic documentation.
Describe that the only portable way to deal with addresses is by using
getaddrinfo(). Describe that some ports may support tuple addresses using
"socket" module (vs "usocket" of native MicroPython).
The original issue leading to crash on startup if no default network
interface was presented, was resolved some time ago.
Note that this enables generic networking subsystem, not networking
on Carbon.
This clarifies return values and the handling of invalid (e.g. newline)
characters.
Encoding conforms to RFC 3548, but decoding does not, as it ignores invalid
characters in base64 input. Instead, it conforms to MIME handling of base64
(RFC 2045).
Note that CPython doesn't document handling of invalid characters in
a2b_base64() docs:
https://docs.python.org/3/library/binascii.html#binascii.a2b_base64 , so
we specify it more explicitly than it, based on CPython's actual behavior
(with which MicroPython now compliant).
This implementation ignores invalid characters in the input. This allows
it to decode the output of b2a_base64, and also mimics the behavior of
CPython.
The value of 0 can't be used because otherwise mp_binary_get_size will let
a null byte through as the type code (intepreted as byterray). This can
lead to invalid type-specifier strings being let through without an error
in the struct module, and even buffer overruns.
current_tid() returns current thread ID. stacks_analyze() calls
k_call_stacks_analyze() which, with CONFIG_INIT_STACKS enabled,
will print stack usage for some well-known threads in the system.
- Changed: ValueError, TypeError, NotImplementedError
- OSError invocations unchanged, because the corresponding utility
function takes ints, not strings like the long form invocation.
- OverflowError, IndexError and RuntimeError etc. not changed for now
until we decide whether to add new utility functions.
Since the stride is specified in pixels, in a 4-bit horizontal format it
has to always be even, otherwise the computation is wrong and we can
write outside of the buffer sometimes.
Before this patch the mperrno.h file could be included and would silently
succeed with incorrect config settings, because mpconfig.h was not yet
included.
If constants (eg mp_const_none_obj) are placed in very high memory
locations that require 64-bits for the pointer then the assembler must be
able to emit instructions to move such pointers to one of the top 8
registers (ie r8-r15).
It's not used anywhere else in the VM loop, and clashes with (is shadowed
by) the n_state variable that's redeclared towards the end of the
mp_execute_bytecode function. Code size is unchanged.
The code conventions suggest using header guards, but do not define how
those should look like and instead point to existing files. However, not
all existing files follow the same scheme, sometimes omitting header guards
altogether, sometimes using non-standard names, making it easy to
accidentally pick a "wrong" example.
This commit ensures that all header files of the MicroPython project (that
were not simply copied from somewhere else) follow the same pattern, that
was already present in the majority of files, especially in the py folder.
The rules are as follows.
Naming convention:
* start with the words MICROPY_INCLUDED
* contain the full path to the file
* replace special characters with _
In addition, there are no empty lines before #ifndef, between #ifndef and
one empty line before #endif. #endif is followed by a comment containing
the name of the guard macro.
py/grammar.h cannot use header guards by design, since it has to be
included multiple times in a single C file. Several other files also do not
need header guards as they are only used internally and guaranteed to be
included only once:
* MICROPY_MPHALPORT_H
* mpconfigboard.h
* mpconfigport.h
* mpthreadport.h
* pin_defs_*.h
* qstrdefs*.h
Prior to this patch Servo numbers 1, 2, 3, 4 mapped to pins X3, X4, X1, X2
on PYBLITE which doesn't match the standard PYB mapping. This patch fixes
the mapping.
The default frozen modules are no longer included (but users can still
specify their own via FROZEN_MPY_DIR), complex numbers are disabled and so
are the native, viper and asm_thumb emitters. Users needing these features
can tune the build to disable other things.
Reasons to disable:
- the code is relatively robust so doesn't need full checking in the
main executable, and the coverage build is used for full testing
with assertions still enabled;
- reduces code size noticeably, by 27k for x86-64 and 20k for x86;
- allows to more easily track changes in code size, since assertions
can skew things.
Got tired of running rm -rf manually. Make should clean, and should clean
fast. Also, fix always-running config-related commands (by having per-board
merged configs).
This makes top-level ToC of the pyboard docs consistent with other ports
(consisting of 3 chapters: QuickRef, General Info, and Tutorial).
Also, some other minor tweaks applied, like local ToC for General Info and
headings mentioning pyboard.
This pseudo-section causes artifacts with latexpdf generation (almost
empty page with list containing literal "genindex", "modeindex", "search"
items). For HTML docs, these sections can be accessed from "home" page.
We don't use alpha/beta/RC, so for us version and release should be the
same, or it leads to confusion (for example, current, 1.9.1 docs are
marked as 1.9 at places).
The idea is to allow to define a kind of "macros" for repeatitive text,
so all occurrances can be updated in one place. Unfortunately, RST doesn't
support replacements with arguments, which limits usefulness of them and
should be taken into account.
We have enough terms or references throughout the docs which may be not
immediately clear or have some important nuances. Referencing terms in
gloassary is the best way to deal with that.
The old intro talked about "differences", but there were hardly any
sections describing differences, mostly MicroPython specific features.
On the other hand, we now have real "differences" chapter, though it's
mostly concerned with stdlib differences.
So, try to avoid confusion by changing wording and linking to the other
chapters and contrasting them with what is described in "MicroPython
language".
Prior to this patch there were 2 paths for creating the namedtuple, one for
when no keyword args were passed, and one when there were keyword args.
And alloca was used in the keyword-arg path to temporarily create the array
of elements for the namedtuple, which would then be copied to a
heap-allocated object (the namedtuple itself).
This patch simplifies the code by combining the no-keyword and keyword
paths, and removing the need for the alloca by constructing the namedtuple
on the heap before populating it.
Heap usage in unchanged, stack usage is reduced, use of alloca is removed,
and code size is not increased and is actually reduced by between 20-30
bytes for most ports.
By default the firmware is built with single-precision floating point.
To build a particular board using double precision instead, put the
following line in the mpconfigboard.mk file:
FLOAT_IMPL = double
The while-loop that calls chop_component will guarantee that level==-1 at
the end of the loop. Hence the code following it is unnecessary.
The check for p==this_name will catch imports that are beyond the
top-level, and also covers the case of new_mod_q==MP_QSTR_ (equivalent to
new_mod_l==0) so that check is removed.
There is also a new check at the start for level>=0 to guard against
__import__ being called with bad level values.
Previously, only "selected chapters" were shown in left-pane ToC (of
Read The Docs theme). These chapters were selected out of order. The
rest of chapters were hidden beyond "Documentation Contents" pseudo-
chapter. This arguably led only to confusion, as many people probably
never tried to open that pseudo-chapter, and those who did, were
confused. Such organization is even worse for PDF output, causing
chapters go in mix-mashed order.
So, instead move to single clean ToC. This will allow readers of HTML
to have access to any doc content at their fingertips (and straight
before their eyes), and will allow to finally have clean PDF docs.
A previous version of the 1-wire driver (which was recently replaced by the
current one) had this behaviour and it allows to create a 1-wire bus
without any external pull-up resistors.
And and FLASH_FS, and use "K" values instead of hex numbers for lengths.
The increase of FLASH_TEXT is to allow more frozen bytecode for a
particular user's project. It's not used for anything else.
Move hardware-specific optimizations to the very end of document, and
add visible note that it gives an example for Pyboard. Remove references
to specific hardware technologies, so the doc can be more naturally
used across ports. Various markup updates to adhere to the latest
docs conventions.
This causes `symbol` syntax to be equivalent to :any:`symbol`, which is
in turn the easiest way to cross-reference an arbitrary symbol in the
docs:
http://www.sphinx-doc.org/en/stable/markup/inline.html#role-any
:any: requires at least Sphinx 1.3 (for reference, Ubuntu 16.03 ships
with 1.3.6, the latest 1.6.3).
Any many of our docs, `symbol` is misused to specify arguments to
functions, etc. Refactoring that is in progress. (CODECONVENTIONS
already specify proper syntax for both arguments and xrefs, based
on CPython conventions).
The idea is that these tests can be run with just a test server running
on a test host, with device under test connecting to it, instead of
requiring Internet connection for testing.
Such setup is however WIP, and some tests in net_hosted/ are so far
written to connect to Internet, as there're not test server written
yet. This is expected to evolve over time.
GC finalization should be enabled for modlwip, or it may lead to GC
problems with socket objects. This decreases usable heap size from
36288 to 35968 (-320) bytes.
It belongs with the other pin config functions in machine_pin.c. Also,
esp_mphal.c is put in iRAM so this change saves about 300 bytes of iRAM
(and mp_hal_pin_open_drain is not a time critical function so doesn't
need to be in iRAM).
These drivers can now be used by any port (so long as that port has the
_onewire driver from extmod/modonewire.c).
These drivers replace the existing 1-wire and DS18X20 drivers in the
drivers/onewire directory. The existing ones were pyboard-specific and
not very efficient nor minimal (although the 1-wire driver was written in
pure Python it only worked at large enough CPU frequency).
This commit brings backwards incompatible API changes to the existing
1-wire drivers. User code should be converted to use the new drivers, or
check out the old version of the code and keep a local copy (it should
continue to work unchanged).
The 1-wire bus is defined with fixed timings so there should be no need to
change them dynamically at runtime. Making the timings fixed saves about
270 bytes of code and 20 bytes of RAM.
The reason it was separated is so that the low-level code could be put in
iRAM, for timing reasons. But:
1. Tests show that it's not necessary to have this code in iRAM for it to
function correctly, and taking it out of iRAM reclaims some of that precious
resource. Furthermore, even though these functions were in iRAM there were
some functions that it called (eg pin get/set functions) which were not in
iRAM, so partially defeated the purpose of putting the 1-wire code in iRAM.
2. It's easier to reuse this 1-wire code in other ports if it's in a single
file.
3. If it turns out that certain code does need to be in iRAM then one can
use the MP_FASTCODE macro to do that.
Previous to this patch, a label with value "0" was used to indicate an
invalid label, but that meant a wasted word (at slot 0) in the array of
label offsets. This patch adjusts the label indices so the first one
starts at 0, and the maximum value indicates an invalid label.
This patch fixes a bug whereby the Python stack was not correctly reset if
there was a break/continue statement in the else black of an optimised
for-range loop.
For example, in the following code the "j" variable from the inner for loop
was not being popped off the Python stack:
for i in range(4):
for j in range(4):
pass
else:
continue
This is now fixed with this patch.
This attempts to bootstrap network tests for MicroPython. This commits
sets test/net_inet/ as place for tests which require access to wide
Internet. They aren't intended to be run as part of the main testsuite,
instead to be run manually on demand.
test_tls_sites.py in particular check that it's possible to establish
SSL/TLS connection to select sites on the Internet: few references ones,
plus those for which problems were reported, and resolved.
The CC3100 only allows to set a timeout for receiving data, not for accept,
connect or send. But it can set non-blocking for all these operations and
this patch uses that feature to implement socket timeout in terms of non-
blocking behaviour combined with a loop.
This method follows the new HW API and allows to set a hard or soft IRQ
callback when a Pin has a level change. It still remains to make this
method return a IRQ object.
In CPython 3.4 this raises a SyntaxError. In CPython 3.5+ having a
positional after * is allowed but uPy has the wrong semantics and passes
the arguments in the incorrect order. To prevent incorrect use of a
function going unnoticed it is important to raise the SyntaxError in uPy,
until the behaviour is fixed to follow CPython 3.5+.
These implementations are incorrect (eg f2d and d2f don't handle special
values like 0.0) and proper versions can be provided by libgcc (or
equivalent depending on the toolchain).
libgcc is now linked with the stmhal port so that library will provide
these functions from now on.
The latest fashion is pushing certificate sub-chains, instead of a single
certificate, during TLS handshake. These are pushed via single TLS record
and effectively put minimum size limit on TLS record buffer. Recently,
these commonly grew over 4K, so we have little choice but to adjust.
This patch fixes 2 things when printing a floating-point number that
requires rounding up of the mantissa:
- retain the correct precision; eg 0.99 becomes 1.0, not 1.00
- if the exponent goes from -1 to 0 then render it as +0, not -0
Baremetal ports standardized on providing localtime(). localtime() offers
more functionality, in particular, strftime() can be completely implemented
in Python with localtime().
Taking the address of a local variable leads to increased stack usage, so
the mp_decode_uint_skip() function is added to reduce the need for taking
addresses. The changes in this patch reduce stack usage of a Python call
by 8 bytes on ARM Thumb, by 16 bytes on non-windowing Xtensa archs, and by
16 bytes on x86-64. Code size is also slightly reduced on most archs by
around 32 bytes.
This adds description of implied AbstractNIC base class, which should be
"subclasses" and implemented by a particular network device class.
This is just an initial step in that direction, the API and description
will be elabotated further.
The list starts with the simplest functionality - GPIO, proceeds to
communication interfaces (UART, SPI, I2C), the to time(r) related
things, then everything else.
For a couple of ports, there was information which directory is set
as current after boot. This information doesn't belong to "uos" module,
and is moved to boards' references (which actually already contained
information on which directory is chosen for boot, even if without
explicit mentioning that it becomes current directory, which is now
done).
The implementation is taken from stmhal/input.c, with code added to handle
ctrl-C. This built-in is controlled by MICROPY_PY_BUILTINS_INPUT and is
disabled by default. It uses readline() to capture input but this can be
overridden by defining the mp_hal_readline macro.
This method isn't implemented in any port. It seemed to have originated
in cc3200 port, but actually never was implemented there either. In
general case, it's impossible to implement this method (for example, for
a perfect GPO, which has only output latch without any feedback look
into a CPU).
For make v3.81, using "make -B" can set $? to empty and in this case the
auto-qstr generation needs to pass all args (ie $^) to cpp. The previous
fix for this (which was removed in 23a693ec2d)
used if statements in the shell command, which gave very long lines that
didn't work on certain systems (eg cygwin).
The fix in this patch is to use an $if(...) expression, which will evaluate
to $? (only newer prerequisites) if it's non empty, otherwise it will use
$^ (all prerequisites).
Previous to this patch the mp_emit_bc_adjust_stack_size function would
adjust the current stack size but would not increase the maximum stack size
if the current size went above it. This meant that certain Python code
(eg a try-finally block with no statements inside it) would not have enough
Python stack allocated to it.
This patch fixes the problem by always checking if the current stack size
goes above the maximum, and adjusting the latter if it does.
This patch fixes a regression introduced by
71a3d6ec3b
Previous to this patch the n_state variable was referring to that computed
at the very start of the mp_execute_bytecode function. This patch fixes it
so that n_state is recomputed when the code_state changes.
For consistent Pin/Signal class hierarchy. With it, Signal is a proper
(while still ducktyped) subclass of a Pin, and any (direct) usage of Pin
can be replace with Signal.
As stmhal's class is reused both as machine.Pin and legacy pyb.Pin,
high/low methods actually retained there.
Both aren't part of generic Hardware API: It's impossible to implement
.id() method in a generic case (e.g., when Pin is instantiated by the
underlying OS/RTOS). .board attribute is an obvious space hog which
instead can be implemented on Python level if needed.
Working on a build with PY_IO enabled (for PY_UJSON support) but PY_SYS_STDFILES disabled (no filesystem). There are multiple references to mp_sys_stdout_obj that should only be enabled if both PY_IO and PY_SYS_STDFILES are enabled.
There're a lot of changes and fixes in 1.8 regarding IP stack,
incompatible with previous Zephyr versions, so supporting them
doesn't make sense.
This is the last commit which should build with Zephyr 1.7.
Tests for an issue with line continuation failing in paste mode due to the
lexer only checking for \n in the "following" character position, before
next_char() has had a chance to convert \r and \r\n to \n.
This ensures that mpy-cross is automatically built (and is up-to-date) for
ports that use frozen bytecode. It also makes sure that .mpy files are
re-built if mpy-cross is changed.
VfsFat no longer has the listdir() method. Rather, if listdir()
functionality is needed then one should use uos.listdir() which will call
VfsFat.ilistdir().
Now consistently uses the EOL processing ("\r" and "\r\n" convert to "\n")
and EOF processing (ensure "\n" before EOF) provided by next_char().
In particular the lexer can now correctly handle input that starts with CR.
Prior to this patch only 'q' and 'Q' type arrays could store big-int
values. With this patch any big int that is stored to an array is handled
by the big-int implementation, regardless of the typecode of the array.
This allows arrays to work with all type sizes on all architectures.
This will lead to crash like:
FATAL: uncaught NLR 80a5420
On x86_32, the minimum heap size is smaller, but not 2 times, so just
use value which works for x86_64.
This patch allows mounting of VFS objects right at the root directory, eg
os.mount(vfs, '/'). It still allows VFS's to be mounted at a path within
the root, eg os.mount(vfs, '/flash'), and such mount points will override
any paths within a VFS that is mounted at the root.
Versions prior to v14.0 have a bug in parsing item functions when used
within a condition: https://github.com/Microsoft/msbuild/issues/368.
Since commit [db9c2e3] this results in an error when building MicroPython
with for example VS2013.
Fix this by creating an intermediate property.
fmode.c should only be included for builds targetting 'pure' windows, i.e.
msvc or mingw builds but not when using msys or cygwin's gcc (see #2298).
Just checking if the OS is windows and UNAME doesn't have msys stil leaves
the gate open for builds with cygwin's gcc since UNAME there is e.g.
CYGWIN_NT-6.1-WOW.
Fix this by checking for 'mingw' explicitly in the compiler version; both
gcc and clang have the -dumpmachine flag so the check should be ok for
all platforms.
The with semantics of this function is close to
pkg_resources.resource_stream() function from setuptools, which
is the canonical way to access non-source files belonging to a package
(resources), regardless of what medium the package uses (e.g. individual
source files vs zip archive). In the case of MicroPython, this function
allows to access resources which are frozen into the executable, besides
accessing resources in the file system.
This is initial stage of the implementation, which actually doesn't
implement "package" part of the semantics, just accesses frozen resources
from "root", or filesystem resource - from current dir.
Do not raise SIGPIPE, instead return EPIPE. Otherwise, e.g. writing
to peer-closed socket will lead to sudden termination of MicroPython
process. SIGPIPE is particularly nasty, because unix shell doesn't
print anything for it, so the above looks like completely sudden and
silent termination for unknown reason. Ignoring SIGPIPE is also what
CPython does. Note that this may lead to problems using MicroPython
scripts as pipe filters, but again, that's what CPython does. So,
scripts which want to follow unix shell pipe semantics (where SIGPIPE
means "pipe was requested to terminate, it's not an error"), should
catch EPIPE themselves.
Peer-closed socket is both readable and writable: read will return EOF,
write - error. Without this poll will hang on such socket.
Note that we don't return POLLHUP, based on argumentation in
http://www.greenend.org.uk/rjk/tech/poll.html that it should apply to
deeper disconnects, for example for networking, that would be link layer
disconnect (e.g. WiFi went down).
Make qstr generation depend on modifications in mpconfigport.h, mpconfig.h
and makeqstrdata.py and if any of those change scan all source files for
qstrs again since they might have changed (for example typcially when
enabling new features in mpconfig.h).
This fixes#2982 for msvc builds.
The standard preprocessor definition to differentiate debug and non-debug
builds is NDEBUG, not DEBUG, so don't rely on the latter:
- just delete the use of it in objint_longlong.c as it has been stale code
for years anyway (since commit [c4029e5]): SUFFIX isn't used anywhere.
- replace DEBUG with MICROPY_DEBUG_NLR in nlr.h: it is rarely used anymore
so can be off by default
This patch allows the following code to run without allocating on the heap:
super().foo(...)
Before this patch such a call would allocate a super object on the heap and
then load the foo method and call it right away. The super object is only
needed to perform the lookup of the method and not needed after that. This
patch makes an optimisation to allocate the super object on the C stack and
discard it right after use.
Changes in code size due to this patch are:
bare-arm: +128
minimal: +232
unix x64: +416
unix nanbox: +364
stmhal: +184
esp8266: +340
cc3200: +128
This patch refactors the handling of the special super() call within the
compiler. It removes the need for a global (to the compiler) state variable
which keeps track of whether the subject of an expression is super. The
handling of super() is now done entirely within one function, which makes
the compiler a bit cleaner and allows to easily add more optimisations to
super calls.
Changes to the code size are:
bare-arm: +12
minimal: +0
unix x64: +48
unix nanbox: -16
stmhal: +4
cc3200: +0
esp8266: -56
Prior to making this a config option it was previously available on these
(and all other) ports, and it makes sense to keep it enabled for mpy-cross
as well as ports that have a decent amount of space for the code.
With this optimisation enabled the compiler optimises the if-else
expression within a return statement. The optimisation reduces bytecode
size by 2 bytes for each use of such a return-if-else statement. Since
such a statement is not often used, and costs bytes for the code, the
feature is disabled by default.
For example the following code:
def f(x):
return 1 if x else 2
compiles to this bytecode with the optimisation disabled (left column is
bytecode offset in bytes):
00 LOAD_FAST 0
01 POP_JUMP_IF_FALSE 8
04 LOAD_CONST_SMALL_INT 1
05 JUMP 9
08 LOAD_CONST_SMALL_INT 2
09 RETURN_VALUE
and to this bytecode with the optimisation enabled:
00 LOAD_FAST 0
01 POP_JUMP_IF_FALSE 6
04 LOAD_CONST_SMALL_INT 1
05 RETURN_VALUE
06 LOAD_CONST_SMALL_INT 2
07 RETURN_VALUE
So the JUMP to RETURN_VALUE is optimised and replaced by RETURN_VALUE,
saving 2 bytes and making the code a bit faster.
Otherwise the type of parse-node and its kind has to be re-extracted
multiple times. This optimisation reduces code size by a bit (16 bytes on
bare-arm).
Add definitions/source files for features which work on the windows
ports but weren't yet enabled.
UTIME related lines are moved a couple of lines up to make comparision
with unix/mpconfigport.h easier in the future.
Sometimes when setting a channel callback the callback fires immediately,
even if the compare register is set to a value far into the future. This
happens when the free running counter has previously been equal to what
happens to be in the compare register.
This patch make sure that there is no pending interrupt when setting a
callback.
It controls the character that's used to (asynchronously) raise a
KeyboardInterrupt exception. Passing "-1" allows to disable the
interception of the interrupt character (as long as a port allows such a
behaviour).
This aligns the I2C class to match the standard machine.I2C API.
Note that this is a (small) breaking change to the existing cc3200 API.
The original API just returned the size of the input buffer so there's no
information lost by this change. To update scripts users should just use
the size of the buffer passed to these functions to get the number of bytes
that are read/written.
This is a user-facing change to the cc3200's API, to make it conform to the
new machine hardware API. The changes are:
- change I2C constructor to: I2C(id=0, *, freq=100000, scl=None, sda=None)
- change I2C init to: init(*, freq, scl, sda)
- removal of machine.I2C.MASTER constant
- I2C str/repr no longer prints I2C.MASTER
To update existing code it should be enough to just remove the I2C.MASTER
constant from contructor/init for I2C.
If we got a CRASH result, return early, similar to SKIP. This is important
because previous refactor changed branching logic a bit, so CRASH now gets
post-processed into CRASH\n, which broke remote hardware tests.
A shorter name takes less code size, less room in scripts and is faster to
type at the REPL.
Tests and HW-API examples are updated to reflect the change.
As Zephyr currently doesn't handle MTU itself (ZEP-1998), limit amount
of data we send on our side.
Also, if we get unsuccessful result from net_nbuf_append(), calculate
how much data it has added still. This works around ZEP-1984.
If a finaliser raises an exception then it must not propagate through the
GC sweep function. This patch protects against such a thing by running
finaliser code via the mp_call_function_1_protected call.
This patch also adds scheduler lock/unlock calls around the finaliser
execution to further protect against any possible reentrancy issues: the
memory manager is already locked when doing a collection, but we also don't
want to allow any scheduled code to run, KeyboardInterrupts to interupt the
code, nor threads to switch.
The common cases for inheritance are 0 or 1 parent types, for both built-in
types (eg built-in exceptions) as well as user defined types. So it makes
sense to optimise the case of 1 parent type by storing just the type and
not a tuple of 1 value (that value being the single parent type).
This patch makes such an optimisation. Even though there is a bit more
code to handle the two cases (either a single type or a tuple with 2 or
more values) it helps reduce overall code size because it eliminates the
need to create a static tuple to hold single parents (eg for the built-in
exceptions). It also helps reduce RAM usage for user defined types that
only derive from a single parent.
Changes in code size (in bytes) due to this patch:
bare-arm: -16
minimal (x86): -176
unix (x86-64): -320
unix nanbox: -384
stmhal: -64
cc3200: -32
esp8266: -108
This implements the orginal idea is that Signal is a subclass of Pin, and
thus can accept all the same argument as Pin, and additionally, "inverted"
param. On the practical side, it allows to avoid many enclosed parenses for
a typical declararion, e.g. for Zephyr:
Signal(Pin(("GPIO_0", 1))).
Of course, passing a Pin to Signal constructor is still supported and is the
most generic form (e.g. Unix port will only support such form, as it doesn't
have "builtin" Pins), what's introduces here is just practical readability
optimization.
"value" kwarg is treated as applying to a Signal (i.e. accounts for possible
inversion).
This buffer is used to allocate objects temporarily, and such objects
require that their underlying memory be correctly aligned for their data
type. Aligning for mp_obj_t should be sufficient for emergency exceptions,
but in general the memory buffer should aligned to the maximum alignment of
the machine (eg on a 32-bit machine with mp_obj_t being 4 bytes, a double
may not be correctly aligned).
This patch fixes a bug for certain nan-boxing builds, where mp_obj_t is 8
bytes and must be aligned to 8 bytes (even though the machine is 32 bit).
Changes made are:
- Use the time module in place of the pyb module for delays.
- Use spi.read/spi.write instead of spi.send/spi.receive.
- Drop some non-portable parameters to spi and pin initialization.
Thanks to @deshipu for the original patch.
Internal structure of k_fifo changed between 1.7 and 1.8, so we need
to abstract it away. This adds more functions than currently used, for
future work.
This allows to execute a command and communicate with its stdin/stdout
via pipes ("exec") or with command-created pseudo-terminal ("execpty"),
to emulate serial access. Immediate usecase is controlling a QEMU process
which emulates board's serial via normal console, but it could be used
e.g. with helper binaries to access real board over other hadware
protocols, etc.
An example of device specification for these cases is:
--device exec:../zephyr/qemu.sh
--device execpty:../zephyr/qemu2.sh
Where qemu.sh contains long-long qemu startup line, or calls another
command. There's a special support in this patch for running the command
in a new terminal session, to support shell wrappers like that (without
new terminal session, only wrapper script would be terminated, but its
child processes would continue to run).
Without this, if there's a large chunk of data coming from hardware (e.g.
clipboard paste, or fed programmatically from the other side of the console),
there's a behavior of initial mass fill-in of the buffer without any
consumption, which starts much later and doesn't catch up with further
filling, leading to buffer overflow.
MONO_xxx is much easier to read if you're not familiar with the code.
MVLSB is deprecated but kept for backwards compatibility, for the time
being.
This patch also updates the associated docs and tests.
This should be a little more efficient (since we anyway scan the input
packet for the interrupt char), and it should also fix any non-atomic read
issues with the buffer state being changed during an interrupt.
Throughput tests show that RX rate is unchanged by this patch.
The previous timeout value of 150ms could lead to data being lost (ie never
received by the host) in some rare cases, eg when the host is under load.
A value of 500ms is quite conservative and allows the host plenty of time
to read our data.
Hashing of float and complex numbers that are exact (real) integers should
return the same integer hash value as hashing the corresponding integer
value. Eg hash(1), hash(1.0) and hash(1+0j) should all be the same (this
is how Python is specified: if x==y then hash(x)==hash(y)).
This patch implements the simplest way of doing float/complex hashing by
just converting the value to int and returning that value.
Split this setting from MICROPY_CPYTHON_COMPAT. The idea is to be able to
keep MICROPY_CPYTHON_COMPAT disabled, but still pass more of regression
testsuite. In particular, this fixes last failing test in basics/ for
Zephyr port.
The first memmove now copies less bytes in some cases (because len_adj <=
slice_len), and the memcpy is replaced with memmove to support the
possibility that dest and slice regions are overlapping.
The foundation of recv() support is per-socket queue of incoming packets,
implemented using Zephyr FIFO object. This patch implements just recv()
for UDP, because TCP recv() requires much more fine-grained control of
network fragments and handling other issues, like EOF condition, etc.
This follows the pattern of how all other headers are now included, and
makes it explicit where the header file comes from. This patch also
removes -I options from Makefile's that specify the mp-readline/timeutils/
netutils directories, which are no longer needed.
Such constants are MCU specific so shouldn't be specified in the board
config file (else it leads to too much duplication of code).
This patch also adds I2C timing values for the F767/F769 for 100k, 400k
and 1MHz I2C bus frequencies.
By default the SDIO (F4) or SDMMC1 (L4, F7) is used as the SD card
peripheral, but if a board config defines MICROPY_HW_SDMMC2_CK and other
pins then the SD card driver will use SDMMC2.
Build happens in 3 stages:
1. Zephyr config header and make vars are generated from prj.conf.
2. libmicropython is built using them.
3. Zephyr is built and final link happens.
Allows to get event time for a head item in the queue. The usecase
if waiting for the next event *OR* I/O completion. I/O completion may
happen before event triggers, and then wait should continue for the
remaining event time (or I/O completion may schedule another earlier
event altogether).
The new function has a strongly provisional status - it may be converted
to e.g. peek() function returning all of the event fields, not just time.
With the existing timeout of 100ms the transfer would end prematurely if
the baudrate was low and the number of bytes to send was high. This patch
fixes the problem by making the timeout proportional to the number of bytes
that are being transferred.
This patch changes mp_uint_t to size_t for the len argument of the
following public facing C functions:
mp_obj_tuple_get
mp_obj_list_get
mp_obj_get_array
These functions take a pointer to the len argument (to be filled in by the
function) and callers of these functions should update their code so the
type of len is changed to size_t. For ports that don't use nan-boxing
there should be no change in generate code because the size of the type
remains the same (word sized), and in a lot of cases there won't even be a
compiler warning if the type remains as mp_uint_t.
The reason for this change is to standardise on the use of size_t for
variables that count memory (or memory related) sizes/lengths. It helps
builds that use nan-boxing.
With this patch all illegal assignments are reported as "can't assign to
expression". Before the patch there were special cases for a literal on
the LHS, and for augmented assignments (eg +=), but it seems a waste of
bytes (and there are lots of bytes used in error messages) to spend on
distinguishing such errors which a user will rarely encounter.
By removing the 'E' code from the operator token encoding mini-language the
tokenising can be simplified. The 'E' code was only used for the !=
operator which is now handled as a special case; the optimisations for the
general case more than make up for the addition of this single, special
case. Furthermore, the . and ... operators can be handled in the same way
as != which reduces the code size a little further.
This simplification also removes a "goto".
Changes in code size for this patch are (measured in bytes):
bare-arm: -48
minimal x86: -64
unix x86-64: -112
unix nanbox: -64
stmhal: -48
cc3200: -48
esp8266: -76
The self variable may be closed-over in the function, and in that case the
call to super() should load the contents of the closure cell using
LOAD_DEREF (before this patch it would just load the cell directly).
Previous to this patch, if the result of the round function overflowed a
small int, or was inf or nan, then a garbage value was returned. With
this patch the correct big-int is returned if necessary and exceptions are
raised for inf or nan.
The C nearbyint function has exactly the semantics that Python's round()
requires, whereas C's round() requires extra steps to handle rounding of
numbers half way between integers. So using nearbyint reduces code size
and potentially eliminates any source of errors in the handling of half-way
numbers.
Also, bare-metal implementations of nearbyint can be more efficient than
round, so further code size is saved (and efficiency improved).
nearbyint is provided in the C99 standard so it should be available on all
supported platforms.
Previous to this patch, if the result of the trunc/ceil/floor functions
overflowed a small int, or was inf or nan, then a garbage value was
returned. With this patch the correct big-int is returned if necessary,
and exceptions are raised for inf or nan.
It improves readability of code and reduces the chance to make a mistake.
This patch also fixes a bug with nan-boxing builds by rounding up the
calculation of the new NSLOTS variable, giving the correct number of slots
(being 4) even if mp_obj_t is larger than the native machine size.
Instead of having the PlatformToolset property hardcoded to a specific
version just set it to the value of DefaultPlatformToolset: this gets
defined according to the commandline environment in which the build was
started.
Instead of just supporting VS2015 the project can now be built by any
version from VS2013 to VS2017 and normally future versions as well, without
quirks like VS asking whether you want to upgrade the project to the latest
version (as was the case when opening the project in VS2017) or not being
able to build at all (as was the case when opening the project in VS2013).
Also adjust the .gitignore file to ignore any artefacts from VS2017.
The -ansi flag is used for C dialect selection and it is equivalent to -std=c90.
Because it goes right before -std=gnu99 it is ignored as for conflicting flags
GCC always uses the last one.
Now, passing a keyword argument that is not expected will correctly report
that fact. If normal or detailed error messages are enabled then the name
of the unexpected argument will be reported.
This patch decreases the code size of bare-arm and stmhal by 12 bytes, and
cc3200 by 8 bytes. Other ports (minimal, unix, esp8266) remain the same in
code size. For terse error message configuration this is because the new
message is shorter than the old one. For normal (and detailed) error
message configuration this is because the new error message already exists
in py/objnamedtuple.c so there's no extra space in ROM needed for the
string.
GIL behaviour should be handled by the port. And ports probably want to
define sleep_us so that it doesn't release the GIL, to improve timing
accuracy.
We can actually handle interrupts during a thread switch (because we always
have a valid stack), but only if those interrupts don't access any of the
thread state (because the state may not correspond to the stack pointer).
So to be on the safe side we disable interrupts during the very short
period of the thread state+stack switch.
The scheduler being locked general means we are running a scheduled
function, and switching to another thread violates that, so don't switch in
such a case (even though we technically could).
And if we are running a scheduled function then we want to finish it ASAP,
so we shouldn't switch to another thread.
Furthermore, ports with threading enabled will lock the scheduler during a
hard IRQ, and this patch to the VM will make sure that threads are not
switched during a hard IRQ (which would crash the VM).
All arguments to pin.irq are converted from keyword-only to positional, and
can still be specified by keyword so it's a backwards compatible change.
The default value for the "trigger" arg is changed from 0 (no trigger)
to rising+falling edge.
ExtInt, Timer and CAN IRQ callbacks are made to work with the scheduler.
They are still hard IRQs by default, but one can now call
micropython.schedule within the hard IRQ to schedule a soft callback.
Instead of always reporting some object cannot be implicitly be converted
to a 'str', even when it is a 'bytes' object, adjust the logic so that
when trying to convert str to bytes it is shown like that.
This will still report bad implicit conversion from e.g. 'int to bytes'
as 'int to str' but it will not result in the confusing
'can't convert 'str' object to str implicitly' anymore for calls like
b'somestring'.count('a').
* Fix mis-spelling of `ticks_add` in code examples.
* Be consistent about parentheses after function names.
* Be consistent about formatting of function, variable and constant names.
* Be consistent about spaces and punctuation.
* Fix some language errors (missing or wrong words, wrong word order).
* Keep line length under 90 chars.
Signed-off-by: Christopher Arndt <chris@chrisarndt.de>
Instead of caching data that is constant (code_info, const_table and
n_state), store just a pointer to the underlying function object from which
this data can be derived.
This helps reduce stack usage for the case when the mp_code_state_t
structure is stored on the stack, as well as heap usage when it's stored
on the heap.
The downside is that the VM becomes a little more complex because it now
needs to derive the data from the underlying function object. But this
doesn't impact the performance by much (if at all) because most of the
decoding of data is done outside the main opcode loop. Measurements using
pystone show that little to no performance is lost.
This patch also fixes a nasty bug whereby the bytecode can be reclaimed by
the GC during execution. With this patch there is always a pointer to the
function object held by the VM during execution, since it's stored in the
mp_code_state_t structure.
This allows to test the PYBV11 target as well as the network drivers
without adding another rule. It also removes the need to use -B,
side-stepping the issue of whether or not -B works with qstr auto
generation.
When make is passed "-B" it seems that everything is considered out-of-date
and so $? expands to all prerequisites. Thus there is no need for a
special check to see if $? is emtpy.
The 'S' typecode is a uPy extension so it should be grouped with the other
extension (namely 'O' typecode). Testing 'S' needs uctypes which is an
extmod module and not always available, so this test is made optional and
will only be run on ports that have (u)struct and uctypes. Otherwise it
will be silently skipped.
Some stack is allocated to format ints, and when the int implementation uses
long-long there should be additional stack allocated compared with the other
cases. This patch uses the existing "fmt_int_t" type to determine the
amount of stack to allocate.
This patch refactors the error handling in the lexer, to simplify it (ie
reduce code size).
A long time ago, when the lexer/parser/compiler were first written, the
lexer and parser were designed so they didn't use exceptions (ie nlr) to
report errors but rather returned an error code. Over time that has
gradually changed, the parser in particular has more and more ways of
raising exceptions. Also, the lexer never really handled all errors without
raising, eg there were some memory errors which could raise an exception
(and in these rare cases one would get a fatal nlr-not-handled fault).
This patch accepts the fact that the lexer can raise exceptions in some
cases and allows it to raise exceptions to handle all its errors, which are
for the most part just out-of-memory errors during construction of the
lexer. This makes the lexer a bit simpler, and also the persistent code
stuff is simplified.
What this means for users of the lexer is that calls to it must be wrapped
in a nlr handler. But all uses of the lexer already have such an nlr
handler for the parser (and compiler) so that doesn't put any extra burden
on the callers.
For example, if the current directory is the root dir then this patch
allows one to do uos.listdir('mnt'), where 'mnt' is a valid mount point.
Previous to this patch such a thing would not work, on needed to do
uos.listdir('/mnt') instead.
Minimal config can be now build with:
./make-minimal BOARD=...
This is required because of Makefile.exports magic, which in its turn depends
on PROJ_CONF to be set correctly at the beginning of Makefile parsing at all
times. Instead of adding more and more workarounds for that, it's better to
just move minimal support to a separate wrapper.
Also, remove Zephyr 1.5 era cruft from Makefile, and add support for Zephyr's
"run" target which supercedes older "qemu" target in upstream.
This is a typical problem with make: we want to trigger rebuilds only
if file actually changed, not if its timestamp changed. In this case,
it's aggravated by the fact that prj.conf depends on the value of
BOARD variable, so we need to do some tricks anyway. We still don't
try to detect if just BOARD changed, just try to generate new
prj.conf.tmp every time (quick), but do actual replacement of prj.conf
only if its content changed.
This is so that the filename of the test doesn't clash with the module name
itself (being "websocket"), and lead to potential problems executing the
test.
MICROPY_LONGINT_IMPL_LONGLONG doesn't have overflow detection, so just
parsing a large number won't give an error, we need to print it out
to check that the whole number was parsed.
These short unit tests test the base uPy methods as well as parts of the
websocket protocol, as implemented by uPy.
@dpgeorge converted the original socket based tests by @hosaka to ones
that only require io.BytesIO.
This test just tests that the basic functions/methods can be called with
the appropriate arguments. There is no real test of underlying
functionality.
Thanks to @hosaka for the initial implementation of this test.
Mostly intended to ease experimentation, no particular plans for APIs
so far (far less their stability), is_preempt_thread() provided is
mostly an example.
By adding back monotonically increasing field in addition to time field.
As heapsort is not stable, without this, among entried added and readded
at the same time instant, some might be always selected, and some might
never be selected, leading to scheduling starvation.
I.e. they don't run successfully with MICROPY_LONGINT_IMPL_NONE
and MICROPY_LONGINT_IMPL_LONGLONG (the problem is that they generate
different output than CPython, TODO to fix that).
The use of large literal numbers is a big no-no when it comes to writing
programs which work with different int representations. Also, some checks
are pretty adhoc (e.g using struct module to check for 64-bitness). This
change bases entire detection on sys.maxsize and integer operarions, and
thus more correct, even if longer.
Note that this change doesn't mean that any of these tests can pass with
anything but MPZ - even despite checking for various int representations,
the tests aren't written to be portable among them.
INT_MAX used previosly is indeed max value for int, whereas on LP64
platforms, long is used for mp_int_t. Using MP_SMALL_INT_MAX is the
correct way to do it anyway.
Each threads needs to have its own private references to its current
locals/globals dicts, otherwise functions running within different
contexts (eg imported from different files) can behave very strangely.
Tests which don't work with small ints are suffixed with _intbig.py. Some
of these may still work with long long ints and need to be reclassified
later.
There were 2 bugs, now fixed by this patch:
- after deleting an element the len of the dict did not decrease by 1
- after deleting an element searching through the dict could lead to
a seg fault due to there being an MP_OBJ_SENTINEL in the ordered array
The renames are:
HAL_Delay -> mp_hal_delay_ms
sys_tick_udelay -> mp_hal_delay_us
sys_tick_get_microseconds -> mp_hal_ticks_us
And mp_hal_ticks_ms is added to provide the full set of timing functions.
Also, a separate HAL_Delay function is added which differs slightly from
mp_hal_delay_ms and is intended for use only by the ST HAL functions.
User can override PYTHON executable before running script,
gen-cpydiff.py works only with Python3 and most systems register
its executable as 'python3'.
In this case, raise an exception without a message.
This would allow to shove few code bytes comparing to currently used
mp_raise_msg(..., "") pattern. (Actual savings depend on function code
alignment used by a particular platform.)
In MicroPython, the path separator is guaranteed to be "/", extra unneeded
things take precious code space (in the port which doesn't have basic things
like floating-port support).
The parser was originally written to work without raising any exceptions
and instead return an error value to the caller. But it's now required
that a call to the parser be wrapped in an nlr handler, so we may as well
make use of that fact and simplify the parser so that it doesn't need to
keep track of any memory errors that it had. The parser anyway explicitly
raises an exception at the end if there was an error.
This patch simplifies the parser by letting the underlying memory
allocation functions raise an exception if they fail to allocate any
memory. And if there is an error parsing the "<id> = const(<val>)" pattern
then that also raises an exception right away instead of trying to recover
gracefully and then raise.
Previous to this patch any non-interned str/bytes objects would create a
special parse node that held a copy of the str/bytes data. Then in the
compiler this data would be turned into a str/bytes object. This actually
lead to 2 copies of the data, one in the parse node and one in the object.
The parse node's copy of the data would be freed at the end of the compile
stage but nevertheless it meant that the peak memory usage of the
parse/compile stage was higher than it needed to be (by an amount equal to
the number of bytes in all the non-interned str/bytes objects).
This patch changes the behaviour so that str/bytes objects are created
directly in the parser and the object stored in a const-object parse node
(which already exists for bignum, float and complex const objects). This
reduces peak RAM usage of the parse/compile stage, simplifies the parser
and compiler, and reduces code size by about 170 bytes on Thumb2 archs,
and by about 300 bytes on Xtensa archs.
This patch allows uPy consts to be bignums, eg:
X = const(1 << 100)
The infrastructure for consts to be a bignum (rather than restricted to
small integers) has been in place for a while, ever since constant folding
was upgraded to allow bignums. It just required a small change (in this
patch) to enable it.
socket.timeout is a subclass of OSError, and using the latter is more
efficient than having a dedicated class. The argument of OSError is
ETIMEDOUT so the error can be distinguished from other kinds of
OSErrors. This follows how the esp8266 port does it.
Since we recently replaced the OSError string messages with simple error
codes, having the uerrno module gets back some user friendly error
messages. The total code size (after removing strings, replacing with
uerrno module) is decreased.
It's configured by MICROPY_PY_UERRNO_ERRORCODE and enabled by default
(since that's the behaviour before this patch).
Without this dict the lookup of errno codes to strings must use the
uerrno module itself.
ftp.c is the only user of this function so making it static in that file
allows it to be inlined. Also, reusing unichar_toupper means we no longer
depend on the C stdlib for toupper, saving about 300 bytes of code space.
This patch introduces the a small framework to track differences between
uPy and CPython. The framework consists of:
- A set of "tests" which test for an individual feature that differs between
uPy and CPy. Each test is like a normal uPy test in the test suite, but
has a special comment at the start with some meta-data: a category (eg
syntax, core language), a human-readable description of the difference, a
cause, and a workaround. Following the meta-data there is a short code
snippet which demonstrates the difference. See tests/cpydiff directory
for the initial set of tests.
- A program (this patch) which runs all the tests (on uPy and CPy) and
generates nicely-formated .rst documenting the differences.
- Integration into the docs build so that everything is automatic, and the
differences appear in a way that is easy for users to read/reference (see
latter commits).
The idea with using this new framework is:
- When a new difference is found it's easy to write a short test for it,
along with a description, and add it to the existing ones. It's also easy
for contributors to submit tests for differences they find.
- When something is no longer different the tool will give an error and
difference can be removed (or promoted to a proper feature test).
These tests are intended to fail, as they provide a programatic record of
differences between uPy and CPython. They also contain a special comment
at the start of the file which has meta-data describing the difference,
including known causes and known workarounds.
Since VS2015 update 2 .db files are used for storing browsing info,
instead of .sdf files. If users don't specify a location for these files
excplicitly they end up in the project directory so ignore them.
It's much more efficient in RAM and code size to do implicit literal string
concatenation in the lexer, as opposed to the compiler.
RAM usage is reduced because the concatenation can be done right away in the
tokeniser by just accumulating the string/bytes literals into the lexer's
vstr. Prior to this patch adjacent strings/bytes would create a parse tree
(one node per string/bytes) and then in the compiler a whole new chunk of
memory was allocated to store the concatenated string, which used more than
double the memory compared to just accumulating in the lexer.
This patch also significantly reduces code size:
bare-arm: -204
minimal: -204
unix x64: -328
stmhal: -208
esp8266: -284
cc3200: -224
Previous to this patch there was an explicit check for errors with line
continuation (where backslash was not immediately followed by a newline).
But this check is not necessary: if there is an error then the remaining
logic of the tokeniser will reject the backslash and correctly produce a
syntax error.
Since the table of keywords is sorted, we can use strcmp to do the search
and stop part way through the search if the comparison is less-than.
Because all tokens that are names are subject to this search, this
optimisation will improve the overall speed of the lexer when processing
a script.
The change also decreases code size by a little bit because we now use
strcmp instead of the custom str_strn_equal function.
Keywords only needs to be searched for if the token is a MP_TOKEN_NAME, so
we can move the seach to the part of the code that does the tokenising for
MP_TOKEN_NAME.
Grammar rules have 2 variants: ones that are attached to a specific
compile function which is called to compile that grammar node, and ones
that don't have a compile function and are instead just inspected to see
what form they take.
In the compiler there is a table of all grammar rules, with each entry
having a pointer to the associated compile function. Those rules with no
compile function have a null pointer. There are 120 such rules, so that's
120 words of essentially wasted code space.
By grouping together the compile vs no-compile rules we can put all the
no-compile rules at the end of the list of rules, and then we don't need
to store the null pointers. We just have a truncated table and it's
guaranteed that when indexing this table we only index the first half,
the half with populated pointers.
This patch implements such a grouping by having a specific macro for the
compile vs no-compile grammar rules (DEF_RULE vs DEF_RULE_NC). It saves
around 460 bytes of code on 32-bit archs.
Allows to iterate over the following without allocating on the heap:
- tuple
- list
- string, bytes
- bytearray, array
- dict (not dict.keys, dict.values, dict.items)
- set, frozenset
Allows to call the following without heap memory:
- all, any, min, max, sum
TODO: still need to allocate stack memory in bytecode for iter_buf.
This patch changes the threading implementation from simple round-robin
with busy waits on mutexs, to proper scheduling whereby threads that are
waiting on a mutex are only scheduled when the mutex becomes available.
This improves efficiency of GIL release within the VM, by only doing the
release after a fixed number of jump-opcodes have executed in the current
thread.
It's more efficient using the system mutexs instead of synthetic ones with
a busy-wait loop. The system can do proper scheduling and blocking of the
threads waiting on the mutex.
Depending on the thread scheduler, a busy-wait loop can hog the CPU and
make the tests very slow. So convert such loops to loops that have an
explicit sleep, allowing the worker threads to do their job.
This allows using the test runner for other scenarios than just
testing uPy itself.
The principle of comparing either to CPython or else to a .exp
file is really handy but to be able to test custom modules not
built into micropython.exe one needs to be able to specify the
module search path a.k.a MICROPYPATH.
MicroPython guarantees '/' to be a path separator, so extra constant taking
precious ROM space are not needed. MicroPython never had such constant, only
one vendor port had it (now unmaintained).
Just one sample is updated with on()/off() for now, there should be
remaining sample(s) showing .value() use (but more can be converted later,
as long as 1 or so good samples of .value() remains).
ipoll() allows to poll streams without allocating any memory: this method
returns an iterator (a poll object itself), and the iterator yields
preallocated "callee-owned tuple" with polling results for each active
stream. The only operation a caller is allowed to do with this tuple is
extracting values from it (storing the tuple as a whole somewhere is
not allowed).
Previous to this patch, for large chunks of bytecode that originated from
a single source-code line, the bytecode-line mapping would generate
something like (for 42 bytecode bytes and 1 line):
BC_SKIP=31 LINE_SKIP=1
BC_SKIP=11 LINE_SKIP=0
This would mean that any errors in the last 11 bytecode bytes would be
reported on the following line. This patch fixes it to generate instead:
BC_SKIP=31 LINE_SKIP=0
BC_SKIP=11 LINE_SKIP=1
This patch implements support for class methods __delattr__ and __setattr__
for customising attribute access. It is controlled by the config option
MICROPY_PY_DELATTR_SETATTR and is disabled by default.
If the mounted object doesn't have a "mount" method then assume it's a
block device and try to detect the filesystem. Since we currently only
support FAT filesystems, the behaviour is to just try and create a VfsFat
object automatically, using the given block device.
Each method asserts and deasserts signal respectively. They are equivalent
to .value(1) and .value(0) but conceptually simpler (and may help to avoid
confusion with inverted signals, where "asserted" state means logical 0
output).
The aapcs-linux ABI is not required, instead the default aapcs ABI is
enough. And using the default ABI means that the provided libgcc will now
link with the firmware without warnings about variable vs fixed enums.
Although the binary size increases by about 1k, RAM usage is slightly
decreased. And libgcc may prove useful in the future for things like
long-long division.
It seems that the gcc toolchain on the RaspberryPi
likes %progbits instead of @progbits. I verified that
%progbits also works under x86, so this should
fix#2848 and fix#2842
I verified that unix and mpy-cross both compile
on my RaspberryPi and on my x64 machine.
The internal map/set functions now use size_t exclusively for computing
addresses. size_t is enough to reach all of available memory when
computing addresses so is the right type to use. In particular, for
nanbox builds it saves quite a bit of code size and RAM compared to the
original use of mp_uint_t (which is 64-bits on nanbox builds).
For archs that have 16-bit pointers, the asmxtensa.h file can give compiler
warnings about left-shift being greater than the width of the type (due to
the inline functions in this header file). Explicitly casting the
constants to uint32_t stops these warnings.
SPI needs to be fast, and calling the EVENT_POLL_HOOK every byte makes it
unusable for ports that need to do non-trivial work in the EVENT_POLL_HOOK
call. And individual SPI transfers should be short enough in time that
EVENT_POLL_HOOK doesn't need to be called.
If something like this proves to be needed in practice then we will need
to introduce separate event hook macros, one for "slow" loops (eg
select/poll) and one for "fast" loops (eg software I2C, SPI).
A few tests still fail on PYBLITE, and that's due to differences in the
available peripheral block numbers on the different MCUs (eg I2C(2)
exists on one, but it's I2C(3) on the other).
This new function controls what happens on a hard-fault:
- debugging disabled: board will do a reset
- debugging enabled: board will print registers and stack and flash LEDs
The default is disabled, ie to do a reset. This is different to previous
behaviour which flashed the LEDs and waited indefinitely.
The port now uses the common mp_utime_ticks_{ms,us,cpu,add,diff} functions
from extmod/utime_mphal.c.
The mp_utime_sleep_XXX functions are still cc3200-specific because they
handle the GIL differently to the ones in extmod.
The files misc/mpsystick.[ch] have been removed because they contain 2
unused functions, and the other remaining function is renamed to
mp_hal_ticks_us and moved to hal/cc3200_hal.c.
machine.time_pulse_us() is intended to provide very fine timing, including
while working with signal bursts, where each transition is tracked in row.
Throwing and handling an exception may take too much time and "signal loss".
So instead, in case of a timeout, just return negative value. Cases of
timeout while waiting for initial signal stabilization, and during actual
timing, are recognized.
The documentation is updated accordingly, and rewritten somewhat to clarify
the function behavior.
This patch fixes two main things:
- dicts can be printed directly using '%s' % dict
- %-formatting should not crash when passed a non-dict to, eg, '%(foo)s'
Updated modbuiltin.c to add conditional support for 3-arg calls to
pow() using MICROPY_PY_BUILTINS_POW3 config parameter. Added support in
objint_mpz.c for for optimised implementation.
This patch brings the _thread module to stmhal/pyboard. There is a very
simple round-robin thread scheduler, which is disabled if there is only
one thread (for efficiency when threading is not used).
The scheduler currently switches threads at a rate of 250Hz using the
systick timer and the pend-SV interrupt.
The GIL is disabled so one must be careful to use lock objects to prevent
concurrent access of objects.
The threading is disabled by default, one can enabled it with the config
option MICROPY_PY_THREAD to test it out.
The first partition is mounted as "/sd" and subsequent partitions are
mounted as "/sd<part_num>". This is backwards compatible with the previous
behaviour, which just mounted the first partition on "/sd".
At this point, only FatFs filesystems are mounted.
A signal is like a pin, but ca also be inverted (active low). As such, it
abstracts properties of various physical devices, like LEDs, buttons,
relays, buzzers, etc. To instantiate a Signal:
pin = machine.Pin(...)
signal = machine.Signal(pin, inverted=True)
signal has the same .value() and __call__() methods as a pin.
Based on the following statistics:
$ git log docs |grep Author | sort | uniq -c | sort -n -r
175 Author: Paul Sokolovsky
135 Author: Damien George
31 Author: Daniel Campora
26 Author: danicampora
14 Author: Peter Hinch
git blame stats script from http://stackoverflow.com/a/13687302/496009:
$ sh git-authors docs
9977 author Damien George
2679 author Paul Sokolovsky
1699 author Daniel Campora
1580 author danicampora
1286 author Peter Hinch
282 author Shuning Bian
249 author Dave Hylands
Total lines per this script: 18417, my contribution is 14.5%.
This patch makes the following configuration changes:
- MICROPY_FSUSERMOUNT is disabled, removing old mounting infrastructure
- MICROPY_VFS is enabled, giving new VFS sub-system
- MICROPY_VFS_FAT is enabled, giving uos.VfsFat type
- MICROPY_FATFS_OO is enabled, to use new ooFatFs lib, R0.12b
User facing API should be almost unchanged. Most notable changes are
removal of os.mkfs (use os.VfsFat.mkfs instead) and pyb.mount doesn't
allow unmounting by passing None as the device.
This provides mp_vfs_XXX functions (eg mount, open, listdir) which are
agnostic to the underlying filesystem type, and just require an object with
the relevant filesystem-like methods (eg .mount, .open, .listidr) which can
then be mounted.
These mp_vfs_XXX functions would typically be used by a port to implement
the "uos" module, and mp_vfs_open would be the builtin open function.
This feature is controlled by MICROPY_VFS, disabled by default.
If MICROPY_VFS_FAT is enabled by a port then the port must switch to using
MICROPY_FATFS_OO. Otherwise a port can continue to use the FatFs code
without any changes.
Previous to this patch the qemu-arm tests were compiled with is_relp=true
meaning that the __repl_print__ function was called for all lines of code
in the outer scope. This is not the right behaviour for scripts that are
executed as though they were a file (eg tests).
With this fix the micropython/heapalloc_str.py test now works so it is
removed from the test blacklist.
In this, don't allocate copy, just return non-empty string. This helps
with a standard pattern of buffering data in case of short reads:
buf = b""
while ...:
s = f.read(...)
buf += s
...
For a typical case when single read returns all data needed, there won't
be extra allocation. This optimization helps uasyncio.
They are one-line functions and having them inline in mp_init/mp_deinit
eliminates the overhead of a function call, and matches how other state
is initialised in mp_init.
The order now follows that in py/mpconfig.h and is a bit cleaner and easier
to maintain. No options were changed/added/removed with this patch, it's
just a reordering.
To use this feature a port should define MICROPY_HW_SPIFLASH_SIZE_BITS
along with x_CS, x_SCK, x_MOSI, x_MISO (x=MICROPY_HW_SPIFLASH). This will
then use external SPI flash on those pins instead of the internal flash.
The SPI is done using the software implementation. There is currently only
support for standard SPI (ie not dual or quad mode).
stmhal will now be built by default with frozen bytecode from scripts
stored in the stmhal/modules/ directory. This can be disabled or
changed to another directory by overridding the make variable
FROZEN_MPY_DIR.
This is how CPython does it, and it's very useful to help users discover
the available modules for a given port, especially built-in and frozen
modules. The function does not list modules that are in the filesystem
because this would require a fair bit of work to do correctly, and is very
port specific (depending on the filesystem).
Ports should no longer use pyhelp_print_obj but instead should define
MICROPY_PY_BUILTINS_HELP to 1 and then specify their help text using
MICROPY_PY_BUILTINS_HELP_TEXT.
If result guaranteedly fits in a small int, it is handled in objint.c.
Otherwise, it is delegated to mp_obj_int_from_bytes_impl(), which should
be implemented by individual objint_*.c, similar to
mp_obj_int_to_bytes_impl().
If GeneratorExit is injected as a throw-value then that should lead to
the close() method being called, if it exists. If close() does not exist
then throw() should not be called, and this patch fixes this.
CPython 3.6 has a few changes that, when run on uPy's test suite, give a
different output to CPython 3.5. uPy currently officially supports the
3.4 language definition, but it's useful to be able to run the test suite
with 3.4/3.5/3.6 versions of CPython. This patch makes such changes to
support 3.6.
The commit d9047d3c8a introduced a bug
whereby "from a.b import c" stopped working for frozen packages. This is
because the path was not properly truncated and became "a//b". Such a
path resolves correctly for a "real" filesystem, but not for a search in
the list of frozen modules.
This effectively reverts the change that introduced this new constant.
The reason is so that users do not need to rebuild the filesystem on
their modules when upgrading the firmware.
Users can change RESERVED_SECS by hand if they need the feature, and in
future firmware it may default to a non-zero value.
This code is no longer pertinent for some time - since switchover to
SDK2.0, there must be correct flash size set for bootloader, or there's
a risk of flash data corruption. And indeed, the correct flash size is
by default auto-detected by esptool.py 1.2.
With caching of map lookups in the bytecode, frozen bytecode can still
work but must be stored in RAM, not ROM. This patch allows mpy-tool.py to
generate code that works with this optimisation, but it's not recommended
to use it on embedded targets (because of lack of RAM).
If sets are not enabled, set literals lead to SyntaxError during parsing,
so it requires feature_check. Set tests are skipped based on set_*.py
pattern.
Starting at esp.flash_user_start(), the reserved sectors are for general
purpose use, for example for native code generation. There is currently
one sector reserved as such.
The driver seems to be be enabling the pullup resistor in most places, but
not this one. Making this one little change allows onewire devices to be
used with no external pullup resistor.
This makes unix "uselect" compatible with baremetal "uselect". Previosuly,
unix version accepted file/socket objects, but internally converted that
to file descriptors, and that's what .poll() returned. To acheive new
behavior, file-like objects are stored internally in an array, in addition
to existing array of struct pollfd. This array is created only on first
case of file-like object being passed to .register(). If only raw fd's are
passed, there will be no additional memory used comparing to the original
implementation.
cc3200tool, https://github.com/ALLTERCO/cc3200tool is a (mostly, some
binary blobs present) open-source, Linux-friendly tool to flash a cc3200
devices. It's an alternative to fully proprietary, Windows-only Uniflash
from TI.
The provided make targets are for erasing flash, flashing the uPy
bootloader and firmware, and flashing vendor's WiFi firmware "servicepacks"
(the latter needs to be downloaded from vendor side, a link is present
inside Makefile).
Sys-tick resolution is 1ms and a value of 2 will give a delay between 1ms
and 2ms (whereas a value of 1 gives a delay between 0ms and 1ms, which is
too short).
The HAL_UART_Transmit function has changed in the latest HAL version such
that the Timeout is a timeout for the entire function, rather than a
timeout between characters as it was before. The HAL function also does
not allow one to reliably tell how many characters were sent before the
timeout (if a timeout occurred).
This patch provides a custom function to do UART transmission, completely
replacing the HAL version, to fix the above-mentioned issues.
UART REPL support was lost in os.dupterm() refactorings, etc. As
os.dupterm() is there, implement UART REPL support at the high level -
if MICROPY_STDIO_UART is set, make default boot.py contain os.dupterm()
call for a UART. This means that changing MICROPY_STDIO_UART value will
also require erasing flash on a module to force boot.py re-creation.
There is a minor functional change with this patch, that the GPIO are now
configured in fast mode, whereas they were in high speed mode before. But
the SDIO should still work because SD CK frequency is at most 25MHz.
They are the same as the existing raw constants (namely 0, 1, 2) but we
want to explicitly show that one can use the HAL's constants if necessary
(eg the mpconfigboard.h files do use the HAL's constants to define the
pull state of certain configurable pins).
Without this the timer will have random values for its State and Lock
entries. The object can then be in a locked state leading to some HAL
functions returning immediately with an error code (which is unchecked).
This patch fixes such a bug which did manifest itself as PWM not working
correctly for LEDs.
This check always fails (ie chr0 is never EOF) because the callers of this
function never call it past the end of the input stream. And even if they
did it would be harmless because 1) reader.readbyte must continue to
return an EOF char if the stream is exhausted; 2) next_char would just
count the subsequent EOF's as characters worth 1 column.
import utimeq, utime
# Max queue size, the queue allocated statically on creation
q = utimeq.utimeq(10)
q.push(utime.ticks_ms(), data1, data2)
res = [0, 0, 0]
# Items in res are filled up with results
q.pop(res)
This is required to avoid extra level of output "cooking" ("\r\r\n") and
make test infrastructure work. On the other hand, this breaks somewhat
Zephyr console abstraction.
Defining and initialising mp_kbd_exception is boiler-plate code and so the
core runtime can provide it, instead of each port needing to do it
themselves.
The exception object is placed in the VM state rather than on the heap.
mp_kbd_exception is now considered the standard variable name to hold the
singleton KeyboardInterrupt exception.
This patch also moves the creation of this object from pyb_usb_init() to
main().
Previous to this patch pyboard.py would open a new serial connection to
the target for each script that was run, and for any command that was run.
Apart from being inefficient, this meant that the board was soft-reset
between scripts/commands, which precludes scripts from accessing variables
set in a previous one.
This patch changes the behaviour of pyboard.py so that the connection to
the target is created only once, and it's not reset between scripts or any
command that is sent with the -c option.
When printing exceptions from files sent to a target by pyboard.py the
filename in the exception is <stdin>, which differs to when running the
script on the PC. So we strip out the filename to make the outputs the
same on all targets (see also misc/print_exception.py test).
sys.exit() is an important function to terminate a program. In particular,
the testsuite relies on it to skip tests (i.e. any other functionality may
be disabled, but sys.exit() is required to at least report that properly).
For all but the last pass the assembler only needs to count how much space
is needed for the machine code, it doesn't actually need to emit anything.
The dummy_data just uses unnecessary RAM and without it the code is not
any more complex (and code size does not increase for Thumb and Xtensa
archs).
This patch moves some common code from the individual inline assemblers to
the compiler, the code that calls the emit-glue to assign the machine code
to the functions scope.
This patch adds the MICROPY_EMIT_INLINE_XTENSA option, which, when
enabled, allows the @micropython.asm_xtensa decorator to be used.
The following opcodes are currently supported (ax is a register, a0-a15):
ret_n()
callx0(ax)
j(label)
jx(ax)
beqz(ax, label)
bnez(ax, label)
mov(ax, ay)
movi(ax, imm) # imm can be full 32-bit, uses l32r if needed
and_(ax, ay, az)
or_(ax, ay, az)
xor(ax, ay, az)
add(ax, ay, az)
sub(ax, ay, az)
mull(ax, ay, az)
l8ui(ax, ay, imm)
l16ui(ax, ay, imm)
l32i(ax, ay, imm)
s8i(ax, ay, imm)
s16i(ax, ay, imm)
s32i(ax, ay, imm)
l16si(ax, ay, imm)
addi(ax, ay, imm)
ball(ax, ay, label)
bany(ax, ay, label)
bbc(ax, ay, label)
bbs(ax, ay, label)
beq(ax, ay, label)
bge(ax, ay, label)
bgeu(ax, ay, label)
blt(ax, ay, label)
bnall(ax, ay, label)
bne(ax, ay, label)
bnone(ax, ay, label)
Upon entry to the assembly function the registers a0, a12, a13, a14 are
pushed to the stack and the stack pointer (a1) decreased by 16. Upon
exit, these registers and the stack pointer are restored, and ret.n is
executed to return to the caller (caller address is in a0).
Note that the ABI for the Xtensa emitters is non-windowing.
This patch allows esp8266 to use @micropython.native and
@micropython.viper function decorators. By default the executable machine
code is written to the space at the end of the iram1 region. The user can
call esp.set_native_code_location() to make the code go to flash instead.
If a port defines MP_PLAT_COMMIT_EXEC then this function is used to turn
RAM data into executable code. For example a port may want to write the
data to flash for execution. The function must return a pointer to the
executable data.
The 512k build recently overflowed because of the newly-enabled uselect
module. uselect is arguable more important than framebuf for small
devices so we disable framebuf to keep the 512k build within its limit.
This is a pure refactoring (and simplification) of code so that stmhal
uses the software SPI class provided in extmod, for the machine.SPI
implementation.
So long as a port defines relevant mp_hal_pin_xxx functions (and delay) it
can make use of this software SPI class without the need for additional
code.
Previous to this patch trying to construct, but not init, a UART that
didn't exist on the target board would actually succeed. Only when
initialising the UART would it then raise an exception that the UART does
not exist.
This patch adds an explicit check that the constructed UART does in fact
exist for the given board.
This follows the pattern of other peripherals (I2C, SPI) to specify the
pins using pin objects instead of a pair of GPIO port and pin number. It
makes it easier to customise the UART pins for a particular board.
These are basic drawing primitives. They work in a generic way on all
framebuf formats by calling the underlying setpixel or fill_rect C-level
primitives.
The STM32 F7 and L4 boards use significantly different code to the F4
boards so it's important to test them with CI. To keep CI build times
within a reasonable limit the STM32F4DISC board is no longer built, it's
anyway very similar to the standard F4 build for PYBv1.0.
If you have longish operations on the db (such as logging data) it may
be desirable to periodically sync the database to the disk. The added
btree.sync() method merely exposes the berkley __bt_sync function to the
user.
The constants MP_IOCTL_POLL_xxx, which were stmhal-specific, are moved
from stmhal/pybioctl.h (now deleted) to py/stream.h. And they are renamed
to MP_STREAM_POLL_xxx to be consistent with other such constants.
All uses of these constants have been updated.
If the destination of os.rename() exists then it will be overwritten if it
is a file. This is the POSIX behaviour, which is also the CPython
behaviour, and so we follow suit.
See issue #2598 for discussion.
Add 2 macros in mphalport.h that clean and invalidate data caches only on
STM32F7 MCUs. They are needed to ensure the cache coherency before/after
DMA transferts.
* MP_HAL_CLEANINVALIDATE_DCACHE cleans and invalidate the data cache. It
must be called before starting a DMA transfer from the peripheral to the
RAM memory.
* MP_HAL_CLEAN_DCACHE cleans the data cache. It must be called before
starting a DMA transfert from the RAM memory to the peripheral.
These macros are called in sdcard.c, before reading from and writing to
the SDCard, when DMA is used.
Fill is a very common operation (eg to clear the screen) and it is worth
optimising it, by providing a specialised fill_rect function for each
framebuffer format.
This patch improved the speed of fill by 10 times for a 16-bit display
with 160*128 pixels.
Rename FrameBuffer1 into FrameBuffer and make it handle different bit
depths via a method table that has getpixel and setpixel. Currently
supported formats are MVLSB (monochrome, vertical, LSB) and RGB565.
Also add blit() and fill_rect() methods.
The temperature sensor on F4 and F7 MCUs is mostly, but not always, on
channel 16. To retain compatibility across all these MCUs this patch
maps the user-facing channel 16 to the internal temperature sensor.
The "mask" parameter is used to select which pins the ADCAll constructor
will initialise to analog mode. It defaults to all pins (0xffffffff),
which is backwards compatible with previous behaviour.
These were inadvertently removed with a recent upgrade to CMSIS, where
those registers were no longer defined in the CMSIS headers, and hence
no longer extracted.
This allows one to construct an I2C object using ids that are specific
to the stmhal port, eg machine.I2C('X'). Right now the implementation
of I2C uses software I2C but the idea is to just change the C-level I2C
protocol functions to hardware implementations later on.
stdio.h was included in all HAL files only to provide
definition of NULL symbol
"stdio.h" includes "types.h" which contains some conflicting definitions
with "drivers/cc3000/inc/socket.h"
HAL Driver before v1.4.2 had a bug which caused clearing all pending
flags in MSR, TSR, RF0R and RF1R instead of only the requested one.
This is why micropython got away without explicitly clearing flags
in IRQ handler.
Current version of HAL drivers optimize IRQ handler by using precalculated
DMA register address and stream bitshift instead of calculating it on every interrupt.
Since we skip call to `HAL_DMA_Init` on reused DMA, fields StreamBaseAddress and StreamIndex
of DMA handle are not initialized and thus leads to SegFault in `DMA_IRQHandler`.
HAL_DMA_Init is a big routine and we do not need to call it on each use of DMA
(ex.: series of I2C operations) and DMA_CalcBaseAndBitshift is really small and
releasing it increases code size by only 8 bytes.
If a port defines MICROPY_READER_POSIX or MICROPY_READER_FATFS then
lexer.c now provides an implementation of mp_lexer_new_from_file using
the mp_reader_new_file function.
Implementations of persistent-code reader are provided for POSIX systems
and systems using FatFS. Macros to use these are MICROPY_READER_POSIX and
MICROPY_READER_FATFS respectively. If an alternative implementation is
needed then a port can define the function mp_reader_new_file.
It is split into 2 functions, one to make small ints and the other to make
a non-small-int leaf node. This reduces code size by 32 bytes on
bare-arm, 64 bytes on unix (x64-64) and 144 bytes on stmhal.
Per the latest HW API, "SPI" class implements only master side of the
protocol, so mode=SPI.MASTER (which was static for WiPy anyway) is not
required (or allowed). This change is required to correspond to updated
documentation of machine.SPI class which no longer lists "mode".
This includes StopIteration and thus are important to make Python-coded
iterables work with yield from/await.
Exceptions in Python send() are still not handled and left for future
consideration and optimization.
We allow 'exc.__traceback__ = None' assignment as a low-level optimization
of pre-allocating exception instance and raising it repeatedly - this
avoids memory allocation during raise. However, uPy will keep adding
traceback entries to such exception instance, so before throwing it,
traceback should be cleared like above.
'exc.__traceback__ = None' syntax is CPython compatible. However, unlike
it, reading that attribute or setting it to any other value is not
supported (and not intended to be supported, again, the only reason for
adding this feature is to allow zero-memalloc exception raising).
Its addition was due to an early exploration on how to add CPython-like
stream interface. It's clear that it's not needed and just takes up
bytes in all ports.
As required for further elaboration of uasyncio, like supporting baremetal
systems with wraparound timesources. This is not intended to be public
interface, and likely will be further refactored in the future.
If an I2C send/recv fails then the peripheral is now checked to see if
it's in a "stuck" state waiting for the stop bit, and if so then it is
reset so that the next I2C transaction can proceed.
This patch also de-inits the I2C peripheral in the init() method, before
init'ing it again.
New keyword option in constructor and init() method is "dma=<bool>".
DMA is now disabled by default for I2C transfers because it currently does
not handle I2C bus errors very well (eg if slave device doesn't ACK or
NACK correctly during a transfer).
The DMA state is always HAL_DMA_STATE_RESET because of the memset clearing
all the data, so prior to this patch HAL_DMA_DeInit was never called. Now
it is always called to make sure the DMA is properly reset.
* The microcontroller: `STM32F405RGT6 <http://www.st.com/web/catalog/mmc/FM141/SC1169/SS1577/LN1035/PF252144>`_ (link to manufacturer's site)
* The accelerometer: `Freescale MMA7660 <http://micropython.org/resources/datasheets/MMA7660FC.pdf>`_ (800kiB PDF)
* The LDO voltage regulator: `Microchip MCP1802 <http://micropython.org/resources/datasheets/MCP1802-22053C.pdf>`_ (400kiB PDF)
Datasheets for other components
===============================
-------------------------------
* The LCD display on the LCD touch-sensor skin: `Newhaven Display NHD-C12832A1Z-FSW-FBW-3V3 <http://micropython.org/resources/datasheets/NHD-C12832A1Z-FSW-FBW-3V3.pdf>`_ (460KiB PDF)
* The touch sensor chip on the LCD touch-sensor skin: `Freescale MPR121 <http://micropython.org/resources/datasheets/MPR121.pdf>`_ (280KiB PDF)
For detailed documentation of the driver for the display see the
:mod:`lcd160cr` module.
Plugging in the display
-----------------------
The display can be plugged directly into a pyboard (all pyboard versions
are supported). You plug the display onto the top of the pyboard either
in the X or Y positions. The display should cover half of the pyboard.
See the picture above for how to achieve this; the left half of the picture
shows the X position, and the right half shows the Y position.
Getting the driver
------------------
You can control the display directly using a power/enable pin and an I2C
bus, but it is much more convenient to use the driver provided by the
:mod:`lcd160cr` module. This driver is included in recent version of the
pyboard firmware (see `here <http://micropython.org/download>`__). You
can also find the driver in the GitHub repository
`here <https://github.com/micropython/micropython/blob/master/drivers/display/lcd160cr.py>`__, and to use this version you will need to copy the file to your
board, into a directory that is searched by import (usually the lib/
directory).
Once you have the driver installed you need to import it to use it::
import lcd160cr
Testing the display
-------------------
There is a test program which you can use to test the features of the display,
and which also serves as a basis to start creating your own code that uses the
LCD. This test program is included in recent versions of the pyboard firmware
@@ -60,10 +60,10 @@ One problem you might find is that if you stop the script and then start it agai
for l in leds:
l.off()
The Fourth Special LED
----------------------
The Special LEDs
----------------
The blue LED is special. As well as turning it on and off, you can control the intensity using the intensity() method. This takes a number between 0 and 255 that determines how bright it is. The following script makes the blue LED gradually brighter then turns it off again. ::
The yellow and blue LEDs are special. As well as turning them on and off, you can control their intensity using the intensity() method. This takes a number between 0 and 255 that determines how bright it is. The following script makes the blue LED gradually brighter then turns it off again. ::
led = pyb.LED(4)
intensity = 0
@@ -72,4 +72,4 @@ The blue LED is special. As well as turning it on and off, you can control the i
led.intensity(intensity)
pyb.delay(20)
You can call intensity() on the other LEDs but they can only be off or on. 0 sets them off and any other number up to 255 turns them on.
You can call intensity() on LEDs 1 and 2 but they can only be off or on. 0 sets them off and any other number up to 255 turns them on.
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
