Printing of uPy floats can differ by the floating-point precision on
different architectures (eg 64-bit vs 32-bit x86), so it's not possible to
using printing of floats in some parts of this test. Instead we can just
check for equivalence with what is known to be the correct answer.
For cases where size_t is smaller than mp_int_t (eg nan-boxing builds) the
difference between two size_t's is not sign extended into mp_int_t and so
the result is never negative. This patch fixes this bug by using ssize_t
for the type of the result.
This gives dir() better behaviour when listing the attributes of a user
type that defines __getattr__: it will now not list those attributes for
which __getattr__ raises AttributeError (meaning the attribute is not
supported by the object).
This patch fixes the possibility of a crash of the REPL when tab-completing
an object which raises an exception when its attributes are accessed.
See issue #3729.
This new helper function acts like mp_load_method_maybe but is wrapped in
an NLR handler so it can catch exceptions. It prevents AttributeError from
propagating out, and optionally all other exceptions. This helper can be
used to fully implement hasattr (see follow-up commit), and also for cases
where mp_load_method_maybe is used but it must now raise an exception.
Commit e269cabe3e added a check that the
first argument to the to_bytes() method is an integer, and now uPy
follows CPython behaviour and raises a TypeError for this test.
Note: CPython checks the argument types before checking the number of
arguments, but uPy does it the other way around, so they give different
exception messages for this test, but still the same type, a TypeError.
On this 32-bit arch there's no need to use the long version of the format
specifier. It's only there to appease the compiler which checks the type
of the args passed to printf. Removing the "l" saves a bit of code space.
The order of function calls in an arithmetic expression is undefined and so
they must be written out as sequential statements.
Thanks to @dv-extrarius for reporting this issue, see issue #3690.
If a socket is cleanly shut down by the peer then reads on this socket
should continue to return zero bytes. The lwIP socket API does not have
this behaviour (it only returns zero once, then blocks on subsequent calls)
so this patch adds explicit checks and logic for peer closed sockets.
Add --init to the submodule update example, thus, all submodules get
initialised including the nested (--recursive) ones. Without it there
might not be a submodule init.
Disabling this saves around 6000 bytes of code space and gets the 512k
build fitting in the available flash again (it increased lately due to an
increase in the size of the ESP8266 SDK).
In adcall.py the pyb module may not be imported, so use ADCAll directly.
In dac.py the DAC object now prints more info, so update .exp file.
In spi.py the SPI should be deinitialised upon exit, so the test can run a
second time correctly.
For a given IRQn (eg UART) there's no need to carry around both a PRI and
SUBPRI value (eg IRQ_PRI_UART, IRQ_SUBPRI_UART). Instead, the IRQ_PRI_UART
value has been changed in this patch to be the encoded hardware value,
using NVIC_EncodePriority. This way the NVIC_SetPriority function can be
used directly, instead of going through HAL_NVIC_SetPriority which must do
extra processing to encode the PRI+SUBPRI.
For a priority grouping of 4 (4 bits for preempt priority, 0 bits for the
sub-priority), which is used in the stm32 port, the IRQ_PRI_xxx constants
remain unchanged in their value.
This patch also "fixes" the use of raise_irq_pri() which should be passed
the encoded value (but as mentioned above the unencoded value is the same
as the encoded value for priority grouping 4, so there was no bug from this
error).
The problem is the existing code which tries to optimise the
reinitialisation of the DMA breaks the abstraction of the HAL. For the
STM32L4 the HAL's DMA setup code maintains two private vars (ChannelIndex,
DmaBaseAddress) and updates a hardware register (CCR).
In HAL_DMA_Init(), the CCR is updated to set the direction of the DMA.
This is a problem because, when using the SD Card interface, the same DMA
channel is used in both directions, so the direction bit in the CCR must
follow that.
A quick and effective fix for the L4 is to simply call HAL_DMA_DeInit() and
HAL_DMA_Init() every time.
This is a more consistent use of errno codes. For example, it may be that
a stream returns MP_EAGAIN but the mp_is_nonblocking_error() macro doesn't
catch this value because it checks for EAGAIN instead (which may have a
different value than MP_EAGAIN when MICROPY_USE_INTERNAL_ERRNO is enabled).
Most modern systems have EWOULDBLOCK aliased to EAGAIN, ie they have the
same value. But some systems use different values for these errnos and if
a uPy port is using the system errno values (ie not the internal uPy
values) then it's important to be able to distinguish EWOULDBLOCK from
EAGAIN. Eg if a system call returned EWOULDBLOCK it must be possible to
check for this return value, and this patch makes this now possible.
If MICROPY_USE_INTERNAL_ERRNO is disabled, MP_EINVAL is not guaranteed
to have the value 22, so we cannot depend on OSError(22,).
Instead, to support any given port's errno values, without relying
on uerrno, we just check that the args[0] is positive.
ADC3 is used because the H7's internal ADC channels are connected to ADC3
and the uPy driver doesn't support more than one ADC.
Only 12-bit resolution is supported because 12 is hard-coded and 14/16 bits
are not recommended on some ADC3 pins (see errata).
Values from internal ADC channels are known to give wrong values at
present.
The esp8266 uses modlwip.c for its usocket implementation, which allows to
easily support callbacks on socket events (like when a socket becomes ready
for reading). This is not as easy to do for the esp32 which uses the
ESP-IDF-provided lwIP POSIX socket API. Socket events are needed to get
WebREPL working, and this patch provides a way for such events to work by
explicitly polling registered sockets for readability, and then calling the
associated callback if the socket is readable.
After calling HAL_SYSTICK_Config the SysTick IRQ priority is set to 15, the
lowest priority. This commit reconfigures the IRQ priority to the desired
TICK_INT_PRIORITY value.
By default the stm module is included in the build, but a board can now
define MICROPY_PY_STM to 0 to not include this module. This reduces the
firmware by about 7k.
To use HSE bypass mode the board should define:
#define MICROPY_HW_CLK_USE_BYPASS (1)
If this is not defined, or is defined to 0, then HSE oscillator mode is
used.
This patch allows a given board to configure which pins are used for the
CAN peripherals, in a similar way to all the other bus peripherals (I2C,
UART, SPI). To enable CAN on a board the mpconfigboard.h file should
define (for example):
#define MICROPY_HW_CAN1_TX (pin_B9)
#define MICROPY_HW_CAN1_RX (pin_B8)
#define MICROPY_HW_CAN2_TX (pin_B13)
#define MICROPY_HW_CAN2_RX (pin_B12)
And the board config file should no longer define MICROPY_HW_ENABLE_CAN.
The individual union members (like SPI, I2C) are never used, only the
generic "reg" entry is. And the union names can clash with macro
definitions in the HAL so better to remove them.
The only configuration that changes with this patch is that on L4 MCUs the
clock prescaler changed from ADC_CLOCK_ASYNC_DIV2 to ADC_CLOCK_ASYNC_DIV1
for the ADCAll object. This should be ok.
A value of DISABLE for EOCSelection is invalid. This would have been
interpreted instead as ADC_EOC_SEQ_CONV, but really it should be
ADC_EOC_SINGLE_CONV for the uses in this code. So this has been fixed.
ExternalTrigConv should be ADC_SOFTWARE_START because all ADC
conversions are started by software. This is now fixed.
This can be used to select the output buffer behaviour of the DAC. The
default values are chosen to retain backwards compatibility with existing
behaviour.
Thanks to @peterhinch for the initial idea to add this feature.
Reading into a bytearray will truncate values to 0xff so the assertions
checking read_timed() would previously always succeed.
Thanks to @peterhinch for finding this problem and providing the solution.
This event queue has UART events posted to it and they need to be drained
for it to operate without error. The queue is not used by the uPy UART
class so it should be removed to prevent the IDF emitting errors.
Fixes#3704.
Instead of emitnative.c having configuration code for each supported
architecture, and then compiling this file multiple times with different
macros defined, this patch adds a file per architecture with the necessary
code to configure the native emitter. These files then #include the
emitnative.c file.
This simplifies emitnative.c (which is already very large), and simplifies
the build system because emitnative.c no longer needs special handling for
compilation and qstr extraction.
Keeping all the stress related tests in one place makes it easier to
stress-test a given port, and to also not run such tests on ports that
can't handle them.
The 2nd and 3rd args of the ternary operator are treated like they are in
the same expression and must have similar types. void is not compatible
with int so that's why the compiler is complaining.
This patch moves the implementation of stream closure from a dedicated
method to the ioctl of the stream protocol, for each type that implements
closing. The benefits of this are:
1. Rounds out the stream ioctl function, which already includes flush,
seek and poll (among other things).
2. Makes calling mp_stream_close() on an object slightly more efficient
because it now no longer needs to lookup the close method and call it,
rather it just delegates straight to the ioctl function (if it exists).
3. Reduces code size and allows future types that implement the stream
protocol to be smaller because they don't need a dedicated close method.
Code size reduction is around 200 bytes smaller for x86 archs and around
30 bytes smaller for the bare-metal archs.
The LHS passed to mp_obj_int_binary_op() will always be an integer, either
a small int or a big int, so the test for this type doesn't need to include
an "other, unsupported type" case.
The string "Q+?" is special in that it hashes to zero with the djb2
algorithm (among other strings), and a zero hash should be incremented to a
hash of 1.
Without the compiler enabled the mp_optimise_value is unused, and the
micropython.opt_level() function is not useful, so exclude these from the
build to save RAM and code size.
The case of a return statement in the try suite of a try-except statement
was previously only tested by builtin_compile.py, and only then in the part
of this test which checked for the existence of the compile builtin. So
this patch adds an explicit unit test for this case.
When pystack is enabled mp_obj_fun_bc_prepare_codestate() will always
return a valid pointer, and if there is no more pystack available then it
will raise an exception (a RuntimeError). So having pystack enabled with
stackless enabled automatically gives strict stackless mode. There is
therefore no need to have code for strict stackless mode when pystack is
enabled, and this patch optimises the VM for such a case.
The VM expects that, if mp_resume() returns MP_VM_RETURN_EXCEPTION, then
the returned value is an exception instance (eg to add a traceback to it).
It's possible that a value passed to a generator's throw() is not an
exception so must be explicitly checked for if the thrown value is not
intercepted by the generator.
Thanks to @jepler for finding the bug.
Prior to this patch the code would crash if a key in a ** dict was anything
other than a str or qstr. This is because mp_setup_code_state() assumes
that keys in kwargs are qstrs (for efficiency).
Thanks to @jepler for finding the bug.
The main() function has a predefined type in C which is not so useful for
embedded contexts. This patch renames main() to stm32_main() so we can
define our own type signature for this function. The type signature is
defined to have a single argument which is the "reset_mode" and is passed
through as r0 from Reset_Handler. This allows, for example, a bootloader
to pass through information into the main application.
The Reset_Handler needs to copy the data section and zero the BSS, and
these operations should be as optimised as possible to reduce start up
time. The versions provided in this patch are about 2x faster (on a Cortex
M4) than the previous implementations.
Rather than pin objects themselves. The actual object is now pin_X_obj and
defines are provided so that pin_X is &pin_X_obj. This makes it so that
code that uses pin objects doesn't need to know if they are literals or
objects (that need pointers taken) or something else. They are just
entities that can be passed to the map_hal_pin_xxx functions. This mirrors
how the core handles constant objects (eg mp_const_none which is
&mp_const_none_obj) and allows for the possibility of different
implementations of the pin layer.
For example, prior to this patch there was the following:
extern const pin_obj_t pin_A0;
#define pyb_pin_X1 pin_A0
...
mp_hal_pin_high(&pin_A0);
and now there is:
extern const pin_obj_t pin_A0_obj;
#define pin_A0 (&pin_A0_obj)
#define pyb_pin_X1 pin_A0
...
mp_hal_pin_high(pin_A0);
This patch should have minimal effect on board configuration files. The
only change that may be needed is if a board has .c files that configure
pins.
This patch forces a board to explicitly define TEXT1_ADDR in order to
split the firmware into two separate pieces. Otherwise the default is now
to produce only a single continuous firmware image with all ISR, text and
data together.
This patch allows a particular board to independently specify the linker
scripts for 1) the MCU memory layout; 2) how the different firmware
sections are arranged in memory. Right now all boards follow the same
layout with two separate firmware section, one for the ISR and one for the
text and data. This leaves room for storage (filesystem data) to live
between the firmware sections.
The idea with this patch is to accommodate boards that don't have internal
flash storage and only need to have one continuous firmware section. Thus
the common.ld script is renamed to common_ifs.ld to make explicit that it
is used for cases where the board has internal flash storage.
Explicitly writing out the implementation of sys_tick_has_passed makes
these bdev files independent of systick.c and more reusable as a general
component. It also reduces the code size slightly.
The irq.h header is added to spibdev.c because it uses declarations in that
file (irq.h is usually included implicitly via mphalport.h but not always).
Taking the address assumes that the pin is an object (eg a struct), but it
could be a literal (eg an int). Not taking the address makes this driver
more general for other uses.
genhdr/pins.h is an internal header file that defines all of the pin
objects and it's cleaner to have pin.h include it (where the struct's for
these objects are defined) rather than an explicit include by every user.
The HAL requires strict aliasing optimisation to be turned on to function
correctly (at least for the SD card driver on F4 MCUs). This optimisation
was recently disabled with the addition of H7 support due to the H7 HAL
having errors with the strict aliasing optimisation enabled. But this is
now fixed in the latest stm32lib and so the optimisation can now be
re-enabled.
Thanks to @chuckbook for finding that there was a problem with the SD card
on F4 MCUs with the strict aliasing optimisation disabled.
The CMSIS files for the STM32 range provide macros to distinguish between
the different MCU series: STM32F4, STM32F7, STM32H7, STM32L4, etc. Prefer
to use these instead of custom ones.
By using pre-compiled regexs, using startswith(), and explicitly checking
for empty lines (of which around 30% of the input lines are), automatic
qstr extraction is speed up by about 10%.
This patch provides a custom (and simple) function to receive data on the
CAN bus, instead of the HAL function. This custom version calls
mp_handle_pending() while waiting for messages, which, among other things,
allows to interrupt the recv() method via KeyboardInterrupt.
mp_spiflash_read had a bug in it where "dest" and "addr" were incremented
twice for a certain special case. This was fixed, which then allowed the
function to be simplified to reduce code size.
mp_spiflash_write had a bug in it where "src" was not incremented correctly
for the case where the data to be written included the caching buffer as
well as some bytes after this buffer. This was fixed and the resulting
code simplified.
Certain pins (eg 4 and 5) seem to behave differently at the hardware level
when in open-drain mode: they glitch when set "high" and drive the pin
active high for a brief period before disabling the output driver. To work
around this make the pin an input to let it float high.
This config variable controls whether to support storage on the internal
flash of the MCU. It is enabled by default and should be explicitly
disabled by boards that don't want internal flash storage.
It makes it cleaner, and simpler to support multiple different block
devices. It also allows to easily extend a given block device with new
ioctl operations.
This patch alters the SPI-flash memory driver so that it uses the new
low-level C SPI protocol (from drivers/bus/spi.h) instead of the uPy SPI
protocol (from extmod/machine_spi.h). This allows the SPI-flash driver to
be used independently from the uPy runtime.
This patch takes the software SPI implementation from extmod/machine_spi.c
and moves it to a dedicated file in drivers/bus/softspi.c. This allows the
SPI driver to be used independently of the uPy runtime, making it a more
general component.
The PWM at full value was not considered as an "active" channel so if no
other channel was used the timer used to mange PWM was not started. So
when another duty value was set the PWM timer restarted and there was a
visible glitch when driving LEDs. Such a glitch can be seen with the
following code (assuming active-low LED on pin 0):
p = machine.PWM(machine.Pin(0))
p.duty(1023) # full width, LED is off
p.duty(1022) # LED flashes brightly then goes dim
This patch fixes the glitch.
This is just to test that the function exists and returns some kind of
valid value. Although this file is for testing ms/us functions, put the
ticks_cpu() test here so not to add a new test file.
The spiflash memory driver is reworked to allow the underlying bus to be
either normal SPI or QSPI. In both cases the bus can be implemented in
software or hardware, as long as the spiflash driver is passed the correct
configuration structure.
All callers of mp_obj_int_formatted() are expected to pass in a valid int
object, and they do:
- mp_obj_int_print() should always pass through an int object because it is
the print special method for int instances.
- mp_print_mp_int() checks that the argument is an int, and if not converts
it to a small int.
This patch saves around 20-50 bytes of code space.
This test for calling gc_realloc() while the GC is locked can be done in
pure Python, so better to do it that way since it can then be tested on
more ports.
Prior to this patch, some architectures (eg unix x86) could render floats
with "negative" digits, like ")". For example, '%.23e' % 1e-80 would come
out as "1.0000000000000000/)/(,*0e-80". This patch fixes the known cases.
Prior to this patch, some architectures (eg unix x86) could render floats
with a ":" character in them, eg 1e+39 would come out as ":e+38" (":" is
just after "9" in ASCII so this is like 10e+38). This patch fixes some of
these cases.
Prior to this patch the %f formatting of some FP values could be off by up
to 1, eg '%.0f' % 123 would return "122" (unix x64). Depending on the FP
precision (single vs double) certain numbers would format correctly, but
others wolud not. This patch should fix all cases of rounding for %f.
This patch eliminates heap allocation in the VFS FAT disk IO layer, when
calling the underlying readblocks/writeblocks methods. The bytearray
object that is passed to these methods is now allocated on the C stack
rather than the heap (it's only 4 words big).
This means that these methods should not retain a pointer to the buffer
object that is passed in, but this was already a restriction because the
original heap-allocated bytearray had its buffer passed by reference.
There's no need to have MP_OBJ_NULL a special case, the code can re-use
the MP_OBJ_STOP_ITERATION value to signal the special case and the VM can
detect this with only one check (for MP_OBJ_STOP_ITERATION).
This patch concerns the handling of an NLR-raised StopIteration, raised
during a call to mp_resume() which is handling the yield from opcode.
Previously, commit 6738c1dded introduced code
to handle this case, along with a test. It seems that it was lucky that
the test worked because the code did not correctly handle the stack pointer
(sp).
Furthermore, commit 79d996a57b improved the
way mp_resume() propagated certain exceptions: it changed raising an NLR
value to returning MP_VM_RETURN_EXCEPTION. This change meant that the
test introduced in gen_yield_from_ducktype.py was no longer hitting the
code introduced in 6738c1dded.
The patch here does two things:
1. Fixes the handling of sp in the VM for the case that yield from is
interrupted by a StopIteration raised via NLR.
2. Introduces a new test to check this handling of sp and re-covers the
code in the VM.
Float parsing (both single and double precision) may have a relative error
of order the floating point precision, so adjust tests to take this into
account by not printing all of the digits of the answer.
These new tests cover cases that can't be reached from Python and get
coverage of py/mpz.c to 100%.
These "unreachable from Python" pieces of code could be removed but they
form an integral part of the mpz C API and may be useful for non-Python
usage of mpz.
This path for src->deg==NULL is never used because mpz_clone() is always
called with an argument that has a non-zero integer value, and hence has
some digits allocated to it (mpz_clone() is a static function private to
mpz.c all callers of this function first check if the integer value is zero
and if so take a special-case path, bypassing the call to mpz_clone()).
There is some unused and commented-out functions that may actually pass a
zero-valued mpz to mpz_clone(), so some TODOs are added to these function
in case they are needed in the future.
There is a finite list of ascending primes used for the size of a hash
table, and this test tests that the code can handle a dict larger than the
maximum value in that list of primes. Adding this tests gets py/map.c to
100% coverage.
All callers of the asm entry function guarantee that num_locals>=0, so no
need to add an explicit check for it. Use an assertion instead.
Also, the signature of asm_x86_entry is changed to match the other asm
entry functions.
This patch just moves the definition of the wrapper object fat_vfs_open_obj
to the location of the definition of its function, which matches how it's
done in most other places in the code base.
The fat_vfs_ilistdir2() function was only used by fat_vfs_ilistdir_func()
so moving the former into the same file as the latter allows it to be
placed directly into the latter function, thus saving code size.
These ports don't need anything from extmod so don't include those files
at all in the build. This speeds up the build by about 10% when building
with a single core.
If a port only needs the core files then it can now use the $(PY_CORE_O)
variable instead of $(PY_O). $(PY_EXTMOD_O) contains the list of extmod
files (including some files from lib/). $(PY_O) retains its original
definition as the list of all object file (including those for frozen code)
and is a convenience variable for ports that want everything.
Saves a few bytes of code space, and is more efficient because with
MICROPY_GC_CONSERVATIVE_CLEAR enabled by default all memory is already
cleared when allocated.
Otherwise passing -1 as maxlen will lead to a zero allocation and
subsequent unbound buffer overflow in deque.append() because i_put is
allowed to grow without bound.
So far, implements just append() and popleft() methods, required for
a normal queue. Constructor doesn't accept an arbitarry sequence to
initialize from (am empty deque is always created), so an empty tuple
must be passed as such. Only fixed-size deques are supported, so 2nd
argument (size) is required.
There's also an extension to CPython - if True is passed as 3rd argument,
append(), instead of silently overwriting the oldest item on queue
overflow, will throw IndexError. This behavior is desired in many
cases, where queues should store information reliably, instead of
silently losing some items.
Currently only the first 2 args are used, but this patch should at least
make getaddrinfo() signature-compatible with CPython and other bare-metal
ports that use the lwip bindings.
The micropython.stack_use() function is useful to query the current C stack
usage, and it's inclusion in the micropython module doesn't need to be tied
to the inclusion of mem_info()/qstr_info() because it doesn't rely on any
of the code from these functions. So this patch introduces the config
option MICROPY_PY_MICROPYTHON_STACK_USE which can be used to independently
control the inclusion of stack_use(). By default it is enabled if
MICROPY_PY_MICROPYTHON_MEM_INFO is enabled (thus not changing any of the
existing ports).
The new option is MICROPY_ENABLE_EXTERNAL_IMPORT and is enabled by default
so that the default behaviour is the same as before. With it disabled
import is only supported for built-in modules, not for external files nor
frozen modules. This allows to support targets that have no filesystem of
any kind and that only have access to pre-supplied built-in modules
implemented natively.
Prior to this patch uPy (on a 32-bit arch) would have severe issues when
calling bytes(-1): such a call would call vstr_init_len(vstr, -1) which
would then +1 on the len and call vstr_init(vstr, 0), which would then
round this up and allocate a small amount of memory for the vstr. The
bytes constructor would then attempt to zero out all this memory, thinking
it had allocated 2^32-1 bytes.
This patch changes the way REPL autocomplete finds matches. It now probes
the target object for all qstrs via mp_load_method_maybe to look for a
match with the given input string. Similar to how the builtin dir()
function works, this new algorithm now find all methods and instances of
user-defined classes including attributes of their parent classes. This
helps a lot at the REPL prompt for user-discovery and to autocomplete names
even for classes that are derived.
The downside is that this new algorithm is slower than the previous one,
and in particular will be slower the more qstrs there are in the system.
But because REPL autocomplete is primarily used in an interactive way it is
not that important to make it fast, as long as it is "fast enough" compared
to human reaction.
On a slow microcontroller (CPU running at 16MHz) the autocomplete time for
a list of 35 names in the outer namespace (pressing tab at a bare prompt)
takes about 160ms with this algorithm, compared to about 40ms for the
previous implementation (this time includes the actual printing of the
names as well). This time of 160ms is very reasonable especially given the
new functionality of listing all the names.
This patch also decreases code size by:
bare-arm: +0
minimal x86: -128
unix x64: -128
unix nanbox: -224
stm32: -88
cc3200: -80
esp8266: -92
esp32: -84
This patch improves the builtin dir() function by probing the target object
with all possible qstrs via mp_load_method_maybe. This is very simple (in
terms of implementation), doesn't require recursion, and allows to list all
methods of user-defined classes (without duplicates) even if they have
multiple inheritance with a common parent. The downside is that it can be
slow because it has to iterate through all the qstrs in the system, but
the "dir()" function is anyway mostly used for testing frameworks and user
introspection of types, so speed is not considered a priority.
In addition to providing a more complete implementation of dir(), this
patch is simpler than the previous implementation and saves some code
space:
bare-arm: -80
minimal x86: -80
unix x64: -56
unix nanbox: -48
stm32: -80
cc3200: -80
esp8266: -104
esp32: -64
This macro is written out explicitly in the two locations that it is used
and then the code is optimised, opening possibilities for further
optimisations and reducing code size:
unix: -48
minimal CROSS=1: -32
stm32: -32
Using the message "maximum recursion depth exceeded" for when the pystack
runs out of memory can be misleading because the pystack can run out for
reasons other than deep recursion (although in most cases pystack
exhaustion is probably indirectly related to deep recursion). And it's
important to give the user more precise feedback as to the reason for the
error: if they know precisely that the pystack was exhausted then they have
a chance to increase the amount of memory available to the pystack (as
opposed to not knowing if it was the C stack or pystack that ran out).
Also, C stack exhaustion is more serious than pystack exhaustion because it
could have been that the C stack overflowed and overwrote/corrupted some
data and so the system must be restarted. The pystack can never corrupt
data in this way so pystack exhaustion does not require a system restart.
Knowing the difference between these two cases is therefore important.
The actual exception type for pystack exhaustion remains as RuntimeError so
that programatically it behaves the same as a C stack exhaustion.
By adding __builtin_unreachable() at the end of nlr_push, we're
essentially telling the compiler that this function will never return.
When GCC LTO is in use, this means that any time nlr_push() is called
(which is often), the compiler thinks this function will never return
and thus eliminates all code following the call.
Note: I've added a 'return 0' for older GCC versions like 4.6 which
complain about not returning anything (which doesn't make sense in a
naked function). Newer GCC versions (tested 4.8, 5.4 and some others)
don't complain about this.
This constant exception instance was once used by m_malloc_fail() to raise
a MemoryError without allocating memory, but it was made obsolete long ago
by 3556e45711. The functionality is now
replaced by the use of mp_emergency_exception_obj which lives in the global
uPy state, and which can handle any exception type, not just MemoryError.
This feature is not often used so is guarded by the config option
MICROPY_PY_BUILTINS_RANGE_BINOP which is disabled by default. With this
option disabled MicroPython will always return false when comparing two
range objects for equality (unless they are exactly the same object
instance). This does not match CPython so if (in)equality between range
objects is needed then this option should be enabled.
Enabling this option costs between 100 and 200 bytes of code space
depending on the machine architecture.
This patch provides inline versions of the utf8 helper functions for the
case when unicode is disabled (MICROPY_PY_BUILTINS_STR_UNICODE set to 0).
This saves code size.
The unichar_charlen function is also renamed to utf8_charlen to match the
other utf8 helper functions, and the signature of this function is adjusted
for consistency (const char* -> const byte*, mp_uint_t -> size_t).
Instead of putting just 'CRASH' in the .py.out file, this patch makes it so
any output from uPy that led to the crash is stored in the .py.out file, as
well as the 'CRASH' message at the end.
Prior to this patch, storage.c was a combination of code that handled
either internal flash or external SPI flash and exposed one of them as a
block device for the local storage. It was also exposed to the USB MSC.
This patch splits out the flash and SPI code to separate files, which each
provide a general block-device interface (at the C level). Then storage.c
just picks one of them to use as the local storage medium. The aim of this
factoring is to allow to add new block devices in the future and allow for
easier configurability.
This patch allows to completely compile-out support for USB, and no-USB is
now the default. If a board wants to enable USB it should define:
#define MICROPY_HW_ENABLE_USB (1)
And then one or more of the following to select the USB PHY:
#define MICROPY_HW_USB_FS (1)
#define MICROPY_HW_USB_HS (1)
#define MICROPY_HW_USB_HS_IN_FS (1)
Newer versions of the HAL use names which are cleaner and more
self-consistent amongst the HAL itself. This patch switches to use those
names in most places so it is easier to update the HAL in the future.
Prior to this patch the USBD driver did not handle the recipient correctly
for setup requests. It was not interpreting the req->wIndex field in the
right way: in some cases this field indicates the endpoint number but the
code was assuming it always indicated the interface number.
This patch fixes this. The only noticeable change is to the MSC
interface, which should now correctly respond to the USB_REQ_CLEAR_FEATURE
request and hence unmount properly from the host when requested.
mpconfigboard_common.h now sets the defaults so there is no longer a need
to explicitly list all configuration options in a board's mpconfigboard.h
file.
This file mirrors py/mpconfig.h but for board-level config options. It
provides a default configuration, to be overridden by a specific
mpconfigboard.h file, as well as setting up certain macros to automatically
configure a board.
Prior to this patch, a float literal that was close to subnormal would
have a loss of precision when parsed. The worst case was something like
float('10000000000000000000e-326') which returned 0.0.
This patch simplifies how sentinel values are stored on the stack when
doing an unwind return or jump. Instead of storing two values on the stack
for an unwind jump it now stores only one: a negative small integer means
unwind-return and a non-negative small integer means unwind-jump with the
value being the number of exceptions to unwind. The savings in code size
are:
bare-arm: -56
minimal x86: -68
unix x64: -80
unix nanbox: -4
stm32: -56
cc3200: -64
esp8266: -76
esp32: -156
The array should be of type unsigned byte because that is the type of the
values being stored. And changing to uint8_t helps to prevent warnings
from some static analysers.
Note that the check for elem!=NULL is removed for the
MP_MAP_LOOKUP_ADD_IF_NOT_FOUND case because mp_map_lookup will always
return non-NULL for such a case.
The calls to rtc_init_start(), sdcard_init() and storage_init() are all
guarded by a check for first_soft_reset, so it's simpler to just put them
all before the soft-reset loop, without the check.
The call to machine_init() can also go before the soft-reset loop because
it is only needed to check the reset cause which can happen once at the
first boot. To allow this to work, the reset cause must be set to SOFT
upon a soft-reset, which is the role of the new function machine_deinit().
Upon boot the RTC early-init function should detect if LSE or LSI is
already selected/running and, if so, use it. When the LSI has previously
(in the previous reset cycle) been selected as the clock source the only
way to reliably tell is if the RTCSEL bits of the RCC_BDCR are set to the
correct LSI value. In particular the RCC_CSR bits for LSI control do not
indicate if the LSI is ready even if it is selected.
This patch removes the check on the RCC_CSR bits for the LSI being on and
ready and only uses the check on the RCC_BDCR to see if the LSI should be
used straightaway. This was tested on a PYBLITEv1.0 and with the patch the
LSI persists correctly as the RTC source as long as the backup domain
remains powered.
Previously, if LSE is selected but fails and the RTC falls back to LSI,
then the rtc_info flags would incorrectly state that LSE is used. This
patch fixes that by setting the bit in rtc_info only after the clock is
ready.
There is an underlying hardware SPI driver (built on top of the STM HAL)
and then on top of this sits the legacy pyb.SPI class as well as the
machine.SPI class. This patch improves the separation between these
layers, in particular decoupling machine.SPI from pyb.SPI.
This patch combines the compiler optimisation code for double and triple
tuple-to-tuple assignment, taking it from two separate if-blocks to one
combined if-block. This can be done because the code for both of these
optimisations has a lot in common. Combining them together reduces code
size for ports that have the triple-tuple optimisation enabled (and doesn't
change code size for ports that have it disabled).
The SPI sub-system is independent from the uPy state (eg the heap) and so
can safely persist across a soft reset. And this is actually necessary for
drivers that rely on SPI and that also need to persist across soft reset
(eg external SPI flash memory).
This patch adds support in the USBD configuration and CDC-MSC-HID class for
high-speed USB mode. To enable it the board configuration must define
USE_USB_HS, and either not define USE_USB_HS_IN_FS, or be an STM32F723 or
STM32F733 MCU which have a built-in HS PHY. High-speed mode is then
selected dynamically by passing "high_speed=True" to the pyb.usb_mode()
function, otherwise it defaults to full-speed mode.
This patch has been tested on an STM32F733.
By defining MICROPY_HW_USB_MAIN_DEV a given board can select to use either
USB_PHY_FS_ID or USB_PHY_HS_ID as the main USBD peripheral, on which the
REPL will appear. If not defined this will be automatically configured.
There's no need to have these as separate functions, they just take up
unnecessary code space and combining them allows to factor common code, and
also allows to support arbitrary string descriptor indices.
The routine waits for the DMA to finish, which is signalled from a DMA IRQ
handler. Using WFI makes the CPU sleep while waiting for the IRQ to arrive
which decreases power consumption. To make it work correctly the check for
the change in state must be atomic and so IRQs must be disabled during the
check. The key feature of the Cortex MCU that makes this possible is that
WFI will exit when an IRQ arrives even if IRQs are disabled.
CPython doesn't allow SEEK_CUR with non-zero offset for files in text mode,
and uPy inherited this behaviour for both text and binary files. It makes
sense to provide full support for SEEK_CUR of binary-mode files in uPy, and
to do this in a minimal way means also allowing to use SEEK_CUR with
non-zero offsets on text-mode files. That seems to be a fair compromise.
Build and test 32bit and 64bit versions of the windows port using gcc
from mingw-w64. Note a bunch of tests which rely on floating point
math/printing have been disabled for now since they fail.
This commit fixes two things:
1. Do not allocate on the heap in readblocks() - unless the block size
is bigger than 512 bytes.
2. Raise an error instead of returning 1 to indicate an error: the FAT
block device layer does not check the return value. And other
backends (e.g. esp32 blockdev) also raise an error instead of
returning non-zero.
The number of registers used should be 10, not 12, to match the assembly
code in nlrx64.c. With this change the 64bit mingw builds don't need to
use the setjmp implementation, and this fixes miscellaneous crashes and
assertion failures as reported in #1751 for instance.
To avoid mistakes in the future where something gcc-related for Windows
only gets fixed for one particular compiler/environment combination,
make use of a MICROPY_NLR_OS_WINDOWS macro.
To make sure everything nlr-related is now ok when built with gcc this
has been verified with:
- unix port built with gcc on Cygwin (i686-pc-cygwin-gcc and
x86_64-pc-cygwin-gcc, version 6.4.0)
- windows port built with mingw-w64's gcc from Cygwin
(i686-w64-mingw32-gcc and x86_64-w64-mingw32-gcc, version 6.4.0)
and MSYS2 (like the ones on Cygwin but version 7.2.0)
Add some features which are already enabled in the unix port and
default to using the Python stack for scoped allocations: this can be
more performant in cases the heap is heavily used because for example
the memory needed for storing *args and **kwargs doesn't require
scanning the heap to find a free block.
For MSVC off_t is defined in sys/types.h but according to the comment
earlier in mpconfigport.h this cannot be included directly.
So just make off_t the same as mp_off_t.
This fixes the build for MSVC with MICROPY_STREAMS_POSIX_API
enabled because stream.h uses off_t.
There are two checks that are always false so can be converted to (negated)
assertions to save code space and execution time. They are:
1. The check of the str parameter, which is required to be non-NULL as per
the original comment that it has enough space in it as calculated by
mp_int_format_size. And for all uses of this function str is indeed
non-NULL.
2. The check of the base parameter, which is already required to be between
2 and 16 (inclusive) via the assertion in mp_int_format_size.
The motivation behind this patch is to remove unreachable code in mpn_div.
This unreachable code was added some time ago in
9a21d2e070, when a loop in mpn_div was copied
and adjusted to work when mpz_dig_t was exactly half of the size of
mpz_dbl_dig_t (a common case). The loop was copied correctly but it wasn't
noticed at the time that the final part of the calculation of num-quo*den
could be optimised, and hence unreachable code was left for a case that
never occurred.
The observation for the optimisation is that the initial value of quo in
mpn_div is either exact or too large (never too small), and therefore the
subtraction of quo*den from num may subtract exactly enough or too much
(but never too little). Using this observation the part of the algorithm
that handles the borrow value can be simplified, and most importantly this
eliminates the unreachable code.
The new code has been tested with DIG_SIZE=3 and DIG_SIZE=4 by dividing all
possible combinations of non-negative integers with between 0 and 3
(inclusive) mpz digits.
Empty __VA_ARGS__ are not allowed in the C preprocessor so adjust the rule
arg offset calculation to not use them. Also, some compilers (eg MSVC)
require an extra layer of macro expansion.
This is the sixth and final patch in a series of patches to the parser that
aims to reduce code size by compressing the data corresponding to the rules
of the grammar.
Prior to this set of patches the rules were stored as rule_t structs with
rule_id, act and arg members. And then there was a big table of pointers
which allowed to lookup the address of a rule_t struct given the id of that
rule.
The changes that have been made are:
- Breaking up of the rule_t struct into individual components, with each
component in a separate array.
- Removal of the rule_id part of the struct because it's not needed.
- Put all the rule arg data in a big array.
- Change the table of pointers to rules to a table of offsets within the
array of rule arg data.
The last point is what is done in this patch here and brings about the
biggest decreases in code size, because an array of pointers is now an
array of bytes.
Code size changes for the six patches combined is:
bare-arm: -644
minimal x86: -1856
unix x64: -5408
unix nanbox: -2080
stm32: -720
esp8266: -812
cc3200: -712
For the change in parser performance: it was measured on pyboard that these
six patches combined gave an increase in script parse time of about 0.4%.
This is due to the slightly more complicated way of looking up the data for
a rule (since the 9th bit of the offset into the rule arg data table is
calculated with an if statement). This is an acceptable increase in parse
time considering that parsing is only done once per script (if compiled on
the target).
Instead of each rule being stored in ROM as a struct with rule_id, act and
arg, the act and arg parts are now in separate arrays and the rule_id part
is removed because it's not needed. This reduces code size, by roughly one
byte per grammar rule, around 150 bytes.
The rule name is only used for debugging, and this patch makes things a bit
cleaner by completely separating out the rule name from the rest of the
rule data.
Each NLR implementation (Thumb, x86, x64, xtensa, setjmp) duplicates a lot
of the NLR code, specifically that dealing with pushing and popping the NLR
pointer to maintain the linked-list of NLR buffers. This patch factors all
of that code out of the specific implementations into generic functions in
nlr.c, along with a helper macro in nlr.h. This eliminates duplicated
code.
If MICROPY_NLR_SETJMP is not enabled and the machine is auto-detected then
nlr.h now defines some convenience macros for the individual NLR
implementations to use (eg MICROPY_NLR_THUMB). This keeps nlr.h and the
implementation in sync, and also makes the nlr_buf_t struct easier to read.
A function with a naked attribute must only contain basic inline asm
statements and no C code.
For nlr_push this means removing the "return 0" statement. But for some
gcc versions this induces a compiler warning so the __builtin_unreachable()
line needs to be added.
For nlr_jump, this function contains a combination of C code and inline asm
so cannot be naked.
This reverts commit 6a3a742a6c.
The above commit has number of faults starting from the motivation down
to the actual implementation.
1. Faulty implementation.
The original code contained functions like:
NORETURN void nlr_jump(void *val) {
nlr_buf_t **top_ptr = &MP_STATE_THREAD(nlr_top);
nlr_buf_t *top = *top_ptr;
...
__asm volatile (
"mov %0, %%edx \n" // %edx points to nlr_buf
"mov 28(%%edx), %%esi \n" // load saved %esi
"mov 24(%%edx), %%edi \n" // load saved %edi
"mov 20(%%edx), %%ebx \n" // load saved %ebx
"mov 16(%%edx), %%esp \n" // load saved %esp
"mov 12(%%edx), %%ebp \n" // load saved %ebp
"mov 8(%%edx), %%eax \n" // load saved %eip
"mov %%eax, (%%esp) \n" // store saved %eip to stack
"xor %%eax, %%eax \n" // clear return register
"inc %%al \n" // increase to make 1, non-local return
"ret \n" // return
: // output operands
: "r"(top) // input operands
: // clobbered registers
);
}
Which clearly stated that C-level variable should be a parameter of the
assembly, whcih then moved it into correct register.
Whereas now it's:
NORETURN void nlr_jump_tail(nlr_buf_t *top) {
(void)top;
__asm volatile (
"mov 28(%edx), %esi \n" // load saved %esi
"mov 24(%edx), %edi \n" // load saved %edi
"mov 20(%edx), %ebx \n" // load saved %ebx
"mov 16(%edx), %esp \n" // load saved %esp
"mov 12(%edx), %ebp \n" // load saved %ebp
"mov 8(%edx), %eax \n" // load saved %eip
"mov %eax, (%esp) \n" // store saved %eip to stack
"xor %eax, %eax \n" // clear return register
"inc %al \n" // increase to make 1, non-local return
"ret \n" // return
);
for (;;); // needed to silence compiler warning
}
Which just tries to perform operations on a completely random register (edx
in this case). The outcome is the expected: saving the pure random luck of
the compiler putting the right value in the random register above, there's
a crash.
2. Non-critical assessment.
The original commit message says "There is a small overhead introduced
(typically 1 machine instruction)". That machine instruction is a call
if a compiler doesn't perform tail optimization (happens regularly), and
it's 1 instruction only with the broken code shown above, fixing it
requires adding more. With inefficiencies already presented in the NLR
code, the overhead becomes "considerable" (several times more than 1%),
not "small".
The commit message also says "This eliminates duplicated code.". An
obvious way to eliminate duplication would be to factor out common code
to macros, not introduce overhead and breakage like above.
3. Faulty motivation.
All this started with a report of warnings/errors happening for a niche
compiler. It could have been solved in one the direct ways: a) fixing it
just for affected compiler(s); b) rewriting it in proper assembly (like
it was before BTW); c) by not doing anything at all, MICROPY_NLR_SETJMP
exists exactly to address minor-impact cases like thar (where a) or b) are
not applicable). Instead, a backwards "solution" was put forward, leading
to all the issues above.
The best action thus appears to be revert and rework, not trying to work
around what went haywire in the first place.
These were copied from the stm32 port (then stmhal) at the very beginning
of this port, with the anticipation that the esp8266 port would have board
definition files with a list of valid pins and their names. But that has
not been implemented and likely won't be, so remove the corresponding lines
from the Makefile.
This patch adds in internal config value MICROPY_HW_ENABLE_HW_I2C that is
automatically configured, and enabled only if one or more hardware I2C
ports are defined in the mpconfigboard.h file. If none are defined then
the pyb.I2C class is excluded from the build, along with all supporting
code. The machine.I2C class will still be available for software I2C.
Disabling all hardware I2C on an F4 board saves around 10,000 bytes of code
and 200 bytes of RAM.
Each NLR implementation (Thumb, x86, x64, xtensa, setjmp) duplicates a lot
of the NLR code, specifically that dealing with pushing and popping the NLR
pointer to maintain the linked-list of NLR buffers. This patch factors all
of that code out of the specific implementations into generic functions in
nlr.c. This eliminates duplicated code.
The factoring also allows to make the machine-specific NLR code pure
assembler code, thus allowing nlrthumb.c to use naked function attributes
in the correct way (naked functions can only have basic inline assembler
code in them).
There is a small overhead introduced (typically 1 machine instruction)
because now the generic nlr_jump() must call nlr_jump_tail() rather than
them being one combined function.
set_equal is called only from set_binary_op, and this guarantees that the
second arg to set_equal is always a set or frozenset. So there is no need
to do a further check.
Previously, testing of stackless build happened (manually) in
travis-stackless branch. However, stackless offers important
featureset, so it's worth to test it as a part of the main
CI. Strict stackless is used because it's the "real" stackless
build, which avoids using C stack as much as possible (non-strict
just prefers heap over C stack, but may end up using the latter).
This implements .pend_throw(exc) method, which sets up an exception to be
triggered on the next call to generator's .__next__() or .send() method.
This is unlike .throw(), which immediately starts to execute the generator
to process the exception. This effectively adds Future-like capabilities
to generator protocol (exception will be raised in the future).
The need for such a method arised to implement uasyncio wait_for() function
efficiently (its behavior is clearly "Future" like, and normally would
require to introduce an expensive Future wrapper around all native
couroutines, like upstream asyncio does).
py/objgenerator: pend_throw: Return previous pended value.
This effectively allows to store an additional value (not necessary an
exception) in a coroutine while it's not being executed. uasyncio has
exactly this usecase: to mark a coro waiting in I/O queue (and thus
not executed in the normal scheduling queue), for the purpose of
implementing wait_for() function (cancellation of such waiting coro
by a timeout).
tinytest is written with the idea that tests won't write to stdout, so it
prints test name witjout newline, then executes test, then writes status.
But MicroPython tests write to stdout, so the test output becomes a mess.
So, instead print it like:
# starting basics/andor.py
... test output ...
basics/andor.py: OK
If TEST is defined, file it refers to will be used as the testsuite
source (should be generated with tools/tinytest-codegen.py).
"make-bin-testsuite" script is introduce to build such a binary.
The whole idea of --list-tests is that we prepare a list of tests to run
later, and currently don't have a connection to target board. Similarly
for --write-exp - only "python3" binary would be required for this operation,
not "micropython".
Some compilers can treat enum types as signed, in which case 3 bits is not
enough to encode all mp_raw_code_kind_t values. So change the type to
mp_uint_t.
Instead of passing thru more and more options from tinytest-codegen to
run-tests --list-tests, pipe output of run-tests --list-tests into
tinytest-codegen.
The idea that --list-tests would be enough to produce list of tests for
tinytest-codegen didn't work, because normal run-tests processing heavily
relies on dynamic target capabilities discovery, and test filtering happens
as the result of that.
So, approach the issue from different end - allow to specify arbitrary
filtering criteria as run-tests arguments. This way, specific filters
will be still hardcoded, but at least on a particular target's side,
instead of constant patching tinytest-codegen and/or run-tests.
Because otherwise the function can return with data still waiting to be
clocked out, and CS might then be disabled before the SPI transaction is
complete. Fixes issue #3487.
Gets passed to run-tests --list-tests to get actual list of tests to use.
If --target= is not given, legacy set hardcoded in tinytest-codegen itself
is used.
Also, get rid of tinytest test groups - they aren't really used for
anything, and only complicate processing. Besides, one of the next
step is to limit number of tests per a generated file to control
the binary size, which also will require "flat" list of tests.
Lists tests to be executed, subject to all other filters requested. This
options would be useful e.g. for scripts like tools/tinytest-codegen.py,
which currently contains hardcoded filters for particular a particular
target and can't work for multiple targets.
The way tinytest was used in qemu-arm test target is that it didn't test
much. MicroPython tests are based on matching the test output against
reference output, but qemu-arm's implementation didn't do that, it
effectively tested just that there was no exception during test
execution. "upytesthelper" wrapper was introduce to fix it, so switch
test implementation to use it.
This requires passing different CFLAGS when building the firmware, so
split out test-related parts to Makefile.test.
The way tinytest was used in qemu-arm test target is that it didn't test
much. MicroPython tests are based on matching the test output against
reference output, but qemu-arm's implementation didn't do that, it
effectively tested just that there was no exception during test
execution. "upytesthelper" wrapper was introduce to fix it, and so
test generator is now switched to generate test code for it.
Also, fix PEP8 and other codestyle issues.
Tinytest is classical assert-style framework, but MicroPython tests work
in different way - they produce content, and that content should be matched
against expected one to see if test passes. upytesthelper exactly adds
helper functions to make that possible.
Code lineage:
osdebug() is based loosely on the version in esp8266, but there didn't
seem to be an obvious way of choosing a particular UART. The basic
behavior is the same, though: provide None, and logging is disabled;
provide an integer and logging is restored to the default level.
To build on that, and because the IDF provides more functionality, a
second parameter has now been implemented which allows the active log
level to be set:
esp.osdebug(uart[, level])
The module has a corresponding set of LOG_ values to set this accordingly.
When configuring a static set of values with ifconfig() the DNS was not
being set. This patch fixes that, and additionally uses the tcpip_adapter
API to ensure it is thread safe.
Further discussion is here:
https://github.com/micropython/micropython-esp32/issues/210/
This commit is a combination of 216 commits from the initial stages of
development of this port, up to and including the point where the code was
moved to the ports/esp32 directory. These commits were mostly concerned
with setting up the build system and getting a reliable port working with
basic features. The following is a digest of the original commits in their
original order (most recent listed first), grouped where possible per
author. The list is here to give credit for the work and provide some
level of traceability and accountability. For the full history of
development please consult the original repository.
All code is MIT licensed and the relevant copyright holders are listed in
the comment-header of each file.
Damien George <damien.p.george@gmail.com>
esp32: Update to latest ESP IDF.
esp32: Update module symlinks now that code lives under the ports dir.
esp32: Update to compile with new ports/esp32 directory structure.
esp32: Move it to the ports/ directory.
esp32/machine_uart: Don't save baudrate but compute it instead.
esp32/modsocket: Add socket.readinto() method.
esp32/modesp: Add esp.gpio_matrix_in and esp.gpio_matrix_out functions.
esp32/machine_uart: Wait for all data to be tx'd before changing config.
NyxCode <moritz.bischof1@gmail.com>
esp32: Add note to README.md about updating the submodules of ESP IDF.
Anthony Briggs <anthony.briggs@gmail.com>
esp32: Update README.md installation and flashing instructions.
Javier Candeira <javier@candeira.com>
esp32: Raise error when setting input-only pin to output.
With help from Adrian Smith (fon@thefon.net)
Javier Candeira <javier@candeira.com>
esp32: Replace exception raising with corresponding mp_raise_XXX funcs.
Tisham Dhar <whatnickd@gmail.com>
esp32: Add some specific notes about building on Windows using WSL.
Ben Gamari <ben@smart-cactus.org>
esp32: Provide machine.Signal class.
Damien George <damien.p.george@gmail.com>
esp32/modnetwork: Implement AP version of network.isconnected().
Eric Poulsen <eric@zyxod.com>
esp32/README.md: Add note about btree submodule initialization.
Damien George <damien.p.george@gmail.com>
esp32: Make firmware.bin start at 0x1000 to allow flash size autodetect.
esp32: Changes to follow latest version of upstream uPy.
esp32/Makefile: Separate ESP-specific inc dirs to avoid header clashes.
esp32: Enable "btree" database module.
esp32: Update to latest ESP IDF.
Roosted7 <thomasroos@live.nl>
esp32: Update to latest ESP-IDF.
Alex King <alex_w_king@yahoo.com>
esp32/machine_timer: Add support for esp32 hardware timer.
Code lineage:
Timer() is based loosely on the version in esp8266, although the
implementation is differs significantly because of the change in
the underlying platform.
Damien George <damien.p.george@gmail.com>
esp32/machine_uart: Increase UART TX buffer size to 64.
esp32/modules: Update dht symlink.
esp32/mpconfigport.h: Enable utimeq module, needed for uasyncio.
esp32: Changes to follow latest version of upstream uPy.
esp32: Update to latest ESP-IDF.
esp32/machine_uart: Add uart.any() method.
esp32/machine_uart: Uninstall the UART driver before installing it.
Thomas Roos <mail@thomasroos.nl>
esp32: Update to latest ESP-IDF.
Eric Poulsen <eric@zyxod.com>
esp32/modsocket: Make read/write return None when in non-blocking mode.
esp32/modsocket.c: Fix send/sendto/write for non-blocking sockets.
Odd Stråbø <oddstr13@openshell.no>
esp32: Initial working implementation of machine.UART.
Code lineage (as seen by previous commits): I copied the ESP8266 code,
renamed pyb -> machine, and used esp-idf as a reference while implementing
minimal functionality. I provide all of my changes under the MIT license.
Odd Stråbø <oddstr13@openshell.no>
esp32/machine_uart: Rename pyb to machine.
esp32: Copy machine_uart.c from esp8266 port.
Damien George <damien.p.george@gmail.com>
esp32/moduos: Add uos.ilistdir() function.
esp32: Mount filesystem at the root of the VFS, following esp8266.
Andy Valencia <vandyswa@gmail.com>
esp32: Add hardware SHA1/SHA256 support via mbedtls API.
Code lineage: a copy of extmod/moduhashlib with the API invocation details
edited. Total derivative work.
Andy Valencia <vandyswa@gmail.com>
esp32: Add PWM support via machine.PWM class.
Code lineage:
I started by copying the esp8266 machine_pwm.c. I used information from the
ESP32 Technical Reference Manual, the esp-idf documentation, and the SDK's
sample ledc example code (but I did not copy that code, just studied it to
understand the SDK's API for PWM). So aside from the code copied from the
esp8266 PWM support, everything else you see is just new code I wrote.
I wasn't an employee of anybody when I wrote it, and I wrote it with the
understanding and intention that it's simply a derivative work of the
existing micropython code. I freely and willingly contribute it to the
project and intend that it not change the legal status of the micropython
code base in any way, even if it is included in that base in whole or part.
Damien George <damien.p.george@gmail.com>
esp32/modules: Add symlinks for upysh and upip.
Eric Poulsen <eric@zyxod.com>
esp32/modmachine: Add unique_id() function to machine module.
Damien George <damien.p.george@gmail.com>
esp32: Change dac_out_voltage to dac_output_voltage for new IDF API.
esp32: Update esp32.custom_common.ld to align with changes in ESP IDF.
Eric Poulsen <eric@zyxod.com>
esp32: Update to latest ESP IDF.
Damien George <damien.p.george@gmail.com>
esp32/modsocket: When resolving IP addr handle the case of host=''.
esp32: Update to latest ESP IDF.
Eric Poulsen <eric@zyxod.com>
esp32/Makefile: Change default FLASH_MODE to dio for WROOM-32 module.
esp32: Move FAT FS to start at 0x200000 and increase size to 2MiB.
Damien George <damien.p.george@gmail.com>
esp32: Remove enable_irq/disable_irq and use ATOMIC_SECTION instead.
esp32/mpconfigport.h: Provide ATOMIC_SECTION macros.
esp32/main: Restart the MCU if there is a failed NLR jump.
Daniel Campora <daniel@pycom.io>
esp32: Enable threading; be sure to exit GIL when a thread will block.
esp32: Trace the registers when doing a gc collect. Also make it thread ready.
esp32: Add threading implementation, disabled for the time being.
Damien George <damien.p.george@gmail.com>
esp32: Update to latest ESP IDF.
esp32/uart: Use high-level function to install UART0 RX ISR handler.
esp32/Makefile: Make FreeRTOS private include dir really private.
Eric Poulsen <eric@zyxod.com>
esp32: Add support for hardware SPI peripheral (block 1 and 2).
Sergio Conde Gómez <skgsergio@gmail.com>
esp32/modules/inisetup.py: Mount filesystem at /flash like ESP8266
Damien George <damien.p.george@gmail.com>
esp32: Convert to use core-provided KeyboardInterrupt exception.
esp32: Pump the event loop while waiting for rx-chr or delay_ms.
esp32: Implement Pin.irq() using "soft" scheduled interrupts.
esp32: Update to latest ESP IDF version.
Eric Poulsen <eric@zyxod.com>
esp32/README: Add troubleshooting section to the end.
tyggerjai <tyggerjai@gmail.com>
esp32: Add support for WS2812 and APA106 RGB LEDs.
Damien George <damien.p.george@gmail.com>
esp32: Add makeimg.py script to build full firmware; use it in Makefile.
esp32/modsocket: Make socket.read return when socket closes.
esp32/modsocket: Initialise the timeout on an accepted socket.
esp32/mphalport: Provide proper implementations of disable_/enable_irq.
esp32/modmachine: Add disable_irq/enable_irq functions.
Nick Moore <nick@zoic.org>
esp32/modsocket.c: add comment explaining timeout behaviour
esp32/modsocket.c: clean up send methods for retries too
esp32/modsocket.c: sockets always nonblocking, accept timeout in modsocket
esp32: Update to latest ESP IDF version.
esp32/modsocket.c: remove MSG_PEEK workaround on select ioctl.
esp32/modsocket.c: Initialize tcp when modsocket loads.
Damien George <damien.p.george@gmail.com>
esp32/main: Bump heap size from 64k to 96k.
esp32/modutime: Add time.time() function.
esp32/modsocket: Convert lwip errnos to uPy ones.
esp32/modules: Provide symlink to ds18x20 module.
esp32: Add support for onewire protocol via OneWire module.
esp32: Add support for DHT11 and DHT22 sensors.
esp32/mphalport: Improve delay and ticks functions.
esp32: Update to latest ESP IDF.
esp32/modules: Provide symlink to urequests from micropython-lib.
esp32: Populate sys.path.
Nick Moore <nick@zoic.org>
esp32/machine_dac.c: implement DAC pins as well
esp32/machine_adc.c: also machine.ADC
esp32/machine_touchpad.c: add support for touchpad
Damien George <damien.p.george@gmail.com>
esp32/README: Add hint about using GNUmakefile on case-insensitive FS.
esp32/mpconfigport.h: Enable maximum speed software SPI.
esp32: Provide improved version of mp_hal_delay_us_fast.
esp32/mpconfigport.h: Enable MICROPY_PY_BUILTINS_POW3 option.
esp32: Update to latest ESP IDF.
esp32: Convert to use new oofatfs library and generic VFS sub-system.
esp32: Enable help('modules') to list builtin modules.
esp32: Convert to use new builtin help function.
Aaron Kelly <AaronKelly@email.com>
esp32/README: Add comment about ESP-IDF version
Damien George <damien.p.george@gmail.com>
esp32: Consistently use size_t instead of mp_uint_t.
esp32: Change "Micro Python" to "MicroPython" in license comments.
esp32/Makefile: Use -C argument to git instead of cd'ing.
esp32/help: Add section to help about using the network module.
esp32/README: Add section about configuring and using an ESP32 board.
esp32/README: Remove paragraph about buggy toolchain, it's now fixed.
esp32/modnetwork: Change network init logging from info to debug.
esp32/modnetwork: Don't start AP automatically when init'ing wifi.
esp32/modsocket: Implement socket.setsockopt, to support SO_REUSEADDR.
esp32: Update to latest ESP IDF.
esp32/sdkconfig.h: Remove unused CONFIG_ESPTOOLPY_xxx config settings.
esp32/modsocket: Add support for DGRAM and RAW, and sendto/recvfrom.
esp32/modsocket: Fix return value of "port" in socket.accept.
esp32/modsocket: Make socket.recv take exactly 2 args.
esp32: Enable ussl module, using mbedtls component from ESP IDF.
esp32/modsocket: Rename "socket" module to "usocket".
esp32/sdkconfig: Increase max number of open sockets from 4 to 8.
esp32/modsocket: Add error checking for creating and closing sockets.
esp32/modsocket: Use _r (re-entrant) versions of LWIP socket API funcs.
esp32/modsocket: Raise an exception if socket.connect did not succeed.
esp32/modsocket: Make socket.accept return a tuple: (client, addr).
esp32/modsocket: Use m_new_obj_with_finaliser instead of calloc.
esp32/Makefile: Add check for IDF version, and warn if not supported.
esp32/esp32.custom_common.ld: Update to follow changes in IDF.
esp32: Update to latest ESP IDF.
nubcore <x@nubcore.com>
esp32: add #define CONFIG_ESP32_WIFI_RX_BUFFER_NUM 25
Nick Moore <nick@zoic.org>
esp32/modsocket.c: add in sendall and makefile methods #10
esp32/modsocket.c: fixups for #10
esp32/modsocket.c: fix copyright, socket_recv gets param and exception
esp32/modnetwork.c: fix copyright, network.active param to bool
Damien George <damien.p.george@gmail.com>
esp32/modnetwork: Implement wlan.isconnected() method.
esp32/modnetwork: Add initial implementation of wlan.config().
esp32/modnetwork: Simplify event_handler messages.
Nick Moore <nick@zoic.org>
esp32/modsocket.c: support for ioctl, settimeout, setblocking, getaddrinfo
Damien George <damien.p.george@gmail.com>
esp32/README: Add comment about FLASH_MODE being dio.
esp32: Update to latest ESP IDF.
esp32/modnetwork: Remove unnecessary indirection variable for scan list.
esp32/modnetwork: Check that STA is active before trying to scan.
esp32/mphalport: Replace portTICK_RATE_MS with portTICK_PERIOD_MS.
esp32/README: Add comment about using $(HOME) in makefile.
esp32/modnetwork: Use memset instead of bzero, the latter is deprecated.
esp32/modnetwork: Improve error handling when STA is connecting to AP.
esp32/Makefile: Use tab instead of spaces, and use shorter variable.
Nick Moore <nick@zoic.org>
esp32/modsocket.c: AF_*, SOCK_* and IPPROTO_* constants
esp32/modsocket.c: socket.settimeout implementation
Damien George <damien.p.george@gmail.com>
esp32/Makefile: Update to latest ESP IDF.
Nick Moore <nick@zoic.org>
esp32/modsocket.c: use mp streams for sockets
esp32: network.WLAN.ifconfig based on esp8266 version
esp32: Fix up exception handling
esp32: sketchy modsocket ... revisit this once modnetwork is sorted
esp32: First cut at modnetwork, manually rebased from prev. version
Damien George <damien.p.george@gmail.com>
esp32/help: Update help text.
esp32: Add info about Microbric Pty Ltd being the sponsor of this port.
esp32: Add README.md file.
esp32/mpconfigport.h: Add weak links to many of the builtin modules.
esp32: Enable soft implementation of machine.SPI class.
esp32/Makefile: Simplify APP_LD_ARGS by using OBJ variable.
esp32/Makefile: Reorganise Makefile and add some comments.
esp32/Makefile: Clean up CFLAGS for ESP IDF components.
esp32/Makefile: Tidy up names of ESP IDF components, to match dir name.
esp32/Makefile: Define and use ESPCOMP variable.
esp32: Update to latest ESP IDF.
esp32/main: Enable filesystem support.
esp32: Use custom ld script to ensure correct code get placed in iram.
esp32: Update to latest ESP IDF.
esp32/main: Pin the uPy task to core 0.
esp32: Update to use latest ESP IDF.
esp32: Disable boot-up scripts, spi_flash_erase_sector no longer works.
esp32: Add scripts to init and mount filesystem.
esp32: Enable frozen bytecode, with scripts stored in "modules/" dir.
esp32/modesp: Increase flash_user_start position to 1Mbyte.
esp32/Makefile: Add "erase" target for convenient erasure.
esp32/sdkconfig: Reorder config settings to put common things together.
esp32/sdkconfig: Change to use single core only.
esp32: Add esp module.
esp32/uart.c: Make sure uart ISR handler is all in iram.
esp32/main.c: Use ESP_TASK_PRIO_MIN + 1 for mp_task's priority.
esp32/Makefile: Use only bare-minimum flags when compiling .S files.
esp32/Makefile: Rename "firmware" to "application".
esp32: Update ESP IDF version.
esp32/Makefile: Add declarations to build bootloader and partitions.
esp32/Makefile: When deploying, write the application last.
esp32/Makefile: Use $(INC) variable instead of listing include dirs.
esp32/Makefile: Use locally built versions of freertos and newlib libs.
esp32: Add low-level uart handler with ISR and ringbuf for stdin.
esp32: Add machine.idle() function.
esp32: Add machine.I2C class.
esp32: Enable machine.time_pulse_us.
esp32: Add initial implementation of machine.Pin class.
esp32: Prepare main.c for using xTaskCreateStatic.
esp32: Clean up mphalport.h.
esp32: Add initial uos module.
esp32: Clean up mpconfigport.h, enable more features.
esp32: Use new reset function.
esp32: Update to latest ESP IDF.
esp32: Add idf-version target to Makefile, to track IDF commit.
esp32: Initial port to ESP32.
This is a bit of a clumsy way of doing it but solves the issue of __init__
not running when a module is imported via its weak-link name. Ideally a
better solution would be found.
"Builtin" tinytest-based testsuite as employed by qemu-arm (and now
generalized by me to be reusable for other targets) performs simplified
detection of skipped tests, it treats as such tests which raised SystemExit
(instead of checking got "SKIP" output). Consequently, each "SKIP" must
be accompanied by SystemExit (and conversely, SystemExit should not be
used if test is not skipped, which so far seems to be true).
Before this patch, if a user defined the __new__() function for a class
then two instances of that class would be created: once before __new__ is
called and once during the __new__ call (assuming the user creates some
instance, eg using super().__new__, which is most of the time). The first
one was then discarded. This refactor makes it so that a new instance is
only created if the user __new__ function doesn't exist.
This patch cleans up and generalises part of the code which handles
overriding and calling a native base-class's __init__ method. It defers
the call to the native make_new() function until after the user (Python)
__init__() method has run. That user method now has the chance to call the
native __init__/make_new and pass it different arguments. If the user
doesn't call the super().__init__ method then it will be called
automatically after the user code finishes, to finalise construction of the
instance.
The nan-boxing representation has an extra 16-bits of space to store
small-int values, and making use of it allows to create and manipulate full
32-bit positive integers (ie up to 0xffffffff) without using the heap.
These tests involves testing allocation-free function calling, and in strict
stackless mode, it's not possible to make a function call with heap locked
(because function activation record aka frame is allocated on the heap).
In strict stackless mode, it's not possible to make a function call with
heap locked (because function activation record aka frame is allocated on
heap). So, if the only purpose of function is to introduce local variable
scope, move heap lock/unlock calls inside the function.
This patch introduces the MICROPY_ENABLE_PYSTACK option (disabled by
default) which enables a "Python stack" that allows to allocate and free
memory in a scoped, or Last-In-First-Out (LIFO) way, similar to alloca().
A new memory allocation API is introduced along with this Py-stack. It
includes both "local" and "nonlocal" LIFO allocation. Local allocation is
intended to be equivalent to using alloca(), whereby the same function must
free the memory. Nonlocal allocation is where another function may free
the memory, so long as it's still LIFO.
Follow-up patches will convert all uses of alloca() and VLA to the new
scoped allocation API. The old behaviour (using alloca()) will still be
available, but when MICROPY_ENABLE_PYSTACK is enabled then alloca() is no
longer required or used.
The benefits of enabling this option are (or will be once subsequent
patches are made to convert alloca()/VLA):
- Toolchains without alloca() can use this feature to obtain correct and
efficient scoped memory allocation (compared to using the heap instead
of alloca(), which is slower).
- Even if alloca() is available, enabling the Py-stack gives slightly more
efficient use of stack space when calling nested Python functions, due to
the way that compilers implement alloca().
- Enabling the Py-stack with the stackless mode allows for even more
efficient stack usage, as well as retaining high performance (because the
heap is no longer used to build and destroy stackless code states).
- With Py-stack and stackless enabled, Python-calling-Python is no longer
recursive in the C mp_execute_bytecode function.
The micropython.pystack_use() function is included to measure usage of the
Python stack.
This function was implemented as an experiment, and was enabled only in
unix port. To remind, it allows to access arbitrary files frozen as
source modules (vs bytecode).
However, further experimentation showed that the same functionality can
be implemented with frozen bytecode. The process requires more steps, but
with suitable toolset it doesn't matter patch. This process is:
1. Convert binary files into "Python resource module" with
tools/mpy_bin2res.py.
2. Freeze as the bytecode.
3. Use micropython-lib's pkg_resources.resource_stream() to access it.
In other words, the extra step is using tools/mpy_bin2res.py (because
there would be wrapper for uio.resource_stream() anyway).
Going frozen bytecode route allows more flexibility, and same/additional
efficiency:
1. Frozen source support can be disabled altogether for additional code
savings.
2. Resources could be also accessed as a buffer, not just as a stream.
There're few caveats too:
1. It wasn't actually profiled the overhead of storing a resource in
"Python resource module" vs storing it directly, but it's assumed that
overhead is small.
2. The "efficiency" claim above applies to the case when resource
file is frozen as the bytecode. If it's not, it actually will take a
lot of RAM on loading. But in this case, the resource file should not
be used (i.e. generated) in the first place, and micropython-lib's
pkg_resources.resource_stream() implementation has the appropriate
fallback to read the raw files instead. This still poses some distribution
issues, e.g. to deployable to baremetal ports (which almost certainly
would require freezeing as the bytecode), a distribution package should
include the resource module. But for non-freezing deployment, presense
of resource module will lead to memory inefficiency.
All the discussion above reminds why uio.resource_stream() was implemented
in the first place - to address some of the issues above. However, since
then, frozen bytecode approach seems to prevail, so, while there're still
some issues to address with it, this change is being made.
This change saves 488 bytes for the unix x86_64 port.
This target removes any stray files (i.e. something not committed to git)
from scripts/ and modules/ dirs (or whatever FROZEN_DIR and FROZEN_MPY_DIR
is set to).
The expected workflow is:
1. make clean-frozen
2. micropython -m upip -p modules <packages_to_freeze>
3. make
As it can be expected that people may drop random thing in those dirs which
they can miss later, the content is actually backed up before cleaning.
This is second part of fun_bc_call() vs mp_obj_fun_bc_prepare_codestate()
common code refactor. This factors out code to initialize codestate
object. After this patch, mp_obj_fun_bc_prepare_codestate() is effectively
DECODE_CODESTATE_SIZE() followed by allocation followed by
INIT_CODESTATE(), and fun_bc_call() starts with that too.
fun_bc_call() starts with almost the same code as
mp_obj_fun_bc_prepare_codestate(), the only difference is a way to
allocate the codestate object (heap vs stack with heap fallback).
Still, would be nice to avoid code duplication to make further
refactoring easier.
So, this commit factors out the common code before the allocation -
decoding and calculating codestate size. It produces two values,
so structured as a macro which writes to 2 variables passed as
arguments.
Command-line argc and argv should be passed, and as we don't have them,
placeholders were passed, but incorrectly. As we don't have them, just
pass 0/NULL. Looking at the source, this migh lead to problems under
Windows, but this test doesn't run under Windows.
Also, use "%d" printf format consistently with the rest of the codebase.
The assembler back-end for most architectures needs to know if a jump is
backwards in order to emit optimised machine code, and they do this by
checking if the destination label has been set or not. So always reset
label offsets to -1 (this reverts partially the previous commit, with some
minor optimisation for the if-logic with the pass variable).
Clearing the labels to -1 is purely a debugging measure. For release
builds there is no need to do it as the label offset table should always
have the correct value assigned.
Accessing them will crash immediately instead still working for some time,
until overwritten by some other data, leading to much less deterministic
crashes.
This is mostly a workaround for forceful rebuilding of mpy-cross on every
codebase change. If this file has debug logging enabled (by patching),
mpy-cross build failed.
Before that, the output was truncated to 32 bits. Only "%x" format is
handled, because a typical use is for addresses.
This refactor actually decreased x86_64 code size by 30 bytes.
This allows the function to raise an exception when unknown keyword args
are passed in. This patch also reduces code size by (in bytes):
bare-arm: -24
minimal x86: -76
unix x64: -56
unix nanbox: -84
stm32: -40
esp8266: -68
cc3200: -48
Furthermore, this patch adds space (" ") to the set of ROM qstrs which
means it doesn't need to be put in RAM if it's ever used.
Return the result of called function. If exception happened, return
MP_OBJ_NULL. Allows to use mp_call_function_*_protected() with callbacks
returning values, etc.
The parameter order in the example for ticks_diff was incorrect. If it's
"too early" that means that scheduled time is greater than current time and
if it's "running late" then scheduled time would be less than current time.
This commit essentially reverts aa9dbb1b03
where this if-condition was added. It seems that even when that commit
was made the code was never reached by any tests, nor reachable by
analysis (see below). The same is true with the code as it currently
stands: no test triggers this if-condition, nor any uasyncio examples.
Analysing the flow of the program also shows that it's not reachable:
==START==
-> to trigger this if condition mp_execute_bytecode() must return
MP_VM_RETURN_YIELD with *sp==MP_OBJ_STOP_ITERATION
-> mp_execute_bytecode() can only return MP_VM_RETURN_YIELD from the
MP_BC_YIELD_VALUE bytecode, which can happen in 2 ways:
-> 1) from a "yield <x>" in bytecode, but <x> must always be a proper
object, never MP_OBJ_STOP_ITERATION; ==END1==
-> 2) via yield from, via mp_resume() which must return
MP_VM_RETURN_YIELD with ret_value==MP_OBJ_STOP_ITERATION, which
can happen in 3 ways:
-> 1) it delegates to mp_obj_gen_resume(); go back to ==START==
-> 2) it returns MP_VM_RETURN_YIELD directly but with a guard that
ret_val!=MP_OBJ_STOP_ITERATION; ==END2==
-> 3) it returns MP_VM_RETURN_YIELD with ret_val set from
mp_call_method_n_kw(), but mp_call_method_n_kw() must return a
proper object, never MP_OBJ_STOP_ITERATION; ==END3==
The above shows there is no way to trigger the if-condition and it can be
removed.
Prior to this fix, enabling WebREPL for the first time via webrepl_setup
did not work at all because "boot.py" did not contain any lines with
"webrepl" in them that could be uncommented.
These checks are assumed to be true in all cases where gc_realloc is
called with a valid pointer, so no need to waste code space and time
checking them in a non-debug build.
So long as the input qstr identifier is valid (below the maximum number of
qstrs) the function will always return a valid pointer. This patch
eliminates the "return 0" dead-code.
This time hopefully should work reliably, using make $(wildcard) function,
which in this case either expands to existing prj_$(BOARD).conf file, or to
an empty string for non-existing one.
This patch improves parsing of floating point numbers by converting all the
digits (integer and fractional) together into a number 1 or greater, and
then applying the correct power of 10 at the very end. In particular the
multiple "multiply by 0.1" operations to build a fraction are now combined
together and applied at the same time as the exponent, at the very end.
This helps to retain precision during parsing of floats, and also includes
a check that the number doesn't overflow during the parsing. One benefit
is that a float will have the same value no matter where the decimal point
is located, eg 1.23 == 123e-2.
Put offset first in OR expressions, and use "offset" var instead of
hardcoded numbers. Hopefully, this will make it more self-describing
and show patterns better.
Dramatically improves TCP sending throughput because without an explicit
call to tcp_output() the data is only sent to the lower layers via the
lwIP slow timer which (by default) ticks every 500ms.
Before this patch MP_BINARY_OP_IN had two meanings: coming from bytecode it
meant that the args needed to be swapped, but coming from within the
runtime meant that the args were already in the correct order. This lead
to some confusion in the code and comments stating how args were reversed.
It also lead to 2 bugs: 1) containment for a subclass of a native type
didn't work; 2) the expression "{True} in True" would illegally succeed and
return True. In both of these cases it was because the args to
MP_BINARY_OP_IN ended up being reversed twice.
To fix these things this patch introduces MP_BINARY_OP_CONTAINS which
corresponds exactly to the __contains__ special method, and this is the
operator that built-in types should implement. MP_BINARY_OP_IN is now only
emitted by the compiler and is converted to MP_BINARY_OP_CONTAINS by
swapping the arguments.
In mp_binary_op, there is no need to explicitly check for type->getiter
being non-null and raising an exception because this is handled exactly by
mp_getiter(). So just call the latter unconditionally.
This patch introduces a new compile-time config option to disable multiple
inheritance at the Python level: MICROPY_MULTIPLE_INHERITANCE. It is
enabled by default.
Disabling multiple inheritance eliminates a lot of recursion in the call
graph (which is important for some embedded systems), and can be used to
reduce code size for ports that are really constrained (by around 200 bytes
for Thumb2 archs).
With multiple inheritance disabled all tests in the test-suite pass except
those that explicitly test for multiple inheritance.
This is a low-cost evaluation kit board from ST based on the STM32
Nucleo-144 form factor. It uses the STM32F746ZG MCU in the LQFP144
package. The MCU has 1MB of flash and 320kB of System RAM.
Cortex-M7 runs at up to 216MHz.
It's possible to use the methods (eg ilistdir) of a VFS FatFS object
without it being mounted in the VFS itself. This previously worked but
only because FatFS was "mounting" the filesystem automatically when any
function (eg f_opendir) was called. But it didn't work for ports that used
synchronisation objects (_FS_REENTRANT) because they are only initialised
via a call to f_mount. So, call f_mount explicitly when creating a new
FatFS object so that everything is set up correctly. Then also provide a
finaliser to do the f_umount call, but only if synchronisation objects are
enabled (since otherwise the f_umount call does nothing).
The function mp_obj_new_str_of_type is a general str object constructor
used in many places in the code to create either a str or bytes object.
When creating a str it should first check if the string data already exists
as an interned qstr, and if so then return the qstr object. This patch
makes the function have such behaviour, which helps to reduce heap usage by
reusing existing interned data where possible.
The old behaviour of mp_obj_new_str_of_type (which didn't check for
existing interned data) is made available through the function
mp_obj_new_str_copy, but should only be used in very special cases.
One consequence of this patch is that the following expression is now True:
'abc' is ' abc '.split()[0]
This patch simplifies the str creation API to favour the common case of
creating a str object that is not forced to be interned. To force
interning of a new str the new mp_obj_new_str_via_qstr function is added,
and should only be used if warranted.
Apart from simplifying the mp_obj_new_str function (and making it have the
same signature as mp_obj_new_bytes), this patch also reduces code size by a
bit (-16 bytes for bare-arm and roughly -40 bytes on the bare-metal archs).
Rationale:
* Calling Python build tool scripts from makefiles should be done
consistently using `python </path/to/script>`, instead of relying on the
correct she-bang line in the script [1] and the executable bit on the
script being set. This is more platform-independent.
* The name/path of the Python executable should always be used via the
makefile variable `PYTHON` set in `py/mkenv.mk`. This way it can be
easily overwritten by the user with `make PYTHON=/path/to/my/python`.
* The Python executable name should be part of the value of the makefile
variable, which stands for the build tool command (e.g. `MAKE_FROZEN` and
`MPY_TOOL`), not part of the command line where it is used. If a Python
tool is substituted by another (non-python) program, no change to the
Makefiles is necessary, except in `py/mkenv.mk`.
* This also solves #3369 and #1616.
[1] There are systems, where even the assumption that `/usr/bin/env` always
exists, doesn't hold true, for example on Android (where otherwise the unix
port compiles perfectly well).
All the asm macro names that convert a particular architecture to a generic
interface now follow the convention whereby the "destination" (usually a
register) is specified first.
Recent vendor SDKs ship libs with code in .text section, which previously
was going into .irom0.text. Adjust the linker script to route these
sections back to iROM (follows upstream change).
The SHA1 hashing functionality is provided via the "axtls" library's
implementation, and hence is unavailable when the "axtls" library is not being
used. This change provides the same SHA1 hashing functionality when using the
"mbedtls" library by using its implementation instead.
Macros to convert big-endian values to host byte order and vice-versa.
These were defined in adhoc way for some ports (e.g. esp8266), allow
reuse, provide default implementations, while allow ports to override.
In the vendor SDK 2.1.0, some of the functions which previously didn't
have prototypes, finally acquired them. Change prototypes on our side
to match those in vendor headers, to avoid warnings-as-errors.
If SSL_EAGAIN is returned (which is a feature of MicroPython's axTLS fork),
return EAGAIN.
Original axTLS returns SSL_OK both when there's no data to return to user
yet and when the underlying stream returns EAGAIN. That's not distinctive
enough, for example, original module code works well for blocking stream,
but will infinite-loop for non-blocking socket with EAGAIN. But if we fix
non-blocking case, blocking calls to .read() will return few None's initially
(while axTLS progresses thru handshake).
Using SSL_EAGAIN allows to fix non-blocking case without regressing the
blocking one.
Note that this only handles case of non-blocking reads of application data.
Initial handshake and writes still don't support non-blocking mode and must
be done in the blocking way.
The technique of using alloca is how dotted import names are composed in
mp_import_from and mp_builtin___import__, so use the same technique in the
compiler. This puts less pressure on the heap (only the stack is used if
the qstr already exists, and if it doesn't exist then the standard qstr
block memory is used for the new qstr rather than a separate chunk of the
heap) and reduces overall code size.
This reverts commit 3289b9b7a7.
The commit broke building on MINGW because the filename became
micropython.exe.exe. A proper solution to support more Windows build
environments requires more thought and testing.
Per the comment found here
https://github.com/micropython/micropython-esp32/issues/209#issuecomment-339855157,
this patch adds finaliser code to prevent memory leaks from ussl objects,
which is especially useful when memory for a ussl context is allocated
outside the uPy heap. This patch is in-line with the finaliser code found
in many modsocket implementations for various ports.
This feature is configured via MICROPY_PY_USSL_FINALISER and is disabled by
default because there may be issues using it when the ussl state *is*
allocated on the uPy heap, rather than externally.
With inplace methods now disabled by default, it makes sense to enable
reverse methods, as they allow for more useful features, e.g. allow
for datetime module to implement both 2 * HOUR and HOUR * 2 (where
HOUR is e.g. timedelta object).
This allows to configure support for inplace special methods separately,
similar to "normal" and reverse special methods. This is useful, because
inplace methods are "the most optional" ones, for example, if inplace
methods aren't defined, the operation will be executed using normal
methods instead.
As a caveat, __iadd__ and __isub__ are implemented even if
MICROPY_PY_ALL_INPLACE_SPECIAL_METHODS isn't defined. This is similar
to the state of affairs before binary operations refactor, and allows
to run existing tests even if MICROPY_PY_ALL_INPLACE_SPECIAL_METHODS
isn't defined.
If MICROPY_PY_ALL_SPECIAL_METHODS is defined, actually define all special
methods (still subject to gating by e.g. MICROPY_PY_REVERSE_SPECIAL_METHODS).
This adds quite a number of qstr's, so should be used sparingly.
VLAs can be expensive on stack usage due to stack alignment requirements,
and also the fact that extra local variables are needed to track the
dynamic size of the stack. So using fixed-size arrays when possible can
help to reduce code size and stack usage.
In this particular case, the maximum value of n_args in the VLA is 2 and so
it's more efficient to just allocate this array with a fixed size. This
reduces code size by around 30 bytes on Thumb2 and Xtensa archs. It also
reduces total stack usage of the function: on Thumb2 the usage with VLA is
between 40 and 48 bytes, which is reduced to 32; on Xtensa, VLA usage is
between 64 and 80 bytes, reduced to 32; on x86-64 it's at least 88 bytes
reduced to 80.
CPython only supports the server_hostname keyword arg via the SSLContext
object, so use that instead of the top-level ssl.wrap_socket. This allows
the test to run on CPython the same as uPy.
Also add the "Host:" header to correctly make a GET request (for URLs that
are hosted on other servers). This is not strictly needed to test the SSL
connection but helps to debug things when printing the response.
Unix naming is historical, before current conventions were established.
All other ports however have it as "modusocket.c", so rename for
consistency and to avoid confusion.
Update makeqstrdata.py to sort strings starting with "__" to the beginning
of qstr list, so they get low qstr id's, guaranteedly fitting in 8 bits.
Then use this property to further compact op_id => qstr mapping arrays.
Per https://docs.python.org/3/library/sys.html#sys.getsizeof:
getsizeof() calls the object’s __sizeof__ method. Previously, "getsizeof"
was used mostly to save on new qstr, as we don't really support calling
this method on arbitrary objects (so it was used only for reporting).
However, normalize it all now.
Not all compilers/analysers are smart enough to realise that this function
is never called if MICROPY_ERROR_REPORTING is not TERSE, because the logic
in the code uses if statements rather than #if to select whether to call
this function or not (MSC in debug mode is an example of this, but there
are others). So just unconditionally compile this helper function. The
code-base anyway relies on the linker to remove unused functions.
The legacy function pyb.repl_uart() is still provided and retains its
original behaviour (it only accepts a UART object). uos.dupterm() will now
accept any object with write/readinto methods. At the moment there is just
1 dupterm slot.
Without this the board will crash when deactivating a stream that doesn't
have a close() method (eg UART) or that raises an exception within the
method (eg user-defined function).
The W5200 and W5500 can support up to 80MHz so 42MHz (the maximum the
pyboard can do in its standard configuration) should be safe.
Tested to give around 1050000 kbytes/sec TCP download speed on a W5500,
which is about 10% more than with the previous SPI speed of 21MHz.
Which Wiznet chip to use is a compile-time option: MICROPY_PY_WIZNET5K
should be set to either 5200 or 5500 to support either one of these
Ethernet chips. The driver is called network.WIZNET5K in both cases.
Note that this commit introduces a breaking-change at the build level
because previously the valid values for MICROPY_PY_WIZNET5K were 0 and 1
but now they are 0, 5200 and 5500.
This patch implements the basic SPI read/write functions for the W5500
chip. It also allows _WIZCHIP_ to be configured externally to select the
specific Wiznet chip.
The uos.dupterm() signature and behaviour is updated to reflect the latest
enhancements in the docs. It has minor backwards incompatibility in that
it no longer accepts zero arguments.
The dupterm_rx helper function is moved from esp8266 to extmod and
generalised to support multiple dupterm slots.
A port can specify multiple slots by defining the MICROPY_PY_OS_DUPTERM
config macro to an integer, being the number of slots it wants to have;
0 means to disable the dupterm feature altogether.
The unix and esp8266 ports are updated to work with the new interface and
are otherwise unchanged with respect to functionality.
So that a pointer to it can be passed as a pointer to math_generic_1. This
patch also makes the function work for single and double precision floating
point.
This patch changes how most of the plain math functions are implemented:
there are now two generic math wrapper functions that take a pointer to a
math function (like sin, cos) and perform the necessary conversion to and
from MicroPython types. This helps to reduce code size. The generic
functions can also check for math domain errors in a generic way, by
testing if the result is NaN or infinity combined with finite inputs.
The result is that, with this patch, all math functions now have full
domain error checking (even gamma and lgamma) and code size has decreased
for most ports. Code size changes in bytes for those with the math module
are:
unix x64: -432
unix nanbox: -792
stm32: -88
esp8266: +12
Tests are also added to check domain errors are handled correctly.
While this console API improves handling on real hardware boards
(e.g. clipboard paste is much more reliable, as well as programmatic
communication), it vice-versa poses problems under QEMU, apparently
because it doesn't emulate UART interrupt handling faithfully. That
leads to inability to run the testsuite on QEMU at all. To work that
around, we have to suuport both old and new console routines, and use
the old ones under QEMU.
We want to close communication object even if there were exceptions
somewhere in the code. This is important for --device exec:/execpty:
which may otherwise leave processing running in the background.
Ideally, these should be configurable from Python (using network module),
but as that doesn't exist, we better off using Zephyr's native bootstrap
configuration facility.
The poweroff() and poweron() methods are used to do soft power control of
the display, and this patch makes these methods work the same for both I2C
and SPI interfaces.
Printing "(null)" when a NULL string pointer is passed to %s is a debugging
feature and not a feature that's relied upon by the code. So it only needs
to be compiled in when debugging (such as assert) is enabled, and saves
roughy 30 bytes of code when disabled.
This patch also fixes this NULL check to not do the check if the precision
is specified as zero.
Header files that are considered internal to the py core and should not
normally be included directly are:
py/nlr.h - internal nlr configuration and declarations
py/bc0.h - contains bytecode macro definitions
py/runtime0.h - contains basic runtime enums
Instead, the top-level header files to include are one of:
py/obj.h - includes runtime0.h and defines everything to use the
mp_obj_t type
py/runtime.h - includes mpstate.h and hence nlr.h, obj.h, runtime0.h,
and defines everything to use the general runtime support functions
Additional, specific headers (eg py/objlist.h) can be included if needed.
Qstr values fit in 16-bits (and this fact is used elsewhere in the code) so
no need to use more than that for the large lookup tables. The compiler
will anyway give a warning if the qstr values don't fit in 16 bits. Saves
around 80 bytes of code space for Thumb2 archs.
Building mpy-cross: this patch adds .exe to the PROG name when building
executables for host (eg mpy-cross) on Windows. make clean now removes
mpy-cross.exe under Windows.
Building MicroPython: this patch sets MPY_CROSS to mpy-cross.exe or
mpy-cross so they can coexist and use cygwin or WSL without rebuilding
mpy-cross. The dependency in the mpy rule now uses mpy-cross.exe for
Windows and mpy-cross for Linux.
CPython docs explicitly state that the RHS of a set/frozenset binary op
must be a set to prevent user errors. It also preserves commutativity of
the ops, eg: "abc" & set() is a TypeError, and so should be set() & "abc".
This change actually decreases unix (x64) code by 160 bytes; it increases
stm32 by 4 bytes and esp8266 by 28 bytes (but previous patch already
introduced a much large saving).
A lot of set's methods (the mutable ones) are not allowed to operate on a
frozenset, and giving frozenset a separate locals dict with only the
methods that it supports allows to simplify the logic that verifies if
args are a set or a frozenset. Even though the new frozenset locals dict
is relatively large (88 bytes on 32-bit archs) there is a much bigger
saving coming from the removal of a const string for an error message,
along with the removal of some checks for set or frozenset type.
Changes in code size due to this patch are (for ports that changed at all):
unix x64: -56
unix nanbox: -304
stm32: -64
esp8266: -124
cc3200: -40
Apart from the reduced code, frozenset now has better tab-completion
because it only lists the valid methods. And the error message for
accessing an invalid method is now more detailed (it includes the
method name that wasn't found).
This returns a complex number, following CPython behaviour. For ports that
don't have complex numbers enabled this will raise a ValueError which gives
a fail-safe for scripts that were written assuming complex numbers exist.
This adds a new configuration option to print runtime warnings and errors to
stderr. On Unix, CPython prints warnings and unhandled exceptions to stderr,
so the unix port here is configured to use this option.
The unix port already printed unhandled exceptions on the main thread to
stderr. This patch fixes unhandled exceptions on other threads and warnings
(issue #2838) not printing on stderr.
Additionally, a couple tests needed to be fixed to handle this new behavior.
This is done by also capturing stderr when running tests.
It removes the need for a wrapper Python function to dispatch to the
framebuf method which makes each function call a bit faster, roughly 2.5x.
This patch also adds the rest of the framebuf methods to the SSD class.
The timer prescaler is buffered by default, and this patch enables ARPE
which buffers the auto-reload register. With both of these registers
buffered it's now possible to smoothly change the timer's frequency and
have a smoothly varying PWM output.
Allow literal minus in char classes to be in trailing position, e.g. [a-c-].
(Previously, minus was allowed only at the start.)
This increases ARM Thumb2 code size by 8 bytes.
Current users of fixed vstr buffers (building file paths) assume that there
is no overflow and do not check for overflow after building the vstr. This
has the potential to lead to NULL pointer dereferences
(when vstr_null_terminated_str returns NULL because it can't allocate RAM
for the terminating byte) and stat'ing and loading invalid path names (due
to the path being truncated). The safest and simplest thing to do in these
cases is just raise an exception if a write goes beyond the end of a fixed
vstr buffer, which is what this patch does. It also simplifies the vstr
code.
The vstr argument to the calls to vstr_add_len are dynamically allocated
(ie fixed_buf=false) and so vstr_add_len will never return NULL. So
there's no need to check for it. Any out-of-memory errors are raised by
the call to m_renew in vstr_ensure_extra.
The aim of this patch is to rewrite the functions that create exception
instances (mp_obj_exception_make_new and mp_obj_new_exception_msg_varg) so
that they do not call any functions that may raise an exception. Otherwise
it's possible to create infinite recursion with an exception being raised
while trying to create an exception object.
The two main things that are done to accomplish this are:
1. Change mp_obj_new_exception_msg_varg to just format the string, then
call mp_obj_exception_make_new to actually create the exception object.
2. In mp_obj_exception_make_new and mp_obj_new_exception_msg_varg try to
allocate all memory first using functions that don't raise exceptions
If any of the memory allocations fail (return NULL) then degrade
gracefully by trying other options for memory allocation, eg using the
emergency exception buffer.
3. Use a custom printer backend to conservatively format strings: if it
can't allocate memory then it just truncates the string.
As part of this rewrite, raising an exception without a message, like
KeyError(123), will now use the emergency buffer to store the arg and
traceback data if there is no heap memory available.
Memory use with this patch is unchanged. Code size is increased by:
bare-arm: +136
minimal x86: +124
unix x64: +72
unix nanbox: +96
stm32: +88
esp8266: +92
cc3200: +80
This allows user classes to implement __abs__ special method, and saves
code size (104 bytes for x86_64), even though during refactor, an issue
was fixed and few optimizations were made:
* abs() of minimum (negative) small int value is calculated properly.
* objint_longlong and objint_mpz avoid allocating new object is the
argument is already non-negative.
Prior to this patch calling pyb.Timer(id) would always create a new timer
instance, even if there was an existing one. This patch fixes this
behaviour to match other peripherals, like UART, such that constructing a
timer with just the id will retrieve any existing instances.
The patch also refactors the way timers are validated on construction to
simplify and reduce code size.
This patch also removes the empty type "pinbase_type" (which crashes if
accessed) and uses "machine_pinbase_type" instead as the type of the
PinBase singleton.
Given that various ports now require submodules, rewrite the section
to be more generic.
Also, add git submodule update command to other sections for easy user
start.
If, for class X, X.__add__(Y) doesn't exist (or returns NotImplemented),
try Y.__radd__(X) instead.
This patch could be simpler, but requires undoing operand swap and
operation switch to get non-confusing error message in case __radd__
doesn't exist.
connect, send, recv, sendto and recvfrom now release the GIL. accept
already releases the GIL because it calls mp_hal_delay_ms() within its
busy-wait loop.
Previous to this patch the i2c.scan() method would do up to 100 probes per
I2C address, to detect the devices on the bus. This repeated probing was a
relic from when the code was copied from the accelerometer initialisation,
which requires to do repeated probes while waiting for the accelerometer
chip to turn on.
But I2C devices shouldn't need more than 1 probe to detect their presence,
and the generic software I2C implementation uses 1 probe successfully. So
this patch changes the implementation to use 1 probe per address, which
significantly speeds up the scan operation.
This is to allow to place reverse ops immediately after normal ops, so
they can be tested as one range (which is optimization for reverse ops
introduction in the next patch).
Originally, there were grouped in blocks of 5, to make it easier e.g.
to assess and numeric code of each. But now it makes more sense to
group it by semantics/properties, and then split in chunks still,
which usually leads to chunks of ~6 ops.
It starts a dichotomy of mp_binary_op_t values which can't appear in the
bytecode. Another reason to move it is to VALUES of OP_* and OP_INPLACE_*
nicely adjacent. This also will be needed for OP_REVERSE_*, to be soon
introduced.
With MBEDTLS_DEBUG_C disabled the function mbedtls_debug_set_threshold()
doesn't exist. There's also no need to call mbedtls_ssl_conf_dbg() so a
few bytes can be saved on disabling that and not needing the mbedtls_debug
callback.
This patch adds a function utf8_check() to check for a valid UTF-8 encoded
string, and calls it when constructing a str from raw bytes. The feature
is selectable at compile time via MICROPY_PY_BUILTINS_STR_UNICODE_CHECK and
is enabled if unicode is enabled. It costs about 110 bytes on Thumb-2, 150
bytes on Xtensa and 170 bytes on x86-64.
This is to keep the top-level directory clean, to make it clear what is
core and what is a port, and to allow the repository to grow with new ports
in a sustainable way.
There are 2 changes:
- remove early initialisation of LSE and replaced it by LSEDRIVE config
(there is no reason to call HAL_RCC_OscConfig twice).
- add initialisation of the variables PLLSAI1Source and PLLSAI1M as they
are needed in Cube HAL 1.8.1.
IEEE floating point is specified such that a comparison of NaN with itself
returns false, and Python respects these semantics. This patch makes uPy
also have these semantics. The fix has a minor impact on the speed of the
object-equality fast-path, but that seems to be unavoidable and it's much
more important to have correct behaviour (especially in this case where
the wrong answer for nan==nan is silently returned).
These are now returned as "operation not supported" instead of raising
TypeError. In particular, this fixes equality for float vs incompatible
types, which now properly results in False instead of exception. This
also paves the road to support reverse operation (e.g. __radd__) with
float objects.
This is achieved by introducing mp_obj_get_float_maybe(), similar to
existing mp_obj_get_int_maybe().
Prior to this patch, the size of the buffer given to pack_into() was checked
for being too small by using the count of the arguments, not their actual
size. For example, a format spec of '4I' would only check that there was 4
bytes available, not 16; and 'I' would check for 1 byte, not 4.
The pack() function is ok because its buffer is created to be exactly the
correct size.
The fix in this patch calculates the total size of the format spec at the
start of pack_into() and verifies that the buffer is large enough. This
adds some computational overhead, to iterate through the whole format spec.
The alternative is to check during the packing, but that requires extra
code to handle alignment, and the check is anyway not needed for pack().
So to maintain minimal code size the check is done using struct_calcsize.
Prior to this patch, the size of the buffer given to unpack/unpack_from was
checked for being too small by using the count of the arguments, not their
actual size. For example, a format spec of '4I' would only check that
there was 4 bytes available, not 16; and 'I' would check for 1 byte, not 4.
This bug is fixed in this patch by calculating the total size of the format
spec at the start of the unpacking function. This function anyway needs to
calculate the number of items at the start, so calculating the total size
can be done at the same time.
This patch makes a repeat counter behave the same as repeating the
typecode, when there are not enough args. For example:
struct.pack('2I', 1) now behave the same as struct.pack('II', 1).
Similar to the existing testcase, but test that returning both value of
native type and instance of another user class from __new__ lead to
__init__ not being called, for better coverage.
NotImplemented means "try other fallbacks (like calling __rop__
instead of __op__) and if nothing works, raise TypeError". As
MicroPython doesn't implement any fallbacks, signal to raise
TypeError right away.
This upgrades the HAL to the versions:
- F4 V1.16.0
- F7 V1.7.0
- L4 V1.8.1
The main changes were in the SD card driver. The vendor changed the SD
read/write functions to accept block number intead of byte address, so
there is no longer any need for a custom patch for this in stm32lib.
The CardType values also changed, so pyb.SDCard().info() will return
different values for the 3rd element of the tuple, but this function was
never documented.
The unary-op/binary-op enums are already defined, and there are no
arithmetic tricks used with these types, so it makes sense to use the
correct enum type for arguments that take these values. It also reduces
code size quite a bit for nan-boxing builds.
The SPI flash driver now supports using an arbitrary SPI object to
communicate with the flash chip, and in particular can use a hardware SPI
peripheral.
Otherwise, it will silently get incorrect result on other values types,
including CPython tuple form like "foo.png".endswith(("png", "jpg"))
(which MicroPython doesn't support for unbloatedness).
Allows for simpler, smaller and faster code at run time when selecting the
boards frequency, and allows more customisation opportunities for the PLL
values depending on the target MCU.
Changes for F7 are:
- machine.reset_cause() now reports DEEPSLEEP_RESET correctly;
- machine.sleep() is further optimised to reduce power consumption;
- machine.deepsleep() is now implemented and working.
For SEEK_SET, offset should be treated as unsigned, to allow full-width
stream sizes (e.g. 32-bit instead of 31-bit). This is now fully documented
in stream.h. Also, seek symbolic constants are added.
Too big positive, or too big negative offset values could lead to overflow
and address space wraparound and thus access to unrelated areas of memory
(a security issue).
State that this doc describes generic, "core" MicroPython functionality,
any particular port may diverge in both directions, by both omitting
some functionality, and adding more, both cases described outside the
generic documentation.
Describe that the only portable way to deal with addresses is by using
getaddrinfo(). Describe that some ports may support tuple addresses using
"socket" module (vs "usocket" of native MicroPython).
The original issue leading to crash on startup if no default network
interface was presented, was resolved some time ago.
Note that this enables generic networking subsystem, not networking
on Carbon.
This clarifies return values and the handling of invalid (e.g. newline)
characters.
Encoding conforms to RFC 3548, but decoding does not, as it ignores invalid
characters in base64 input. Instead, it conforms to MIME handling of base64
(RFC 2045).
Note that CPython doesn't document handling of invalid characters in
a2b_base64() docs:
https://docs.python.org/3/library/binascii.html#binascii.a2b_base64 , so
we specify it more explicitly than it, based on CPython's actual behavior
(with which MicroPython now compliant).
This implementation ignores invalid characters in the input. This allows
it to decode the output of b2a_base64, and also mimics the behavior of
CPython.
The value of 0 can't be used because otherwise mp_binary_get_size will let
a null byte through as the type code (intepreted as byterray). This can
lead to invalid type-specifier strings being let through without an error
in the struct module, and even buffer overruns.
current_tid() returns current thread ID. stacks_analyze() calls
k_call_stacks_analyze() which, with CONFIG_INIT_STACKS enabled,
will print stack usage for some well-known threads in the system.
- Changed: ValueError, TypeError, NotImplementedError
- OSError invocations unchanged, because the corresponding utility
function takes ints, not strings like the long form invocation.
- OverflowError, IndexError and RuntimeError etc. not changed for now
until we decide whether to add new utility functions.
Since the stride is specified in pixels, in a 4-bit horizontal format it
has to always be even, otherwise the computation is wrong and we can
write outside of the buffer sometimes.
Before this patch the mperrno.h file could be included and would silently
succeed with incorrect config settings, because mpconfig.h was not yet
included.
If constants (eg mp_const_none_obj) are placed in very high memory
locations that require 64-bits for the pointer then the assembler must be
able to emit instructions to move such pointers to one of the top 8
registers (ie r8-r15).
It's not used anywhere else in the VM loop, and clashes with (is shadowed
by) the n_state variable that's redeclared towards the end of the
mp_execute_bytecode function. Code size is unchanged.
The code conventions suggest using header guards, but do not define how
those should look like and instead point to existing files. However, not
all existing files follow the same scheme, sometimes omitting header guards
altogether, sometimes using non-standard names, making it easy to
accidentally pick a "wrong" example.
This commit ensures that all header files of the MicroPython project (that
were not simply copied from somewhere else) follow the same pattern, that
was already present in the majority of files, especially in the py folder.
The rules are as follows.
Naming convention:
* start with the words MICROPY_INCLUDED
* contain the full path to the file
* replace special characters with _
In addition, there are no empty lines before #ifndef, between #ifndef and
one empty line before #endif. #endif is followed by a comment containing
the name of the guard macro.
py/grammar.h cannot use header guards by design, since it has to be
included multiple times in a single C file. Several other files also do not
need header guards as they are only used internally and guaranteed to be
included only once:
* MICROPY_MPHALPORT_H
* mpconfigboard.h
* mpconfigport.h
* mpthreadport.h
* pin_defs_*.h
* qstrdefs*.h
Prior to this patch Servo numbers 1, 2, 3, 4 mapped to pins X3, X4, X1, X2
on PYBLITE which doesn't match the standard PYB mapping. This patch fixes
the mapping.
The default frozen modules are no longer included (but users can still
specify their own via FROZEN_MPY_DIR), complex numbers are disabled and so
are the native, viper and asm_thumb emitters. Users needing these features
can tune the build to disable other things.
Reasons to disable:
- the code is relatively robust so doesn't need full checking in the
main executable, and the coverage build is used for full testing
with assertions still enabled;
- reduces code size noticeably, by 27k for x86-64 and 20k for x86;
- allows to more easily track changes in code size, since assertions
can skew things.
Got tired of running rm -rf manually. Make should clean, and should clean
fast. Also, fix always-running config-related commands (by having per-board
merged configs).
This makes top-level ToC of the pyboard docs consistent with other ports
(consisting of 3 chapters: QuickRef, General Info, and Tutorial).
Also, some other minor tweaks applied, like local ToC for General Info and
headings mentioning pyboard.
This pseudo-section causes artifacts with latexpdf generation (almost
empty page with list containing literal "genindex", "modeindex", "search"
items). For HTML docs, these sections can be accessed from "home" page.
We don't use alpha/beta/RC, so for us version and release should be the
same, or it leads to confusion (for example, current, 1.9.1 docs are
marked as 1.9 at places).
The idea is to allow to define a kind of "macros" for repeatitive text,
so all occurrances can be updated in one place. Unfortunately, RST doesn't
support replacements with arguments, which limits usefulness of them and
should be taken into account.
We have enough terms or references throughout the docs which may be not
immediately clear or have some important nuances. Referencing terms in
gloassary is the best way to deal with that.
The old intro talked about "differences", but there were hardly any
sections describing differences, mostly MicroPython specific features.
On the other hand, we now have real "differences" chapter, though it's
mostly concerned with stdlib differences.
So, try to avoid confusion by changing wording and linking to the other
chapters and contrasting them with what is described in "MicroPython
language".
Prior to this patch there were 2 paths for creating the namedtuple, one for
when no keyword args were passed, and one when there were keyword args.
And alloca was used in the keyword-arg path to temporarily create the array
of elements for the namedtuple, which would then be copied to a
heap-allocated object (the namedtuple itself).
This patch simplifies the code by combining the no-keyword and keyword
paths, and removing the need for the alloca by constructing the namedtuple
on the heap before populating it.
Heap usage in unchanged, stack usage is reduced, use of alloca is removed,
and code size is not increased and is actually reduced by between 20-30
bytes for most ports.
By default the firmware is built with single-precision floating point.
To build a particular board using double precision instead, put the
following line in the mpconfigboard.mk file:
FLOAT_IMPL = double
The while-loop that calls chop_component will guarantee that level==-1 at
the end of the loop. Hence the code following it is unnecessary.
The check for p==this_name will catch imports that are beyond the
top-level, and also covers the case of new_mod_q==MP_QSTR_ (equivalent to
new_mod_l==0) so that check is removed.
There is also a new check at the start for level>=0 to guard against
__import__ being called with bad level values.
Previously, only "selected chapters" were shown in left-pane ToC (of
Read The Docs theme). These chapters were selected out of order. The
rest of chapters were hidden beyond "Documentation Contents" pseudo-
chapter. This arguably led only to confusion, as many people probably
never tried to open that pseudo-chapter, and those who did, were
confused. Such organization is even worse for PDF output, causing
chapters go in mix-mashed order.
So, instead move to single clean ToC. This will allow readers of HTML
to have access to any doc content at their fingertips (and straight
before their eyes), and will allow to finally have clean PDF docs.
A previous version of the 1-wire driver (which was recently replaced by the
current one) had this behaviour and it allows to create a 1-wire bus
without any external pull-up resistors.
And and FLASH_FS, and use "K" values instead of hex numbers for lengths.
The increase of FLASH_TEXT is to allow more frozen bytecode for a
particular user's project. It's not used for anything else.
Move hardware-specific optimizations to the very end of document, and
add visible note that it gives an example for Pyboard. Remove references
to specific hardware technologies, so the doc can be more naturally
used across ports. Various markup updates to adhere to the latest
docs conventions.
This causes `symbol` syntax to be equivalent to :any:`symbol`, which is
in turn the easiest way to cross-reference an arbitrary symbol in the
docs:
http://www.sphinx-doc.org/en/stable/markup/inline.html#role-any
:any: requires at least Sphinx 1.3 (for reference, Ubuntu 16.03 ships
with 1.3.6, the latest 1.6.3).
Any many of our docs, `symbol` is misused to specify arguments to
functions, etc. Refactoring that is in progress. (CODECONVENTIONS
already specify proper syntax for both arguments and xrefs, based
on CPython conventions).
The idea is that these tests can be run with just a test server running
on a test host, with device under test connecting to it, instead of
requiring Internet connection for testing.
Such setup is however WIP, and some tests in net_hosted/ are so far
written to connect to Internet, as there're not test server written
yet. This is expected to evolve over time.
GC finalization should be enabled for modlwip, or it may lead to GC
problems with socket objects. This decreases usable heap size from
36288 to 35968 (-320) bytes.
It belongs with the other pin config functions in machine_pin.c. Also,
esp_mphal.c is put in iRAM so this change saves about 300 bytes of iRAM
(and mp_hal_pin_open_drain is not a time critical function so doesn't
need to be in iRAM).
These drivers can now be used by any port (so long as that port has the
_onewire driver from extmod/modonewire.c).
These drivers replace the existing 1-wire and DS18X20 drivers in the
drivers/onewire directory. The existing ones were pyboard-specific and
not very efficient nor minimal (although the 1-wire driver was written in
pure Python it only worked at large enough CPU frequency).
This commit brings backwards incompatible API changes to the existing
1-wire drivers. User code should be converted to use the new drivers, or
check out the old version of the code and keep a local copy (it should
continue to work unchanged).
The 1-wire bus is defined with fixed timings so there should be no need to
change them dynamically at runtime. Making the timings fixed saves about
270 bytes of code and 20 bytes of RAM.
The reason it was separated is so that the low-level code could be put in
iRAM, for timing reasons. But:
1. Tests show that it's not necessary to have this code in iRAM for it to
function correctly, and taking it out of iRAM reclaims some of that precious
resource. Furthermore, even though these functions were in iRAM there were
some functions that it called (eg pin get/set functions) which were not in
iRAM, so partially defeated the purpose of putting the 1-wire code in iRAM.
2. It's easier to reuse this 1-wire code in other ports if it's in a single
file.
3. If it turns out that certain code does need to be in iRAM then one can
use the MP_FASTCODE macro to do that.
Previous to this patch, a label with value "0" was used to indicate an
invalid label, but that meant a wasted word (at slot 0) in the array of
label offsets. This patch adjusts the label indices so the first one
starts at 0, and the maximum value indicates an invalid label.
This patch fixes a bug whereby the Python stack was not correctly reset if
there was a break/continue statement in the else black of an optimised
for-range loop.
For example, in the following code the "j" variable from the inner for loop
was not being popped off the Python stack:
for i in range(4):
for j in range(4):
pass
else:
continue
This is now fixed with this patch.
This attempts to bootstrap network tests for MicroPython. This commits
sets test/net_inet/ as place for tests which require access to wide
Internet. They aren't intended to be run as part of the main testsuite,
instead to be run manually on demand.
test_tls_sites.py in particular check that it's possible to establish
SSL/TLS connection to select sites on the Internet: few references ones,
plus those for which problems were reported, and resolved.
The CC3100 only allows to set a timeout for receiving data, not for accept,
connect or send. But it can set non-blocking for all these operations and
this patch uses that feature to implement socket timeout in terms of non-
blocking behaviour combined with a loop.
This method follows the new HW API and allows to set a hard or soft IRQ
callback when a Pin has a level change. It still remains to make this
method return a IRQ object.
In CPython 3.4 this raises a SyntaxError. In CPython 3.5+ having a
positional after * is allowed but uPy has the wrong semantics and passes
the arguments in the incorrect order. To prevent incorrect use of a
function going unnoticed it is important to raise the SyntaxError in uPy,
until the behaviour is fixed to follow CPython 3.5+.
These implementations are incorrect (eg f2d and d2f don't handle special
values like 0.0) and proper versions can be provided by libgcc (or
equivalent depending on the toolchain).
libgcc is now linked with the stmhal port so that library will provide
these functions from now on.
The latest fashion is pushing certificate sub-chains, instead of a single
certificate, during TLS handshake. These are pushed via single TLS record
and effectively put minimum size limit on TLS record buffer. Recently,
these commonly grew over 4K, so we have little choice but to adjust.
This patch fixes 2 things when printing a floating-point number that
requires rounding up of the mantissa:
- retain the correct precision; eg 0.99 becomes 1.0, not 1.00
- if the exponent goes from -1 to 0 then render it as +0, not -0
Baremetal ports standardized on providing localtime(). localtime() offers
more functionality, in particular, strftime() can be completely implemented
in Python with localtime().
Taking the address of a local variable leads to increased stack usage, so
the mp_decode_uint_skip() function is added to reduce the need for taking
addresses. The changes in this patch reduce stack usage of a Python call
by 8 bytes on ARM Thumb, by 16 bytes on non-windowing Xtensa archs, and by
16 bytes on x86-64. Code size is also slightly reduced on most archs by
around 32 bytes.
This adds description of implied AbstractNIC base class, which should be
"subclasses" and implemented by a particular network device class.
This is just an initial step in that direction, the API and description
will be elabotated further.
The list starts with the simplest functionality - GPIO, proceeds to
communication interfaces (UART, SPI, I2C), the to time(r) related
things, then everything else.
For a couple of ports, there was information which directory is set
as current after boot. This information doesn't belong to "uos" module,
and is moved to boards' references (which actually already contained
information on which directory is chosen for boot, even if without
explicit mentioning that it becomes current directory, which is now
done).
The implementation is taken from stmhal/input.c, with code added to handle
ctrl-C. This built-in is controlled by MICROPY_PY_BUILTINS_INPUT and is
disabled by default. It uses readline() to capture input but this can be
overridden by defining the mp_hal_readline macro.
This method isn't implemented in any port. It seemed to have originated
in cc3200 port, but actually never was implemented there either. In
general case, it's impossible to implement this method (for example, for
a perfect GPO, which has only output latch without any feedback look
into a CPU).
For make v3.81, using "make -B" can set $? to empty and in this case the
auto-qstr generation needs to pass all args (ie $^) to cpp. The previous
fix for this (which was removed in 23a693ec2d)
used if statements in the shell command, which gave very long lines that
didn't work on certain systems (eg cygwin).
The fix in this patch is to use an $if(...) expression, which will evaluate
to $? (only newer prerequisites) if it's non empty, otherwise it will use
$^ (all prerequisites).
Previous to this patch the mp_emit_bc_adjust_stack_size function would
adjust the current stack size but would not increase the maximum stack size
if the current size went above it. This meant that certain Python code
(eg a try-finally block with no statements inside it) would not have enough
Python stack allocated to it.
This patch fixes the problem by always checking if the current stack size
goes above the maximum, and adjusting the latter if it does.
This patch fixes a regression introduced by
71a3d6ec3b
Previous to this patch the n_state variable was referring to that computed
at the very start of the mp_execute_bytecode function. This patch fixes it
so that n_state is recomputed when the code_state changes.
For consistent Pin/Signal class hierarchy. With it, Signal is a proper
(while still ducktyped) subclass of a Pin, and any (direct) usage of Pin
can be replace with Signal.
As stmhal's class is reused both as machine.Pin and legacy pyb.Pin,
high/low methods actually retained there.
Both aren't part of generic Hardware API: It's impossible to implement
.id() method in a generic case (e.g., when Pin is instantiated by the
underlying OS/RTOS). .board attribute is an obvious space hog which
instead can be implemented on Python level if needed.
Working on a build with PY_IO enabled (for PY_UJSON support) but PY_SYS_STDFILES disabled (no filesystem). There are multiple references to mp_sys_stdout_obj that should only be enabled if both PY_IO and PY_SYS_STDFILES are enabled.
There're a lot of changes and fixes in 1.8 regarding IP stack,
incompatible with previous Zephyr versions, so supporting them
doesn't make sense.
This is the last commit which should build with Zephyr 1.7.
Tests for an issue with line continuation failing in paste mode due to the
lexer only checking for \n in the "following" character position, before
next_char() has had a chance to convert \r and \r\n to \n.
This ensures that mpy-cross is automatically built (and is up-to-date) for
ports that use frozen bytecode. It also makes sure that .mpy files are
re-built if mpy-cross is changed.
VfsFat no longer has the listdir() method. Rather, if listdir()
functionality is needed then one should use uos.listdir() which will call
VfsFat.ilistdir().
Now consistently uses the EOL processing ("\r" and "\r\n" convert to "\n")
and EOF processing (ensure "\n" before EOF) provided by next_char().
In particular the lexer can now correctly handle input that starts with CR.
Prior to this patch only 'q' and 'Q' type arrays could store big-int
values. With this patch any big int that is stored to an array is handled
by the big-int implementation, regardless of the typecode of the array.
This allows arrays to work with all type sizes on all architectures.
This will lead to crash like:
FATAL: uncaught NLR 80a5420
On x86_32, the minimum heap size is smaller, but not 2 times, so just
use value which works for x86_64.
This patch allows mounting of VFS objects right at the root directory, eg
os.mount(vfs, '/'). It still allows VFS's to be mounted at a path within
the root, eg os.mount(vfs, '/flash'), and such mount points will override
any paths within a VFS that is mounted at the root.
Versions prior to v14.0 have a bug in parsing item functions when used
within a condition: https://github.com/Microsoft/msbuild/issues/368.
Since commit [db9c2e3] this results in an error when building MicroPython
with for example VS2013.
Fix this by creating an intermediate property.
fmode.c should only be included for builds targetting 'pure' windows, i.e.
msvc or mingw builds but not when using msys or cygwin's gcc (see #2298).
Just checking if the OS is windows and UNAME doesn't have msys stil leaves
the gate open for builds with cygwin's gcc since UNAME there is e.g.
CYGWIN_NT-6.1-WOW.
Fix this by checking for 'mingw' explicitly in the compiler version; both
gcc and clang have the -dumpmachine flag so the check should be ok for
all platforms.
The with semantics of this function is close to
pkg_resources.resource_stream() function from setuptools, which
is the canonical way to access non-source files belonging to a package
(resources), regardless of what medium the package uses (e.g. individual
source files vs zip archive). In the case of MicroPython, this function
allows to access resources which are frozen into the executable, besides
accessing resources in the file system.
This is initial stage of the implementation, which actually doesn't
implement "package" part of the semantics, just accesses frozen resources
from "root", or filesystem resource - from current dir.
Do not raise SIGPIPE, instead return EPIPE. Otherwise, e.g. writing
to peer-closed socket will lead to sudden termination of MicroPython
process. SIGPIPE is particularly nasty, because unix shell doesn't
print anything for it, so the above looks like completely sudden and
silent termination for unknown reason. Ignoring SIGPIPE is also what
CPython does. Note that this may lead to problems using MicroPython
scripts as pipe filters, but again, that's what CPython does. So,
scripts which want to follow unix shell pipe semantics (where SIGPIPE
means "pipe was requested to terminate, it's not an error"), should
catch EPIPE themselves.
Peer-closed socket is both readable and writable: read will return EOF,
write - error. Without this poll will hang on such socket.
Note that we don't return POLLHUP, based on argumentation in
http://www.greenend.org.uk/rjk/tech/poll.html that it should apply to
deeper disconnects, for example for networking, that would be link layer
disconnect (e.g. WiFi went down).
Make qstr generation depend on modifications in mpconfigport.h, mpconfig.h
and makeqstrdata.py and if any of those change scan all source files for
qstrs again since they might have changed (for example typcially when
enabling new features in mpconfig.h).
This fixes#2982 for msvc builds.
The standard preprocessor definition to differentiate debug and non-debug
builds is NDEBUG, not DEBUG, so don't rely on the latter:
- just delete the use of it in objint_longlong.c as it has been stale code
for years anyway (since commit [c4029e5]): SUFFIX isn't used anywhere.
- replace DEBUG with MICROPY_DEBUG_NLR in nlr.h: it is rarely used anymore
so can be off by default
This patch allows the following code to run without allocating on the heap:
super().foo(...)
Before this patch such a call would allocate a super object on the heap and
then load the foo method and call it right away. The super object is only
needed to perform the lookup of the method and not needed after that. This
patch makes an optimisation to allocate the super object on the C stack and
discard it right after use.
Changes in code size due to this patch are:
bare-arm: +128
minimal: +232
unix x64: +416
unix nanbox: +364
stmhal: +184
esp8266: +340
cc3200: +128
This patch refactors the handling of the special super() call within the
compiler. It removes the need for a global (to the compiler) state variable
which keeps track of whether the subject of an expression is super. The
handling of super() is now done entirely within one function, which makes
the compiler a bit cleaner and allows to easily add more optimisations to
super calls.
Changes to the code size are:
bare-arm: +12
minimal: +0
unix x64: +48
unix nanbox: -16
stmhal: +4
cc3200: +0
esp8266: -56
Prior to making this a config option it was previously available on these
(and all other) ports, and it makes sense to keep it enabled for mpy-cross
as well as ports that have a decent amount of space for the code.
With this optimisation enabled the compiler optimises the if-else
expression within a return statement. The optimisation reduces bytecode
size by 2 bytes for each use of such a return-if-else statement. Since
such a statement is not often used, and costs bytes for the code, the
feature is disabled by default.
For example the following code:
def f(x):
return 1 if x else 2
compiles to this bytecode with the optimisation disabled (left column is
bytecode offset in bytes):
00 LOAD_FAST 0
01 POP_JUMP_IF_FALSE 8
04 LOAD_CONST_SMALL_INT 1
05 JUMP 9
08 LOAD_CONST_SMALL_INT 2
09 RETURN_VALUE
and to this bytecode with the optimisation enabled:
00 LOAD_FAST 0
01 POP_JUMP_IF_FALSE 6
04 LOAD_CONST_SMALL_INT 1
05 RETURN_VALUE
06 LOAD_CONST_SMALL_INT 2
07 RETURN_VALUE
So the JUMP to RETURN_VALUE is optimised and replaced by RETURN_VALUE,
saving 2 bytes and making the code a bit faster.
Otherwise the type of parse-node and its kind has to be re-extracted
multiple times. This optimisation reduces code size by a bit (16 bytes on
bare-arm).
Add definitions/source files for features which work on the windows
ports but weren't yet enabled.
UTIME related lines are moved a couple of lines up to make comparision
with unix/mpconfigport.h easier in the future.
Sometimes when setting a channel callback the callback fires immediately,
even if the compare register is set to a value far into the future. This
happens when the free running counter has previously been equal to what
happens to be in the compare register.
This patch make sure that there is no pending interrupt when setting a
callback.
It controls the character that's used to (asynchronously) raise a
KeyboardInterrupt exception. Passing "-1" allows to disable the
interception of the interrupt character (as long as a port allows such a
behaviour).
This aligns the I2C class to match the standard machine.I2C API.
Note that this is a (small) breaking change to the existing cc3200 API.
The original API just returned the size of the input buffer so there's no
information lost by this change. To update scripts users should just use
the size of the buffer passed to these functions to get the number of bytes
that are read/written.
This is a user-facing change to the cc3200's API, to make it conform to the
new machine hardware API. The changes are:
- change I2C constructor to: I2C(id=0, *, freq=100000, scl=None, sda=None)
- change I2C init to: init(*, freq, scl, sda)
- removal of machine.I2C.MASTER constant
- I2C str/repr no longer prints I2C.MASTER
To update existing code it should be enough to just remove the I2C.MASTER
constant from contructor/init for I2C.
If we got a CRASH result, return early, similar to SKIP. This is important
because previous refactor changed branching logic a bit, so CRASH now gets
post-processed into CRASH\n, which broke remote hardware tests.
A shorter name takes less code size, less room in scripts and is faster to
type at the REPL.
Tests and HW-API examples are updated to reflect the change.
As Zephyr currently doesn't handle MTU itself (ZEP-1998), limit amount
of data we send on our side.
Also, if we get unsuccessful result from net_nbuf_append(), calculate
how much data it has added still. This works around ZEP-1984.
If a finaliser raises an exception then it must not propagate through the
GC sweep function. This patch protects against such a thing by running
finaliser code via the mp_call_function_1_protected call.
This patch also adds scheduler lock/unlock calls around the finaliser
execution to further protect against any possible reentrancy issues: the
memory manager is already locked when doing a collection, but we also don't
want to allow any scheduled code to run, KeyboardInterrupts to interupt the
code, nor threads to switch.
The common cases for inheritance are 0 or 1 parent types, for both built-in
types (eg built-in exceptions) as well as user defined types. So it makes
sense to optimise the case of 1 parent type by storing just the type and
not a tuple of 1 value (that value being the single parent type).
This patch makes such an optimisation. Even though there is a bit more
code to handle the two cases (either a single type or a tuple with 2 or
more values) it helps reduce overall code size because it eliminates the
need to create a static tuple to hold single parents (eg for the built-in
exceptions). It also helps reduce RAM usage for user defined types that
only derive from a single parent.
Changes in code size (in bytes) due to this patch:
bare-arm: -16
minimal (x86): -176
unix (x86-64): -320
unix nanbox: -384
stmhal: -64
cc3200: -32
esp8266: -108
This implements the orginal idea is that Signal is a subclass of Pin, and
thus can accept all the same argument as Pin, and additionally, "inverted"
param. On the practical side, it allows to avoid many enclosed parenses for
a typical declararion, e.g. for Zephyr:
Signal(Pin(("GPIO_0", 1))).
Of course, passing a Pin to Signal constructor is still supported and is the
most generic form (e.g. Unix port will only support such form, as it doesn't
have "builtin" Pins), what's introduces here is just practical readability
optimization.
"value" kwarg is treated as applying to a Signal (i.e. accounts for possible
inversion).
This buffer is used to allocate objects temporarily, and such objects
require that their underlying memory be correctly aligned for their data
type. Aligning for mp_obj_t should be sufficient for emergency exceptions,
but in general the memory buffer should aligned to the maximum alignment of
the machine (eg on a 32-bit machine with mp_obj_t being 4 bytes, a double
may not be correctly aligned).
This patch fixes a bug for certain nan-boxing builds, where mp_obj_t is 8
bytes and must be aligned to 8 bytes (even though the machine is 32 bit).
Changes made are:
- Use the time module in place of the pyb module for delays.
- Use spi.read/spi.write instead of spi.send/spi.receive.
- Drop some non-portable parameters to spi and pin initialization.
Thanks to @deshipu for the original patch.
Internal structure of k_fifo changed between 1.7 and 1.8, so we need
to abstract it away. This adds more functions than currently used, for
future work.
This allows to execute a command and communicate with its stdin/stdout
via pipes ("exec") or with command-created pseudo-terminal ("execpty"),
to emulate serial access. Immediate usecase is controlling a QEMU process
which emulates board's serial via normal console, but it could be used
e.g. with helper binaries to access real board over other hadware
protocols, etc.
An example of device specification for these cases is:
--device exec:../zephyr/qemu.sh
--device execpty:../zephyr/qemu2.sh
Where qemu.sh contains long-long qemu startup line, or calls another
command. There's a special support in this patch for running the command
in a new terminal session, to support shell wrappers like that (without
new terminal session, only wrapper script would be terminated, but its
child processes would continue to run).
Without this, if there's a large chunk of data coming from hardware (e.g.
clipboard paste, or fed programmatically from the other side of the console),
there's a behavior of initial mass fill-in of the buffer without any
consumption, which starts much later and doesn't catch up with further
filling, leading to buffer overflow.
MONO_xxx is much easier to read if you're not familiar with the code.
MVLSB is deprecated but kept for backwards compatibility, for the time
being.
This patch also updates the associated docs and tests.
This should be a little more efficient (since we anyway scan the input
packet for the interrupt char), and it should also fix any non-atomic read
issues with the buffer state being changed during an interrupt.
Throughput tests show that RX rate is unchanged by this patch.
The previous timeout value of 150ms could lead to data being lost (ie never
received by the host) in some rare cases, eg when the host is under load.
A value of 500ms is quite conservative and allows the host plenty of time
to read our data.
Hashing of float and complex numbers that are exact (real) integers should
return the same integer hash value as hashing the corresponding integer
value. Eg hash(1), hash(1.0) and hash(1+0j) should all be the same (this
is how Python is specified: if x==y then hash(x)==hash(y)).
This patch implements the simplest way of doing float/complex hashing by
just converting the value to int and returning that value.
Split this setting from MICROPY_CPYTHON_COMPAT. The idea is to be able to
keep MICROPY_CPYTHON_COMPAT disabled, but still pass more of regression
testsuite. In particular, this fixes last failing test in basics/ for
Zephyr port.
The first memmove now copies less bytes in some cases (because len_adj <=
slice_len), and the memcpy is replaced with memmove to support the
possibility that dest and slice regions are overlapping.
The foundation of recv() support is per-socket queue of incoming packets,
implemented using Zephyr FIFO object. This patch implements just recv()
for UDP, because TCP recv() requires much more fine-grained control of
network fragments and handling other issues, like EOF condition, etc.
This follows the pattern of how all other headers are now included, and
makes it explicit where the header file comes from. This patch also
removes -I options from Makefile's that specify the mp-readline/timeutils/
netutils directories, which are no longer needed.
Such constants are MCU specific so shouldn't be specified in the board
config file (else it leads to too much duplication of code).
This patch also adds I2C timing values for the F767/F769 for 100k, 400k
and 1MHz I2C bus frequencies.
By default the SDIO (F4) or SDMMC1 (L4, F7) is used as the SD card
peripheral, but if a board config defines MICROPY_HW_SDMMC2_CK and other
pins then the SD card driver will use SDMMC2.
Build happens in 3 stages:
1. Zephyr config header and make vars are generated from prj.conf.
2. libmicropython is built using them.
3. Zephyr is built and final link happens.
Allows to get event time for a head item in the queue. The usecase
if waiting for the next event *OR* I/O completion. I/O completion may
happen before event triggers, and then wait should continue for the
remaining event time (or I/O completion may schedule another earlier
event altogether).
The new function has a strongly provisional status - it may be converted
to e.g. peek() function returning all of the event fields, not just time.
With the existing timeout of 100ms the transfer would end prematurely if
the baudrate was low and the number of bytes to send was high. This patch
fixes the problem by making the timeout proportional to the number of bytes
that are being transferred.
This patch changes mp_uint_t to size_t for the len argument of the
following public facing C functions:
mp_obj_tuple_get
mp_obj_list_get
mp_obj_get_array
These functions take a pointer to the len argument (to be filled in by the
function) and callers of these functions should update their code so the
type of len is changed to size_t. For ports that don't use nan-boxing
there should be no change in generate code because the size of the type
remains the same (word sized), and in a lot of cases there won't even be a
compiler warning if the type remains as mp_uint_t.
The reason for this change is to standardise on the use of size_t for
variables that count memory (or memory related) sizes/lengths. It helps
builds that use nan-boxing.
With this patch all illegal assignments are reported as "can't assign to
expression". Before the patch there were special cases for a literal on
the LHS, and for augmented assignments (eg +=), but it seems a waste of
bytes (and there are lots of bytes used in error messages) to spend on
distinguishing such errors which a user will rarely encounter.
By removing the 'E' code from the operator token encoding mini-language the
tokenising can be simplified. The 'E' code was only used for the !=
operator which is now handled as a special case; the optimisations for the
general case more than make up for the addition of this single, special
case. Furthermore, the . and ... operators can be handled in the same way
as != which reduces the code size a little further.
This simplification also removes a "goto".
Changes in code size for this patch are (measured in bytes):
bare-arm: -48
minimal x86: -64
unix x86-64: -112
unix nanbox: -64
stmhal: -48
cc3200: -48
esp8266: -76
The self variable may be closed-over in the function, and in that case the
call to super() should load the contents of the closure cell using
LOAD_DEREF (before this patch it would just load the cell directly).
Previous to this patch, if the result of the round function overflowed a
small int, or was inf or nan, then a garbage value was returned. With
this patch the correct big-int is returned if necessary and exceptions are
raised for inf or nan.
The C nearbyint function has exactly the semantics that Python's round()
requires, whereas C's round() requires extra steps to handle rounding of
numbers half way between integers. So using nearbyint reduces code size
and potentially eliminates any source of errors in the handling of half-way
numbers.
Also, bare-metal implementations of nearbyint can be more efficient than
round, so further code size is saved (and efficiency improved).
nearbyint is provided in the C99 standard so it should be available on all
supported platforms.
Previous to this patch, if the result of the trunc/ceil/floor functions
overflowed a small int, or was inf or nan, then a garbage value was
returned. With this patch the correct big-int is returned if necessary,
and exceptions are raised for inf or nan.
It improves readability of code and reduces the chance to make a mistake.
This patch also fixes a bug with nan-boxing builds by rounding up the
calculation of the new NSLOTS variable, giving the correct number of slots
(being 4) even if mp_obj_t is larger than the native machine size.
Instead of having the PlatformToolset property hardcoded to a specific
version just set it to the value of DefaultPlatformToolset: this gets
defined according to the commandline environment in which the build was
started.
Instead of just supporting VS2015 the project can now be built by any
version from VS2013 to VS2017 and normally future versions as well, without
quirks like VS asking whether you want to upgrade the project to the latest
version (as was the case when opening the project in VS2017) or not being
able to build at all (as was the case when opening the project in VS2013).
Also adjust the .gitignore file to ignore any artefacts from VS2017.
The -ansi flag is used for C dialect selection and it is equivalent to -std=c90.
Because it goes right before -std=gnu99 it is ignored as for conflicting flags
GCC always uses the last one.
Now, passing a keyword argument that is not expected will correctly report
that fact. If normal or detailed error messages are enabled then the name
of the unexpected argument will be reported.
This patch decreases the code size of bare-arm and stmhal by 12 bytes, and
cc3200 by 8 bytes. Other ports (minimal, unix, esp8266) remain the same in
code size. For terse error message configuration this is because the new
message is shorter than the old one. For normal (and detailed) error
message configuration this is because the new error message already exists
in py/objnamedtuple.c so there's no extra space in ROM needed for the
string.
GIL behaviour should be handled by the port. And ports probably want to
define sleep_us so that it doesn't release the GIL, to improve timing
accuracy.
We can actually handle interrupts during a thread switch (because we always
have a valid stack), but only if those interrupts don't access any of the
thread state (because the state may not correspond to the stack pointer).
So to be on the safe side we disable interrupts during the very short
period of the thread state+stack switch.
The scheduler being locked general means we are running a scheduled
function, and switching to another thread violates that, so don't switch in
such a case (even though we technically could).
And if we are running a scheduled function then we want to finish it ASAP,
so we shouldn't switch to another thread.
Furthermore, ports with threading enabled will lock the scheduler during a
hard IRQ, and this patch to the VM will make sure that threads are not
switched during a hard IRQ (which would crash the VM).
All arguments to pin.irq are converted from keyword-only to positional, and
can still be specified by keyword so it's a backwards compatible change.
The default value for the "trigger" arg is changed from 0 (no trigger)
to rising+falling edge.
ExtInt, Timer and CAN IRQ callbacks are made to work with the scheduler.
They are still hard IRQs by default, but one can now call
micropython.schedule within the hard IRQ to schedule a soft callback.
Instead of always reporting some object cannot be implicitly be converted
to a 'str', even when it is a 'bytes' object, adjust the logic so that
when trying to convert str to bytes it is shown like that.
This will still report bad implicit conversion from e.g. 'int to bytes'
as 'int to str' but it will not result in the confusing
'can't convert 'str' object to str implicitly' anymore for calls like
b'somestring'.count('a').
* Fix mis-spelling of `ticks_add` in code examples.
* Be consistent about parentheses after function names.
* Be consistent about formatting of function, variable and constant names.
* Be consistent about spaces and punctuation.
* Fix some language errors (missing or wrong words, wrong word order).
* Keep line length under 90 chars.
Signed-off-by: Christopher Arndt <chris@chrisarndt.de>
Instead of caching data that is constant (code_info, const_table and
n_state), store just a pointer to the underlying function object from which
this data can be derived.
This helps reduce stack usage for the case when the mp_code_state_t
structure is stored on the stack, as well as heap usage when it's stored
on the heap.
The downside is that the VM becomes a little more complex because it now
needs to derive the data from the underlying function object. But this
doesn't impact the performance by much (if at all) because most of the
decoding of data is done outside the main opcode loop. Measurements using
pystone show that little to no performance is lost.
This patch also fixes a nasty bug whereby the bytecode can be reclaimed by
the GC during execution. With this patch there is always a pointer to the
function object held by the VM during execution, since it's stored in the
mp_code_state_t structure.
This allows to test the PYBV11 target as well as the network drivers
without adding another rule. It also removes the need to use -B,
side-stepping the issue of whether or not -B works with qstr auto
generation.
When make is passed "-B" it seems that everything is considered out-of-date
and so $? expands to all prerequisites. Thus there is no need for a
special check to see if $? is emtpy.
The 'S' typecode is a uPy extension so it should be grouped with the other
extension (namely 'O' typecode). Testing 'S' needs uctypes which is an
extmod module and not always available, so this test is made optional and
will only be run on ports that have (u)struct and uctypes. Otherwise it
will be silently skipped.
Some stack is allocated to format ints, and when the int implementation uses
long-long there should be additional stack allocated compared with the other
cases. This patch uses the existing "fmt_int_t" type to determine the
amount of stack to allocate.
This patch refactors the error handling in the lexer, to simplify it (ie
reduce code size).
A long time ago, when the lexer/parser/compiler were first written, the
lexer and parser were designed so they didn't use exceptions (ie nlr) to
report errors but rather returned an error code. Over time that has
gradually changed, the parser in particular has more and more ways of
raising exceptions. Also, the lexer never really handled all errors without
raising, eg there were some memory errors which could raise an exception
(and in these rare cases one would get a fatal nlr-not-handled fault).
This patch accepts the fact that the lexer can raise exceptions in some
cases and allows it to raise exceptions to handle all its errors, which are
for the most part just out-of-memory errors during construction of the
lexer. This makes the lexer a bit simpler, and also the persistent code
stuff is simplified.
What this means for users of the lexer is that calls to it must be wrapped
in a nlr handler. But all uses of the lexer already have such an nlr
handler for the parser (and compiler) so that doesn't put any extra burden
on the callers.
For example, if the current directory is the root dir then this patch
allows one to do uos.listdir('mnt'), where 'mnt' is a valid mount point.
Previous to this patch such a thing would not work, on needed to do
uos.listdir('/mnt') instead.
Minimal config can be now build with:
./make-minimal BOARD=...
This is required because of Makefile.exports magic, which in its turn depends
on PROJ_CONF to be set correctly at the beginning of Makefile parsing at all
times. Instead of adding more and more workarounds for that, it's better to
just move minimal support to a separate wrapper.
Also, remove Zephyr 1.5 era cruft from Makefile, and add support for Zephyr's
"run" target which supercedes older "qemu" target in upstream.
This is a typical problem with make: we want to trigger rebuilds only
if file actually changed, not if its timestamp changed. In this case,
it's aggravated by the fact that prj.conf depends on the value of
BOARD variable, so we need to do some tricks anyway. We still don't
try to detect if just BOARD changed, just try to generate new
prj.conf.tmp every time (quick), but do actual replacement of prj.conf
only if its content changed.
This is so that the filename of the test doesn't clash with the module name
itself (being "websocket"), and lead to potential problems executing the
test.
MICROPY_LONGINT_IMPL_LONGLONG doesn't have overflow detection, so just
parsing a large number won't give an error, we need to print it out
to check that the whole number was parsed.
These short unit tests test the base uPy methods as well as parts of the
websocket protocol, as implemented by uPy.
@dpgeorge converted the original socket based tests by @hosaka to ones
that only require io.BytesIO.
This test just tests that the basic functions/methods can be called with
the appropriate arguments. There is no real test of underlying
functionality.
Thanks to @hosaka for the initial implementation of this test.
Mostly intended to ease experimentation, no particular plans for APIs
so far (far less their stability), is_preempt_thread() provided is
mostly an example.
By adding back monotonically increasing field in addition to time field.
As heapsort is not stable, without this, among entried added and readded
at the same time instant, some might be always selected, and some might
never be selected, leading to scheduling starvation.
I.e. they don't run successfully with MICROPY_LONGINT_IMPL_NONE
and MICROPY_LONGINT_IMPL_LONGLONG (the problem is that they generate
different output than CPython, TODO to fix that).
The use of large literal numbers is a big no-no when it comes to writing
programs which work with different int representations. Also, some checks
are pretty adhoc (e.g using struct module to check for 64-bitness). This
change bases entire detection on sys.maxsize and integer operarions, and
thus more correct, even if longer.
Note that this change doesn't mean that any of these tests can pass with
anything but MPZ - even despite checking for various int representations,
the tests aren't written to be portable among them.
INT_MAX used previosly is indeed max value for int, whereas on LP64
platforms, long is used for mp_int_t. Using MP_SMALL_INT_MAX is the
correct way to do it anyway.
Each threads needs to have its own private references to its current
locals/globals dicts, otherwise functions running within different
contexts (eg imported from different files) can behave very strangely.
Tests which don't work with small ints are suffixed with _intbig.py. Some
of these may still work with long long ints and need to be reclassified
later.
There were 2 bugs, now fixed by this patch:
- after deleting an element the len of the dict did not decrease by 1
- after deleting an element searching through the dict could lead to
a seg fault due to there being an MP_OBJ_SENTINEL in the ordered array
The renames are:
HAL_Delay -> mp_hal_delay_ms
sys_tick_udelay -> mp_hal_delay_us
sys_tick_get_microseconds -> mp_hal_ticks_us
And mp_hal_ticks_ms is added to provide the full set of timing functions.
Also, a separate HAL_Delay function is added which differs slightly from
mp_hal_delay_ms and is intended for use only by the ST HAL functions.
User can override PYTHON executable before running script,
gen-cpydiff.py works only with Python3 and most systems register
its executable as 'python3'.
In this case, raise an exception without a message.
This would allow to shove few code bytes comparing to currently used
mp_raise_msg(..., "") pattern. (Actual savings depend on function code
alignment used by a particular platform.)
In MicroPython, the path separator is guaranteed to be "/", extra unneeded
things take precious code space (in the port which doesn't have basic things
like floating-port support).
The parser was originally written to work without raising any exceptions
and instead return an error value to the caller. But it's now required
that a call to the parser be wrapped in an nlr handler, so we may as well
make use of that fact and simplify the parser so that it doesn't need to
keep track of any memory errors that it had. The parser anyway explicitly
raises an exception at the end if there was an error.
This patch simplifies the parser by letting the underlying memory
allocation functions raise an exception if they fail to allocate any
memory. And if there is an error parsing the "<id> = const(<val>)" pattern
then that also raises an exception right away instead of trying to recover
gracefully and then raise.
Previous to this patch any non-interned str/bytes objects would create a
special parse node that held a copy of the str/bytes data. Then in the
compiler this data would be turned into a str/bytes object. This actually
lead to 2 copies of the data, one in the parse node and one in the object.
The parse node's copy of the data would be freed at the end of the compile
stage but nevertheless it meant that the peak memory usage of the
parse/compile stage was higher than it needed to be (by an amount equal to
the number of bytes in all the non-interned str/bytes objects).
This patch changes the behaviour so that str/bytes objects are created
directly in the parser and the object stored in a const-object parse node
(which already exists for bignum, float and complex const objects). This
reduces peak RAM usage of the parse/compile stage, simplifies the parser
and compiler, and reduces code size by about 170 bytes on Thumb2 archs,
and by about 300 bytes on Xtensa archs.
This patch allows uPy consts to be bignums, eg:
X = const(1 << 100)
The infrastructure for consts to be a bignum (rather than restricted to
small integers) has been in place for a while, ever since constant folding
was upgraded to allow bignums. It just required a small change (in this
patch) to enable it.
socket.timeout is a subclass of OSError, and using the latter is more
efficient than having a dedicated class. The argument of OSError is
ETIMEDOUT so the error can be distinguished from other kinds of
OSErrors. This follows how the esp8266 port does it.
Since we recently replaced the OSError string messages with simple error
codes, having the uerrno module gets back some user friendly error
messages. The total code size (after removing strings, replacing with
uerrno module) is decreased.
It's configured by MICROPY_PY_UERRNO_ERRORCODE and enabled by default
(since that's the behaviour before this patch).
Without this dict the lookup of errno codes to strings must use the
uerrno module itself.
ftp.c is the only user of this function so making it static in that file
allows it to be inlined. Also, reusing unichar_toupper means we no longer
depend on the C stdlib for toupper, saving about 300 bytes of code space.
This patch introduces the a small framework to track differences between
uPy and CPython. The framework consists of:
- A set of "tests" which test for an individual feature that differs between
uPy and CPy. Each test is like a normal uPy test in the test suite, but
has a special comment at the start with some meta-data: a category (eg
syntax, core language), a human-readable description of the difference, a
cause, and a workaround. Following the meta-data there is a short code
snippet which demonstrates the difference. See tests/cpydiff directory
for the initial set of tests.
- A program (this patch) which runs all the tests (on uPy and CPy) and
generates nicely-formated .rst documenting the differences.
- Integration into the docs build so that everything is automatic, and the
differences appear in a way that is easy for users to read/reference (see
latter commits).
The idea with using this new framework is:
- When a new difference is found it's easy to write a short test for it,
along with a description, and add it to the existing ones. It's also easy
for contributors to submit tests for differences they find.
- When something is no longer different the tool will give an error and
difference can be removed (or promoted to a proper feature test).
These tests are intended to fail, as they provide a programatic record of
differences between uPy and CPython. They also contain a special comment
at the start of the file which has meta-data describing the difference,
including known causes and known workarounds.
Since VS2015 update 2 .db files are used for storing browsing info,
instead of .sdf files. If users don't specify a location for these files
excplicitly they end up in the project directory so ignore them.
It's much more efficient in RAM and code size to do implicit literal string
concatenation in the lexer, as opposed to the compiler.
RAM usage is reduced because the concatenation can be done right away in the
tokeniser by just accumulating the string/bytes literals into the lexer's
vstr. Prior to this patch adjacent strings/bytes would create a parse tree
(one node per string/bytes) and then in the compiler a whole new chunk of
memory was allocated to store the concatenated string, which used more than
double the memory compared to just accumulating in the lexer.
This patch also significantly reduces code size:
bare-arm: -204
minimal: -204
unix x64: -328
stmhal: -208
esp8266: -284
cc3200: -224
Previous to this patch there was an explicit check for errors with line
continuation (where backslash was not immediately followed by a newline).
But this check is not necessary: if there is an error then the remaining
logic of the tokeniser will reject the backslash and correctly produce a
syntax error.
Since the table of keywords is sorted, we can use strcmp to do the search
and stop part way through the search if the comparison is less-than.
Because all tokens that are names are subject to this search, this
optimisation will improve the overall speed of the lexer when processing
a script.
The change also decreases code size by a little bit because we now use
strcmp instead of the custom str_strn_equal function.
Keywords only needs to be searched for if the token is a MP_TOKEN_NAME, so
we can move the seach to the part of the code that does the tokenising for
MP_TOKEN_NAME.
Grammar rules have 2 variants: ones that are attached to a specific
compile function which is called to compile that grammar node, and ones
that don't have a compile function and are instead just inspected to see
what form they take.
In the compiler there is a table of all grammar rules, with each entry
having a pointer to the associated compile function. Those rules with no
compile function have a null pointer. There are 120 such rules, so that's
120 words of essentially wasted code space.
By grouping together the compile vs no-compile rules we can put all the
no-compile rules at the end of the list of rules, and then we don't need
to store the null pointers. We just have a truncated table and it's
guaranteed that when indexing this table we only index the first half,
the half with populated pointers.
This patch implements such a grouping by having a specific macro for the
compile vs no-compile grammar rules (DEF_RULE vs DEF_RULE_NC). It saves
around 460 bytes of code on 32-bit archs.
Allows to iterate over the following without allocating on the heap:
- tuple
- list
- string, bytes
- bytearray, array
- dict (not dict.keys, dict.values, dict.items)
- set, frozenset
Allows to call the following without heap memory:
- all, any, min, max, sum
TODO: still need to allocate stack memory in bytecode for iter_buf.
This patch changes the threading implementation from simple round-robin
with busy waits on mutexs, to proper scheduling whereby threads that are
waiting on a mutex are only scheduled when the mutex becomes available.
This improves efficiency of GIL release within the VM, by only doing the
release after a fixed number of jump-opcodes have executed in the current
thread.
It's more efficient using the system mutexs instead of synthetic ones with
a busy-wait loop. The system can do proper scheduling and blocking of the
threads waiting on the mutex.
Depending on the thread scheduler, a busy-wait loop can hog the CPU and
make the tests very slow. So convert such loops to loops that have an
explicit sleep, allowing the worker threads to do their job.
This allows using the test runner for other scenarios than just
testing uPy itself.
The principle of comparing either to CPython or else to a .exp
file is really handy but to be able to test custom modules not
built into micropython.exe one needs to be able to specify the
module search path a.k.a MICROPYPATH.
MicroPython guarantees '/' to be a path separator, so extra constant taking
precious ROM space are not needed. MicroPython never had such constant, only
one vendor port had it (now unmaintained).
Just one sample is updated with on()/off() for now, there should be
remaining sample(s) showing .value() use (but more can be converted later,
as long as 1 or so good samples of .value() remains).
ipoll() allows to poll streams without allocating any memory: this method
returns an iterator (a poll object itself), and the iterator yields
preallocated "callee-owned tuple" with polling results for each active
stream. The only operation a caller is allowed to do with this tuple is
extracting values from it (storing the tuple as a whole somewhere is
not allowed).
Previous to this patch, for large chunks of bytecode that originated from
a single source-code line, the bytecode-line mapping would generate
something like (for 42 bytecode bytes and 1 line):
BC_SKIP=31 LINE_SKIP=1
BC_SKIP=11 LINE_SKIP=0
This would mean that any errors in the last 11 bytecode bytes would be
reported on the following line. This patch fixes it to generate instead:
BC_SKIP=31 LINE_SKIP=0
BC_SKIP=11 LINE_SKIP=1
This patch implements support for class methods __delattr__ and __setattr__
for customising attribute access. It is controlled by the config option
MICROPY_PY_DELATTR_SETATTR and is disabled by default.
If the mounted object doesn't have a "mount" method then assume it's a
block device and try to detect the filesystem. Since we currently only
support FAT filesystems, the behaviour is to just try and create a VfsFat
object automatically, using the given block device.
Each method asserts and deasserts signal respectively. They are equivalent
to .value(1) and .value(0) but conceptually simpler (and may help to avoid
confusion with inverted signals, where "asserted" state means logical 0
output).
The aapcs-linux ABI is not required, instead the default aapcs ABI is
enough. And using the default ABI means that the provided libgcc will now
link with the firmware without warnings about variable vs fixed enums.
Although the binary size increases by about 1k, RAM usage is slightly
decreased. And libgcc may prove useful in the future for things like
long-long division.
It seems that the gcc toolchain on the RaspberryPi
likes %progbits instead of @progbits. I verified that
%progbits also works under x86, so this should
fix#2848 and fix#2842
I verified that unix and mpy-cross both compile
on my RaspberryPi and on my x64 machine.
The internal map/set functions now use size_t exclusively for computing
addresses. size_t is enough to reach all of available memory when
computing addresses so is the right type to use. In particular, for
nanbox builds it saves quite a bit of code size and RAM compared to the
original use of mp_uint_t (which is 64-bits on nanbox builds).
For archs that have 16-bit pointers, the asmxtensa.h file can give compiler
warnings about left-shift being greater than the width of the type (due to
the inline functions in this header file). Explicitly casting the
constants to uint32_t stops these warnings.
SPI needs to be fast, and calling the EVENT_POLL_HOOK every byte makes it
unusable for ports that need to do non-trivial work in the EVENT_POLL_HOOK
call. And individual SPI transfers should be short enough in time that
EVENT_POLL_HOOK doesn't need to be called.
If something like this proves to be needed in practice then we will need
to introduce separate event hook macros, one for "slow" loops (eg
select/poll) and one for "fast" loops (eg software I2C, SPI).
A few tests still fail on PYBLITE, and that's due to differences in the
available peripheral block numbers on the different MCUs (eg I2C(2)
exists on one, but it's I2C(3) on the other).
This new function controls what happens on a hard-fault:
- debugging disabled: board will do a reset
- debugging enabled: board will print registers and stack and flash LEDs
The default is disabled, ie to do a reset. This is different to previous
behaviour which flashed the LEDs and waited indefinitely.
The port now uses the common mp_utime_ticks_{ms,us,cpu,add,diff} functions
from extmod/utime_mphal.c.
The mp_utime_sleep_XXX functions are still cc3200-specific because they
handle the GIL differently to the ones in extmod.
The files misc/mpsystick.[ch] have been removed because they contain 2
unused functions, and the other remaining function is renamed to
mp_hal_ticks_us and moved to hal/cc3200_hal.c.
machine.time_pulse_us() is intended to provide very fine timing, including
while working with signal bursts, where each transition is tracked in row.
Throwing and handling an exception may take too much time and "signal loss".
So instead, in case of a timeout, just return negative value. Cases of
timeout while waiting for initial signal stabilization, and during actual
timing, are recognized.
The documentation is updated accordingly, and rewritten somewhat to clarify
the function behavior.
This patch fixes two main things:
- dicts can be printed directly using '%s' % dict
- %-formatting should not crash when passed a non-dict to, eg, '%(foo)s'
Updated modbuiltin.c to add conditional support for 3-arg calls to
pow() using MICROPY_PY_BUILTINS_POW3 config parameter. Added support in
objint_mpz.c for for optimised implementation.
This patch brings the _thread module to stmhal/pyboard. There is a very
simple round-robin thread scheduler, which is disabled if there is only
one thread (for efficiency when threading is not used).
The scheduler currently switches threads at a rate of 250Hz using the
systick timer and the pend-SV interrupt.
The GIL is disabled so one must be careful to use lock objects to prevent
concurrent access of objects.
The threading is disabled by default, one can enabled it with the config
option MICROPY_PY_THREAD to test it out.
The first partition is mounted as "/sd" and subsequent partitions are
mounted as "/sd<part_num>". This is backwards compatible with the previous
behaviour, which just mounted the first partition on "/sd".
At this point, only FatFs filesystems are mounted.
A signal is like a pin, but ca also be inverted (active low). As such, it
abstracts properties of various physical devices, like LEDs, buttons,
relays, buzzers, etc. To instantiate a Signal:
pin = machine.Pin(...)
signal = machine.Signal(pin, inverted=True)
signal has the same .value() and __call__() methods as a pin.
Based on the following statistics:
$ git log docs |grep Author | sort | uniq -c | sort -n -r
175 Author: Paul Sokolovsky
135 Author: Damien George
31 Author: Daniel Campora
26 Author: danicampora
14 Author: Peter Hinch
git blame stats script from http://stackoverflow.com/a/13687302/496009:
$ sh git-authors docs
9977 author Damien George
2679 author Paul Sokolovsky
1699 author Daniel Campora
1580 author danicampora
1286 author Peter Hinch
282 author Shuning Bian
249 author Dave Hylands
Total lines per this script: 18417, my contribution is 14.5%.
This patch makes the following configuration changes:
- MICROPY_FSUSERMOUNT is disabled, removing old mounting infrastructure
- MICROPY_VFS is enabled, giving new VFS sub-system
- MICROPY_VFS_FAT is enabled, giving uos.VfsFat type
- MICROPY_FATFS_OO is enabled, to use new ooFatFs lib, R0.12b
User facing API should be almost unchanged. Most notable changes are
removal of os.mkfs (use os.VfsFat.mkfs instead) and pyb.mount doesn't
allow unmounting by passing None as the device.
This provides mp_vfs_XXX functions (eg mount, open, listdir) which are
agnostic to the underlying filesystem type, and just require an object with
the relevant filesystem-like methods (eg .mount, .open, .listidr) which can
then be mounted.
These mp_vfs_XXX functions would typically be used by a port to implement
the "uos" module, and mp_vfs_open would be the builtin open function.
This feature is controlled by MICROPY_VFS, disabled by default.
If MICROPY_VFS_FAT is enabled by a port then the port must switch to using
MICROPY_FATFS_OO. Otherwise a port can continue to use the FatFs code
without any changes.
Previous to this patch the qemu-arm tests were compiled with is_relp=true
meaning that the __repl_print__ function was called for all lines of code
in the outer scope. This is not the right behaviour for scripts that are
executed as though they were a file (eg tests).
With this fix the micropython/heapalloc_str.py test now works so it is
removed from the test blacklist.
In this, don't allocate copy, just return non-empty string. This helps
with a standard pattern of buffering data in case of short reads:
buf = b""
while ...:
s = f.read(...)
buf += s
...
For a typical case when single read returns all data needed, there won't
be extra allocation. This optimization helps uasyncio.
They are one-line functions and having them inline in mp_init/mp_deinit
eliminates the overhead of a function call, and matches how other state
is initialised in mp_init.
The order now follows that in py/mpconfig.h and is a bit cleaner and easier
to maintain. No options were changed/added/removed with this patch, it's
just a reordering.
To use this feature a port should define MICROPY_HW_SPIFLASH_SIZE_BITS
along with x_CS, x_SCK, x_MOSI, x_MISO (x=MICROPY_HW_SPIFLASH). This will
then use external SPI flash on those pins instead of the internal flash.
The SPI is done using the software implementation. There is currently only
support for standard SPI (ie not dual or quad mode).
stmhal will now be built by default with frozen bytecode from scripts
stored in the stmhal/modules/ directory. This can be disabled or
changed to another directory by overridding the make variable
FROZEN_MPY_DIR.
This is how CPython does it, and it's very useful to help users discover
the available modules for a given port, especially built-in and frozen
modules. The function does not list modules that are in the filesystem
because this would require a fair bit of work to do correctly, and is very
port specific (depending on the filesystem).
Ports should no longer use pyhelp_print_obj but instead should define
MICROPY_PY_BUILTINS_HELP to 1 and then specify their help text using
MICROPY_PY_BUILTINS_HELP_TEXT.
If result guaranteedly fits in a small int, it is handled in objint.c.
Otherwise, it is delegated to mp_obj_int_from_bytes_impl(), which should
be implemented by individual objint_*.c, similar to
mp_obj_int_to_bytes_impl().
If GeneratorExit is injected as a throw-value then that should lead to
the close() method being called, if it exists. If close() does not exist
then throw() should not be called, and this patch fixes this.
CPython 3.6 has a few changes that, when run on uPy's test suite, give a
different output to CPython 3.5. uPy currently officially supports the
3.4 language definition, but it's useful to be able to run the test suite
with 3.4/3.5/3.6 versions of CPython. This patch makes such changes to
support 3.6.
The commit d9047d3c8a introduced a bug
whereby "from a.b import c" stopped working for frozen packages. This is
because the path was not properly truncated and became "a//b". Such a
path resolves correctly for a "real" filesystem, but not for a search in
the list of frozen modules.
This effectively reverts the change that introduced this new constant.
The reason is so that users do not need to rebuild the filesystem on
their modules when upgrading the firmware.
Users can change RESERVED_SECS by hand if they need the feature, and in
future firmware it may default to a non-zero value.
This code is no longer pertinent for some time - since switchover to
SDK2.0, there must be correct flash size set for bootloader, or there's
a risk of flash data corruption. And indeed, the correct flash size is
by default auto-detected by esptool.py 1.2.
With caching of map lookups in the bytecode, frozen bytecode can still
work but must be stored in RAM, not ROM. This patch allows mpy-tool.py to
generate code that works with this optimisation, but it's not recommended
to use it on embedded targets (because of lack of RAM).
If sets are not enabled, set literals lead to SyntaxError during parsing,
so it requires feature_check. Set tests are skipped based on set_*.py
pattern.
Starting at esp.flash_user_start(), the reserved sectors are for general
purpose use, for example for native code generation. There is currently
one sector reserved as such.
The driver seems to be be enabling the pullup resistor in most places, but
not this one. Making this one little change allows onewire devices to be
used with no external pullup resistor.
This makes unix "uselect" compatible with baremetal "uselect". Previosuly,
unix version accepted file/socket objects, but internally converted that
to file descriptors, and that's what .poll() returned. To acheive new
behavior, file-like objects are stored internally in an array, in addition
to existing array of struct pollfd. This array is created only on first
case of file-like object being passed to .register(). If only raw fd's are
passed, there will be no additional memory used comparing to the original
implementation.
cc3200tool, https://github.com/ALLTERCO/cc3200tool is a (mostly, some
binary blobs present) open-source, Linux-friendly tool to flash a cc3200
devices. It's an alternative to fully proprietary, Windows-only Uniflash
from TI.
The provided make targets are for erasing flash, flashing the uPy
bootloader and firmware, and flashing vendor's WiFi firmware "servicepacks"
(the latter needs to be downloaded from vendor side, a link is present
inside Makefile).
Sys-tick resolution is 1ms and a value of 2 will give a delay between 1ms
and 2ms (whereas a value of 1 gives a delay between 0ms and 1ms, which is
too short).
The HAL_UART_Transmit function has changed in the latest HAL version such
that the Timeout is a timeout for the entire function, rather than a
timeout between characters as it was before. The HAL function also does
not allow one to reliably tell how many characters were sent before the
timeout (if a timeout occurred).
This patch provides a custom function to do UART transmission, completely
replacing the HAL version, to fix the above-mentioned issues.
UART REPL support was lost in os.dupterm() refactorings, etc. As
os.dupterm() is there, implement UART REPL support at the high level -
if MICROPY_STDIO_UART is set, make default boot.py contain os.dupterm()
call for a UART. This means that changing MICROPY_STDIO_UART value will
also require erasing flash on a module to force boot.py re-creation.
There is a minor functional change with this patch, that the GPIO are now
configured in fast mode, whereas they were in high speed mode before. But
the SDIO should still work because SD CK frequency is at most 25MHz.
They are the same as the existing raw constants (namely 0, 1, 2) but we
want to explicitly show that one can use the HAL's constants if necessary
(eg the mpconfigboard.h files do use the HAL's constants to define the
pull state of certain configurable pins).
Without this the timer will have random values for its State and Lock
entries. The object can then be in a locked state leading to some HAL
functions returning immediately with an error code (which is unchecked).
This patch fixes such a bug which did manifest itself as PWM not working
correctly for LEDs.
This check always fails (ie chr0 is never EOF) because the callers of this
function never call it past the end of the input stream. And even if they
did it would be harmless because 1) reader.readbyte must continue to
return an EOF char if the stream is exhausted; 2) next_char would just
count the subsequent EOF's as characters worth 1 column.
import utimeq, utime
# Max queue size, the queue allocated statically on creation
q = utimeq.utimeq(10)
q.push(utime.ticks_ms(), data1, data2)
res = [0, 0, 0]
# Items in res are filled up with results
q.pop(res)
This is required to avoid extra level of output "cooking" ("\r\r\n") and
make test infrastructure work. On the other hand, this breaks somewhat
Zephyr console abstraction.
Defining and initialising mp_kbd_exception is boiler-plate code and so the
core runtime can provide it, instead of each port needing to do it
themselves.
The exception object is placed in the VM state rather than on the heap.
mp_kbd_exception is now considered the standard variable name to hold the
singleton KeyboardInterrupt exception.
This patch also moves the creation of this object from pyb_usb_init() to
main().
Previous to this patch pyboard.py would open a new serial connection to
the target for each script that was run, and for any command that was run.
Apart from being inefficient, this meant that the board was soft-reset
between scripts/commands, which precludes scripts from accessing variables
set in a previous one.
This patch changes the behaviour of pyboard.py so that the connection to
the target is created only once, and it's not reset between scripts or any
command that is sent with the -c option.
When printing exceptions from files sent to a target by pyboard.py the
filename in the exception is <stdin>, which differs to when running the
script on the PC. So we strip out the filename to make the outputs the
same on all targets (see also misc/print_exception.py test).
sys.exit() is an important function to terminate a program. In particular,
the testsuite relies on it to skip tests (i.e. any other functionality may
be disabled, but sys.exit() is required to at least report that properly).
For all but the last pass the assembler only needs to count how much space
is needed for the machine code, it doesn't actually need to emit anything.
The dummy_data just uses unnecessary RAM and without it the code is not
any more complex (and code size does not increase for Thumb and Xtensa
archs).
This patch moves some common code from the individual inline assemblers to
the compiler, the code that calls the emit-glue to assign the machine code
to the functions scope.
This patch adds the MICROPY_EMIT_INLINE_XTENSA option, which, when
enabled, allows the @micropython.asm_xtensa decorator to be used.
The following opcodes are currently supported (ax is a register, a0-a15):
ret_n()
callx0(ax)
j(label)
jx(ax)
beqz(ax, label)
bnez(ax, label)
mov(ax, ay)
movi(ax, imm) # imm can be full 32-bit, uses l32r if needed
and_(ax, ay, az)
or_(ax, ay, az)
xor(ax, ay, az)
add(ax, ay, az)
sub(ax, ay, az)
mull(ax, ay, az)
l8ui(ax, ay, imm)
l16ui(ax, ay, imm)
l32i(ax, ay, imm)
s8i(ax, ay, imm)
s16i(ax, ay, imm)
s32i(ax, ay, imm)
l16si(ax, ay, imm)
addi(ax, ay, imm)
ball(ax, ay, label)
bany(ax, ay, label)
bbc(ax, ay, label)
bbs(ax, ay, label)
beq(ax, ay, label)
bge(ax, ay, label)
bgeu(ax, ay, label)
blt(ax, ay, label)
bnall(ax, ay, label)
bne(ax, ay, label)
bnone(ax, ay, label)
Upon entry to the assembly function the registers a0, a12, a13, a14 are
pushed to the stack and the stack pointer (a1) decreased by 16. Upon
exit, these registers and the stack pointer are restored, and ret.n is
executed to return to the caller (caller address is in a0).
Note that the ABI for the Xtensa emitters is non-windowing.
This patch allows esp8266 to use @micropython.native and
@micropython.viper function decorators. By default the executable machine
code is written to the space at the end of the iram1 region. The user can
call esp.set_native_code_location() to make the code go to flash instead.
If a port defines MP_PLAT_COMMIT_EXEC then this function is used to turn
RAM data into executable code. For example a port may want to write the
data to flash for execution. The function must return a pointer to the
executable data.
The 512k build recently overflowed because of the newly-enabled uselect
module. uselect is arguable more important than framebuf for small
devices so we disable framebuf to keep the 512k build within its limit.
This is a pure refactoring (and simplification) of code so that stmhal
uses the software SPI class provided in extmod, for the machine.SPI
implementation.
So long as a port defines relevant mp_hal_pin_xxx functions (and delay) it
can make use of this software SPI class without the need for additional
code.
Previous to this patch trying to construct, but not init, a UART that
didn't exist on the target board would actually succeed. Only when
initialising the UART would it then raise an exception that the UART does
not exist.
This patch adds an explicit check that the constructed UART does in fact
exist for the given board.
This follows the pattern of other peripherals (I2C, SPI) to specify the
pins using pin objects instead of a pair of GPIO port and pin number. It
makes it easier to customise the UART pins for a particular board.
These are basic drawing primitives. They work in a generic way on all
framebuf formats by calling the underlying setpixel or fill_rect C-level
primitives.
The STM32 F7 and L4 boards use significantly different code to the F4
boards so it's important to test them with CI. To keep CI build times
within a reasonable limit the STM32F4DISC board is no longer built, it's
anyway very similar to the standard F4 build for PYBv1.0.
If you have longish operations on the db (such as logging data) it may
be desirable to periodically sync the database to the disk. The added
btree.sync() method merely exposes the berkley __bt_sync function to the
user.
The constants MP_IOCTL_POLL_xxx, which were stmhal-specific, are moved
from stmhal/pybioctl.h (now deleted) to py/stream.h. And they are renamed
to MP_STREAM_POLL_xxx to be consistent with other such constants.
All uses of these constants have been updated.
If the destination of os.rename() exists then it will be overwritten if it
is a file. This is the POSIX behaviour, which is also the CPython
behaviour, and so we follow suit.
See issue #2598 for discussion.
Add 2 macros in mphalport.h that clean and invalidate data caches only on
STM32F7 MCUs. They are needed to ensure the cache coherency before/after
DMA transferts.
* MP_HAL_CLEANINVALIDATE_DCACHE cleans and invalidate the data cache. It
must be called before starting a DMA transfer from the peripheral to the
RAM memory.
* MP_HAL_CLEAN_DCACHE cleans the data cache. It must be called before
starting a DMA transfert from the RAM memory to the peripheral.
These macros are called in sdcard.c, before reading from and writing to
the SDCard, when DMA is used.
Fill is a very common operation (eg to clear the screen) and it is worth
optimising it, by providing a specialised fill_rect function for each
framebuffer format.
This patch improved the speed of fill by 10 times for a 16-bit display
with 160*128 pixels.
Rename FrameBuffer1 into FrameBuffer and make it handle different bit
depths via a method table that has getpixel and setpixel. Currently
supported formats are MVLSB (monochrome, vertical, LSB) and RGB565.
Also add blit() and fill_rect() methods.
The temperature sensor on F4 and F7 MCUs is mostly, but not always, on
channel 16. To retain compatibility across all these MCUs this patch
maps the user-facing channel 16 to the internal temperature sensor.
The "mask" parameter is used to select which pins the ADCAll constructor
will initialise to analog mode. It defaults to all pins (0xffffffff),
which is backwards compatible with previous behaviour.
These were inadvertently removed with a recent upgrade to CMSIS, where
those registers were no longer defined in the CMSIS headers, and hence
no longer extracted.
This allows one to construct an I2C object using ids that are specific
to the stmhal port, eg machine.I2C('X'). Right now the implementation
of I2C uses software I2C but the idea is to just change the C-level I2C
protocol functions to hardware implementations later on.
stdio.h was included in all HAL files only to provide
definition of NULL symbol
"stdio.h" includes "types.h" which contains some conflicting definitions
with "drivers/cc3000/inc/socket.h"
HAL Driver before v1.4.2 had a bug which caused clearing all pending
flags in MSR, TSR, RF0R and RF1R instead of only the requested one.
This is why micropython got away without explicitly clearing flags
in IRQ handler.
Current version of HAL drivers optimize IRQ handler by using precalculated
DMA register address and stream bitshift instead of calculating it on every interrupt.
Since we skip call to `HAL_DMA_Init` on reused DMA, fields StreamBaseAddress and StreamIndex
of DMA handle are not initialized and thus leads to SegFault in `DMA_IRQHandler`.
HAL_DMA_Init is a big routine and we do not need to call it on each use of DMA
(ex.: series of I2C operations) and DMA_CalcBaseAndBitshift is really small and
releasing it increases code size by only 8 bytes.
If a port defines MICROPY_READER_POSIX or MICROPY_READER_FATFS then
lexer.c now provides an implementation of mp_lexer_new_from_file using
the mp_reader_new_file function.
Implementations of persistent-code reader are provided for POSIX systems
and systems using FatFS. Macros to use these are MICROPY_READER_POSIX and
MICROPY_READER_FATFS respectively. If an alternative implementation is
needed then a port can define the function mp_reader_new_file.
It is split into 2 functions, one to make small ints and the other to make
a non-small-int leaf node. This reduces code size by 32 bytes on
bare-arm, 64 bytes on unix (x64-64) and 144 bytes on stmhal.
Per the latest HW API, "SPI" class implements only master side of the
protocol, so mode=SPI.MASTER (which was static for WiPy anyway) is not
required (or allowed). This change is required to correspond to updated
documentation of machine.SPI class which no longer lists "mode".
This includes StopIteration and thus are important to make Python-coded
iterables work with yield from/await.
Exceptions in Python send() are still not handled and left for future
consideration and optimization.
We allow 'exc.__traceback__ = None' assignment as a low-level optimization
of pre-allocating exception instance and raising it repeatedly - this
avoids memory allocation during raise. However, uPy will keep adding
traceback entries to such exception instance, so before throwing it,
traceback should be cleared like above.
'exc.__traceback__ = None' syntax is CPython compatible. However, unlike
it, reading that attribute or setting it to any other value is not
supported (and not intended to be supported, again, the only reason for
adding this feature is to allow zero-memalloc exception raising).
Its addition was due to an early exploration on how to add CPython-like
stream interface. It's clear that it's not needed and just takes up
bytes in all ports.
As required for further elaboration of uasyncio, like supporting baremetal
systems with wraparound timesources. This is not intended to be public
interface, and likely will be further refactored in the future.
If an I2C send/recv fails then the peripheral is now checked to see if
it's in a "stuck" state waiting for the stop bit, and if so then it is
reset so that the next I2C transaction can proceed.
This patch also de-inits the I2C peripheral in the init() method, before
init'ing it again.
New keyword option in constructor and init() method is "dma=<bool>".
DMA is now disabled by default for I2C transfers because it currently does
not handle I2C bus errors very well (eg if slave device doesn't ACK or
NACK correctly during a transfer).
The DMA state is always HAL_DMA_STATE_RESET because of the memset clearing
all the data, so prior to this patch HAL_DMA_DeInit was never called. Now
it is always called to make sure the DMA is properly reset.
The ESP SDK supports configuring the hostname that is
reported when doing a DHCP request in station mode. This commit
exposes that under network.WLAN(network.STA_IF).config('dhcp_hostname')
as a read/write value similar to other parameters.
With this patch one can now do "make FROZEN_MPY_DIR=../../frozen" to
specify a directory containing scripts to be frozen (as well as absolute
paths).
The compiled .mpy files are now stored in $(BUILD)/frozen_mpy/.
Now, to use frozen bytecode all a port needs to do is define
FROZEN_MPY_DIR to the directory containing the .py files to freeze, and
define MICROPY_MODULE_FROZEN_MPY and MICROPY_QSTR_EXTRA_POOL.
Previously, it was included only in release builds, but it's important
tool which should be always at the fingertips to be useful (and to
pump up its usage).
To make its inclusion as frozen modules in multiple ports less magic.
Ports are just expected to symlink 2 files into their scripts/modules
subdirs.
Unix port updated to use this and in general follow frozen modules setup
tested and tried on baremetal ports, where there's "scripts" predefined
dir (overridable with FROZEN_DIR make var), and a user just drops Python
files there.
At the WS2812 driver level, a 400ns value was used for T0H (time high to
send a 0 bit) but LED specification says it should be 350ns +- 150ns.
Due to loop overhead the 400ns value could lead to T0H close to 500ns
which is too close from the limit value and gave glitches (bad data to
pixels) in some cases. This patch makes the calculated T0H value 350ns.
Previously they used historical "pyb" affix causing confusion and
inconsistency (there's no "pyb" module in modern ports; but people
took esp8266 port as an example, and "pyb" naming kept proliferating,
while other people complained that source structure is not clear).
This helps to test floating point code on Cortex-M hardware.
As part of this patch the link-time-optimisation was disabled because it
wasn't compatible with software FP support. In particular, the linker
could not find the __aeabi_f2d, __aeabi_d2f etc functions even though they
were provided by lib/libm/math.c.
In both parse.c and qstr.c, an internal chunking allocator tidies up
by calling m_renew to shrink an allocated chunk to the size used, and
assumes that the chunk will not move. However, when MICROPY_ENABLE_GC
is false, m_renew calls the system realloc, which does not guarantee
this behaviour. Environments where realloc may return a different
pointer include:
(1) mbed-os with MBED_HEAP_STATS_ENABLED (which adds a wrapper around
malloc & friends; this is where I was hit by the bug);
(2) valgrind on linux (how I diagnosed it).
The fix is to call m_renew_maybe with allow_move=false.
Size 64 was incorrect and will lead to stack corruption. Size 88 was
verified empirically. Also, allow to skip defining it if MD5_CTX
preprocessor macro is already defined (to avoid header conflict).
ESP8266 SDK2.0 fixes (at least, I can't reproduce it) an infamous bug
with crash during scan. 36K seams to be a safe value based on a download
test (test_dl.py), over 1GB was downloaded. More testing is needed, but
let's have other people participate by committing it now.
There is no automatic reconnect after wlan.active(False);
wlan.active(True). This commit provide the possibility to run
wlan.connect() without parameter, to reconnect to the previously
connected AP.
resolve#2493
It's mandatory function which should be present in every port. Even if
it's not, in the stdlib intro we waarn users that a particular port can
lack anything of described in the docs.
Now the function properly uses ring arithmetic to return signed value
in range (inclusive):
[-MICROPY_PY_UTIME_TICKS_PERIOD/2, MICROPY_PY_UTIME_TICKS_PERIOD/2-1].
That means that function can properly process 2 time values away from
each other within MICROPY_PY_UTIME_TICKS_PERIOD/2 ticks, but away in
both directions. For example, if tick value 'a' predates tick value 'b',
ticks_diff(a, b) will return negative value, and positive value otherwise.
But at positive value of MICROPY_PY_UTIME_TICKS_PERIOD/2-1, the result
of the function will wrap around to negative -MICROPY_PY_UTIME_TICKS_PERIOD/2,
in other words, if a follows b in more than MICROPY_PY_UTIME_TICKS_PERIOD/2 - 1
ticks, the function will "consider" a to actually predate b.
It's implemented in terms of usleep(), and POSIX doesn't guarantee that
usleep() can sleep for more than a second. This restriction unlikely
applies to any real-world system, but...
Based on the earlier discussed RFC. Practice showed that the most natural
order for arguments corresponds to mathematical subtraction:
ticks_diff(x, y) <=> x - y
Also, practice showed that in real life, it's hard to order events by time
of occurance a priori, events tend to miss deadlines, etc. and the expected
order breaks. And then there's a need to detect such cases. And ticks_diff
can be used exactly for this purpose, if it returns a signed, instead of
unsigned, value. E.g. if x is scheduled time for event, and y is the current
time, then if ticks_diff(x, y) < 0 then event has missed a deadline (and e.g.
needs to executed ASAP or skipped). Returning in this case a large unsigned
number (like ticks_diff behaved previously) doesn't make sense, and such
"large unsigned number" can't be reliably detected per our definition of
ticks_* function (we don't expose to user level maximum value, it can be
anything, relatively small or relatively large).
The integration with Zephyr is fairly clean but as MicroPython Hardware API
requires pin ID to be a single value, but Zephyr operates GPIO in terms of
ports and pins, not just pins, a "hierarchical" ID is required, using tuple
of (port, pin). Port is a string, effectively a device name of a GPIO port,
per Zephyr conventions these are "GPIO_0", "GPIO_1", etc.; pin is integer
number of pin with the port (supposed to be in range 0-31).
Example of pin initialization:
pin = Pin(("GPIO_1", 21), Pin.OUT)
(an LED on FRDM-K64F's Port B, Pin 21).
There is support for in/out pins and pull up/pull down but currently
there is no interrupt support.
Signed-off-by: Daniel Thompson <daniel.thompson@linaro.org>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@linaro.org>
Signed-off-by: Paul Sokolovsky <paul.sokolovsky@linaro.org>
This happens with some compilers on some architectures, which don't define
size_t as unsigned int. MicroPython's printf() dooesn't support obscure
format specifiers for size_t, so the obvious choice is to explicitly cast
to unsigned, to match %u used in printf().
Clarify the class implements master side of the protocol, also put adhoc
WiPy paramter after the generic, described in the current Hardware API
version.
The NeoPixel class now handles 4 bytes-per-pixel LEDs (extra byte is
intensity) and arbitrary byte ordering. APA102 class is now derived from
NeoPixel to reduce code size and support fill() operation.
To build, "make 512k".
Disabled are FatFs support (no space for filesystem), Python functionality
related to files, btree module, and recently enabled features. With all
this, there's only one free FlashROM page.
As we're looking towards adding OTA support, calculation of a FlashROM
area which can be used for filesystem (etc.) may become complex, so
introduce C function for that. So far it just hardcodes current value,
0x90000. In the future the function may be extended (and renamed) to
return the size of area too.
This provides time and sleep together with the usual ticks_us/_ms/_diff
and sleep_us/ms family.
We also provide access to Zephyr's high precision timer as ticks_cpu().
Signed-off-by: Daniel Thompson <daniel.thompson@linaro.org>
Builtin functions with a fixed number of arguments (0, 1, 2 or 3) are
quite common. Before this patch the wrapper for such a function cost
3 machine words. After this patch it only takes 2, which can reduce the
code size by quite a bit (and pays off even more, the more functions are
added). It also makes function dispatch slightly more efficient in CPU
usage, and furthermore reduces stack usage for these cases. On x86 and
Thumb archs the dispatch functions are now tail-call optimised by the
compiler.
The bare-arm port has its code size increase by 76 bytes, but stmhal drops
by 904 bytes. Stack usage by these builtin functions is decreased by 48
bytes on Thumb2 archs.
In order to have more fine-grained control over how builtin functions are
constructed, the MP_DECLARE_CONST_FUN_OBJ macros are made more specific,
with suffix of _0, _1, _2, _3, _VAR, _VAR_BETEEN or _KW. These names now
match the MP_DEFINE_CONST_FUN_OBJ macros.
Running Python code on a hard interrupt is incompatible with having a GIL,
because most of the time the GIL will be held by the user thread when the
interrupt arrives. Hard interrupts mean that we should process them right
away and hence can't wait until the GIL is released.
The problem with the current code is that a hard interrupt will try to
exit/enter the GIL while it is still held by the user thread, hence leading
to a deadlock.
This patch works around such a problem by just making GIL exit/enter a
no-op when in an interrupt context, or when interrupts are disabled.
See issue #2406.
SDK 2.0.0 goes into boot loop if a firmware is programmed over erased flash,
causing problems with user experience. This change implements behavior
similar to older SDKs': if clean flash is detected, default system
parameters are used.
As long as a port implement mp_hal_sleep_ms(), mp_hal_ticks_ms(), etc.
functions, it can just use standard implementations of utime.sleel_ms(),
utime.ticks_ms(), etc. Python-level functions.
Now there is just one function to allocate a new vstr, namely vstr_new
(in addition to vstr_init etc). The caller of this function should know
what initial size to allocate for the buffer, or at least have some policy
or config option, instead of leaving it to a default (as it was before).
This refactors ujson.loads(s) to behave as ujson.load(StringIO(s)).
Increase in code size is: 366 bytes for unix x86-64, 180 bytes for
stmhal, 84 bytes for esp8266.
The boot issue text mentions a help() function and encourages
the user to run it. It is very disconcerting to find that the
function does not exist...
Signed-off-by: Daniel Thompson <daniel.thompson@linaro.org>
"Forced exit" is treated as soft-reboot (Ctrl+D). But expected effect of
calling sys.exit() is termination of the current script, not any further
and more serious actions like mentioned soft reboot.
Setting emit_dent=0 is unnecessary because arriving in that part of the
if-logic will guarantee that emit_dent is already zero.
The block to check indent_top(lex)>0 is unreachable because a newline is
always inserted an the end of the input stream, and hence dedents are
always processed before EOF.
As per discussion in #2449, using write requests instead of read requests
for I2C.scan() seems to support a larger number of devices, especially
ones that are write-only. Even a read-only I2C device has to implement
writes in order to be able to receive the address of the register to read.
Adds check that LZ offsets fall into the sliding dictionary used. This
catches a case when uzlib.DecompIO with a smaller dictionary is used
to decompress data which was compressed with a larger dictionary.
Previously, this would lead to producing invalid data or crash, now
an exception will be thrown.
The outputexpors target, which exports Zephyr environment variables, was
recently added to Zephyr. By exploiting this feature we can hugely simplify
the build system, improving robustness at the same time.
Signed-off-by: Daniel Thompson <daniel.thompson@linaro.org>
The two variables, GENERIC_TARGETS and CONFIG_TARGETS come, respectively,
from the the lists shown during "make help" and "make kconfig-help".
Signed-off-by: Daniel Thompson <daniel.thompson@linaro.org>
Currently to compile for anything that except ARCH=x86 we have to
provide ARCH via the environment or make arguments. We can do better
than that!
Signed-off-by: Daniel Thompson <daniel.thompson@linaro.org>
Added options to make deploy so it can be used for ESP8266 boards with
other flash configurations. For example NodeMCU DEVKIT V1.0 can now use:
$ make FLASH_MODE=dio FLASH_SIZE=32m deploy
The output might contain more than one line ending in 5b so properly skip
everything until the next known point.
This fixes test failures in appveyor debug builds.
In particular, this makes the L4 .isr_vector section 16K in size so it's
the same as the F4/F7 MCUs. The patch also moves the L4 filesystem to
the end of flash, which allows for 512K filesystem on the 1Mb devices
like the STM32L476DISC.
The delay_half parameter must be specified by the port to set up the
timing of the software SPI. This allows the port to adjust the timing
value to better suit its timing characteristics, as well as provide a
more accurate printing of the baudrate.
It's simpler to just default to always using software SPI if no specific
peripheral id/name is given. To use hardware SPI users must specify a
hardware peripheral id as the first parameter to the SPI constructor.
There is no need to take src_len and dest_len arguments. The case of
reading-only with a single output byte (originally src_len=1, dest_len>1)
is now handled by using the output buffer as the input buffer, and using
memset to fill the output byte into this buffer. This simplifies the
implementations of the spi_transfer protocol function.
Similar to how binary op already works. Common unary operations already
have fast paths for bool so there's no need to have explicit handling of
ops in bool_unary_op, especially since they have the same behaviour as
integers.
On 32-bit archs this makes the scope_t struct 48 bytes in size, which fits
in 3 GC blocks (previously it used 4 GC blocks). This will lead to some
savings when compiling scripts because there are usually quite a few scopes,
one for each function and class.
Note that qstrs will fit in 16 bits, this assumption is made in a few other
places.
The memory read/write I2C functions now take an optional keyword-only
parameter that specifies the number of bits in the memory address.
Only mem-addrs that are a multiple of 8-bits are supported (otherwise
the behaviour is undefined).
Due to the integer type used for the address, for values larger than 32
bits, only 32 bits of address will be sent, and the rest will be padded
with 0s. Right now no exception is raised when that happens. For values
smaller than 8, no address is sent. Also no exception then.
Tested with a VL6180 sensor, which has 16-bit register addresses.
Due to code refactoring, this patch reduces stmhal and esp8266 builds
by about 50 bytes.
Following how other objects work, set/frozenset methods should use the
mp_check_self() macro to check the type of the self argument, because in
most cases this check can be a null operation.
Saves about 100-180 bytes of code for builds with set and frozenset
enabled.
To reset the flags we should write to the single bit only, not the entire
register (otherwise all other settings in the register are cleared).
Fixes#2457.
Having a micropython.const identity function, and writing "from micropython
import const" at the start of scripts that use the const feature, allows to
write scripts which are compatible with CPython, and with uPy builds that
don't include const optimisation.
This patch adds such a function and updates the tests to do the import.
When an exception is raised and is to be handled by the VM, it is stored
on the Python value stack so the bytecode can access it. CPython stores
3 objects on the stack for each exception: exc type, exc instance and
traceback. uPy followed this approach, but it turns out not to be
necessary. Instead, it is enough to store just the exception instance on
the Python value stack. The only place where the 3 values are needed
explicitly is for the __exit__ handler of a with-statement context, but
for these cases the 3 values can be extracted from the single exception
instance.
This patch removes the need to store 3 values on the stack, and instead
just stores the exception instance.
Code size is reduced by about 50-100 bytes, the compiler and VM are
slightly simpler, generate bytecode is smaller (by 2 bytes for each try
block), and the Python value stack is reduced in size for functions that
handle exceptions.
This fixes constant substitution so that only standalone identifiers are
replaced with their constant value (if they have one). I.e. don't
replace NAME in expressions like obj.NAME or NAME = expr.
qstrs ids are restricted to fit within 2 bytes already (eg in persistent
bytecode) so it's safe to use a uint16_t to store them in mp_arg_t. And
the flags member only needs a maximum of 2 bytes so can also use uint16_t.
Savings in code size can be significant when many mp_arg_t structs are
used for argument parsing. Eg, this patch reduces stmhal by 480 bytes.
When the clock is too fast for the i2c slave, it can temporarily hold
down the scl line to signal to the master that it needs to wait. The
master should check the scl line when it is releasing it after
transmitting data, and wait for it to be released.
This change has been tested with a logic analyzer and an i2c slace
implemented on an atmega328p using its twi peripheral, clocked at 8Mhz.
Without the change, the i2c communication works up to aboy 150kHz
frequency, and above that results in the slave stuck in an unresponsive
state. With this change, communication has been tested to work up to
400kHz.
The system printf is no longer used by the core uPy code. Instead, the
platform print stream or DEBUG_printf is used. Using DEBUG_printf in the
showbc functions would mean that the code can't be tested by the test
suite, so use the normal output instead.
This patch also fixes parsing of bytecode-line-number mappings.
UART1 can be used even if the switch is enabled. The schematics for this
board make I2C1 available on PB8/PB9, even though it can also be mapped
to PB6/PB7.
See #2396 and #2427.
The vstr.had_error flag was a relic from the very early days which assumed
that the malloc functions (eg m_new, m_renew) returned NULL if they failed
to allocate. But that's no longer the case: these functions will raise an
exception if they fail.
Since it was impossible for had_error to be set, this patch introduces no
change in behaviour.
An alternative option would be to change the malloc calls to the _maybe
variants, which return NULL instead of raising, but then a lot of code
will need to explicitly check if the vstr had an error and raise if it
did.
The code-size savings for this patch are, in bytes: bare-arm:188,
minimal:456, unix(NDEBUG,x86-64):368, stmhal:228, esp8266:360.
With the previous patch combining 3 emit functions into 1, it now makes
sense to also combine the corresponding VM opcodes, which is what this
patch does. This eliminates 2 opcodes which simplifies the VM and reduces
code size, in bytes: bare-arm:44, minimal:64, unix(NDEBUG,x86-64):272,
stmhal:92, esp8266:200. Profiling (with a simple script that creates many
list/dict/set comprehensions) shows no measurable change in performance.
The 3 kinds of comprehensions are similar enough that merging their emit
functions reduces code size. Decreases in code size in bytes are:
bare-arm:24, minimal:96, unix(NDEBUG,x86-64):328, stmhal:80, esp8266:76.
bool(None) has a fast path in mp_obj_is_true so doesn't need to be
handled in none_unary_op. The only caveat is that subclassing may
bypass the mp_obj_is_true function, but actually you aren't allowed to
subclass classes that have singleton instances like NoneType (see
https://mail.python.org/pipermail/python-dev/2002-March/020822.html for
reference on this point).
This is actually long overdue: the README in the windows directory has been
updated once to indicate mingw32 is abandoned and not ok to use with uPy,
but we forgot travis builds were still using it.
As a bonus the travis build will succeed again since moduerrno.c now compiles.
(see https://github.com/micropython/micropython/pull/2399)
py/makeqstrdefs.py declares that it works with python 2.6 however the
syntax used to initialise of a set with values was only added in python
2.7. This leads to build failures when the host system doesn't have
python 2.7 or newer.
Instead of using the new syntax pass a list of initial values through
set() to achieve the same result. This should work for python versions
from at least 2.6 onwards.
Helped-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Chris Packham <judge.packham@gmail.com>
If a user tries to call `swint()` while interrupt is disabled the flag in
SWIER is set but the interrupt is not triggered and therefore the SWIER bit
is not cleared. When the interrupt is again enabled the next call to
`swint()` won't trigger the IRQ because a 0 to 1 transition will not occur.
The LCD interface library fails to deassert the chip select of the LCD
after an SPI transmission. Consequently using the SPI with other
peripherals disturbs the state of the LCD. This patch changes
lcd.lcd_out() to deassert CS after each transmission to the LCD.
Tested on a STM32F7DISCO at 216MHz. All tests generating code (inlineasm,
native, viper) now pass, except pybnative/while.py, but that's because
there is no LED(2).
machine.POWER_ON is renamed to machine.PWRON_RESET to match other
reset-cause constants that all end in _RESET. The cc3200 port keeps a
legacy definition of POWER_ON for backwards compatibility.
- Refers to the technique of instantiating an object for use in an ISR by
specifying it as a default argument.
- Footnote detailing the fact that interrupt handlers continue to be
executed at the REPL.
When dealing with a board which controls chip reset with UART's DTR/RTS,
we never see REASON_DEFAULT_RST (0), only REASON_EXT_SYS_RST (6). However,
trying a "raw" module with with just TXD/RXD UART connection, on power up
it has REASON_DEFAULT_RST as a reset reason.
According to the Arduino ESP8266 implementation the first argument to the
wifi scan callback is actually a bss_info pointer. This patch fixes the
iteration over this data so the first 2 entries are no longer skipped.
Fixes issue #2372.
Interrupts during neopixel_write causes timing problems and therefore
wrong light patterns. Switching off IRQs should help to keep the strict
timing schedule.
This new config option allows to control whether MicroPython uses its own
internal printf or not (if not, an external one should be linked in).
Accompanying this new option is the inclusion of lib/utils/printf.c in the
core list of source files, so that ports no longer need to include it
themselves.
Adds horizontal scrolling. Right now, I'm just leaving the margins
created by the scrolling as they were -- so they will repeat the
edge of the framebuf. This is fast, and the user can always fill
the margins themselves.
modpybhspi now does the needed multiplexing, calling out to modpybspi
(bitbanging SPI) for suitable peripheral ID's. modmachinespi (previous
multiplexer class) thus not needed and removed.
modpybhspi also updated to following standard SPI peripheral naming:
SPI0 is used for FlashROM and thus not supported so far. SPI1 is available
for users, and thus needs to be instantiated as:
spi = machine.SPI(1, ...)
There was a bug in `framebuf1_fill` function, that makes it leave a few
lines unfilled at the bottom if the height is not divisible by 8.
A similar bug is fixed in the scroll method.
Arguments of an unknown type cannot be skipped and continuing to parse a
format string after encountering an unknown format specifier leads to
undefined behaviour. This patch helps to find use of unsupported formats.
The idea is that all ports can use these helper methods and only need to
provide initialisation of the SPI bus, as well as a single transfer
function. The coding pattern follows the stream protocol and helper
methods.
This extra forward slash for the starting-point directory is unnecessary
and leads to additional slashes on Max OS X which mean that the frozen
files cannot be imported.
Fixes#2374.
The OneWire class is now in its own onewire.py module, and the temperature
sensor class is in its own ds18x20.py module. The latter is renamed to
DS18X20 to reflect the fact that it will support both the "S" and "B"
variants of the device.
These files are moved to the modules/ subdirectory to take advantage of
frozen bytecode.
This patch makes second and next calls to <socket>.close() a no-op.
It prevents GC from closing the underlying resource after user
already used <socket>.close() explicitly.
fixes#2355
There can be stray pointers in memory blocks that are not properly zero'd
after allocation. This patch adds a new config option to always zero all
allocated memory (via gc_alloc and gc_realloc) and hence help to eliminate
stray pointers.
See issue #2195.
This is an object-oriented approach, where uos is only a proxy for the
methods on the vfs object. Some internals had to be exposed (the STATIC
keyword removed) for this to work.
Fixes#2338.
In current state `mp_get_stream_raise` assumes that `self_in` is an object
and always performs a pointer derefence which may cause a segfault.
This function shall throw an exception whenever `self_in` does not implement
a stream protocol, that includes qstr's and numbers.
fixes#2331
In `btree_seq()`, when `__bt_seq()` gets called with invalid
`flags` argument it will return `RET_ERROR` and it won't
initialize `val`. If field `data` of uninitialized `val`
is passed to `mp_obj_new_bytes()` it causes a segfault.
It turns out that TIM1 and TIM8 have their own Capture/Compare
interrupt vector. For all of the other timers, the capture/compare
interrupt vector is the same as the update vector.
So we need to add handlers for these vectors and enable them
when using capture/compare callbacks.
During testing of this, I also found that passing a channel callback
into the channel constructor would not enable interrupts properly.
I tested using:
```
>>> pyb.Timer(1, freq=4).channel(1, pyb.Timer.OC_TOGGLE, callback=lambda t: print('.', end=''))
```
I tested the above with channels 1, 4, and 8
This type was used only for the typedef of mp_obj_t, which is now defined
by the object representation. So we can now remove this unused typedef,
to simplify the mpconfigport.h file.
The machine_ptr_t type is long obsolete as the type of mp_obj_t is now
defined by the object representation, ie by MICROPY_OBJ_REPR. So just use
void* explicitly for the typedef of mp_obj_t.
If a port wants to use something different then they should define a new
object representation.
Only tuple, namedtuple and attrtuple use the tuple_cmp_helper function,
and they all have getiter=mp_obj_tuple_getiter, so the check here is only
to ensure that the self object is consistent. Hence use mp_check_self.
Checks for number of args removes where guaranteed by function descriptor,
self checking is replaced with mp_check_self(). In few cases, exception
is raised instead of assert.
Indended to replace raw asserts in bunch of files. Expands to empty
if MICROPY_BUILTIN_METHOD_CHECK_SELF_ARG is defined, otehrwise by
default still to assert, though a particular port may define it to
something else.
Introduce mp_raise_msg(), mp_raise_ValueError(), mp_raise_TypeError()
instead of previous pattern nlr_raise(mp_obj_new_exception_msg(...)).
Save few bytes on each call, which are many.
Fixing Issue #2243. Main problems were:
- HAL_ADC_GetState(adcHandle) may return other bits set (not only
HAL_ADC_STATE_EOC_REG) when called - so I AND-ed it out as proposed by
mattbrejza in Issue #2243.
- ADC Pin has to be configured as GPIO_MODE_ANALOG_ADC_CONTROL not only
GPIO_MODE_ANALOG.
- Resolved ADC resolution L4 specific (Use L4 define ADC_RESOLUTION_12B).
- Changed setting of Init.EOCSelection toADC_EOC_SINGLE_CONV for L4.
- Added call to ADC_MultiModeTypeDef as this is done on a STM32Cube
generated project too.
- Clean up: Configuration of ADC is done only in ONE function not the same
is done in two functions.
Test is done on PA5 pin of STM32L4Discovery-Kit which is connected to the
DOWN button.
Thanks to mattbrejza for discovering the bug.
This includes file and socket objects, backed by Unix file descriptor.
This improves compatibility with stmhal's uselect (and convenience of
use), though not completely: return value from poll.poll() is still
raw file descriptor.
That apparently will only help folks who read the docs on how to disable,
but could use a quick reminder straight in boot.py. For the developers,
it's important to have debug logging enabled in development branch
(master).
This goes bit against websocket nature (message-based communication),
as it ignores boundaries bertween messages, but may be very practical
to do simple things with websockets.
Name recv() based a "simplistic", as it can't work robustly in every
environment. All this is to let people concentreate on proper, read()-
based one (and to turn recv() based into a "negative showcase",
explaining what are the pitfalls of such approach).
In the sense that while GET_FILE transfers its data, REPL still works.
This is done by requiring client to send 1-byte block before WebREPL
server transfers next block of data.
md5 is calculated over the entire file, except first 4 bytes, which contain
flash parameters and may be changed by flashing tool or MicroPython flash
auto-config.
These macros are broken and are anyway unused on these two ports. If they
are ever needed in the future then their implementation can be taken from
either stmhal (working macros in mpconfigport.h) or esp8266 (functions).
Using __errno() function, and redirect it to use mp_stream_errno from
stream module. This is pre-requisite for integrating with 3rd-party libs,
like BerkeleyDB.
When compiling with msys2's gcc there's no need to apply the binary fmode
so adjust the Makefile to reflect that.
When compiling with mingw we need to include malloc.h since there is no
alloca.h, and the 64bit detection in mpconfigport.h needs some adjustment.
To filter out even prototypes of mp_stream_posix_*() functions, which
require POSIX types like ssize_t & off_t, which may be not available in
some ports.
Helpful when porting existing C libraries to MicroPython. abort()ing in
embedded environment isn't a good idea, so when compiling such library,
-Dabort=abort_ option can be given to redirect standard abort() to this
"safe" version.
The configuration bits for the UART register were wrong and the parity
couldn't be enabled, because the exist_parity member hasn't been updated. I
took this ESP8266 register description (http://esp8266.ru/esp8266-uart-reg/)
as reference.
Verification has been done with a logic analyzer.
The idea behind decrease is: bytecode and other static data is also kept on
heap, and can easily become half of heap, then setting threshold to half of
heap will have null effect - GC will happen on complete heap exhaustion like
before. But exactly in such config maintaining heap defragmented is very
important, so lower threshold to accommodate that.
This is a fix for https://github.com/micropython/micropython/issues/2209:
by default a file created using open() uses text translation mode so writing
\n to it will result in the file having \r\n. This is obviously problematic
for binary .mpy files, so provide functions for setting the open mode
and use binary mode in mpy-cross' main().
Something like:
if foo == "bar":
will be always false if foo is b"bar". In CPython, warning is issued if
interpreter is started as "python3 -b". In MicroPython,
MICROPY_PY_STR_BYTES_CMP_WARN setting controls it.
Currently, MicroPython runs GC when it could not allocate a block of memory,
which happens when heap is exhausted. However, that policy can't work well
with "inifinity" heaps, e.g. backed by a virtual memory - there will be a
lot of swap thrashing long before VM will be exhausted. Instead, in such
cases "allocation threshold" policy is used: a GC is run after some number of
allocations have been made. Details vary, for example, number or total amount
of allocations can be used, threshold may be self-adjusting based on GC
outcome, etc.
This change implements a simple variant of such policy for MicroPython. Amount
of allocated memory so far is used for threshold, to make it useful to typical
finite-size, and small, heaps as used with MicroPython ports. And such GC policy
is indeed useful for such types of heaps too, as it allows to better control
fragmentation. For example, if a threshold is set to half size of heap, then
for an application which usually makes big number of small allocations, that
will (try to) keep half of heap memory in a nice defragmented state for an
occasional large allocation.
For an application which doesn't exhibit such behavior, there won't be any
visible effects, except for GC running more frequently, which however may
affect performance. To address this, the GC threshold is configurable, and
by default is off so far. It's configured with gc.threshold(amount_in_bytes)
call (can be queries without an argument).
3-arg form:
stream.write(data, offset, length)
2-arg form:
stream.write(data, length)
These allow efficient buffer writing without incurring extra memory
allocation for slicing or creating memoryview() object, what is
important for low-memory ports.
All arguments must be positional. It might be not so bad idea to standardize
on 3-arg form, but 2-arg case would need check and raising an exception
anyway then, so instead it was just made to work.
The minimum thread stack size is set by pthreads (16k bytes) so we must
use that value for our minimum. The stack limit check is also adjusted
to work correctly for 32-bit builds.
Since "read-exactly" stream refactor, where stream.read(N) will read
exactly N bytes (unless EOF), http_server* examples can't any longer do
client_socket.read(4096) and expect to get full request (it will block
on HTTP/1.1 client). Instead, read request line by line, as the HTTP
protocol requires.
Threading support is still very new so stay conservative at this point
and enable threading without the GIL. This requires users to protect
concurrent access of mutatable Python objects (eg lists) with locks at
the Python level (something you should probably do anyway). The
advantage is that there is less of a performance hit for non-threaded
code, because the VM does not need to constantly release/acquire the GIL.
In the future the GIL will be made more efficient. There is also room to
improve the efficiency of non-GIL code by not using mutex's if there is
only one thread active.
Due to the way modern compilers work (allocating space for stack vars once
at tha start of function, and deallocating once on exit from), using
intermediate stack buffer of big size caused blockage of 4K (PATH_MAX)
on stack for the entire duration of MicroPython execution.
This follows source code/header file organization similar to few other
objects, and intended to be used only is special cases, where efficiency/
simplicity matters.
Previously, if there was chain of allocated blocks ending with the last
block of heap, it wasn't included in number of 1/2-block or max block
size stats.
To start with, the critical scripts _boot.py and flashbdev.py are frozen
to improve performance and reduce RAM consumption.
Saves about 1000 bytes of heap RAM for a bare boot with filesystem.
The 16k FreeRTOS heap originally had all TCBs and stacks dynamically
allocated within it (plus semaphores and some other things). Now that
xTaskCreateStatic is used instead of xTaskCreate, the TCBs and stacks
are allocated statically and no longer use any of the FreeRTOS heap.
Therefore, the FreeRTOS stack can be shrunk by the amount that has been
made static. Furthermore, the TCBs and stack that are now static should
be placed in the .rtos_heaps section of RAM because this RAM is treated
specially by the bootloader (the bootloader executes from the first 16k
of RAM and loads the firmware into the section starting after the 16k).
After this patch the FreeRTOS heap (ucHeap) is 7200 bytes. The memory
available for the MicroPython heap is 54936 bytes (including GC overhead).
In VStartSimpleLinkSpawnTask we change xTaskCreate to xTaskCreateStatic
so that the task is created using statically allocated memory for the TCB
and stack.
This means that xTaskCreate function is no longer needed (the static
version is now used exclusively).
This function is no longer used. Having the .boot section attribute
meant that it was included in the firmware regargless of use. Without
this attribute the linker can remove the function.
This allows to statically allocate the TCB (thread control block) and
thread stack in the BSS segment, reducing the need for dynamic memory
allocation.
Now only the bits that really need to be written in assembler are written
in it, otherwise C is used. This means that the assembler code no longer
needs to know about the global state structure which makes it much easier
to maintain.
This is a pristine copy (actually a subset of files) of upstream FreeRTOS
v9.0.0.
Modifications to the previous version of FreeRTOS (v8.1.2) included
addition of __attribute__ ((section (".boot"))) to the following
functions:
pxPortInitialiseStack
prvTaskExitError
prvPortStartFirstTask
xPortStartScheduler
vPortSetupTimerInterrupt
xTaskGenericCreate
vTaskStartScheduler
prvInitialiseTCBVariables
prvInitialiseTaskLists
prvAllocateTCBAndStack
This attribute will need to be reinstated on a case-by-case basis
because some of the above functions are now removed/changed.
SA_SIGINFO allows the signal handler to access more information about
the signal, especially useful in a threaded environment. The extra
information is not currently used but it may prove useful in the future.
GC_EXIT() can cause a pending thread (waiting on the mutex) to be
scheduled right away. This other thread may trigger a garbage
collection. If the pointer to the newly-allocated block (allocated by
the original thread) is not computed before the switch (so it's just left
as a block number) then the block will be wrongly reclaimed.
This patch makes sure the pointer is computed before allowing any thread
switch to occur.
By using a single, global mutex, all memory-related functions (alloc,
free, realloc, collect, etc) are made thread safe. This means that only
one thread can be in such a function at any one time.
The linker flag --gc-sections is not available on the linker used on
Mac OS X which results in an error when linking micropython on Mac OS X.
Therefore move this option to the LDFLAGS_ARCH variable on non Darwin
systems. According to http://stackoverflow.com/a/17710056 the equivalent
to --gc-sections is -dead_strip thus this option is used for the
LDFLAGS_ARCH on Darwin systems.
In particular, the WeMOS D1 Mini board comes with a shield that has a
64x48 OLED display. This patch makes it display properly, with the upper
left pixel being at (0, 0) and not (32, 0).
I tried to do this with the configuration commands, but there doesn't
seem to be a command that would set the column offset (there is one for
the line offset, though).
gcc 6.1.1 warns when indentation is misleading, and in this case the
formatting of the code really is misleading. So adjust the formatting
to be clear of the meaning of the code.
Storing a chain of pbuf was an original design of @pfalcon's lwIP socket
module. The problem with storing just one, like modlwip does is that
"peer closed connection" notification is completely asynchronous and out of
band. So, there may be following sequence of actions:
1. pbuf #1 arrives, and stored in a socket.
2. pbuf #2 arrives, and rejected, which causes lwIP to put it into a
queue to re-deliver later.
3. "Peer closed connection" is signaled, and socket is set at such status.
4. pbuf #1 is processed.
5. There's no stored pbufs in teh socket, and socket status is "peer closed
connection", so EOF is returned to a client.
6. pbuf #2 gets redelivered.
Apparently, there's no easy workaround for this, except to queue all
incoming pbufs in a socket. This may lead to increased memory pressure,
as number of pending packets would be regulated only by TCP/IP flow
control, whereas with previous setup lwIP had a global overlook of number
packets waiting for redelivery and could regulate them centrally.
This allows to define an abstract base class which would translate
C-level protocol to Python method calls, and any subclass inheriting
from it will support this feature. This in particular actually enables
recently introduced machine.PinBase class.
Allows to translate C-level pin API to Python-level pin API. In other
words, allows to implement a pin class and Python which will be usable
for efficient C-coded algorithms, like bitbanging SPI/I2C, time_pulse,
etc.
When built for Linux, libffi includes very bloated and workaround exec-alloc
implementation required to work around SELinux and other "sekuritee" features
which real people don't use. MicroPython has own alloc-exec implementation,
used to alloc memory for @micropython.native code. With this option enabled,
uPy's implementation will override libffi's. This saves 11K on x86_64 (and
that accounts for more than half of the libffi code size).
TODO: Possibly, we want to refactor this option to allow either use uPy's
implementation even for libffi, or allow to use libffi's implementation even
for uPy.
This actually saves "only" 6K for x86_64 build, as we're still more or less
careful to #ifdef unneeded code. But relying on --gc-sections in a "lazy"
manner would allow to make #ifdef'ing less pervasive (not suggested right
away, but an option for the future).
The time stamp is taken from the RTC for all newly generated
or changed files. RTC must be maintained separately.
The dummy time stamp of Jan 1, 2000 is set in vfs.stat() for the
root directory, avoiding invalid time values.
MicroPython own readline implementation is superior now by providing
automatic indentation and completion (completion for GNU Readline was
never implemented). MICROPY_USE_READLINE=2 also wasn't build for a long
time and probably broken.
If GNU Readline is still beneficial for some cases, it can be achieved
with external wrappers like "rlwrap" (there will be the same level of
functionality, as again, there never was deep integration, like completion
support).
That's arbitrary restriction, in case of embedding, a source file path may
be absolute. For the purpose of filtering out system includes, checking
for ".c" suffix is enough.
From https://github.com/pfalcon/berkeley-db-1.xx, which so far contains
pristine 1.85, but will get patches and compile warning fixes going
forward.
Berkeley DB 1.xx is BSD-licensed, and will form the basis of "btree"
simple database module.
Docs are now by default rebuilt from scratch, as required to build
conditionalized (i.e. using only:: directive) docs across different
output types. We have pretty small docset, so that's still rather fast.
However, if that's a concern, incremental rebuilds can be used by
passing "FORCE=" (nothing after =) as a make parameter. This will work
when using the same output type (e.g. only "html").
Based on my experience, there's rather non-zero chance to have an image be
flashed incorrectly. As --verify option is now works well in teh latest
esptool.py, enable it by default.
For modindex_exclude extension, per-port module excludes are also added.
With these changes, it's possible to generate docs for a particular port
devoid of any superfluous and unrelated content, including in indexes and
full-text search - with small caveat: when generating PDF docs after HTML,
or vice-versa cached internal doctree representation (build/*/doctrees/)
must be removed first.
Designed specifically to workaround issues we were facing with generating
multiple conditionalized output docsets from a single master doctree.
Extensions were factored out into a separate project, based on the fact
that many other Sphinx users experience similar or related problems:
https://github.com/pfalcon/sphinx_selective_exclude
Corresponds to the 182f4a8da57 upstream revision.
Assignments of the form "_id = const(value)" are treated as private
(following a similar CPython convention) and code is no longer emitted
for the assignment to a global variable.
See issue #2111.
By design, at wake up from deepsleep, the RTC timer will be reset, but
the data stored in RTC memory will not [1]. Therefore, we have to adjust
delta in RTC memory before going into deepsleep to get almost correct
time after waking up.
[1] http://bbs.espressif.com/viewtopic.php?t=1184#p4082
Instead of calling strlen(), which won't work if there're 32 chars in
returned ESSID. struct bss_info::ssid_len is not documented in SDK API
Guide, but is present in SDK headers since 1.4.0. Just in case, previous
code is left commented.
ESP-SDK system_get_rtc_time() returns uint32 and therefore overflow
about every 7:45h. Let's write the last state of system_get_rtc_time()
in RTC mem and use it to check for overflow. This commit require running
pyb_rtc_get_us_since_2000() at least once within 7 hours to avoid
overflow.
This introductions makes explicit the fact that whenever possible,
the documentation describes full MicroPython functionality, and arbitrary
functions/classes/modules may be missing in a paricular port or build.
This implementation makes use of vfs.stat() and therefore has the same
properties. Known issues for all ports: uos.stat(".") on the top level
returns the error code 22, EINVAL. The same happens with
uos.stat("dirname/") where dirname IS the name of a directory.
The call to stat() returns a 10 element tuple consistent to the os.stat()
call. At the moment, the only relevant information returned are file
type and file size.
Ctrl-C will raise a KeyboardInterrupt and stop the scan (although it will
continue to run in the background, it won't report anything). If
interrupted, and another scan() is started before the old one completes
in the background, then the second scan will fail with an OSError.
Avoid using system libraries, use copies bundled with MicroPython as
submodules (currently affects only libffi, other dependencies either
already used as bundled-only (axtls), or can't be bundled (so far),
like libjni).
Using usual method of virtual method tables. Single virtual method,
ioctl, is defined currently for all operations. This universal and
extensible vtable-based method is also defined as a default MPHAL
GPIO implementation, but a specific port may override it with its
own implementation (e.g. close-ended, but very efficient, e.g. avoiding
virtual method dispatch).
ets_loop_iter processes pending tasks, and tasks are considered lower
priority than interrupts, so tasks shouldn't be processed if interrupts
are disabled.
There appears to be issue signature problem with the PPA package we use,
so workaround it this way for now. Warning: with broken signature, there's
always a possibility that PPA was hacked and ships trojaned binaries.
.mpy files contain the name of the source file that they were compiled
from. This patch adds a way to change this name to an arbitrary string,
specified on the command line with the -s option. The default is to use
the full name of the input filename.
This new -s option is useful to strip off a leading directory name so
that mpy-tool.py can freeze packages.
Disabled by default, enabled in unix port. Need for this method easily
pops up when working with text UI/reporting, and coding workalike
manually again and again counter-productive.
Frozen modules are now stored with extensions and with '/' as path
separator. In other words, frozen modules paths stored as they are
in normal filesystem.
Now frozen modules is treated just as a kind of VFS, and all operations
performed on it correspond to operations on normal filesystem. This allows
to support packages properly, and potentially also data files.
This change also have changes to rework frozen bytecode modules support to
use the same framework, but it's not finished (and actually may not work,
as older adhox handling of any type of frozen modules is removed).
Make dupterm subsystem close a term stream object when EOF or error occurs.
There's no other party than dupterm itself in a better position to do this,
and this is required to properly reclaim stream resources, especially if
multiple dupterm sessions may be established (e.g. as networking
connections).
Adding a very first start section to get people going after flashing.
I tried to condense it to a minimum to avoid as much as possible
redundancy and bloating.
Both read and write operations support variants where either a) a single
call is made to the undelying stream implementation and returned buffer
length may be less than requested, or b) calls are repeated until requested
amount of data is collected, shorter amount is returned only in case of
EOF or error.
These operations are available from the level of C support functions to be
used by other C modules to implementations of Python methods to be used in
user-facing objects.
The rationale of these changes is to allow to write concise and robust
code to work with *blocking* streams of types prone to short reads, like
serial interfaces and sockets. Particular object types may select "exact"
vs "once" types of methods depending on their needs. E.g., for sockets,
revc() and send() methods continue to be "once", while read() and write()
thus converted to "exactly" versions.
These changes don't affect non-blocking handling, e.g. trying "exact"
method on the non-blocking socket will return as much data as available
without blocking. No data available is continued to be signaled as None
return value to read() and write().
From the point of view of CPython compatibility, this model is a cross
between its io.RawIOBase and io.BufferedIOBase abstract classes. For
blocking streams, it works as io.BufferedIOBase model (guaranteeing
lack of short reads/writes), while for non-blocking - as io.RawIOBase,
returning None in case of lack of data (instead of raising expensive
exception, as required by io.BufferedIOBase). Such a cross-behavior
should be optimal for MicroPython needs.
To use frozen bytecode make a subdirectory under the unix/ directory
(eg frozen/), put .py files there, then run:
make FROZEN_MPY_DIR=frozen
Be sure to build from scratch. The .py files will then be available for
importing.
When an mpy file is frozen it must know the values of certain
configuration variables. This patch provides an explicit check in the
generated C file that the configuration variables are what they are
supposed to be.
That one was missing in the module, even if it was available in the
vfs object. The change consist of adding the name and preparing the
call to the underlying vfs module, similar to what was already
implemented e.g. for remove.
Rename is useful by itself, or for instance for a safe file replace,
consisting of the sequence:
write to a temp file
delete the original file
rename the temp file to the original file's name
Calling it from lwIP accept callback will lead incorrect functioning
and/or packet leaks if Python callback has any networking calls, due
to lwIP non-reentrancy. So, instead schedule "poll" callback to do
that, which will be called by lwIP when it does not perform networking
activities. "Poll" callback is called infrequently though (docs say
every 0.5s by default), so for better performance, lwIP needs to be
patched to call poll callback soon after accept callback, but when
current packet is already processed.
For example, the following code now works with a file on the SD card:
f = open('test', 'rb') # test must be 1024 bytes or more in size
f.seek(511)
f.read(513)
Also works for writing.
Fixes issue #1863.
Address printed was truncated anyway and in general confusing to outsider.
A line which dumps it is still left in the source, commented, for peculiar
cases when it may be needed (e.g. when running under debugger).
In some compliation enviroments (e.g. mbed online compiler) with
strict standards compliance, <math.h> does not define constants such
as M_PI. Provide fallback definitions of M_E and M_PI where needed.
If an OSError is raised with an integer argument, and that integer
corresponds to an errno, then the string for the errno is used as the
argument to the exception, instead of the integer. Only works if
the uerrno module is enabled.
These are typical consumers of large chunks of memory, so it's useful to
see at least their number (how much memory isn't clearly shown, as the data
for these objects is allocated elsewhere).
These symbols are still defined in terms of the system Exxx symbols, and
can be switched to internal numeric definitions at a later stage.
Note that extmod/modlwip still uses many system Exxx symbols.
Effect measured on esp8266 port:
Before:
>>> pystone_lowmem.main(10000)
Pystone(1.2) time for 10000 passes = 44214 ms
This machine benchmarks at 226 pystones/second
>>> pystone_lowmem.main(10000)
Pystone(1.2) time for 10000 passes = 44246 ms
This machine benchmarks at 226 pystones/second
After:
>>> pystone_lowmem.main(10000)
Pystone(1.2) time for 10000 passes = 44343ms
This machine benchmarks at 225 pystones/second
>>> pystone_lowmem.main(10000)
Pystone(1.2) time for 10000 passes = 44376ms
This machine benchmarks at 225 pystones/second
vstr_null_terminated_str is almost certainly a vstr finalization operation,
so it should add the requested NUL byte, and not try to pre-allocate more.
The previous implementation could actually allocate double of the buffer
size.
Previous to this patch bignum division and modulo would temporarily
modify the RHS argument to the operation (eg x/y would modify y), but on
return the RHS would be restored to its original value. This is not
allowed because arguments to binary operations are const, and in
particular might live in ROM. The modification was to normalise the arg
(and then unnormalise before returning), and this patch makes it so the
normalisation is done on the fly and the arg is now accessed as read-only.
This change doesn't increase the order complexity of the operation, and
actually reduces code size.
This is kind of compensation for 4K FatFs buffer size which is eaten away
from it on FS mount. This should still leave enough of networking ("OS")
heap.
When DIG_SIZE=32, a uint32_t is used to store limbs, and no normalisation
is needed because the MSB is already set, then there will be left and
right shifts (in C) by 32 of a 32-bit variable, leading to undefined
behaviour. This patch fixes this bug.
Also do that only for the first word in a line. The idea is that when you
start up interpreter, high chance that you want to do an import. With this
patch, this can be achieved with "i<tab>".
The type is an unsigned 8-bit value, since bytes objects are exactly
that. And it's also sensible for unicode strings to return unsigned
values when accessed in a byte-wise manner (CPython does not allow this).
It interferes with running testsuite. master branch should be optimized for
development, so any features which interfere with that, would need to be
disabled by default.
The main thing is to change the DMA code in a way that the structure
DMA_Stream_TypeDef (which is similar to DMA_Channel_TypeDef on stm32l4)
is no longer used outside of dma.c, as this structure only exists for the
F4 series. Therefore I introduced a new structure (dma_descr_t) which
handles all DMA specific stuff for configuration. Further the periphery
(spi, i2c, sdcard, dac) does not need to know the internals of the dma.
Useful for testing fragmentation issues in OS heap. E.g. freemem() may
report large amount, but is it possible to actually allocate block of
a given size? Issue malloc() (followed by free()) to find out.
Previously, "import _io" worked on both CPython and MicroPython (essentially
by a chance on CPython, as there's not guarantee that its contents will stay
the same across versions), but as the module was renamed to uio, need to use
more robust import sequence for compatibility.
A standard I2C address is 7 bits but addresses 0b0000xxx and 0b1111xxx
are reserved. The scan() method is changed to reflect this, along with
the docs.
If there's no port_config.py file, or it lacks WEBREPL_PASS variable,
"initial setup mode" will be entered on first WebREPLconnection. User
will be asked for password, which will be written to
port_config.WEBREPL_PASS, and system restarted to work in normal mode
with password active.
While just a websocket is enough for handling terminal part of WebREPL,
handling file transfer operations requires demultiplexing and acting
upon, which is encapsulated in _webrepl class provided by this module,
which wraps a websocket object.
The current install command uses the flag -D which is specific to the
install command from GNU coreutils, but isn't available for the BSD
version. This solution uses the -d flag which should be commonly
available to create the target directory. Afterwards the target files
are installed to this directory seperately.
Changes are:
- added OneWireError exception and used where errors can occur
- renamed read/write functions to use same names as C _onewire funcs
- read_bytes is now read, write_bytes is now write
- add ability to read/write DS18B20 scratch pad
- rename start_measure to convert_temp (since that's what it does)
- rename get_temp to read_temp (consistency with other read names)
- removed test function
All functionality of the pyb module is available in other modules, like
time, machine and os. The only outstanding function, info(), is
(temporarily) moved to the esp module and the pyb module is removed.
Even the modules whose names don't start with "u" prefix are micro-ified
anyway, i.e. provide only subset of CPython's functionality (and sometimes
extensions to it). So, it doesn't make much sense to devide them by
criteria of having/not having "u" prefix.
The C standard says that left-shifting a signed value (on the LHS of the
operator) is undefined. So we cast to an unsigned integer before the
shift. gcc does not issue a warning about this, but clang does.
All Flash sans firmware at the beginning and 16K SDK param block at the
end is used for filesystem (and that's calculated depending on the Flash
size).
Builds have been broken since reworking autogeneration in c618f91 and
related, this gets fixed here by applying similar qstr generation logic
for the msvc builds: c files are only preprocessed when changed (or not
yet preprocessed) and the concatenated output is fed into makeqstrdefs.py.
To speed up this process, the concatenated output is already filtered to
contain only lines which makeqstrdefs really needs: this makes the qstr
generation stage about twice as fast (checked on different machines).
- msvc preprocessor output contains full paths with backslashes so the
':' and '\' characters needs to be erased from the paths as well
- use a regex for extraction of filenames from preprocessor output so it
can handle both gcc and msvc preprocessor output, and spaces in paths
(also thanks to a PR from @travnicekivo for part of that regex)
- os.rename will fail on windows if the destination file already exists,
so simply attempt to delete that file first
Several ports use identical code for the 1-argument form of the builtin
help function. Move this code to a library function to allow easier
re-use by ports.
Most pin I/O can be done just knowing the pin number as a simple
integer, and it's more efficient this way (code size, speed) because it
doesn't require a memory lookup to get the pin id from the pin object.
If the full pin object is needed then it can be easily looked up in the
pin table.
Qstr auto-generation is now much faster so this optimisation for start-up
time is no longer needed. And passing "-s -S" breaks some things, like
stmhal's "make deploy".
The L4 MCU supports 40 Events/IRQs lines of the type configurable and
direct. But this L4 port only supports configurable line types which are
already supported by uPy. For details see page 330 of RM0351, Rev 1.
The USB_FS_WAKUP event is a direct type and there is no support for it.
__GPIOI_CLK_ENABLE is defined in hal/l4/inc/Legacy/stm32_hal_legacy.h
as __HAL_RCC_GPIOI_CLK_ENABLE, and that latter macro is not defined
anywhere else (because the L4 does not have port GPIOI). So the test
for GPIOI is needed, along with the test for the CLK_ENABLE macro.
Use the machine.deepsleep() function to enter the sleep mode. Use the
RTC to configure the alarm to wake the device.
Basic use is the following:
import machine
# configure RTC's ALARM0 to wake device from deep sleep
rtc = machine.RTC()
rtc.irq(trigger=rtc.ALARM0, wake=machine.DEEPSLEEP)
# do other things
# ...
# set ALARM0's alarm to wake after 10 seconds
rtc.alarm(rtc.ALARM0, 10000)
# enter deep-sleep state (system is reset upon waking)
machine.deepsleep()
To detect if the system woke from a deep sleep use:
if machine.reset_cause() == machine.DEEPSLEEP_RESET:
print('woke from deep sleep')
Flash size as seen by vendor SDK doesn't depend on real size, but rather on
a particular value in firmware header, as put there by flash tool. That means
it's user responsibility to know what flash size a particular device has, and
specify correct parameters during flashing. That's not end user friendly
however, so we try to make it "flash and play" by detecting real size vs
from-header size mismatch, and correct the header accordingly.
E.g. for stmhal, accumulated preprocessed output may grow large due to
bloated vendor headers, and then reprocessing tens of megabytes on each
build make take couple of seconds on fast hardware (=> potentially dozens
of seconds on slow hardware). So instead, split once after each change,
and only cat repetitively (guaranteed to be fast, as there're thousands
of lines involved at most).
If make -B is run, the rule is run with $? empty. Extract fron all file in
this case. But this gets fragile, really "make clean" should be used instead
with such build complexity.
When there're C files to be (re)compiled, they're all passed first to
preprocessor. QSTR references are extracted from preprocessed output and
split per original C file. Then all available qstr files (including those
generated previously) are catenated together. Only if the resulting content
has changed, the output file is written (causing almost global rebuild
to pick up potentially renumbered qstr's). Otherwise, it's not updated
to not cause spurious rebuilds. Related make rules are split to minimize
amount of commands executed in the interim case (when some C files were
updated, but no qstrs were changed).
A port which uses lib/utils/pyexec.c but which does not enable garbage
collection should not need to implement the gc_collect function.
This patch also moves the gc_collect call to after printing the qstr
info. Since qstrs cannot be collected it should not make any difference
to the printed statistics.
To use: .setsockopt(SOL_SOCKET, 20, lambda sock: print(sock)). There's a
single underlying callback slot. For normal sockets, it serves as data
received callback, for listening sockets - connection arrived callback.
L4 does not have UART6, and has similar registers to the F7.
Original patch was authored by Tobias Badertscher / @tobbad, but it was
reworked to split UART edits from USB edits.
64-bit integer division brings a dependency on library functions. It is
avoided here by dividing fck and baud by a common divisior. The error
is the better (1/(2*0x300)) as with 64 bit division (1/(0x300)).
These files come from STM32Cube_FW_L4_V1.3.0, with Windows line endings
converted to unix. Only basic HAL files are added. In addition the QSPI
support is included to support later external QSPI flash as mass storage.
- any architecture may explicitely build with qstring make
QSTR_AUTOGEN_DISABLE=1 autogeneration disabled and provide its
own list of qstrings by the standard
mechanisms (qstrdefsport.h).
Note this still needs some work: currently all source files are always
preprocessed no matter which one actually changed, moreover that happens
file by file without any parallellism so builds are painstakingly slow.
- add template rule that converts a specified source file into a qstring file
- add special rule for generating a central header that contains all
extracted/autogenerated strings - defined by QSTR_DEFS_COLLECTED
variable. Each platform appends a list of sources that may contain
qstrings into a new build variable: SRC_QSTR. Any autogenerated
prerequisities are should be appened to SRC_QSTR_AUTO_DEPS variable.
- remove most qstrings from py/qstrdefs, keep only qstrings that
contain special characters - these cannot be easily detected in the
sources without additional annotations
- remove most manual qstrdefs, use qstrdef autogen for: py, cc3200,
stmhal, teensy, unix, windows, pic16bit:
- remove all micropython generic qstrdefs except for the special strings that contain special characters (e.g. /,+,<,> etc.)
- remove all port specific qstrdefs except for special strings
- append sources for qstr generation in platform makefiles (SRC_QSTR)
This script will search for patterns of the form Q(...) and generate a
list of them.
The original code by Pavel Moravec has been significantly simplified to
remove the part that searched for C preprocessor directives (eg #if).
This is because all source is now run through CPP before being fed into
this script.
Small hash tables (eg those used in user class instances that only have a
few members) now only use the minimum amount of memory necessary to hold
the key/value pairs. This can reduce performance for instances that have
many members (because then there are many reallocations/rehashings of the
table), but helps to conserve memory.
See issue #1760.
"" is the correct name of the root directory when mounting a device there
(as opposed to "/"). One can now do os.listdir('/') and open('/abc'), as
well as os.listdir() and open('abc').
Most grammar rules can optimise to the identity if they only have a single
argument, saving a lot of RAM building the parse tree. Previous to this
patch, whether a given grammar rule could be optimised was defined (mostly
implicitly) by a complicated set of logic rules. With this patch the
definition is always specified explicitly by using "and_ident" in the rule
definition in the grammar. This simplifies the logic of the parser,
making it a bit smaller and faster. RAM usage in unaffected.
The config variable MICROPY_MODULE_FROZEN is now made of two separate
parts: MICROPY_MODULE_FROZEN_STR and MICROPY_MODULE_FROZEN_MPY. This
allows to have none, either or both of frozen strings and frozen mpy
files (aka frozen bytecode).
They are sugar for marking function as generator, "yield from"
and pep492 python "semantically equivalents" respectively.
@dpgeorge was the original author of this patch, but @pohmelie made
changes to implement `async for` and `async with`.
The idea is that if dupterm object can handle exceptions, it will handle
them itself. Otherwise, object state can be compromised and it's better
to terminate dupterm session. For example, disconnected socket will keep
throwing exceptions and dump messages about that.
The idea is that if dupterm object can handle exceptions, it will handle
them itself. Otherwise, object state can be compromised and it's better
to terminate dupterm session. For example, disconnected socket will keep
throwing exceptions and dump messages about that.
nlr_pop must be called if no exception was raised.
Also, return value of these callback helpers is made void because ther
is (currently) no use for it.
Main entry point is _boot.py which checks whether FAT FS in flash mountable,
and if so, mounts it. Otherwise, it checks if flash is empty, and if so,
performs initial module setup: makes FAT FS, configures default AP name,
etc. As a last option, if flash is not empty, and could not be mounted,
it means filesystem corruption, and warning message with instructions is
printed in an infinite loop.
When lwIP creates a incoming connection socket of a listen socket, it
sets its recv callback to one which discards incoming data. We set
proper callback only in accept() call, when we allocate Python-level
socket where we can queue incoming data. So, in lwIP accept callback
be sure to set recv callback to one which tells lwIP to not discard
incoming data.
Upon start-up, _boot module is executed from frozen files to do early
initialization, e.g. create and mount the flash filesystem. Then
"boot.py" is executed if it exists in the filesystem. Finally, "main.py"
is executed if exists to allow start-on-boot user applications.
This allows a user to make a custom boot file or startup application
without recompiling the firmware, while letting to do early initialization
in Python code.
Based on RFC https://github.com/micropython/micropython/issues/1955.
Will call underlying C virtual methods of stream interface. This isn't
intended to be added to every stream object (it's not in CPython), but
is convenient way to expose extra operation on Python side without
adding bunch of Python-level methods.
This is strange asymmetry which is sometimes needed, e.g. for WebREPL: we
want to process only available input and no more; but for output, we want
to get rid of all of it, because there's no other place to buffer/store
it. This asymmetry is akin to CPython's asyncio asymmetry, where reads are
asynchronous, but writes are synchronous (asyncio doesn't expect them to
block, instead expects there to be (unlimited) buffering for any sync write
to completely immediately).
With .rodata being in FlashROM now, gap can be much smaller now. InstRAM
can be max 32K, and with segment headers, that already makes it more than
32K. Then there's some .data still, and the next Flash page boundary is
0x9000. That figure should be more or less future-proof.
TODO: Refactor makeimg to take FlashROM segment offset from file name.
This was originally used for non-event based REPL processing. Then it
was unused when event-based processing was activated. But now that event
based is disabled, and non-event based is back, there has been new ring
buffer code to process the chars.
Event-driven loop (push-style) is still supported and default (controlled
by MICROPY_REPL_EVENT_DRIVEN setting, as expected).
Dedicated loop worked even without adding ets_loop_iter(), though that
needs to be revisited later.
Before this change, if REPL blocked executing some code, it was possible
to still input new statememts and excuting them, all leading to weird,
and portentially dangerous interaction.
TODO: Current implementation may have issues processing input accumulated
while REPL was blocked.
The idea is following: underlying interrupt-driven or push-style data source
signals that more data is available for dupterm processing via call to
mp_hal_signal_dupterm_input(). This triggers a task which pumps data between
actual dupterm object (which may perform additional processing on data from
low-level data source) and input ring buffer.
But now it's generic ring buffer implemented via ringbuf.h, and is intended
for any type of input, including dupterm's, not just UART. The general
process work like this: an interrupt-driven input source puts data into
input_buf, and then signals new data available via call to
mp_hal_signal_input().
Features inline get/put operations for the highest performance. Locking
is not part of implementation, operation should be wrapped with locking
externally as needed.
When taking the logarithm of the float to determine the exponent, there
are some edge cases that finish the log loop too large. Eg for an
input value of 1e32-epsilon, this is actually less than 1e32 from the
log-loop table and finishes as 10.0e31 when it should be 1.0e32. It
is thus rendered as :e32 (: comes after 9 in ascii).
There was the same problem with numbers less than 1.
See https://github.com/micropython/micropython/issues/1736 for the
list of complications. This workaround instead of duplicating REPL
to another stream, switches to it, because read(STDIN) we use otherwise
is blocking call, so it and custom REPL stream can't be used together.
PWM implementation uses a timer and interrupts (FRC1), taken from
Espressif's/NodeMCU's implementation and adapted for our use.
8 channels are supported, on pins 0, 2, 4, 5, 12, 13, 14, 15.
Usage:
import machine
pwm0 = machine.PWM(machine.Pin(0))
pwm0.freq(1000)
pwm0.duty(500)
Frequency is shared (ie the same) for all channels. Frequency is
between 1 and 1000. Duty is between 0 and 1023.
Per POSIX http://pubs.opengroup.org/onlinepubs/9699919799/functions/send.html :
"If space is not available at the sending socket to hold the message to be
transmitted, and the socket file descriptor does not have O_NONBLOCK set,
send() shall block until space is available. If space is not available at the
sending socket to hold the message to be transmitted, and the socket file
descriptor does have O_NONBLOCK set, send() shall fail [with EAGAIN]."
Previous to this patch, the "**b" in "a**b" had its own parse node with
just one item (the "b"). Now, the "b" is just the last element of the
power parse-node. This saves (a tiny bit of) RAM when compiling.
Previous to this patch, all qemu-arm tests were running in the same
session, and global variables could be left over from the previous test.
This patch makes it so that the heap and runtime are reinitialised at
the start of each test.
All tests in basics/ directory can now run and pass using 64-bit unix
port with only a 16k heap (./run-tests --heapsize 16k). Tests in this
directory should remain small so they can be used for ports with a
small heap.
Passing an mp_uint_t to a %d printf format is incorrect for builds where
mp_uint_t is larger than word size (eg a nanboxing build). This patch
adds some simple casting to int in these cases.
The code is based on Damien George's implementation for esp8266 port,
avoids use of global variables and associated re-entrancy issues, and
fixes returning stale data in some cases.
This implementation provides the same interface and uses the same
datastructures as used by BootROM, i.e. is a drop-in replacement for it.
But it offers one advantage: it allows to run single iteration of
event-pumping loop.
Original BootROM function are renamed, prefixed with underscore. There's
a switch which allows to use forward calls to them, for compatibility
testing.
The implementation also includes workarounds for hardware timer handler,
and these workarounds may be SDK version specific.
Allows to set (in case keyword args are given) or query (in case a single
"symbolic keyword" (a string, value is the same as keyword)) arbitrary
interface paramters (i.e. extensible and adaptable to various hardware).
Example usage:
ap_if = network.WLAN(1)
ap_if.config(essid="MicroPython on Air")
print(ap_if.config("essid"))
Allows to up/down interface when called with a boolean, or query current
state if called without args. This per-interface method is intended to
supersede adhoc network.wifi_mode() function.
On ESP8266, there're 2 different interfaces. Pretending it's not the case
desn't make sense. So, network.WLAN() now takes interface id, and returns
interface object. Individual operations are then methods of interface
object. Some operations require i/f of specific type (e.g. .connect()
makes sense only for STA), other are defined for any (e.g. .ifconfig(),
.mac()).
Building in 32-bit mode was only to reduce binary size on 64-bit machines
and is otherwise not needed. Having it forced to 32-bit meant an
unnecessary dependency on 32-bit libraries that is now removed.
It can happen that a socket gets closed while the pbuf is not completely
drained by the application. It can also happen that a new pbuf comes in
via the recv callback, and then a "peer closed" event comes via the same
callback (pbuf=NULL) before the previous event has been handled. In both
cases the socket is closed but there is remaining data. This patch makes
sure such data is passed to the application.
If the heap is locked, or memory allocation fails, then calling a bound
method will still succeed by allocating the argument state on the stack.
The new code also allocates less stack than before if less than 4
arguments are passed. It's also a tiny bit smaller in code size.
This was done as part of the ESA project.
tools/pydfu.py is now the recommended way of deploying a DFU file. Old
behaviour of dfu-util can be obtained by passing USE_PYDFU=0 when invoking
make.
The main README.md file has been updated to reflect this change.
None of the other ports do, since introduction of mp_state_ctx_t. In
the case of current esp8266 port, heap is inside BSS, so scanning it
picked up a lot of dead pointers.
NameError may either include offending name or not. Unfortunately, this
change makes test float-dependent. And using integer division leads to
different error message than CPython.
Enabling standard assert() (by removing -DNDEBUG) produces non-bootable
binary (because all messages go to .rodata which silently overflows).
So, for once-off debugging, have a custom _assert().
Initialize RTC period coefficients, etc. if RTC RAM doesn't contain valid
values. time.time() then will return number of seconds since power-on, unless
set to different timebase.
This reuses MEM_MAGIC for the purpose beyond its initial purpose (but the whole
modpybrtc.c need to be eventually reworked completely anyway).
For some reason, Travis now has Google Chrome PPA included in the builder
image, that lacks i386 arch, that leads to apt-get update error. So, ignore
it (this is not ideal as may lead to actual repo update failures to be missed,
leading to installation of old package, leading to weird errors; let's keep
that in mind).
This new compile-time option allows to make the bytecode compiler
configurable at runtime by setting the fields in the mp_dynamic_compiler
structure. By using this feature, the compiler can generate bytecode
that targets any MicroPython runtime/VM, regardless of the host and
target compile-time settings.
Options so far that fall under this dynamic setting are:
- maximum number of bits that a small int can hold;
- whether caching of lookups is used in the bytecode;
- whether to use unicode strings or not (lexer behaviour differs, and
therefore generated string constants differ).
Reduces code size by 112 bytes on Thumb2 arch, and makes assembler faster
because comparison can be a simple equals instead of a string compare.
Not all ops have been converted, only those that were simple to convert
and reduced code size.
The chunks of memory that the parser allocates contain parse nodes and
are pointed to from many places, so these chunks cannot be relocated
by the memory manager. This patch makes it so that when a chunk is
shrunk to fit, it is not relocated.
Properly calculate the period and the prescaler, this now allows to
set the PWM frequency down to 5Hz. Make Timer IDs go from 0 to 3.
Add the trigger definitions for the channel IRQ.
These can be used to insert arbitrary checks, polling, etc into the VM.
They are left general because the VM is a highly tuned loop and it should
be up to a given port how that port wants to modify the VM internals.
One common use would be to insert a polling check, but only done after
a certain number of opcodes were executed, so as not to slow down the VM
too much. For example:
#define MICROPY_VM_HOOK_COUNT (30)
#define MICROPY_VM_HOOK_INIT static uint vm_hook_divisor = MICROPY_VM_HOOK_COUNT
#define MICROPY_VM_HOOK_POLL if (--vm_hook_divisor == 0) { \
vm_hook_divisor = MICROPY_VM_HOOK_COUNT;
extern void vm_hook_function(void);
vm_hook_function();
}
#define MICROPY_VM_HOOK_LOOP MICROPY_VM_HOOK_POLL
#define MICROPY_VM_HOOK_RETURN MICROPY_VM_HOOK_POLL
If None was returned for such requests (which likely means that user simply
didn't handle them), it means successful init and default sector size of 512
bytes respectively. This makes only BP_IOCTL_SEC_COUNT a mandatory request,
and thus re-establishes parity with old interface, where only .count() is
mandatory().
This implements OO interface based on existing fsusermount code and with
minimal changes to it, to serve as a proof of concept of OO interface.
Examle of usage:
bdev = RAMFS(48)
uos.VfsFat.mkfs(bdev)
vfs = uos.VfsFat(bdev, "/ramdisk")
f = vfs.open("foo", "w")
f.write("hello!")
f.close()
This patch adds support to fsusermount for multiple block devices
(instead of just one). The maximum allowed is fixed at compile time by
the size of the fs_user_mount array accessed via MP_STATE_PORT, which
in turn is set by MICROPY_FATFS_VOLUMES.
With this patch, stmhal (which is still tightly coupled to fsusermount)
is also modified to support mounting multiple devices And the flash and
SD card are now just two block devices that are mounted at start up if
they exist (and they have special native code to make them more
efficient).
You can now create (singleton) objects representing the flash and SD
card, using:
flash = pyb.Flash()
sdcard = pyb.SDCard()
These objects provide the block protocol.
This enables MICROPY_HW_HAS_FLASH which got missed.
The HW has UART2 on the 401 connected to the STLINK procesor
which exposes it as USB serial. This connects that up so that
you can get a REPL using the USB serial.
If MICROPY_FATFS_MAX_SS is defined to power of 2 value between 1024 and
4096, support for dynamic sector size in FatFs will be enabled. Note
that FatFs reserves static buffer of MICROPY_FATFS_MAX_SS size for each
filesystem in use, so that value should be set sparingly.
Initial patch provided by @pfalcon.
The new block protocol is:
- readblocks(self, n, buf)
- writeblocks(self, n, buf)
- ioctl(self, cmd, arg)
The new ioctl method handles the old sync and count methods, as well as
a new "get sector size" method.
The old protocol is still supported, and used if the device doesn't have
the ioctl method.
Per the previously discussed plan. mount() still stays backward-compatible,
and new mkfs() is rought and takes more args than needed. But is a step
in a forward direction.
This allows you to pass a number (being an address) to a viper function
that expects a pointer, and also allows casting of integers to pointers
within viper functions.
This was actually the original behaviour, but it regressed due to native
type identifiers being promoted to 4 bits in width.
If MICROPY_VFS_FAT is defined, mp_type_fileio & mp_type_textio won't be
defined, as these may be alredy defined elsewhere. The idea is to have
compartmentalized VFS FatFs class, which can work in parallel with some
other "main" filesystem. E.g., for unix port, mp_type_fileio, etc. will
be defined for the main POSIX filesystem, while stmhal/file.c will be
a self-contained VFS file class.
Move definition of mp_builtin_open_obj to a separate module, then file.c
becomes more or less compartmentalized FatFs file class, which can be used
together with file class implementations for other (V)FSes.
This function computes (x**y)%z in an efficient way. For large arguments
this operation is otherwise not computable by doing x**y and then %z.
It's currently not used, but is added in case it's useful one day.
For these 3 bitwise operations there are now fast functions for
positive-only arguments, and general functions for arbitrary sign
arguments (the fast functions are the existing implementation).
By default the fast functions are not used (to save space) and instead
the general functions are used for all operations.
Enable MICROPY_OPT_MPZ_BITWISE to use the fast functions for positive
arguments.
Before this patch, the native types for uint and ptr/ptr8/ptr16/ptr32
all overlapped and it was possible to make a mistake in casting. Now,
these types are all separate and any coding mistakes will be raised
as runtime errors.
Eg: '{:{}}'.format(123, '>20')
@pohmelie was the original author of this patch, but @dpgeorge made
significant changes to reduce code size and improve efficiency.
This function is provided by some MicroPython ports for compatibility
and is not efficient. Usage of :class:`Poll` is recommended instead.
.._class: Poll
class ``Poll``
--------------
Methods
~~~~~~~
..method:: poll.register(obj[, eventmask])
Register `stream`*obj* for polling. *eventmask* is logical OR of:
*``uselect.POLLIN`` - data available for reading
*``uselect.POLLOUT`` - more data can be written
Note that flags like ``uselect.POLLHUP`` and ``uselect.POLLERR`` are
*not* valid as input eventmask (these are unsolicited events which
will be returned from `poll()` regardless of whether they are asked
for). This semantics is per POSIX.
*eventmask* defaults to ``uselect.POLLIN | uselect.POLLOUT``.
..method:: poll.unregister(obj)
Unregister *obj* from polling.
..method:: poll.modify(obj, eventmask)
Modify the *eventmask* for *obj*.
..method:: poll.poll(timeout=-1)
Wait for at least one of the registered objects to become ready or have an
exceptional condition, with optional timeout in milliseconds (if *timeout*
arg is not specified or -1, there is no timeout).
Returns list of (``obj``, ``event``, ...) tuples. There may be other elements in
tuple, depending on a platform and version, so don't assume that its size is 2.
The ``event`` element specifies which events happened with a stream and
is a combination of ``uselect.POLL*`` constants described above. Note that
flags ``uselect.POLLHUP`` and ``uselect.POLLERR`` can be returned at any time
(even if were not asked for), and must be acted on accordingly (the
corresponding stream unregistered from poll and likely closed), because
otherwise all further invocations of `poll()` may return immediately with
these flags set for this stream again.
In case of timeout, an empty list is returned.
..admonition:: Difference to CPython
:class:attention
Tuples returned may contain more than 2 elements as described above.
..method:: poll.ipoll(timeout=-1, flags=0)
Like :meth:`poll.poll`, but instead returns an iterator which yields a
`callee-owned tuple`. This function provides an efficient, allocation-free
way to poll on streams.
If *flags* is 1, one-shot behavior for events is employed: streams for
which events happened will have their event masks automatically reset
(equivalent to ``poll.modify(obj, 0)``), so new events for such a stream
won't be processed until new mask is set with `poll.modify()`. This
behavior is useful for asynchronous I/O schedulers.
..admonition:: Difference to CPython
:class:attention
This function is a MicroPython extension.
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
