This improves REPL usage consistency across ports, by utilizing the pyexec
code for the unix REPL.
Only enabled when MICROPY_USE_READLINE == 1 (the default).
Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>
This allows out-of-tree tests like:
$ make -C ports/unix/ BUILD=/tmp MICROPY_MPYCROSS=~/bin/mpy-cross test
Signed-off-by: Yanfeng Liu <yfliu2008@qq.com>
Currently testing on the unix port with `--via-mpy` only runs tests in the
`basics`, `float` and `micropython` test directories.
This commit removes that restriction and now runs `--via-mpy` tests using
all possible test directories. This improves test coverage.
Signed-off-by: Damien George <damien@micropython.org>
These are convenience targets for running specific tests as a developer.
They are more useful that invoking run-tests directly as they take account
of the VARIANT= specified on the make command-line.
For instance, you can run all tests matching the regular expression "int"
with `make VARIANT=... test//int`.
The new targets are all documented in README.md.
Signed-off-by: Jeff Epler <jepler@gmail.com>
The unix port can now be built with the GIL enabled, by passing
MICROPY_PY_THREAD_GIL=1 on the make command line.
Signed-off-by: Damien George <damien@micropython.org>
When detecting the target platform, also check if it has threading and
whether the GIL is enabled or not (using the new attribute
`sys.implementation._thread`). If threading is available, add the thread
tests to the set of tests to run (unless the set of tests is explicitly
given).
With this change, the unix port no longer needs to explicitly run the set
of thread tests, so that line has been removed from the Makefile.
This change will make sure thread tests are run with other testing
combinations. In particular, thread tests are now run:
- on the unix port with the native emitter
- on macOS builds
- on unix qemu, the architectures MIPS, ARM and RISCV-64
Signed-off-by: Damien George <damien@micropython.org>
This drops use of non-existing path `/usr/include/i686-linux-gnu` as
default include paths shall suffice.
Signed-off-by: Yanfeng Liu <yfliu2008@qq.com>
This is code makes sure that time functions work properly on a
reasonable date range, on all platforms, regardless of the epoch.
The suggested minimum range is 1970 to 2099.
In order to reduce code footprint, code to support far away dates
is only enabled specified by the port.
New types are defined to identify timestamps.
The implementation with the smallest code footprint is when
support timerange is limited to 1970-2099 and Epoch is 1970.
This makes it possible to use 32 bit unsigned integers for
all timestamps.
On ARM4F, adding support for dates up to year 3000 adds
460 bytes of code. Supporting dates back to 1600 adds
another 44 bytes of code.
Signed-off-by: Yoctopuce dev <dev@yoctopuce.com>
This commit fixes CI test runs for the `nanbox` target, which were
broken by the unconditional native emitter code output changes in the
test runner.
The `nanbox` configuration does not enable native emitters of any kind,
and with a full test run that includes executing emitted native code
things would break when doing CI runs.
This is worked around by introducing a common subset of tests that do
not involve the native emitter, and a more comprehensive set of tests
that include both non-emitter and emitter tests. The `nanbox` CI test
run will stop at the first subset, whilst other configurations will run
that and execute further tests.
Function names have been kept the same for steps that involve native
code, with the `nanbox` subset having another one. This should not
trigger any breakage in existing CI configurations or external scripts.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
On 64-bit systems this should have no effect.
On 32-bit systems it will force 64-bit file sizes and fixes directory
listing on certain 32-bit ports. This option should work on pretty much
all 32-bit systems these days.
Signed-off-by: Damien George <damien@micropython.org>
Libffi is updated to the latest stable version (v3.4.6) as the version
of libffi that was linked by the repository predates the introduction of
RISC-V processors (both 32 and 64 bits).
This is necessary as the packaged version of libffi used by the CI for
RISC-V 64 turned out to simply not work correctly, whilst a source-built
version of libffi worked just fine.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This is a code factoring to have the dict for the machine module in one
location, and all the ports use that same dict. The machine.soft_reset()
function implementation is also factored because it's the same for all
ports that did already implement it. Eventually more functions/bindings
can be factored.
All ports remain functionally the same, except:
- cc3200 port: gains soft_reset, mem8, mem16, mem32, Signal; loses POWER_ON
(which was a legacy constant, replaced long ago by PWRON_RESET)
- nrf port: gains Signal
- qemu-arm port: gains soft_reset
- unix port: gains soft_reset
- zephyr port: gains soft_reset, mem8, mem16, mem32
Signed-off-by: Damien George <damien@micropython.org>
This provides a way to build a non-DEBUG host binary that still has symbols
and debug information.
Document this for the unix port, and update a comment in the unix port
Makefile.
This work was funded through GitHub Sponsors.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
The unix port has a custom select module which only works with objects that
have a file descriptor, eg files and sockets. On the other hand, bare
metal ports use the common extmod/modselect.c implementation of the select
module that supports polling of arbitrary objects, as long as those objects
provide a MP_STREAM_POLL in their ioctl implementation (which can be done
in C or Python).
This commit removes the unix-specific code and makes unix use the common
one provided by extmod/modselect.c instead. All objects with file
descriptors implement MP_STREAM_POLL so they continue to work.
Signed-off-by: Damien George <damien@micropython.org>
Following how mkrules.cmake works. This makes it easy for a port to enable
frozen code, by defining FROZEN_MANIFEST in its Makefile.
Signed-off-by: Damien George <damien@micropython.org>
Updates any includes, and references from Makefiles/CMake.
This essentially reverts what was done long ago in commit
136b5cbd76
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This modifies the unix port Makefile to use += for QSTR_DEFS and
QSTR_GLOBAL_DEPENDENCIES so that variants can add additional files if
needed (similar to stm32 port).
Signed-off-by: David Lechner <david@pybricks.com>
This allows:
$ make BOARD_DIR=path/to/board
to infer BOARD=board, rather than the previous behavior that required
additionally setting BOARD explicitly.
Also makes the same change for VARIANT_DIR -> VARIANT on Unix.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Signed-off-by: Damien George <damien@micropython.org>
This makes it so that all a port needs to do is set the relevant variables
and "include extmod.mk" and doesn't need to worry about adding anything to
OBJ, CFLAGS, SRC_QSTR, etc.
Make all extmod variables (src, flags, etc) private to extmod.mk.
Also move common/shared, extmod-related fragments (e.g. wiznet, cyw43,
bluetooth) into extmod.mk.
Now that SRC_MOD, CFLAGS_MOD, CXXFLAGS_MOD are unused by both extmod.mk
(and user-C-modules in a previous commit), remove all uses of them from
port makefiles.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This applies to nimble, btstack, axtls, mbedtls, lwip.
Rather than having the ports individually manage GIT_SUBMODULES for these
components, make extmod.mk append them when the relevant feature is
enabled.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This conflicts with the triple-usage of these variables for user-C-modules
and extmod source.
For CFLAGS_MOD, just use CFLAGS directly. For SRC, use SRC_C directly as
the relevant files are all guarded by the preprocessor anyway.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Binaries built using the Make build system now no longer appear in the
working directory of the build, but rather in the build directory. Thus
some paths had to be adjusted.
This moves the libffi submodule variable modifier inside of the if
statement where it is actually used so that the submodule will only be
checked out if it is actually being used.
A new DEPLIBS variable is also introduced to prevent building the libffi
submodule when not needed.
Signed-off-by: David Lechner <david@pybricks.com>
All ports that use mbedtls use the custom error messages in
mp_mbedtls_errors.c. This commit simplifies the build so that ports don't
need to explicitly add this file, it's now used by default when mbedtls is
enabled.
Signed-off-by: Damien George <damien@micropython.org>
Based on existing bare-metal config from stm32. Also uses shorter error
messages from lib/mbedtls_errors.
Signed-off-by: Damien George <damien@micropython.org>
This separates extmod source files from `py.mk`. Previously, `py.mk`
assumed that every consumer of the py/ directory also wanted to include
extmod/. However, this is not the case. For example, building mpy-cross
uses py/ but doesn't need extmod/.
This commit moves all extmod-specific items from `py.mk` to `extmod.mk` and
explicitly includes `extmod.mk` in ports that use it.
Signed-off-by: David Lechner <david@pybricks.com>
All variants now use extmod/moduos.c as their uos module implementation.
In particular this means they all have MICROPY_VFS enabled and use VfsPosix
for their filesystem.
As part of this, the available functions in uos become more consistent with
other ports:
- coverage variant gets uos.urandom
- minimal and standard variant get: unlink, chdir, getcwd, listdir
Signed-off-by: Damien George <damien@micropython.org>
For the coverage build this reduces the binary size to about 1/4 of its
size, and seems to help gcov/lcov coverage analysis so that it doesn't miss
lines.
Signed-off-by: Damien George <damien@micropython.org>
This commit removes all parts of code associated with the existing
MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE optimisation option, including the
-mcache-lookup-bc option to mpy-cross.
This feature originally provided a significant performance boost for Unix,
but wasn't able to be enabled for MCU targets (due to frozen bytecode), and
added significant extra complexity to generating and distributing .mpy
files.
The equivalent performance gain is now provided by the combination of
MICROPY_OPT_LOAD_ATTR_FAST_PATH and MICROPY_OPT_MAP_LOOKUP_CACHE (which has
been enabled on the unix port in the previous commit).
It's hard to provide precise performance numbers, but tests have been run
on a wide variety of architectures (x86-64, ARM Cortex, Aarch64, RISC-V,
xtensa) and they all generally agree on the qualitative improvements seen
by the combination of MICROPY_OPT_LOAD_ATTR_FAST_PATH and
MICROPY_OPT_MAP_LOOKUP_CACHE.
For example, on a "quiet" Linux x64 environment (i3-5010U @ 2.10GHz) the
change from CACHE_MAP_LOOKUP_IN_BYTECODE, to LOAD_ATTR_FAST_PATH combined
with MAP_LOOKUP_CACHE is:
diff of scores (higher is better)
N=2000 M=2000 bccache -> attrmapcache diff diff% (error%)
bm_chaos.py 13742.56 -> 13905.67 : +163.11 = +1.187% (+/-3.75%)
bm_fannkuch.py 60.13 -> 61.34 : +1.21 = +2.012% (+/-2.11%)
bm_fft.py 113083.20 -> 114793.68 : +1710.48 = +1.513% (+/-1.57%)
bm_float.py 256552.80 -> 243908.29 : -12644.51 = -4.929% (+/-1.90%)
bm_hexiom.py 521.93 -> 625.41 : +103.48 = +19.826% (+/-0.40%)
bm_nqueens.py 197544.25 -> 217713.12 : +20168.87 = +10.210% (+/-3.01%)
bm_pidigits.py 8072.98 -> 8198.75 : +125.77 = +1.558% (+/-3.22%)
misc_aes.py 17283.45 -> 16480.52 : -802.93 = -4.646% (+/-0.82%)
misc_mandel.py 99083.99 -> 128939.84 : +29855.85 = +30.132% (+/-5.88%)
misc_pystone.py 83860.10 -> 82592.56 : -1267.54 = -1.511% (+/-2.27%)
misc_raytrace.py 21490.40 -> 22227.23 : +736.83 = +3.429% (+/-1.88%)
This shows that the new optimisations are at least as good as the existing
inline-bytecode-caching, and are sometimes much better (because the new
ones apply caching to a wider variety of map lookups).
The new optimisations can also benefit code generated by the native
emitter, because they apply to the runtime rather than the generated code.
The improvement for the native emitter when LOAD_ATTR_FAST_PATH and
MAP_LOOKUP_CACHE are enabled is (same Linux environment as above):
diff of scores (higher is better)
N=2000 M=2000 native -> nat-attrmapcache diff diff% (error%)
bm_chaos.py 14130.62 -> 15464.68 : +1334.06 = +9.441% (+/-7.11%)
bm_fannkuch.py 74.96 -> 76.16 : +1.20 = +1.601% (+/-1.80%)
bm_fft.py 166682.99 -> 168221.86 : +1538.87 = +0.923% (+/-4.20%)
bm_float.py 233415.23 -> 265524.90 : +32109.67 = +13.756% (+/-2.57%)
bm_hexiom.py 628.59 -> 734.17 : +105.58 = +16.796% (+/-1.39%)
bm_nqueens.py 225418.44 -> 232926.45 : +7508.01 = +3.331% (+/-3.10%)
bm_pidigits.py 6322.00 -> 6379.52 : +57.52 = +0.910% (+/-5.62%)
misc_aes.py 20670.10 -> 27223.18 : +6553.08 = +31.703% (+/-1.56%)
misc_mandel.py 138221.11 -> 152014.01 : +13792.90 = +9.979% (+/-2.46%)
misc_pystone.py 85032.14 -> 105681.44 : +20649.30 = +24.284% (+/-2.25%)
misc_raytrace.py 19800.01 -> 23350.73 : +3550.72 = +17.933% (+/-2.79%)
In summary, compared to MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE, the new
MICROPY_OPT_LOAD_ATTR_FAST_PATH and MICROPY_OPT_MAP_LOOKUP_CACHE options:
- are simpler;
- take less code size;
- are faster (generally);
- work with code generated by the native emitter;
- can be used on embedded targets with a small and constant RAM overhead;
- allow the same .mpy bytecode to run on all targets.
See #7680 for further discussion. And see also #7653 for a discussion
about simplifying mpy-cross options.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This compiler is unable to optimise out the giant strcmp match generated
by MP_MATCH_COMPRESSED.
See github.com/micropython/micropython/pull/7659#issuecomment-899479793
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
It reschedules the BT HCI poll soft timer so that it is called exactly when
the next timer expires.
Signed-off-by: Damien George <damien@micropython.org>
Also known as L2CAP "connection oriented channels". This provides a
socket-like data transfer mechanism for BLE.
Currently only implemented for NimBLE on STM32 / Unix.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This requires that the event handlers are called from non-interrupt context
(i.e. the MicroPython scheduler).
This will allow the BLE stack (e.g. NimBLE) to run from the scheduler
rather than an IRQ like PENDSV, and therefore be able to invoke Python
callbacks directly/synchronously. This allows writing Python BLE handlers
for events that require immediate response such as _IRQ_READ_REQUEST (which
was previous a hard IRQ) and future events relating to pairing/bonding.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Support building .cpp files and linking them into the micropython
executable in a way similar to how it is done for .c files. The main
incentive here is to enable user C modules to use C++ files (which are put
in SRC_MOD_CXX by py.mk) since the core itself does not utilize C++.
However, to verify build functionality a unix overage test is added. The
esp32 port already has CXXFLAGS so just add the user modules' flags to it.
For the unix port use a copy of the CFLAGS but strip the ones which are not
usable for C++.
This commit adds support for using Bluetooth on the unix port via a H4
serial interface (distinct from a USB dongle), with both BTstack and NimBLE
Bluetooth stacks.
Note that MICROPY_PY_BLUETOOTH is now disabled for the coverage variant.
Prior to this commit Bluetooth was anyway not being built on Travis because
libusb was not detected. But now that bluetooth works in H4 mode it will
be built, and will lead to a large decrease in coverage because Bluetooth
tests cannot be run on Travis.
Previously the interaction between the different layers of the Bluetooth
stack was different on each port and each stack. This commit defines
common interfaces between them and implements them for cyw43, btstack,
nimble, stm32, unix.
