GitLab

Prior to this commit, pyboard.py used eval() to "parse" file data received
from the board.  Using eval() on received data from a device is dangerous,
because a malicious device may inject arbitrary code execution on the PC
that is doing the operation.

Consider the following scenario:

Eve may write a malicious script to Bob's board in his absence.  On return
Bob notices that something is wrong with the board, because it doesn't work
as expected anymore.  He wants to read out boot.py (or any other file) to
see what is wrong.  What he gets is a remote code execution on his PC.

Proof of concept:

Eve:

  $ cat boot.py
  _print = print
  print = lambda *x, **y: _print("os.system('ls /; echo Pwned!')", end="\r\n\x04")
  $ ./pyboard.py -f cp boot.py :
  cp boot.py :boot.py

Bob:

  $ ./pyboard.py -f cp :boot.py /tmp/foo
  cp :boot.py /tmp/foo
  bin   chroot  dev  home  lib32  media  opt   root  sbin  sys  usr
  boot  config  etc  lib   lib64  mnt    proc  run   srv   tmp  var
  Pwned!

There's also the possibility that the device is malfunctioning and sends
random and possibly dangerous data back to the PC, to be eval'd.

Fix this problem by using ast.literal_eval() to parse the received bytes,
instead of eval().
Signed-off-by: Michael Buesch <m@bues.ch>

Prior to this commit, if you configure a pin as an output type (I2C in this
example) and then later configure it back as an input, then it will report
the type incorrectly.  Example:

    >>> import machine
    >>> b6 = machine.Pin('B6')
    >>> b6
    Pin(Pin.cpu.B6, mode=Pin.IN)
    >>> machine.I2C(1)
    I2C(1, scl=B6, sda=B7, freq=420000)
    >>> b6
    Pin(Pin.cpu.B6, mode=Pin.ALT_OPEN_DRAIN, pull=Pin.PULL_UP, af=Pin.AF4_I2C1)
    >>> b6.init(machine.Pin.IN)
    >>> b6
    Pin(Pin.cpu.B6, mode=Pin.ALT_OPEN_DRAIN, af=Pin.AF4_I2C1)

With this commit the last print now works:

    >>> b6
    Pin(Pin.cpu.B6, mode=Pin.IN)

Latest versions of Sphinx (at least 3.1.0) do not need the `*` escaped and
will render the `\` in the output if it is there, so remove it.

Fixes issue #6209.

Adds a job to build the zephyr port in CI using the same docker container
that the zephyr project uses for its own CI.

Always make clean zephyr builds to ensure we don't just rebuild C code, but
we also rebuild Kconfig and dts.  This is required when switching between
boards, which have different Kconfigs and device trees.

Uses the tagged zephyr 2.3.0 release.
Signed-off-by: Maureen Helm <maureen.helm@nxp.com>

Include storage/flash_map.h unconditionally so we always have access to the
FLASH_AREA_LABEL_EXISTS macro, even if CONFIG_FLASH_MAP is not defined.

This fixes a build error for the qemu_x86 board:

main.c:108:63: error: missing binary operator before token "("
  108 |     #elif defined(CONFIG_FLASH_MAP) && FLASH_AREA_LABEL_EXISTS(storage)
      |                                                               ^
../../py/mkrules.mk:88: recipe for target 'build/genhdr/qstr.i.last' failed
Signed-off-by: Maureen Helm <maureen.helm@nxp.com>

mp_reader_new_file() is used to read in files for importing, either .py or
.mpy files, for the lexer and persistent code loader respectively.  In both
cases the file should be opened in raw bytes mode: the lexer handles
unicode characters itself, and .mpy files contain 8-bit bytes by nature.

Before this commit importing was working correctly because, although the
file was opened in text mode, all native filesystem implementations (POSIX,
FAT, LFS) would access the file in raw bytes mode via mp_stream_rw()
calling mp_stream_p_t.read().  So it was only an issue for non-native
filesystems, such as those implemented in Python.  For Python-based
filesystem implementations, a call to mp_stream_rw() would go via IOBase
and then to readinto() at the Python level, and readinto() is only defined
on files opened in raw bytes mode.
Signed-off-by: Damien George <damien@micropython.org>

If mpy-cross exits with an error be sure to print that error in a way that
is readable, instead of a long bytes object.
Signed-off-by: Damien George <damien@micropython.org>

On ports where normal heap memory can contain executable code (eg ARM-based
ports such as stm32), native code loaded from an .mpy file may be reclaimed
by the GC because there's no reference to the very start of the native
machine code block that is reachable from root pointers (only pointers to
internal parts of the machine code block are reachable, but that doesn't
help the GC find the memory).

This commit fixes this issue by maintaining an explicit list of root
pointers pointing to native code that is loaded from an .mpy file.  This
is not needed for all ports so is selectable by the new configuration
option MICROPY_PERSISTENT_CODE_TRACK_RELOC_CODE.  It's enabled by default
if a port does not specify any special functions to allocate or commit
executable memory.

A test is included to test that native code loaded from an .mpy file does
not get reclaimed by the GC.

Fixes #6045.
Signed-off-by: Damien George <damien@micropython.org>

All imports are now tested to see if the test should be skipped,
UserFile.read is removed, and UserFile.readinto is made more efficient.
Signed-off-by: Damien George <damien@micropython.org>

These tests are specific to MicroPython so have a better home in the
micropython/ test subdir, and putting them here allows them to be run by
all targets, not just those that have access to the local filesystem (eg
the unix port).
Signed-off-by: Damien George <damien@micropython.org>

To make this code more portable, across different platforms.
Signed-off-by: Damien George <damien@micropython.org>

It raises on EOFError instead of an IncompleteReadError (which is what
CPython does).  But the latter is derived from EOFError so code compatible
with MicroPython and CPython can be written by catching EOFError (eg see
included test).

Fixes issue #6156.
Signed-off-by: Damien George <damien@micropython.org>

The SCSI driver calls GetCapacity to get the block size and number of
blocks of the underlying block-device/LUN.  It caches these values and uses
them later on to verify that reads/writes are within the bounds of the LUN.
But, prior to this commit, there was only one set of cached values for all
LUNs, so the bounds checking for a LUN could use incorrect values, values
from one of the other LUNs that most recently updated the cached values.
This would lead to failed SCSI requests.

This commit fixes this issue by having separate cached values for each LUN.
Signed-off-by: Damien George <damien@micropython.org>

MicroPython's original implementation of __aiter__ was correct for an
earlier (provisional) version of PEP492 (CPython 3.5), where __aiter__ was
an async-def function.  But that changed in the final version of PEP492 (in
CPython 3.5.2) where the function was changed to a normal one.  See
https://www.python.org/dev/peps/pep-0492/#why-aiter-does-not-return-an-awaitable
See also the note at the end of this subsection in the docs:
https://docs.python.org/3.5/reference/datamodel.html#asynchronous-iterators
And for completeness the BPO: https://bugs.python.org/issue27243

To be consistent with the Python spec as it stands today (and now that
PEP492 is final) this commit changes MicroPython's behaviour to match
CPython:  __aiter__ should return an async-iterable object, but is not
itself awaitable.

The relevant tests are updated to match.

See #6267.

Also remove trailing spaces on other lines.

On original pyboard UART 5 isn't available; added pyboard D availability.

Enabling the following features for all targets, except for nrf51
targets compiled to be used with SoftDevice:

- MICROPY_PY_ARRAY_SLICE_ASSIGN
- MICROPY_PY_SYS_STDFILES
- MICROPY_PY_UBINASCII

Splitting mpconfigport.h into multiple device specific
files in order to facilitate variations between devices.

Due to the fact that the devices might have variations in
features and also variations in flash size it makes sense
that some devices offers more functionality than others
without being limited by restricted devices.

For example more micropython features can be activated for
nrf52840 with 1MB flash, compared to nrf51 with 256KB.

Fixing 98e58343, the semantics of strncpy
require that the remainder of dst be filled with null bytes.
Signed-off-by: Damien George <damien@micropython.org>

This code is imported from musl, to match existing code in libm_dbl.

The file is also added to the build in stm32/Makefile.  It's not needed by
the core code but, similar to c5cc6417,
allows round() to be used by user C modules or board extensions.

Because the argument arrays may overlap, as show by the new tests in this
commit.

Also remove the debugging comments for these macros, add a new comment
about overlapping regions, and separate the macros by blank lines to make
them easier to read.

Fixes issue #6244.
Signed-off-by: Damien George <damien@micropython.org>

Signed-off-by: Damien George <damien@micropython.org>

Changes in this new library version are:
- Update H7 HAL to v1.6.0.
- Update WB HAL to v1.6.0.
- Add patches to fix F4 ll_uart clock selection for UART9/UART10.
Signed-off-by: Damien George <damien@micropython.org>

Signed-off-by: Kenneth Ryerson <kenneth.ryerson@gmail.com>

This explains how looping now works, and removes the warning about calling
wait_done().

A previous commit 3a9d9480 can cause
lock-ups of the RMT driver, so this commit reverses that, adds a loop_en
flag, and explicitly controls the TX interrupt in write_pulses().  This
provides correct looping, non-blocking writes and sensible behaviour for
wait_done().

See also #6167.

The stm32 and esp32 ports now use shorter error strings for mbedtls errors.
Also, MBEDTLS_ERROR_C is enabled on stm32 by default to get these strings.

The file `mbedtls_errors/mp_mbedtls_errors.c` can be used instead of
`mbedtls/library/error.c` to give shorter error strings, reducing the build
size of the error strings from about 12-16kB down to about 2-5kB.

This commit adds human readable error messages when mbedtls or axtls raise
an exception.  Currently often just an EIO error is raised so the user is
lost and can't tell whether it's a cert error, buffer overrun, connecting
to a non-ssl port, etc.  The axtls and mbedtls error raising in the ussl
module is modified to raise:

    OSError(-err_num, "error string")

For axtls a small error table of strings is added and used for the second
argument of the OSErrer.  For mbedtls the code uses mbedtls' built-in
strerror function, and if there is an out of memory condition it just
produces OSError(-err_num).  Producing the error string for mbedtls is
conditional on them being included in the mbedtls build, via
MBEDTLS_ERROR_C.

This commit adds the IRQ_GATTS_INDICATE_DONE BLE event which will be raised
with the status of gatts_indicate (unlike notify, indications require
acknowledgement).

An example of its use is added to ble_temperature.py, and to the multitests
in ble_characteristic.py.

Implemented for btstack and nimble bindings, tested in both directions
between unix/btstack and pybd/nimble.

Also clarify behavior of `gatts_notify` and add some TODOs about adding an
event for indication acknowledgement.

The goal of this commit is to allow using ble.gatts_notify() at any time,
even if the stack is not ready to send the notification right now.  It also
addresses the same issue for ble.gatts_indicate() and ble.gattc_write()
(without response).  In addition this commit fixes the case where the
buffer passed to write-with-response wasn't copied, meaning it could be
modified by the caller, affecting the in-progress write.

The changes are:

- gatts_notify/indicate will now run in the background if the ACL buffer is
  currently full, meaning that notify/indicate can be called at any time.

- gattc_write(mode=0) (no response) will now allow for one outstanding
  write.

- gattc_write(mode=1) (with response) will now copy the buffer so that it
  can't be modified by the caller while the write is in progress.

All four paths also now track the buffer while the operation is in
progress, which prevents the GC free'ing the buffer while it's still
needed.