GitLab

This commit adds many new sections to the existing "Developing and building
MicroPython" chapter to make it all about the internals of MicroPython.

This work was done as part of Google's Season of Docs 2020.

It's not necessary to fetch all branches.
Signed-off-by: Damien George <damien@micropython.org>

Changes are:
- Use ubuntu-20.04 so that gcc-multilib installs without error.
- Use "fetch-depth: 100" to get history prior to pull request.
Signed-off-by: Damien George <damien@micropython.org>

* Fix a typo in the Makefile that prevented the debug build to be actually
  enabled when BTYPE=debug is used.

* Add a missing header in modmachine.c that is used when a debug build is
  created.

This commit improves some FTP implementation details for better
compatibility with FTP clients:

* The PWD command now puts quotes around the directory name before
  returning it.  This fixes BBEdit’s FTP client, which performs a PWD after
  each CWD and gets confused if the returned directory path is not
  surrounded by quotes.

* The FEAT command is now allowed before logging in. This fixes the lftp
  client, which send FEAT first and gets confused (tries to use TLS) if the
  server responds with 332.

With MICROPY_FLOAT_IMPL_FLOAT the results of utime.time(), gmtime() and
localtime() change only every 129 seconds.  As one consequence
tests/extmod/vfs_lfs_mtime.py will fail on a unix port with LFS support.

With this patch these functions only return floats if
MICROPY_FLOAT_IMPL_DOUBLE is used.  Otherwise they return integers.

According to documentation time() has a precision of at least 1 second.
This test runs for 2.5 seconds and calls all utime functions every 100ms.
Then it checks if they returned enough different results.  All functions
with sub-second precision will return ~25 results.  This test passes with
15 results or more.  Functions that do not exist are skipped silently.

Currently supports only polling read and write.
Signed-off-by: Yonatan Schachter <yonatan.schachter@gmail.com>

This allows sending arbitrary HCI commands and getting the response.  The
return value of the function is the status of the command.

This is intended for debugging and not to be a part of the public API, and
must be enabled via mpconfigboard.h.  It's currently only implemented for
NimBLE bindings.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>

To match the definition of GENERATE_PACK_DFU, so a board can customise the
location/name of this file if needed.
Signed-off-by: Damien George <damien@micropython.org>

Signed-off-by: Damien George <damien@micropython.org>

To have at least one board configured with MBOOT_ENABLE_PACKING, for CI
testing purposes and demonstration of the feature.
Signed-off-by: Damien George <damien@micropython.org>

This commit adds support to stm32's mboot for signe, encrypted and
compressed DFU updates.  It is based on inital work done by Andrew Leech.

The feature is enabled by setting MBOOT_ENABLE_PACKING to 1 in the board's
mpconfigboard.mk file, and by providing a header file in the board folder
(usually called mboot_keys.h) with a set of signing and encryption keys
(which can be generated by mboot_pack_dfu.py).  The signing and encryption
is provided by libhydrogen.  Compression is provided by uzlib.  Enabling
packing costs about 3k of flash.

The included mboot_pack_dfu.py script converts a .dfu file to a .pack.dfu
file which can be subsequently deployed to a board with mboot in packing
mode.  This .pack.dfu file is created as follows:
- the firmware from the original .dfu is split into chunks (so the
  decryption can fit in RAM)
- each chunk is compressed, encrypted, a header added, then signed
- a special final chunk is added with a signature of the entire firmware
- all chunks are concatenated to make the final .pack.dfu file

The .pack.dfu file can be deployed over USB or from the internal filesystem
on the device (if MBOOT_FSLOAD is enabled).

See #5267 and #5309 for additional discussion.
Signed-off-by: Damien George <damien@micropython.org>

Prior to this fix, the final piece of data in a compressed file may have
been lost when decompressing.
Signed-off-by: Damien George <damien@micropython.org>

This library is a small and easy-to-use cryptographic library which is well
suited to embedded systems.
Signed-off-by: Damien George <damien@micropython.org>

Without this fix, each time service registration happened it would do an
increasingly large malloc() for service state.

See https://github.com/apache/mynewt-nimble/issues/896.

Notably git-cmd which comes with git installations on Windows alters the
encoding resulting in CPython tracing encodings/cp1252.py calls.

The original logic of reducing a full path to a relative one assumes
"tests/misc" is in the filename which is limited in usage: it never works
for CPython on Windows since that will use a backslash as path separator,
and also won't work when the filename is a path not relative to the tests
directory which happens for example in the common case of running
"./run-tests -d misc".

Fix all cases by printing only the bare filename, which requires them all
to start with sys_settrace_ hence the renaming.

Changes are:
- Remove include of stm32's adc.h because it was recently changed and is
  no longer compatible with teensy (and not used anyway).
- Remove define of __disable_irq in mpconfigport.h because it was clashing
  with an equivalent definition in core/mk20dx128.h.
- Add -Werror to CFLAGS, and change -std=gnu99 to -std=c99.
Signed-off-by: Damien George <damien@micropython.org>

Mboot builds do not use the external SPI flash in caching mode, and
explicitly disabling it saves RAM and a small bit of flash.
Signed-off-by: Damien George <damien@micropython.org>

This only needs to be enabled if a board uses FAT FS on external SPI flash.
When disabled (and using external SPI flash) 4k of RAM can be saved.
Signed-off-by: Damien George <damien@micropython.org>

When littlefs is enabled extended reading must be supported, and using this
function to read the first block for auto-detection is more efficient (a
smaller read) and does not require a cached SPI-flash read.
Signed-off-by: Damien George <damien@micropython.org>

Signed-off-by: Damien George <damien@micropython.org>

These functions enable SDRAM data retention in stop mode.  Example usage,
in mpconfigboard.h:

    #define MICROPY_BOARD_ENTER_STOP sdram_enter_low_power();
    #define MICROPY_BOARD_LEAVE_STOP sdram_leave_low_power();

Calculate the bit timing from baudrate if provided, allowing sample point
override.  This makes it a lot easier to make CAN work between different
MCUs with different clocks, prescalers etc.

Tested on F4, F7 and H7 Y/V variants.

This commit prevents uos.mount() from raising an AttributeError.
vfs_autodetect() is supposed to return an object that has a "mount" method,
so if no filesystem is found it should raise an OSError(ENODEV) and not
return the bdev itself which has no "mount" method.

Mounting a bdev directly tries to auto-detect the filesystem and if none is
found an OSError(19,) should be raised.

The fourth parameter of readblocks() and writeblocks() must be optional to
support ports with MICROPY_VFS_FAT=1.  Otherwise mounting a bdev may fail
because looking for a FATFS will call readblocks() with only 3 parameters.

Since CPython 3.8 the optional "sep" argument to hexlify is officially
supported, so update comments in the code and the docs to reflect this.
Signed-off-by: Damien George <damien@micropython.org>

Signed-off-by: Damien George <damien@micropython.org>

As a general pattern, required positional arguments that are not named do
not need to be parsed using mp_arg_parse_all().
Signed-off-by: Damien George <damien@micropython.org>

Because the version included in xtensa-lx106-elf-standalone.tar.gz needs
Python 2 (and pyserial for Python 2).
Signed-off-by: Damien George <damien@micropython.org>

Because the setup functions may print other information which should not be
added to the path.
Signed-off-by: Damien George <damien@micropython.org>

Signed-off-by: Damien George <damien@micropython.org>

This much buffer space is required for CDC data out endpoints to avoid any
buffer overflows when the USB CDC is saturated with data.
Signed-off-by: Damien George <damien@micropython.org>

Changes in this new library version are:
- Update WB HAL to v1.10.0.
Signed-off-by: Damien George <damien@micropython.org>

To reduce code size, since mboot does not modify the filesystem.
Signed-off-by: Damien George <damien@micropython.org>

To prevent warnings about this function being unused when assertions are
disabled.
Signed-off-by: Damien George <damien@micropython.org>