GitLab

* Clean up some useless constants
* made legacy/builder.go independent from legacy/constants
* Now 'compile --only-compilation-database' will run all pre-* hooks

Some platforms make sketch preparation see #1547

Fix #1547
Co-authored-by: per1234 <accounts@perglass.com>

This method allows to bound process execution to a context.

Unlike the IDE and Web Editor, Arduino CLI is intended to be used directly only by advanced users. However, all types of
users are likely to visit the repository and documentation website. Some of these readers will not be familiar with the
meaning of the acronym "CLI".

The previous documentation introduces Arduino CLI as:

> an all-in-one solution that provides [wonderful things] to use any Arduino compatible board and platform

Which sounds like something every Arduino user would be interested in.

Those readers might therefore get the impression this is some sort of GUI application like the IDE and be frustrated and confused
after spending time to install and run the program, only to find that it doesn't apparently do anything when they run the
executable (because they did it from a file browser or shortcut).

It is a bit difficult to describe Arduino CLI's interface in a short introduction because, despite the "CLI" in the name,
command line is only one of the interfaces offered by Arduino CLI (the others being the gRPC and Go APIs). I added the
term "machine" to cover the others.

* Add an introduction to install docs

This document provides a list of installation options. The user may pick any one option that is most convenient for them.
However, a reader might get the impression that it is a list of instructions that must be followed from top to bottom.

An introduction makes the situation immediately clear to the reader.

* Clarify that sh is not required in template install docs

A significant portion of Arduino users use Windows. The first option listed in the installation instructions is for an
installation shell script, which can be used on Windows, but might not be convenient for some Windows users.

This could be discouraging or frustrating to potential new users, but not necessarily so because the script is an
optional convenience and directly downloading and installing the tool is also an option, as documented in the
instructions for the second listed option.

A little refinement of the wording of the script's `sh` requirement makes the situation clear to those users.

* Add a reference link re: PATH to install docs

A reference link provides easy access to the relevant information for readers who are not familiar with this concept
while not harming readability for everyone else by increasing verbosity.

* Align heading levels in install docs

This document is intended to provide a list of installation options:

- brew
- script
- download
- build

but the heading levels were misaligned, which resulted in this structure:

-brew
- script
  - download
  - build

* fix packages being added to the package manager even if they were not respecting the specification

* add test to check if if the bug it's fixed

* fix platform release being nil and causing panic with indexes not compliant with specs

* add test to check if the bug is fixed

* add comments and optimize the code a little bit

* Apply suggestions from code review
Co-authored-by: per1234 <accounts@perglass.com>
Co-authored-by: per1234 <accounts@perglass.com>

* Split CI execution of integration tests

* Make Github workflows runners cry

* Different approach

* Moved errors to arduino package

* Handled package name change cause of errors move

* Renamed some errors

* Add CI workflow to check for unapproved Go dependency licenses

A task and GitHub Actions workflow are provided here for checking the license types of Go project dependencies.

On every push and pull request that affects relevant files, the CI workflow will check:

- If the dependency licenses cache is up to date
- If any of the project's dependencies have an unapproved license type.

Approval can be based on:

- Universally allowed license type
- Individual dependency

* Make initial commit of dependency license metadata

The  folder contains a cache of license metadata for all the project's Go dependencies. This serves two purposes:

- Allow the Licensed dependency license checker tool to only check licenses when a dependency is added or updated

- Allow the maintainer to manually define license metadata when the licensee tool is unable to automatically detect it

* Manually define dependency license metadata that was not detected

The "Licensed" dependency license checker tool uses the licensee tool to automatically determine the license type based
on metadata provided by the dependency author. This must be in a standardized format without any modifications. In cases
where that wasn't done, it is necessary to determine the license type and update the dependency license metadata cache in
the `.licenses` folder manually.

The Licensed tool will check this data whenever the dependency version is updated to make sure the license hasn't changed.

* Fixed network upload for avr packages

* Added test for serial pluggable-monitor compatibility conversion

* Added data for unit testing

* Fixed other collateral tests

* Perform regexp compile at package level

* fix i18n

* [skip changelog] Standardize license file

Standardization in license documentation is important because, in addition to making it easy for humans to find this vital information, it allows machines to automate the process of license type determination, which is useful both for discovering suitable open source projects as well as checking open source license compliance.

The open source license of Arduino CLI is already stored in a standardized location at `/LICENSE.txt`. However, even though Arduino CLI is licensed under the industry standard GPL 3.0 open source license, additional text was added to the license file which offers the option to purchase an exception for proprietary use of the code. Even though this offer does not have any legal effect on the GPL 3.0 license, it does make it so that the license file can no longer be identified with 100% confidence by machines as GPL 3.0, which means a human must make that identification instead.

Since there is no need to place the exception offer in the license file, it can be moved to the readme and the license file left as the verbatim GPL 3.0 license text, as looked for by the license type detection tools such as licensee.

* Add CI workflow to check the license file

Whenever one of the recognized license file names are modified in the repository, the workflow runs to check whether the license can be recognized and whether it is of the expected type.

GitHub has a useful automated license detection system that determines the license type used by a repository, and surfaces that information in the repository home page, the search web interface, and the GitHub API. This license detection system requires that the license be defined by a dedicated file with one of several standardized filenames and paths.

GitHub's license detection system uses the popular licensee tool, so this file also serves to define the license type for any other usages of licensee, as well as to human readers of the file.

For this reason, and to ensure it remains a valid legal instrument, it's important that there be no non-standard modifications to the license file or collisions with other supported licence files. This workflow ensures that any changes which would change the license type or which license file is used by the detection are caught automatically.

* Free contributors from i18n data sync responsibility

* Update .github/workflows/check-i18n-task.yml
Co-authored-by: per1234 <accounts@perglass.com>
Co-authored-by: per1234 <accounts@perglass.com>

* add completion for `-b` or `--fqbn` for every command

* add completion for `-l` or `--protocol`
But does only work for connected boards and do not list every protocol installed

* the previous implementation was working only with a board connected to the pc
RPC was not exposing that functionality

* add static completion for `--log-level, `--log-format` and `--format`

* add completion for `-P` or `--programmer` & fix typo

* add completion for `core uninstall`
maybe could be done better (filter and remove the manually installed ones)

* add completion for `core install` and `core download`

* add completion for `lib uninstall`

* add completion for `lib install`, `lib download`

* add completion for `lib examples`

* add completion for `config add`, `config remove`, `config delete` and `config set`

* add completion for `lib deps`

* add tests

* enhance the completion for `config add` and `config remove`

* add description completion suggestion for core, lib, fqbn, programmer

* add completion also for `-p` or `--port` flag

* remove the `toComplete` parameter from all the completion functions
as of now this parameter is useless because if a string is typed in the terminal it cannot be swapped with a different one.
For example if I write `arduino-cli compile -b avr<TAB><TAB>` the completion function returns all elements starting with `arduino:avr...`.
So the completions are not showed because they cannot be swapped with a string that starts differently.
The only shell which seems to support this seems to be zsh

* fixes after rebase

* update docs

* add `-b` or `--fqbn` completion for the monitor command and tests

* apply suggestions from code review

* fixes after rebase

[skip-changelog] Drop `go.rice` library in favor of golang native `embed` (available from go1.16) (#1513)

* Removed 'go.rice' library in favor of 'embed'

* fix i18n

* #1456 - Add "new sketch" in gRPC .proto, regenerate .pg.go. Use "user" dir for sketches created via gRPC

* #1456 - Avoid using hardcoded extension (address comment).

* #1456 - Add optional dir field in the request, use it.

* #1456 - Use go-paths-helper

* Use gRPC function from CLI to create new sketch
Co-authored-by: Silvano Cerza <silvanocerza@gmail.com>

In the event an Arduino boards platform is deprecated by the maintainer, it will be useful to clearly communicate this
fact to the user and ensure they will easily find a preferred alternative platform. In order to facilitate this, a
`packages[*].platforms[*].deprecated`field was added to the Arduino package index format and this information is used to
enhance the user interfaces of the official Arduino development software.

This useful feature is now documented in the Arduino package index specification.

* [skip changelog] Correct context key name in "Sync Labels" workflow

Incorrect context key name resulted in impossible to satisfy conditional, meaning the dry run determination code was
solely dependent on the check for whether the workflow was triggered from the default branch name.

* [skip changelog] Sync labels in write mode on schedule trigger

In order to facilitate the testing and review of proposed changes to the repository label infrastructure, the
"Sync Labels" template workflow does a dry run when triggered under conditions that indicate it would not be appropriate
to make real changes to the repository's labels. The changes that would have resulted are printed to the log, but not
actually made.

One of the criteria used to determine "dry run" mode usage is whether the event occurred on the repository's default
branch. A trigger on a development branch or for a pull request should not result in a change to the labels.
It turns out that GitHub does not define a `github.event.repository.default_branch` context item when a workflow is
triggered by a `schedule` event. This resulted in the workflow always running in "dry run" mode on a `schedule` trigger.
Since `schedule` and `repository_dispatch` triggers are only permitted for the default branch, there is no need to check
whether the event's ref matches the default branch and it is safe to always run in write mode on these events.

Add gRPC interface function to request user fields supported by a board/protocol combination (#1503)

Add support for pluggable monitor (tracks branch `feature/pluggable-monitor`)

* Add --fqbn parameter to monitor command

* Add support for pluggable monitors with explicit recipe in platform.txt

* increase logging

* fix i18n

* Factored pluggable monitor creation

* Factored out code duplication

* EnumerateMonitorPortSettings does require only protocol

* Monitor.Open does require only address and protocol

* Added GetPortAddressAndProtocol helper

* fix i18n

* Apply suggestions from code review
Co-authored-by: Silvano Cerza <3314350+silvanocerza@users.noreply.github.com>

People might get the impression that, since the documentation is in the Arduino CLI repo, it only applies to Arduino CLI.
However, since Arduino CLI is used in some form in most of Arduino's development tools, much of the information is
universal.

* Added documentation for pluggable monitor

* Apply suggestions from code review
Co-authored-by: per1234 <accounts@perglass.com>

* Add pluggable monitor spec in nav bar; fixed link in source code comments

* fix i18n and lint warnings
Co-authored-by: per1234 <accounts@perglass.com>

* Added silent mode for monitor

* Added --config flag to monitor command

* fix i18n

* Applied review suggestions

* rename --silent to --quiet
* removed useless 'settings' variable
* translation string for "Monitor port settings" in CLI mode

* Implementation of Monitor business logic

* Implementation of 'monitor' command of the CLI

* fix i18n

* Update commands/monitor/monitor.go
Co-authored-by: Silvano Cerza <3314350+silvanocerza@users.noreply.github.com>
Co-authored-by: Silvano Cerza <3314350+silvanocerza@users.noreply.github.com>

* Resolve monitors from platform.txt

* Implementation of monitor --describe command

* fix i18n

* Added new Monitor command (will replace current Monitor service)

* Added new EnumerateMonitorPortParameters gRPC call

* Force compile error on gRPC deamon when there are unimplemented methods

* Updated deamon implementation stub

* fix i18n

* Added bundled instance of serial-monitor

* Download serial-monitor together with other bundle tools

* fix i18n

* Fixed race condition in pluggable monitor

Many methods had the stateLock held for the entire scope of the function
call but the 'port_closed' message can be received at any moment,
asyncronously, and it requires a stateLock as well. In this case the
worst case is that the decode loop is blocked for 10 seconds until the
timeout occurs, but this is not ideal.

This bug has been fixed by removing the state, since it's not really
useful.

* Improved message logging in pluggable monitor

* Refactored message processing in pluggable monitor

* Fix i18n

* fixed lint suggestion

* fix from code review

* Addde QUIT command in monitor-client

* Use rpc.Port in pluggable monitor Open method

* Slightly improved monitor client error messages

* Fixed resource leaking in PluggableMonitor client

* fixed i18n

* Fixed test... ooops

A bug introduced in the 3.9.7 release of Python causes a spurious failure of the
`test\test_lib.py::test_install_git_invalid_library` integration test:
https://bugs.python.org/issue45121

As a workaround, the last working version of Python must be used: 3.9.6. In order to ensure this is done when running the
integration tests locally, this version is specified in the Poetry configuration. Even though only the integration tests
require this version limitation, the Poetry configuration change affects all processes that use Poetry. Since only the
integration test workflow was adjusted accordingly, the other workflows using Poetry continued to explicitely install
a version of Python that did not fulfill the version constraints specified in the Poetry configuration. It seems it was
possible for Poetry to find a compatible version on the runner:

> The currently activated Python version 3.9.7 is not supported by the project (>=3.8, <3.9.7).
> Trying to find and use a compatible version.
> Using python3.8 (3.8.10)

but this additional output caused a failure of the "Deploy Website" workflow due to expecting only valid JSON as the
output from the siteversion.py script:

```
Error: The template is not valid. Newtonsoft.Json.JsonReaderException: Unexpected character encountered while parsing value: T. Path '', line 0, position 0.
```

Even though the other workflows were able to continue to work with the discovered Python 3.8.10, it seems safest avoid
reliance on whatever Python the runner happens to have installed and instead explicitly install the Python version we
want to be used. So the full pin of Python is done in all workflows that use Poetry.

Since it is convenient to get automatic updates for Python patch releases, this full pin should be reverted back to the
"3.9" minor version pin once a new version of Python is released with the bug fixed and added to versions available for
installation via the `actions/setup-python` GitHub Actions action.