GitLab

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

* Added test for issue #1823

* Do not assume that version in platforms.txt is correct semver

* When parsing version in platform.txt perform a variable replace

* Update internal/integrationtest/core/testdata/issue_1823/DxCore-dev/megaavr/README.md

* Update internal/integrationtest/core/core_list_test.go
Co-authored-by: per1234 <accounts@perglass.com>

* testsuite: Added helper functions to handle test envs

* testsuite: Added helper functions to run arduino-cli

* testsuite: Added colored output to arduino-cli Run helper

* testsuite: Pass cli config through env variables

* testsuite: added env commands to download and extract files

* testsuite: added commands to start cli daemon and run some gRPC calls

* testsuite: moved test harness inside 'internal' package

* testsuite: added first daemon test for gRPC board watch

* testsuite: added http server helper

* testsuite: added JSON helpers

* testsuite: Added possibility to use shared download staging folder

* testsuite: Converted a core_test.py test (WIP)

* REMOVEME: Deactivate daemon integration test for now

* testsuite: force colored output

* testsuite: moved all generic subroutines into their own library

* testsuite: moved daemon test in his own dir

* fixed typo

* testsuite: added some helpers to improve daemon testing

* testsuite: added test for #1614

* Removed converted test

* testsuite: perform build before test

Otherwise people running the task locally will have a bad time when the test
either fails due to the missing executable, or far worse when they get
incorrect test results because they don't realize it is using whichever random
executable happened to be sitting around from the last time they did a build.

* Added missing comment

* Renamed test file

* Skip failing tests

* Updated licensed cache

* re-enable test for fixed bug

* testsuite: disable postinstall

Otherwise Windows CI will get stuck.

* Removed useless startDaemon

* Renamed inst -> grpcInst

* Close daemon gRPC connection on test cleanup

* Added comment

* Apply suggestions from code review
Co-authored-by: per1234 <accounts@perglass.com>

* Update internal/integrationtest/arduino-cli.go
Co-authored-by: per1234 <accounts@perglass.com>
Co-authored-by: per1234 <accounts@perglass.com>

`github.com/arduino/arduino-cli/i18n.Tr` provides a `fmt.Printf`-style interface, where the first argument may serve as
a template filled by subsequent arguments according to its format verbs.

This call contains a subsequent argument, but no verb in the format string to use it. This generates some cryptic
content in the error message.

For example:

Error during install: Error downloading tool arduino:bossac@1.7.0-arduino3: no versions available for the current OS%!(EXTRA *cores.ToolRelease=arduino:bossac@1.7.0-arduino3)

The tool name is already provided in the correctly configured `Message` field of the error, so there is no need to
include it in the `Cause` field as well, so the spurious argument can be removed entirely rather than the alternative
fix of adjusting the format string argument. I suspect the spurious argument was simply the result of a copy/paste error.

* Made PackageManager.TempDir private

* Merged InstallToolRelease function into the proper packagemanager method

* Moved uninstallToolRelease into the proper packagamanger method

* Moved uninstallPlatformRelease into the proper packagamanger method

* Moved downloadToolRelease into the proper packagamanger method

* Merged downloadTool with the proper packagamanger method

* Merged downloadPlatform with the proper packagamanger method

* Moved installPlatform into a packagamanger method

* Moved upgradePlatform into a packagamanger method

* Made PackageManager.Log field private

* Removed the massive code duplication in the 'upgrade' command

* Removed the massive code duplication in the 'outdated' command

* Updated docs

* Update arduino/cores/packagemanager/install_uninstall.go
Co-authored-by: per1234 <accounts@perglass.com>

* Update arduino/cores/packagemanager/install_uninstall.go
Co-authored-by: per1234 <accounts@perglass.com>
Co-authored-by: per1234 <accounts@perglass.com>

* Bugfix: do not overwrite board properies during compile

Fix #1614

* Removed AddBuildPropertyIfMissinc 'legacy' (inlined in other functions)

* Increase code coverage

Versions of the `codecov/codecov-action` GitHub Actions action prior to 1.0.6 required the use of a token provided by
Codecov in order to upload data to Codecov. This token was stored in secret in the Arduino CLI repository and used in
the test workflow.

For security reasons, secrets are not accessible when a workflow is triggered by an event generated by a fork of the
repository. This meant that it was impossible to upload coverage data for the test runs triggered by PRs from forks. A
conditional was added to the upload step of the workflow to cause it to only run on `push` event triggers, which
effectively prevented its failure for runs on PRs from forks.

The token requirement was removed in the 1.0.6 release of `codecov/codecov-action`, but the now pointless conditional
was never removed from the workflow. This prevented PRs from forks from receiving the automated code coverage report
comments that would otherwise encourage those contributors to resolve coverage deficiencies and facilitate the review
process.

The harmful conditional is hereby removed from the coverage data upload steps of the workflow and PRs from forks will
now receive coverage report comments, just as PRs from branches do already.

* Added tests for #1669

* Fixed regession in discoveries startup

Fix #1669

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

This is done by providing the desired revision in the fragment, e.g.
`…/Library.git#0.1.0`. When set, this disables the clone depth limit so
all remote references will be available.

* packagemanager: Show size difference between package_index and downloaded file

This will help identify the problem:
- if the file is corrupted
- if the package_index is wrong
- if there is a bug in the CLI (more rare... but actually possible :-)

* Load package indexes after installed packages

The installed packages keeps an extract of the original
package_index.json, at the moment of the installation, inside
a file named installed.json. This extract may turn out useful
if the original package_XXX_index.json is lost or removed to keep the
platform functional. On the other hand, if the original
package_XXX_index.json is modified the information kept in the
installed.json may be outdated and should be replaced by the upstream
index: this is the reason why it's loaded after the hardware platforms.

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

* Fix path in error messages on sketch loading

* Added integration tests

* Updated unit-test

* Updated integration test

* Even better error messages

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

* Added skip-libraries-discovery flag in Compile

* Always send compile response even in case of error

* Renamed variable

* Moved 'alread-installed' check in the proper place

* Added --no-overwrite flag in core install

* Fixed message

* Added tests

* Update cli/core/install.go
Co-authored-by: per1234 <accounts@perglass.com>

* Update rpc/cc/arduino/cli/commands/v1/core.proto
Co-authored-by: per1234 <accounts@perglass.com>
Co-authored-by: per1234 <accounts@perglass.com>

* Added no_overwrite field in LibraryInstall

* Added tests

* Check if libraries needs to be installed prior to download/install

* Update rpc/cc/arduino/cli/commands/v1/lib.proto
Co-authored-by: per1234 <accounts@perglass.com>
Co-authored-by: per1234 <accounts@perglass.com>

* Provide a way to define extra UserAgent fields via env vars

* Add secret token to User Agent

This will allow to possibly bypass spam-checks on the server side
reducing the jobs failures due to rate limiting.

[breaking] Fix gRPC streaming synchronization on close (should fix missing lines at the end of compile) (#1787)

* Correctly synchronize streams (Fix: some lines cut-off at the end of the output)

* Updated docs

* Fixed go-docs

* Use pipes with channel buffering to smooth out communications

* Update docs/UPGRADING.md
Co-authored-by: per1234 <accounts@perglass.com>
Co-authored-by: per1234 <accounts@perglass.com>

Arduino libraries may specify dependencies on other libraries in their metadata. The Arduino Library Manager offers the
option to install these dependencies when the user installs the library.

By default, the latest version of the dependency is installed. However, libraries may be compatible or tested working
only with specific versions of a dependency. In this case, the library developer may wish to specify a particular
version or version range of the dependency.

Arduino CLI and the Arduino Library index system has had support for such dependency version constraints for the last
2.5 years, but it was never documented so has only been used by the few library developers who discovered it via other
means.

The dependencies version constraint system is hereby fully documented in the Arduino library specification. This will
allow meaningful benefit to finally come from the work done to add this valuable capability.

* [skip changelog] Revert "codecov: Disable pull request comments"

This reverts commit 5c9eee34.

Codecov will now automatically comment on pull requests to provide a report of the code coverage impact that would
result from the proposed changes.

These comments are valuable for several reasons:

- Brings test deficiencies to the contributor's attention
- Encourages contributors to improve PRs that originally had insufficient test coverage for the added code
- Facilitates the review process for the maintainers
- Increases awareness of code coverage trends in the project (the Codecov website is great for this, but likely rarely visited)

* [skip changelog] Add support to "Test Go" workflow for alternative Codecov configuration paths

For the sake of efficiency, the "Test Go" GitHub Actions workflow is configured to run only when relevant files are
modified. Since the workflow uploads code coverage data to Codecov, the Codecov configuration file is one of these files.

The standard filename for the Codecov configuration file is codecov.yml, and the workflow's path filter was configured
for that filename. It turns out an alternative filename is also recognized: .codecov.yml, which is used by this project.
Two subfolders are also supported in addition to the root of the repository as locations for the configuration file.

The workflow's paths filter was not configured for the alternative filename and locations, meaning the workflow would
not be triggered on change to the Codecov configuration in projects that use the alternative configuration file name or
locations.

The workflow's paths filter is hereby configured to recognize changes to any valid Codecov configuration file.

During a refactoring of the "Publish Nightly Build" workflow, globbing of the macOS build artifact filename in the
repackaging step was accidentally lost, causing the glob pattern to be treated instead as a string and the updated
package saved to a file named by that string instead of to the original filename.

This resulted in the workflow failing with errors like:

Artifact path is not valid: /arduino-cli_nightly-*macOS_64bit.tar.gz. Contains the following character: Asterisk *

The previous `basename` command was providing the globbing. After the refactoring, `basename` is no longer needed, so an
alternative way to achieve globbing is needed. It seems the preferred approach is use of an array. Since the globbing
will only expand to a single filename, the array can be referenced as before by the rest of the step, since it will
resolve to the first element in the array (equivalent to `$PACKAGE_FILENAME[0]`).

* add support for macos arm64 build in the taskfile

* update workflows to handle darwin_arm64 notarization

- the gon config is now hardcoded in the workflow (it allows customization)
- the notarization step is run in parallel now
- the updated checksum is passed to the `create-release` job
- the `create-release` job handles the checksum update in `checksum.txt` file

* use working-directory, this should simplify a bit the workflow

* we cannot upload all the artifacts since the job is being parallelized

* Apply suggestions from code review
Co-authored-by: per1234 <accounts@perglass.com>

* add macos arm 64 download links in `installation.md`

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

[breaking] Remove old gRPC Monitor service, in favor of new gRPC API based on pluggable monitor (#1504)

* Removed old Monitor gRPC service

* Updated UPGRADING.md

* Fixed broken links

* Updated licensed cache

* Rate limit the number of outgoing gRPC messages

This change should allow a better buffering of the outgoing gRPC
messages (less messages with bigger data blocks -> less fragmentation).
This should allow the clients (IDE) to better handle incoming data.

* Update arduino/utils/stream.go
Co-authored-by: Umberto Baldi <34278123+umbynos@users.noreply.github.com>
Co-authored-by: Umberto Baldi <34278123+umbynos@users.noreply.github.com>

* Bump actions/setup-python from 3 to 4

Bumps [actions/setup-python](https://github.com/actions/setup-python) from 3 to 4.
- [Release notes](https://github.com/actions/setup-python/releases)
- [Commits](https://github.com/actions/setup-python/compare/v3...v4)

---
updated-dependencies:
- dependency-name: actions/setup-python
  dependency-type: direct:production
  update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>

* Use non-version specific reference URLs for tool version workflow variables

GitHub Actions actions are used by the workflows to set up development tools in the runner workspace.

In order to facilitate updates to new versions of these tools, we set the version to be set up via environment variables
at the top of the workflow.

Since this variable definition is separate from the step using the action, it might not be immediately apparent to the
maintainer which version syntaxes are supported. For this reason, comments were added with the URL to the relevant
section of the consuming action's documentation. Previously, these URLs were made to point to the version of the
documentation that matched the version of the action in use by the workflow. Since we only use a major version ref, the
expectation was that this would only need to be updated rarely. However, it turned out that the major version bump cycle
is significantly shorter than expected. In addition, it is easy to forget the update because action version update PRs
are provided by Dependabot, which obviously won't update the URLs in the comments.

So it will be best to use a URL that points to the documentation at the tip of the default branch of the action
repository. The likelihood of the documentation provided by this URL not matching the behavior of the release version of
the action in use is likely less than it is for an outdated URL.
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: per1234 <accounts@perglass.com>

Fix #1755

Fix #1738

Fix #1653

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>