GitLab

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

* Added test for os-specific config options

* Build board config options structures only once and cache them

* Board's build options properties are now calculated only once and cached

* Added tests for config options ordering

It required insertion of test data with the properties.Set method to
preserve ordering.

* Renamed some variables to improve code readability

* Added board config identification subroutines

* Added board config detection in 'commands.identify' function

* Updated docs

* Apply suggestions from code review
Co-authored-by: per1234 <accounts@perglass.com>

* Fixed comment
Co-authored-by: per1234 <accounts@perglass.com>

* add step to generate a windows msi installer of the Arduino CLI

* remove warning:

`warning : Solution properties are only available during IDE builds or when building the solution file from the command line. To turn off this warning set <DefineSolutionProperties>false</DefineSolutionProperties> in your .wixproj`

* fix error caused by wix peculiar version constraints:

`error CNDL0108: The Product/@Version attribute's value, '0.0.0-test', is not a valid version.  Legal version values should look like 'x.x.x.x' where x is an integer from 0 to 65534.`

* fix `error LGHT0103: The system cannot find the file 'arduino-cli.exe'`

* final adjustments in wix config

* exclude prettier from checking the installer config files, prettier does not support xml out of the box

* add download links

* temporarily disable s3 push and homebrew update for testing

* Upload nightly artifacts for testing

* add windows check certs

* bump to latest windows-sdk and document better

specify timestamp algorithm. Apparently this is required in the latest version of the sdk

* remove useless job id not used anymore (followup of #1883)

* fixed comments

* It's useless to use the output from setupmsbuild

* use YAML-style syntax

* try to uniform the usage of version/tag across the workflow

* Revert "Upload nightly artifacts for testing" && "temporarily disable s3 push and homebrew update for testing"

This reverts commit 3123dfd8ea0b2fa4d2fd33aa4ccd5696f34355d2.
This reverts commit 24fa25ca65be2d8f149729f3c169479e1848fa51.

* Apply suggestions from code review
Co-authored-by: per1234 <accounts@perglass.com>
Co-authored-by: per1234 <accounts@perglass.com>

* Update publish-go-tester-task

* Update publish-go-nightly-task

* Update release-go-task

* Migrate TestCompileSketchWithPdeExtension from test_compile_part_3.py to compile_part_3_test.go

* Migrate TestCompileSketchWithMultipleMainFiles from test_compile_part_3.py to compile_part_3_test.go

* Migrate TestCompileSketchCaseMismatchFails from test_compile_part_3.py to compile_part_3_test.go

A new function that sets the working directory to a specific path has been added in order to run all the tests as intended.

* Migrate TestCompileWithOnlyCompilationDatabaseFlag from test_compile_part_3.py to compile_part_3_test.go

* Migrate TestCompileUsingPlatformLocalTxt from test_compile_part_3.py to compile_part_3_test.go

* Migrate TestCompileUsingBoardsLocalTxt from test_compile_part_3.py to compile_part_3_test.go

* Migrate TestCompileManuallyInstalledPlatform from test_compile_part_3.py to compile_part_3_test.go

* Migrate TestCompileManuallyInstalledPlatformUsingPlatformLocalTxt from test_compile_part_3.py to compile_part_3_test.go

* Migrate TestCompileWithFullyPrecompiledLibrary to compile_part_3_test.go and delete test_compile_part_3.py

* Rearrange tests to have them share the same environment

Bumps [geekyeggo/delete-artifact](https://github.com/geekyeggo/delete-artifact) from 1 to 2.
- [Release notes](https://github.com/geekyeggo/delete-artifact/releases)
- [Commits](https://github.com/geekyeggo/delete-artifact/compare/v1...v2)

---
updated-dependencies:
- dependency-name: geekyeggo/delete-artifact
  dependency-type: direct:production
  update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

The original copy sketch function written in conftest.py copies sketches into the testsuite's working directory. Changing the copy sketch function present in the golang testsuite to match the original one,
should prevent errors that could occur while migrating tests.

* Move upload of coverage data to dedicated job in test workflow

The code coverage data generated by running the unit tests in the "Test Go" GitHub Actions workflow is uploaded to
Codecov.

This upload is prone to occasional transient failures. With the previous workflow configuration this caused a failure of
the Linux test run job. Contributors would need to review the logs to understand that these failures were not caused by
a test failing. The failure also causes the ongoing jobs that run the tests on other operating systems to be immediately
canceled, meaning their full test results are not available. The entire test suite must be reran just to attempt the
upload again.

Moving the upload to a dedicated workflow job makes it faster and easier for contributors to interpret the cause of a
test failure. It also allows the test suite to complete for all operating systems, making their results immediately
available to the contributor even when the coverage data upload fails. The coverage upload job can be reran, making
recovery from a transient failure more efficient.

* Use a single step to upload coverage data in "Test Go" workflow

The code coverage data generated by running the unit tests in the "Test Go" GitHub Actions workflow is uploaded to
Codecov. Arduino CLI's unit tests are split into two collections: those covering the modernized part of the codebase,
and those covering the "legacy" code inherited from arduino-builder. A separate code coverage data file is produced for
each of these collections.

The `codecov/codecov-action` GitHub Actions action is used to perform this upload. Previously, a separate step was used
for the upload of each of the code coverage files. The action provides the capability to specify multiple files for
upload in a comma-separated list. This approach will make the workflow more efficient and reliable.

* Migrated TestBoardList from test_board.py to board_test.go

* Migrated TestBoardListWithInvalidDiscovery from test_board.py to board_test.go

* Migrated TestBoardListall from test_board.py to board_test.go

* Migrated TestBoardListallWithManuallyInstalledPlatform from test_board.py to board_test.go

* Migrated TestBoardDetails from test_board.py to board_test.go

* Migrated TestBoardDetailsNoFlags from test_board.py to board_test.go

* Migrated TestBoardDetailsListProgrammersWithoutFlags from test_board.py to board_test.go

* Migrated TestBoardDetailsListProgrammersFlags from test_board.py to board_test.go

* Migrated TestBoardSearch from test_board.py to board_test.go

* Migrated TestBoardAttachWithoutSketchJson from test_board.py to board_test.go

* Migrated TestBoardSearchWithOutdatedCore to board_test.go and deleted test_board.py

To ensure functionality on all platforms, the "Test Go" GitHub Actions workflow runs the tests on Windows, Linux, and
macOS. This is done via a "job matrix", which runs the three jobs in parallel.

By default GitHub Actions uses a "fail fast" behavior for job matrixes, where all in progress or pending jobs of the
matrix are immediately canceled upon failure of any job.

The benefit of this "fail fast" behavior is efficiency in the case where a failure is universal and those other jobs
were certain to eventually fail. However, in other cases a failure is either specific to the configuration of a single
job (e.g., a Windows-specific bug), or a spurious result of a transient network outage. In the latter case, canceling
the other jobs is very harmful. Running the jobs for the other operating systems would highlight the nature of an
OS-specific failure for the contributor. Canceling other jobs due to a transient failure means all those jobs must be
reran instead of only the specific job that suffered the failure.

The workflow is hereby configured to disable the "fail fast" behavior. All the matrix jobs will now run even if one of
them has failed.

* Factor all compile_part_1 and 2 test setup

* Made an object to simplify running of subtests

* Moved all subtests in the same file

Just moved functions around, no code change.

* Reuse installed platform to speedup debug integration tests

* Integrated 'go.bug.st/testsuite'

It doesn't make much sense to keep it separate in a different library

* Do not use shared download dir for 'cache clean' command tests

* Added method to get cli download dir

* Create infra for sharing directories between tests

* Slighlty improved test output

* Non-parallel testing is no longer required

* Add more time to complete tests

* Update internal/integrationtest/cache/cache_test.go
Co-authored-by: MatteoPologruto <109663225+MatteoPologruto@users.noreply.github.com>
Co-authored-by: MatteoPologruto <109663225+MatteoPologruto@users.noreply.github.com>

* Migrate TestInit from test_config.py to config_test.go

* Migrate TestInitWithExistingCustomConfig from test_config.py to config_test.go

* Migrate TestInitOverwriteExistingCustomFile from test_config.py to config_test.go

* Migrate TestInitDestAbsolutePath from test_config.py to config_test.go

* Migrate TestInitDestRelativePath from test_config.py to config_test.go

* Migrate TestInitDestFlagWithOverwriteFlag from test_config.py to config_test.go

* Migrate TestInitDestAndConfigFileFlags from test_config.py to config_test.go

* Migrate TestInitConfigFileFlagAbsolutePath from test_config.py to config_test.go

* Migrate TestInitConfigFileFlagWithOverwriteFlag from test_config.py to config_test.go

* Migrate TestDump from test_config.py to config_test.go

* Migrate TestDumpWithConfigFileFlag from test_config.py to config_test.go

* Migrate TestInitConfigFileFlagRelativePath from test_config.py to config_test.go

* Migrate TestAddRemoveSetDeleteOnUnexistingKey from test_config.py to config_test.go

* Migrate TestAddSingleArgument from test_config.py to config_test.go

* Migrate TestAddMultipleArguments from test_config.py to config_test.go

* Migrate TestAddOnUnsupportedKey from test_config.py to config_test.go

* Migrate TestRemoveSingleArgument from test_config.py to config_test.go

* Migrate TestRemoveMultipleArguments from test_config.py to config_test.go

* Migrate TestRemoveOnUnsupportedKey from test_config.py to config_test.go

* Migrate TestSetSliceWithSingleArgument from test_config.py to config_test.go

* Migrate TestSetSliceWithMultipleArguments from test_config.py to config_test.go

* Migrate TestSetStringWithSingleArgument from test_config.py to config_test.go

* Migrate TestSetStringWithMultipleArguments from test_config.py to config_test.go

* Migrate TestSetBoolWithSingleArgument from test_config.py to config_test.go

* Migrate TestSetBoolWithMultipleArguments from test_config.py to config_test.go

* Migrate TestDelete to config_test.go and delete config_test.py

* Migrate TestCompileWithOutputDirFlag from test_compile_part_2.py to compile_part_2_test.go

* Migrate TestCompileWithExportBinariesFlag from test_compile_part_2.py to compile_part_2_test.go

* Migrate TestCompileWithCustomBuildPath from test_compile_part_2.py to compile_part_2_test.go

* Migrate TestCompileWithInvalidUrl from test_compile_part_2.py to compile_part_2_test.go

* Migrate TestCompileWithExportBinariesEnvVar from test_compile_part_2.py to compile_part_2_test.go

* Migrate TestCompileWithExportBinariesConfig from test_compile_part_2.py
to compile_part_2_test.go

* Migrate TestCompileWithCustomLibraries from test_compile_part_2.py to compile_part_2_test.go

* Migrate TestCompileWithArchivesAndLongPaths from test_compile_part_2.py to compile_part_2_test.go

* Migrate TestCompileWithPrecompileLibrary to compile_part_2_test.go and delete test_compile_part_2.py

* Increase lapse before timeout by 5 minutes

* Download library index as a combined archive+signature

* Fixed integration tests

* uniquify configuration arrays in `config add`

* Add tests for unique board manager URLs

* fix linting

* lint: update function name

* update config set to uniquify slices

* Refactored DownloadProgress protocol

* Updated integration tests

* Do not create overly verbose errors

* Updated docs

* Update rpc/cc/arduino/cli/commands/v1/commands.proto
Co-authored-by: per1234 <accounts@perglass.com>
Co-authored-by: per1234 <accounts@perglass.com>

* Removed real_name field from gRPC Library message

* Update docs/UPGRADING.md
Co-authored-by: per1234 <accounts@perglass.com>
Co-authored-by: per1234 <accounts@perglass.com>

* Factored function to get search terms

* Made 'core search' logic identical to Arduino IDE 1.8.x search dialog

* Drastically simplified search algorithm for libs

* Fixed SearchTermsFromQueryString function

* Moving SearchTermsFromQueryString into proper place

Each Arduino library has a name, which is used as its sole identifier by the user in `arduino-cli lib` commands, and by
Arduino CLI when referring to the library in messages displayed to the user.

The name is defined by:

- "1.5 format" libraries: `name` field in the library.properties metadata file
- "1.0 format" (AKA "legacy") libraries: installation folder name

The name is resolved when loading the library and stored in the `Name` field of the
`github.com/arduino/arduino-cli/arduino/libraries.Library` struct.

The name of the library's installation folder is used by Arduino CLI in several other ways, most notably for determining
"folder name priority" for use in library dependency resolution. For this reason, the folder name is also stored in the
struct when loading the library.

Arduino CLI and arduino-builder have been plagued by problems caused by the inappropriate use of this folder name as the
identifier for the library instead of the sole correct identifier (which is only the folder name in the case of
"1.0 format" libraries. The design of the `github.com/arduino/arduino-cli/arduino/libraries.Library` struct may have
been a contributing factor in those bugs, since at the time of their occurrence the folder name was stored in the `Name`
field, the metadata-defined name in a `RealName`. In addition to the fact that no one field could be used as a source of
the name in all cases, I suspect the ambiguous field names themselves caused confusion to developers.

This situation was improved by providing the library identifier via a single field for all library formats. The name
provided by this field is the "canonical" name of the library. Inexplicably, at that time the field containing the
folder name was renamed "CanonicalName". The string contained by this field is in no way a "canonical" name for the
library, so the field name is bound to cause more of the same bugs and confusion the redesign of the struct was intended
to prevent.

The inappropriately named `github.com/arduino/arduino-cli/arduino/libraries.Library.CanonicalName` field is hereby
renamed to the accurate `DirName`.

* Do not run go integration tests in parallel

This is required since the integration tests shares the download folder.
Still not ideal, it would be better if the cli could handle concurrent
access to the download folder (from multiple cli processes...), but at
least allows to run the tests without random failures.

* Fixed integration test

`board listall` (in json/machine readable output) return boards sorted with the same order as in the original `boards.txt` (#1903)

* Simplified internal boards menu maps construction

* Redesign boards build loop to iterate following boards.txt boards order

* Return board list with the same ordering as in original boards.txt

* Added integration test

* [skip changelog] Use correct markup for PR template headings

This repository uses a pull request template in order to provide a standardized structure for pull request messages.

The template imposes a framework for the organization of the pull request message by providing a section to contain each
of the distinct types of important information to be included in the pull request. As should be obvious, the only
correct formatting for section headings is heading markup, yet somehow list markup was chosen instead. This collides
with the appropriate use of lists within the sections.

The template is hereby changed to use the correct markup for headings.

* [skip changelog] Move contributor guide link to appropriate section of PR template

This repository uses a pull request template in order to provide a standardized structure for pull request messages.

The template includes a link to the project's contributor guide. Previously, this link was at the bottom of the
template, separated from the other content by a horizontal rule. One of the unfortunate ambiguities of the Markdown
syntax is that the horizontal rule syntax is the same as the "setext" H2 heading syntax, with the differentiation
depending on whether the prior line is empty. This results in text intended to be part of the "Other information"
section of the PR message instead being formatted as an H2 heading if the contributor does not add an empty line at the
end of the "Other information" section.

Moving the link to the section of the template that contains a checklist for the PR requirements makes the relevant
information readily accessible to the contributor while also avoiding the creation of a formatting trap.

* Use a matrix create parallel builds for each os

Using a matrix to run each build task greatly improves performances,
since they can all start concurrently. The finishing time of the job
will be equal to the one of the longer task to build, instead of being
the sum of each individual task's finishing time.

* Calculate checksums during release creation

Checksums of the output files where previously calculated during the initial creation of the artifacts, during the notarization process and, finally, at the release creation's step.
The whole process has been simplified and checksums are now computed only during the creation of the release.

* Set condition to create changelog once

The changelog is the same for each OS. It does not make sense to generate it more than once.

* Disable s3 push for testing

* Upload nightly artifacts for testing

* Fix linux_arm_6 typo

* Stop uploading nigthly artifacts

* Enable s3 pushing

* Upload build artifacts separately

Previously, the different builds were firstly uploaded using a single artifact, which was then downloaded to create different ones and eventually deleted.
Now, since builds are created concurrently, the same matrix can be used to directly upload an artifact for each build. It's necessary to use a second job to calculate the checksum related to
each build and save them all in a single .txt file.

* Set test environment directory as CLI WorkingDir

By default, the working directory is the one containing the test.go file. This causes problems when executing commands that have to create files specifically in the working directory, because they either must be deleted manually or the user has to be aware of it and defer a deleting instruction. Furthermore, it messes with tests using relative paths.
Setting the environment directory as the CLI's WorkingDir prevents the above mentioned issues from occurring.

* Fix errors related to the change of the working directory

* Use absolute path to create daemon environment

* Migrate TestCompileWithoutFqbn from test_compile_part_1.py to compile_part_1_test.go

* Migrate TestCompileErrorMessage from test_compile_part_1.py to compile_part_1_test.go

* Migrate TestCompileWithSimpleSketch from test_compile_part_1.py to compile_part_1_test.go

* Migrate TestOutputFlagDefaultPath from test_compile_part_1.py to compile_part_1_test.go

* Migrate TestCompileWithSketchWithSymlinkSelfloop from test_compile_part_1.py to compile_part_1_test.go

* Migrate TestCompileBlacklistedSketchname from test_compile_part_1.py to compile_part_1_test.go

* Migrate TestCompileWithoutPrecompiledLibraries from test_compile_part_1.py to compile_part_1_test.go

* Migrate TestCompileWithBuildPropertiesFlag from test_compile_part_1.py to compile_part_1_test.go

* Migrate TestCompileWithBuildPropertyContainingQuotes from
test_compile_part_1.py to compile_part_1_test.go

* Migrate TestCompileWithMultipleBuildPropertyFlags to compile_part_1_test.go and delete test_compile_part_1.py

* Shorten testsuite ProjectName and DataDir to prevent errors on Windows runner

Long paths caused certain commands to fail on the Windows runner. Reducing their lengths prevents those errors from occurring.

* Do not treat custom menu items without label as 'malformed'

* Fixed tests

* Removed LibraryAlternatives object in favor of libraries.List

* Use RealName instead of (dir) Name as key in librarymanager.Library map

* Refactored some librarymanager functions to query libraries list

* Simplified librariesmanager.Install

It does not require installLocation since it can be obtained with
libPath.Parent()

* Added checks for multiple library installation

* Added test to check that the cli is able to handle multiple lib installations

* Renamed fields in Library structure: Name->CanonicalName; RealName->Name

* Use Name instead of CanonicalName in LibraryList

This fixes also `lib list` and `lib examples`.

* Use `Name` field where appropriate instead of `CanonicalName`

* Updated documentation

* Update arduino/libraries/librarieslist.go
Co-authored-by: Umberto Baldi <34278123+umbynos@users.noreply.github.com>

* Improved integration test
Co-authored-by: Umberto Baldi <34278123+umbynos@users.noreply.github.com>

* Migrate TestCompileWithProfiles from test_profiles.py to profiles_test.go

* Migrate TestBuilderDidNotCatchLibsFromUnusedPlatforms to profiles_test.go and delete test_profiles.py

CopySketch is a function that copies a sketch present in the "testdata"
folder and copies it into the testing environment.

* Fixed NPE on 'lib upgrade' command

* Added tests

* Added flag to skip 3rd party package_index in UpdateIndex gRPC API

Fix #1788

* UpdateIndex gRPC call does not stop at the first error

* Updated docs

* Fixed integration tests

* Ensure that a 'Completed' download progress is always sent

* UpdateIndex: Fixed a wrong success report and added test.

* Improved test error messages