Improved JSON decoding (improves arduino-cli startup time) (#1402)

* Updated library_index.json for testing

* Use easyjson for parsing library_index

* Added benchmarks for json deconding.

Results:

$ go test -v -benchmem -bench BenchmarkIndexParsing github.com/arduino/arduino-cli/arduino/libraries/librariesindex
=== RUN   TestIndexer
--- PASS: TestIndexer (0.16s)
goos: linux
goarch: amd64
pkg: github.com/arduino/arduino-cli/arduino/libraries/librariesindex
cpu: Intel(R) Core(TM) i7-8550U CPU @ 1.80GHz
BenchmarkIndexParsingStdJSON
BenchmarkIndexParsingStdJSON-8                 5         214872730 ns/op          94.52 MB/s    58956539 B/op     418973 allocs/op
BenchmarkIndexParsingEasyJSON
BenchmarkIndexParsingEasyJSON-8               16          69215472 ns/op         293.42 MB/s    56162664 B/op     418966 allocs/op
PASS
ok      github.com/arduino/arduino-cli/arduino/libraries/librariesindex 4.442s

easyjson is 3x faster.

* Updated license check cache

* Added easyjson to package_index.json parser

* Allow easyjson to match tags in case-insensitive

See https://github.com/mailru/easyjson/pull/372

* Added easyjson generation task in Taskfile

* Added github action check

* Update .github/workflows/check-easyjson.yml
Co-authored-by: per1234 <accounts@perglass.com>
Co-authored-by: per1234 <accounts@perglass.com>

.github/workflows/check-easyjson.yml

+name: Check easyjson generated files
+
+env:
+  # See: https://github.com/actions/setup-go/tree/v2#readme
+  GO_VERSION: "1.17"
+
+# See: https://docs.github.com/en/actions/reference/events-that-trigger-workflows
+on:
+  push:
+    paths:
+      - ".github/workflows/check-easyjson.ya?ml"
+      - "go.mod"
+      - "go.sum"
+      - "Taskfile.ya?ml"
+      - "**.go"
+  pull_request:
+    paths:
+      - ".github/workflows/check-easyjson.ya?ml"
+      - "go.mod"
+      - "go.sum"
+      - "Taskfile.ya?ml"
+      - "**.go"
+  workflow_dispatch:
+  repository_dispatch:
+
+jobs:
+  check-easyjson-generated-files:
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v3
+
+      - name: Install Go
+        uses: actions/setup-go@v3
+        with:
+          go-version: ${{ env.GO_VERSION }}
+
+      - name: Install Task
+        uses: arduino/setup-task@v1
+        with:
+          repo-token: ${{ secrets.GITHUB_TOKEN }}
+          version: 3.x
+
+      - name: Regenerate easyjson source code
+        run: task go:easyjson-generate
+
+      - name: Check for outdated cache
+        id: diff
+        run: |
+          git add .
+          if ! git diff --cached --color --exit-code; then
+            echo
+            echo "::error::easyjson generated files out of sync."
+            exit 1
+          fi

.licenses/go/github.com/josharian/intern.dep.yml

+---
+name: github.com/josharian/intern
+version: v1.0.0
+type: go
+summary: Package intern interns strings.
+homepage: https://pkg.go.dev/github.com/josharian/intern
+license: mit
+licenses:
+- sources: license.md
+  text: |
+    MIT License
+
+    Copyright (c) 2019 Josh Bleecher Snyder
+
+    Permission is hereby granted, free of charge, to any person obtaining a copy
+    of this software and associated documentation files (the "Software"), to deal
+    in the Software without restriction, including without limitation the rights
+    to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+    copies of the Software, and to permit persons to whom the Software is
+    furnished to do so, subject to the following conditions:
+
+    The above copyright notice and this permission notice shall be included in all
+    copies or substantial portions of the Software.
+
+    THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+    IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+    FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+    AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+    LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+    OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+    SOFTWARE.
+notices: []

.licenses/go/github.com/mailru/easyjson.dep.yml

+---
+name: github.com/mailru/easyjson
+version: v0.7.7
+type: go
+summary: Package easyjson contains marshaler/unmarshaler interfaces and helper functions.
+homepage: https://pkg.go.dev/github.com/mailru/easyjson
+license: mit
+licenses:
+- sources: LICENSE
+  text: |
+    Copyright (c) 2016 Mail.Ru Group
+
+    Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
+
+    The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
+
+    THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+notices: []

.licenses/go/github.com/mailru/easyjson/buffer.dep.yml

+---
+name: github.com/mailru/easyjson/buffer
+version: v0.7.7
+type: go
+summary: Package buffer implements a buffer for serialization, consisting of a chain
+  of []byte-s to reduce copying and to allow reuse of individual chunks.
+homepage: https://pkg.go.dev/github.com/mailru/easyjson/buffer
+license: mit
+licenses:
+- sources: easyjson@v0.7.7/LICENSE
+  text: |
+    Copyright (c) 2016 Mail.Ru Group
+
+    Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
+
+    The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
+
+    THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+notices: []

.licenses/go/github.com/mailru/easyjson/jlexer.dep.yml

+---
+name: github.com/mailru/easyjson/jlexer
+version: v0.7.7
+type: go
+summary: Package jlexer contains a JSON lexer implementation.
+homepage: https://pkg.go.dev/github.com/mailru/easyjson/jlexer
+license: mit
+licenses:
+- sources: easyjson@v0.7.7/LICENSE
+  text: |
+    Copyright (c) 2016 Mail.Ru Group
+
+    Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
+
+    The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
+
+    THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+notices: []

.licenses/go/github.com/mailru/easyjson/jwriter.dep.yml

+---
+name: github.com/mailru/easyjson/jwriter
+version: v0.7.7
+type: go
+summary: Package jwriter contains a JSON writer.
+homepage: https://pkg.go.dev/github.com/mailru/easyjson/jwriter
+license: mit
+licenses:
+- sources: easyjson@v0.7.7/LICENSE
+  text: |
+    Copyright (c) 2016 Mail.Ru Group
+
+    Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
+
+    The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
+
+    THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+notices: []

Taskfile.yml

@@ -119,6 +119,13 @@ tasks:
     cmds:
       - go vet {{default .DEFAULT_GO_PACKAGES .GO_PACKAGES}}
 
+  go:easyjson-generate:
+    desc: Run easyjson code generation
+    cmds:
+      - go install github.com/mailru/easyjson/...@v0.7.7
+      - easyjson arduino/cores/packageindex/index.go
+      - easyjson arduino/libraries/librariesindex/json.go
+
   # Source: https://github.com/arduino/tooling-project-assets/blob/main/workflow-templates/assets/check-markdown-task/Taskfile.yml
   markdown:check-links:
     desc: Check for broken links

arduino/cores/packageindex/index.go

@@ -24,17 +24,20 @@ import (
 	"github.com/arduino/arduino-cli/arduino/security"
 	"github.com/arduino/arduino-cli/i18n"
 	"github.com/arduino/go-paths-helper"
+	easyjson "github.com/mailru/easyjson"
 	"github.com/sirupsen/logrus"
 	semver "go.bug.st/relaxed-semver"
 )
 
 // Index represents Cores and Tools struct as seen from package_index.json file.
+//easyjson:json
 type Index struct {
 	Packages  []*indexPackage `json:"packages"`
 	IsTrusted bool
 }
 
 // indexPackage represents a single entry from package_index.json file.
+//easyjson:json
 type indexPackage struct {
 	Name       string                  `json:"name"`
 	Maintainer string                  `json:"maintainer"`
@@ -47,6 +50,7 @@ type indexPackage struct {
 }
 
 // indexPlatformRelease represents a single Core Platform from package_index.json file.
+//easyjson:json
 type indexPlatformRelease struct {
 	Name                  string                     `json:"name"`
 	Architecture          string                     `json:"architecture"`
@@ -65,6 +69,7 @@ type indexPlatformRelease struct {
 }
 
 // indexToolDependency represents a single dependency of a core from a tool.
+//easyjson:json
 type indexToolDependency struct {
 	Packager string                 `json:"packager"`
 	Name     string                 `json:"name"`
@@ -72,18 +77,21 @@ type indexToolDependency struct {
 }
 
 // indexDiscoveryDependency represents a single dependency of a core from a pluggable discovery tool.
+//easyjson:json
 type indexDiscoveryDependency struct {
 	Packager string `json:"packager"`
 	Name     string `json:"name"`
 }
 
 // indexMonitorDependency represents a single dependency of a core from a pluggable monitor tool.
+//easyjson:json
 type indexMonitorDependency struct {
 	Packager string `json:"packager"`
 	Name     string `json:"name"`
 }
 
 // indexToolRelease represents a single Tool from package_index.json file.
+//easyjson:json
 type indexToolRelease struct {
 	Name    string                    `json:"name"`
 	Version *semver.RelaxedVersion    `json:"version"`
@@ -91,6 +99,7 @@ type indexToolRelease struct {
 }
 
 // indexToolReleaseFlavour represents a single tool flavor in the package_index.json file.
+//easyjson:json
 type indexToolReleaseFlavour struct {
 	OS              string      `json:"host"`
 	URL             string      `json:"url"`
@@ -100,17 +109,20 @@ type indexToolReleaseFlavour struct {
 }
 
 // indexBoard represents a single Board as written in package_index.json file.
+//easyjson:json
 type indexBoard struct {
 	Name string         `json:"name"`
 	ID   []indexBoardID `json:"id,omitempty"`
 }
 
 // indexBoardID represents the ID of a single board. i.e. uno, yun, diecimila, micro and the likes
+//easyjson:json
 type indexBoardID struct {
 	USB string `json:"usb"`
 }
 
 // indexHelp represents the help URL
+//easyjson:json
 type indexHelp struct {
 	Online string `json:"online,omitempty"`
 }
@@ -352,7 +364,7 @@ func LoadIndex(jsonIndexFile *paths.Path) (*Index, error) {
 		return nil, err
 	}
 	var index Index
-	err = json.Unmarshal(buff, &index)
+	err = easyjson.Unmarshal(buff, &index)
 	if err != nil {
 		return nil, err
 	}
@@ -381,7 +393,7 @@ func LoadIndexNoSign(jsonIndexFile *paths.Path) (*Index, error) {
 		return nil, err
 	}
 	var index Index
-	err = json.Unmarshal(buff, &index)
+	err = easyjson.Unmarshal(buff, &index)
 	if err != nil {
 		return nil, err
 	}

arduino/discovery/discovery_client/go.sum

@@ -192,6 +192,7 @@ github.com/ianlancetaylor/demangle v0.0.0-20200824232613-28f6c0f3b639/go.mod h1:
 github.com/inconshreveable/mousetrap v1.0.0/go.mod h1:PxqpIevigyE2G7u3NXJIT2ANytuPF1OarO4DADm73n8=
 github.com/jbenet/go-context v0.0.0-20150711004518-d14ea06fba99/go.mod h1:1lJo3i6rXxKeerYnT8Nvf0QmHCRC1n8sfWVwXF2Frvo=
 github.com/jessevdk/go-flags v1.4.0/go.mod h1:4FA24M0QyGHXBuZZK/XkWh8h0e1EYbRYJSGM75WSRxI=
+github.com/josharian/intern v1.0.0/go.mod h1:5DoeVV0s6jJacbCEi61lwdGj/aVlrQvzHFFd8Hwg//Y=
 github.com/json-iterator/go v1.1.11/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4=
 github.com/jstemmer/go-junit-report v0.0.0-20190106144839-af01ea7f8024/go.mod h1:6v2b51hI/fHJwM22ozAgKL4VKDeJcHhJFhtBdhmNjmU=
 github.com/jstemmer/go-junit-report v0.9.1/go.mod h1:Brl9GWCQeLvo8nXZwPNNblvFj/XSXhF0NWZEnDohbsk=
@@ -219,6 +220,7 @@ github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
 github.com/leonelquinteros/gotext v1.4.0 h1:2NHPCto5IoMXbrT0bldPrxj0qM5asOCwtb1aUQZ1tys=
 github.com/leonelquinteros/gotext v1.4.0/go.mod h1:yZGXREmoGTtBvZHNcc+Yfug49G/2spuF/i/Qlsvz1Us=
 github.com/magiconair/properties v1.8.5/go.mod h1:y3VJvCyxH9uVvJTWEGAELF3aiYNyPKd5NZ3oSwXrF60=
+github.com/mailru/easyjson v0.7.7/go.mod h1:xzfreul335JAWq5oZzymOObrkdz5UnU4kGfJJLY9Nlc=
 github.com/marcinbor85/gohex v0.0.0-20210308104911-55fb1c624d84/go.mod h1:Pb6XcsXyropB9LNHhnqaknG/vEwYztLkQzVCHv8sQ3M=
 github.com/mattn/go-colorable v0.0.9/go.mod h1:9vuHe8Xs5qXnSaW/c/ABM9alt+Vo+STaOChaDxuIBZU=
 github.com/mattn/go-colorable v0.1.8/go.mod h1:u6P/XSegPjTcexA+o6vUJrdnUu04hMope9wVRipJSqc=

arduino/libraries/librariesindex/index_test.go

@@ -16,11 +16,13 @@
 package librariesindex
 
 import (
+	json "encoding/json"
 	"fmt"
 	"testing"
 
 	"github.com/arduino/arduino-cli/arduino/libraries"
 	"github.com/arduino/go-paths-helper"
+	easyjson "github.com/mailru/easyjson"
 	"github.com/stretchr/testify/require"
 	semver "go.bug.st/relaxed-semver"
 )
@@ -36,16 +38,16 @@ func TestIndexer(t *testing.T) {
 
 	index, err := LoadIndex(paths.New("testdata/library_index.json"))
 	require.NoError(t, err)
-	require.Equal(t, 2380, len(index.Libraries), "parsed libraries count")
+	require.Equal(t, 4124, len(index.Libraries), "parsed libraries count")
 
 	alp := index.Libraries["Arduino Low Power"]
 	require.NotNil(t, alp)
-	require.Equal(t, 4, len(alp.Releases))
-	require.Equal(t, "Arduino Low Power@1.2.1", alp.Latest.String())
+	require.Equal(t, 5, len(alp.Releases))
+	require.Equal(t, "Arduino Low Power@1.2.2", alp.Latest.String())
 	require.Len(t, alp.Latest.Dependencies, 1)
 	require.Equal(t, "RTCZero", alp.Latest.Dependencies[0].GetName())
 	require.Equal(t, "", alp.Latest.Dependencies[0].GetConstraint().String())
-	require.Equal(t, "[1.0.0 1.1.0 1.2.0 1.2.1]", fmt.Sprintf("%v", alp.Versions()))
+	require.Equal(t, "[1.0.0 1.1.0 1.2.0 1.2.1 1.2.2]", fmt.Sprintf("%v", alp.Versions()))
 
 	rtc100ref := &Reference{Name: "RTCZero", Version: semver.MustParse("1.0.0")}
 	require.Equal(t, "RTCZero@1.0.0", rtc100ref.String())
@@ -96,12 +98,12 @@ func TestIndexer(t *testing.T) {
 	oauth010 := index.FindRelease(&Reference{Name: "Arduino_OAuth", Version: semver.MustParse("0.1.0")})
 	require.NotNil(t, oauth010)
 	require.Equal(t, "Arduino_OAuth@0.1.0", oauth010.String())
-	eccx133 := index.FindRelease(&Reference{Name: "ArduinoECCX08", Version: semver.MustParse("1.3.3")})
-	require.NotNil(t, eccx133)
-	require.Equal(t, "ArduinoECCX08@1.3.3", eccx133.String())
-	bear130 := index.FindRelease(&Reference{Name: "ArduinoBearSSL", Version: semver.MustParse("1.3.0")})
-	require.NotNil(t, bear130)
-	require.Equal(t, "ArduinoBearSSL@1.3.0", bear130.String())
+	eccx135 := index.FindRelease(&Reference{Name: "ArduinoECCX08", Version: semver.MustParse("1.3.5")})
+	require.NotNil(t, eccx135)
+	require.Equal(t, "ArduinoECCX08@1.3.5", eccx135.String())
+	bear172 := index.FindRelease(&Reference{Name: "ArduinoBearSSL", Version: semver.MustParse("1.7.2")})
+	require.NotNil(t, bear172)
+	require.Equal(t, "ArduinoBearSSL@1.7.2", bear172.String())
 	http040 := index.FindRelease(&Reference{Name: "ArduinoHttpClient", Version: semver.MustParse("0.4.0")})
 	require.NotNil(t, http040)
 	require.Equal(t, "ArduinoHttpClient@0.4.0", http040.String())
@@ -109,7 +111,31 @@ func TestIndexer(t *testing.T) {
 	resolve2 := index.ResolveDependencies(oauth010)
 	require.Len(t, resolve2, 4)
 	require.Contains(t, resolve2, oauth010)
-	require.Contains(t, resolve2, eccx133)
-	require.Contains(t, resolve2, bear130)
+	require.Contains(t, resolve2, eccx135)
+	require.Contains(t, resolve2, bear172)
 	require.Contains(t, resolve2, http040)
 }
+
+func BenchmarkIndexParsingStdJSON(b *testing.B) {
+	indexFile := paths.New("testdata/library_index.json")
+	buff, err := indexFile.ReadFile()
+	require.NoError(b, err)
+	b.SetBytes(int64(len(buff)))
+	for i := 0; i < b.N; i++ {
+		var i indexJSON
+		err = json.Unmarshal(buff, &i)
+		require.NoError(b, err)
+	}
+}
+
+func BenchmarkIndexParsingEasyJSON(b *testing.B) {
+	indexFile := paths.New("testdata/library_index.json")
+	buff, err := indexFile.ReadFile()
+	require.NoError(b, err)
+	b.SetBytes(int64(len(buff)))
+	for i := 0; i < b.N; i++ {
+		var i indexJSON
+		err = easyjson.Unmarshal(buff, &i)
+		require.NoError(b, err)
+	}
+}

arduino/libraries/librariesindex/json.go

@@ -16,19 +16,21 @@
 package librariesindex
 
 import (
-	"encoding/json"
 	"fmt"
 
 	"github.com/arduino/arduino-cli/arduino/resources"
 	"github.com/arduino/arduino-cli/i18n"
 	"github.com/arduino/go-paths-helper"
+	easyjson "github.com/mailru/easyjson"
 	semver "go.bug.st/relaxed-semver"
 )
 
+//easyjson:json
 type indexJSON struct {
 	Libraries []indexRelease `json:"libraries"`
 }
 
+//easyjson:json
 type indexRelease struct {
 	Name             string             `json:"name"`
 	Version          *semver.Version    `json:"version"`
@@ -49,6 +51,7 @@ type indexRelease struct {
 	ProvidesIncludes []string           `json:"providesIncludes"`
 }
 
+//easyjson:json
 type indexDependency struct {
 	Name    string `json:"name"`
 	Version string `json:"version,omitempty"`
@@ -64,7 +67,7 @@ func LoadIndex(indexFile *paths.Path) (*Index, error) {
 	}
 
 	var i indexJSON
-	err = json.Unmarshal(buff, &i)
+	err = easyjson.Unmarshal(buff, &i)
 	if err != nil {
 		return nil, fmt.Errorf(tr("parsing library_index.json: %s"), err)
 	}

client_example/go.sum

@@ -187,6 +187,7 @@ github.com/ianlancetaylor/demangle v0.0.0-20200824232613-28f6c0f3b639/go.mod h1:
 github.com/inconshreveable/mousetrap v1.0.0/go.mod h1:PxqpIevigyE2G7u3NXJIT2ANytuPF1OarO4DADm73n8=
 github.com/jbenet/go-context v0.0.0-20150711004518-d14ea06fba99/go.mod h1:1lJo3i6rXxKeerYnT8Nvf0QmHCRC1n8sfWVwXF2Frvo=
 github.com/jessevdk/go-flags v1.4.0/go.mod h1:4FA24M0QyGHXBuZZK/XkWh8h0e1EYbRYJSGM75WSRxI=
+github.com/josharian/intern v1.0.0/go.mod h1:5DoeVV0s6jJacbCEi61lwdGj/aVlrQvzHFFd8Hwg//Y=
 github.com/json-iterator/go v1.1.11/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4=
 github.com/jstemmer/go-junit-report v0.0.0-20190106144839-af01ea7f8024/go.mod h1:6v2b51hI/fHJwM22ozAgKL4VKDeJcHhJFhtBdhmNjmU=
 github.com/jstemmer/go-junit-report v0.9.1/go.mod h1:Brl9GWCQeLvo8nXZwPNNblvFj/XSXhF0NWZEnDohbsk=
@@ -212,6 +213,7 @@ github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
 github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
 github.com/leonelquinteros/gotext v1.4.0/go.mod h1:yZGXREmoGTtBvZHNcc+Yfug49G/2spuF/i/Qlsvz1Us=
 github.com/magiconair/properties v1.8.5/go.mod h1:y3VJvCyxH9uVvJTWEGAELF3aiYNyPKd5NZ3oSwXrF60=
+github.com/mailru/easyjson v0.7.7/go.mod h1:xzfreul335JAWq5oZzymOObrkdz5UnU4kGfJJLY9Nlc=
 github.com/marcinbor85/gohex v0.0.0-20210308104911-55fb1c624d84/go.mod h1:Pb6XcsXyropB9LNHhnqaknG/vEwYztLkQzVCHv8sQ3M=
 github.com/mattn/go-colorable v0.0.9/go.mod h1:9vuHe8Xs5qXnSaW/c/ABM9alt+Vo+STaOChaDxuIBZU=
 github.com/mattn/go-colorable v0.1.8/go.mod h1:u6P/XSegPjTcexA+o6vUJrdnUu04hMope9wVRipJSqc=

docs/CONTRIBUTING.md

@@ -349,6 +349,13 @@ func main() {
 }
 ```
 
+### About `easyjson` golang library
+
+We use the hi-performance `easyjson` library to parse the large JSON index files for libraries and platforms. To obtain
+the best performance we must do some code generation, this is done via `task go:easyjson-generate`. If you ever touch
+source code using the `easyjson` library, make sure to re-run the `go:easyjson-generate` task to see if there are
+changes in the generated code.
+
 ### Additional settings
 
 If you need to push a commit that's only shipping documentation changes or example files, thus a complete no-op for the

docsgen/go.mod

@@ -32,11 +32,13 @@ require (
 	github.com/hashicorp/hcl v1.0.0 // indirect
 	github.com/inconshreveable/mousetrap v1.0.0 // indirect
 	github.com/jbenet/go-context v0.0.0-20150711004518-d14ea06fba99 // indirect
+	github.com/josharian/intern v1.0.0 // indirect
 	github.com/juju/errors v0.0.0-20181118221551-089d3ea4e4d5 // indirect
 	github.com/kevinburke/ssh_config v0.0.0-20190725054713-01f96b0aa0cd // indirect
 	github.com/konsorten/go-windows-terminal-sequences v1.0.1 // indirect
 	github.com/leonelquinteros/gotext v1.4.0 // indirect
 	github.com/magiconair/properties v1.8.5 // indirect
+	github.com/mailru/easyjson v0.7.7 // indirect
 	github.com/marcinbor85/gohex v0.0.0-20210308104911-55fb1c624d84 // indirect
 	github.com/mattn/go-colorable v0.1.8 // indirect
 	github.com/mattn/go-isatty v0.0.12 // indirect

docsgen/go.sum

@@ -213,6 +213,8 @@ github.com/inconshreveable/mousetrap v1.0.0/go.mod h1:PxqpIevigyE2G7u3NXJIT2ANyt
 github.com/jbenet/go-context v0.0.0-20150711004518-d14ea06fba99 h1:BQSFePA1RWJOlocH6Fxy8MmwDt+yVQYULKfN0RoTN8A=
 github.com/jbenet/go-context v0.0.0-20150711004518-d14ea06fba99/go.mod h1:1lJo3i6rXxKeerYnT8Nvf0QmHCRC1n8sfWVwXF2Frvo=
 github.com/jessevdk/go-flags v1.4.0/go.mod h1:4FA24M0QyGHXBuZZK/XkWh8h0e1EYbRYJSGM75WSRxI=
+github.com/josharian/intern v1.0.0 h1:vlS4z54oSdjm0bgjRigI+G1HpF+tI+9rE5LLzOg8HmY=
+github.com/josharian/intern v1.0.0/go.mod h1:5DoeVV0s6jJacbCEi61lwdGj/aVlrQvzHFFd8Hwg//Y=
 github.com/json-iterator/go v1.1.11/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4=
 github.com/jstemmer/go-junit-report v0.0.0-20190106144839-af01ea7f8024/go.mod h1:6v2b51hI/fHJwM22ozAgKL4VKDeJcHhJFhtBdhmNjmU=
 github.com/jstemmer/go-junit-report v0.9.1/go.mod h1:Brl9GWCQeLvo8nXZwPNNblvFj/XSXhF0NWZEnDohbsk=
@@ -247,6 +249,8 @@ github.com/leonelquinteros/gotext v1.4.0 h1:2NHPCto5IoMXbrT0bldPrxj0qM5asOCwtb1a
 github.com/leonelquinteros/gotext v1.4.0/go.mod h1:yZGXREmoGTtBvZHNcc+Yfug49G/2spuF/i/Qlsvz1Us=
 github.com/magiconair/properties v1.8.5 h1:b6kJs+EmPFMYGkow9GiUyCyOvIwYetYJ3fSaWak/Gls=
 github.com/magiconair/properties v1.8.5/go.mod h1:y3VJvCyxH9uVvJTWEGAELF3aiYNyPKd5NZ3oSwXrF60=
+github.com/mailru/easyjson v0.7.7 h1:UGYAvKxe3sBsEDzO8ZeWOSlIQfWFlxbzLZe7hwFURr0=
+github.com/mailru/easyjson v0.7.7/go.mod h1:xzfreul335JAWq5oZzymOObrkdz5UnU4kGfJJLY9Nlc=
 github.com/marcinbor85/gohex v0.0.0-20210308104911-55fb1c624d84 h1:hyAgCuG5nqTMDeUD8KZs7HSPs6KprPgPP8QmGV8nyvk=
 github.com/marcinbor85/gohex v0.0.0-20210308104911-55fb1c624d84/go.mod h1:Pb6XcsXyropB9LNHhnqaknG/vEwYztLkQzVCHv8sQ3M=
 github.com/mattn/go-colorable v0.0.9/go.mod h1:9vuHe8Xs5qXnSaW/c/ABM9alt+Vo+STaOChaDxuIBZU=

go.mod

@@ -2,6 +2,9 @@ module github.com/arduino/arduino-cli
 
 go 1.17
 
+// We must use this fork until https://github.com/mailru/easyjson/pull/372 is merged
+replace github.com/mailru/easyjson => github.com/cmaglie/easyjson v0.8.1
+
 require (
 	github.com/arduino/board-discovery v0.0.0-20180823133458-1ba29327fb0c
 	github.com/arduino/go-paths-helper v1.7.0
@@ -19,6 +22,7 @@ require (
 	github.com/juju/loggo v0.0.0-20190526231331-6e530bcce5d8 // indirect
 	github.com/kr/text v0.2.0 // indirect
 	github.com/leonelquinteros/gotext v1.4.0
+	github.com/mailru/easyjson v0.7.7
 	github.com/marcinbor85/gohex v0.0.0-20210308104911-55fb1c624d84
 	github.com/mattn/go-colorable v0.1.8
 	github.com/mattn/go-isatty v0.0.12
@@ -62,6 +66,7 @@ require (
 	github.com/hashicorp/hcl v1.0.0 // indirect
 	github.com/inconshreveable/mousetrap v1.0.0 // indirect
 	github.com/jbenet/go-context v0.0.0-20150711004518-d14ea06fba99 // indirect
+	github.com/josharian/intern v1.0.0 // indirect
 	github.com/juju/errors v0.0.0-20181118221551-089d3ea4e4d5 // indirect
 	github.com/kevinburke/ssh_config v0.0.0-20190725054713-01f96b0aa0cd // indirect
 	github.com/konsorten/go-windows-terminal-sequences v1.0.1 // indirect

go.sum

@@ -68,6 +68,8 @@ github.com/chzyer/logex v1.1.10/go.mod h1:+Ywpsq7O8HXn0nuIou7OrIPyXbp3wmkHB+jjWR
 github.com/chzyer/readline v0.0.0-20180603132655-2972be24d48e/go.mod h1:nSuG5e5PlCu98SY8svDHJxuZscDgtXS6KTTbou5AhLI=
 github.com/chzyer/test v0.0.0-20180213035817-a1ea475d72b1/go.mod h1:Q3SI9o4m/ZMnBNeIyt5eFwwo7qiLfzFZmjNmxjkiQlU=
 github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw=
+github.com/cmaglie/easyjson v0.8.1 h1:nKQ6Yew57jsoGsuyRJPgm8PSsjbU3eO/uA9BsTu3E/8=
+github.com/cmaglie/easyjson v0.8.1/go.mod h1:xzfreul335JAWq5oZzymOObrkdz5UnU4kGfJJLY9Nlc=
 github.com/cmaglie/pb v1.0.27 h1:ynGj8vBXR+dtj4B7Q/W/qGt31771Ux5iFfRQBnwdQiA=
 github.com/cmaglie/pb v1.0.27/go.mod h1:GilkKZMXYjBA4NxItWFfO+lwkp59PLHQ+IOW/b/kmZI=
 github.com/cncf/udpa/go v0.0.0-20191209042840-269d4d468f6f/go.mod h1:M8M6+tZqaGXZJjfX53e64911xZQV5JYwmTeXPW+k8Sc=
@@ -213,6 +215,8 @@ github.com/inconshreveable/mousetrap v1.0.0/go.mod h1:PxqpIevigyE2G7u3NXJIT2ANyt
 github.com/jbenet/go-context v0.0.0-20150711004518-d14ea06fba99 h1:BQSFePA1RWJOlocH6Fxy8MmwDt+yVQYULKfN0RoTN8A=
 github.com/jbenet/go-context v0.0.0-20150711004518-d14ea06fba99/go.mod h1:1lJo3i6rXxKeerYnT8Nvf0QmHCRC1n8sfWVwXF2Frvo=
 github.com/jessevdk/go-flags v1.4.0/go.mod h1:4FA24M0QyGHXBuZZK/XkWh8h0e1EYbRYJSGM75WSRxI=
+github.com/josharian/intern v1.0.0 h1:vlS4z54oSdjm0bgjRigI+G1HpF+tI+9rE5LLzOg8HmY=
+github.com/josharian/intern v1.0.0/go.mod h1:5DoeVV0s6jJacbCEi61lwdGj/aVlrQvzHFFd8Hwg//Y=
 github.com/json-iterator/go v1.1.11/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4=
 github.com/jstemmer/go-junit-report v0.0.0-20190106144839-af01ea7f8024/go.mod h1:6v2b51hI/fHJwM22ozAgKL4VKDeJcHhJFhtBdhmNjmU=
 github.com/jstemmer/go-junit-report v0.9.1/go.mod h1:Brl9GWCQeLvo8nXZwPNNblvFj/XSXhF0NWZEnDohbsk=