[skip changelog] Sync "Check Python" CI workflow with template (#1393)

* [skip changelog] Sync "Check Python" CI workflow with template

We have assembled a collection of reusable project assets:
https://github.com/arduino/tooling-project-assets
These assets will be used in the repositories of all Arduino tooling projects.

Some minor improvements and standardizations have been made in the upstream "template" assets, and those are
introduced to this repository here.

Notable:

- Check for black formatting style compliance
- Update tool dependencies
- Standardize linting task name and deprecate old task name
- Add cyclomatic complexity check
- Add pep8-naming plugin to flake8

* [skip changelog] Restore formatting compliance of Python code after black update

After updating the black formatter tool, the new version wanted a small change in the Python formatting.

* Remove obsolete Python linting task

The sync with the template documentation assets has resulted in new standardized names for some of the tasks.

The `python:check` task has been renamed to `python:lint`

.flake8

-# TODO: move this to pyproject.toml when supported: https://gitlab.com/pycqa/flake8/merge_requests/245
+# Source: https://github.com/arduino/tooling-project-assets/blob/main/workflow-templates/assets/check-python/.flake8
+# See: https://flake8.pycqa.org/en/latest/user/configuration.html
+# The code style defined in this file is the official standardized style to be used in all Arduino tooling projects and
+# should not be modified.
 
 [flake8]
-max-line-length = 120
+doctests = True
 ignore =
     E741,
-    # W503 and W504 are mutually exclusive, so one or the other must be ignored.
-    # PEP 8 recommends line break before, so we keep W504.
+    # W503 and W504 are mutually exclusive. PEP 8 recommends line break before.
     W503
+max-complexity = 10
+max-line-length = 120
+select = E,W,F,C,N

.github/workflows/check-python-task.yml

+# Source: https://github.com/arduino/tooling-project-assets/blob/main/workflow-templates/check-python-task.md
+name: Check Python
+
+env:
+  # See: https://github.com/actions/setup-python/tree/v2#available-versions-of-python
+  PYTHON_VERSION: "3.9"
+
+# See: https://docs.github.com/en/actions/reference/events-that-trigger-workflows
+on:
+  push:
+    paths:
+      - ".github/workflows/check-python-task.ya?ml"
+      - "**/.flake8"
+      - "**/poetry.lock"
+      - "**/pyproject.toml"
+      - "**/setup.cfg"
+      - "Taskfile.ya?ml"
+      - "**/tox.ini"
+      - "**.py"
+  pull_request:
+    paths:
+      - ".github/workflows/check-python-task.ya?ml"
+      - "**/.flake8"
+      - "**/poetry.lock"
+      - "**/pyproject.toml"
+      - "**/setup.cfg"
+      - "Taskfile.ya?ml"
+      - "**/tox.ini"
+      - "**.py"
+  workflow_dispatch:
+  repository_dispatch:
+
+jobs:
+  lint:
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v2
+
+      - name: Install Python
+        uses: actions/setup-python@v2
+        with:
+          python-version: ${{ env.PYTHON_VERSION }}
+
+      - name: Install Poetry
+        run: pip install poetry
+
+      - name: Install Task
+        uses: arduino/setup-task@v1
+        with:
+          repo-token: ${{ secrets.GITHUB_TOKEN }}
+          version: 3.x
+
+      - name: Run flake8
+        uses: liskin/gh-problem-matcher-wrap@v1
+        with:
+          linters: flake8
+          run: task python:lint
+
+  formatting:
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v2
+
+      - name: Install Python
+        uses: actions/setup-python@v2
+        with:
+          python-version: ${{ env.PYTHON_VERSION }}
+
+      - name: Install Poetry
+        run: pip install poetry
+
+      - name: Install Task
+        uses: arduino/setup-task@v1
+        with:
+          repo-token: ${{ secrets.GITHUB_TOKEN }}
+          version: 3.x
+
+      - name: Format Python code
+        run: task python:format
+
+      - name: Check formatting
+        run: git diff --color --exit-code

.github/workflows/python-lint.yaml

-name: "Lints Python code"
-
-on:
-  push:
-    paths:
-      - "**.py"
-      - ".flake8"
-  pull_request:
-    paths:
-      - "**.py"
-      - ".flake8"
-
-jobs:
-  python-linting:
-    name: "Lints Python code"
-    runs-on: ubuntu-latest
-
-    steps:
-      - name: Checkout
-        uses: actions/checkout@v2
-
-      - name: Install Taskfile
-        uses: arduino/setup-task@v1
-        with:
-          repo-token: ${{ secrets.GITHUB_TOKEN }}
-          version: 3.x
-
-      - name: Activate Python
-        uses: actions/setup-python@v2.2.2
-        with:
-          python-version: "3.8"
-          architecture: "x64"
-
-      - name: Install Poetry
-        run: pip install poetry
-
-      - name: Lints Python files
-        run: task python:check

Taskfile.yml

@@ -189,6 +189,22 @@ tasks:
     cmds:
       - clang-format -i rpc/cc/arduino/cli/*/*/*.proto
 
+  # Source: https://github.com/arduino/tooling-project-assets/blob/main/workflow-templates/assets/check-python-task/Taskfile.yml
+  python:format:
+    desc: Format Python files
+    deps:
+      - task: poetry:install-deps
+    cmds:
+      - poetry run black .
+
+  # Source: https://github.com/arduino/tooling-project-assets/blob/main/workflow-templates/assets/check-python-task/Taskfile.yml
+  python:lint:
+    desc: Lint Python code
+    deps:
+      - task: poetry:install-deps
+    cmds:
+      - poetry run flake8 --show-source
+
   build:
     desc: Build the project
     deps:
@@ -219,18 +235,6 @@ tasks:
       - task: python:check
       - task: protoc:check
 
-  python:check:
-    desc: Linting for Python files
-    cmds:
-      - poetry install --no-root
-      - poetry run flake8
-
-  python:format:
-    desc: Automatically formats Python files
-    cmds:
-      - poetry install --no-root
-      - poetry run black .
-
   check-legacy:
     desc: Check fmt and lint for the `legacy` package
     cmds:

docs/CONTRIBUTING.md

@@ -212,7 +212,7 @@ pytest test_lib.py::test_list
 When editing any Python file in the project remember to run linting checks with:
 
 ```shell
-task python:check
+task python:lint
 ```
 
 This will run `flake8` automatically and return any error in the code formatting, if not already installed it will also

pyproject.toml

@@ -15,8 +15,6 @@ prometheus-client = "0.7.1"
 requests = "2.22.0"
 pytest-timeout = "1.3.4"
 invoke = "1.4.1"
-flake8 = "^3.8.3"
-black = { version = "^19.10b0", allow-prereleases = true }
 filelock = "^3.0.12"
 pytest-xdist = "^2.1.0"
 pytest_httpserver = "^0.3.5"
@@ -28,6 +26,9 @@ mkdocs-material = "^7.1.8"
 mdx-truly-sane-lists = "^1.2"
 GitPython = "^3.1.20"
 mike = "^1.0.1"
+black = "^21.7b0"
+flake8 = "^3.9.2"
+pep8-naming = "^0.12.1"
 
 [tool.black]
 line-length = 120

test/test_core.py

@@ -201,7 +201,8 @@ def test_core_install_without_updateindex(run_command):
 
 
 @pytest.mark.skipif(
-    platform.system() == "Windows", reason="core fails with fatal error: bits/c++config.h: No such file or directory",
+    platform.system() == "Windows",
+    reason="core fails with fatal error: bits/c++config.h: No such file or directory",
 )
 def test_core_install_esp32(run_command, data_dir):
     # update index
@@ -685,7 +686,9 @@ def test_core_with_wrong_custom_board_options_is_loaded(run_command, data_dir):
 
     # Install platform in Sketchbook hardware dir
     shutil.copytree(
-        Path(__file__).parent / "testdata" / test_platform_name, platform_install_dir, dirs_exist_ok=True,
+        Path(__file__).parent / "testdata" / test_platform_name,
+        platform_install_dir,
+        dirs_exist_ok=True,
     )
 
     assert run_command("update")
@@ -721,7 +724,9 @@ def test_core_with_missing_custom_board_options_is_loaded(run_command, data_dir)
 
     # Install platform in Sketchbook hardware dir
     shutil.copytree(
-        Path(__file__).parent / "testdata" / test_platform_name, platform_install_dir, dirs_exist_ok=True,
+        Path(__file__).parent / "testdata" / test_platform_name,
+        platform_install_dir,
+        dirs_exist_ok=True,
     )
 
     assert run_command("update")