test: require CI workflow validator in release gates

This commit is contained in:
Hermes Agent
2026-09-07 10:35:22 +00:00
parent e4d4db3541
commit fa6be94a3e
6 changed files with 11 additions and 2 deletions
+1
View File
@@ -11,6 +11,7 @@
- Added focused regression coverage for the workflow validator. - Added focused regression coverage for the workflow validator.
- `tools/validate_ci_workflow.py` now checks release commands inside actual `run` steps, so commented-out commands cannot make CI coverage appear complete. - `tools/validate_ci_workflow.py` now checks release commands inside actual `run` steps, so commented-out commands cannot make CI coverage appear complete.
- The validator now requires its own executable CI command, protecting the workflow coverage check from being removed silently.
### Object-alias field CHECK TYPE diagnostics ### Object-alias field CHECK TYPE diagnostics
+2
View File
@@ -28,6 +28,8 @@ python3 tools/validate_package_security.py
python3 tools/validate_ci_workflow.py python3 tools/validate_ci_workflow.py
``` ```
The CI workflow validator also requires its own command to remain in an executable `run` step, so the release-gate check cannot silently disappear from CI.
## Beginner-first syntax ## Beginner-first syntax
Claro accepts both the sentence-like style and the shorter simple style: Claro accepts both the sentence-like style and the shorter simple style:
+1 -1
View File
@@ -175,7 +175,7 @@ Ready now:
- metadata JSON - metadata JSON
- completion list - completion list
- diagnostics helper - diagnostics helper
- Forgejo/Gitea CI coverage for the current release validation gates (`claro validate`, typecheck diagnostics validation, version convention validation, package security validation, and the CI workflow coverage check); the workflow validator verifies these commands occur in executable `run` steps rather than comments - Forgejo/Gitea CI coverage for the current release validation gates (`claro validate`, typecheck diagnostics validation, version convention validation, package security validation, and the CI workflow coverage check); the workflow validator verifies these commands, including its own command, occur in executable `run` steps rather than comments
Still needed: Still needed:
- syntax highlighting package - syntax highlighting package
+1 -1
View File
@@ -31,7 +31,7 @@ See `CURRENT_STATUS.md` for the detailed feature matrix.
1. Keep beginner-facing docs current and separate from historical release notes. 1. Keep beginner-facing docs current and separate from historical release notes.
2. Keep examples aligned with the modern simple syntax (`END`, `DO`, short `SET`, short `ASK`) while documenting older compatibility forms separately. 2. Keep examples aligned with the modern simple syntax (`END`, `DO`, short `SET`, short `ASK`) while documenting older compatibility forms separately.
3. Expand validation around typecheck diagnostics and package/networking safety. Current Forgejo/Gitea CI runs the documented release gates (`claro validate`, typecheck diagnostics validation, version convention validation, package security validation, and CI workflow coverage validation). The workflow coverage validator checks executable `run` steps rather than comments. Current package safety validation covers safe project creation, rejecting unsafe project names during `claro new`, safe manifest/lockfile creation, rejecting unsafe package names during `package add`, detecting unsafe package names already present in `claro.project` during `package doctor`, refusing to write lockfile data for unsafe package names during `package lock`, making `package list` fail instead of displaying unsafe package entries as normal package names, making `package init` fail instead of reporting the project ready when unsafe package names are already present, making `package add` fail before changing files when unsafe package names are already present, making `package remove` fail instead of reporting success when unsafe package names remain during lockfile refresh, allowing `package remove` to remove an exact unsafe package entry so learners can repair a bad `claro.project`, making `package doctor` reject stale lockfile checksums for listed packages, and making `package doctor` reject lockfile package entries that are not listed in `claro.project`. Current function validation covers correct checked calls, wrong-type arguments, missing checked-argument diagnostics, missing unchecked-argument diagnostics for simple functions in both modern `DO greet` and empty compatibility `CALL greet WITH` forms, extra-argument diagnostics, and beginner-facing unknown-function diagnostics for mistyped modern `DO` and compatibility `CALL ... WITH` calls; current object-method parameter validation covers correct modern `DO object.method ...` and compatibility `CALL object.method WITH ...` checked calls, wrong-type diagnostics for both call forms, checked methods after another method in the same class, missing and extra checked-argument diagnostics including the extra-argument case for a checked method after another method in the same class, missing unchecked-argument diagnostics for simple object methods, missing-object guidance when `DO player.method ...` or compatibility `CALL object.method WITH ...` appears before `NEW` even if the method name is also wrong, and class-specific unknown-method diagnostics for both modern and compatibility calls when a learner calls a method the class does not declare; current object-field validation covers direct NUMBER/TEXT/YESNO field-assignment positives, direct object-field `CHECK TYPE` metadata positives for NUMBER/TEXT/YESNO fields, negative NUMBER/TEXT/YESNO field metadata mismatches, NUMBER/TEXT/YESNO-expectation unknown-field `CHECK TYPE` diagnostics, direct wrong-type field assignment diagnostics, field collection when a `HAS` field appears after a simple method, direct unknown-field diagnostics for NUMBER/TEXT/YESNO values, a beginner-facing fallback for unknown fields assigned from expressions whose type is not inferable yet, and missing-object diagnostics for both `SET object.field value` and `CHECK TYPE object.field IS TYPE` before `NEW`. 3. Expand validation around typecheck diagnostics and package/networking safety. Current Forgejo/Gitea CI runs the documented release gates (`claro validate`, typecheck diagnostics validation, version convention validation, package security validation, and CI workflow coverage validation). The workflow coverage validator checks executable `run` steps rather than comments and requires its own validator command. Current package safety validation covers safe project creation, rejecting unsafe project names during `claro new`, safe manifest/lockfile creation, rejecting unsafe package names during `package add`, detecting unsafe package names already present in `claro.project` during `package doctor`, refusing to write lockfile data for unsafe package names during `package lock`, making `package list` fail instead of displaying unsafe package entries as normal package names, making `package init` fail instead of reporting the project ready when unsafe package names are already present, making `package add` fail before changing files when unsafe package names are already present, making `package remove` fail instead of reporting success when unsafe package names remain during lockfile refresh, allowing `package remove` to remove an exact unsafe package entry so learners can repair a bad `claro.project`, making `package doctor` reject stale lockfile checksums for listed packages, and making `package doctor` reject lockfile package entries that are not listed in `claro.project`. Current function validation covers correct checked calls, wrong-type arguments, missing checked-argument diagnostics, missing unchecked-argument diagnostics for simple functions in both modern `DO greet` and empty compatibility `CALL greet WITH` forms, extra-argument diagnostics, and beginner-facing unknown-function diagnostics for mistyped modern `DO` and compatibility `CALL ... WITH` calls; current object-method parameter validation covers correct modern `DO object.method ...` and compatibility `CALL object.method WITH ...` checked calls, wrong-type diagnostics for both call forms, checked methods after another method in the same class, missing and extra checked-argument diagnostics including the extra-argument case for a checked method after another method in the same class, missing unchecked-argument diagnostics for simple object methods, missing-object guidance when `DO player.method ...` or compatibility `CALL object.method WITH ...` appears before `NEW` even if the method name is also wrong, and class-specific unknown-method diagnostics for both modern and compatibility calls when a learner calls a method the class does not declare; current object-field validation covers direct NUMBER/TEXT/YESNO field-assignment positives, direct object-field `CHECK TYPE` metadata positives for NUMBER/TEXT/YESNO fields, negative NUMBER/TEXT/YESNO field metadata mismatches, NUMBER/TEXT/YESNO-expectation unknown-field `CHECK TYPE` diagnostics, direct wrong-type field assignment diagnostics, field collection when a `HAS` field appears after a simple method, direct unknown-field diagnostics for NUMBER/TEXT/YESNO values, a beginner-facing fallback for unknown fields assigned from expressions whose type is not inferable yet, and missing-object diagnostics for both `SET object.field value` and `CHECK TYPE object.field IS TYPE` before `NEW`.
3a. Keep package diagnostics actionable: invalid package manifest format fields now name the file and explain how to repair it, separately from an absent manifest. Keep package validation honest by checking that the project manifest has one non-empty project name, that the project and each listed package manifest declare the exact supported manifest version, package version, local source, and expected package name, and that lockfiles declare exactly one current release version. 3a. Keep package diagnostics actionable: invalid package manifest format fields now name the file and explain how to repair it, separately from an absent manifest. Keep package validation honest by checking that the project manifest has one non-empty project name, that the project and each listed package manifest declare the exact supported manifest version, package version, local source, and expected package name, and that lockfiles declare exactly one current release version.
3b. Keep package security validation in Forgejo/Gitea CI so unsafe names, exact manifest-version/name/version/checksum mismatches, missing manifests, duplicate project manifest versions and packages, duplicate lock packages, duplicate package manifest names, versions, and checksums, and lockfile errors remain release blockers. 3b. Keep package security validation in Forgejo/Gitea CI so unsafe names, exact manifest-version/name/version/checksum mismatches, missing manifests, duplicate project manifest versions and packages, duplicate lock packages, duplicate package manifest names, versions, and checksums, and lockfile errors remain release blockers.
4. Add small examples for each foundation feature before adding bigger syntax. The object-field foundation now includes positive and negative validation for field expressions such as `SET player.score player.name`, `SET player.score player.score + 1`, `SET player.name player.score + 1`, explicitly typed field assignments, simple and chained aliases such as `SET alias player`, `SET backup alias`, followed by `SET backup.score ...` or `CHECK TYPE backup.score IS ...`, chained aliases in object-method calls through both modern `DO` and compatibility `CALL ... WITH` forms (including a dedicated positive modern `DO` fixture), and arithmetic/text expression mismatches; typed containers also accept a map with nested type metadata when it is added to a `LIST OF MAP`; broader alias/control-flow checking remains planned. 4. Add small examples for each foundation feature before adding bigger syntax. The object-field foundation now includes positive and negative validation for field expressions such as `SET player.score player.name`, `SET player.score player.score + 1`, `SET player.name player.score + 1`, explicitly typed field assignments, simple and chained aliases such as `SET alias player`, `SET backup alias`, followed by `SET backup.score ...` or `CHECK TYPE backup.score IS ...`, chained aliases in object-method calls through both modern `DO` and compatibility `CALL ... WITH` forms (including a dedicated positive modern `DO` fixture), and arithmetic/text expression mismatches; typed containers also accept a map with nested type metadata when it is added to a `LIST OF MAP`; broader alias/control-flow checking remains planned.
+5
View File
@@ -26,9 +26,14 @@ class ValidateCiWorkflowTests(unittest.TestCase):
run: python3 tools/validate_version_convention.py run: python3 tools/validate_version_convention.py
- name: Packages - name: Packages
run: python3 tools/validate_package_security.py run: python3 tools/validate_package_security.py
- name: CI workflow
run: python3 tools/validate_ci_workflow.py
""" """
self.assertEqual(MODULE.missing_release_gates(workflow), []) self.assertEqual(MODULE.missing_release_gates(workflow), [])
def test_requires_the_workflow_validator_to_run_in_ci(self):
self.assertIn("python3 tools/validate_ci_workflow.py", MODULE.REQUIRED_COMMANDS)
def test_rejects_commands_that_only_appear_in_comments(self): def test_rejects_commands_that_only_appear_in_comments(self):
workflow = """ workflow = """
steps: steps:
+1
View File
@@ -11,6 +11,7 @@ REQUIRED_COMMANDS = [
"python3 tools/validate_typecheck_diagnostics.py", "python3 tools/validate_typecheck_diagnostics.py",
"python3 tools/validate_version_convention.py", "python3 tools/validate_version_convention.py",
"python3 tools/validate_package_security.py", "python3 tools/validate_package_security.py",
"python3 tools/validate_ci_workflow.py",
] ]