Skip to content

feat: check test fields (info-only) - #934

Open
a-zw wants to merge 3 commits into
eclipse-score:mainfrom
etas-contrib:check-all-testcases
Open

a-zw wants to merge 3 commits into
eclipse-score:mainfrom
etas-contrib:check-all-testcases

Conversation

@a-zw

@a-zw a-zw commented Oct 9, 2026

Copy link
Copy Markdown
Contributor

📌 Description

Check all testcase need items - even the external ones.

🚨 Impact Analysis

  • This change does not violate any tool requirements and is covered by existing tool requirements
  • This change does not violate any design decisions
  • Otherwise I have created a ticket for new tool qualification

✅ Checklist

  • Added/updated documentation for new or changed features
  • Added/updated tests to cover the changes
  • Followed project coding standards and guidelines

@github-actions

Copy link
Copy Markdown
Contributor

Documentation preview for this pull request is available at:
pr-934: https://eclipse-score.github.io/docs-as-code/pr-934/

@@ -0,0 +1,92 @@
# *******************************************************************************

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What exactly are we testing here?
The links inside the full/partially verifies?

@a-zw a-zw Oct 13, 2026 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@MaximilianSoerenPollak

Copy link
Copy Markdown
Contributor

Should these be tested & asserted here? I guess in a way we need it to have better coverage, but those tests & checks I would see actually in the plugin so in score_tools?

There we also have commented out a bunch of asserts so that incomplete tests-case decorators still get created etc.
Probably need to think if it's time for it to not do that? Not sure.

@a-zw

a-zw commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor Author

Should these be tested & asserted here? I guess in a way we need it to have better coverage, but those tests & checks I would see actually in the plugin so in score_tools?

There we also have commented out a bunch of asserts so that incomplete tests-case decorators still get created etc. Probably need to think if it's time for it to not do that? Not sure.

Cannot really write tests yet because of the "info-only". Waiting for a later release to make these mandatory.

The whole point of this PR is to have checks not in score_tools. Checking there does not catch testcase items introduced from other sources.

#
# SPDX-License-Identifier: Apache-2.0
# *******************************************************************************
"""Validation rules for testcase needs, independent of their source."""

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you explain that/why metadata.yml constraints are not applied here

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because they are external needs which are excluded from the graph & local checks via our filter.

But I think it makes more sense to include them in that filter instead of making a whole new file for them specifically.

@a-zw
a-zw marked this pull request as draft October 15, 2026 07:15
@MaximilianSoerenPollak

Copy link
Copy Markdown
Contributor

@a-zw Is this still up to date and relevant, or should I not review this?

@a-zw

a-zw commented Nov 10, 2026

Copy link
Copy Markdown
Contributor Author

@a-zw Is this still up to date and relevant, or should I not review this?

There is a design decision to be made: This PR or rather the approach in #928 ?

@a-zw
a-zw marked this pull request as ready for review November 10, 2026 14:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

3 participants