Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature Request: [Web UI] Show assertion details when using assert-set #919

Open
sjsinju opened this issue Jun 11, 2024 · 0 comments
Open
Labels
enhancement New feature or request

Comments

@sjsinju
Copy link

sjsinju commented Jun 11, 2024

Thank you for adding #765 (assert-set) and #903 (__threshold for csv). 馃憤

But I found an issue while using it.
I thought below should be shown same behavior.

a.yaml

- vars:
      abc: "test"
- assert:
      type: assert-set
      threshold: 0.5
      assert:
        - type: python
          value: "file://a.py"
        - type: python
          value: "file://b.py

a.csv

abc __expected1 __expected2 __threshold
test file://a.py file://b.py 0.5

But I got results on Web UI

  1. for a.yaml : Just shown Aggregate score 1 > 0.5 threshold in overview and detail view
    image

  2. for a.csv : Aggregate score 1 > 0.5 threshold in overview is same with a.yaml, but detail view shows pass/fail details for all of assertions

Is this the intended result?
I think 1 should also show assertions details in detail view same as 2. If no, could you explain this differences?

@sjsinju sjsinju changed the title [Web UI] Show assertion details when using assert-set Feature Request: Show assertion details when using assert-set Jun 12, 2024
@sjsinju sjsinju changed the title Feature Request: Show assertion details when using assert-set Feature Request: [Web UI] Show assertion details when using assert-set Jun 12, 2024
@typpo typpo added the enhancement New feature or request label Jun 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants