Skip to content

Issues: promptfoo/promptfoo

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

53 Open 373 Closed
53 Open 373 Closed
Author
Filter by author
Label
Filter by label
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Milestones
Filter by milestone
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

[Feature suggestion]: Exposing model costs for usage with Custom Providers
#940 opened Jun 16, 2024 by Codeshark-NET
[BUG] When using eval command there are diff between using config option and not using config option
#927 opened Jun 14, 2024 by suhwan-kang
1
Feature Request: [Web UI] Show assertion details when using assert-set
#919 opened Jun 11, 2024 by sjsinju
Failed to fetch when attempting to log in/sign up on the web-UI after promptfoo share
#892 opened Jun 7, 2024 by MehakGopal
expression to select subset of output to display in view
#879 opened Jun 6, 2024 by pelikhan
3
[Web UI] Image previewer not working for variables or failed tests
#865 opened Jun 4, 2024 by chrismaltais
Not working using text prompt with Palm2 Google Vertex API
#826 opened May 28, 2024 by rmonteiro89
1
Using vars on rubricPrompt lacks serialization, leading to the failure of the entire assertion.
#823 opened May 28, 2024 by YiFanChen99
3
Allow overriding provider config in prompt or test case
#815 opened May 24, 2024 by tomquist
n parameters does not work with openai as provider
#814 opened May 24, 2024 by harsh-writesonic
1
Markdown appearing when manually failing test case
#810 opened May 23, 2024 by chrismaltais
2
Web ui doesn't support object prompts config
#803 opened May 21, 2024 by kongmoumou
How can we secure self hosted promptfoo server?
#801 opened May 20, 2024 by harshaa947
Standardize how LLM output is stored in a ProviderResponse output object
#786 opened May 16, 2024 by efung
3
Browser exception (Minified React error #31; visit) when clicking magnifying glass in output
#768 opened May 9, 2024 by efung
1
Feature Request: Have optional vars in assertions with ability to define fallbacks
#766 opened May 7, 2024 by reallyinvincible
Multiline vars replacement break indentation in config generation
#759 opened May 2, 2024 by yleflour
Multiple prompts in a single prompt file when using multiple prompt files
#744 opened Apr 30, 2024 by alexclaydon
Random sampling of n data rows
#740 opened Apr 30, 2024 by alexclaydon
1
Docs request: where does histogram come from?
#738 opened Apr 30, 2024 by jamesbraza
1
Request: ability to exclude Description from promptfoo view and persist UI settings across refresh
#736 opened Apr 30, 2024 by jamesbraza
3
Caching concurrent calls in Python provider does not use new caching behaviour
#730 opened Apr 29, 2024 by JonnySpruce
Request: suppress warning Treating it as a text prompt. from each eval
#727 opened Apr 29, 2024 by jamesbraza
3
Request: JSON schema for promptfooconfig.yaml
#726 opened Apr 29, 2024 by jamesbraza
allow provider to report updated prompt?
#718 opened Apr 26, 2024 by pelikhan
1
ProTip! Updated in the last three days: updated:>2024-06-13.