Skip to content
@prometheus-eval

prometheus-eval

Codebase to inference and train foundation models specialized on evaluating other foundation models
README.md

We train language models specialized in evaluating other language models and optimize evaluation pipelines!

Repositories

Below are our key projects, with links to their repositories and related publications:

Repository Description Paper
prometheus-eval A repository for evaluating LLMs in generation tasks. Supports Prometheus 2, GPT-4, and others. Link
prometheus An Evaluator LM that is open-source, offers reproducible evaluation, and inexpensive to use. Link
prometheus-vision An Evaluator VLM that is open-source, offers reproducible evaluation, and inexpensive to use. Link

Popular repositories Loading

  1. prometheus-eval prometheus-eval Public

    Evaluate your LLM's response with Prometheus and GPT4 đź’Ż

    Python 655 34

  2. prometheus prometheus Public

    [ICLR 2024 & NeurIPS 2023 WS] An Evaluator LM that is open-source, offers reproducible evaluation, and inexpensive to use. Specifically designed for fine-grained evaluation on a customized score ru…

    Python 277 18

  3. prometheus-vision prometheus-vision Public

    [ACL 2024 Findings & ICLR 2024 WS] An Evaluator VLM that is open-source, offers reproducible evaluation, and inexpensive to use. Specifically designed for fine-grained evaluation on customized scor…

    Python 45 6

  4. .github .github Public

    Organization README for prometheus-eval

  5. prometheus-eval.github.io prometheus-eval.github.io Public

    Documentation and blogposts for Prometheus

    1

  6. leaderboard leaderboard Public

    BiGGen-Bench Leaderboard

    Python

Repositories

Showing 6 of 6 repositories
  • prometheus-eval Public

    Evaluate your LLM's response with Prometheus and GPT4 đź’Ż

    prometheus-eval/prometheus-eval’s past year of commit activity
    Python 655 Apache-2.0 34 3 0 Updated Jun 15, 2024
  • .github Public

    Organization README for prometheus-eval

    prometheus-eval/.github’s past year of commit activity
    0 0 0 0 Updated Jun 11, 2024
  • leaderboard Public

    BiGGen-Bench Leaderboard

    prometheus-eval/leaderboard’s past year of commit activity
    Python 0 0 0 0 Updated Jun 4, 2024
  • prometheus-vision Public

    [ACL 2024 Findings & ICLR 2024 WS] An Evaluator VLM that is open-source, offers reproducible evaluation, and inexpensive to use. Specifically designed for fine-grained evaluation on customized score rubric, Prometheus-Vision is a good alternative for human evaluation and GPT-4V evaluation.

    prometheus-eval/prometheus-vision’s past year of commit activity
    Python 45 Apache-2.0 6 2 0 Updated May 16, 2024
  • prometheus-eval.github.io Public

    Documentation and blogposts for Prometheus

    prometheus-eval/prometheus-eval.github.io’s past year of commit activity
    0 1 0 0 Updated May 1, 2024
  • prometheus Public

    [ICLR 2024 & NeurIPS 2023 WS] An Evaluator LM that is open-source, offers reproducible evaluation, and inexpensive to use. Specifically designed for fine-grained evaluation on a customized score rubric, Prometheus is a good alternative for human evaluation and GPT-4 evaluation.

    prometheus-eval/prometheus’s past year of commit activity
    Python 277 MIT 18 4 0 Updated Nov 11, 2023

Top languages

Python

Most used topics

Loading…