[Internal] Improve the documentation of the Pydantic models

[peterschmidt85]

Problem

Currently, the reference pages generate the documentation for Pydantic models from two different sources: from the Pydantic fields (via typing_extensions.Annotated and scripts/docs/gen_schema_reference.py - for YAML schema) and from docstrings (via mkdocstrings - for the Python API). This leads to having the documentation of the Pydantic fields duplicated.

Solution

• Minimum: For Pydantic models that don't have methods, in Python API reference, switch to typing_extensions.Annotated via scripts/docs/gen_schema_reference.py. Remove the docstrings that are duplicated by typing_extensions.Annotated.
  • Switch to scripts/docs/gen_schema_reference.py for dstack.api.Task and dstack.api.Service, dstack.api.Resources, dstack.api.GPU, and dstack.api.Disk
  • Include the dostrings for the Pydantic class into the documentation generated by scripts/docs/gen_schema_reference.py
  • Implement a setting format: yaml|python for scripts/docs/gen_schema_reference.py (to use the table format, include type, examples link, etc)
• Maximum: Switch from mkdocstrings to scripts/docs/gen_schema_reference.py entirely for Python API reference generation by adding generation of the documentation for methods to scripts/docs/gen_schema_reference.py.

[pawamoy]

You might be interested in https://mkdocstrings.github.io/griffe-pydantic/ and https://github.com/pyapp-kit/griffe-fieldz 🙂

[peterschmidt85]

@pawamoy Thanks a lot for sharing this. Actually, I tried I suppose both but couldn't find a way to make them work. It simply didn't render any annotated fields.

Do you have any experience with any of these two?

[pawamoy]

griffe-pydantic is only available to sponsors (see the docs), the package on pypi does nothing. I didn't try griffe-fieldz.