[Roadmap] Q3 2024

[peterschmidt85]

This issue outlines the major items planned for Q3 2024. Doesn't include bug fixes, except for major ones.

Important

Community feedback and contributions are very welcome. Please suggest additions or changes in the comments.

Core features

• Support volumes [Feature]: Support dstack volumes #1158 [Feature]: Support Runpod network volumes #1296
• Support private subnets (incl. gateways) Support provisioning instances without public IPs on AWS #1201 Support internal-facing gateways on AWS #1171 Support gateways behind ALB with ACM certificate #1264 Support specifying non-default VPC for GCP backend #1304
• Unified interface for managing resources of different types (dstack apply, etc) Implement dstack apply for gateways #1223 Support running run configurations with dstack apply #1326 Support adding instances with dstack apply #1327
• Support for reserved instances [Feature]: Support the Capacity Blocks for ML feature for AWS #1155
• Better support for clusters (incl. TPU, on-prem, and other user cases) Support multi-device TPU Pods #1337
• Support secrets Implement secrets #708

Supported architectures

• TPU [Feature]: Add TPU support #956
• AMD (Epyc/etc)
• AWS (Trainium/Inferentia)
• Intel (Gaudi/etc)

Examples

Important

Community help is needed!

• Multi-node HuggingFace Alignment Handbook
• TensorRT-LLM
• Triton
• GitHub Action
• Function calling
• SD3

Improvements

• Support healthchecks [Feature]: Continuous health checks on services with reactions like pod-replacements  #1349
• Support for H100 in AWS/GCP/Azure
• Support for L4 in AWS
• Support for H200 in AWS/GCP/Azure
• Support non-wildcard domains for gateway

Research

Important

Research and feedback is required!

• Fault-tolerant training: Research how dstack can be used for fault-tolerant training of massive models.
• Control plane: Research the importance of including the web UI for managing runs, projects, and users in dstack. #1361
• Metrics: Research whether dstack should collect certain metrics out of the box (at least hardware utilization) or if it should be integrated with more enterprise-grade tools.