Memory Management

Jump to bottom

Gary Miguel edited this page Oct 20, 2021 · 2 revisions

BFCArena

Why we need an arena? What problem does it solve?

Memory Planning

Memory Pattern

How does ORT come up with a peak memory consumption?

External Torch's CUDACachingAllocator

How does ORTModule uses pytorch allocator? See _graph_execution_manager.py
Advanced: What's the difference between BFCArena and CUDACachingAllocator?