// out-of-memory
Fix Axolotl RAM OOM on large datasets
Axolotl issue #2975 documents a large-dataset preload spiking RAM into the terabytes. The problem is not compute — it is materializing the entire corpus in memory before training starts.
Who it's for: Teams fine-tuning on multi-GB or TB-scale corpora that hit RAM limits during dataset load.
the problem
The stock pipeline holds the whole dataset in memory at once, so peak RAM scales with dataset size. On a large corpus that means an OOM kill before the first training step.
how fast-axolotl helps
fast-axolotl replaces the reader with a Rust streaming_dataset_reader that iterates fixed-size batches and never materializes the full dataset. Peak RAM is bounded by batch size. The README reports a 77x speedup on Parquet streaming (50,000 rows, 16-core Linux).
What that looks like in practice
- →Peak RAM bounded by batch_size, not dataset size — the OOM simply does not occur.
- →Parquet, Arrow, JSON, JSONL, CSV, and text supported, with transparent ZSTD/Gzip decompression.
- →No source edits to Axolotl — the shim installs itself into sys.modules.
- →Set dataset_use_rust_streaming: true and keep the rest of your YAML unchanged.
# axolotl config.yml
dataset_use_rust_streaming: true
sequence_len: 32768
dedupe: trueTry it on this workflow
Follow a guide, read the architecture, or see more use cases.