Axolotl runs out of RAM loading my large dataset — how do I fix it?
Stock Axolotl materializes the dataset in memory before training, which blows up on large corpora — Axolotl issue #2975 documents a 1.3 TB RAM spike loading a large dataset. fast-axolotl replaces the reader with a Rust streaming_dataset_reader that iterates the file in fixed-size batches and never holds the whole dataset in RAM. The README reports a 77x speedup on Parquet streaming (50,000 rows, 16-core Linux). Install with uv add fast-axolotl (or pip), import fast_axolotl before axolotl, and set dataset_use_rust_streaming: true in your YAML.
The failure mode in issue #2975 is a preload, not a compute problem: the pipeline tries to hold the entire dataset in memory at once. Streaming sidesteps it by reading fixed-size batches on demand, so peak RAM is bounded by the batch size rather than the dataset size. Parquet, Arrow, JSON, JSONL, CSV, and text are all supported, with transparent ZSTD/Gzip decompression.
How do I make Axolotl data loading faster for big datasets?
fast-axolotl swaps two data-pipeline hot paths for Rust at import time. Streaming readers (Parquet, Arrow, JSON, JSONL, CSV, text — with ZSTD/Gzip) carry the README's 77x Parquet-streaming number, and parallel SHA256 deduplication runs multi-threaded at 1.9x over a hashlib loop (100,000 rows, 16 cores). There is no config diff beyond importing fast_axolotl before axolotl; set dataset_use_rust_streaming: true for datasets over ~1 GB or long sequence lengths.
The shim installs itself into sys.modules, so existing axolotl.utils.* import paths get the Rust implementations without any source change to Axolotl or your training script. The wins are on the read and dedupe stages — the parts that dominate wall-clock when the data, not the GPU, is the bottleneck.
Can I use fast-axolotl alongside Unsloth?
Yes — they are complementary, not competitors. Unsloth accelerates the training / kernel phase (its published positioning is roughly 2-5x faster training and about 80% less memory via fused attention and LoRA kernels). fast-axolotl accelerates the earlier data-loading and preprocessing phase (streaming reads, dedup, packing, padding). They target different bottlenecks, so you can run both together: Unsloth on the compute-bound training step, fast-axolotl on the data pipeline that feeds it.
If your wall-clock is dominated by reading and deduplicating a large dataset, fast-axolotl is the lever; if it is dominated by attention/LoRA throughput on the GPU, Unsloth is. Most real fine-tuning runs have both costs. See the side-by-side on the fast-axolotl vs Unsloth comparison page for what each layer optimizes.
When does fast-axolotl NOT help?
At small batch sizes the token-packing and batch-padding paths are currently slower than plain Python, not faster: the README benchmark shows pack_sequences at about 0.4x and pad_sequences at about 0.5x on 10,000-sequence batches, because the FFI (Python-to-Rust) crossing cost dominates at that size. fast-axolotl also will not help if data loading is not your bottleneck, or if you cannot install a Rust-built wheel for policy reasons.
This is documented honestly rather than hidden. The streaming reader and parallel hashing are the clean wins today; packing and padding are exposed but only expected to pay off at larger training-scale batches where the per-call FFI overhead is amortized. If your bottleneck is GPU compute rather than the data pipeline, reach for a training-phase accelerator like Unsloth instead.