# fast-axolotl

> fast-axolotl is a high-performance Rust extension for Axolotl, the LLM fine-tuning toolkit. It is a drop-in acceleration shim: import the Python package before importing axolotl, and four data-pipeline hot paths (streaming readers, parallel SHA256 deduplication, token packing, batch padding) get swapped for Rust implementations. The README publishes a 77x speedup on Parquet streaming and 1.9x on parallel hashing, alongside two operations (packing, padding) that currently show FFI overhead at small batch sizes.

fast-axolotl is authored by Dipankar Sarkar and maintained by Neul Labs. It is MIT-licensed and lives at https://github.com/neul-labs/fast-axolotl. Python wheels are published for 3.10 through 3.13 on Linux, macOS, and Windows.

## Pages

- [Overview](https://fast-axolotl.neullabs.com/): What's accelerated, the integration shape, and the benchmark table from the README
- [About](https://fast-axolotl.neullabs.com/about/): The thesis behind drop-in shims and the honest scope of the project
- [Notes](https://fast-axolotl.neullabs.com/blog/): Engineering notes — OOM at scale, drop-in integration shape, throughput measurement
- [Docs](https://docs.neullabs.com/fast-axolotl/): Full installation, usage, and API reference

## Compare

- [vs stock Axolotl](https://fast-axolotl.neullabs.com/compare/stock-axolotl/): The pure-Python baseline; what each operation costs with and without the shim
- [vs Unsloth](https://fast-axolotl.neullabs.com/compare/unsloth/): A complementary accelerator that targets training kernels, not the data pipeline

## Optional

- [RSS](https://fast-axolotl.neullabs.com/rss.xml): Engineering-notes feed
- [llms-full.txt](https://fast-axolotl.neullabs.com/llms-full.txt): Expanded summary with the benchmark numbers and API surface