RAGE Retrieval Augmented Generative Engine

Day 1 of the AMD × lablab.ai Developer Hackathon. Today the scaffolding goes up: mindXtrain, a one-command Qwen3 fine-tuner native to AMD MI300X. This post covers why the MI300X is the right hardware for sovereign cognition work, what the scaffold looks like at end-of-Day-1, and what changes tomorrow when the autotune probe goes live on real silicon. 1. Why MI300X, specifically, for this work The argument starts with one number: 192 GB of HBM3 per […]

The 2024 article on production_transformer.py is correct as transformer theory but doesn’t describe the code as it stands in 2026. Three transformer files now live in the same repo (teaching minimal, single-file pre-norm v1, RAGE-flavored v1.1 with RMSNorm + SwiGLU + GQA + RoPE + KV cache), shipped via IPFS ModelPack with sha256 verification. Here is the operational ground truth.