Rigel
Rigel: a hybrid Mamba–attention MoE approaching Llama-3.2 quality with a fraction of the training compute. Trained on a mix of A100s, H100s, and TPUs, thanks to Mayank.
Rigel: a hybrid Mamba–attention MoE approaching Llama-3.2 quality with a fraction of the training compute. Trained on a mix of A100s, H100s, and TPUs, thanks to Mayank.