Introducing Falcon-H1-Arabic: Pushing the Boundaries of Arabic Language AI with Hybrid Architecture

TII announced Falcon-H1-Arabic, a family of Arabic language models in 3B, 7B, and 34B parameter sizes, published on Hugging Face. The models use a hybrid Mamba-Transformer architecture and extend context windows up to 256K tokens, an increase from Falcon-Arabic's 32K limit. TII trained the models on approximately 300 billion tokens combining Arabic, English, and multilingual content. The team includes Basma Boussaha, Mohammed Alyafeai, Hakim Hacid, and other TII researchers.