Musk's AI company xAI has officially released the Grok-1 source code

ChainCatcher news, Elon Musk's artificial intelligence company xAI has announced the release of the weights and architecture of its 314 billion parameter mixture of experts model Grok-1. Grok-1 is a large language model trained from scratch by xAI, utilizing a mixture of experts model architecture where 25% of the weights are active on a given token. The base model weights and network architecture of the model have now been made available under the Apache 2.0 license.

It is reported that the pre-training phase of Grok-1 concluded in October 2023, and the current release is the original base model checkpoint, which has not been fine-tuned for any specific application.