xAI open sources the base model weights and network architecture of Grok-1, a 314B parameter Mixture-of-Experts model trained in October 2023, under Apache 2.0
We are releasing the base model weights and network architecture of Grok-1, our large language model.
xAI
Context & Ripple Effects
Days after saying it would make Grok available as open source, xAI has moved from a subscription-only initial rollout toward distributing a model artifact developers can inspect and run. The prior pledge to open-source Grok followed its limited-user launch.
This is the first step in a release pattern that later included Grok 2.5 weights on Hugging Face, making this release an early indicator of xAI pairing proprietary product access with periodic weight releases.
First-order effects
Developers and researchers can inspect Grok-1's architecture and use its released weights under Apache 2.0, rather than relying solely on xAI-hosted access.
xAI gives Grok-1 a distribution path outside its own service while retaining the ability to advance newer Grok models separately.
Second-order effects
Organizations with sufficient compute can evaluate or adapt Grok-1 in their own environments, increasing the relevance of infrastructure, hosting, and tooling around portable model weights.
Competing model providers face added pressure to distinguish hosted offerings through newer capabilities, product integration, or their own openness terms rather than access to a base model alone.
Third-order effects
If periodic releases continue, frontier-model competition may increasingly split between open, reusable older generations and closed access to the newest systems.
The practical value of open weights will depend on governance around deployment and downstream use, not just the permissiveness of the license.
The trend: Grok-1 is an early example of a broader pattern in which AI vendors use open-weight releases to widen ecosystem adoption while reserving their newest models for managed access.
Grok-1 314B parameters mixture of experts (MoE) - 8 experts top 2 selection. - The MoE implementation is different from Mixtral 8x7b. Mixtral 8x7b applies softmax over top2 experts, while Grok-1 applies top2 over the softmax of all 8 experts. - written in jax - training code is…
Chunky beast, needs 320 Gb VRAM likely 4 bit, likely is being run 8 bit on 8 x 80 Gb GPUs. As 25% of weights active any given time runs at 70b LLaMA 2 speeds. MoE models tail off larger so this is very interesting to test to see tradeoffs.
Yep, thanks to @elonmusk and xAI team for open-sourcing the base model for Grok. We will fine-tune it for conversational search and optimize the inference, and bring it up for all Pro users!
Grok-1 open-source- Why this is a huge deal Grok-1 has decent metrics, though it will need some fine-tuning to make it usable. We at Abacus AI will start working on it and should have an update/release in a few weeks. The model is too large for the open-source community to... [im…
Thank you @elonmusk, we will network this into @HyperspaceAI to continously build smarter, cheaper and more abundant AI. Together we can compete with the non-networked AIs. Wisdom of the crowd wins > those who fly solo..
from the RELEASE file in the torrent: xAI Grok-1 (314B) @xai @grok 314B parameter Mixture of Experts model - Base model (not finetuned) - 8 experts (2 active) - 86B active parameters - Apache 2.0 license - Code: https://github.com/... - Happy coding! [image]
here's your DEEP DIVE into @grok's architecture! I just went through the https://model.py/, for this 314B open source behemoth with *no strings attached*. 👇🧵 [image]
The largest ever open LLM, trained by a world class team, dropped by a magnet link. Apache 2.0. I wonder what it feels like to be out-opened by @grok . 314B, mixture of expert (2 out of 8 active). Even the active parameters only (86B) is more than the biggest Llama. Can't wa…
if one more dude emails me about Robert Heinlein I'm gonna snap. literally go on the xAI website and they talk about the inspiration for Grok coming from Hitchhikers. leave me alone!!
Musk's Grok AI was just released open source in a way that is more open than most other open models (it has open weights) but less than what is needed to reproduce it (there is no information on training data). Won't change much, there are stronger open source models out there. […
Grok weights are out. Download them quickly at https://huggingface.co/... huggingface-cli download xai-org/grok-1 —repo-type model —include ckpt/tensor* —local-dir checkpoints/ckpt-0 —local-dir-use-symlinks False Learn about mixture of experts at https://hf.co/...
Grok-1 is open sourced. Releasing Grok-1 increases LLMs' diffusion rate through society. Democratizing access helps us work through the technology's implications more quickly and increases our preparedness for more capable AI systems. Grok-1 doesn't pose severe bioweapon or cy…