Meta debuts Code Llama, which can generate code and debug human-written work, under the same community license as Llama 2, free for research and commercial use
Code Llama, Code Llama - Python, and Code Llama - Instruct, fine-tuned for understanding natural language instructions. … Yann LeCun / @yannlecun : You knew that was coming: Code LLama !!! - Llama-2 tuned for code generation, debugging, etc - base models, Python-specific, and instruction-tuned. - 7B, 13B, 33B models - same license as Llama-2 - Blog: https://ai.meta.com/... - Paper: https://ai.meta.com/... - Code: https://github.com/... - Models: https://ai.meta.com/... X: Georgi Gerganov / @ggerganov : Here are some inference numbers for Code Llama on M2 Ultra at different quantum levels using latest llama.cpp pp - prompt processing tg - text generation Code Llama 7B [image] Nat Friedman / @natfriedman : Local AI will drive multiple hardware cycles for Apple Sebastian Raschka / @rasbt : Llama 2 is awesome, however, coding tasks were not its strong suite. (See HumanEval, a coding-related evaluation task from the paper Evaluating Large Language Models Trained on Code.) The new 34B CodeLlama model is twice as good as the original 70B Llama 2 model and closes the gap to (the much larger) GPT 4. Excited to read the 47-page CodeLlama paper (via https://ai.meta.com/research/ publications/code-llama-open-foundation - models-for-code/) in more detail in the next couple of days. Sridhar / @ramaswmysridhar : Excited to be kicking the tires on the Code Llama model! This is likely to be a game changer for those of us working in the LLM world! Thanks, @MetaAI ! Joelle Pineau / @jpineau1 : Exciting day today, as we're launching Code Llama, an extension of Llama 2 specialized for code generation, and natural language related to code. As per our commitment to openness, it comes with paper, code, model weights, responsible use guide, and the same license as Llama 2! Andrej Karpathy / @karpathy : Looks very nice on initial skim! But about this “Unnatural Code Llama”... Baptiste Rozière / @b_roziere : We initialize CodeLlama with Llama 2 and train on 500B tokens of a code-heavy dataset. At mid-training, it is already equivalent to a full training from scratch. The perplexities of our models still go down at the end of training, so extra budget could be used to train longer. [image] Gabriel Synnaeve / @syhw : Happy to be releasing Code Llama! We've built it on Llama 2 and improved it for code use cases. In particular it supports infilling out of the box, and was trained with sequences up to 16k tokens. Looking forward to what the community will build with it! 1/7 Baptiste Rozière / @b_roziere : It allows CodeLlama to outperform other open base models on programming benchmarks, even those trained on more tokens. We also obtain significant performance gains compared to Llama 2. Even CodeLlama 7B outperforms Llama 2 70B on a multilingual benchmark. [image] Gabriel Synnaeve / @syhw : Here are some fun things we did with it: > I have a pandas dataframe with the columns “decoding”, “Capabilities”, “Fine-tuning”, “Model size”, “HE pass@1”, “MBPP pass@1”. I want a seaborn figure with two scatterplots side-by-side. [...] [image] Gabriel Synnaeve / @syhw : We're releasing base models, Python-specialized models, and Instruct(ion following) models, all in sizes 7B, 13B, 34B params. Get the code and weights: https://github.com/... Read the research paper: https://ai.meta.com/... Read the blog post: https://ai.meta.com/... [image] Tobi Lutke / @tobi : Fantastic model Soumith Chintala / @soumithchintala : CodeLlama — a version of Llama2 that was fine-tuned for code tasks is live now. Available in 7B, 13B and 34B. https://ai.meta.com/... [image] Omar Sanseviero / @osanseviero : Welcome...Code Llama! Official open-access @MetaAI models for code!🔥Three different models: - Code Llama: Code model - Python specialist - Instruct - natural language instructions 7B, 13B, and 34B parameters versions. Same license as Llama 2! https://ai.meta.com/... [image] Anton / @abacaj : Meta does it again, releasing code llama and in a 34B variant (good size for 4bit performance). Surpassing publicly available state of the art code models [image] @garybasin : They don't want you to know that synthetic data is the future. LLMs generating synthetic data to train on drives a huuuge boost in “unnatural” code llama — the one model they aren't releasing. Surpasses gpt-3.5 and gets close to gpt-4 performance on a 34B model [image] Daniel Gross / @danielgross : Meta released CodeLlama today, except the most powerful variant (Unnatural Code Llama), which it is not releasing. [image] Aman Sanger / @amanrsanger : Tough... The best code-llama model drastically underperforms gpt-4 prompted (87%) and gpt-3.5-turbo prompted (75%) on humaneval OSS still has some ways to go on code [image] @metaai : Today we're releasing Code Llama, a large language model built on top of Llama 2, fine-tuned for coding & state-of-the-art for publicly available coding tools. Keeping with our open approach, Code Llama is publicly-available now for both research & commercial use. More ⬇️ LinkedIn: Joelle Pineau : Exciting day today, as we're launching Code Llama, an extension of Llama 2 specialized for code generation, and natural language related to code. … Justin Gerrard : Meta has released a tool called Code Llama, built on top of its Llama 2 large language model, to generate new code and debug human-written work, the company said. … Amit Sangani : Super excited to see Code Llama, a SOTA large language model (LLM) for coding is released and available under a commercially permissive license! … Will Knight : This morning, Meta released Code Llama, a version of its “open source” large language model fine-tuned for writing code. … Adnan Boz : 🚀Exciting News for Product Managers!: Meta just introduced Code Llama! 🚀 Today marks a significant milestone in the world of AI and coding. … Sahar Mor : Breaking: Meta releases Code Llama - a Llama 2 state-of-the-art coding LLM. — The best part? It's free for commercial use ⭐️ A few details: … Jia Li : Meta launched Code Llama for coding productivity. Again research and commercially available. Model sizes are 7B, 13B and 34B parameters. https://lnkd.in/... Sam Burgoyne : Now we have Code Llama for Python! We're cracking this open to see what it can do. If this performs at the level of GPT-4, it's a real leap forward. … Forums: Hacker News : Meta launches own AI code-writing tool: Code Llama r/Python : Introducing Code Llama, a state-of-the-art large language model for coding
Context & Ripple Effects
Code Llama extends the Llama 2 line into a specialized coding model family, offering general, Python-focused, and instruction-tuned variants at several sizes. The commercial-use community license makes the release relevant not just to researchers but to teams evaluating models for development workflows.
The launch became a base for subsequent specialization: Meta later released a 70B Code Llama update focused on code correctness and built LLM Compiler models for code optimization on the Code Llama family. That progression matters because it turns a general model release into a reusable technical layer for distinct software-engineering tasks.
First-order effects
- Developers and organizations can obtain Code Llama weights and tailor deployments around code generation and debugging, with a Python-specific option and an instruction-following variant.
- Meta broadens Llama 2’s addressable use cases from general language tasks to software development, where the reported 34B model benchmark performance gives the family a more concrete capability claim.
Second-order effects
- Coding-assistant providers and enterprise engineering teams gain a model family they can test against proprietary or hosted alternatives, increasing pressure to differentiate on reliability, tooling, deployment, and cost rather than basic code completion alone.
- The separate Python and instruction variants establish a product pattern that later supports more targeted models, including Code Llama-derived optimization models, rather than relying on one general-purpose system.
Third-order effects
- If specialized, commercially usable model families continue to improve, software-development AI is likely to fragment into task-specific layers—generation, debugging, and optimization—with evaluation and integration becoming key buyer considerations.
- The later 70B Code Llama release suggests that model access is paired with an ongoing capability race; the durable competitive question is whether open-weight ecosystems can sustain performance and deployment advantages across specialized workloads.
The trend: Code Llama is an early instance of general LLM platforms being converted into commercially usable, domain-specific model families for engineering work.