Xiao Zha officially announced the heavyweight update of Code Llama, adding a 70B version, but there are still capacity restrictions

The more powerful Code Llama 70B model is here.

Today, Meta officially launches the Code Llama 70B, the largest and best-performing model ever in the Code Llama series.

We are open sourcing an improved version of Code Llama that includes a larger 70B parameter model. Writing and editing code has become one of the most important uses of modern artificial intelligence models. It turns out that for AI models, the ability to write code is key to processing information in other fields more rigorously and rationally. I'm very proud of this development and look forward to incorporating it into Llama 3 and future models.

Code Llama 70B is available in the same three versions as previously released Code Llama models:

• CodeLlama - 70B, basic code model;
• CodeLlama - 70B - Python, 70B specifically for Python;
• Code Llama - 70B - Instruct 70B, which is fine-tuned for understanding natural language instructions.

Code Llama was originally a tool launched by Meta in August 2023 and is free for research and commercial use.

According to a Meta AI blog post, Code Llama 70B can handle more queries than other versions, which means developers can provide it with more hints when programming. And can be more accurate.

In the HumanEval benchmark test, the accuracy of Code Llama 70B reached 53%, which is better than GPT-3.5’s 48.1%, and is further close to GPT-4 in the OpenAI paper report 67%.

Code Llama is built on Llama 2 and helps developers create code from prompts and debug human-written code. At the beginning of the release, Meta stated that Code Llama's open source license is the same as Llama 2 and is free for research and commercial purposes, as is Code Llama 70B.

At that time, there were three versions of the Code Llama series models, with parameter amounts of 7B, 13B and 34B respectively. And supports multiple programming languages, including Python, C, Java, PHP, Typescript (Javascript), C#, and Bash.

The Code Llama versions provided by Meta include:

• ##Code Llama, basic code model ;
• Code Llama-Python, a fine-tuned version of Python;
• Code Llama-Instruct, a fine-tuned version of natural language instructions.

#Among them, the latter two models focus on specific coding languages. Now Code Llama 70B has been added to this model family, taking coding tasks to another level.

Code Llama 70B is trained on 1TB of code and code-related data, hosted on the code repository Hugging Face.

Meta says its larger models, the 34B and 70B models, return the best results and provide better coding assistance.

But in the developer’s experience, Code Llama 70B still has obvious shortcomings.

For example, someone found that Code Llama 70B could not complete the request to generate prime numbers:

Some people say that this restriction is to avoid some bad uses, such as generating illegal prime numbers. Illegal primes can be used to represent material that is prohibited from possession or distribution, and distribution of such programs is illegal under the U.S. Digital Millennium Copyright Act.

^{Reference link: https://www.theverge.com/2024/1/29/24055011/meta-llama2-code-generator-generative-ai}

Related reading

• One day after its release, Code Llama’s coding capabilities improved by leaps and bounds, and the fine-tuned version of HumanEval scored over GPT-4
• Official Code Llama open source: free for commercial use, the mysterious version is close to GPT-4

The above is the detailed content of Xiao Zha officially announced the heavyweight update of Code Llama, adding a 70B version, but there are still capacity restrictions. For more information, please follow other related articles on the PHP Chinese website!