Beating GPT-4o in seconds, beating Llama 3 70B in 22B, Mistral AI opens its first code model-AI-php.cn

Home

Technology peripherals

Beating GPT-4o in seconds, beating Llama 3 70B in 22B, Mistral AI opens its first code model

WBOYWBOYWBOYWBOYWBOYWBOYWBOYWBOYWBOYWBOYWBOYWBOYWB

Jun 01, 2024 pm 06:32 PM

Modeltrain

Mistral AI, the French AI unicorn benchmarking against OpenAI, has made a new move: Codestral, the first large code model, was born.

速度秒掉GPT-4o、22B击败Llama 3 70B，Mistral AI开放首个代码模型

As an open generative AI model designed specifically for code generation tasks, Codestral helps developers write and Interact with the code. Codestral's proficiency in coding and English allows software developers to design advanced AI applications.

Codestral’s parameter size is 22B and follows the new Mistral AI Non-Production License. It can be used for research and testing purposes, but commercial use is prohibited.

速度秒掉GPT-4o、22B击败Llama 3 70B，Mistral AI开放首个代码模型

Currently, the model is available for download on HuggingFace.

速度秒掉GPT-4o、22B击败Llama 3 70B，Mistral AI开放首个代码模型

##Download address: https://huggingface .co/mistralai/Codestral-22B-v0.1
Free trial address: https://t.co/LsgC84GCYw

Guillaume Lample, co-founder and chief scientist of Mistral AI, said that Codestral can be easily integrated into the VS Code plug-in.

速度秒掉GPT-4o、22B击败Llama 3 70B，Mistral AI开放首个代码模型

Some users compared Codestral with GPT-4o, and Codestral was directly faster than GPT-4o.

速度秒掉GPT-4o、22B击败Llama 3 70B，Mistral AI开放首个代码模型

Proficient in 80 Programming Languages

Codestral in a diverse dataset of 80 programming languages Online training, including Python, Java, C, C, JavaScript, Bash and other popular programming languages. It also performs well on programming languages such as Swift and Fortran.

Thus, a broad language base ensures that Codestral can help developers in a variety of coding environments and projects.

Codestral can competently write code, write tests and use the fill-in-the-middle mechanism to complete any code part, saving developers time and energy. Using Codestral at the same time can also help improve developers' coding skills and reduce the risk of errors and bugs.

New standard for code generation performance

As a 22B parameter model, Codestral has better code generation performance than previous large code models. and latency headroom set new standards.

As you can see from Figure 1 below, the context window length of Codestral is 32k, the competing product CodeLlama 70B is 4k, DeepSeek Coder 33B is 16k, and Llama 3 70B is 8k. Results show that Codestral outperforms other models on the code generation remote evaluation benchmark RepoBench.

速度秒掉GPT-4o、22B击败Llama 3 70B，Mistral AI开放首个代码模型

Mistral AI also compared Codestral to existing code-specific models, which require higher hardware requirements.

Performance on Python. The researchers used the HumanEval pass@1 and MBPP sanitised pass@1 benchmarks to evaluate Codestral's Python code generation capabilities; in addition, the researchers also used CruxEval and RepoBench EM benchmark evaluations.

Performance on SQL. To evaluate the performance of Codestral in SQL, the researchers used the Spider benchmark.

Performance on other programming languages. The researchers also evaluated Codestral in six other programming languages, including C, bash, Java, PHP, Typescript, and C#, and calculated the average of these evaluations.

速度秒掉GPT-4o、22B击败Llama 3 70B，Mistral AI开放首个代码模型

FIM Benchmark. The researchers also evaluated Codestral's ability to complete code when there are gaps in the code fragments, mainly conducting experiments on Python, JavaScript and Java. The results showed that users can run the code completed by Codestral immediately.

速度秒掉GPT-4o、22B击败Llama 3 70B，Mistral AI开放首个代码模型

Blog address: https://mistral.ai/news/codestral/

The above is the detailed content of Beating GPT-4o in seconds, beating Llama 3 70B in 22B, Mistral AI opens its first code model. For more information, please follow other related articles on the PHP Chinese website!

Statement

The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn

You Must Build Workplace AI Behind A Veil Of IgnoranceApr 29, 2025 am 11:15 AM

In John Rawls' seminal 1971 book The Theory of Justice, he proposed a thought experiment that we should take as the core of today's AI design and use decision-making: the veil of ignorance. This philosophy provides a simple tool for understanding equity and also provides a blueprint for leaders to use this understanding to design and implement AI equitably. Imagine that you are making rules for a new society. But there is a premise: you don’t know in advance what role you will play in this society. You may end up being rich or poor, healthy or disabled, belonging to a majority or marginal minority. Operating under this "veil of ignorance" prevents rule makers from making decisions that benefit themselves. On the contrary, people will be more motivated to formulate public

Decisions, Decisions… Next Steps For Practical Applied AIApr 29, 2025 am 11:14 AM

Numerous companies specialize in robotic process automation (RPA), offering bots to automate repetitive tasks—UiPath, Automation Anywhere, Blue Prism, and others. Meanwhile, process mining, orchestration, and intelligent document processing speciali

The Agents Are Coming – More On What We Will Do Next To AI PartnersApr 29, 2025 am 11:13 AM

The future of AI is moving beyond simple word prediction and conversational simulation; AI agents are emerging, capable of independent action and task completion. This shift is already evident in tools like Anthropic's Claude. AI Agents: Research a

Why Empathy Is More Important Than Control For Leaders In An AI-Driven FutureApr 29, 2025 am 11:12 AM

Rapid technological advancements necessitate a forward-looking perspective on the future of work. What happens when AI transcends mere productivity enhancement and begins shaping our societal structures? Topher McDougal's upcoming book, Gaia Wakes:

AI For Product Classification: Can Machines Master Tax Law?Apr 29, 2025 am 11:11 AM

Product classification, often involving complex codes like "HS 8471.30" from systems such as the Harmonized System (HS), is crucial for international trade and domestic sales. These codes ensure correct tax application, impacting every inv

Could Data Center Demand Spark A Climate Tech Rebound?Apr 29, 2025 am 11:10 AM

The future of energy consumption in data centers and climate technology investment This article explores the surge in energy consumption in AI-driven data centers and its impact on climate change, and analyzes innovative solutions and policy recommendations to address this challenge. Challenges of energy demand: Large and ultra-large-scale data centers consume huge power, comparable to the sum of hundreds of thousands of ordinary North American families, and emerging AI ultra-large-scale centers consume dozens of times more power than this. In the first eight months of 2024, Microsoft, Meta, Google and Amazon have invested approximately US$125 billion in the construction and operation of AI data centers (JP Morgan, 2024) (Table 1). Growing energy demand is both a challenge and an opportunity. According to Canary Media, the looming electricity

AI And Hollywood's Next Golden AgeApr 29, 2025 am 11:09 AM

Generative AI is revolutionizing film and television production. Luma's Ray 2 model, as well as Runway's Gen-4, OpenAI's Sora, Google's Veo and other new models, are improving the quality of generated videos at an unprecedented speed. These models can easily create complex special effects and realistic scenes, even short video clips and camera-perceived motion effects have been achieved. While the manipulation and consistency of these tools still need to be improved, the speed of progress is amazing. Generative video is becoming an independent medium. Some models are good at animation production, while others are good at live-action images. It is worth noting that Adobe's Firefly and Moonvalley's Ma

Is ChatGPT Slowly Becoming AI's Biggest Yes-Man?Apr 29, 2025 am 11:08 AM

ChatGPT user experience declines: is it a model degradation or user expectations? Recently, a large number of ChatGPT paid users have complained about their performance degradation, which has attracted widespread attention. Users reported slower responses to models, shorter answers, lack of help, and even more hallucinations. Some users expressed dissatisfaction on social media, pointing out that ChatGPT has become “too flattering” and tends to verify user views rather than provide critical feedback. This not only affects the user experience, but also brings actual losses to corporate customers, such as reduced productivity and waste of computing resources. Evidence of performance degradation Many users have reported significant degradation in ChatGPT performance, especially in older models such as GPT-4 (which will soon be discontinued from service at the end of this month). this

See all articles