


Researchers find much of the code generated by ChatGPT is insecure, but it won't tell you
News on April 23, the ChatGPT chatbot can generate a variety of text, including code, based on user input. However, four researchers from the University of Quebec in Canada found that the code generated by ChatGPT often has serious security problems, and it does not proactively alert users of these problems, and will only admit its mistakes when users ask.
The researchers introduced their findings in a paper. IT House reviewed the paper and found that they had ChatGPT generate 21 programs and scripts involving languages such as C, C, Python and Java. These programs and scripts are designed to demonstrate specific security vulnerabilities, such as memory corruption, denial of service, deserialization, and encryption implementation vulnerabilities. The results showed that only 5 out of 21 programs generated by ChatGPT were safe on the first try. After further prompting to correct its erroneous steps, the large language model managed to generate 7 more secure applications, although this was only "safe" with respect to the specific vulnerability being evaluated, not to say that the final code didn't have anything else that could be done. Exploited vulnerabilities.
Researchers pointed out that part of ChatGPT’s problem is that it does not take into account the adversarial code execution model. It will repeatedly tell users that security issues can be avoided by "not entering invalid data," but this is not feasible in the real world. However, it appears to be aware of and admit to critical vulnerabilities in its proposed code.
Raphaël Khoury, a professor of computer science and engineering at the University of Quebec and one of the paper's co-authors, told The Register: "Obviously, it's just an algorithm. It doesn't know anything, but it can identify insecure behavior." He said that initially ChatGPT's response to the security issue was to recommend only using valid input, which was clearly unreasonable. It only provides useful guidance when later asked to improve the problem.
Researchers believe that this behavior of ChatGPT is not ideal because users knowing what questions to ask require some knowledge of specific vulnerabilities and coding techniques.
The researchers also pointed out that there are ethical inconsistencies in ChatGPT. It will deny the creation of attacking code, but will create vulnerable code. They gave an example of a Java deserialization vulnerability, "The chatbot generated vulnerable code and provided suggestions on how to make it more secure, but said it could not create a more secure version of the code."
Khoury believes that ChatGPT is a risk in its current form, but that’s not to say there aren’t sensible ways to use this unstable, underperforming AI assistant. "We've seen students using this tool, and programmers are using this tool in real life," he said. "So having a tool that generates unsafe code is very dangerous. We need to make students aware that if the code is Generated with this type of tool, then it's probably unsafe." He also said that he was surprised that when they asked ChatGPT to generate code for the same task in different languages, sometimes for one language, It will generate secure code, and for another language, it will generate vulnerable code. "Because this language model is kind of like a black box, I don't really have a good explanation or theory for this. ”
The above is the detailed content of Researchers find much of the code generated by ChatGPT is insecure, but it won't tell you. For more information, please follow other related articles on the PHP Chinese website!

Running large language models at home with ease: LM Studio User Guide In recent years, advances in software and hardware have made it possible to run large language models (LLMs) on personal computers. LM Studio is an excellent tool to make this process easy and convenient. This article will dive into how to run LLM locally using LM Studio, covering key steps, potential challenges, and the benefits of having LLM locally. Whether you are a tech enthusiast or are curious about the latest AI technologies, this guide will provide valuable insights and practical tips. Let's get started! Overview Understand the basic requirements for running LLM locally. Set up LM Studi on your computer

Guy Peri is McCormick’s Chief Information and Digital Officer. Though only seven months into his role, Peri is rapidly advancing a comprehensive transformation of the company’s digital capabilities. His career-long focus on data and analytics informs

Introduction Artificial intelligence (AI) is evolving to understand not just words, but also emotions, responding with a human touch. This sophisticated interaction is crucial in the rapidly advancing field of AI and natural language processing. Th

Introduction In today's data-centric world, leveraging advanced AI technologies is crucial for businesses seeking a competitive edge and enhanced efficiency. A range of powerful tools empowers data scientists, analysts, and developers to build, depl

This week's AI landscape exploded with groundbreaking releases from industry giants like OpenAI, Mistral AI, NVIDIA, DeepSeek, and Hugging Face. These new models promise increased power, affordability, and accessibility, fueled by advancements in tr

But the company’s Android app, which offers not only search capabilities but also acts as an AI assistant, is riddled with a host of security issues that could expose its users to data theft, account takeovers and impersonation attacks from malicious

You can look at what’s happening in conferences and at trade shows. You can ask engineers what they’re doing, or consult with a CEO. Everywhere you look, things are changing at breakneck speed. Engineers, and Non-Engineers What’s the difference be

Simulate Rocket Launches with RocketPy: A Comprehensive Guide This article guides you through simulating high-power rocket launches using RocketPy, a powerful Python library. We'll cover everything from defining rocket components to analyzing simula


Hot AI Tools

Undresser.AI Undress
AI-powered app for creating realistic nude photos

AI Clothes Remover
Online AI tool for removing clothes from photos.

Undress AI Tool
Undress images for free

Clothoff.io
AI clothes remover

AI Hentai Generator
Generate AI Hentai for free.

Hot Article

Hot Tools

SecLists
SecLists is the ultimate security tester's companion. It is a collection of various types of lists that are frequently used during security assessments, all in one place. SecLists helps make security testing more efficient and productive by conveniently providing all the lists a security tester might need. List types include usernames, passwords, URLs, fuzzing payloads, sensitive data patterns, web shells, and more. The tester can simply pull this repository onto a new test machine and he will have access to every type of list he needs.

WebStorm Mac version
Useful JavaScript development tools

ZendStudio 13.5.1 Mac
Powerful PHP integrated development environment

Safe Exam Browser
Safe Exam Browser is a secure browser environment for taking online exams securely. This software turns any computer into a secure workstation. It controls access to any utility and prevents students from using unauthorized resources.

MinGW - Minimalist GNU for Windows
This project is in the process of being migrated to osdn.net/projects/mingw, you can continue to follow us there. MinGW: A native Windows port of the GNU Compiler Collection (GCC), freely distributable import libraries and header files for building native Windows applications; includes extensions to the MSVC runtime to support C99 functionality. All MinGW software can run on 64-bit Windows platforms.