Is it an achievement or a weakening? The dispute between AI code generation tools and programmers

Are programmers destined to be replaced? The computer programming community has been gripped by this question ever since OpenAI's large-scale language model GPT-3 demonstrated its stunning ability to create HTML websites based on simple written instructions.

In the months since, it has emerged that simple but complete computer programs can be written based on natural language descriptions (spoken or written human language) and the programmer's work process can be accelerated. Automatic programming assistant. To what extent will AI replace or enhance the work of programmers?

IEEE Spectrum asked some experts about this, and the bad news is that programming may be destined to be replaced. But the good news is that it looks like computer programming and software development will remain a highly human-involved job for the foreseeable future. At the same time, AI-powered automatic code generation is increasingly accelerating software development by allowing more code to be written in less time.

Vasi Philomin, Vice President of Amazon AI Services, said, “I do not believe that AI will completely replace human developers. AI tools can free programmers from daily tasks, but computer programming Creative jobs are here to stay."

If someone wants to be a developer in 10 years, they don't necessarily need to learn a programming language. Instead, they need to understand the semantics, concepts, and logical sequences of creating computer programs, which will open software development to a wider population.

Computer programming is becoming more intelligent and automated

When electronic computer programming began in the 1940s, programmers wrote using digital machine code. It was not until the mid-1950s that Grace Hopper, a distinguished American female computer scientist, and her team at the Remington Rand Company developed FLOW-MATIC, which allowed programmers to write programs using a limited English vocabulary.

Since then, programming languages ​​have become more and more efficient, and programmers have become more efficient in their work.

AI-written code is at the forefront of a broader change that allows people to write software without coding at all. There are now no-code AI development platforms like Akkio that allow people to build machine learning models with simple drag-and-drop and click-of-button functionality. Microsoft's Power Platform includes a series of low-code products that allow users to generate simple applications simply by describing them.

In June this year, Amazon launched a preview version of CodeWhisperer, a machine learning-based programming assistant similar to Copilot for GitHub. Both tools are based on large language models (LLMs), which have been trained on massive code bases. As programmers write code, both CodeWhisperer and Copilot provide autocompletion suggestions and suggest executable instructions based on simple natural language phrases.

## A few days ago, GitHub conducted a survey among 2,000 developers and found that Copilot Task time is cut in half and developers’ overall satisfaction with their jobs increases.

Beyond code auto-completion is not easy

But if you want to transcend auto-completion, the problem lies in teaching intent to the computer. Software requirements are often vague, and natural language is notoriously inaccurate.

British artificial intelligence startup Diffblue is committed to using AI to check and correct code. Peter Schrammel, one of its co-founders, said that it wants to solve the ambiguities in English writing standards. Then some incremental improvements are needed, some dialogue between humans and machines.

To address these issues, Microsoft researchers recently proposed adding a feedback mechanism to LLM-based code generation so that the computer would ask the programmer to clarify any ambiguities before generating code. Microsoft's interactive system, called TiCoder, refines and formalizes user intent by generating what it calls "test-driven user intent formalization" (TDUIF).

TDUIF attempts to use iterative feedback to determine the programmer's algorithmic intent and then generate code that is consistent with the programmer's expressed intent. The figure below shows the workflow of TDUIF.

##Paper address: https://arxiv.org/abs/2208.05950

According to related papers, TiCoder improved the accuracy of automatically generated code from 48% to 85% when evaluated on the Mostly Basic Programming Problems (MBPP) benchmark. Designed to evaluate machine-generated code, MBPP consists of approximately 1,000 crowdsourced Python programming problems to be solved by entry-level programmers.

A unit of code can be hundreds of lines long and is the smallest part of a program that can be independently maintained and executed. A suite of unit tests typically consists of dozens of unit tests, each containing 10 to 20 lines of code, that check that the unit performs as expected, so that when you stack the units on top of each other, the program works as expected.

Unit tests are useful for debugging individual functions and detecting errors when manually changing code. Additionally, a unit test serves as a specification for a unit of code and serves to guide programmers in writing clean, bug-free code. Although not many programmers pursue true test-driven development, where unit tests are written first, unit tests are often written together with units.

Experts believe: Human programmers will not lose their jobs due to AI code generation

According to the results of the 2019 Diffblue developer survey, developers spend approximately 35% of their time using to write quality control tests instead of writing code for production use, so automating this part of the work can significantly improve productivity.

At the same time, Copilot, CodeWhisperer and other AI programming assistant packages can be used as interactive auto-completion tools for writing unit tests. Programmers get coding suggestions and choose the ones that work best.

In February this year, DeepMind further used AlphaCode for fully automated code production. AlphaCode is a large-scale language model that can write simple computer programs based on natural language instructions. It uses an encoder-decoder Transformer architecture that first encodes a natural language description of the problem and then decodes the resulting vector into the code required for the solution.

AlphaCode Overview. Image source: https://arxiv.org/pdf/2203.07814.pdf

The model is first trained on the GitHub code repository until it can generate views Code that looks reasonable. To fine-tune the model, DeepMind used 15,000 pairs of natural language problem descriptions and successful code solutions from previous programming competitions to create a dedicated dataset of input-output examples.

Once AlphaCode has been trained and fine-tuned, it can be tested on problems it has not seen before. The final step is to generate a large number of solutions and then use a filtering algorithm to select the best one. Oriol Vinyals, head of the deep learning team at DeepMind, believes that they created many different programs by sampling the language model nearly 1 million times.

To optimize the sample selection process, DeepMind uses a clustering algorithm to group solutions. The clustering process tends to group together workable solutions, making it easier to find a small subset of candidates that are likely to work as programmers wrote them.

To test the system, DeepMind submitted 10 programs written by AlphaCode to a human programming competition on the popular Codeforces platform, with solutions placing in the top 54%.

In a recent interview, Oriol Vinyals asked rhetorically, “In order to generate a program, do you just write it in natural language without coding, and then the solution will be in What about coming out the other side?" Vinyals and others are cautious, saying it will take time, possibly decades, to get there.

Landing AI founder and CEO Andrew Ng, a well-known AI scholar, said that we are still far away from a person being able to tell a computer the requirements to write an arbitrarily complex computer program and automatically complete the coding.

But given how quickly AI code generation has advanced in just a few years, it seems inevitable that AI systems will eventually be able to write code based on natural language instructions. And hand-programming software programs will become more and more like hand-knitting sweaters.

In order to provide natural language instructions to computers, developers still need to understand some logical and functional concepts and how to structure things. Even if a developer doesn't learn a specific programming language or write computer code, they still need to learn the basics of programming. In turn, this will allow more programmers to create increasingly diverse software.

Vasi Philomin says he doesn’t believe AI will replace human developers. AI can eliminate the menial, routine work that developers have to do, allowing them to focus on higher value things.

Peter Schrammel also agrees that AI automatic code generation allows software developers to focus on more difficult and challenging tasks. However, he added, it is necessary to at least confirm whether what the machine understands is human intention.

He also said that software developers will not lose their jobs because automated coding tools replace them, and there will always be more software to be written.

The above is the detailed content of Is it an achievement or a weakening? The dispute between AI code generation tools and programmers. For more information, please follow other related articles on the PHP Chinese website!