Goose Factory has built an AI translation company: specializing in online novels, automatically adapting language styles, and both real people and GPT-4 can read it well

Goose Factory has set up a "translation company" with more than 150 people. From the boss to the employees are all AI agents!

The main business is the translation of online novels. The quality is extremely high. Readers who participated in the evaluation believe that is even better than real-person translation.

And compared to hiring real people to translate literary works, the cost is reduced by nearly 80 times.

ATransAgents company, each position is equipped with 30 different employees, who can adapt different translation styles according to language, genre and target audience.

Compared with traditional translation, the translation produced is more flexible and diverse, more in line with the expression habits of the target language, and is more literary.

So, although TransAgents "failed" in the automatic evaluation based on similarity, it has won strong recognition from readers and professionals.

Such a performance even makes people lament that perhaps the era is coming when humans will be more interested in content generated by artificial intelligence.

Some people still say that TransAgents is another evidence that they misunderstood artificial intelligence - they thought that due to model limitations, the translation of novels would be extremely difficult for AI, but as a result The AI ​​agent accomplishes this task very well.

So, are TransAgents that amazing?

Both real people and GPT-4 agree

In order to evaluate the translation quality of TransAgents, the author chose the WMT2023 data set, which needs to be translated at the chapter level.

This data set intercepts 20 consecutive chapters from 12 online novels, involving the following eight types:

• Games (Video Games, VG)
• Eastern Fantasy, EF
• Sci-fi Romance,SR
• Contemporary Romance,CR
• Fantasy (Fantasy, F)
• Science Fiction (SF)
• Horror & Thriller (HT)
• Fantasy Romance (Fantasy Romance, FR)

Initially, the author used d-BLEU for automatic evaluation. This method will perform similarity calculation with the reference sample, the higher the similarity. The score will be higher.

Specifically for this project, there are two sets of reference samples. Sample one is the translation result given manually, and sample two is obtained by aligning bilingual texts on the Internet.

As a result, TransAgents' score was not ideal, only 25 points, less than half of SOTA.

But this does not mean that the translation quality of TransAgents is not good, but because using similarity to measure the performance of translation of literary works is inherently biased.

Literary translation is not a word-for-word comparison, but requires creative transformations in semantics, tone, style, etc. These transformations may result in The translation and the reference translation are significantly different on the surface, it’s not surprising that the similarity is not high.

So, the author simply asked real people (at least 10 people) to evaluate the quality of the translation, and also asked GPT-4(0125-Preview) to take a look. In TransAgents, GPT -4（1106-Preview）Which one has the best translation between and the real person?

Evaluators will see different translations of the same original text. The real person only sees the translation, while GPT-4 sees both the original text and the translation.

As a result, more than half of the real-person reviewers believed that TransAgents translated better than humans, and 8.4% believed that the quality of the two was equivalent. GPT-4 also believed that TransAgents was of higher quality than itself (and humans).

In addition to these public reviews, two professional translators also believed that although manual translation is more faithful to the original text, the translation given by TransAgents is obviously more literary, It is more concise and reflects the language talent and profound literary expression in the choice of words and sentences.

MATTR and MTLD test indicators also prove this, especially on MTLD, the language diversity of TransAgents is one-third higher than that of real people and GPT-4 about.

In the case shown by the author, TransAgents will adjust the translation content according to the habits of the target language. Although the translations by real people (Ref1) and GPT-4 are correct, , but in comparison, it is not as consistent with language habits as TransAgents.

In addition, in terms of consistency, TransAgents also goes beyond simply using GPT-4 and keeps using the same original text. Same translation.

Of course, this does not mean that it is good at all genres. Among the 8 genres mentioned above, TransAgents has outstanding performance in games, science fiction romance and other genres, and In genres such as horror and thriller, it is relatively mediocre.

△The dotted line in the picture represents 50% Win rate

At the same time, the author also found that TransAgents did not "accept everything" when translating, but would There is a certain degree of omissions.

However, judging from the ratings given by readers who have not read the original text in the test, such omissions do not seem to affect their reading experience.

So, how does this "translation company" operate?

Multi-agent division of labor

In this company, different agents play the roles of CEO, junior/senior editor, true translator, localization expert and proofreader(Proofreader )Except for the CEO, there are 30 people in each of these different positions. Each person has different areas of expertise. There is also a Ghost Agent.

These agents are driven by GPT-4-Turbo. Each role includes multi-dimensional settings such as name, age, position, years of experience and languages ​​mastered.

After receiving the translation request from the "client", the CEO will comprehensively analyze the original text and target language, genre, target audience and other information, and select areas of expertise from several senior editors The best match.

At this time, Ghost Agent will evaluate the CEO's choice and inform him whether the candidate is suitable, thereby reducing the phenomenon of improper selection.

The selected senior editor will work with the CEO to further select team members such as junior editors, translators, localization experts, and proofreaders from the company's talent pool based on task requirements and personal characteristics.

After the team is formed, the junior editor first identifies all potential key terms chapter by chapter, generates an initial glossary, and submits it to the senior editor for review and deletes common terms. A revised glossary is generated and iterated until no further revisions are required.

Then, senior editors will translate the key terms in the glossary into the target language based on the meaning of the terms in different contexts.

After having a glossary, the junior editor will generate a detailed chapter summary for each chapter, retaining key information and details as much as possible, and then let the senior editor review it and remove redundant or unnecessary information , making chapter summaries more concise and to the point.

Then, the senior editor writes a summary of the entire book based on the revised chapter summary, summarizing the main plot, characters, and themes, and randomly selects a chapter from the book, analyzes its tone, style, and target audience, and develops a translation style guide.

The translation style guide will be sent to all members of the project team to ensure consistency in translation style.

According to the style guide, the company's real translators will perform preliminary translations chapter by chapter, and submit the first draft to the junior editor for review to check whether the translation style guide is followed and provide suggestions for improvement.

On this basis, the senior editor will evaluate the quality of the modified translation and decide whether further modifications are needed. The translator, junior editor and senior editor will iterate until the translation quality meets the requirements.

However, the translated text obtained at this time is not the final draft and needs to be handed over to localization experts for adjustments. Experts will identify content that may require cultural adaptation, such as idioms, metaphors, etc., and adjust these content to make it more relevant to the target language and culture while retaining the intent of the original text.

The adjusted text will be reviewed again by the junior editor and senior editor to ensure that the translation strikes a balance between cultural adaptability and fidelity.

After this, proofreaders will check the grammar, spelling, punctuation and formatting errors again. If there are any modifications, they will be further reviewed by the editor.

After these processes are completed, the senior editor will conduct a final review, focusing on the coherence between adjacent chapters to ensure that plot, characters, themes and other elements are consistent throughout the book. Once problems are discovered Then it is sent back to the previous team members for modification until the final translation is formed.

If the client has any suggestions for modifications to the translation, the senior editor will again take the lead and organize relevant personnel to make modifications until the final version is finalized.

Not only is the work process extremely strict, but after the client is satisfied with the translation, the senior editor will also organize a "meeting" for the project team to summarize the project and share experiences and lessons learned.

The valuable terminology, translation skills, cultural adaptation strategies and other knowledge accumulated during the process will be summarized and uploaded to the company's knowledge base for reference in subsequent projects.

One More Thing

TransAgents is not the first "company" composed of intelligent agents. Last year, ChatDev, a "game company" from Tsinghua University, became popular on the Internet. The technology used behind it The core technology is multi-agent.

If we look wider, we don’t just look at the “company”, but also Stanford’s AI Town and Tsinghua University’s AI Werewolf Game, all of which are using multi-agent agents to simulate real human society.

In short, with the deepening of large model research, agent and swarm intelligence experiments have become one of the most popular directions in AI research, and judging from this TransAgents, multi-agent collaboration has begun to show practical results. benefit.

(By the way, some netizens discovered that from Stanford Town to ChatDev, and then to TransAgents this time, the author of multi-agent research really likes the "Stardew Valley" style Painting style. )

## Of course, some people have expressed concerns about this, believing that AI-led translation will lead to the homogenization of languages ​​and make various Unique expressions in the language disappear.

What's more, I have jumped out of TransAgents itself and thought of Ilya's fear of large-scale Agent cooperation...

So what novel results do you think can be created on the road of swarm intelligence? Feel free to share your thoughts in the comment area.

Paper address: https://arxiv.org/abs/2405.11804

The above is the detailed content of Goose Factory has built an AI translation company: specializing in online novels, automatically adapting language styles, and both real people and GPT-4 can read it well. For more information, please follow other related articles on the PHP Chinese website!