How AI Agent thinks about enterprise-level design

After the explosion of large-scale models this year, the current artificial intelligence technology is still in the second-generation system stage and has not yet reached the level of artificial general intelligence (AGI). However, in some fields, AI Agent can already be used for processing. As a typical B-end application, how to design an AI Agent that meets business needs?

With the rapid development of artificial intelligence, various advanced large-scale models, products and tools are emerging one after another. As a B-end product manager, we need to actively embrace this transformative artificial intelligence technology and apply it to our own business to ensure that the efficiency of the enterprise does not lag behind the development of the industry

Generally speaking, if an enterprise does not independently develop large-scale models, it needs to rely on third-party large-scale model capabilities to develop AI capabilities to build AI products or tools suitable for the enterprise itself.

As shown in the figure below, common large-scale model directions include natural language processing (NLP), multimodal (multimodal) and speech recognition. Large model manufacturers conduct AI training based on capabilities in specific directions, allowing AI to replace humans in "repeatable" and "standardizable" tasks in certain fields.

The picture is quoted from "Battle of 100 Models!" of Open Privacy Computing! Which large AI model do you prefer? 》

In order to determine the AI ​​capabilities required for our business, we need to conduct research on the corresponding AI models and evaluate the output capabilities of large models. We can evaluate large models from multiple dimensions such as recall, accuracy, security, interpretability, stability, cost, and development potential to ultimately select a large model that suits our needs. On this topic, I have detailed how to choose a large model that is right for you in a previous article.

However, the current AI technology is still in the second-generation system stage and has not yet reached the AGI level. There are still some problems with the currently accessible large-scale models, mainly including the following aspects:

1. AI Illusion:

AI illusion refers to the limitations of artificial intelligence in knowledge memory, understanding ability, training methods and model technology, which leads to inaccurate or unreliable performance in output results. Common problems include data bias and poor interpretability

Due to the existence of AI illusion, even if we expect AI to be able to stably output reliable solutions, there will still be a certain proportion of wrong answers. For example, if the AI ​​is 50% accurate in a certain field, there will be 25 wrong answers out of 50. For businesses that require high accuracy, AI cannot be applied directly.

2. Answers to Compliance Questions

The essence of AI is prediction of results based on statistics and lacks clear ability to judge right and wrong. Therefore, when it comes to ethical, legal and other issues, AI cannot make accurate judgments or distinctions, which may have a negative impact on the enterprise

3. Not native enough:

Currently, the interaction method of artificial intelligence is mainly through input-output. Users input content and artificial intelligence outputs results. However, this process does not suit the usage habits of all business personnel

Taking the translation scenario as an example, the business process of a translator usually includes the following steps:

1. Determine translation needs: Confirm the source language, target language, translation style, cultural differences in different regions and adjustment plans, etc.
2. Batch translation: Translation tasks usually involve multiple pieces of content, and translators need to complete the processing in batches at the same time.
3. Proofreading: After the translation is completed, it needs to be proofread by a proofreader to determine whether it meets business needs.
4. Modification: If the translation result does not meet the requirements, it needs to be modified. After the revision is completed, continue the process of translation, proofreading and revision until the revision is approved.
5. Delivery: Apply the proofread content to the business.

However, if you want to use AI for translation to save labor costs, directly using the dialogue input interaction method of a large language model will bring the following problems:

1) Interrupt the existing workflow and make it difficult to develop usage habits:

Using large language model dialogue windows will interrupt the business process of the original translation work. Originally, you only needed to complete the work on the translation tool or document. After joining the large-scale model dialogue, each translated text requires input interaction on the input box.

2) The upper limit of operating cost increase:

Large language models have limitations on conversation length. If the translation content is large, interactions need to be conducted in batches, which will increase labor costs.

3) Increase in operating costs to meet special translation needs:

If there are specific translation needs, such as term translation or specified translation style, interaction is required every time, which further takes up manpower.

The above problems prevent AI from effectively improving business translation efficiency. Due to the cumbersome operation, it is difficult for users to form usage habits. They often subconsciously think that it is better to translate directly by themselves than to use AI, so the replacement cost is high.

In addition, AI translation has an illusion problem and cannot provide a user experience beyond expectations.

Thus, [(New Experience-Old Experience)-Replacement Cost] is not greater than 0. Directly using AI’s native interaction method cannot effectively improve the business rate, so some more native methods are needed. The way.

1. What is AI Agent

In view of the current challenges faced by artificial intelligence, we need to think about how to build our own applications on the B side. One possible way is to try to build an AI Agent

customized for the business

The so-called AI Agent, also known as artificial intelligence agent, refers to an automated program that can understand, learn and perform tasks. It can be compared to "treating AI as an intern and letting it take on trivial tasks, while we are responsible for guiding this intern to ensure that its output meets the expected results."

Unlike large models, AI Agent's interaction methods are not limited to prompting with people. It is a system that is based on specific work goals and outputs results that meet the needs. The core of AI Agent is a large model, which also includes extensions of perception module, planning module and action module

1. Perception module: The perception module assembles data into prompt words for input through connections with business data sources and external data sources.
2. Action module: The action module can perform corresponding business operations based on the results of perception and planning through connection with the business capability interface. In addition, the feedback data in business practice can also be optimized through the feedback system with codes, strategies and prompt words.
3. Planning module: The planning module is the core of AI Agent. In addition to leveraging the ability of large models, it is also necessary to design based on business needs combined with code logic. Here we need to conceive of how the brain works and adopt appropriate input and output methods to drive business.

2. How to design AI Agent

So how to design an AI Agent that meets business needs?

1. Find the right business scenario

First, we need to determine the scenario suitable for the AI ​​Agent, and evaluate the suitability of the selected scenario by simulating data input and collecting output results. When judging the output results, it is necessary to examine whether they meet expectations. If the results are not as expected, you need to evaluate the severity of the error and whether there is room for improvement in precision and recall. If there is room for improvement, AI can be made to meet our scene needs by optimizing prompt words, or by introducing the construction of perception, action, and planning modules.

2. Sort out input and output expectations

We need to clarify the 'goals' and 'requirements' for the AI ​​Agent to determine what we input and what the Agent needs to output.

Based on these 'goals' and 'requirements', we should think about how to choose the appropriate input and output methods to meet business needs and empower the business. This will help us design the subsequent output process and evaluate it during the design verification phase to determine whether it meets the requirements.

3. Sort out the output process

When we face complex output requirements, we need to design multiple AI conversation processes so that the various AIs can collaborate with each other to ultimately achieve output results that meet the requirements. For example, in text classification scenarios, we can first use the large model of version 3.5 for accurate first-level classification to meet the needs.

Since the cost of version 4.0 is dozens of times that of version 3.5, and version 3.5 is sufficient for first-level classification, you can choose to use version 3.5 to save costs. Then, we can use version 4.0 for secondary classification to obtain better classification results and ensure that we output the required content.

When designing the process, the following factors need to be considered:

• Cost factors: The costs of different large models and their versions are different. We need to weigh the business benefits and choose a reasonable large model usage plan to avoid losing more than the gain.
• Effective factors: The effects of different large models and their versions are different, and they have their strengths in different fields. We need to choose the most suitable solution based on the usage requirements.

After comprehensively considering the factors of cost and effect, we finally determined a suitable plan

4. Output verification mechanism

In order to avoid misleading content in AI output results from affecting the business, we need to establish an effective verification mechanism. Common verification methods include lexicon matching, regular expression matching, and manual inspection. Block misleading content by using thesaurus or manual means. In addition, we can also build a quality inspection agent to let the AI ​​itself perform quality inspection on the output to filter out problematic content and improve the detection rate of misleading content.

5. Illusion cover-up plan

The occurrence of hallucinations is a phenomenon that cannot be completely eradicated. In order to avoid adverse effects on the business, we need to formulate a safety plan, such as:

1) Manual inspection: Before the AI ​​output is passed to the user, a manual inspection link is introduced. Only after the manual inspection passes, the results are output to the user. In this way, we can perfectly prevent AI illusion from having a negative impact on business, and we can also use the output of AI to improve efficiency. However, this method requires human review, so it will increase certain labor costs.

2) Reasonable packaging: Considering that we are an AI application for the B-side, we can use methods such as packaging the output application as "AI assistant" to directly make it clear to users: "The output results here are generated by AI and are for reference only. ". In this way, we can allow users to form reasonable psychological expectations and avoid adverse reactions when hallucinatory output occurs.

3. Summary

Based on the above ideas, we can build B-side translation Agent, data classification Agent, intelligent customer service Agent and other services. Of course, these are just some of my personal thoughts. Everyone is welcome to exchange and discuss.

Please vote for me

I am participating in the 2023 Everyone is a Product Manager selection. I hope that friends who like my articles can come and support me~

Click the link below to enter my personal election page, and click on the red heart to vote for me.

Each person can vote up to 30 votes per day. By voting, you will get a chance to win a lottery, and you will receive gifts such as books, Everyone is a Product Manager commemorative peripherals, and membership in the Starting Point Classroom!

Please click the following link to vote: https://996.pm/7d9yE

Columnist

Lemon cake is clean and hygienic. Public account: Lemon cake is clean and hygienic. Everyone is a product manager columnist. A B-end product in the game industry. He has been responsible for CRM, risk control, BI, SDK, and AI-related content in the game industry. He regularly outputs personal thoughts or summary articles~

This article was originally published on Everyone is a Product Manager. Reprinting without permission is prohibited

The title picture comes from Unsplash, based on the CC0 protocol

The above is the detailed content of How AI Agent thinks about enterprise-level design. For more information, please follow other related articles on the PHP Chinese website!