Alibaba’s intelligent “assembly factory” is open source! 0 experience to handle tens of thousands of agents concurrently

Make multi-agent development like building blocks, Alibaba Tongyi Lab’s open source multi-agent programming framework and development platformAgentScope.

This platform is specially built for multi-agent application developers, aiming to provide an easy-to-use programming experience, stable and reliable runtime guarantee, and provides developers with distributed and multi-modal technical support. .

It has built-in model APIs for various platforms such as OpenAI, DashScope, Gemini, Ollama, etc., and is deeply compatible with the current large model open source ecosystem.

AgentScope provides a variety of out-of-the-box functions, and you can build multi-agent applications by simply dragging and dropping.

Even developers without distributed development experience can easily achieve tens of thousands of levels of multi-agent concurrency on the AgentScope platform.

#In order to allow more users to quickly and easily develop their own multi-agent applications. AgentScope provides the following functions:

• Drag-and-drop programming paradigm-AgentScope Workstation: Provides users with visual Drag-and-drop development interface
• Interactive programming assistant——AgentScope Copilot: Answer developers’ questions about AgentScope
• Transparent and controllable development process - real-time application monitoring: Real-time monitoring of application operating costs and multi-agent status to achieve transparent and controllable development
• Rich development resources: Facilitate fast and convenient secondary development, building applications without "starting from scratch"

AgentScope Workstation

AgentScope Workstation provides a convenient "drag-and-drop" multi-agent application orchestration paradigm.

Here, programming experience is no longer a factor that limits your imagination. Each developer can select and drag their favorite large models, agents and pipelines in the rich toolbar without coding, and combine them freely like building blocks to create unique and innovative multi-agent applications.

In order to ensure that these multi-agent applications built through drag-and-drop are truly usable, AgentScope Workstation introduces static rule checking to ensure the correctness of the application. This ensures correct characterization of the application.

For advanced developers seeking further customization and in-depth development, AgentScope Workstation also provides strong support.

Developers can either export the application as configuration information and run it with the help of the AgentScope Workstation engine, or use AgentScope Workstation Compiler to convert the configuration information into Python code with one click. In this way, developers can further edit and optimize the code to achieve more refined and personalized application adjustments.

AgentScope Copilot

AgentScope Copilot is a development assistant built on the AgentScope framework itself, designed to help Developers solve the problems encountered in the development process of multi-agent applications. Its technical implementation combines multi-agent group chat (Multi-agent Conversation) and data retrieval generation (Retrieval-Augmented Generation, RAG) , agent call (Mention) and many other features.

In interacting with AgentScope Copilot, developers can interact with the Guide Assistant (Guide Assistant) to get help directly; they can also call a dedicated agent assistant, such as a question and answer assistant （Tutoring Assistant）or Code Programming Assistant（Coding Assistant）, so as to get more professional and specific answers. More specifically, the code programming assistant can help developers quickly clarify the definition and usage of each module in the framework, and provide better programming suggestions.

It is worth mentioning that AgentScope Copilot itself is built based on the RAG module in the AgentScope framework and supports popular data retrieval frameworks such as LlamaIndex, as well as a variety of vectors Database type, and supports access to various large language models. Developers can quickly carry out secondary development and easily build the Copilot assistant for their own projects.

Transparent and controllable development process

A friendly application development process should be controllable and transparent to developers .

However, in a multi-agent scenario, there are many types of model APIs and different calling interfaces. How to effectively manage and monitor the usage cost of model APIs and avoid resource waste and unexpected expenses poses a higher challenge to resource monitoring capabilities.

For example, taking a complex web page as input to a large model when using a search engine will cause high overhead, and developer perception often lags behind. In order to solve this problem, AgentScope designed the Monitor module to implement:

Automatic API cost statistics: Accurately record the token usage of different model APIs, and automatically calculate the current cost to ensure that developers Every expense of model API costs is clearly understood.

Budget setting and overage alarm: Support developers to set the budget upper limit of each model API. When the total expenditure exceeds the budget, the system automatically triggers an alarm and promptly notifies the developer to make inspections and adjustments to avoid overspending.

Support custom monitoring indicators: In addition to the preset model API related indicators, Monitor also allows developers to customize other monitoring indicators, such as the cost of search tools and the cost of data storage services. , network traffic, etc., allowing developers to comprehensively and automatically monitor the status of applications.

△ Monitor module

Ready-to-use development resources

AgentScope It has built-in rich tool functions, agents, and application examples. Developers can easily develop their own multi-agent applications through lightweight modifications.

Tool functions: AgentScope supports various types of tool functions including network search, database query, file operation, text processing, etc. Each category contains different implementation forms. For example, at the network search level, AgentScope already supports multiple search engines such as Bing, arXiv and DBLP.

Agent: AgentScope has built-in agents with different functions, including basic dialogue, formatted dialogue, Reasoning, RAG, distributed and many different types of agents. Developers only need to initialize the agent instance with different parameters to specialize the agent into the agent instance they need.

Application Example: AgentScope is preset with dialogue (Conversation) , game (Game) , and distributed ( Distribution) and many different types of applications. On the one hand, these samples can help developers reduce development costs, and on the other hand, they also provide templates and references for application development.

In order to allow developers to better and faster understand the built-in resources in AgentScope, AgentScope provides rich and detailed documentation, including tutorial (Tutorial) and interface documentation (API Document) and design papers to help developers better understand and use AgentScope.

Stable and reliable

Multi-agent applications based on large models will face many new challenges such as model illusion and insufficient ability to follow model instructions. In order to ensure that multi-agent applications can run stably and reliably, AgentScope first classifies errors that occur in the application, and then provides a complete set of fault-tolerance mechanisms and customized fault-tolerance processing accordingly.

Randomness-oriented fault tolerance: Random errors are often caused by unstable network conditions or uncertainty in the content generated by the model. They are the most common type when building applications based on large models. mistake. Such errors are often very trivial and difficult to exhaust. Therefore, AgentScope uses the built-in retry mechanism to filter and shield such random errors, making it easier for developers to devote their energy to application orchestration.

Rule-based fault tolerance: Some errors encountered in the application can be repaired through rules. For example, when a large model is required to generate a response in a specified format, the large model sometimes generates additional content, which can be truncated by preset rules to ensure the normal operation of the application.

Model-based fault tolerance: Using the large model's own capabilities to correct errors is one of the characteristics of multi-agent applications. AgentScope will try to provide input and error information to the large model, using the large model's own capabilities to correct errors. Model understanding and knowledge to correct errors.

Fault tolerance for agents/developers: When preset rules and large models cannot solve the error, it often requires the intervention of developers or agents to solve the problem, so AgentScope In the event of such an error, the formatted attribution, error information, input and output information of the error will be fully submitted to the developer or agent to help solve the problem encountered.

Prompt Optimization

The improvement of multi-agent application performance depends largely on the quality of the prompt (Prompt) of the large model. A good prompt can greatly increase the probability of successful application operation. The AgentScope programming framework provides a prompt tuning module to help developers continuously optimize applications.

Automatic generation of prompts: For developers, generating a good prompt is often a time-consuming and labor-intensive task. AgentScope is preset with an agent, which internally uses In-context learning (ICL) to comprehensively integrate the developer's application scenarios and directly generate the required prompts to help developers quickly start development. .

Support sample input: In AgentScope, developers can input several samples as templates at the same time, and AgentScope can generate specific prompt words for specific downstream tasks based on these samples.

Dynamic tuning of prompts: During the application running process, the prompt words of the large model need to be further adjusted according to the operating conditions, such as adding new rules to avoid errors. AgentScope automates this process. In the process of the agent interacting with developers and the environment, its historical data will become the basis for adjusting prompts; AgentScope modifies the agent's system prompt(System prompt)# based on the scene at that time. ##Thus improving the performance of the agent during operation. Distributed Parallel

As a multi-agent programming framework, AgentScope took improving the collaboration efficiency between agents as one of its main goals from the beginning of its design, and designed distribution for this purpose. style mode. In this mode, multiple agents can run in different processes and machines, thereby making full use of computing resources and improving operating efficiency. The distributed design in AgentScope mainly has the following characteristics:

Automatic parallel optimization

: The distributed design in AgentScope follows the Actor programming paradigm and can automatically identify potential differences between different agents in application process orchestration. Parallel possibilities, and automatic parallel optimization is performed to improve operating efficiency. At the same time, each agent can run independently on local or remote machines, making full use of computing resources and supporting large-scale deployment.

The threshold for getting started is extremely low

: AgentScope completely shields developers from the implementation details of distributed technologies. Developers can develop distributed multi-agent applications at zero cost, or use existing multi-agent applications to The agent application is converted into a distributed mode to run. When converted into distributed applications, distributed application orchestration in AgentScope is fully compatible with localized orchestration methods. Even without distributed background knowledge, developers can easily orchestrate distributed multi-agent applications.

Support large-scale deployment

: AgentScope currently supports running more than 16,000 agent instances at a time on a single machine (64-core 8-card A100) , and the Scale can grow linearly as the number of machines increases. For example, AgentScope can complete more than 64,000 agent calls in 30 seconds on a cluster of 4 machines. This feature makes large-scale parallelism and simulation of agents possible.

△

Conversion from local mode to distributed mode

Multi-modal support

AgentScope supports developer use Multimodal data and multimodal models to build powerful multi-agent applications. In order to allow developers to interact with their programmed multi-agent applications more intuitively and conveniently, AgentScope provides a developer-friendly, easy-to-use interactive interface AgentScope Studio, which allows different modalities such as text, sound, and images to be used. The data is vividly presented, ensuring that developers can experience and adjust the intelligent applications they create in the most intuitive way.

△

AgentScope Studio Next, AgentScope will continue to bring more to developers With the convenience of development, we also look forward to more developers joining the construction of the AgentScope open source community and exploring more interesting multi-agent applications.

AgentScope open source warehouse address:

https://github.com/modelscope/agentscope

Welcome Trial:

https://agentscope.aliyun.com

The above is the detailed content of Alibaba’s intelligent “assembly factory” is open source! 0 experience to handle tens of thousands of agents concurrently. For more information, please follow other related articles on the PHP Chinese website!