A large model development toolset has been created!

The content that needs to be rewritten is: Author Richard MacManus

Planning | Yan Zheng

Web3 failed to subvert Web2, but the emerging large model development stack is allowing developers to start from the "cloud" The "native" era is moving towards a new AI technology stack.

Tip engineers may not be able to touch the nerves of developers to rush to large models, but a sentence from a product manager or leader: Can an "agent" be developed, can a "chain" be implemented, and "Which vector database to use?" , but it has become the difficulty for driving technology students in major mainstream large model application companies to overcome the development of generative AI.

What are the layers of the emerging technology stack? Where is the most difficult part? This article will lead you to find out

1. The technology stack needs to be updated. Developers are ushering in the era of AI engineers

In the past year, some tools have emerged, such as LangChain and LlamaIndex. This has allowed the developer ecosystem for AI applications to begin to mature. There is even a term now used to describe those who focus on the development of artificial intelligence, namely "AI engineer". According to Shawn @swyx Wang, this is the next step for "prompt engineers". He also created a coordinate chart to visualize where AI engineers fit into the broader artificial intelligence ecosystem

Source: swyx

Large-scale language model (LLM) is the core technology of AI engineers. It is no coincidence that both LangChain and LlamaIndex are tools that extend and complement LLM. But what other tools are available for this new breed of developer?

So far, the best diagram I’ve seen of the LLM stack comes from venture capital firm Andreessen Horowitz (a16z). The following is its view on the "LLM app stack":

Source: a16z

2. Yes, the top layer is still data

In the LLM technology stack, data is the most important component, this is very obvious. According to a16z's chart, the data is at the top. In LLM, "embedded model" is a very critical area, and you can choose from OpenAI, Cohere, Hugging Face, or dozens of other LLM options, including the increasingly popular open source LLM

Before using LLM, a "data pipeline" needs to be established. For example, consider Databricks and Airflow as two examples, or the data can be processed "unstructured". This also applies to the periodicity of data and can help companies "clean" or simply organize the data before entering it into a custom LLM. "Data intelligence" companies like Alation offer this type of service, which sounds a bit like tools such as "business intelligence" that are better known in the IT technology stack

The last part of the data layer is very popular these days A vector database for storing and processing LLM data. According to Microsoft's definition, this is a database that stores data as high-dimensional vectors, which are mathematical representations of features or attributes. Data is stored as vectors using embedding technology. In a media chat, leading vector database vendor Pinecone noted that their tools are often used with data pipeline tools such as Databricks. In this case, the data is typically stored elsewhere (such as a data lake) and then transformed into embedded data via a machine learning model. After processing and chunking, the resulting vectors are sent to Pinecone

3, Hints and Queries

The next two levels can be summarized as hints and queries - this is an artificial intelligence application The point of interaction where the program interfaces with LLM and (optionally) other data tools. A16z positions LangChain and LlamaIndex as "orchestration frameworks," meaning that once developers understand which LLM they are using, they can leverage these tools

According to a16z, orchestration like LangChain and LlamaIndex The framework "abstracts away many of the details of prompt linking," which means querying and managing data between the application and the LLM. This orchestration process includes interacting with external API interfaces, retrieving context data from the vector database, and maintaining memory across multiple LLM calls. The most interesting box in a16z’s diagram is “Playground,” which includes OpenAI, nat.dev, and Humanloop

A16z isn’t exactly defined in the blog post, but we can infer that the “Playground” tool can help The developers perform what A16z calls "cue jiu-jitsu." In these places, developers can experiment with various prompting techniques.

Humanloop is a British company whose platform features a “collaborative prompt workspace.” It further describes itself as a "complete development toolkit for production LLM functionality." So basically it allows you to try LLM stuff and then deploy it into your application if it works

4. Assembly line operations: LLMOps

At present, the layout of large-scale production lines is gradually becoming clear. On the right side of the orchestration box, there are many operation boxes, including LLM caching and verification. In addition, there are a series of LLM-related cloud services and API services, including open API repositories such as Hugging Face, and proprietary API providers such as OpenAI

This may be our first step in "cloud native" It’s no coincidence that many DevOps companies have added artificial intelligence to their product lists in the most similar place in the tech stack that developers are used to. In May, I spoke with Harness CEO Jyoti Bansal. Harness runs a "software delivery platform" that focuses on the "CD" part of the CI/CD process.

Bansai told me that AI can alleviate the tedious and repetitive tasks involved in the software delivery life cycle, from generating specifications based on existing functionality to writing code. Additionally, he said AI can automate code reviews, vulnerability testing, bug fixes, and even create CI/CD pipelines for builds and deployments. According to another conversation I had in May, AI is also changing developer productivity. Trisha Gee from the build automation tool Gradle told me that AI can speed up development by reducing time on repetitive tasks, like writing boilerplate code, and allowing developers to focus on the big picture, like making sure the code meets business needs.

5. Web3 is out, and the large model development stack is coming

In the emerging LLM development technology stack, we can observe a series of new product types, such as orchestration frameworks (such as LangChain and LlamaIndex), vector databases and "playground" platforms such as Humanloop. All of these products are extending and/or supplementing the core technologies of the current era: large language models, just like the rise of cloud-native era tools such as Spring Cloud and Kubernetes in previous years. However, at present, almost all large, small, and top companies in the cloud native era are trying their best to adapt their tools to AI engineering, which will be very beneficial to the future development of the LLM technology stack.

Yes, this time the big model seems to be "standing on the shoulders of giants." The best innovations in computer technology are always based on the past. Maybe that's why the "Web3" revolution failed - it wasn't so much building on the previous generation as trying to usurp it.

The LLM technology stack seems to have done it, and it has become a bridge from the cloud development era to a newer, artificial intelligence-based developer ecosystem

Reference link:

https:/ /www.php.cn/link/c589c3a8f99401b24b9380e86d939842

The above is the detailed content of A large model development toolset has been created!. For more information, please follow other related articles on the PHP Chinese website!