ChatGPT started to connect to the Internet, and the final seal was lifted

ChatGPT is an artificial intelligence with "super high IQ", especially the upgraded version with GPT-4 core. However, we know that its training data is as of 2021. Without training, it is impossible for AI to answer accurately.

Since its launch in November last year, ChatGPT has been used by countless people. People have been asking for this large language model to access more data in various forms. On March 24, OpenAI finally announced that it had partially lifted ChatGPT. Unable to connect to the Internet.

OpenAI’s solution is to use third-party plug-ins as a bridge to allow AI to "see" external data in a safer environment. Yesterday the agency opened its first list of ChatGPT plugins. This batch of plugins was created by Expedia, FiscalNote, Instacart, KAYAK, Klarna, Milo, OpenTable, Shopify, Slack, Speak, Wolfram, and Zapier.

Specifically, the plugin now lets you do the following with ChatGPT:

• Retrieve real-time information: e.g. Sports scores, stock prices, latest news, etc.;
• retrieves knowledge base information: such as company documents, personal notes, etc.;
• performs operations on behalf of users: for example, booking air tickets, ordering meals, etc.

In addition, OpenAI also provides two plug-ins, including a web browser and a code interpreter, and open source the code of a knowledge base retrieval plug-in. Now, any developer can build their own plug-ins to enhance ChatGPT's repository.

Currently access to the alpha version of the plug-in has been extended to more users and developers on the waiting list. Although OpenAI said it will prioritize a small number of developers and ChatGPT Plus users, it also plans to Larger scale opening will be carried out in the future.

To give everyone an intuitive feeling, here is an example: users can select and install the Wolfram plug-in from ChatGPT to improve its computational intelligence.

The data in Wolfram Alpha comes from major academic websites, publications and scientific institutions, and its professionalism is absolutely guaranteed. Is there such a ChatGPT? Make you feel stronger?

Overview

Although today's large language models can complete various tasks, their role is still limited. Training data is the only information they can learn from, which may be out of date but still adaptable to all human needs. Furthermore, the only ability a language model has out of the box is to output text. This text may contain useful instructions, but to actually follow them would require a lot more processing on the part of the person.

While not a perfect analogy, plug-ins can become the "eyes and ears" of a language model, allowing the language model to access new, private, or specific data that is not included in information in the training data.

In response to explicit user requests, plug-ins can also enable language models to perform safe, restricted operations on their behalf, thereby increasing the usefulness of the entire system.

OpenAI anticipates the emergence of an open standard that unifies applications for AI interaction, and they are making early attempts at such a standard.

Today, OpenAI begins to gradually open the plug-ins built by OpenAI's early collaborators for ChatGPT users. The first targets covered are ChatGPT Plus subscribers. In addition, it has also begun to allow developers to create their own plug-ins for ChatGPT. plug-in capabilities.

In the coming months, as the security system improves, OpenAI plans to enable developers using OpenAI models to integrate plug-ins into their own applications, not just It's ChatGPT.

Security and Wider Implications

Of course, connecting language models to external tools opens new opportunities, but also significant new risks .

Plugins offer the potential to solve various challenges associated with large language models, including large model "hallucinations", tracking recent events, and accessing (permissioned) proprietary information sources. By integrating explicit access to external data, such as online latest information, code-based calculations, or custom plug-ins to retrieve information, language models can enhance their responses with evidence-based reference content.

These references not only enhance the utility of the model, but also enable users to evaluate the trustworthiness of the model output and double-check its accuracy, potentially mitigating conflicts with the recent GPT-4 Risks associated with over-reliance are discussed in the system card. Finally, the value of plug-ins may span existing limitations by helping users handle a variety of new use cases, from browsing product catalogs to booking flights or ordering food.

But at the same time, plug-ins can take harmful or unintentional actions, increasing the ability of bad actors to defraud, mislead, or abuse others, thereby increasing security challenges. By increasing the range of possible applications, plug-ins may increase the risk that the model will take incorrect or misaligned actions in new domains with negative consequences.

These factors guided the development of the ChatGPT plug-in platform, for which OpenAI has introduced a number of safeguards.

Previously, OpenAI has conducted "red team exercises" internally and with external collaborators, practicing many possible related scenarios. For example, red teams have discovered that if a plugin is released without security, it can perform complex prompt injections, send spoofed emails and spam, bypass security restrictions, or misuse the information sent to the plugin.

OpenAI is using these findings to drive security design mitigations to limit risky plugin behavior and increase transparency into how and when they run as part of the user experience, in addition to using These findings serve to confirm the decision to gradually deploy access to the plug-in.

Plug-ins may have widespread social impact. For example, in one paper, OpenAI researchers found that language models that can use tools may have a greater economic impact than language models without tools. More generally, according to what other researchers have found, current artificial intelligence technology Waves will have a large impact on the speed at which jobs are transformed, replaced and created.

Let ChatGPT browse the web

Inspired by a series of work such as WebGPT, GopherCite, BlenderBot2, LaMDA2, etc., allowing language models to read information from the Internet will strictly It greatly expands the scope of what can be discussed, going beyond the training corpus and incorporating current and fresh information.

The image below is an example of how browsing opens up an experience for ChatGPT users where a previous model might politely point out that its training data did not include enough Information makes it answer. In this example, ChatGPT retrieved information about the most recent Academy Awards (awarded on March 13, 2023) and then performed a familiar ChatGPT verse performance. Browsing becomes a way to add to the experience.

Q: Can you tell me which person/movie won an Oscar in these categories?

• Best Actor
• Best Score
• Best Picture
• Best Supporting Actor

Then come up with a poem to tie it all together.

ChatGPT will give you a series of search results, and you can click directly to view relevant information sources.

In addition to providing practical value to end users, language and chat models enable thorough and interpretable The research shows the promise of scalable alignment work.

It is important to note that the plug-in for text-based web browsers is limited to making GET requests, which reduces but does not eliminate certain categories of security risks. Browsing plug-ins are scoped to retrieve information, but do not include "transactional" operations such as form submissions - which pose greater security risks.

The Browse feature uses the Microsoft Bing Search API to retrieve content from the web. As such, this feature inherits Microsoft's extensive work on source reliability and information authenticity, as well as "Safe Mode" that prevents the retrieval of questionable content. The plugin runs in a separate service, and ChatGPT's browsing activity is separate from the rest of the infrastructure.

To respect content creators and adhere to web norms, the ChatGPT browser plug-in’s user-agent token is ChatGPT-User and is configured to respect the website’s robots.txt file. Occasionally it may result in a "click failure," which indicates that the plugin is following the website's instructions to avoid crawling it. This user-agent will only be used to take direct action on behalf of ChatGPT users and will not be used to scrape the web in any automated manner. OpenAI also publishes IP egress ranges and implements rate throttling to avoid sending too much traffic to websites.

Code Interpreter

Experimental ChatGPT model that can handle uploading and downloading using Python

OpenAI for The model behind ChatGPT provides a Python interpreter that works in a sandboxed, firewalled execution environment, as well as some temporary disk space. Code run by the interpreter plugin is evaluated in a persistent session that is active for the duration of the chat conversation (with a capped timeout), and subsequent calls can build upon each other. Currently this feature supports uploading files to the current conversation workspace and downloading work results.

Click on Finished Calculating in the picture:

From During initial user research, OpenAI identified some valuable use cases for using the code interpreter:

• Solving quantitative and qualitative mathematical problems
• Perform data analysis and visualization
• Convert files between formats

Security mechanisms

The first precaution in connecting a ChatGPT model to a programming language interpreter is to properly sandbox the execution so that the AI-generated code does not have unintended side effects in the real world. OpenAI executes code in a secure environment and uses strict network controls to prevent external internet access from executing the code. Additionally, OpenAI places resource limits on each session.

Disabling internet access limits the functionality of the code sandbox, but it may be the safest initial form of AI-assisted programming. Third-party plug-ins are designed with security as a top priority and connect ChatGPT to the outside world.

Data Acquisition

Open source retrieval plug-in enables ChatGPT to access an individual or organization's information sources with permission. It allows users to fetch the most relevant document snippets from their data sources, such as files, notes, emails, or public documents, by asking questions or expressing requirements in natural language.

As an open source and self-hosted solution, developers can deploy their own version of the plugin and register it on ChatGPT. Its plugin leverages the OpenAI embed and allows developers to choose a vector database such as Milvus, Pinecone, Qdrant, Redis, Weaviate or Zilliz to index and search documents. Information sources can be synchronized with the database using webhooks.

The retrieval plugin allows ChatGPT to search the content vector database and add the best results to the ChatGPT session. This means it is not subject to any external influence, with the main risks being data authorization and privacy. Developers should only add content to retrieval plugins that they have permission to use and can be shared within a user's ChatGPT session.

Third-party plug-ins

Using third-party plug-ins on ChatGPT is like this:

OpenAI prepares a description manifest file for third-party plug-ins, which includes a machine-readable description of the plug-in functions, calling methods, and user-facing documentation.

{
"schema_version": "v1",
"name_for_human": "TODO Manager",
"name_for_model": "todo_manager",
"description for human": "Manages your TODOs!",
"description_for_model": "An app for managing a user's TODOs",
"api": { "url": "/openapi.json" },
"auth": { "type": "none" },
"logo_url": "https://example.com/logo.png",
"legal_info_url": "http://example.com",
"contact_email": "hello@example.com"
}

Steps to build a plug-in:

1. Build the API endpoint you want the language model to call (can be a new API, an existing API APIs or wrappers around existing APIs designed specifically for LLM).

2. Create an OpenAPI specification that documents your API, and a manifest file that links to the OpenAPI specification and contains some plugin-specific metadata.

When starting a conversation on chat.openai.com, users can select which third-party plugins they want to enable. Documents about enabled plug-ins are displayed to the language model as part of the conversation context, enabling the model to call the appropriate plug-in APIs as needed to fulfill the user intent. Currently, plugins are designed for calling backend APIs, and OpenAI is exploring plugins that can also call client APIs.

OpenAI said they are working hard to develop the plug-in and promote it to a wider audience.

This means that the storm brought by ChatGPT is sweeping everything.

The above is the detailed content of ChatGPT started to connect to the Internet, and the final seal was lifted. For more information, please follow other related articles on the PHP Chinese website!