How does Flask achieve ChatGPT-like real-time streaming response?

Simulate real-time streaming response of ChatGPT using Flask

Many developers hope to achieve ChatGPT-like real-time response effects in Flask applications: content generation is continuously transmitted to the client. However, a simple Flask response object cannot meet this requirement, and it will wait until the generator function is fully executed before sending the result. This article discusses how to use the Flask framework to achieve true streaming.

The root cause of the problem is that the original code directly uses response object to wrap the generator function, causing the browser to wait for the generator to fully execute before the content can be displayed, which is contrary to the expected real-time response effect.

The core of the improvement is stream_with_context decorator. The following code snippet shows the improved approach:

 from flask import stream_with_context, request

@app.route('/stream')
def streamed_response():
    def generate():
        yield 'Hello'
        yield request.args['name']
        yield '!'
    return app.response_class(stream_with_context(generate()))

stream_with_context(generate()) wraps the generator function. The role of stream_with_context is crucial, ensuring that the generator returns data to the client immediately after each yield , rather than waiting for the entire generator to complete. In this example, the program first returns "Hello", then returns the corresponding name according to the request parameter name , and finally returns "!", realizing the effect of generating and transmitting.

Compared with the original code, the improved code uses stream_with_context to avoid the problem of waiting for the entire generator function to be executed before returning data, real streaming is achieved, and the client can receive data in real time, thereby simulating the real-time response effect of ChatGPT. It should be noted that request.args['name'] shows the parameter passing method. In actual applications, it can be replaced with other data acquisition methods as needed, such as obtaining data from a database or other API. In this way, more dynamic and interactive web applications can be built.

The above is the detailed content of How does Flask achieve ChatGPT-like real-time streaming response?. For more information, please follow other related articles on the PHP Chinese website!