How to Efficiently Log Raw HTTP Request/Response Bodies in FastAPI?

Logging Raw HTTP Request/Response in Python FastAPI

Requirement:

Capture and save the raw JSON bodies of specific route requests and responses, with data size around 1MB, without significantly affecting response times.

Option 1: Utilizing Middleware

Middleware Concept

Middleware intercepts every request before it reaches endpoints and responses before they go to clients, allowing for data manipulation. However, the issue with consuming the request body stream in middleware is that it becomes unavailable to downstream endpoints. Hence, we'll use the set_body() function to make it available.

For Responses, Use BackgroundTask

Logging can be performed using BackgroundTask, which ensures that logging happens after the response has been sent to the client, avoiding delays in response times.

Middleware Example

# Logging middleware
async def some_middleware(request: Request, call_next):
    req_body = await request.body()
    await set_body(request, req_body)
    response = await call_next(request)

    # Body storage in RAM
    res_body = b''
    async for chunk in response.body_iterator:
        res_body += chunk

    # Background logging task
    task = BackgroundTask(log_info, req_body, res_body)
    return Response(...)

# Endpoint using middleware
@app.post('/')
async def main(payload: Dict):
    pass

Option 2: Custom APIRoute Class

APIRoute Class Extension

By creating a custom APIRoute class, we can control request and response bodies, limiting its usage to specific routes via an APIRouter.

Important Considerations

For large responses (e.g., streaming media), the custom route may encounter RAM issues or client-side delays due to reading the entire response into RAM. Hence, consider excluding such endpoints from the custom route.

Custom APIRoute Class Example

class LoggingRoute(APIRoute):
    async def custom_route_handler(request: Request) -> Response:
        req_body = await request.body()
        response = await original_route_handler(request)

        # Response handling based on type
        if isinstance(response, StreamingResponse):
            res_body = b''
            async for item in response.body_iterator:
                res_body += item
            response = Response(...)
        else:
            response.body

        # Logging task
        task = BackgroundTask(log_info, req_body, response.body)
        response.background = task
        return response

# Endpoint using custom APIRoute
@router.post('/')
async def main(payload: Dict):
    return payload

Choosing an Option

Both options provide solutions for logging request and response data without significantly impacting response times. Option 1 allows for general logging, while Option 2 provides granular control over routes that require logging.

The above is the detailed content of How to Efficiently Log Raw HTTP Request/Response Bodies in FastAPI?. For more information, please follow other related articles on the PHP Chinese website!