Does the requests module truncate headers?

Question content

I'm trying to make a POST request to my API using a fairly long (1017 characters) bearer token for authorization.

No matter what I try, when I initiate the request I get an error message stating that \u2026 (horizontal ellipsis) in position 512 cannot be encoded using "latin-1" encoding. Since I can't find any problem with the header I specified (or the ellipsis), my only explanation is that the requests module is silently truncating my token internally. Is that so? Can it be disabled? Are there any other solutions?

Correct answer

No, it does not truncate the header. You can see this by running the localhost http server which prints all requests received. Here's an example using 2000 character tags.

# from https://gist.github.com/mdonkers/63e115cc0c79b4f6b8b3a6b797e485c7
from http.server import basehttprequesthandler, httpserver
import logging

class s(basehttprequesthandler):
    def _set_response(self):
        self.send_response(200)
        self.send_header('content-type', 'text/html')
        self.end_headers()

    def do_post(self):
        content_length = int(self.headers['content-length']) 
        post_data = self.rfile.read(content_length) 
        logging.info("post request,\npath: %s\nheaders:\n%s\n\nbody:\n%s\n",
                str(self.path), str(self.headers), post_data.decode('utf-8'))

        self._set_response()
        self.wfile.write("post request for {}".format(self.path).encode('utf-8'))

def run(server_class=httpserver, handler_class=s):
    logging.basicconfig(level=logging.info)
    server_address = ('', 8080)
    httpd = server_class(server_address, handler_class)
    try:
        httpd.serve_forever()
    except keyboardinterrupt:
        pass
    httpd.server_close()

if __name__ == '__main__':
    run()

Run it on localhost and then use requests to post the request:

import requests
import random
import string

token = ''.join(random.choices(string.ascii_letters, k=2000))
requests.request('post', 'http://localhost:8080', headers={'token': token}, body='hello world')

Produces the following output:

INFO:root:POST request,
Path: /
Headers:
Host: localhost:8080
User-Agent: python-requests/2.31.0
Accept-Encoding: gzip, deflate, br
Accept: */*
Connection: keep-alive
token: CrJlFIgJbvNdpTFPvkvCpaCPmugqMbjCUOYJaCCoibZdLfrEmLRDyxXiABGqHPTpkDYXznIWyVHwmThzBbzISh
umCnRXzQveOrSCAgfYWsfNSQDKGRvsTLnwCAWuVofumJPIhJQgQibpVJPQzqjncqjDbbHVjHidOJievmATjRexUPtVtke
zuMaifyNkPiMPJwwoofurbMJyueoKKdnDuTUxhTrryuuMSUYhoRjVIvtxtBdpIBjvrRcDFbLZtQXSBwcqpuSRWfnNMjll
xebGylOpGEqsfQJtfzkUbtJgbSvzuolDcyXvVjVTqnFakPPGRojYVBioUsllKABIjqzzZblTncrxKYjPSTgzoBIPnjcIn
WYWBsKXkRoRZbVFJnaEERnXjNTMkqGfqMDuADdGXUASRTOUsSDNcpRUzMwwWVhjQEWIpqtrDrxazIkvcoGwERSFDyUgGZ
FtAiWTkiBiwKmcFbwYkZXIJBWGvXJAVylHvMcYrgyVYycbbwabTRAbviddMQCopTZLgbriyZTMvepQOWabQaVdMRGeeZU
zOrLXiqjnulhyNWQOkOgTVVZCNTavrVnJJtBGPMDqzDvcenXziMRIQeXmvyuenZnTDgTSWSAMMPEPwHYPHKzbjLQspeeY
SETSPfgCfoGWattjpnghHJgvarBSLkwwLYfgZhjejobPXRiVZLdzIXhjXSsomeEwAsIUOgcuKBNIOuyZGZVGTeqYMhoSR
JHtfnLWJQIgyDbppALRMmZtAIrerBgNMeDueanSZmogaRNsrQVjIlWnSjqAjEWRtkdmRXoIQWkAYZniMjXOAfBDnQnUoW
uEFQRvbDxCFlwdBnjJDFYaWhHLPYDzLzZOfZtJqVEucAujDXBQhTtXXgnhSHrnCGuWsTzTDUzmKbIDaYdPTmVMDArvufd
fKXLUbriqvrJBMVoTXltcnCpqEELnnYKfgDfWgLCDHRaqJlkiZYUIANHsIrZxhFEJTpsCCwOMwaqjNQtgDDWfWseOgnwc
LUfGzQUKyayNdkwXEOCNsWoDfHGEfNatWvdNeiMTlXiztljvmXLDgJqzIwlqnkXmnVkPtsUJiJbieDYYDALTLNQNHbQjx
uiquCHQscKViGcAgVCJZqoZFhzDtuDRewTYHomvpttxoTTRePRxYoGFDIRUBcTlLhJnouqNjQgZDaPdvyWSLIxddmYYNU
bKoKVWNwfYQsFQgoVsSFSboBgIvJjkxMnifwdDqkNJAcZxQYcwhUUbDwCyYbMSNmpGYaJlxeNUlcrLWXSrzAtAfblXtnC
UzpnOJVYIsGONvdZyJIDwGdbaXsGZtILdPFTanTIhkoPUzDCjdaEslDXCqcaCwjIUKJMEiileCiIGASLyFYFLkKBHepbn
dgmzeJQqEKycchmquNLGoVTJRPPwFBBRyzoEXylieyAUihLHHulYFJIWKXayiwkqIWjPKxmHaHUWanrzMbiHaMYdubXxk
YAANJbNKvIWjZhheFMSOeIPogikXHTspixWKdFTZWWOKqsahTkFKJzrmUtBtqoYlXSjBErsbkbzJOtuXAeXHbKZsekmmU
ohgOAdwaTVJgQLsYRIzIHnhErjKYEViADXfdXncenufRJDmAkEcjpqpxmFKdQmtqKwlUskVtgBAeVwnOTxdUnucxOcdyD
uHoWAntZculQybCDYDEuqnzzjRklcGgAsNkyogLeZCRDSNWhQJCEQoiSChtAFWpmmzAjlhEDOnTzJyIUCzdbbPjhuDynk
AuCggemjZIuisiVqgtJirqrbQeFlWckKVWsjIURcpeyGhQXzcQBQraqMJgBUafQyJiCcOefBrvwJOqyARmUZnunpwkBlD
ytgwnwbBhdTUfyfUYeJjbDMDKiIYSOcyosmZvINlUYUuPhkkyojtYOzYtSFCdwTwxFJcqCDOTxyalYkvNYDvuaQKFwWIQ
fkQAXiaoDQYREuhyntQBDCJohSqcdjZntuwgOVCvOPyJYMSNzksVDx
Content-Length: 11


Body:
hello world

The above is the detailed content of Does the requests module truncate headers?. For more information, please follow other related articles on the PHP Chinese website!