How to Overcome HTTP Error 403 Forbidden When Downloading Historic Stock Data?

HTTP Error 403: Forbidden when Attempting to Retrieve Historic Stock Data

This issue stems from an attempt to automatically download historic stock data using urllib2, which resulted in an HTTP Error 403 Forbidden response.

To address this error, several solutions have been proposed, including modifying the user agent string and accepting response cookies. However, these attempts have been unsuccessful.

Upon further investigation, it was discovered that by adding additional headers to the request, particularly the 'Accept' header, the data could be successfully retrieved.

The following code snippet illustrates the successful implementation:

import urllib2,cookielib

site= "http://www.nseindia.com/live_market/dynaContent/live_watch/get_quote/getHistoricalData.jsp?symbol=JPASSOCIAT&fromDate=1-JAN-2012&toDate=1-AUG-2012&datePeriod=unselected&hiddDwnld=true"
hdr = {'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.11 (KHTML, like Gecko) Chrome/23.0.1271.64 Safari/537.11',
       'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
       'Accept-Charset': 'ISO-8859-1,utf-8;q=0.7,*;q=0.3',
       'Accept-Encoding': 'none',
       'Accept-Language': 'en-US,en;q=0.8',
       'Connection': 'keep-alive'}

req = urllib2.Request(site, headers=hdr)

try:
    page = urllib2.urlopen(req)
except urllib2.HTTPError, e:
    print e.fp.read()

content = page.read()
print content

Notably, the addition of only the 'Accept' header was sufficient to resolve the error.

The above is the detailed content of How to Overcome HTTP Error 403 Forbidden When Downloading Historic Stock Data?. For more information, please follow other related articles on the PHP Chinese website!