Home > Article > Backend Development > Crawler parsing method 1: JOSN parsing
Crawler parsing method 1: JOSN parsing
- 爱喝马黛茶的安东尼forward
- 2019-06-05 11:14:324936browse
Many languages can crawl, but crawlers based on python are more concise and convenient. Crawlers have also become an essential part of the python language. There are many ways for crawlers to parse data. In the previous article, I introduced to you what kind of data the crawler can obtain and the specific parsing methods. ,This This article brings you JSON parsing.
Json is simply an object and an array in JavaScript, so these two structures are objects and arrays. Structure, through these two structures various complex structures can be represented.
Object:The object is represented in js as the content enclosed by { }, and the data structure is { key: value, key: value, ... } The structure of key-value pairs. In object-oriented languages, key is the attribute of the object, and value is the corresponding attribute value, so it is easy to understand. The value method is object.key to obtain the attribute value. The type of this attribute value can be a number. , strings, arrays, objects.
Array:The array in js is the content enclosed by square brackets [ ], and the data structure is ["Python", "javascript", "C ", ...], the value acquisition method is the same as in all languages, using index to obtain, the type of field value can be numbers, strings, arrays, objects.
JSON (JavaScript Object Notation) is a lightweight data exchange format that makes it easy for people to read and write. It also facilitates machine analysis and generation. Suitable for data interaction scenarios, such as data interaction between the front and backend of a website.
import json
def resolveJson(path):
file = open(path, "rb")
fileJson = json.load(file)
field = fileJson["field"]
futures = fileJson["futures"]
type = fileJson["type"]
name = fileJson["name"]
time = fileJson["time"]
return (field, futures, type, name, time)
def output():
result = resolveJson(path)
print(result)
for x in result:
for y in x:
print(y)
path = r"C:\Users\dell\Desktop\kt\test.json"
output()Note that when the function returns multiple values, it returns a tuple;
When performing a for loop on a string, each character will be traversed
Python JSON
In this chapter we will introduce how to use Python language to encode and decode JSON objects.
JSON (JavaScript Object Notation) is a lightweight data exchange format that is easy for humans to read and write.
Using the JSON function requires importing the json library: import json.
Function Encoded JSON string decoded into Python object
json.dumps##json.dumps Used to encode Python objects into JSON strings.
Syntax
Example
The following example encodes the array into JSON format Data:#!/usr/bin/python
import json
data = [ { 'a' : 1, 'b' : 2, 'c' : 3, 'd' : 4, 'e' : 5 } ]
json = json.dumps(data)
print jsonThe execution result of the above code is: [{"a": 1, "c": 3, "b": 2, "e": 5, "d": 4}]Use parameters to format JSON data for output: >>> import json
>>> print json.dumps({'a': 'Runoob', 'b': 7}, sort_keys=True, indent=4, separators=(',', ': '))
{
"a": "Runoob",
"b": 7
}Conversion control of python original type to json type Table:
Python #list, tuple array
None null
json.loads
json.loads 用于解码 JSON 数据。该函数返回 Python 字段的数据类型。
语法
json.loads(s[, encoding[, cls[, object_hook[, parse_float[, parse_int[, parse_constant[, object_pairs_hook[, **kw]]]]]]]])
实例
以下实例展示了Python 如何解码 JSON 对象：
<pre class="brush:php;toolbar:false">
#!/usr/bin/python
import json
jsonData = '{"a":1,"b":2,"c":3,"d":4,"e":5}';
text = json.loads(jsonData)
print text
以上代码执行结果为：
{u'a': 1, u'c': 3, u'b': 2, u'e': 5, u'd': 4}json 类型转换到 python 的类型对照表：
JSON Python
object dict
array list
string unicode
number (int) int, long
number (real) float
true True
false False
null None
使用第三方库：Demjson
Demjson 是 python 的第三方模块库，可用于编码和解码 JSON 数据，包含了 JSONLint 的格式化及校验功能。
Github 地址：https://github.com/dmeranda/demjson
官方地址：http://deron.meranda.us/python/demjson/
环境配置
在使用 Demjson 编码或解码 JSON 数据前，我们需要先安装 Demjson 模块。本教程我们会下载 Demjson 并安装：
$ tar -xvzf demjson-2.2.3.tar.gz $ cd demjson-2.2.3 $ python setup.py install
JSON 函数
函数 描述
encode 将 Python 对象编码成 JSON 字符串
decode 将已编码的 JSON 字符串解码为 Python 对象
encode
Python encode() 函数用于将 Python 对象编码成 JSON 字符串。
语法
demjson.encode(self, obj, nest_level=0)
实例
以下实例将数组编码为 JSON 格式数据：
<pre class="brush:php;toolbar:false">
#!/usr/bin/python
import demjson
data = [ { 'a' : 1, 'b' : 2, 'c' : 3, 'd' : 4, 'e' : 5 } ]
json = demjson.encode(data)
print json
以上代码执行结果为：
[{"a":1,"b":2,"c":3,"d":4,"e":5}]decode
Python 可以使用 demjson.decode() 函数解码 JSON 数据。该函数返回 Python 字段的数据类型。
语法
demjson.decode(self, txt)
实例
以下实例展示了Python 如何解码 JSON 对象：
<pre class="brush:php;toolbar:false">
#!/usr/bin/python
import demjson
json = '{"a":1,"b":2,"c":3,"d":4,"e":5}';
text = demjson.decode(json)
print text
以上代码执行结果为：
{u'a': 1, u'c': 3, u'b': 2, u'e': 5, u'd': 4}The above is the detailed content of Crawler parsing method 1: JOSN parsing. For more information, please follow other related articles on the PHP Chinese website!