search
HomeWeb Front-endHTML TutorialJsoup代码解读之一-概述_html/css_WEB-ITnose

今天看到一个用python写的抽取正文的东东,美滋滋的用Java实现了一番,放到了webmagic里,然后发现Jsoup里已经有了…觉得自己各种不靠谱啊!算了,静下心来学学好东西吧!

Jsoup是Java世界用作html解析和过滤的不二之选。支持将html解析为DOM树、支持CSS Selector形式选择、支持html过滤,本身还附带了一个Http下载器。从今天开始会写一个Jsoup源码解读系列,比起之前的博客,尽量会写的详尽一些。

概述

Jsoup的代码相当简洁,Jsoup总共53个类,且没有任何第三方包的依赖,对比最终发行包9.8M的SAXON,实在算得上是短小精悍了。

jsoup

├── examples #样例,包括一个将html转为纯文本和一个抽取所有链接地址的例子。

├── helper #一些工具类,包括读取数据、处理连接以及字符串转换的工具

├── nodes #DOM节点定义

├── parser #解析html并转换为DOM树

├── safety #安全相关,包括白名单及html过滤

└── select #选择器,支持CSS Selector以及NodeVisitor格式的遍历

使用

Jsoup的入口是Jsoup类。examples包里提供了两个例子,解析html后,分别用CSS Selector以及NodeVisitor来操作Dom元素。

这里用ListLinks里的例子来说明如何调用Jsoup:

public static void main(String[] args) throws IOException { Validate.isTrue(args.length == 1, "usage: supply url to fetch"); String url = args[0]; print("Fetching %s...", url);// 下载url并解析成html DOM结构 Document doc = Jsoup.connect(url).get(); // 使用select方法选择元素,参数是CSS Selector表达式 Elements links = doc.select("a[href]");print("\nLinks: (%d)", links.size()); for (Element link : links) { //使用abs:前缀取绝对url地址 print(" * a: <%s> (%s)", link.attr("abs:href"), trim(link.text(), 35)); }}

Jsoup使用了自己的一套DOM代码体系,这里的Elements、Element等虽然名字和概念都与Java XML APIorg.w3c.dom类似,但并没有代码层面的关系。就是说你想用XML的一套API来操作Jsoup的结果是办不到的,但是正因为如此,才使得Jsoup可以抛弃xml里一些繁琐的API,使得代码更加简单。

还有一种方式是通过NodeVisitor来遍历DOM树,这个在对整个html做分析和替换时比较有用:

public interface NodeVisitor {//遍历到节点开始时,调用此方法 public void head(Node node, int depth);//遍历到节点结束时(所有子节点都已遍历完),调用此方法 public void tail(Node node, int depth);}HtmlToPlainText的例子说明了如何使用NodeVisitor来遍历DOM树,将html转化为纯文本,并将需要换行的标签替换为换行\n:public static void main(String... args) throws IOException { Validate.isTrue(args.length == 1, "usage: supply url to fetch"); String url = args[0];// fetch the specified URL and parse to a HTML DOM Document doc = Jsoup.connect(url).get();HtmlToPlainText formatter = new HtmlToPlainText(); String plainText = formatter.getPlainText(doc); System.out.println(plainText);}public String getPlainText(Element element) { //自定义一个NodeVisitor - FormattingVisitor FormattingVisitor formatter = new FormattingVisitor(); //使用NodeTraversor来装载FormattingVisitor NodeTraversor traversor = new NodeTraversor(formatter); //进行遍历 traversor.traverse(element); return formatter.toString();}

下一节将从DOM结构开始对Jsoup代码进行分析。

Statement
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn
What are self-closing tags? Give an example.What are self-closing tags? Give an example.Apr 27, 2025 am 12:04 AM

Self-closingtagsinHTMLandXMLaretagsthatclosethemselveswithoutneedingaseparateclosingtag,simplifyingmarkupstructureandenhancingcodingefficiency.1)TheyareessentialinXMLforelementswithoutcontent,ensuringwell-formeddocuments.2)InHTML5,usageisflexiblebutr

Beyond HTML: Essential Technologies for Web DevelopmentBeyond HTML: Essential Technologies for Web DevelopmentApr 26, 2025 am 12:04 AM

To build a website with powerful functions and good user experience, HTML alone is not enough. The following technology is also required: JavaScript gives web page dynamic and interactiveness, and real-time changes are achieved by operating DOM. CSS is responsible for the style and layout of the web page to improve aesthetics and user experience. Modern frameworks and libraries such as React, Vue.js and Angular improve development efficiency and code organization structure.

What are boolean attributes in HTML? Give some examples.What are boolean attributes in HTML? Give some examples.Apr 25, 2025 am 12:01 AM

Boolean attributes are special attributes in HTML that are activated without a value. 1. The Boolean attribute controls the behavior of the element by whether it exists or not, such as disabled disable the input box. 2.Their working principle is to change element behavior according to the existence of attributes when the browser parses. 3. The basic usage is to directly add attributes, and the advanced usage can be dynamically controlled through JavaScript. 4. Common mistakes are mistakenly thinking that values ​​need to be set, and the correct writing method should be concise. 5. The best practice is to keep the code concise and use Boolean properties reasonably to optimize web page performance and user experience.

How can you validate your HTML code?How can you validate your HTML code?Apr 24, 2025 am 12:04 AM

HTML code can be cleaner with online validators, integrated tools and automated processes. 1) Use W3CMarkupValidationService to verify HTML code online. 2) Install and configure HTMLHint extension in VisualStudioCode for real-time verification. 3) Use HTMLTidy to automatically verify and clean HTML files in the construction process.

HTML vs. CSS and JavaScript: Comparing Web TechnologiesHTML vs. CSS and JavaScript: Comparing Web TechnologiesApr 23, 2025 am 12:05 AM

HTML, CSS and JavaScript are the core technologies for building modern web pages: 1. HTML defines the web page structure, 2. CSS is responsible for the appearance of the web page, 3. JavaScript provides web page dynamics and interactivity, and they work together to create a website with a good user experience.

HTML as a Markup Language: Its Function and PurposeHTML as a Markup Language: Its Function and PurposeApr 22, 2025 am 12:02 AM

The function of HTML is to define the structure and content of a web page, and its purpose is to provide a standardized way to display information. 1) HTML organizes various parts of the web page through tags and attributes, such as titles and paragraphs. 2) It supports the separation of content and performance and improves maintenance efficiency. 3) HTML is extensible, allowing custom tags to enhance SEO.

The Future of HTML, CSS, and JavaScript: Web Development TrendsThe Future of HTML, CSS, and JavaScript: Web Development TrendsApr 19, 2025 am 12:02 AM

The future trends of HTML are semantics and web components, the future trends of CSS are CSS-in-JS and CSSHoudini, and the future trends of JavaScript are WebAssembly and Serverless. 1. HTML semantics improve accessibility and SEO effects, and Web components improve development efficiency, but attention should be paid to browser compatibility. 2. CSS-in-JS enhances style management flexibility but may increase file size. CSSHoudini allows direct operation of CSS rendering. 3.WebAssembly optimizes browser application performance but has a steep learning curve, and Serverless simplifies development but requires optimization of cold start problems.

HTML: The Structure, CSS: The Style, JavaScript: The BehaviorHTML: The Structure, CSS: The Style, JavaScript: The BehaviorApr 18, 2025 am 12:09 AM

The roles of HTML, CSS and JavaScript in web development are: 1. HTML defines the web page structure, 2. CSS controls the web page style, and 3. JavaScript adds dynamic behavior. Together, they build the framework, aesthetics and interactivity of modern websites.

See all articles

Hot AI Tools

Undresser.AI Undress

Undresser.AI Undress

AI-powered app for creating realistic nude photos

AI Clothes Remover

AI Clothes Remover

Online AI tool for removing clothes from photos.

Undress AI Tool

Undress AI Tool

Undress images for free

Clothoff.io

Clothoff.io

AI clothes remover

Video Face Swap

Video Face Swap

Swap faces in any video effortlessly with our completely free AI face swap tool!

Hot Tools

Dreamweaver Mac version

Dreamweaver Mac version

Visual web development tools

SublimeText3 Mac version

SublimeText3 Mac version

God-level code editing software (SublimeText3)

SAP NetWeaver Server Adapter for Eclipse

SAP NetWeaver Server Adapter for Eclipse

Integrate Eclipse with SAP NetWeaver application server.

MinGW - Minimalist GNU for Windows

MinGW - Minimalist GNU for Windows

This project is in the process of being migrated to osdn.net/projects/mingw, you can continue to follow us there. MinGW: A native Windows port of the GNU Compiler Collection (GCC), freely distributable import libraries and header files for building native Windows applications; includes extensions to the MSVC runtime to support C99 functionality. All MinGW software can run on 64-bit Windows platforms.

PhpStorm Mac version

PhpStorm Mac version

The latest (2018.2.1) professional PHP integrated development tool