Recommended basic introductory tutorials for dynamic websites

Regular matching is usually used when crawling the web content of a single website. However, the structures of different websites are so strange that it is difficult to match them with a unified regular expression. The author of "General Web Page Text Extraction Algorithm Based on Line Block Distribution Function" summarized the general methods of extracting article text from web pages, proposed a text extraction algorithm based on line block distribution, and provided implementations in PHP, Java, etc. The main principles of this algorithm are based on two points: 1. Text area density: after removing all tags in HTML, the character density in the text area is higher and there are fewer multiple lines of blanks; 2. Line block length: the content in non-text areas is average Shorter in individual labels (line blocks). The algorithm steps are as follows: 1. Remove all tags, including styles, Js script content, etc., but retain the original line breaks\n2. Split the web page content by lines, and define the line block $block_i$ as the first $[i, i + blockSize ]$ The sum of lines of text and gives the distribution function of line block length based on line numbers: 3. The text appears in the longest line block, intercept the range from both sides to the line block length of 0: 4. If you need to extract the pictures that appear in the text area , you only need to keep

1 when removing the tag in the first step. [python tutorial] Web page text and content image extraction algorithm

#Introduction: Regular matching is usually used when crawling the web content of a single website. However, the structures of different websites are so strange that it is difficult to use unified regular expressions. formula to match. The author of "General Web Page Text Extraction Algorithm Based on Line Block Distribution Function" summarized the general methods of extracting article text from web pages, proposed a text extraction algorithm based on line block distribution, and provided implementations in PHP, Java, etc. The main principles of this algorithm are based on two points:

2. Example of php extracting web page text_PHP tutorial

Introduction: Example of php extracting the body content of a web page. Example of php extracting the text content of a web page because the difficulty lies in how to identify and retain the article part of the web page, and delete other useless information, and it must be universal and cannot be like a train

3. Where is the text information of web pages generally stored_html/css_WEB-ITnose

Introduction: Where is the text information of web pages generally stored?

4. Example of php extracting the body content of a web page

Introduction: Example of php extracting the body content of a web page. Example of php extracting the text content of a web page because the difficulty lies in how to identify and retain the article part of the web page, and delete other useless information, and it must be universal and cannot be like a train

5. In-depth analysis of the source code of using python to capture the text of web pages

Introduction: Usually when you open a web page, in addition to the text content of the article, there is usually a lot of navigation. , advertising and other information. The purpose of this article is to explain how to extract the text content of an article from a web page and transition away other irrelevant information.

6. javascript collection of methods to change font size [original]_javascript skills

Introduction: Provide the text of the web page, Switching function between small, medium and large fonts. Use js code to set the fontSize attribute of div style.

7. js Get the height and width of dom (visible area and part, etc.)_javascript skills

Introduction : The width or height of the visible area of ​​the web page, the width or height of the full body text of the web page, and the left or right part of the body text of the web page. Please see below for details. I hope it will be helpful to everyone.

[Related Q&A recommendations] ：

objective-c - iOS web page text extraction open source library

##javascript - What is the implementation principle of Evernote's Chrome plug-in clipping

The above is the detailed content of Recommended basic introductory tutorials for dynamic websites. For more information, please follow other related articles on the PHP Chinese website!