A must-have for front-end people: Detailed explanation of packet capture principles and usage of packet capture tool whistle

After 3 years, I came into contact with mobile h5 page development again. The last time I was an intern was my senior year. This time it is hybrid development, which involves the interaction between the h5 page and the native app. The h5 page needs to open the login state with the native app, and call the interface of the native app, such as calling the native camera to scan the QR code. Different from WeChat mini program development, WeChat provides WeChat developer tools for local development, which can be simulated and called locally. However, I need to package static files every time and upload them to the server for debugging, which is very troublesome.

Can I run local code when the native app loads online h5? The answer is yes. You can use packet capture tools such as whistle to intercept online page request data and then respond to local code. This article mainly describes the principle of packet capture and the use of the packet capture tool whistle.

1. Principle of packet capture

1.1 What is packet capture?

Packet capture is to intercept, resend, edit, and transfer the data packets sent and received during network transmission. Through packet capture, you can:

• Analyze network problems

• Business analysis

• Analyze network information flow

• Network big data Financial risk control

• Detect attacks attempting to invade the network

• Detect abuse of network resources by internal and external users

• Detect the impact of network intrusion

• Monitor link Internet broadband traffic

• Monitor network usage traffic (including internal users, external Users and systems)

• Monitor the security status of the Internet and user computers

• Penetration and deception

• ...

Recalling computer network knowledge, data is transmitted on the network in small units of frames, and frames pass through a specific network called The driver program is formed, then sent to the network cable through the network card, reaches the destination machine through the network cable, and the reverse process is performed at one end of the destination machine. The receiving machine's Ethernet captures these frames, tells the operating system that the frame has arrived, and then stores it. During this transmission and reception process, you can use packet capture tools (Sniffers) to capture packets. As a front-end developer, you usually capture HTTP/HTTPS packets at the application layer.

1.2 HTTP/HTTPS packet capture principle

HTTP/HTTPS is a communication protocol used by the application layer, a common application layer system The architecture is client-server architecture.

How do client programs and server programs running on different systems communicate with each other? In fact, in operating system terms, it is actually the process that communicates rather than the program. A process can be thought of as a program running on the end system.

In a web application, a client browser process exchanges session messages with a server process.

The browser process needs to know the host address of the receiving process and the identifier of the receiving process defined in the destination host, which is the destination port.

Most applications consist of pairs of communicating processes, with the two processes in each pair sending messages to each other. A process sends messages to and receives messages from the network through a software interface called a socket.

The process can be compared to a house, and its socket can be its door. The socket is the port between the application layer and the transport layer.

#Now that we know the communication process between the two processes, how do we capture the packets? To give an example in life, Xiao Ming had a crush on Xiao Wen, so he wrote a love letter, but he was a little shy and asked Xiao Wen's good friend Xiao Hua to help deliver the love letter. At this time, Xiaohua can be responsible for delivering love letters between Xiaowen and Xiaoming. As a middleman, she can secretly view the contents of their love letters.

The idea is to set up a middleman process to be responsible for capturing packets. Each session between the target processes first communicates with the middleman process and then forwards it.

1.2.1 HTTP packet capture principle

In the http standard, there is no standard for communication end authentication. For the server, as long as the format of the HTTP request message it receives meets the specification, it will send a response message.

The same is true for the client. It cannot verify the identity of the server, such as the host it connects to http://www.jecyu.com, but due to the existence of intermediate nodes , the final connection may be the host of http://www.jerry.com.

Therefore, for HTTP packet capture, there is no need to do too much processing. You only need to let the middleman be responsible for forwarding the data packets of the client and server.

1.2.2 HTTPS packet capture principle

HTTP is a clear text transmission, which is vulnerable to man-in-the-middle attacks and is not safe.

HTTPS semantics are still HTTP, but a security layer SSL/TSL is inserted between http and tcp in the HTTP protocol stack.

The security layer uses symmetric encryption to encrypt the transmission data and asymmetric encryption to transmit the symmetric key to solve three problems: http data is not encrypted, identity cannot be verified, and data is easily tampered with key problem.

HTTP Encryption Authentication Integrity Protection = HTTPS

The identity verification problem is achieved by verifying the server's certificate, which is a third-party organization (CA Certificate issuing authority) uses digital signature technology to manage , including creating certificates, storing certificates, updating certificates, and revoking certificates.

The browser connects to an HTTPS website. The server sends not only the server entity certificate, but a certificate chain, but does not include the root certificate. The root certificate will be included. Embedded in operating systems such as Windows, Linux, macOS, Android, and iOS.

The verification certificate is divided into two steps, the issuer verification of the certificate and the server entity certificate verification

1. Certificate chain verification:

• 1.1 The browser obtains the public key from the upper-level certificate of the server entity certificate (such as B certificate) to verify the signature of the server entity certificate (the signature is through the private key of the CA organization signed), continue if the verification is successful, otherwise the certificate verification fails.

• 1.2 The browser obtains the public key from the upper-level certificate of certificate B (such as certificate C) to verify the signature of certificate B,

Continue if the verification is successful, otherwise the certificate verification fails.

• 1.3 The browser iteratively verifies the signature of each certificate, and finally finds the self-signed root certificate (the issuer and user are the same person). Since the browser has integrated the root certificate, you can Fully trust the public key of the root certificate to complete the final signature.

2. Server entity certificate verification: Whether the accessed domain name information is consistent with the certificate, date, certificate extension verification, etc.

After understanding the certificate verification, let’s take a look at the specific https communication process:

• The first is the three-way handshake of tcp to establish the connection

• Follows the asymmetric encryption handshake process

• client sends a random numberrandom1 Supported encryption algorithm set

• server receives the information and returns a selected encryption algorithm certificate (including S_public key) random2

• client verifies the validity of the certificate and uses random1 random2 Generate pre-master-secure and send it to server through serverpublic key encryption

• server receives To pre-master-secure, use S_private key to decrypt pre-master-secure according to the agreed algorithm,

• Then use the encryption algorithm to generate master-secure (symmetric encryption key), and then send it to the client

• client receives the generated master-secure, the symmetric encryption key transmission is completed

• Finally, you can use master-secure for real symmetric encryption transmission of data.

If the middleman wants to capture the packet, he needs to intercept the certificate sent by the client before encrypting the HTTPS communication:

• message, pretending to be the server, sending its certificate to the client, and then getting the [message containing the symmetric encryption communication key returned by the client] to generate the symmetric encryption key between the middleman and the client.

• Also pretends to be the client, encrypts it with the server's own asymmetric public key [the message returned by the client containing the symmetric encryption communication key] and sends it to the server to obtain the service Symmetric encryption key generated by the end.

• In this way, the encrypted communication is established, and the middleman obtains the communication data key and can view and modify HTTPS communication messages.

• Here, the client communicates with the middleman, and the middleman communicates with the server, all of which establish HTTPS encrypted connections normally.

One of the most important steps is the browser’s root certificate verification. It is impossible for the CA agency to issue it to an intermediary casually. The domain name certificate that does not belong to it does not exist in the client's operating system. Therefore, can only import the root certificate of the middleman into the client's operating system , so as to complete the verification of the middleman certificate when establishing encrypted communication. verification.

1.3 How does the computer capture the packets of the mobile phone?

If you want to obtain the data packets of the mobile web application through the computer, according to what you have learned before, you need a middleman strategy.

Create a server middleman process on the PC side, pretending to be the target server of the web application. The request data sent by the mobile web application first passes through the middleman, which intercepts and processes it and then sends it to the target server. In turn, the data packets sent by the target server first pass through the middleman, and then the middleman responds to the browser client.

What should be noted here is that whether it is a personal computer or a mobile phone, it needs to be connected to the Internet network and can find each other to establish communication.

Generally speaking, for development, the server process started locally on the personal computer cannot be accessed on the public network. Generally, it is a wireless LAN. A personal computer and a mobile phone can communicate with each other by connecting to the Wi-Fi sent by the same router.

Specific steps:

• Start a server process locally on the PC and listen to a port such as 8899

• Connect to the same LAN on the mobile phone, configure the network proxy, point to the IP address of the PC and port 8899

• In this way, all network communications on the mobile phone will be blocked. First forward it to the 8899 port on the PC side, and then you can analyze and process the data packet

Take accessing youtuBe as an example. For example, the computer has been successfully accessed using [server software]. At this time, as long as the mobile phone configuration The proxy points to the computer's IP address and specified port, and the mobile phone can also access youtuBe.

2. Packet capture tool whistle

2.1 What is whistle

Whistle is a cross-platform packet capture based on Node Free debugging tool, its main features:

1. Completely cross-platform: supports desktop systems such as Mac and Windows, and supports command line systems such as servers

2. Powerful functions:

• Support as HTTP, HTTPS, SOCKS proxy and reverse proxy

• ##Support packet capture and modification of HTTP, HTTPS, HTTP2, WebSocket, TCP requests

• Supports replaying and constructing HTTP, HTTPS, HTTP2, WebSocket, and TCP requests

• Supports setting upstream agents, PAC scripts, Hosts, delays (limited Fast) request response, etc.

• Supports viewing the console log and DOM node of the remote page

• Supports using Node to develop plug-in extension functions, and can also be used as Independent npm package reference

3. Simple operation

• View the package capture and modify the request directly through the browser

• All modification operations can be implemented through configuration (similar to system Hosts), and support group management

• Projects can bring their own proxy rules and configure them to the local Whistle proxy with one click. You can also simplify the operation through customized plug-ins

How to quickly use whistle

• Install node first, it is recommended to use nvm management

• Global installation of whistle

npm i -g whistle & w2 start

After installation, you can set a global proxy on your computer. The proxy port is 8899.

w2 proxy // 设置全局代理

w2 proxy off // 关闭全局代理

You can access

http://127.0.0.1:8899/ through the browser to view packet capture, modify requests, etc.

If you don’t want to use a global proxy, you can install the

SwitchyOmega plug-in and set whistle proxies for certain websites on demand.

• Select Whistle proxy

• Set Whistle proxy

2.2 Things you can do with whistle

There are many things you can do with whistle. Here is the official website picture:

Some example configurations are shown below:

3. Whistle practical case

3.1 Native app loading PC local code development

On the native app, the web page has been loaded through the h5 domain name, but when developing locally, I don’t want to go through the assembly line or locally package and upload the code every time.

You need to proxy the request of the native app to the local server. The prerequisite is that the wifi phone and the computer can access each other, that is, the computer captures the pc package mentioned earlier.

Because my web server is an https application, I need to download the root certificate provided by whistle and manually import it to my phone.

Click the HTTPS menu, then use your mobile phone to scan the QR code, use your mobile browser to open it to download, set up import and set trust in the mobile phone certificate.

此时，再在手机上配置代理指向 PC 电脑的 IP和 whistle 监听的端口即可在电脑上截获数据包。

我本地webpack 启动的服务器应用访问地址为：xxx.xxx.xxx.xxx:8080

whistle 的配置规则：

# Rules

# 访问首页走本地

jecyu.com/webs/short-transport http://xxx.xxx.xxx.xxx:8080?deptCode=755DL # 首页路径

# 后续的请求都使用本地代码

jecyu.com http://xxx.xxx.xxx.xxx:8080?deptCode=755DL

其中试过在原生 app 访问本地应用时出现错误“ webpack 会提示 invalid host header”，解决方案是在 devServer 配置添加即可：

devServer: {

allowedHosts: 'all',

}

至此，成功让原生 app 访问PC 端本地的开发代码。

3.2 查看移动端的 DOM 样式

Whistle 能够通过内置的 Weinre 去实现查看移动端的 DOM 样式，配置规则如下

# 设置 weinre

https://juejin.cn weinre://test

手机上重新访问 juejin.cn 网站，然后打开 weinre 可以看到如下，绿色表示远程连接成功。

可以点击 Element 查看手机上网页 DOM 结构、样式等信息。

也可以在 console 控制台中，执行代码，比如 alert ，手机应用上会显示弹框。

4. 总结

学会抓包是软件开发人员必须掌握的调试技能，本文先介绍抓包的原理，再介绍抓包工具 whistle 的使用，whistle 非常强大，本文只是粗略的介绍，更多的使用技巧，大家可以查看官方文档whistle 文档。

参考资料

• 扯一扯HTTPS单向认证、双向认证、抓包原理、反抓包策略

• 网络抓包原理及Fiddler的使用

• 前端应该知道的web调试工具——whistle

• 移动端调试痛点？——送你五款前端开发利器

• whistle工具全程入门

• Whistle 配置 https 抓包

• 解决webpack5 invalid host Header问题

• whistle localhost不能代理

• Proxying to localhost no longer works

• 使用线上域名访问本地代码调试——whistle代理

• wiki数据包分析器

原文地址：https://juejin.cn/post/7140040425129115684

（学习视频分享：web前端）