Detailed explanation of the principles of how Java implements native socket communication mechanism

This article mainly introduces the principle of implementing the native socket communication mechanism in JAVA. The editor thinks it is quite good, so I will share it with you now and give it as a reference. Let’s follow the editor and take a look.

This article introduces the principle of implementing native socket communication mechanism in JAVA and shares it with everyone. The details are as follows:

Current environment

jdk == 1.8

Knowledge points

• Socket connection processing

• IO input and output stream processing

• Request data format processing

• Request model optimization

Scenario

Today, let’s talk about socket communication issues in JAVA. Here we take the simplest one-request-one-response model as an example, assuming that we now need to communicate with the Baidu site. How can we use JAVA's native socket to achieve this?

Establish a socket connection

First, we need to establish a socket connection (core code)

import java.net.InetSocketAddress;
import java.net.Socket;
import java.net.SocketAddress;
    
// 初始化 socket
Socket socket = new Socket();
// 初始化远程连接地址
SocketAddress remote = new InetSocketAddress(host, port);
// 建立连接
socket.connect(remote);

Processing socket input and output streams

After successfully establishing a socket connection, we can obtain its input and output streams. The essence of communication is the processing of input and output streams. Through the input stream, the data from the network connection is read, and through the output stream, the local data is sent to the remote end.

Socket connection is actually somewhat similar to processing file streams, both are performing IO operations.

The code to obtain the input and output streams is as follows:

// 输入流
InputStream in = socket.getInputStream();
// 输出流
OutputStream out = socket.getOutputStream();

Regarding the processing of IO streams, we generally use corresponding packaging classes to process IO streams. If If we deal with it directly, we need to operate on byte[], which is relatively cumbersome. If we use a wrapper class, we can directly process it with types such as string and int, which simplifies IO byte operations.

The following uses BufferedReader and PrintWriter as input and output packaging classes for processing.

// 获取 socket 输入流
private BufferedReader getReader(Socket socket) throws IOException {
  InputStream in = socket.getInputStream();
  return new BufferedReader(new InputStreamReader(in));
}

// 获取 socket 输出流
private PrintWriter getWriter(Socket socket) throws IOException {
  OutputStream out = socket.getOutputStream();
  return new PrintWriter(new OutputStreamWriter(out));
}

Data request and response

With the socket connection and IO input and output stream, the next step is to send Request data and get the response results of the request.

Because of the support of the IO packaging class, we can directly transmit in string format, and the packaging class will help us convert the data into the corresponding byte stream.

Because we are accessing the baidu site through HTTP, we do not need to define additional output formats. Using the standard HTTP transmission format, you can perform request responses (some specific RPC frameworks may have customized communication formats).

The requested data content is processed as follows:

public class HttpUtil {

  public static String compositeRequest(String host){

    return "GET / HTTP/1.1\r\n" +
        "Host: " + host + "\r\n" +
        "User-Agent: curl/7.43.0\r\n" +
        "Accept: */*\r\n\r\n";
  }
  
}

The code for sending the request data is as follows:

// 发起请求
PrintWriter writer = getWriter(socket);
writer.write(HttpUtil.compositeRequest(host));
writer.flush();
接收响应数据代码如下：

// 读取响应
String msg;
BufferedReader reader = getReader(socket);
while ((msg = reader.readLine()) != null){
  System.out.println(msg);
}

So far, we have finished talking about all the core codes for creating connections, sending requests and receiving responses under native sockets.

The complete code is as follows:

import java.io.*;
import java.net.InetSocketAddress;
import java.net.Socket;
import java.net.SocketAddress;
import com.test.network.util.HttpUtil;

public class SocketHttpClient {

  public void start(String host, int port) {

    // 初始化 socket
    Socket socket = new Socket();

    try {
      // 设置 socket 连接
      SocketAddress remote = new InetSocketAddress(host, port);
      socket.setSoTimeout(5000);
      socket.connect(remote);

      // 发起请求
      PrintWriter writer = getWriter(socket);
      System.out.println(HttpUtil.compositeRequest(host));
      writer.write(HttpUtil.compositeRequest(host));
      writer.flush();

      // 读取响应
      String msg;
      BufferedReader reader = getReader(socket);
      while ((msg = reader.readLine()) != null){
        System.out.println(msg);
      }

    } catch (IOException e) {
      e.printStackTrace();
    } finally {
      try {
        socket.close();
      } catch (IOException e) {
        e.printStackTrace();
      }
    }

  }

  private BufferedReader getReader(Socket socket) throws IOException {
    InputStream in = socket.getInputStream();
    return new BufferedReader(new InputStreamReader(in));
  }

  private PrintWriter getWriter(Socket socket) throws IOException {
    OutputStream out = socket.getOutputStream();
    return new PrintWriter(new OutputStreamWriter(out));
  }

}

Below, we show the results of socket communication by instantiating a client.

public class Application {

  public static void main(String[] args) {

    new SocketHttpClient().start("www.baidu.com", 80);

  }
}

Result output:

Request model optimization

In this way, there is no problem in implementing the function. But if we look closely, we find that IO blocking occurs during the IO writing and reading process. That is:

// 会发生 IO 阻塞
writer.write(HttpUtil.compositeRequest(host));
reader.readLine();

So if you want to request 10 different sites at the same time, as follows:

public class SingleThreadApplication {

  public static void main(String[] args) {

    // HttpConstant.HOSTS 为 站点集合
    for (String host: HttpConstant.HOSTS) {

      new SocketHttpClient().start(host, HttpConstant.PORT);

    }

  }
}

It must be After the first request response is completed, the next site processing will be initiated.

This is more obvious on the server side. Although the code here is a client connection, the specific operations are similar to those on the server side. Requests can only be processed serially one by one, which will definitely not meet the response time standard.

• Multi-threading

Some people think this is not a problem at all. JAVA is a multi-threaded programming language. For this situation, a multi-threaded model is most appropriate.

public class MultiThreadApplication {

  public static void main(String[] args) {

    for (final String host: HttpConstant.HOSTS) {

      Thread t = new Thread(new Runnable() {
        public void run() {
          new SocketHttpClient().start(host, HttpConstant.PORT);
        }
      });

      t.start();

    }
  }
}

This method seems useful at first, but if the amount of concurrency is large, the application will use a lot of threads. As we all know, on the server, each thread actually occupies a file handle. The number of handles on the server is limited, and a large number of threads will cause considerable consumption of switching between threads. Therefore, this method must be unbearable in scenarios with large concurrency.

• Multi-threading + thread pool processing

Since too many threads are not enough, then we can just control the number of threads created. Only a fixed number of threads are started for socket processing, which not only utilizes multi-threaded processing, but also controls system resource consumption.

public class ThreadPoolApplication {

  public static void main(String[] args) {

    ExecutorService executorService = Executors.newFixedThreadPool(8);

    for (final String host: HttpConstant.HOSTS) {

      Thread t = new Thread(new Runnable() {
        public void run() {
          new SocketHttpClient().start(host, HttpConstant.PORT);
        }
      });

      executorService.submit(t);
      new SocketHttpClient().start(host, HttpConstant.PORT);

    }

  }
}

Regarding the number of started threads, generally the CPU-intensive type will be set at N+1 (N is the number of CPU cores), and the IO-intensive type will be set at 2N + 1.

This method seems to be the optimal one. Is there anything better? If a thread can handle multiple socket connections at the same time and does not block when the input and output data of each socket is not ready, is it better? This technology is called "IO multiplexing". The corresponding implementation is provided in JAVA's nio package.

The above is the detailed content of Detailed explanation of the principles of how Java implements native socket communication mechanism. For more information, please follow other related articles on the PHP Chinese website!