A preliminary exploration of python socket programming

Socket programming steps

The server creates a socket, binds the address and port, and then listens for incoming connections on the port. Once a connection comes in, it receives the incoming connection through the accept function.

The client also creates a socket. Bind the remote address and port, then establish a connection and send data.

Server-side socket

The following is a detailed description of the server-side socker_server.py

import socket
import sys

HOST = "127.0.0.1"               
PORT = 10000              
s = None
for res in socket.getaddrinfo(HOST, PORT, socket.AF_UNSPEC,
                              socket.SOCK_STREAM, 0, socket.AI_PASSIVE):
    af, socktype, proto, canonname, sa = res
    try:
        s = socket.socket(af, socktype, proto)
    except socket.error as msg:
        s = None
        continue
    try:
        s.bind(sa)
        s.listen(5)
    except socket.error as msg:
        s.close()
        s = None
        continue
    break
if s is None:
    print 'could not open socket'
    sys.exit(1)
conn, addr = s.accept()
print 'Connected by', addr
while 1:
    data = conn.recv(1024)
    if not data: break
    conn.send(data)
conn.close()

through a piece of example code. First, we use the socket.getaddrinnfo function to convert host/port into a sequence containing 5 tuples. This 5-tuple contains all the necessary parameters we need to create a socket connection. The returned 5-tuples are (family, sockettype, proto, canonname, sockaddr)

family address cluster, used as the first parameter of the socket() function. There are mainly the following ones:

socket.AF_UNIX is used to communicate with processes on a single machine.
socket.AF_INET is used to communicate with servers. This is usually used.
socket.AF_INET6 supports IPv6
sockettype socket type, used as the second parameter of the socket() function. Commonly used ones are

socket.SOCK_STREAM default, used for TCP protocol
socket.SOCK_DGRAM used for UDP protocol
proto protocol, used The third parameter of the socket() function. The getaddrinnfo function will return the appropriate protocol

canonname and a standardized host name based on the address format and socket type.

sockaddr describes a socket address. It is a tuple, mainly used for bind() and connect() functions

Next, create a socket object and pass in the af, sockettype, proto returned by the getaddrinnfo function.

s = socket.socket(af, socktype, proto)

Then bind my socket address

s.bind(sa)

Enable listening mode

s.listen(5)

The listen function will listen for connections connected to the socket. The parameter indicates that the maximum number of connection queues that the system can suspend before rejecting the connection is 5. These connections have not yet been accepted. The quantity cannot be infinite, usually 5 is specified.

Once we listen to the connection, we will call the accept function to receive the connection

conn, addr = s.accept()

The accept function returns a tuple, and conn is a new socket object used to receive and send data. addr represents the socket address of the other end.

Next we can use the conn object to send and receive data

 data = conn.recv(1024) # 接收数据, 这里指定一次最多接收的字符数量为1024
 conn.send(data) # 发送数据

Here we will stop running when we receive a connection socket, so if we want to loop the connection, put the accept function into an infinite loop.

Client socket

Client socket programming is relatively simple. After establishing a connection with the server through connect, you can communicate with each other. socket_client.py is as follows

for res in socket.getaddrinfo(HOST, PORT, socket.AF_UNSPEC, socket.SOCK_STREAM):
    af, socktype, proto, canonname, sa = res
    try:
        s = socket.socket(af, socktype, proto)
    except socket.error as msg:
        s = None
        continue
    try:
        s.connect(sa)
    except socket.error as msg:
        s.close()
        s = None
        continue
    break
if s is None:
    print 'could not open socket'
    sys.exit(1)
s.sendall('Hello, world')
data = s.recv(1024)
s.close()
print 'Received', repr(data)

The above is mainly for socket programming of TCP stream data. For UDP protocol data, the processing is slightly different. For example, the processing function for sending and receiving UDP packets is:

socket.sendto(string, flags, address)
socket.recvfrom(bufsize[, flags]) #返回(string, address)，string是返回的数据，address是发送方的socket地址

SocketServer module

In addition to the socket module, network programming in Python also provides the SocketServer module. This module mainly encapsulates the socket module and binds the creation of socket objects. Determining, connecting, receiving, sending, and closing are all encapsulated in it, which greatly simplifies the programming of network services.

This module provides the following 2 main network service classes for creating corresponding socket streams

TCPServer creates a socket stream of the TCP protocol

UDPServer creates a socket stream of the UDP protocol

We have In addition to the socket stream object, a request processing class is also needed. The SocketServer module provides request processing classes including BaseRequestHandler, and its derived classes StreamRequestHandler and DatagramRequestHandler. So just inherit one of these 3 classes and then override the handle function. This function will be used to handle the received request. Let’s look at a server-side code example

import SocketServer

class MyTCPHandler(SocketServer.StreamRequestHandler):
   """创建请求处理类，重写handle方法。此外也可以重写setup()和finish()来做一些请求处理前和处理后的一些工作"""
    def handle(self):
        # self.request is the TCP socket connected to the client
        self.data = self.request.recv(1024).strip()
        print "{} wrote:".format(self.client_address[0])
        print self.data
        # just send back the same data, but upper-cased
        self.request.sendall(self.data.upper())

if __name__ == "__main__":
    HOST, PORT = "localhost", 10000

    server = SocketServer.TCPServer((HOST, PORT), MyTCPHandler)

    # Activate the server; this will keep running until you
    # interrupt the program with Ctrl-C
    # server.shutdown()
    server.serve_forever()  # 一直循环接收请求
    # server.handle_request() # 只处理一次请求就退出

看着是不是代码简单了很多，而且SocketServer模块内部使用了多路复用IO技术，可以实现更好的连接性能。看serve_forever函数的源代码用到了select模块。通过传入socket对象调用select.select()来监听socket对象的文件描述符，一旦发现socket对象就绪，就通知应用程序进行相应的读写操作。源代码如下：

def serve_forever(self, poll_interval=0.5):
        """Handle one request at a time until shutdown.

        Polls for shutdown every poll_interval seconds. Ignores
        self.timeout. If you need to do periodic tasks, do them in
        another thread.
        """
        self.__is_shut_down.clear()
        try:
            while not self.__shutdown_request:
                # XXX: Consider using another file descriptor or
                # connecting to the socket to wake this up instead of
                # polling. Polling reduces our responsiveness to a
                # shutdown request and wastes cpu at all other times.
                r, w, e = _eintr_retry(select.select, [self], [], [],
                                       poll_interval)
                if self in r:
                    self._handle_request_noblock()
        finally:
            self.__shutdown_request = False
            self.__is_shut_down.set()

即使使用了select技术，TCPServer，UDPServer处理请求仍然是同步的，意味着一个请求处理完，才能处理下一个请求。但SocketServer模块提供了另外2个类用来支持异步的模式。

ForkingMixIn 利用多进程实现异步

ThreadingMixIn 利用多线程实现异步

看名字就知道使用了mixin模式。而mixin模式可以通过多继承来实现，所以通过对网络服务类进行多继承的方式就可以实现异步模式

class ThreadedTCPServer(SocketServer.ThreadingMixIn, SocketServer.TCPServer):
    pass

针对ThreadindMixIn,实现异步的原理也就是在内部对每个请求创建一个线程来处理。看源码

def process_request(self, request, client_address):
        """Start a new thread to process the request."""
        t = threading.Thread(target = self.process_request_thread,
                             args = (request, client_address))
        t.daemon = self.daemon_threads
        t.start()

下面提供一个异步模式的示例

import socket
import threading
import SocketServer

class ThreadedTCPRequestHandler(SocketServer.BaseRequestHandler):

    def handle(self):
        data = self.request.recv(1024)
        cur_thread = threading.current_thread()
        response = "{}: {}".format(cur_thread.name, data)
        self.request.sendall(response)

class ThreadedTCPServer(SocketServer.ThreadingMixIn, SocketServer.TCPServer):
    pass

def client(ip, port, message):
    sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    sock.connect((ip, port))
    try:
        sock.sendall(message)
        response = sock.recv(1024)
        print "Received: {}".format(response)
    finally:
        sock.close()

if __name__ == "__main__":
    # Port 0 means to select an arbitrary unused port
    HOST, PORT = "localhost", 0

    server = ThreadedTCPServer((HOST, PORT), ThreadedTCPRequestHandler)
    ip, port = server.server_address

    # Start a thread with the server -- that thread will then start one
    # more thread for each request
    server_thread = threading.Thread(target=server.serve_forever)
    # Exit the server thread when the main thread terminates
    server_thread.daemon = True
    server_thread.start()
    print "Server loop running in thread:", server_thread.name

    client(ip, port, "Hello World 1")
    client(ip, port, "Hello World 2")
    client(ip, port, "Hello World 3")

    server.shutdown()
    server.server_close()

以上是本人对socket相关的理解，有什么不当或错误之处，还请指出。