Detailed explanation of Python daemon and script singleton execution

This article mainly introduces the Python daemon and script singleton operation. The editor thinks it is quite good. Now I will share it with you and give it as a reference. Let’s follow the editor to take a look.

1. Introduction

The most important feature of the daemon process is to run in the background; it must be isolated from the environment before it runs. These environments include unclosed file descriptors, control terminals, sessions and process groups, working directories, file creation masks, etc.; it can be started from the startup script /etc/rc.d when the system starts, and can be started by the inetd daemon It can also be started by crond, the job planning process, or executed by a user terminal (usually a shell).

Python sometimes needs to ensure that only one script instance is running to avoid data conflicts.

2. Python daemon process

1. Function implementation

#!/usr/bin/env python 
#coding: utf-8 
import sys, os 
  
'''将当前进程fork为一个守护进程 
  注意：如果你的守护进程是由inetd启动的，不要这样做！inetd完成了 
  所有需要做的事情，包括重定向标准文件描述符，需要做的事情只有chdir()和umask()了 
'''
  
def daemonize (stdin='/dev/null', stdout='/dev/null', stderr='/dev/null'): 
   #重定向标准文件描述符（默认情况下定向到/dev/null） 
  try:  
    pid = os.fork()  
     #父进程(会话组头领进程)退出，这意味着一个非会话组头领进程永远不能重新获得控制终端。 
    if pid > 0: 
      sys.exit(0)  #父进程退出 
  except OSError, e:  
    sys.stderr.write ("fork #1 failed: (%d) %s\n" % (e.errno, e.strerror) ) 
    sys.exit(1) 
  
   #从母体环境脱离 
  os.chdir("/") #chdir确认进程不保持任何目录于使用状态，否则不能umount一个文件系统。也可以改变到对于守护程序运行重要的文件所在目录 
  os.umask(0)  #调用umask(0)以便拥有对于写的任何东西的完全控制，因为有时不知道继承了什么样的umask。 
  os.setsid()  #setsid调用成功后，进程成为新的会话组长和新的进程组长，并与原来的登录会话和进程组脱离。 
  
   #执行第二次fork 
  try:  
    pid = os.fork()  
    if pid > 0: 
      sys.exit(0)  #第二个父进程退出 
  except OSError, e:  
    sys.stderr.write ("fork #2 failed: (%d) %s\n" % (e.errno, e.strerror) ) 
    sys.exit(1) 
  
   #进程已经是守护进程了，重定向标准文件描述符 
  
  for f in sys.stdout, sys.stderr: f.flush() 
  si = open(stdin, 'r') 
  so = open(stdout, 'a+') 
  se = open(stderr, 'a+', 0) 
  os.dup2(si.fileno(), sys.stdin.fileno())  #dup2函数原子化关闭和复制文件描述符 
  os.dup2(so.fileno(), sys.stdout.fileno()) 
  os.dup2(se.fileno(), sys.stderr.fileno()) 
  
#示例函数：每秒打印一个数字和时间戳 
def main(): 
  import time 
  sys.stdout.write('Daemon started with pid %d\n' % os.getpid()) 
  sys.stdout.write('Daemon stdout output\n') 
  sys.stderr.write('Daemon stderr output\n') 
  c = 0
  while True: 
    sys.stdout.write('%d: %s\n' %(c, time.ctime())) 
    sys.stdout.flush() 
    c = c+1
    time.sleep(1) 
  
if __name__ == "__main__": 
   daemonize('/dev/null','/tmp/daemon_stdout.log','/tmp/daemon_error.log') 
   main()

You can view the inheritance running in the background through the command ps -ef | grep daemon.py, in/ tmp/daemon_error.log will record error operation logs, and /tmp/daemon_stdout.log will record standard output logs.

2. Class implementation

#!/usr/bin/env python 
#coding: utf-8 
  
#python模拟linux的守护进程 
  
import sys, os, time, atexit, string 
from signal import SIGTERM 
  
class Daemon: 
 def __init__(self, pidfile, stdin='/dev/null', stdout='/dev/null', stderr='/dev/null'): 
   #需要获取调试信息，改为stdin='/dev/stdin', stdout='/dev/stdout', stderr='/dev/stderr'，以root身份运行。 
  self.stdin = stdin 
  self.stdout = stdout 
  self.stderr = stderr 
  self.pidfile = pidfile 
   
 def _daemonize(self): 
  try: 
   pid = os.fork()  #第一次fork，生成子进程，脱离父进程 
   if pid > 0: 
    sys.exit(0)   #退出主进程 
  except OSError, e: 
   sys.stderr.write('fork #1 failed: %d (%s)\n' % (e.errno, e.strerror)) 
   sys.exit(1) 
   
  os.chdir("/")   #修改工作目录 
  os.setsid()    #设置新的会话连接 
  os.umask(0)    #重新设置文件创建权限 
   
  try: 
   pid = os.fork() #第二次fork，禁止进程打开终端 
   if pid > 0: 
    sys.exit(0) 
  except OSError, e: 
   sys.stderr.write('fork #2 failed: %d (%s)\n' % (e.errno, e.strerror)) 
   sys.exit(1) 
   
   #重定向文件描述符 
  sys.stdout.flush() 
  sys.stderr.flush() 
  si = file(self.stdin, 'r') 
  so = file(self.stdout, 'a+') 
  se = file(self.stderr, 'a+', 0) 
  os.dup2(si.fileno(), sys.stdin.fileno()) 
  os.dup2(so.fileno(), sys.stdout.fileno()) 
  os.dup2(se.fileno(), sys.stderr.fileno()) 
   
   #注册退出函数，根据文件pid判断是否存在进程 
  atexit.register(self.delpid) 
  pid = str(os.getpid()) 
  file(self.pidfile,'w+').write('%s\n' % pid) 
   
 def delpid(self): 
  os.remove(self.pidfile) 
  
 def start(self): 
   #检查pid文件是否存在以探测是否存在进程 
  try: 
   pf = file(self.pidfile,'r') 
   pid = int(pf.read().strip()) 
   pf.close() 
  except IOError: 
   pid = None
   
  if pid: 
   message = 'pidfile %s already exist. Daemon already running!\n'
   sys.stderr.write(message % self.pidfile) 
   sys.exit(1) 
    
  #启动监控 
  self._daemonize() 
  self._run() 
  
 def stop(self): 
  #从pid文件中获取pid 
  try: 
   pf = file(self.pidfile,'r') 
   pid = int(pf.read().strip()) 
   pf.close() 
  except IOError: 
   pid = None
   
  if not pid:  #重启不报错 
   message = 'pidfile %s does not exist. Daemon not running!\n'
   sys.stderr.write(message % self.pidfile) 
   return
  
   #杀进程 
  try: 
   while 1: 
    os.kill(pid, SIGTERM) 
    time.sleep(0.1) 
    #os.system('hadoop-daemon.sh stop datanode') 
    #os.system('hadoop-daemon.sh stop tasktracker') 
    #os.remove(self.pidfile) 
  except OSError, err: 
   err = str(err) 
   if err.find('No such process') > 0: 
    if os.path.exists(self.pidfile): 
     os.remove(self.pidfile) 
   else: 
    print str(err) 
    sys.exit(1) 
  
 def restart(self): 
  self.stop() 
  self.start() 
  
 def _run(self): 
  """ run your fun"""
  while True: 
   #fp=open('/tmp/result','a+') 
   #fp.write('Hello World\n') 
   sys.stdout.write('%s:hello world\n' % (time.ctime(),)) 
   sys.stdout.flush()  
   time.sleep(2) 
    
  
if __name__ == '__main__': 
  daemon = Daemon('/tmp/watch_process.pid', stdout = '/tmp/watch_stdout.log') 
  if len(sys.argv) == 2: 
    if 'start' == sys.argv[1]: 
      daemon.start() 
    elif 'stop' == sys.argv[1]: 
      daemon.stop() 
    elif 'restart' == sys.argv[1]: 
      daemon.restart() 
    else: 
      print 'unknown command'
      sys.exit(2) 
    sys.exit(0) 
  else: 
    print 'usage: %s start|stop|restart' % sys.argv[0] 
    sys.exit(2)

Running result:

It is when Daemon is designed A template, from daemon import Daemon in other files, then define a subclass, and override the run() method to implement your own functions.

class MyDaemon(Daemon):
  def run(self):
    while True:
      fp=open('/tmp/run.log','a+')
      fp.write('Hello World\n')
      time.sleep(1)

Inadequacy: signal processing signal.signal (signal.SIGTERM, cleanup_handler) is not installed temporarily, and the callback function delpid() when the registration program exits is not called.

Then, write a shell command, add the startup service, check whether the daemon process is started every 2 seconds, if not, start it, and automatically monitor the recovery process.

#/bin/sh
while true
do
 count=`ps -ef | grep "daemonclass.py" | grep -v "grep"`
 if [ "$?" != "0" ]; then
   daemonclass.py start
 fi
 sleep 2
done

3. python guarantees that only one script instance can be run

1. Open the file and lock it

#!/usr/bin/env python
#coding: utf-8
import fcntl, sys, time, os
pidfile = 0
  
def ApplicationInstance():
  global pidfile
  pidfile = open(os.path.realpath(__file__), "r")
  try:
    fcntl.flock(pidfile, fcntl.LOCK_EX | fcntl.LOCK_NB) #创建一个排他锁,并且所被锁住其他进程不会阻塞
  except:
    print "another instance is running..."
    sys.exit(1)
  
if __name__ == "__main__":
  ApplicationInstance()
  while True:
    print 'running...'
    time.sleep(1)

Note: The open() parameter cannot use w, otherwise the file itself will be overwritten; pidfile must be declared as a global variable, otherwise the life cycle of the local variable will end and the file descriptor will be referenced. The count is 0 and is recycled by the system (if the entire function is written in the main function, there is no need to define it as global).

2. Open the custom file and lock it

#!/usr/bin/env python
#coding: utf-8
import fcntl, sys, time
pidfile = 0
  
def ApplicationInstance():
  global pidfile
  pidfile = open("instance.pid", "w")
  try:
    fcntl.lockf(pidfile, fcntl.LOCK_EX | fcntl.LOCK_NB) #创建一个排他锁,并且所被锁住其他进程不会阻塞
  except IOError:
    print "another instance is running..."
    sys.exit(0)
  
if __name__ == "__main__":
  ApplicationInstance()
  while True:
    print 'running...'
    time.sleep(1)

3. Detect the PID# in the file ##

#!/usr/bin/env python
#coding: utf-8
import time, os, sys
import signal
  
pidfile = '/tmp/process.pid'
  
def sig_handler(sig, frame):
  if os.path.exists(pidfile):
    os.remove(pidfile)
  sys.exit(0)
  
def ApplicationInstance():
  signal.signal(signal.SIGTERM, sig_handler)
  signal.signal(signal.SIGINT, sig_handler)
  signal.signal(signal.SIGQUIT, sig_handler)
  
  try:
   pf = file(pidfile, 'r')
   pid = int(pf.read().strip())
   pf.close()
  except IOError:
   pid = None
   
  if pid:
   sys.stdout.write('instance is running...\n')
   sys.exit(0)
  
  file(pidfile, 'w+').write('%s\n' % os.getpid())
  
if __name__ == "__main__":
  ApplicationInstance()
  while True:
    print 'running...'
    time.sleep(1)

#4. Detect specific folders or files

#!/usr/bin/env python
#coding: utf-8
import time, commands, signal, sys
  
def sig_handler(sig, frame):
  if os.path.exists("/tmp/test"):
    os.rmdir("/tmp/test")
  sys.exit(0)
  
def ApplicationInstance():
  signal.signal(signal.SIGTERM, sig_handler)
  signal.signal(signal.SIGINT, sig_handler)
  signal.signal(signal.SIGQUIT, sig_handler)
  if commands.getstatusoutput("mkdir /tmp/test")[0]:
    print "instance is running..."
    sys.exit(0)
  
if __name__ == "__main__":
  ApplicationInstance()
  while True:
    print 'running...'
    time.sleep(1)

You can also detect a specific file to determine whether the file exists:

import os
import os.path
import time
   
   
#class used to handle one application instance mechanism
class ApplicationInstance:
   
  #specify the file used to save the application instance pid
  def __init__( self, pid_file ):
    self.pid_file = pid_file
    self.check()
    self.startApplication()
   
  #check if the current application is already running
  def check( self ):
    #check if the pidfile exists
    if not os.path.isfile( self.pid_file ):
      return
    #read the pid from the file
    pid = 0
    try:
      file = open( self.pid_file, 'rt' )
      data = file.read()
      file.close()
      pid = int( data )
    except:
      pass
    #check if the process with specified by pid exists
    if 0 == pid:
      return
   
    try:
      os.kill( pid, 0 )  #this will raise an exception if the pid is not valid
    except:
      return
   
    #exit the application
    print "The application is already running..."
    exit(0) #exit raise an exception so don't put it in a try/except block
   
  #called when the single instance starts to save it's pid
  def startApplication( self ):
    file = open( self.pid_file, 'wt' )
    file.write( str( os.getpid() ) )
    file.close()
   
  #called when the single instance exit ( remove pid file )
  def exitApplication( self ):
    try:
      os.remove( self.pid_file )
    except:
      pass
   
   
if __name__ == '__main__':
  #create application instance
  appInstance = ApplicationInstance( '/tmp/myapp.pid' )
   
  #do something here
  print "Start MyApp"
  time.sleep(5)  #sleep 5 seconds
  print "End MyApp"
   
  #remove pid file
  appInstance.exitApplication()

The above os.kill (pid, 0) is used to detect whether a process with pid is still alive. If the process of pid has stopped, an exception will be thrown. If it is running, the kill signal will not be sent.

5. The socket listens to a specific port

#!/usr/bin/env python
#coding: utf-8
import socket, time, sys
  
  
def ApplicationInstance():
  try:  
    global s
    s = socket.socket()
    host = socket.gethostname()
    s.bind((host, 60123))
  except:
    print "instance is running..."
    sys.exit(0)
  
if __name__ == "__main__":
  ApplicationInstance()
  while True:
    print 'running...'
    time.sleep(1)

This function can be implemented using a decorator for easy reuse (the effect is the same as above Same):

#!/usr/bin/env python
#coding: utf-8
import socket, time, sys
import functools
  
#使用装饰器实现
def ApplicationInstance(func):
  @functools.wraps(func)
  def fun(*args,**kwargs):
    import socket
    try:
      global s
      s = socket.socket()
      host = socket.gethostname()
      s.bind((host, 60123))
    except:
      print('already has an instance...')
      return None
    return func(*args,**kwargs)
  return fun
  
@ApplicationInstance
def main():
  while True:
    print 'running...'
    time.sleep(1)
  
if __name__ == "__main__":
  main()

4. Summary

(1) Daemon process and single script running are more important in practical applications. There are also many methods. You can choose the appropriate ones to modify. You can make them into a separate class or template, and then subclass them to implement customization.

(2) The automatic recovery of the daemon monitoring process avoids the use of nohup and &, and combined with the shell script can save a lot of trouble of starting and hanging the server from time to time.

The above is the entire content of this article. I hope it will be helpful to everyone's learning. I also hope that everyone will support the PHP Chinese website.

For more detailed explanations of Python daemons and script singleton operations, please pay attention to the PHP Chinese website!