linux - 如何查看python脚本自动退出原因？

Question

使用gunicorn+flask框架，同时通过apscheduler实现对一款爬虫的定时调用，但是在这其中遇到了一个问题，如下：
1，我利用shell终端，通过gunicorn -c直接运行，然后通过flask构造url调用爬虫，并在终端观察输出信息，可以完美调用爬虫。但是当我运行gunicorn -c之后，关闭终端，则爬虫不能完美运行，通过对爬虫调试，发现在运行一段时间后就被终止。
2，之后我在爬虫中使用os.system('nohup python spider.py &'),并且一直打开终端可以完美运行，但是在关闭终端的情况下，又无法执行到底。
所以请问大家怎么来看一下这个脚本运行中为何自动结束。或者说产生的原因是什么。

Answer 1

Your process has not left the terminal. After the terminal is closed, all child processes associated with the terminal will be terminated without any output.
It is recommended to make spaider.py into a daemon process, out of the control of the terminal.

Answer 2

logging

Answer 3

Thank you for your answers. The real reason has been found. It is because too much print output during the debugging process exceeds the cache and causes the process to end.

Answer 4

Although you execute the process in the background, it is not detached from the terminal that started the process. Therefore, the parent process shell terminal is closed and the child process will also be closed.

Solution 1, you can refer to How to make a Python script run like a service or daemon in Linux, which teaches you how to program a process into a daemon process under Linux. The sample program is quite long, so I’ll explain it at the end.

Solution 2, there are python libraries that can daemonize their own processes, and there are quite a few. For example, python-daemon. Usage examples are as follows:

import daemon

def do_main_program():
    pass  # 你的程序

with daemon.DaemonContext():
    do_main_program()
    

Additional solution 1 processing:

# -*- coding: utf-8 -*-
import os
import sys
import datetime

def daemonize(stdin='/dev/null', stdout='/dev/null', stderr='/dev/null'):
    try:
        pid = os.fork()
        if pid > 0:
            sys.exit(0)  # Exit first parent.
    except OSError, e:
        sys.stderr.write("fork #1 failed: (%d) %s\n" % (e.errno, e.strerror))
        sys.exit(1)
    # Decouple from parent environment.
    os.chdir("/")
    os.umask(0)
    os.setsid()
    # Do second fork.
    try:
        pid = os.fork()
        if pid > 0:
            sys.exit(0)  # Exit second parent.
    except OSError, e:
        sys.stderr.write("fork #2 failed: (%d) %s\n" % (e.errno, e.strerror))
        sys.exit(1)
    # Redirect standard file descriptors.
    si = file(stdin, 'r')
    so = file(stdout, 'a+')
    se = file(stderr, 'a+', 0)
    os.dup2(si.fileno(), sys.stdin.fileno())
    os.dup2(so.fileno(), sys.stdout.fileno())
    os.dup2(se.fileno(), sys.stderr.fileno())
    
if __name__ == "__main__":
    daemonize()
    # 然后开始执行你的其他程序。