Summary of writing back-end running programs in PHP

1. Scheduled execution

It’s crontab, Linux command, how to use it, google it yourself. I just want to say that you need to pay attention to permissions when using crontab. Operation and maintenance often use root to start, resulting in some files that web users do not have permissions for.

2. Background guard

It is divided into two steps: 1. You need to write an infinite loop in the script, because php is not like python. I usually do {....} while (true); but generally add a sleep, otherwise the machine will be killed; 2. When starting the process, you need to add an "&" at the end. You can also google this. If you need to record the output information, you need to write php xxxx.php > /tmp/phplog & , so that the program information is recorded in the file, making it easier to troubleshoot problems later.

3. Operation monitoring

The background daemon process is started, but you cannot be 100% sure that your program will not display warning messages. Once these messages appear, PHP will terminate the current process. At this time, the background program will directly quit. So in addition to the program that handles things normally, you also need a program that checks the running status of that program. I usually call it xxxxDefend.php. An example of this program is as follows

#!/usr/local/php5/bin/php
php

//Startup command
$action = '/usr/local/php5/bin/php xxxxxx.php';
$logPath = '/tmp/logs/';

do {
$result = array ();
exec("ps aux | grep 'xxxxxx.php'", $result);

$isOk = 0;
foreach ($result as $v) {
$is = strpos($v, $action );
if (false !== $is) {
$isOk++;
}
}

$exec = $action . " > " . $logPath . "xxxxxx_log &";
for ($i=1; $i exec($exec);
}
sleep(5);
} while (true);

Start this program in the background as well, and it will Every 5 seconds, the ps command will be used to check whether the working program exists. If it is not there or there are less than 5, it will start up to 5. Of course, this work can also be left to operation and maintenance using the shell, but when the operation and maintenance capabilities are insufficient, you have to do it yourself.

4. Multi-process

In order to increase efficiency, the working PHP will usually start multiple processes at the same time, or even run on multiple machines. At this time, we must consider the problem of multiple processes processing the same data at the same time. At this time, I usually make a queue for the tasks (usually using redis, which has pretty good performance. How to do it, you can google it), and then the working program pops out a record to be worked on each time. For example, if you have a large Files need to be processed. At this time, I usually process the files and put them one by one into the redis list, so that the working program can be popped by multiple programs at the same time and executed in parallel without duplication. If you really don’t have redis, you can use mysql to build an innodb table. Before the program processes the data, be sure to add a read lock to the data to be processed, and then add a mark after processing, or directly delete the data. This can also be achieved The problem of non-duplication in multiple processes.

5. Log

The background program will usually keep running, and basically no one will care about it unless something goes wrong, so the log is extremely important, because once something goes wrong, you need to rely on the log to find the cause, unlike the front-end program. echo to see where the error is. Don’t be afraid of logging too much or wasting space. Hard drives are worthless, but a bug may directly affect your income. I usually keep logs like this

[Machine IP] [Process pid] [Time] [Current program file name] [Number of file lines] [Necessary parameters and information] [Others]

These are what you can think of in a normal program Problem, usually a try catch is added to the outermost layer of the program, so that most exceptions can be caught and then recorded (the warning catch is not available, which is quite depressing)

6. Performance optimization

A background like this Generally, the operation and maintenance of programs will be done separately for the machine. At this time, you need to do a stress test to see how many processes the machine can run. This is usually done when the processor is busy processing tasks, looking at the machine's CPU, memory, network, and For the usage of the hard disk, it is best to reach the maximum value at the same time, so that your machine is not wasted. If the hard disk usage is very high and the others are very low, you need to optimize the program. In this case, it is usually the case of reading and The written data is temporarily stored in the memory for a period of time, and then written to the hard disk at once; if the CPU is extremely high, it means that your algorithm is too messy, so optimize it; memory and network generally will not become bottlenecks, and PHP will not be used much. Memory and the server must have at least a gigabit network card. These two items are generally not bottlenecks. So I usually open a memcache on these machines, haha, no waste.

7. Digression

A question about server close_wait. PHP programmers are generally not very rigorous. Few people will actively close the connection after opening it. For example, when connecting to a database or memcache, many programmers will create a link, operate it, and then the program execution will be completed. If PHP does not actively close the connection, the other party's machine will always wait for the closing operation. What you see on the other party's server is a close_wait state, and a machine can only open more than 60,000 links, especially After the background program started running, the other party's machine was quickly filled up and could not be connected. At this time, both sides need to make some modifications. On the one hand, PHP needs to actively disconnect, and on the other hand, the other party's machine needs to change the default timeout of close_wait to a shorter time (how to change it? Google it yourself). I usually only use memcache. 5 seconds, the database is longer, which is 2 minutes. After this processing, the server's continuity will be greatly increased, and the concurrency capability will also be improved.