Home > Article > Backend Development > Web-side PHP code function coverage testing solution_PHP tutorial
Web-side PHP code function coverage testing solution_PHP tutorial
- WBOYOriginal
- 2016-07-12 09:05:13911browse
Web-side PHP code function coverage testing solution
1. About code coverage
There are many levels of measuring code coverage, such as line coverage, function/method coverage, class coverage, branch coverage, etc. Code coverage is also an important criterion for measuring test quality. For black box testing, if you are not sure whether your test case has actually run through every line of code in the system, you will always have to discount the integrity of the test. . Therefore, the industry has its own set of code coverage solutions for almost every programming language. PHP, the most beautiful language in the world, is certainly no exception. PHPUnit and Spike PHPCoverage provide a set of code coverage testing solutions based on xdebug. In this article, I will describe my own solution for PHP code function coverage testing based on the specific business scenarios I encountered.
2. Business background
Suppose we develop a website online and hand it over to business test colleagues for functional testing. So how did they test it? Usually, it is nothing more than developers deploying the website, and then testers try out all the online functions, including some abnormal usage conditions. For business testing, as long as I test all function points and detect all abnormal usage conditions, it is done. But for development, what I'm more curious about is, did you run all the code I wrote? Is there some code that can only be triggered under very special circumstances, and you have never measured these circumstances? At this point, code coverage may be needed to help.
In fact, I first thought of xdebug to test coverage, which only requires two or three functions, as follows:
xdebug_start_code_coverage(); //Start collecting code line coverage
xdebug_get_code_coverage(); //Get the code file name and line number that have been run so far
xdebug_stop_code_coverage(); //Stop collecting code line coverage
The interface provided by xdebug can be used to test line coverage. Does this meet the requirements? In fact, the row coverage granularity is a bit fine. In actual projects, developers may fine-tune the code. For example, in this test, you ran through line 10 of the A.php file, but one day I fine-tuned A.php and added two more lines of code between lines 9 and 10 of A.php. . As a result, the original line 10 became line 12, and the line coverage information of xdebug only recorded the line number... Wouldn't the previous data be inaccurate? . . After careful consideration, I think function coverage is a good granularity. In relatively mature projects, there are rarely large-scale function changes. But the problem is that xdebug does not provide an interface for function coverage.
So, the scene we are encountering now is:
1] I hope to measure a list of all functions covered in a certain test, know how many functions there are in this project, and calculate whether the coverage rate is high enough.
2] After the test is completed, a coverage report should be generated to visualize the code coverage.
3] The complete test process is as follows:
Instrumentation means some preparation work before test execution.
3. Function coverage solution
1) Principle
xdebug inherently provides support for line coverage, and we need to calculate the function coverage ourselves. Function coverage requires two pieces of data, one is which functions are executed, and the other is the total number of functions in the file.
The total number of functions in the file. Since it is impossible for us to execute all functions, this part can only be achieved through static scanning of the code. If it is in C or Java, you may need lexical analysis tools, but in front of the most beautiful language PHP, we don't need to be so complicated at all. Starting from PHP 4.3, PHP Zend Engine has a built-in tokenizer function to help developers perform source code lexical analysis. We only need to find the corresponding lexical rules when defining functions in PHP, and we can easily get all the functions in the specified PHP file.
The interface defined by tokenizer is also very simple:
array token_get_all (string $source)
This function performs file parsing and splits the PHP source code into an array composed of tokens.
string token_name (int $token)
Convert the token in integer form to string form. Similar to the strerror function in C language. With tokenizer, you can design a finite state machine according to the rules and formats defined by PHP functions to complete the analysis of all functions. I wrote a relatively simple version of this part of the code, and took it out separately for your reference only: PHPFunctionParser
Another difficulty in finding function coverage is obtaining the list of executed functions. This place took us on some detours. The simplest method at the beginning is that since we get the executed line through xdebug, we can use the line number to infer which function this line belongs to. However, the amount of line number information obtained by each request is very large. If a request is executed for 1,000 lines, 1,000 judgments will be made, which will be relatively inefficient. After some research, I found that xdebug provides the function trace function, which can obtain the function call relationship in a request. However, I only got the function name, but I couldn't get the file where it is located. So, after some research again, I discovered Reflection. Given the method name and class name, you can deduce which file it is defined in. So we use function trace to temporarily store the function call relationship in a temporary file, and then parse the file to get the executed function name (if it is a class method, it is in the form of "class name::function name"), and then use the reflection mechanism Just export the file that defines this function. I once again realized the power of the most beautiful language in the world.
2) Instrumentation
In order to lower the threshold of use, we change the PHP source code as little as possible. The principle of xdebug collecting information is to call xdebug_start_code_coverage and xdebug_stop_code_coverage respectively to control the start and end of coverage information collection, so it is inevitable to change the source code. Our solution here is to register xdebug_stop_code_coverage through register_shutdown_function as a program that must be run before the PHP program ends (similar to the atexit function in C language), encapsulate it into a file, and then require this file in the first line of the source code That’s it. If your PHP framework is a framework like CodeIgniter where all requests have a unified entry index.php, then you only need to change this one file, and only one line of change to the source code! In fact, basically all current PHP frameworks use an index.php file as the entry point for all requests.
The only change we made to the source code was to add a sentence to the first line of the entry file index.php:
<ol class="dp-j"><li class="alt"><span><span><?php require_once </span><span class="string">"/file/path/to/phpcoverage.php"</span><span>; ?> </span></span></li></ol>
The core code logic of phpcoverage.php is roughly as follows:
<ol class="dp-j"><li class="alt"><span><span><?php </span></span></li><li><span>…… </span></li><li class="alt"><span>function xdebugPhpcoverageBeforeShutdown(){ </span></li><li><span>…… </span></li><li class="alt"><span>$lineCovData = xdebug_get_code_coverage(); </span></li><li><span>xdebug_stop_code_coverage(); </span></li><li class="alt"><span>…… </span></li><li><span>xdebug_stop_trace(); </span></li><li class="alt"><span>…… </span></li><li><span>} </span></li><li class="alt"><span>register_shutdown_function(‘xdebugPhpcoverageBeforeShutdown’); </span></li><li><span>…… </span></li><li class="alt"><span>xdebug_start_trace(……); </span></li><li><span>xdebug_start_code_coverage(); </span></li><li class="alt"><span><span class="comment">//备注：上面省略号表示非关键代码，这里就不展示了</span><span> </span></span></li></ol>
3) Information storage
We have an idea for function coverage testing. We use xdebug's function trace to obtain the calling relationship of all functions in a request, obtain all executed functions, output them to a file, and obtain the executed functions through file parsing and reflection. name and the file where the function is located. Just store this information in a database or file.
When we tried Spike before, we found that the information was stored in files in xml format, and the data redundancy was very high. As a result, after several tests, the files were already very large. This is obviously not what we want to see. Therefore, when storing data, we directly serialize the data in json format and store it in the file in string form, which greatly reduces the file size. At the same time, we use the IP and date of the request source as separation to store different files separately. In this way, the daily request data from each machine can be seen at a glance, which is another step towards "accuracy" and can accurately monitor each request of the tester. The picture below is a screenshot of some data files we collected in business practice:
In this way, every web request from any IP, the lines and function information it covers, will be recorded in the file. For general project testing, only a few testers are using it, so there is no need to consider some performance issues.
4. Report generation
The above explains the principle of generating coverage data, but what we have obtained so far is only data files. How to summarize them into a complete report? This requires us to write a script ourselves to parse the data file just generated. Our approach is to draw lessons from the template of the open source tool spike phpcoverage and add our own code logic, especially the function coverage statistics that this tool does not have. The report generated by the web page we tested ourselves is as follows:
In the figure, you can see the line coverage, function coverage, and total coverage statistics of each file. If you need more accurate data, you can click on the file link to see which lines of code are covered (blue means covered, red means not covered):
5. Summary
When doing web testing in business testing, code coverage is an important indicator to measure test quality. We hope to use this method to be as "accurate" as possible. After the test is executed, we can accurately see which lines of code have been executed and which lines have not been executed. Analyze the reasons why the test cases were not executed to improve the test cases. The process of using the tool is also very simple. Instrumentation=>Testing=>Collecting data=>Producing a report. And this solution minimizes the impact on business code and only requires changing one line of code. Even if something goes wrong in the middle, you can quickly restore the code to its original state. Let testing be at ease and development can also be at ease.
However, the last thing that needs to be emphasized is that covering all the code does not mean that the test is complete. But if it is not covered, it must be incomplete. Therefore, the greatest significance of this solution is to be able to discover some missing codes in the test and find some problems. In fact, it can also help new employees understand the entire project code structure. We can clearly know which code on the server is running for each of our browser requests.
