Design a code statistics tool using Python

This article mainly introduces relevant information about using Python to design a code statistics tool, including the number of files, the number of code lines, the number of comment lines, and the number of blank lines. Interested friends, please follow the editor of Script House to take a look

Question

Design a program to count the number of projects in a project The number of lines of code, including the number of files, lines of code, number of comment lines, and number of blank lines. Try to be more flexible in the design by inputting different parameters to count items in different languages, for example:

# type用于指定文件类型
python counter.py --type python

Output:

files: 10
code_lines:200
comments:100
blanks:20

Analysis

This is a We can simplify the design problem that looks simple but is a bit complicated to solve. As long as we can correctly count the number of lines of code in a file, it is not a problem to count the number of lines of code in a directory. The most complicated one is about multi-line comments. Taking Python as an example, comment code lines have the following situations:

1. Single-line comments starting with a pound sign

# Single-line comments

2. Multi-line comments When the comment characters are on the same line

"""This is a multi-line comment"""
'''This is also a multi-line comment'''
3. Multi-line comment characters

"""
These 3 lines are all comment symbols
"""

Our idea adopts a line-by-line parsing method. Multi-line comments require an additional identifier in_multi_comment to identify them. Whether the current line is in a multi-line comment, the default is False, when the multi-line comment starts, it is set to True, and when the next multi-line comment is encountered, it is set to False. The code between the start symbol of a multi-line comment and the next end symbol should belong to the comment line.

Knowledge points

How to read files correctly, common methods of string processing when reading files, etc.

Simplified version

We iterate step by step, first implement a simplified version of the program, which only counts a single file of Python code, and does not consider multi-line comments In this case, this is a function that anyone who has started with Python can achieve. The key point is that after reading each line, first use the strip() method to remove the spaces and carriage returns on both sides of the string

# -*- coding: utf-8 -*-
"""
只能统计单行注释的py文件
"""
def parse(path):
 comments = 0
 blanks = 0
 codes = 0
 with open(path, encoding='utf-8') as f:
 for line in f.readlines():
  line = line.strip()
  if line == "":
  blanks += 1
  elif line.startswith("#"):
  comments += 1
  else:
  codes += 1
 return {"comments": comments, "blanks": blanks, "codes": codes}
if __name__ == '__main__':
 print(parse("xxx.py"))

Multi-line comment version

If you can only count the code of single-line comments, it is of little significance. Only by solving the statistics of multi-line comments can it be regarded as a real code statistician

# -*- coding: utf-8 -*-
"""

Can count py files containing multi-line comments

"""
def parse(path):
 in_multi_comment = False # 多行注释符标识符号
 comments = 0
 blanks = 0
 codes = 0
 with open(path, encoding="utf-8") as f:
 for line in f.readlines():
  line = line.strip()
  # 多行注释中的空行当做注释处理
  if line == "" and not in_multi_comment:
  blanks += 1
  # 注释有4种
  # 1. # 井号开头的单行注释
  # 2. 多行注释符在同一行的情况
  # 3. 多行注释符之间的行
  elif line.startswith("#") or \
    (line.startswith('"""') and line.endswith('"""') and len(line)) > 3 or \
   (line.startswith("'''") and line.endswith("'''") and len(line) > 3) or \
   (in_multi_comment and not (line.startswith('"""') or line.startswith("'''"))):
  comments += 1
  # 4. 多行注释符的开始行和结束行
  elif line.startswith('"""') or line.startswith("'''"):
  in_multi_comment = not in_multi_comment
  comments += 1
  else:
  codes += 1
 return {"comments": comments, "blanks": blanks, "codes": codes}
if __name__ == '__main__':
 print(parse("xxx.py"))

The fourth situation above , when encountering multi-line comment symbols, the key operation is to negate the in_multi_comment identifier, instead of simply setting it to False or True. The first time it encounters """, it is True, and the second time it encounters """ It is the end character of the multi-line comment. If it is negated, it is False, and so on. The third time is the beginning, and if it is negated, it is True again.

So how to judge whether other languages ​​need to rewrite a parsing function? If you observe carefully, the four situations of multi-line comments can abstract four judgment conditions, because most languages ​​​​have single-line comments and multi-line comments, but their symbols are different.

CONF = {"py": {"start_comment": ['"""', "'''"], "end_comment": ['"""', "'''"], "single": "#"},
 "java": {"start_comment": ["/*"], "end_comment": ["*/"], "single": "//"}}
start_comment = CONF.get(exstansion).get("start_comment")
end_comment = CONF.get(exstansion).get("end_comment")
cond2 = False
cond3 = False
cond4 = False
for index, item in enumerate(start_comment):
 cond2 = line.startswith(item) and line.endswith(end_comment[index]) and len(line) > len(item)
 if cond2:
 break
for item in end_comment:
 if line.startswith(item):
 cond3 = True
 break
for item in start_comment+end_comment:
 if line.startswith(item):
 cond4 = True
 break
if line == "" and not in_multi_comment:
 blanks += 1
# 注释有4种
# 1. # 井号开头的单行注释
# 2. 多行注释符在同一行的情况
# 3. 多行注释符之间的行
elif line.startswith(CONF.get(exstansion).get("single")) or cond2 or \
 (in_multi_comment and not cond3):
 comments += 1
# 4. 多行注释符分布在多行时，开始行和结束行
elif cond4:
 in_multi_comment = not in_multi_comment
 comments += 1
else:
 codes += 1

Only one configuration constant is needed to mark the symbols of single-line and multi-line comments in all languages, corresponding to cond1 to cond4. It is ok. The remaining task is to parse multiple files, which can be done using the os.walk method.

def counter(path):
 """
 可以统计目录或者某个文件
 :param path:
 :return:
 """
 if os.path.isdir(path):
 comments, blanks, codes = 0, 0, 0
 list_dirs = os.walk(path)
 for root, dirs, files in list_dirs:
  for f in files:
  file_path = os.path.join(root, f)
  stats = parse(file_path)
  comments += stats.get("comments")
  blanks += stats.get("blanks")
  codes += stats.get("codes")
 return {"comments": comments, "blanks": blanks, "codes": codes}
 else:
 return parse(path)

Of course, there is still a lot of work to be done to perfect this program, including command line parsing and parsing only a certain language based on specified parameters. .

Supplement:

Python implementation of code line counting tool

We often want to count The number of lines of code of the project, but if you want to have a more complete statistical function, it may not be that simple. Today we will take a look at how to use python to implement a line of code statistics tool.

Idea:

First get all the files, then count the number of lines of code in each file, and finally add the number of lines.

Functions implemented:

Count the number of lines in each file;
Count the total number of lines;
Count the running time;
Support specified statistical files Type, exclude file types that do not want to be counted;
Recursively count the number of lines of files under the folder including sub-files;

Exclude empty lines;

# coding=utf-8
import os
import time
basedir = '/root/script'
filelists = []
# 指定想要统计的文件类型
whitelist = ['php', 'py']
#遍历文件, 递归遍历文件夹中的所有
def getFile(basedir):
 global filelists
 for parent,dirnames,filenames in os.walk(basedir):
  #for dirname in dirnames:
  # getFile(os.path.join(parent,dirname)) #递归
  for filename in filenames:
   ext = filename.split('.')[-1]
   #只统计指定的文件类型，略过一些log和cache文件
   if ext in whitelist:
    filelists.append(os.path.join(parent,filename))
#统计一个文件的行数
def countLine(fname):
 count = 0
 for file_line in open(fname).xreadlines():
  if file_line != '' and file_line != '\n': #过滤掉空行
   count += 1
 print fname + '----' , count
 return count
if __name__ == '__main__' :
 startTime = time.clock()
 getFile(basedir)
 totalline = 0
 for filelist in filelists:
  totalline = totalline + countLine(filelist)
 print 'total lines:',totalline
 print 'Done! Cost Time: %0.2f second' % (time.clock() - startTime)

Result:

[root@pythontab script]# python countCodeLine.py
/root/script/test /gametest.php---- 16
/root/script/smtp.php---- 284
/root/script/gametest.php---- 16
/root/script/countCodeLine .py---- 33
/root/script/sendmail.php---- 17
/root/script/test/gametest.php---- 16
total lines: 382
Done! Cost Time: 0.00 second
[root@pythontab script]

#Only counts php and python files, which is very convenient.

Related recommendations:

Complete example sharing of Python design calculator function implementation

Visitors and observations in Python design pattern programming User mode example introduction

The above is the detailed content of Design a code statistics tool using Python. For more information, please follow other related articles on the PHP Chinese website!