Detailed explanation of the use of regex and Linux's three major text processing tools

This time I will bring you a detailed explanation of the use of the three major text processing tools of Regex and Linux. What are the precautions when using the three major text processing tools of Regex and Linux. The following is a practical case. Let’s take a look. one time.

grep, sed and awk are all text processing tools. Although they are all text processing tools, they all have their own advantages and disadvantages. One text processing command cannot be completely replaced by another, otherwise it will not Three text processing commands appear.

1. Regular expression

1. Type of matching characters

[a-z]: Lowercase letters

[A-Z]: Uppercase letters

[a-Z]: Small or uppercase letters

[0-9]: Numbers

[a-zA-Z0-9]: Matches a character that is a letter or number

. : Matches any character, except spaces

[0-f] : Hexadecimal number

abc | def: abc or def
a (bc | de) f: abcf or adef

\<: The first word is usually preceded by a space or Special characters are used as separators, and continuous strings are treated as words

\>: end of words

[^expression]: except lowercase letters All characters except , and so on.

2, followed by the following symbols to control the number of matches

The left side of such symbols must have the expression of the first point above

expression *: 0 or n characters

expression +: 1 or n characters

expression?: 0 or 1 character

expression {n}: n characters

Expression{n:m}:n to m characters

Expression{n,}:at least n characters

[Example][a-z] * means matching 0 or more lowercase letters

3. Control the matching characters at the beginning and end
^ Expression: The head matches

Expression$: The tail matches

2. Three major text processing tools in Linux

1. egrep filtering tool

Extension of grep version, you can use regular expressions

Syntax:

egrep -option 'regular expression' file name

Options:

-n: display lines No.
-o: Only display matching content
-q: Silent mode, no output, you have to use $? to determine whether the execution is successful, that is, whether the desired content is filtered
-l: If the match is successful, only the file name will be printed. If it fails, it will not be printed. Usually -rl is used together. grep -rl 'root' /etc
-A: If the match is successful, the matching line and the following n lines will be printed. Print them out together
-B: If the match is successful, print out the matching line and the n lines before and after it.
-C: If the match is successful, print out the matching line and the n lines before and after it.
--color
-c: If the match is successful, print out the number of matched lines
-i: Ignore case
-v: Negate, do not match
-w: Match words

2. sed streamEditor

Syntax:

Syntax 1: sed - option 'Number positioning + command' File name

Options:

-n: Silent mode, no output
-e: Multiple edits, this is not very clear
-i: Directly modify the file content, and Not output
-r: Extended mode, you can use regular expressions
-f: Specify the file name and write the action in a new file

Command:

a: Append,
c: Change,
d: Delete,
i: Insert, i can be followed by strings, and these strings will appear in a new line (the current previous line )
p: Print print
s: Replace substitute, you can directly perform the replacement work. Usually this s action can be paired with a regular expression. For example, 1,20s/old/new/g

*s command special instructions:

Use {Command 1: Command 2: Command 3} to add multiple commands

语法2：sed  -r  '替换命令s/正则表达式/替换内容/贪婪选项g'  文件名

定位的两种方法：

①数字定位（输入行序号定位）

十进制数
1：单行
1,3：范围 从第一行到第三行
2,+4：匹配行后若干行
4,~3：从第四行到下一个3的倍数行
2~3：第二行起每间隔三行的行
$：尾行
1!：除了第一行以外的行

【例】sed -n '1p' /etc/passwd

②正则表达式定位

正则必须用//包裹起来

扩展正则需要用 -r 参数或转义

替换可使用正则表达式的子模式，即小括号（），可以\1、\2代表子模式

【例】sed -r 's/(.)(.)/\2\1/ file1 表示将匹配到的第一部分和第二部分替换

*贪婪选项：填上g，代表把一行中所有匹配项替换

3、awk 文本分析工具

由命令、正则（需要用//包围起来）、比较和关系运算组合而成

使用option中的-F参数定义间隔符号

用$1,$2,$3等的顺序表示files中每行以间隔符号分隔的各列不同域，NF变量表示当前记录的字段数

语法

awk -选项 参数  '逻辑判断{命令 变量1,变量2,变量3}' 文件名

选项

-F 定义字段分隔符,默认的分隔符是连续的空格或制表符
-v 定义变量并赋值 也可以借用次方式从shell变量中引入

AWK变量

NR 当前记录的个数(全部文件连接后的统计)
FNR 当前记录的个数(仅为当前文件的统计,非全部)
FS 字段分隔符 默认为连续空格或制表符,可以使用多个不同的符号做分隔符 -F[:/]
OFS 输出字符的分隔符 默认是空格
# awk -F: 'OFS="=====" {print $1,$2}' /etc/passwd
root=====x
NF 当前读入行的字段个数
ORS 输出记录分隔符 默认是换行
# awk -F: 'ORS="=====" {print $1,$2}' /etc/passwd
root x=====bin x=====
FILENAME 当前文件名

【例1】使用AWK变量

# awk '{print NR,FNR,$1}' file1 file2 
1 1 aaaaa
2 2 bbbbb
3 3 ccccc
4 1 dddddd
5 2 eeeeee
6 3 ffffff
#

【例2】引用shell变量的方法

# a=root
# awk -v var=$a -F: '$1 == var {print $0}' /etc/passwd
或者 把整个命令拆开传递,让shell变量外露,
# awk -F: '$1 == "'$a'" {print $0}' /etc/passwd
# a=NF
# awk -F: '{print $'$a'}' /etc/passwd

逻辑运算（可直接引用域进行运算）

= += -= /= *= 赋值

&& || ! 逻辑与 逻辑或 逻辑非

~ !~  匹配正则或不匹配,正则需要用 /正则/ 包围住
< <= > >= != ==  关系 比较字符串时要把字符串用双引号引起来

$ 字段引用：字段引用需要加$,而变量引用直接用变量名取

+ - * / % ++ --  运算符

转义序列

\\ \自身
\$ 转义$
\t 制表符
\b 退格符
\r 回车符
\n 换行符
\c 取消换行

相信看了本文案例你已经掌握了方法，更多精彩请关注php中文网其它相关文章！

推荐阅读：

Linux grep与正则表达式的使用

怎么用正则把字符串分组

The above is the detailed content of Detailed explanation of the use of regex and Linux's three major text processing tools. For more information, please follow other related articles on the PHP Chinese website!