Detailed explanation of xargs command under Linux and the difference between xargs and pipes

Why use xargs, the source of the problem

I often come into contact with the xargs command at work, especially in scripts written by others, but it is easy to get confused with pipelines. This article will explain in detail what the xargs command is, why you should use the xargs command, and the difference with pipes. Why use xargs? We know that the linux command can read the content to be processed from two places, one is through the command line parameters, and the other is the standard input. For example, cat and grep are such commands. For example:

echo 'main' | cat test.cpp

In this case, cat will output the content of test.cpp instead of 'main'String, if test. If cpp does not exist, the cat command will report that the file does not exist and will not attempt to read from standard input. echo 'main' | will import the standard output of echo (that is, the string 'main') into the standard input of cat through a pipeline. That is to say, there is content in the standard input of cat at this time, and its content is the string' main' but in the above content, cat will not read the content to be processed from its standard input. (Note: The standard input has a buffer, just like we use scanffunction to read from the standard input in the program, it actually reads from the standard input buffer). In fact, basically many Linux commands are designed to first obtain the parameters from the command line parameters, and then read them from the standard input, which is reflected in the program. The command line parameters are int main(int argc,char through the main function *argv[]) Function parameters are obtained, and the standard input is read through the standard input function such as scanf in C language. Where they get it is different. For example:

echo 'main' | cat

This command will cause cat to read the content from its standard input and process it, that is, it will output the 'main' string. The echo command directs the contents of its standard output 'main' to the standard output of cat through a pipe.

cat

If you just enter cat and press Enter, the program will wait for input. We need to input the content to be processed from the keyboard to cat. At this time, cat also gets the content to be processed from the standard input, because Our cat command line also does not specify the file name to be processed. Most commands have a parameter - if specified directly at the end of the command - which means reading from the standard input,

For example:

echo 'main' | cat -

This is also feasible and the 'main' character will be displayed String, also enter cat - Enter directly and enter cat directly, the effect is the same, but what if:

echo 'main' | cat test.cpp -

Specify test.cpp and - parameters at the same time, then the cat program will still display test. cpp content. But there is a program with a different strategy, it is grep, for example:

echo 'main' | grep 'main' test.cpp -

The output of this command is:

test.cpp:int main()
(standard input):main

At this time grep will process the standard input and the file test.cpp at the same time The content, that is to say, will be searched for 'main' in the standard input and 'main' will also be searched for in the file test.cpp (the file name is obtained from the grep command line parameter). That is to say, when the two parameters test.cpp and - exist at the same time in the command line, different programs handle them differently. We have seen that cat and grep are processed differently. But one thing is the same: first find the source of the content to be processed on the command line (whether from a file, standard input, or both), if the parameters related to the source of the content to be processed are not found in the command line By default, the content to be processed is read from the standard

input.

In addition, many programs do not process standard input, such as kill and rm. If the content to be processed is not specified in the command line parameters, these programs will not read from standard input by default. Therefore:

echo '516' | kill

This kind of destiny cannot be carried out.

echo 'test' | rm -f

This is also ineffective.

These two commands only accept the processing content specified in the command line parameters and do not obtain the processing content from the standard input. It's normal when you think about it. Kill means to end the process, and rm means delete the file. If the pid of the process to be ended and the name of the file to be deleted need to be read from the standard input, this is also It's weird. But it is natural for word processing tools like cat and grep to read content to be processed from standard input.

But sometimes our scripts need the effect of echo '516' | kill, such as ps -ef | grep 'ddd' | kill, to filter out the process pid that meets certain conditions and then end it. This need is natural and very common for us, so how should we achieve this effect? There are several solutions:

1. In the form of

kill `ps -ef | grep 'ddd'`

, this is actually equivalent to the command obtained by splicing strings, and its effect is similar to kill $pid

2.

for procid in $(ps -aux | grep "some search" | awk '{print $2}'); do kill -9 $procid; done

其实与第一种原理一样，只不过需要多次kill的时候是循环处理的，每次处理一个

3.

 ps -ef | grep 'ddd' | xargs kill

OK，使用了xargs命令，铺垫了这么久终于铺到了主题上。xargs命令可以通过管道接受字符串，并将接收到的字符串通过空格分割成许多参数(默认情况下是通过空格分割) 然后将参数传递给其后面的命令，作为后面命令的命令行参数

xargs是什么，与管道有什么不同

xargs与管道有什么不同呢，这是两个很容易混淆的东西，看了上面的xargs的例子还是有点云里雾里的话，我们来看下面的例子弄清楚为什么需要xargs：

echo '--help' | cat

输出：

--help
echo '--help' | xargs cat

输出：

Usage: cat [OPTION]... [FILE]...
Concatenate FILE(s), or standard input, to standard output.
 -A, --show-all   equivalent to -vET
 -b, --number-nonblank number nonempty output lines
 -e      equivalent to -vE
 -E, --show-ends   display $ at end of each line
 -n, --number    number all output lines
 -s, --squeeze-blank  suppress repeated empty output lines
 -t      equivalent to -vT
 -T, --show-tabs   display TAB characters as ^I
 -u      (ignored)
 -v, --show-nonprinting use ^ and M- notation, except for LFD and TAB
  --help  display this help and exit
  --version output version information and exit

可以看到 echo '--help' | cat   该命令输出的是echo的内容，也就是说将echo的内容当作cat处理的文件内容了，实际上就是echo命令的输出通过管道定向到cat的输入了。然后cat从其标准输入中读取待处理的文本内容。这等价于在test.txt文件中有一行字符 '--help' 然后运行  cat test.txt 的效果。

而 echo '--help' | xargs cat 等价于 cat --help 什么意思呢，就是xargs将其接受的字符串 --help 做成cat的一个命令参数来运行cat命令，同样  echo 'test.c test.cpp' | xargs cat 等价于 cat test.c test.cpp 此时会将test.c和test.cpp的内容都显示出来。

xargs的一些有用的选项

相信到这里应该都知道xargs的作用了，那么我们看看xargs还有一些有用的选项：

1. -d 选项

默认情况下xargs将其标准输入中的内容以空白(包括空格、Tab、回车换行等)分割成多个之后当作命令行参数传递给其后面的命令，并运行之，我们可以使用 -d 命令指定分隔符，例如：

echo '11@22@33' | xargs echo

输出：

11@22@33

默认情况下以空白分割，那么11@22@33这个字符串中没有空白，所以实际上等价于 echo 11@22@33 其中字符串 '11@22@33' 被当作echo命令的一个命令行参数

echo '11@22@33' | xargs -d '@' echo

输出：

11 22 33

指定以@符号分割参数，所以等价于 echo 11 22 33 相当于给echo传递了3个参数，分别是11、22、33

2. -p 选项

使用该选项之后xargs并不会马上执行其后面的命令，而是输出即将要执行的完整的命令(包括命令以及传递给命令的命令行参数)，询问是否执行，输入 y 才继续执行，否则不执行。这种方式可以清楚的看到执行的命令是什么样子，也就是xargs传递给命令的参数是什么，例如：

echo '11@22@33' | xargs -p -d '@' echo

输出：

echo 11 22 33

 ?...y      ==>这里询问是否执行命令 echo 11 22 33 输入y并回车，则显示执行结果，否则不执行

 11 22 33   ==>执行结果

3. -n 选项

该选项表示将xargs生成的命令行参数，每次传递几个参数给其后面的命令执行，例如如果xargs从标准输入中读入内容，然后以分隔符分割之后生成的命令行参数有10个，使用 -n 3 之后表示一次传递给xargs后面的命令是3个参数，因为一共有10个参数，所以要执行4次，才能将参数用完。例如：

echo '11@22@33@44@55@66@77@88@99@00' | xargs -d '@' -n 3 echo

输出结果：

11 22 33
44 55 66
77 88 99
00

等价于：

echo 11 22 33
echo 44 55 66
echo 77 88 99
echo 00

实际上运行了4次，每次传递3个参数，最后还剩一个，就直接传递一个参数。

4. -E 选项，有的系统的xargs版本可能是-e  eof-str

该选项指定一个字符串，当xargs解析出多个命令行参数的时候，如果搜索到-e指定的命令行参数，则只会将-e指定的命令行参数之前的参数(不包括-e指定的这个参数)传递给xargs后面的命令

echo '11 22 33' | xargs -E '33' echo

输出：

11 22

可以看到正常情况下有3个命令行参数 11、22、33 由于使用了-E '33' 表示在将命令行参数 33 之前的参数传递给执行的命令，33本身不传递。等价于 echo 11 22 这里-E实际上有搜索的作用，表示只取xargs读到的命令行参数前面的某些部分给命令执行。

注意：-E只有在xargs不指定-d的时候有效，如果指定了-d则不起作用，而不管-d指定的是什么字符，空格也不行。

echo '11 22 33' | xargs -d ' ' -E '33' echo => 输出 11 22 33
echo '11@22@33@44@55@66@77@88@99@00 aa 33 bb' | xargs -E '33' -d '@' -p echo => 输出 11 22 33 44 55 66 77 88 99 00 aa 33 bb
## -0 选项表示以 '\0' 为分隔符，一般与find结合使用
find . -name "*.txt"

输出：

./2.txt
./3.txt
./1.txt     => 默认情况下find的输出结果是每条记录后面加上换行，也就是每条记录是一个新行
find . -name "*.txt" -print0

输出：

./2.txt./3.txt./1.txt     => 加上 -print0 参数表示find输出的每条结果后面加上 '\0' 而不是换行
find . -name "*.txt" -print0 | xargs -0 echo

输出：

./2.txt ./3.txt ./1.txt
find . -name "*.txt" -print0 | xargs -d '\0' echo

输出：

./2.txt ./3.txt ./1.txt

xargs的 -0 和 -d '\0' 表示其从标准输入中读取的内容使用 '\0' 来分割，由于 find 的结果是使用 '\0' 分隔的，所以xargs使用 '\0' 将 find的结果分隔之后得到3个参数： ./2.txt ./3.txt ./1.txt   注意中间是有空格的。上面的结果就等价于 echo ./2.txt ./3.txt ./1.txt

In fact, it is also possible to use the default whitespace delimiter of xargs find . -name "*.txt" | xargs echo because the newline character is also one of the default whitespace characters of xargs. If the find command does not add -print0, a newline is actually added after each string in the search results.

The above is the detailed content of Detailed explanation of xargs command under Linux and the difference between xargs and pipes. For more information, please follow other related articles on the PHP Chinese website!