How to use Linux uniq command

Linux uniq command is used to check and delete repeated rows and columns in text files. It is generally used in conjunction with the sort command.

uniq can check for repeated rows and columns in text files.

Syntax:

uniq [-cdu][-f<栏位>][-s<字符位置>][-w<字符位置>][--help][--version][输入文件][输出文件]

Parameters:

-c or --count Displays duplicate occurrences of the row next to each column number of times.

-d or --repeated displays only repeated rows and columns.

-f or --skip-fields= ignores the fields specified by the comparison.

-s or --skip-chars= Ignores the comparison of the specified characters.

-u or --unique displays rows and columns only once.

-w or --check-chars= specifies the characters to compare.

--help Display help.

--version Display version information.

[Input file] Specify the sorted text file. If this item is not specified, data is read from the standard;

[output file] specifies the output file. If this option is not specified, the content is displayed to the standard output device (display terminal).

Example:

The 2nd, 3rd, 5th, 6th, 7th and 9th lines in the file testfile have the same lines. Use the uniq command to delete duplicate lines. You can use The following command:

uniq

The original content in testfile is:

$ cat testfile      #原有内容  
test 30  
test 30  
test 30  
Hello 95  
Hello 95  
Hello 95  
Hello 95  
Linux 85  
Linux 85

After using the uniq command to delete duplicate lines, the following output results:

$ uniq testfile     #删除重复行后的内容  
test 30  
Hello 95  
Linux 85

Check the file and delete it Lines that appear repeatedly in the file, and the number of times the line appears again is displayed at the beginning of the line. Use the following command:

uniq

The result output is as follows:

$ uniq -c testfile      #删除重复行后的内容  
3 test 30             #前面的数字的意义为该行共出现了3次  
4 Hello 95            #前面的数字的意义为该行共出现了4次  
2 Linux 85

When the repeated lines are not adjacent, the uniq command does not work. That is, if the file content is as follows, the uniq command Doesn’t work:

$ cat testfile1      # 原有内容 
test 30  
Hello 95  
Linux 85 
test 30  
Hello 95  
Linux 85 
test 30  
Hello 95  
Linux 85

Then we can use sort:

$ sort  testfile1 | uniq
Hello 95  
Linux 85 
test 30

Count the number of times each line appears in the file:

$ sort testfile1 | uniq -c
   3 Hello 95  
   3 Linux 85 
   3 test 30

Find duplicate lines in the file :

$ sort testfile1 | uniq -d
Hello 95  
Linux 85 
test 30

The above is the detailed content of How to use Linux uniq command. For more information, please follow other related articles on the PHP Chinese website!