简洁的Bash编程技巧 <转>

下面这几条是我自己在写shell代码的时候，比较喜欢的几种写法，抛砖引玉。 1) 检查命令执行是否成功 第一种写法，比较常见： echo abcdee | grep -q abcd if [ $? -eq 0 ] ; then echo "Found" else echo "Not found" fi 简洁的写法： if echo abcdee | grep

下面这几条是我自己在写shell代码的时候，比较喜欢的几种写法，抛砖引玉。

1) 检查命令执行是否成功

第一种写法，比较常见：

<span>echo</span> abcdee <span>|</span> <span>grep</span> <span>-q</span> abcd
 
<span>if</span> <span>[</span> <span>$?</span> <span>-eq</span> <span>0</span> <span>]</span>; <span>then</span>
    <span>echo</span> <span>"Found"</span>
<span>else</span>
    <span>echo</span> <span>"Not found"</span>
<span>fi</span>

简洁的写法：

<span>if</span> <span>echo</span> abcdee <span>|</span> <span>grep</span> <span>-q</span> abc; <span>then</span>
    <span>echo</span> <span>"Found"</span>
<span>else</span>
    <span>echo</span> <span>"Not found"</span>
<span>fi</span>

当然你也可以不要if/else,不过这样可读性比较差:

<span>[</span>Sun Nov 04 05:<span>58</span> AM<span>]</span> <span>[</span>kodango<span>@</span>devops<span>]</span> ~<span>/</span>workspace 
$ <span>echo</span> abcdee <span>|</span> <span>grep</span> <span>-q</span> abc <span>&&</span> <span>echo</span> <span>"Found"</span> <span>||</span> <span>echo</span> <span>"Not found"</span>
Found

2) 将标准输出与标准错误输出重定向到/dev/null

第一种写法，比较常见：

<span>grep</span> <span>"abc"</span> test.txt <span>1</span><span>>/</span>dev<span>/</span>null <span>2</span><span>>&</span><span>1</span>

常见的错误写法：

<span>grep</span> <span>"abc"</span> test.txt <span>2</span><span>>&</span><span>1</span> <span>1</span><span>>/</span>dev<span>/</span>null

简洁的写法：

<span>grep</span> <span>"abc"</span> test.txt <span>&></span> <span>/</span>dev<span>/</span>null

3) awk的使用

举一个实际的例子，获取Xen DomU的id。

常见的写法：

<span>sudo</span> xm li <span>|</span> <span>grep</span> vm_name <span>|</span> <span>awk</span> <span>'{print $2}'</span>

简洁的写法：

<span>sudo</span> xm li <span>|</span> <span>awk</span> <span>'/vm_name/{print $2}'</span>

4) 将一个文本的所有行用逗号连接起来

假设文件内容如下所示:

<span>[</span>Sat Nov 03 <span>10</span>:04 PM<span>]</span> <span>[</span>kodango<span>@</span>devops<span>]</span> ~<span>/</span>workspace 
$ <span>cat</span> <span>/</span>tmp<span>/</span>test.txt 
<span>1</span>
<span>2</span>
<span>3</span>

使用Sed命令：

<span>[</span>Sat Nov 03 <span>10</span>:<span>14</span> PM<span>]</span> <span>[</span>kodango<span>@</span>devops<span>]</span> ~<span>/</span>workspace 
$ <span>sed</span> <span>':a;$!N;s/\n/,/;ta'</span> <span>/</span>tmp<span>/</span>test.txt 
<span>1</span>,<span>2</span>,<span>3</span>

简洁的写法：

<span>[</span>Sat Nov 03 <span>10</span>:04 PM<span>]</span> <span>[</span>kodango<span>@</span>devops<span>]</span> ~<span>/</span>workspace 
$ paste -sd, <span>/</span>tmp<span>/</span>test.txt 
<span>1</span>,<span>2</span>,<span>3</span>

5) 过滤重复行

假设文件内容如下所示：

<span>[</span>Sat Nov 03 <span>10</span>:<span>16</span> PM<span>]</span> <span>[</span>kodango<span>@</span>devops<span>]</span> ~<span>/</span>workspace 
$ <span>sort</span> <span>/</span>tmp<span>/</span>test.txt 
<span>1</span>
<span>1</span>
<span>2</span>
<span>3</span>

常用的方法：

<span>[</span>Sat Nov 03 <span>10</span>:<span>16</span> PM<span>]</span> <span>[</span>kodango<span>@</span>devops<span>]</span> ~<span>/</span>workspace 
$ <span>sort</span> <span>/</span>tmp<span>/</span>test.txt <span>|</span> <span>uniq</span>
<span>1</span>
<span>2</span>
<span>3</span>

简单的写法：

<span>[</span>Sat Nov 03 <span>10</span>:<span>16</span> PM<span>]</span> <span>[</span>kodango<span>@</span>devops<span>]</span> ~<span>/</span>workspace 
$ <span>sort</span> <span>/</span>tmp<span>/</span>test.txt <span>-u</span>
<span>1</span>
<span>2</span>
<span>3</span>

6) grep查找单词

假设一个文本的每一行是一个ip地址，例如

<span>[</span>Sat Nov 03 <span>10</span>:<span>20</span> PM<span>]</span> <span>[</span>kodango<span>@</span>devops<span>]</span> ~<span>/</span>workspace 
$ <span>cat</span> <span>/</span>tmp<span>/</span>ip.list 
10.0.0.1
10.0.0.12
10.0.0.123

使用grep查找是否包括10.0.0.1这个ip地址。常见的写法：

<span>[</span>Sat Nov 03 <span>10</span>:<span>22</span> PM<span>]</span> <span>[</span>kodango<span>@</span>devops<span>]</span> ~<span>/</span>workspace 
$ <span>grep</span> <span>'10.0.0.1\>'</span> <span>/</span>tmp<span>/</span>ip.list 
10.0.0.1

简单的方法（其实这方法不见得简单，只是为了说明-w这个参数还是很有用的)

<span>[</span>Sat Nov 03 <span>10</span>:<span>23</span> PM<span>]</span> <span>[</span>kodango<span>@</span>devops<span>]</span> ~<span>/</span>workspace 
$ <span>grep</span> <span>-w</span> <span>'10.0.0.1'</span> <span>/</span>tmp<span>/</span>ip.list 
10.0.0.1

顺便grep的-n/-H/-v/-f/-c这几参数都很有用。

7) 临时设置环境变量

常见的写法：

<span>[</span>Sat Nov 03 <span>10</span>:<span>26</span> PM<span>]</span> <span>[</span>kodango<span>@</span>devops<span>]</span> ~<span>/</span>workspace 
$ <span>export</span> <span>LC_ALL</span>=zh_CN.UTF-<span>8</span> 
 
<span>[</span>六 <span>11</span>月 03 <span>10</span>:<span>26</span> 下午<span>]</span> <span>[</span>kodango<span>@</span>devops<span>]</span> ~<span>/</span>workspace 
$ <span>date</span>
<span>2012</span>年 <span>11</span>月 03日 星期六 <span>22</span>:<span>26</span>:<span>55</span> CST

简洁的写法:

<span>[</span>六 <span>11</span>月 03 <span>10</span>:<span>26</span> 下午<span>]</span> <span>[</span>kodango<span>@</span>devops<span>]</span> ~<span>/</span>workspace 
$ <span>unset</span> LC_ALL
 
<span>[</span>Sat Nov 03 <span>10</span>:<span>27</span> PM<span>]</span> <span>[</span>kodango<span>@</span>devops<span>]</span> ~<span>/</span>workspace 
$ <span>LC_ALL</span>=zh_CN.UTF-<span>8</span> <span>date</span> 
<span>2012</span>年 <span>11</span>月 03日 星期六 <span>22</span>:<span>27</span>:<span>43</span> CST

在命令之前加上环境变更的设置，只是临时改变当前执行命令的环境。

8) $1,$2...等位置参数的使用

假设只想使用$2,$3..这几个参数，常见的做法是:

<span>shift</span>
<span>echo</span> <span>"$@"</span>

为什么不这样写呢？

<span>echo</span> <span>"<span>${@:2}</span>"</span>

9）退而求其次的写法

相信大家会有这种需求，当一个参数值没有提供时，可以使用默认值。常见的写法是：

<span>arg</span>=<span>$1</span>
 
<span>if</span> <span>[</span> <span>-z</span> <span>"<span>$arg</span>"</span> <span>]</span>; <span>then</span>
   <span>arg</span>=<span>0</span>
<span>fi</span>

简洁的写法是这样的:

<span>arg</span>=<span>${1:-0}</span>

10）bash特殊参数--的用法

假设要用grep查找字符串中是否包含-i，我们会这样尝试：

<span>[</span>Sat Nov 03 <span>10</span>:<span>45</span> PM<span>]</span> <span>[</span>kodango<span>@</span>devops<span>]</span> ~<span>/</span>workspace 
$ <span>echo</span> <span>'abc-i'</span> <span>|</span> <span>grep</span> <span>"-i"</span>
Usage: <span>grep</span> <span>[</span>OPTION<span>]</span>... PATTERN <span>[</span>FILE<span>]</span>...
Try <span>'grep --help'</span> <span>for</span> <span>more</span> information.
 
<span>[</span>Sat Nov 03 <span>10</span>:<span>45</span> PM<span>]</span> <span>[</span>kodango<span>@</span>devops<span>]</span> ~<span>/</span>workspace 
$ <span>echo</span> <span>'abc-i'</span> <span>|</span> <span>grep</span> <span>"\-i"</span>
abc-i

简洁的方法是：

<span>[</span>Sat Nov 03 <span>10</span>:<span>45</span> PM<span>]</span> <span>[</span>kodango<span>@</span>devops<span>]</span> ~<span>/</span>workspace 
$ <span>echo</span> <span>'abc-i'</span> <span>|</span> <span>grep</span> <span>--</span> <span>-i</span>
abc-i

bash中--后面的参数不会被当作选项解析。

11）函数的返回值默认是最后一行语句的返回值

<span># Check whether an item is a function</span>
<span># $1: the function name</span>
<span># Return: 0(yes) or 1(no)</span>
<span>function</span> is_function<span>(</span><span>)</span>
<span>{</span>
    <span>local</span> <span>func_name</span>=<span>$1</span>
    <span>test</span> <span>"<span>`type -t $1 2>/dev/null`</span>"</span> = <span>"function"</span>
<span>}</span>

不要画蛇添足再在后面加一行return $?了。

12) 将printf格式化的结果赋值给变量

例如将数字转换成其十六进制形式，常见的写法是：

<span>[</span>Sat Nov 03 <span>10</span>:<span>55</span> PM<span>]</span> <span>[</span>kodango<span>@</span>devops<span>]</span> ~<span>/</span>workspace 
$ <span>var</span>=$<span>(</span><span>printf</span> <span>'%%%02x'</span> <span>111</span><span>)</span>

简单的写法是：

<span>[</span>Sat Nov 03 <span>10</span>:<span>54</span> PM<span>]</span> <span>[</span>kodango<span>@</span>devops<span>]</span> ~<span>/</span>workspace 
$ <span>printf</span> <span>-v</span> var <span>'%%%02x'</span> <span>111</span>

看看printf的help

<span>[</span>Sat Nov 03 <span>10</span>:<span>53</span> PM<span>]</span> <span>[</span>kodango<span>@</span>devops<span>]</span> ~<span>/</span>workspace 
$ <span>help</span> <span>printf</span> <span>|</span> <span>grep</span> <span>-A</span> <span>1</span> <span>-B</span> <span>1</span> <span>--</span> <span>-v</span>
printf: <span>printf</span> <span>[</span>-v var<span>]</span> format <span>[</span>arguments<span>]</span>
    Formats and prints ARGUMENTS under control of the FORMAT.
<span>--</span>
    Options:
      <span>-v</span> var	assign the output to shell variable VAR rather than
    		display it on the standard output

13）打印文件行

打印文件的第一行：

<span>head</span> <span>-1</span> test.txt

打印文件的第2行：

<span>sed</span> <span>-n</span> <span>'2p'</span> test.txt

打印文件的第2到5行：

<span>sed</span> <span>-n</span> <span>'2,5p'</span> test.txt

打印文件的第2行始（包括第2行在内）5行的内容：

<span>sed</span> <span>-n</span> <span>'2,+4p'</span> test.txt

打印倒数第二行：

$ <span>tail</span> <span>-2</span> test.txt <span>|</span> <span>head</span> <span>-1</span>
$ <span>tac</span> test.txt <span>|</span> <span>sed</span> <span>-n</span> <span>'2p'</span>

14）善用let或者(())命令做算术运算

如何对一个数字做++运算，可能你会这样用：

<span>a</span>=<span>1</span>
<span>a</span>=<span>`</span><span>expr</span> a + <span>1</span><span>`</span>

为何不用你熟悉的:

<span>a</span>=<span>1</span>
<span>let</span> a++
<span>let</span> a+=<span>2</span>

15）获取软连接指定的真实文件名

如果你不知道，你可能会这样获取：

<span>[</span>Sat Nov 03 <span>11</span>:<span>12</span> PM<span>]</span> <span>[</span>kodango<span>@</span>devops<span>]</span> ~<span>/</span>workspace 
$ <span>ls</span> <span>-l</span> <span>/</span>usr<span>/</span>bin<span>/</span>python <span>|</span> <span>awk</span> <span>-F</span><span>'->'</span> <span>'{print $2}'</span> <span>|</span> <span>tr</span> <span>-d</span> <span>' '</span>
<span>/</span>usr<span>/</span>bin<span>/</span>python2

如果你知道有一个叫readlink的命令，那么：

<span>[</span>Sat Nov 03 <span>11</span>:<span>13</span> PM<span>]</span> <span>[</span>kodango<span>@</span>devops<span>]</span> ~<span>/</span>workspace 
$ <span>readlink</span> <span>/</span>usr<span>/</span>bin<span>/</span>python
<span>/</span>usr<span>/</span>bin<span>/</span>python2

16）获取一个字符的ASCII码

<span>[</span>Sat Nov 03 <span>11</span>:<span>14</span> PM<span>]</span> <span>[</span>kodango<span>@</span>devops<span>]</span> ~<span>/</span>workspace 
$ <span>printf</span> <span>'%02x'</span> <span>"'+"</span>
2b
<span>[</span>Sat Nov 03 <span>11</span>:<span>30</span> PM<span>]</span> <span>[</span>kodango<span>@</span>devops<span>]</span> ~<span>/</span>workspace 
$ <span>echo</span> <span>-n</span> <span>'+'</span> <span>|</span> <span>od</span> <span>-tx1</span> <span>-An</span> <span>|</span> <span>tr</span> <span>-d</span> <span>' '</span>
2b

17）清空一个文件

常见的用法:

<span>echo</span> <span>""</span> <span>></span> test.txt

简单的写法：

<span>></span> test.txt

18) 不要忘记有here document

下面一段代码：

<span>grep</span> <span>-v</span> <span>1</span> <span>/</span>tmp<span>/</span>test.txt <span>|</span> <span>while</span> <span>read</span> line; <span>do</span>
    <span>let</span> a++
    <span>echo</span> --<span>$line</span>--
<span>done</span>
 
<span>echo</span> a:<span>$a</span>

执行后有什么问题吗？

<span>[</span>Sun Nov 04 05:<span>35</span> AM<span>]</span> <span>[</span>kodango<span>@</span>devops<span>]</span> ~<span>/</span>workspace 
$ <span>sh</span> test.sh 
<span>--2--</span>
<span>--3--</span>
a:

发现a这个变量没有被赋值，为什么呢？因为管道后面的代码是在在一个子shell中执行的，所做的任何更改都不会对当前shell有影响，自然a这个变量就不会有赋值了。

换一种思路，可以这样做：

<span>grep</span> <span>-v</span> <span>1</span> <span>/</span>tmp<span>/</span>test.txt <span>></span> <span>/</span>tmp<span>/</span>test.tmp
 
<span>while</span> <span>read</span> line; <span>do</span>
    <span>let</span> a++
    <span>echo</span> --<span>$line</span>--
<span>done</span> <span> <span>/</span>tmp<span>/</span>test.tmp
 
<span>echo</span> a:<span>$a</span>
<span>rm</span> <span>-f</span> <span>/</span>tmp<span>/</span>test.tmp</span>

不过多了一个临时文件，最后还要删除。这里其实可以用到here document：

<span>b</span>=<span>1</span>
<span>while</span> <span>read</span> line2; <span>do</span>
    <span>let</span> b++
    <span>echo</span> ??<span>$line2</span>??
<span>done</span> <span> <span> EOF
<span>`</span><span>grep</span> <span>-v</span> <span>1</span> <span>/</span>tmp<span>/</span>test.txt<span>`</span>
EOF
 
<span>echo</span> b: <span>$b</span></span></span>

here document往往用于需要输出一大段文本的地方，例如脚本的help函数。

19）删除字符串中的第一个或者最后一个字符

假设字符串为：

<span>[</span>Sun Nov 04 <span>10</span>:<span>21</span> AM<span>]</span> <span>[</span>kodango<span>@</span>devops<span>]</span> ~<span>/</span>workspace 
$ <span>str</span>=<span>"aremoveb"</span>

可能你第一个想法是通过sed或者其它命令来完成这个功能，但是其实有很简单的方法：

<span>[</span>Sun Nov 04 <span>10</span>:<span>24</span> AM<span>]</span> <span>[</span>kodango<span>@</span>devops<span>]</span> ~<span>/</span>workspace 
$ <span>echo</span> <span>"<span>${str#?}</span>"</span>
removeb
 
<span>[</span>Sun Nov 04 <span>10</span>:<span>24</span> AM<span>]</span> <span>[</span>kodango<span>@</span>devops<span>]</span> ~<span>/</span>workspace 
$ <span>echo</span> <span>"<span>${str%?}</span>"</span>
aremove

类似地，你也可以删除2个、3个、4个……

有没有一次性删除第一个和最后一个字符的方法呢？答案当然是肯定的：

<span>[</span>Sun Nov 04 <span>10</span>:<span>26</span> AM<span>]</span> <span>[</span>kodango<span>@</span>devops<span>]</span> ~<span>/</span>workspace 
$ <span>echo</span> <span>"<span>${str:1:-1}</span>"</span>
remove

关于这些变量替换的内容在bash的man手册中都有说明。

20）使用逗号join数组元素

假设数组元素没有空格，可以用这种方法：

<span>[</span>Sun Nov 04 <span>10</span>:<span>14</span> AM<span>]</span> <span>[</span>kodango<span>@</span>devops<span>]</span> ~<span>/</span>workspace 
$ <span>a</span>=<span>(</span><span>1</span> <span>2</span> <span>3</span><span>)</span> 
$ <span>b</span>=<span>"<span>${a[*]}</span>"</span>
 
<span>[</span>Sun Nov 04 <span>10</span>:<span>15</span> AM<span>]</span> <span>[</span>kodango<span>@</span>devops<span>]</span> ~<span>/</span>workspace 
$ <span>echo</span> <span>${b// /,}</span>
<span>1</span>,<span>2</span>,<span>3</span>

注意：当该数组的长度非常长时，使用这种替换的时间开销很高，性能很差，推荐用sed。

假设数组元素包含有空格，可以借用printf命令来达到：

<span>[</span>Sun Nov 04 <span>10</span>:<span>15</span> AM<span>]</span> <span>[</span>kodango<span>@</span>devops<span>]</span> ~<span>/</span>workspace 
$ <span>a</span>=<span>(</span><span>1</span> <span>"2 3"</span> <span>4</span><span>)</span>
 
<span>[</span>Sun Nov 04 <span>10</span>:<span>15</span> AM<span>]</span> <span>[</span>kodango<span>@</span>devops<span>]</span> ~<span>/</span>workspace 
$ <span>printf</span> <span>",%s"</span> <span>"<span>${a[@]}</span>"</span> <span>|</span> <span>cut</span> <span>-c2-</span>   
<span>1</span>,<span>2</span> <span>3</span>,<span>4</span>

21) Shell中的多进程

在命令行下，我们会在命令行后面加上&符号来让该命令在后台执行，在shell脚本中，使用"(cmd)"可以让fork一个子shell来执行该命令。利用这两点，可以实现shell的多线程：

<span>job_num</span>=<span>10</span>
 
<span>function</span> do_work<span>(</span><span>)</span>
<span>{</span>
    <span>echo</span> <span>"Do work.."</span>
<span>}</span>
 
<span>for</span> <span>(</span><span>(</span><span>i</span>=<span>0</span>; i<span>job_num ;i++<span>)</span><span>)</span>; <span>do</span>
    <span>echo</span> <span>"Fork job <span>$i</span>"</span>
    <span>(</span>do_work<span>)</span> <span>&</span>
<span>done</span>
 
<span>wait</span>   <span># wait for all job done</span>
<span>echo</span> <span>"All job have been done!"</span></span>

注意最后的wait命令，作用是等待所有子进程结束。

附几则小技巧：
1）sudo iptables -L -n | vim -
2）grep -v xxx | vim -
3）echo $'\''
4）set -- 1 2 3; echo "$@"
5）搜索stackoverflow/superuser等站点

 

 

http://kodango.me/simple-bash-programming-skills