• shell lint: shellcheck (ShellCheck – Online shell script analyzer)
• Ctrl - R: 向前查询 history
• Ctrl - S: 向后查询 history, 如果不行, 则需运行: stty -ixon. Unable to forward search Bash history similarly as with CTRL-r - Stack Overflow

github 上有 bash-it, 用来增强 bash 功能, 有时间看看. 这个是模仿了 oh-my-zsh — Xiaopei 2013/08/07 22:47

1. What the Unix tools are
   A core set of Unix tools are used over and over again when shell scripting. We cover the basics of the shell and regular expressions, and present each core tool within the context of a particular kind of problem. Besides covering what the tools do, for each tool we show you why it exists and why it has particular options.
2. How to combine the tools to get your job done
   In shell scripting, it really is true that “the whole is greater than the sum of its parts.” By using the shell as “glue” to combine individual tools, you can accomplish some amazing things, with little effort.

1. script 的运行若以 source 来运行时，代表在父程序的 bash 内运行之意！
2. 我们可使用 sh -x script.sh 来进行程序的 debug

#!/bin/bash
 
_gini()
{
COMPREPLY=()
 
local cur
 
cur=$(_get_cword)
 
unset COMP_WORDS[0]
 
case "$cur" in
@)
COMPREPLY=($( compgen -W "$(gini -- ${COMP_WORDS[@]})" -- "$cur" ))
;;
*)
COMPREPLY=( $( compgen -W "$(gini -- ${COMP_WORDS[@]})" -- "$cur" ) )
;;
esac
 
return 0
} &&
complete -F _gini gini

主要参考 Scripting with style [Bash Hackers Wiki]

• Indention: 4个空格
• Breaking up lines: 断行后增加一级缩进

  activate some_very_long_option \
    some_other_option

• Compound commands

  HEAD_KEYWORD parameters; BODY_BEGIN
    BODY_COMMANDS
  BODY_END

  • if/then/elif/else

    if ...; then
      ...
    elif ...; then
      ...
    else
      ...
    fi

  • for

    for f in /etc/*; do
      ...
    done

  • while/until

    while [[ $answer != [YyNn] ]]; do
      ...
    done

  • case

    case $input in
      hello)
        echo "You said hello"
      ;;
      bye)
        echo "You said bye"
        if foo; then
          bar
        fi
      ;;
      *)
        echo "You said something weird..."
      ;;
    esac

• 判断用 [[ ... ]]
• 变量名用小写字母和下划线, 不用大写字母, 若不得不用大写, 加前缀(如 MY_), 以避免与Environment Variables冲突
• 注意变量作用域, 尽量用 local
• 变量必须初始化

  my_input=""
  my_array=()
  my_number=0

• 使用变量/参数时, 用双引号包裹, 除非有不包的理由, 如

  list="one two three"
   
  # you MUST NOT quote $list here
  for word in $list; do
    ...
  done

• 函数名用小写字母, 命名要避免与系统命令类似
• 定义函数用

  getargs() {
    ...
  }

• 使用其他命令的结果(Command substitution)时, 用 “$( … )”, 如

  now="$(date)"

• 整体结构

  #!SHEBANG
  
  CONFIGURATION_VARIABLES
  
  FUNCTION_DEFINITIONS
  
  MAIN_CODE

• Fail early
• Exit meaningfully(0 if everything is okay, non-zero if there was an error)
• write normal output to STDOUT and error, warning and diagnostic messages to STDERR
• if applicable, write a logfile that contains all the details
• always check the input

while true; do
  echo "$(date) $(nc -z baidu.com 80; echo $?)" >> conn2baidu.log
  sleep 2
done

若想通用的 UNIX 工具实现锁, 则应使用 mkdir. 因为 mkdir:

1. create a given directory only if it did not exist before, and set a successful exit code
2. it will set an unsuccesful exit code if an error occours - for example if the given directory already existed

而 touch 文件实现锁需要两步 ↓ 不是原子操作, 所以不推荐:

1. check the existance of the lockfile (if)
2. if it doesn't exist, it would create one (lock) and continue(touch)

一个简单的 mkdir 互斥锁(MUTEX) 如下:

if mkdir /var/lock/mylock; then
  echo "Locking succeeded" >&2
else
  echo "Lock failed - exit" >&2
  exit 1
fi

• 另外 linux 下也可用 flock 锁: » Linuxaria – Everything about GNU/Linux and Open source Linux shell: Introduction to Flock

• IFS, Internal_field_separator

• TOREAD: UNIX/Linux Bash Shell Scripting: Math in Shell Scripts
• Performing Math calculation in Bash | Shell Tips ! ↓

I use math in bash scripts a lot, from simple crontab reports to Nagios monitoring plugins… Here is few small examples on how to do some maths in Bash with integers or float.

Integer Math

First way to do math with integer (and only integer) is to use the command “expr — evaluate expression“.

When doing a “multiply by” make sure to backslash the “asterisk”  as it’s a wildcard in Bash used for expansion.

Another alternative to expr, is to use the bash builtin command let.

Also, you can simply use the parentheses or square brackets :

This allow you to use C-style programming :

Floating point arithmetic

If you need to do floating point arithmetic, you will have to use a command line tool, the most common one is “bc – An arbitrary precision calculator language“.

Of course you can use the STDIN to send your formula to “bc” then get the output on STDOUT.

I encourage you too take a look at the man pages to get more detail on how it works (man bc).

There are four special variables, scale, ibase, obase, and last.  scale defines how some operations use digits after the decimal point.  The default value of scale is 0. ibase and obase define the conver-
sion base for input and output numbers.  The default for both input and output is base 10.  last (an extension) is a variable that has the value of the last printed number.

The “scale” variable is really important for the precision of your results, especially when using integers only (Note: you can also use “bc -l” to use mathlib and see the result at max scale) .

You can also use the here-doc notation to pass your formula to bc :

• http://tldp.org/LDP/abs/html/
• http://www.richardma.org/blog/2012/03/12/linux-bash%E5%86%99%E5%87%BA%E5%81%A5%E5%A3%AE%E7%9A%84bash%E8%84%9A%E6%9C%AC/

# clean line end spaces
alias clend='sed -i "s/[ \t]*$//"'
alias clendgitdiff='clend `git diff --name-only`'
alias clendgitstaged='clend `git diff --staged --name-only`; git add `git diff --name-only`'
 
alias regxp_debug='perl -Mre=debug -e '
 
# safe rm(xiaopei.li@2012-12-13)
rm() {
  RMDIR=/home/xiaopei.li/.recycle/`date +%s`/
  mkdir $RMDIR
  for var in "$@"; do
    is_arg=`echo $var | grep '^\-'`
    if [[ ! $is_arg ]]; then
      mv "$var" $RMDIR
    fi
  done
}

debugging - How do you debug a regex? - Stack Overflow

$ perl -Mre=debug -e'"foobar"=~/(.)\1/'
Compiling REx "(.)\1"
Final program:
   1: OPEN1 (3)
   3:   REG_ANY (4)
   4: CLOSE1 (6)
   6: REF1 (8)
   8: END (0)
minlen 1
Matching REx "(.)\1" against "foobar"
   0 <> <foobar>             |  1:OPEN1(3)
   0 <> <foobar>             |  3:REG_ANY(4)
   1 <f> <oobar>             |  4:CLOSE1(6)
   1 <f> <oobar>             |  6:REF1(8)
                                  failed...
   1 <f> <oobar>             |  1:OPEN1(3)
   1 <f> <oobar>             |  3:REG_ANY(4)
   2 <fo> <obar>             |  4:CLOSE1(6)
   2 <fo> <obar>             |  6:REF1(8)
   3 <foo> <bar>             |  8:END(0)
Match successful!
Freeing REx: "(.)\1"

RegExr

Kodos - The Python Regex Debugger

• 笔记
• 来源

http://www.ibm.com/developerworks/cn/linux/l-bash-test.html

• 根据表达式 expr 求值的结果返回 0（真）或 1（假）
• test expr 和 [ expr ] 是等价的, 需注意中括号与其中的内容间需有空格
• 可以用 $? 检查返回值
• 可以使用 && 和 || 操作返回值
• 可以用 -eq、 -ne、-lt、 -le、 -gt 或 -ge 比较算术值
• 可以分别用操作符 =、 !=、< 和 > 比较字符串
• 字符串比较中, 不等号比较字典序, 且必须用 \< 或 \> 加以转义
• 单目操作符 -z 测试 null 字符串，如果字符串非空 -n 返回 True
• 文件测试↓

• -o 操作符允许测试利用 set -o 选项 设置的各种 shell 选项
• 可以用括号把表达式分组，但需要用 \( 和 \) 转义括号，或者把这些操作符括在单引号或双引号内
• 测试目录是否为空 [ “$(ls -A $DIR)” ]

• 一些简单测试

$ test 3 -gt 4 && echo True || echo false
false
$ [ "abc" != "def" ];echo $?
0
$ test -d "$HOME" ;echo $?
0

• 一些字符串测试

$ test "abc" = "def" ;echo $?
1
$ [ "abc" != "def" ];echo $?
0
$ [ "abc" \< "def" ];echo $?
0
$ [ "abc" \> "def" ];echo $?
1
$ [ "abc" \<"abc" ];echo $?
1
$ [ "abc" \> "abc" ];echo $?
1

• 测试 shell 选项

$ set +o nounset
$ [ -o nounset ];echo $?
1
$ set -u
$ test  -o nounset; echo $?
0

• 组合和分组测试

$ test "a" != "$HOME" -a 3 -ge 4 ; echo $?
1
$ [ ! \( "a" = "$HOME" -o 3 -lt 4 \) ]; echo $?
1
$ [ ! \( "a" = "$HOME" -o '(' 3 -lt 4 ')' ")" ]; echo $?
1

• 由于 test 命令很难满足其转义需求以及字符串和算术比较之间的区别, 所以 bash 提供了其他两种测试方式
• (( )) 复合命令计算算术表达式，如果表达式求值为 0，则设置退出状态为 False(1)；如果求值为非 0 值，则设置为 True(0)。不需要对 (( 和 )) 之间的操作符转义。算术只对整数进行。除 0 会产生错误，但不会产生溢出。可以执行 C 语言中常见的算术、逻辑和位操作。
• let 命令也能执行一个或多个算术表达式。它通常用来为算术变量分配值。
• 利用复合命令 [[ ]] 可以对文件名和字符串使用更自然的语法
• 在使用 == 或 != 操作符时，复合命令 [[ 还能在字符串上进行模式匹配
• [[ 复合命令内可使用 -gt 或 (( )) 执行算术测试

• 分配和测试算术表达式

[ian@pinguino ~]$ let x=2 y=2**3 z=y*3;echo $? $x $y $z
0 2 8 24
[ian@pinguino ~]$ (( w=(y/x) + ( (~ ++x) & 0x0f ) )); echo $? $x $y $w
0 3 8 16
[ian@pinguino ~]$ (( w=(y/x) + ( (~ ++x) & 0x0f ) )); echo $? $x $y $w
0 4 8 13

• 使用 [[ 复合命令

[ian@pinguino ~]$ [[ ( -d "$HOME" ) && ( -w "$HOME" ) ]] &&  
>  echo "home is a writable directory"
home is a writable directory

• 用 [[ 进行通配符测试

[ian@pinguino ~]$ [[ "abc def .d,x--" == a[abc]*\ ?d* ]]; echo $?
0
[ian@pinguino ~]$ [[ "abc def c" == a[abc]*\ ?d* ]]; echo $?
1
[ian@pinguino ~]$ [[ "abc def d,x" == a[abc]*\ ?d* ]]; echo $?
1

• 用 [[ 包含算术测试

[ian@pinguino ~]$ [[ "abc def d,x" == a[abc]*\ ?d* || (( 3 > 2 )) ]]; echo $?
0
[ian@pinguino ~]$ [[ "abc def d,x" == a[abc]*\ ?d* || 3 -gt 2 ]]; echo $?
0
[ian@pinguino ~]$ [[ "abc def d,x" == a[abc]*\ ?d* || 3 > 2 ]]; echo $?
0
[ian@pinguino ~]$ [[ "abc def d,x" == a[abc]*\ ?d* || a > 2 ]]; echo $?
0
[ian@pinguino ~]$ [[ "abc def d,x" == a[abc]*\ ?d* || a -gt 2 ]]; echo $?
-bash: a: unbound variable

[ian@pinguino ~]$ function mycalc ()
> {
>   local x
>   if [ $# -lt 1 ]; then
>     echo "This function evaluates arithmetic for you if you give it some"
>   elif (( $* )); then
>     let x="$*"
>     echo "$* = $x"
>   else
>     echo "$* = 0 or is not an arithmetic expression"
>   fi
> }

在 shell 脚本中检测某服务器的某端口是否可访问

#!/bin/bash
 
# 定时运行, 检测本地 memcached(port 11211) 连接是否正常
# 若不正常, 则重启 memcached, 并发信通知管理员
# (xiaopei.li@2012-04-28)
exec 3>/dev/tcp/localhost/11211
if [ $? -ne 0 ]
then
	service memcached restart
	echo 'memcached has a problem and restarted ' | sendmail admin@example.com
fi

http://www.unix.com/shell-programming-scripting/105041-how-create-shell-script-test-telnet-connection-status.html

if diff foo bar > /dev/null 2>&1 ; then
    echo same
fi

经常可以在一些脚本，尤其是在crontab调用时发现如下形式的命令调用

/tmp/test.sh > /tmp/test.log 2>&1

前半部分/tmp/test.sh > /tmp/test.log很容易理解，那么后面的2>&1是怎么回事呢？

要解释这个问题，还是得提到文件重定向。我们知道>和<是文件重定向符。那么1和2是什么？在shell中，每个进程都和三个系统文件相关联：标准输入stdin，标准输出stdout和标准错误stderr，三个系统文件的文件描述符分别为0，1和2。所以这里2>&1的意思就是将标准错误也输出到标准输出当中。

下面通过一个例子来展示2>&1有什么作用：

$ cat test.sh
t
date

test.sh中包含两个命令，其中t是一个不存在的命令，执行会报错，默认情况下，错误会输出到stderr。date则能正确执行，并且输出时间信息，默认输出到stdout

./test.sh > test1.log
./test.sh: line 1: t: command not found
 
$ cat test1.log
Tue Oct 9 20:51:50 CST 2007

可以看到，date的执行结果被重定向到log文件中了，而t无法执行的错误则只打印在屏幕上。

$ ./test.sh > test2.log 2>&1
 
$ cat test2.log
./test.sh: line 1: t: command not found
Tue Oct 9 20:53:44 CST 2007

这次，stderr和stdout的内容都被重定向到log文件中了。

实际上， > 就相当于 1> 也就是重定向标准输出，不包括标准错误。通过2>&1，就将标准错误重定向到标准输出了，那么再使用>重定向就会将标准输出和标准错误信息一同重定向了。如果只想重定向标准错误到文件中，则可以使用2> file。

2>1的意思是将stderr重定向输出到名字为1的文件中了

&1是引用stdout的文件句柄，也就是将stderr合并到stdout中去

shell 编程之2>&1