Bash Scripting Learn Notes

References:

[1] http://www.tldp.org/LDP/Bash-Beginners-Guide/html/

1. Executing programs from a script

When the program being executed is a shell script, bash will create a new bash process using a fork. This subshell reads the lines from the shell script one line at a time. Commands on each line are read, interpreted and executed as if they would have come directly from the keyboard.

While the subshell processes each line of the script, the parent shell waits for its child process to finish. When there are no more lines in the shell script to read, the subshell terminates. The parent shell awakes and displays a new prompt.

If input is not commented, the shell reads it and divides it into words and operators, employing quoting rules to define the meaning of each character of input. Then these words and operators are translated into commands and other constructs, which return an exit status available for inspection or processing. The above fork-and-exec scheme is only applied after the shell has analyzed input in the following way:

• The shell reads its input from a file, from a string or from the user's terminal.

• Input is broken up into words and operators, obeying the quoting rules, see Chapter 3. These tokens are separated by metacharacters. Alias expansion is performed.

• The shell parses (analyzes and substitutes) the tokens into simple and compound commands.

• Bash performs various shell expansions, breaking the expanded tokens into lists of filenames and commands and arguments.

• Redirection is performed if necessary, redirection operators and their operands are removed from the argument list.

• Commands are executed.

• Optionally the shell waits for the command to complete and collects its exit status.

It is preferred to execute the script like this in a subshell. The variables, functions and aliases created in this subshell are only known to the particular bash session of that subshell. When that shell exits and the parent regains control, everything is cleaned up and all changes to the state of the shell made by the script, are forgotten. If you don't want to start a new shell but execute the script in the current shell, you source it: source script_name.sh

2. Debug the script

set -x			# activate debugging from here
w
set +x			# stop debugging from here

3. Introduction of some configuration files

1) /etc/profile

When invoked interactively with the --login option or when invoked as sh, Bash reads the /etc/profile instructions. These usually set the shell variables PATH, USER, MAIL, HOSTNAME and HISTSIZE.

2) /etc/bashrc

You might also find that /etc/profile on your system only holds shell environment and program startup settings, while /etc/bashrc contains system-wide definitions for shell functions and aliases. The /etc/bashrc file might be referred to in /etc/profile or in individual user shell initialization files.

3) ~/.bash_profile

This is the preferred configuration file for configuring user environments individually. In this file, users can add extra configuration options or change default settings:

4) ~/.bash_login

This file contains specific settings that are normally only executed when you log in to the system.

5) ~/.profile

In the absence of ~/.bash_profile and ~/.bash_login, ~/.profile is read. It can hold the same configurations, which are then also accessible by other shells. Mind that other shells might not understand the Bash syntax.

6) ~/.bashrc

Today, it is more common to use a non-login shell, for instance when logged in graphically using X terminal windows. Upon opening such a window, the user does not have to provide a user name or password; no authentication is done. Bash searches for ~/.bashrc when this happens, so it is referred to in the files read upon login as well, which means you don't have to enter the same settings in multiple files.

7) ~/.bash_logout

This file contains specific instructions for the logout procedure. 

When making changes to any of the above files, users have to either reconnect to the system or source the altered file for the changes to take effect.

4. Special Bash Variables

Character	Definition
$*	Expands to the positional parameters, starting from one. When the expansion occurs within double quotes, it expands to a single word with the value of each parameter separated by the first character of the IFS special variable.
$@	Expands to the positional parameters, starting from one. When the expansion occurs within double quotes, each parameter expands to a separate word.
$#	Expands to the number of positional parameters in decimal.
$?	Expands to the exit status of the most recently executed foreground pipeline.
$-	A hyphen expands to the current option flags as specified upon invocation, by the set built-in command, or those set by the shell itself (such as the -i).
$$	Expands to the process ID of the shell.
$!	Expands to the process ID of the most recently executed background (asynchronous) command.
$0	Expands to the name of the shell or shell script.
$_	The underscore variable is set at shell startup and contains the absolute file name of the shell or script being executed as passed in the argument list. Subsequently, it expands to the last argument to the previous command, after expansion. It is also set to the full pathname of each command executed and placed in the environment exported to that command. When checking mail, this parameter holds the name of the mail file.

The positional parameters are the words following the name of a shell script. They are put into the variables $1, $2, $3 and so on. As long as needed, variables are added to an internal array. $# holds the total number of parameters. The implementation of "$*" has always been a problem and realistically should have been replaced with the behavior of "$@". In almost every case where coders use "$*", they mean "$@". "$*" Can cause bugs and even security holes in your software. e.g. User franky starts entering the grep command, which results in the assignment of the _ variable. The process ID of his shell is 10662. After putting a job in the background, the ! holds the process ID of the backgrounded job. The shell running is bash. When a mistake is made, ? holds an exit code different from 0 (zero).

$* 这个程式的所有参数
$# 这个程式的参数个数
$@ 跟$*类似，但是可以当作数组用

5. Single quotes and double quotes

franky ~> echo '$date'
$date

franky ~> echo "$date"
20021226

6. Basic Substitutions

* Shell parameter and variable expansion

${Parameter}       e.g. echo ${USER:=chi}

* Command Substitution

$(Command) or `Command`      e.g. echo "$(date)"     or     echo `date`

* Arithmetic expansion

$((Expression))            e.g. echo "$((3*2))"  or     echo "$((2 == 2 ? 1 : 0))"

* Process Substitution

<(LIST)  or  >(LIST)     e.g. ls -l /proc/self/fd > /var/tmp/fdtest.at

* $[ EXPRESSION ]

However, this will only calculate the result of EXPRESSION, and do no tests:

franky ~> echo $[365*24]
8760

7. String Comparisons

An example of comparing strings for testing the user ID:

if [ "$(whoami)" != 'root' ]; then
        echo "You have no permission to run $0 as non-root user."
        exit 1;
fi

With Bash, you can shorten this type of construct. The compact equivalent of the above test is as follows:

[ "$(whoami)" != 'root' ] && ( echo you are using a non-privileged account; exit 1 )

Similar to the "&&" expression which indicates what to do if the test proves true, "||" specifies what to do if the test is false.

Regular expressions may also be used in comparisons:

anny > gender="female"

anny > if [[ "$gender" == f* ]]
More input> then echo "Pleasure to meet you, Madame."; fi
Pleasure to meet you, Madame.

8. [] vs [[]]

Contrary to [, [[ prevents word splitting of variable values. So, if VAR="var with spaces", you do not need to double quote $VAR in a test - eventhough using quotes remains a good habit. Also, [[ prevents pathname expansion, so literal strings with wildcards do not try to expand to filenames. Using [[, == and != interpret strings to the right as shell glob patterns to be matched against the value to the left, for instance: [[ "value" == val* ]].

#!/bin/bash

# This script gives information about a file.

FILENAME="$1"

echo "Properties for $FILENAME:"

if [ -f $FILENAME ]; then

The above example will fail if the value of $1 can be parsed as multiple words. In that case, the if command can be fixed either using double quotes around the filename, or by using [[ instead of [.

()会开启一个新的子shell，{}不会开启一个新的子shell

(())常用于算术运算比较,[[]]常用于字符串的比较.

$()返回括号中命令执行的结果

$(())返回的结果是括号中表达式值

${ }参数替换与扩展

9. echo

The echo built-in command outputs its arguments, separated by spaces and terminated with a newline character. The return status is always zero. echo takes a couple of options:

• -e: interprets backslash-escaped characters.

• -n: suppresses the trailing newline.

 echo -e "Feeding $menu to $animal...\n"

10. > /dev/null/ 2>&1

> is for redirect

/dev/null is a black hole where any data sent, will be discarded

2 is the file descriptor for Standard Error

> is for redirect

& is the symbol for file descriptor (without it, the following 1 would be considered a filename)

1 is the file descriptor for Standard Out

Therefore >/dev/null 2>&1 is redirect the output of your program to /dev/null. Include both the Standard Error and Standard Out.

Much more information is available at The Linux Documentation Project's I/O Redirection page.

cron will only email you if there is some output from you job. With everything redirected to null, there is no output and hence cron will not email you.

11. Array

[bob in ~] ARRAY=(one two three)

[bob in ~] echo ${ARRAY[*]}
one two three

[bob in ~] echo $ARRAY[*]
one[*]

[bob in ~] echo ${ARRAY[2]}
three

[bob in ~] ARRAY[3]=four

[bob in ~] echo ${ARRAY[*]}
one two three four

[bob in ~] unset ARRAY[1]

[bob in ~] echo ${ARRAY[*]}
one three four

[bob in ~] unset ARRAY

[bob in ~] echo ${ARRAY[*]}
<--no output-->

[bob in ~] echo $SHELL
/bin/bash

[bob in ~] echo ${#SHELL}
9

[bob in ~] ARRAY=(one two three)

[bob in ~] echo ${#ARRAY}
3

12. Substitution and String

${VAR:-WORD}

If VAR is not defined or null, the expansion of WORD is substituted; otherwise the value of VAR is substituted:

[bob in ~] echo ${TEST:-test}
test

[bob in ~] echo $TEST

This form is often used in conditional tests, for instance in this one:

[ -z "${COLUMNS:-}" ] && COLUMNS=80

If the hyphen (-) is replaced with the equal sign (=), the value is assigned to the parameter if it does not exist:

[bob in ~] echo $TEST2


[bob in ~] echo ${TEST2:=$TEST}
a_string

[bob in ~] echo $TEST2
a_string

To strip a number of characters, equal to OFFSET, from a variable, use this syntax:

${VAR:OFFSET:LENGTH}

The LENGTH parameter defines how many characters to keep, starting from the first character after the offset point. If LENGTH is omitted, the remainder of the variable content is taken:

[bob in ~] export STRING="thisisaverylongname"

[bob in ~] echo ${STRING:4}
isaverylongname

[bob in ~] echo ${STRING:6:5}
avery

13. Functions

Special built-in commands are found before shell functions during command lookup. The special built-ins are: break, :, ., continue, eval, exec, exit, export,readonly, return, set, shift, trap and unset.

[lydia@cointreau ~/test] cat showparams.sh
#!/bin/bash
                                                                                
echo "This script demonstrates function arguments."
echo
                                                                                
echo "Positional parameter 1 for the script is $1."
echo
                                                                                
test ()
{
echo "Positional parameter 1 in the function is $1."
RETURN_VALUE=$?
echo "The exit code of this function is $RETURN_VALUE."
}
                                                                                
test other_param

[lydia@cointreau ~/test] ./showparams.sh parameter1
This script demonstrates function arguments.
 
Positional parameter 1 for the script is parameter1.
 
Positional parameter 1 in the function is other_param.
The exit code of this function is 0.

[lydia@cointreau ~/test]

14. Signals

Standard key combination	Meaning
Ctrl+C	The interrupt signal, sends SIGINT to the job running in the foreground.
Ctrl+Y	The delayed suspend character. Causes a running process to be stopped when it attempts to read input from the terminal. Control is returned to the shell, the user can foreground, background or kill the process. Delayed suspend is only available on operating systems supporting this feature.
Ctrl+Z	The suspend signal, sends a SIGTSTP to a running program, thus stopping it and returning control to the shell.

Both kill commands send the TERM signal if none is given.

When killing a process or series of processes, it is common sense to start trying with the least dangerous signal, SIGTERM. That way, programs that care about an orderly shutdown get the chance to follow the procedures that they have been designed to execute when getting the SIGTERM signal, such as cleaning up and closing open files. If you send a SIGKILL to a process, you remove any chance for the process to do a tidy cleanup and shutdown, which might have unfortunate consequences.

But if a clean termination does not work, the INT orKILL signals might be the only way. For instance, when a process does not die using Ctrl+C, it is best to use thekill -9 on that process ID.

Signal name	Signal value	Effect
SIGHUP	1	Hangup
SIGINT	2	Interrupt from keyboard
SIGKILL	9	Kill signal
SIGTERM	15	Termination signal
SIGSTOP	17,19,23	Stop the process

15. Traps

trap [COMMANDS] [SIGNALS]

This instructs the trap command to catch the listed SIGNALS, which may be signal names with or without the SIG prefix, or signal numbers. If a signal is 0 or EXIT, the COMMANDS are executed when the shell exits. If one of the signals is DEBUG, the list of COMMANDS is executed after every simple command. A signal may also be specified as ERR; in that case COMMANDS are executed each time a simple command exits with a non-zero status. Note that these commands will not be executed when the non-zero exit status comes from part of an if statement, or from a while or until loop. Neither will they be executed if a logical AND (&&) or OR (||) result in a non-zero exit code, or when a command's return status is inverted using the ! operator.

#!/bin/bash
# traptest.sh

trap "echo Booh!" SIGINT SIGTERM
echo "pid is $$"

while :			# This is the same as "while true".
do
        sleep 60	# This script is not really doing anything.
done

Special built-in commands are found before shell functions during command lookup. The special built-ins are: break, :, ., continue, eval, exec, exit, export,readonly, return, set, shift, trap and unset.

16. System environment variables 

Environment variables set without export will not be inherited in the environment of the commands you are calling. e.g.

HOSTNAME is the environment variable and could be got by $HOSTNAME. But it couldn't be accessed by java -jar example.jar.

1) export HOSTNAME=$HOSTNAME and then you could access it by System.getenv("HOSTNAME"). 

2) java -DHOSTNAME='XXX' -jar example.jar

17. About other people

• w --- tells you who's logged in, and what they're doing. Especially useful: the 'idle' part. This allows you to see whether they're actually sitting there typing away at their keyboards right at the moment.
• who --- tells you who's logged on, and where they're coming from. Useful if you're looking for someone who's actually physically in the same building as you, or in some other particular location.
• finger username --- gives you lots of information about that user, e.g. when they last read their mail and whether they're logged in. Often people put other practical information, such as phone numbers and addresses, in a file called .plan. This information is also displayed by 'finger'.
• last -1 username --- tells you when the user last logged on and off and from where. Without any options, last will give you a list of everyone's logins.
• talk username --- lets you have a (typed) conversation with another user
• write username --- lets you exchange one-line messages with another user
• elm --- lets you send e-mail messages to people around the world (and, of course, read them). It's not the only mailer you can use, but the one we recommend. See the elm page, and find out about the departmental mailing lists (which you can also find in /user/linguistics/helpfile).