System Programming

" One vision, one purpose. "

Copyright © Tony's Studio 2020 - 2022

Chapter Four - Shell

4.1 Get Ready for Shell

4.1.1 Redirect

1
2
3
 stdin: <
stdout: > or 1>
stderr: 2>

4.1.2 Pipe

Just wire the output of a command to the input of another command.

Here is a comprehensive example. Output of the first cat, which is the content of a.txt is wired to the input of the second cat, whose stdout is redirected to file b.txt, and stderr is redirected to stdout. However, there is no such thing like 2>>&1.

1
cat a.txt | cat >>b.txt 2>&1

4.1.3 Environment Variable

4.1.3.1 Add Environment Variable

Well, as you’ve known before, it is easy to add environment variables to current bash process.

1
export environment_variable=xxx

But if you want this variable added to the bash permanently, you have to put that line to ~/.bashrc file, which is loaded every time for initialization.

4.1.3.2 Get Environment Variable

For C, there are several ways to get environment variable.

1
2
3
4
5
6
7
// Method 1, use global environ.
extern char** environ;
// Method 2, use parameter of main.
int main(int argc, char* argv[], char* envp[]) {...}
// Method 3, use function.
#include <stdlib.h>
char* value = getenvp("var_name");

4.1.3.3 Execute Permission

See [chmod](#1.4.8 chmod).

4.2 Shell Programming

4.2.1 Special Symbols

#!/bin/bash: Must be at the first line to indicate interpreter.

#: In-line comment. Use \# if you really want a #.

;: Separate multiple commands. e.g. echo Hello; echo there!

,: Separate arithmetic expression, just like C. Here is an example.

1
2
3
4
5
$ a=$((b = 9, 15 / 3))   # check out 4.2.4
$ echo $a;
$ 5
$ echo $b
$ 9

\: Escape character, similar to what is in C.

1
2
$ echo \$num
$ $num

4.2.2 Run Bash Script

There are several ways to run a bash script. If we directly run it by ./filename, it must have execute permission. Use chmod u+x script.sh. And #!/bin/bash is preferred, although default bash will be selected if this is missing. The other two don’t need these.

1
2
3
$ ./script.sh
$ bash script.sh
$ source script.sh

One more thing, why ./ is needed? Because not like what is in Windows, in Linux, current directory is not added to the environment path for… for security reason. Not any ordinary environment variable, but PATH. Once added, we can simply run a bash script just by its name.

1
export PATH=$PATH:.	# ':' connects variables

4.2.3 Exit Status

We can use exit ret to exit a bash script with a return value ret. 0 means success, anything else is considered not good. If exit is missing, then the return value depends on the last statement executed in the script.

For the return value, we are recommended to use 0 ~ 125, 126 means not executable, 127 means command not found, and 128+ means signal received. The last return value is stored in $?.

Like C, expression can be put together by some logic operator. && and || remains the same meaning and function. But here in bash, 0 means true, others are regarded as false. For &&, if previous statement is false, bash will abandon latter statements and return false.

4.2.3 Variables

“I always hate variables in script language.”

4.2.3.1 Declaration

Variable name is the same as C, but could not contain ‘$’. The value… I hate this. String value is just plain text if contains no space. Quote it with ‘’ or “” like var="Hello there!". These two are a little different, though.

1
2
$ var=value                 # no space, no $
$ readonly const=constant   # read-only variable

If you do not want a variable any more, you can use unset to… unset it. :P

1
unset var

4.2.3.2 Quotes

In Bash, there are three types of quotes. A little tricky, huh?

1
2
3
4
5
6
7
8
9
10
11
# Three types of quote have different behaviors.
$ plain='this is $plain "text" $?'
$ rich="this is not '$plain' text"
$ super=`echo this is not \'$rich\' text`
# Here are the results.
$ echo $plain
this is $plain "text" $?
$ echo $rich
this is not 'this is $plain "text" $?' text
$ echo $super
this is not 'this is not 'this is $plain "text" $?' text' text

4.2.3.3 Internal Variables

Interval Variable Meaning
$? last return value
$# number of parameters, just like argc
$* all parameters passed to the script, just like argv as a string
$@ all parameters, no IFS, more like argv, seperated strings
$$ pid of current bash process, commonly used as temp file name
$! pid of the last process in background
$0 current process name, just like argv[0]
$1 to $9 parameters, just like argv[i]

For $# and $@, things can be really tricky. Here is a good example.

This example comes from here: What’s the difference between $@ and $*.

1
2
3
4
5
#!/bin/bash
echo "With *:"
for arg in "$*"; do echo "<$arg>"; done
echo "With @:"
for arg in "$@"; do echo "<$arg>"; done

Its output can be as follows.

1
2
3
4
5
6
7
$ ifs A B "C D"
With *:
<A B C D>
With @:
<A>
<B>
<C D>

For $1 to $9, we can use shift to shit them. After shift, $1 will become $2, and so on. But this won’t affect $0.

4.2.3.4 Variable Replacement

There are some advanced, but tricky variable assignment.

var=${param:-word}: If param is set, then var is set to param. Or var is set to word, like a default value.

var=${param:=word}: If param is set, then var is set to param. Or both var and param is set to word.

var=${param:?word}: If param is set, then var is set to param. Otherwise, script will display error info and exit with return value 1. The error info is as follows.

1
2
-bash: param: word                       # word is not null
-bash: param: parameter null or not set  # word is omitted var=${param:?}

var=${param:+word}: If param is set, then var is set to word. Or var is set to null.

Err… you know, we can simply use ${param:-word} alone… It’s just a value, and this is it.

4.2.4 Expression

“I hate expressions in these script languages.”

4.2.4.1 Variables in Expression

Just add dollar sign before a variable to use it. For command variables, both $() and `` are OK.

1
2
3
$variable
$(command)
`command`

For arithmetic variables, use $((...)) to wrap them, or use expr command. Or more directly, use let command.

1
2
3
4
5
$ a=$((3 + 4 + $b))
$ c=$(expr 5 + $d + 7)
$ let a=a++
$ let a=a+1+5
$ let "a = a * (3 + 4)"

For a more powerful way to use variables, you can consider ${}. In this way, I guess, number is regarded as string. However, in x64 architecture, numbers seem to be a 64 bits signed value in calculation.

format return value
$var or ${var} value of the variable
${#var} length of the variable
${var:start} substring of the variable from start to end
${var:start:length} substring of the variable from start in given length, or till the end

4.2.4.2 Condition Expression

In Bash, we can use test condition to check condition, or just [condition]. There are four types of condition check, string, number, logic and file.

For string comparation, there are five operations. Be aware of the space around ‘[‘, ‘]’ and ‘=’.

test meaning
[ str1 = str2 ] or [ str1 == str2 ] return true if str1 and str2 are the same
[ str1 != str2] return true if str1 and str2 are different
[ str ] return true if str is not null
[ -n str ] return true if length of str is greater than zero
[ -z str ] return true if length of str is zero

For arithmetic comparation, we can also use condition expression.

test meaning
[ a -eq b ] return true if a == b
[ a -ne b ] return true if a != b
[ a -ge b] return true if a >= b
[ a -le b ] return true if a <= b
[ a -gt b ] return true if a > b
[ a -lt b ] return true if a < b

Or, we can just use ((expression)) to replace [ condition ], which is easier to understand.

For logic condition, it is quite like arithmetic.

test meaning
!expr not
expr1 -a expr2 and
expr1 -o expr2 or

For file condition, things are almost the same.

test meaning
[ -d file ] return true if file is a directory
[ -f file ] return true if file is a file
[ -r file ] return true if file is readable
[ -w file ] return true if file is writable
[ -x file ] return true if file is executable
[ -s file ] return true if file size greater than zero

4.2.4.3 Branch and Loop

It’s a little wired to have If-then-elif-then-else.

1
2
3
4
5
6
7
if [ condition ]; then
	# statements
elif [ condition ]; then
	# statements
else
	# statements
fi

It’s even more wired with switch-case. :(

1
2
3
4
5
6
7
8
9
10
11
case expression in
	pattern1)
		statements
		;;
	pattern2)
		statements
		;;
	*)			# default
		statements
		;;
esac

There are three kinds of loop in Bash, while, until and for.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
# loop while condition is true
while [ condition ]; do
	# statements
done
# loop until condition is true
until [ condition ]; do
	# statements
done
# for is a little tricky
for arg in [list]; do
	# statements
done
# for can also be like this in a simpler format
for ((i = 0; i < 5; i++)); do
	# statements
done

Hmm… An extra example, I think, redirect can be appended to loop.

1
2
3
4
5
6
7
8
9
10
# print the 10th line in a file
file="test.txt"
i=0
while read line; do
	i=$(($i+1))
	if (($i==10)); then
		echo $line
		break
	fi
done < $file

For for loop, list is a string or array, for example, we can write this.

1
2
3
for i in 1 2 3; do
	echo $i
done

And it will print things out.

1
2
3
1
2
3

Actually you can refer to previous example with [$@ and $*](#4.2.3.3 Internal Variables).

4.2.5 Array

Ahh… tired… leave it alone. :(

4.2.x Example

“Talk is cheap, show me the code.”

Here is a comprehensive example. The script has one parameter, and its meaning is as follows.

  • mine: find all files that belongs to the current user under the current directory and print their filenames. (not recursively)
  • largest：print the largest file under the current directory. (not recursively)
  • expand: move all files recursively to the current directory, and then delete all subfolders (they should be empty now).
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
#!/bin/bash
dir=`pwd`
# A recursive function example.
function move {
	if [ "$(ls -A ${1})" ]; then	# dir must not be empty
		for file in ${1}/*; do	    # empty dir will cause file match plain text 
			filename=`basename $file`
			if [ -d $filename ]; then
				new="${1}/$filename"
                  move $new
			else
				if [ ${1} != $dir ]; then
					old="${1}/$filename"
                      new="$dir/$filename"
                      mv $old $new
                      # echo "Move $old to $new"
				fi
			fi
		done
	fi
}

# empty string check
if [ -z $1 ]; then
	echo "Usage: ./manage.sh OPERATION"
	exit 1
fi

# string compa
if [ $1 == "mine" ]; then
	flag=0
	for file in ${dir}/*; do
		filename=`basename $file`
		if [ $filename == `basename $0` ]; then		# we don't want itself
			continue
		fi
		if [ -d $filename ]; then		# we don't want a directory
			continue
		fi
		if [ -O $filename ]; then		# if belongs to the current user
			echo -n "$filename "
			flag=1
		fi
	done
	if (( $flag == 1 )); then
		echo ""
	fi
elif [ $1 == "largest" ]; then
	largestSize=-1
	largestFile=""
	if [ "$(ls -A ${dir})" ]; then
		for file in ${dir}/*; do
			filename=`basename $file`
			if [ $filename == `basename $0` ]; then
	         	continue
			fi
			if [ -d $filename ]; then
				continue
			fi
			size=$(stat -c %s "$filename")
			if (( $size > $largestSize )); then
				largestSize=$size
	             largestFile=$filename
			fi
		done
		if [ $largestFile != "" ]; then
			echo "Largest file: $largestFile"
		fi
	fi	# if [ "$(ls -A ${dir})" ]; then
elif [ $1 == "expand" ]; then
	move $dir
	if [ "$(ls -A ${dir})" ]; then
		for file in ${dir}/*; do
			filename=`basename $file`
	         if [ -d $filename ]; then
				rm -r $filename
			fi
		done
	fi
else
	echo "Invalid argument! [mine, largest, expand]"
fi

" Do or do not. There is no try. "

Copyright © Tony's Studio 2020 - 2022

- EOF -