C.2. Awk

Awkis a full-featured text processinglanguage with a syntax reminiscent of C. Whileit possesses an extensive set of operators and capabilities,we will cover only a couple of these here - the ones most usefulfor shell scripting.

Awk breaks each line of input passed to it intofields. By default, a field isa string of consecutive characters separated by whitespace, though there areoptions for changing the delimiter. Awk parses and operates oneach separate field. This makes awk ideal for handling structuredtext files -- especially tables -- data organized into consistentchunks, such as rows and columns.

Strong quoting (single quotes) and curly brackets enclosesegments of awk code within a shell script.

   1 echo one two | awk '{print $1}'
   2 # one
   3
   4 echo one two | awk '{print $2}'
   5 # two
   6
   7
   8 awk '{print $3}' $filename
   9 # Prints field #3 of file $filename to stdout.
  10
  11 awk '{print $1 $5 $6}' $filename
  12 # Prints fields #1, #5, and #6 of file $filename.

We have just seen the awk printcommandin action. The only other feature of awk we need to deal withhere is variables. Awk handles variables similarly to shellscripts, though a bit more flexibly.

   1 { total += ${column_number} }

This adds the value of column_numbertothe running total of "total". Finally, to print"total", there is an ENDcommandblock, executed after the script has processed all its input.

   1 END { print total }

Corresponding to the END, there is aBEGIN, for a code block to be performed before awkstarts processing its input.

The following example illustrates how awkcanadd text-parsing tools to a shell script.

Example C-1. Counting Letter Occurrences

   1 #! /bin/sh
   2 # letter-count2.sh: Counting letter occurrences in a text file.
   3 #
   4 # Script by nyal [nyal@voila.fr].
   5 # Used with permission.
   6 # Recommented by document author.
   7 # Version 1.1: Modified to work with gawk 3.1.3.
   8 #              (Will still work with earlier versions.)
   9
  10
  11 INIT_TAB_AWK=""
  12 # Parameter to initialize awk script.
  13 count_case=0
  14 FILE_PARSE=$1
  15
  16 E_PARAMERR=65
  17
  18 usage()
  19 {
  20     echo "Usage: letter-count.sh file letters" 2>&1
  21     # For example:   ./letter-count2.sh filename.txt a b c
  22     exit $E_PARAMERR  # Not enough arguments passed to script.
  23 }
  24
  25 if [ ! -f "$1" ] ; then
  26     echo "$1: No such file." 2>&1
  27     usage                 # Print usage message and exit.
  28 fi
  29
  30 if [ -z "$2" ] ; then
  31     echo "$2: No letters specified." 2>&1
  32     usage
  33 fi
  34
  35 shift                      # Letters specified.
  36 for letter in `echo $@`    # For each one . . .
  37   do
  38   INIT_TAB_AWK="$INIT_TAB_AWK tab_search[${count_case}] = \"$letter\"; final_tab[${count_case}] = 0; "
  39   # Pass as parameter to awk script below.
  40   count_case=`expr $count_case + 1`
  41 done
  42
  43 # DEBUG:
  44 # echo $INIT_TAB_AWK;
  45
  46 cat $FILE_PARSE |
  47 # Pipe the target file to the following awk script.
  48
  49 # ----------------------------------------------------------------------------------
  50 # Earlier version of script used:
  51 # awk -v tab_search=0 -v final_tab=0 -v tab=0 -v nb_letter=0 -v chara=0 -v chara2=0 \
  52
  53 awk \
  54 "BEGIN { $INIT_TAB_AWK } \
  55 { split(\$0, tab, \"\"); \
  56 for (chara in tab) \
  57 { for (chara2 in tab_search) \
  58 { if (tab_search[chara2] == tab[chara]) { final_tab[chara2]++ } } } } \
  59 END { for (chara in final_tab) \
  60 { print tab_search[chara] \" => \" final_tab[chara] } }"
  61 # ----------------------------------------------------------------------------------
  62 #  Nothing all that complicated, just . . .
  63 #+ for-loops, if-tests, and a couple of specialized functions.
  64
  65 exit $?
  66
  67 # Compare this script to letter-count.sh.

For simpler examples of awk within shell scripts, see:

1. Example 11-12
2. Example 16-8
3. Example 12-29
4. Example 33-5
5. Example 9-23
6. Example 11-18
7. Example 27-2
8. Example 27-3
9. Example 10-3
10. Example 12-55
11. Example 9-28
12. Example 12-4
13. Example 9-13
14. Example 33-16
15. Example 10-8
16. Example 33-4

That's all the awk we'll cover here, folks, but there's lotsmore to learn. See the appropriate references in the Bibliography.