Using Bash shell scripts for function testing
功能测试是软件开发的一个关键部分 -- 而已经装入 Linux 的 Bash 可以帮您轻而易举地完成功能测试。在本文中，Angel Rivera 将说明如何运用 Bash shell 脚本通过行命令来执行 Linux 应用程序的功能测试。由于此脚本依赖于命令行的返回码，因而您不能将这种方法运用于 GUI 应用程序
功能测试是开发周期的一个阶段，在这个阶段中将测试软件应用程序以确保软件的函数如预期的那样，同时能正确处理代码中错误。此项工作通常在单个模块的单元测试结束之后，在负载／重压条件下整个产品的系统测试之前进行的。
市场上有许多测试工具提供了有助于功能测试的功能。然而，首先要获取它们，然后再安装、配置，这将占用您宝贵的时间和精力。Bash 可以帮您免去这些烦琐的事从而可以加快测试的进程。
使用 Bash shell 脚本进行功能测试的优点在于：
- Bash shell 脚本已经在 Linux 系统中安装和配置好了。不必再花时间准备它。
- 可以使用由 Linux 提供的文本编辑器如 vi 创建和修改 Bash shell 脚本。不需要再为创建测试程序而获取专门的工具。
- 如果已经知道了如何开发 Bourne 或 Korn shell 脚本，那对于如何运用 Bash shell 脚本已经足够了。对您来说，学习曲线已不存在了。
- Bash shell 提供了大量的编程构造用于开发从非常简单到中等复杂的脚本。
将脚本从 Korn 移植到 Bash 时的建议
如果已有现成的 Korn shell 脚本，而想要将它们移植到 Bash，就需要考虑下列情况：
- Korn 的 "print" 命令在 Bash 中不能使用；而是改为使用 "echo" 命令。
- 需要将脚本的第一行：
#!/usr/bin/ksh
修改成：
#!/bin/bash
创建 Bash shell 脚本进行功能测试
这些基本的步骤和建议适用于许多在 Linux 上运行的客户机／服务器应用程序。
- 记录运行脚本的先决条件和主要步骤
- 将操作分成若干个逻辑组
- 基于一般方案制定执行步骤
- 在每个 shell 脚本中提供注释和说明
- 做一个初始备份以创建基准线
- 检查输入参数和环境变量
- 尝试提供 "usuage" 反馈
- 尝试提供一个“安静”的运行模式
- 当出现错误时，提供一个函数终止脚本
- 如可能，提供可以执行单个任务的函数
- 当显示正在生成的输出时，捕获每个脚本的输出
- 在每个脚本内，捕获每个行命令的返回码
- 计算失败事务的次数
- 在输出文件中，突出显示错误消息，以便于标识
- 如有可能，“实时”生成文件
- 在执行脚本的过程中提供反馈
- 提供脚本执行的摘要
- 提供一个容易解释的输出文件
- 如有可能，提供清除脚本及返回基准线的方法
下面详细讲述了每一条建议以及用于说明问题的脚本。若要下载此脚本，请参阅本文后面的 参考资料部分。
1. 记录运行脚本的先决条件和主要步骤
记录，尤其是以有自述标题的单个文件（例如 "README-testing.txt"）记录功能测试的主要想法是很重要的，包括，如先决条件、服务器和客户机的设置、脚本遵循的整个（或详细的）步骤、如何检查脚本的成功／失败、如何执行清除和重新启动测试。
2. 将操作分成若干个逻辑组
如果仅仅执行数量非常少的操作，可以将它们全部放在一个简单的 shell 脚本中。
但是，如果需要执行一些数量很多的操作，那最好是将它们分成若干个逻辑集合，例如将一些服务器操作放在一个文件而将客户机操作放在在另一个文件中。通过这种方法，划分适当的颗粒度来执行测试和维护测试。
3. 基于一般方案制定执行步骤
一旦决定对操作进行分组，需要根据一般方案考虑执行操作的步骤。此想法是模拟实际生活中最终用户的情形。作为一个总体原则，只需集中测试 80% 最常调用函数的 20% 用法即可。
例如，假设应用程序要求 3 个测试组以某个特定的顺序排列。每个测试组可以放在一个带有自我描述文件名（如果可能）的文件中，并用号码来帮助识别每个文件的顺序，例如：
|
4. 在每个 shell 脚本中提供注释和说明
在每个 shell 脚本的头文件中提供相关的注释和说明是一个良好的编码习惯。这样的话，当另一个测试者运行该脚本时，测试者就能清楚地了解每个脚本中测试的范围、所有先决条件和警告。
下面是一个 Bash 脚本 "test-bucket-1" 的示例 。
|
5. 做一个初始备份以创建基准线
您可能需要多次执行功能测试。第一次运行它时，也许会找到脚本或进程中的一些错误。因而，为了避免因从头重新创建服务器环境而浪费大量时间 -- 特别是如果涉及到数据库 -- 您在测试之前或许想做个备份。
在运行完功能测试之后，就可以从备份中恢复服务器了，同时也为下一轮测试做好了准备。
6. 检查输入参数和环境变量
最好校验一下输入参数，并检查环境变量是否设置正确。如果有问题，显示问题的原因及其修复方法，然后终止脚本。
当测试者准备运行脚本，而此时如果没有正确设置脚本所调用的环境变量，但由于发现及时，终止了脚本，那测试者会相当感谢。没有人喜欢等待脚本执行了很久却发现没有正确设置变量。
|
关于此脚本的说明如下：
- 使用语句
CALLER=`basename $0`可以得到正在运行的脚本名称。这样的话，无须在脚本中硬编码脚本名称。因此当复制脚本时，采用新派生的脚本可以减少工作量。 - 调用脚本时，语句
TEMP=`getopt hs $*`用于得到输入变量（例如 -h 代表帮助，-s 代表安静模式）。 - 语句
[ -z "$X" ]和echo "The environment variable X is not set."以及usage都是用于检测字符串是否为空 (-z)，如果为空，随后就执行 echo 语句以显示未设置字符串并调用下面要讨论的 "usage" 函数。 - 若脚本未使用标志，可以使用变量 "$#"，它可以返回正在传递到脚本的变量数量。
7. 尝试提供“usage”反馈
脚本中使用 "usage" 语句是个好主意，它用来说明如何使用脚本。
|
调用脚本时，使用“-h”标志可以调用 "usage" 语句，如下所示：
./test-bucket-1 -h
8. 尝试使用“安静”的运行模式
您或许想让脚本有两种运行模式：
- 在 "verbose" 模式（您也许想将此作为缺省值）中提示用户输入值，或者只需按下 Enter 继续运行。
- 在 "silent" 模式中将不提示用户输入数据。
下列摘录说明了在安静模式下运用所调用标志 "-s" 来运行脚本：
|
9. 当出现错误时，提供一个函数终止脚本
遇到严重错误时，提供一个中心函数以终止运行的脚本不失为一个好主意。此函数还可提供附加的说明，用于指导在此情况下应做些什么：
|
10. 如有可能，提供可以执行简单任务的函数
例如，不使用许多很长的行命令，如：
|
……而是创建一个如下所示的函数，此函数也可以处理返回码，如果有必要，还可以增加错误计数器：
|
……然后，以易读和易扩展的方式调用此函数：
|
11. 当显示正在生成的输出时，捕获每个脚本的输出
如果脚本不能自动地将输出发送到文件的话，可以利用 Bash shell 的一些函数来捕获所执行脚本的输出，如：
|
让我们来分析上面的命令：
- "2>&1" 命令：
使用 "2>&1" 将标准错误重定向到标准输出。字符串 "2>&1" 表明任何错误都应送到标准输出，即 UNIX/Linux 下 2 的文件标识代表标准错误，而 1 的文件标识代表标准输出。如果不用此字符串，那么所捕捉到的仅仅是正确的信息，错误信息会被忽略。
- 管道 "|" 和 "tee" 命令：
UNIX/Linux 进程和简单的管道概念很相似。既然这样，可以做一个管道将期望脚本的输出作为管道的输入。下一个要决定的是如何处理管道所输出的内容。在这种情况下，我们会将它捕获到输出文件中，在此示例中将之称为 "test-bucket-1.out"。
但是，除了要捕获到输出结果外，我们还想监视脚本运行时产生的输出。为达到此目的，我们连接允许两件事同时进行的 "tee" （T- 形管道）：将输出结果放在文件中同时将输出结果显示在屏幕上。其管道类似于：
process --> T ---> output file | V screen
如果 只 想捕获输出结果而不想在屏幕上看到输出结果，那可以忽略多余的管道：
./test-bucket-1 -s 2>&1 > test-bucket-1.out假若这样，相类似的管道如下：
process --> output file
12. 在每个脚本内，捕获每个行命令所返回码
决定功能测试成功还是失败的一种方法是计算已失败行命令的数量，即返回码不是 0。变量 "$?" 提供最近所调用命令的返回码；在下面的示例中，它提供了执行 "ls" 命令的返回码。
|
13. 记录失败事务的次数
在功能测试中决定其成功或失败的一个方法是计算返回值不是 0 的行命令数量。但是，从我个人的经验而言，我习惯于在我的 Bash shell 脚本中仅使用字符串而不是整数。在我所参考的手册中没有清楚地说明如何使用整数，这就是我为什么想在此就关于如何使用整数和计算错误（行命令失败）数量的方面多展开讲的原因：
首先，需要按如下方式对计数器变量进行初始化：
|
然后，发出行命令并使用 $? 变量捕获返回码。如果返回码不是 0，那么计数器增加 1（见蓝色粗体语句）：
|
顺便说一下，与其它变量一样，可以使用 "echo" 显示整数变量。
14. 在输出文件中，为了容易标识，突出显示错误消息
当遇到错误（或失败的事务）时，除了错误计数器的数量会增加外，最好标识出此处有错。较理想的做法是，字符串有一个如 ERROR 或与之相似的子串（见蓝色粗体的语句），这个子串允许测试者很快地在输出文件中查找到错误。此输出文件可能很大，而且它对于迅速找到错误非常重要。
|
15. 如有可能，“实时”生成文件
在某些情况下，有必要处理应用程序使用的文件。可以使用现有文件，也可以在脚本中添加语句来创建文件。如果要使用的文件很长，那最好将其作为独立的实体。如果文件很小而且内容简单或不相关（重要的一点是文本文件而不考虑它的内容），那就可以决定“实时”创建这些临时文件。
下面几行代码显示如何“实时”创建临时文件：
|
第一个 echo 语句使用单个的 > 强行创建新文件。第二个 echo 语句使用两个 >> 将数据附加到现有文件的后面。顺便说一下，如果该文件不存在，那么会创建一个文件。
16. 在执行脚本的过程中提供反馈
最好在脚本中包含 echo 语句以表明它执行的逻辑进展状况。可以添加一些能迅速表明输出目的的语句。
如果脚本要花费一些时间执行，那或许应在执行脚本的开始和结束的地方打印时间。这样可以计算出所花费的时间。
在脚本样本中，一些提供进展说明的 echo 语句如下所示：
|
17. 提供脚本执行的摘要
如果正在计算错误或失败事务的次数，那最好表明是否有错误。此方法使得测试者在看到输出文件的最后能迅速地辨认出是否存在错误。
在下面的脚本示例中，代码语句提供了上述脚本的执行摘要：
|
18. 提供一个容易解释的输出文件
在脚本生成的实际输出中提供一些关键信息是非常有用的。那样，测试者就可以很容易地确定正在查看的文件是否与自己所做的相关以及它是否是当前产生的。附加的时间戳记对于是否是当前状态是很重要的。摘要报告对于确定是否有错误也是很有帮助的；如果有错误，那么测试者就必须搜索指定的关键字，例如 ERROR，并确认出个别失败的事务。
以下是一段输出文件样本的片段：
|
当遇到错误时输出文件最后部分的示例如下所示：
|
19. 如有可能，提供清除脚本及返回基准线的方法
测试脚本可以生成临时文件；假若这样，最好能让脚本删除所有临时文件。这就会避免由于测试者也许没有删除所有临时文件而引起的错误，更糟糕的是将所需要的文件当作临时文件而删除了。
运行功能测试的 Bash shell 脚本
本节描述如何运用 Bash shell 脚本进行功能测试。假设您已经执行了在前面部分中所述步骤。
设置必要的环境变量
根据需要在 .profile 中或手工指定下列环境变量。该变量用于说明在脚本中如何处理，所需环境变量的验证必须在脚本执行前定义。
|
将 Bash shell 脚本复制到正确的目录下
Bash shell 脚本和相关文件需要复制到要进行功能测试的用户标识的目录结构下。
- 登录进某个帐户。您应该在主目录下。假设它是 /home/tester。
- 为测试案例创建目录：
mkdir fvt - 复制 Bash shell 脚本和相关文件。获取压缩文件（请参阅 参考资料 ）并将其放在 $HOME 下。然后将其按下列方式解压：
unzip trfvtbash.zip - 为了执行这个文件，更改文件的许可权：
chmod u+x * - 更改名称以除去文件的后缀：
mv test-bucket-1.bash test-bucket-1
运行脚本
执行下列步骤以运行脚本：
- 以测试者的用户标识登录
- 更改目录至所复制脚本的位置：
cd $HOME/fvt - 从 $HOME/fvt 运行脚本：
./test-bucket-1 -s 2>&1 | tee test-bucket-1.out - 看一下输出文件 "test-bucket-1.out" 的尾部并查看摘要报告的结论。
原文：
Function testing is the phase during a development cycle in which the software application is tested to ensure that the functionality is working as desired and that any errors in the code are properly handled. It is usually done after the unit testing of individual modules, and before a more thorough system test of the entire product under load/stress conditions.
There are many testing tools in the marketplace that offer a lot of functionalityto help with the testing efforts. However, they need to be obtained, installed, and configured, which could take up valuable time and effort. Bash can help to speed things along.
The advantages of using Bash shell scripts for function testing are:
- The Bash shell is already installed and configured in yourLinux system. You do not have to spend time in getting it ready.
- You can create and modify the Bash shell scripts using text editors already providedby Linux, such as vi. You do not need to acquire specialized toolsto create the test cases.
- If you already know how to develop Bourne or Korn shell scripts, thenyou already know enough tostart working with Bash shell scripts. Your learning curve isgreatly diminished.
- The Bash shell provides plenty of programming constructs to developscripts that have a range from very simple to medium complexity.
Recommendations when porting scripts from Korn to Bash
If you have existing Korn shell scripts that you want to port to Bash, you need to take into account the following:
- The Korn "print" command is not available in Bash; use the "echo" command instead.
- You will need to change the first line of the script from:
#!/usr/bin/kshto:#!/bin/bash
Creating Bash shell scripts for function testing
These basic steps and recommendations can be applied to many client/server applications that run in Linux.
- Document the prerequisites and main sequence for running scripts
- Divide actions into logical groups
- Develop an execution sequence based on a common usage scenario
- Provide comments and instructions in each shell script
- Make an initial backup to create a baseline
- Check for input parameters and environment variables
- Try to provide "usage" feedback
- Try to provide a "silent" running mode
- Provide one function to terminate the script when there are errors
- When possible, provide functions that do a single task well
- Capture the output of each script, while watching the output being produced
- Inside each script, capture the return code of each line command
- Keep a count of the failed transactions
- Highlight the error messages for easy identification in the output file
- When possible, generate files "on the fly"
- Provide feedback on the progress of the execution of the script
- Provide a summary of the execution of the script
- Try to provide an output file that is easy to interpret
- When possible, provide cleanup scripts and a way to return to the baseline
Each recommendation is detailed below along with a Bash shell script for illustration. To download this script, see theResources section later in this article.
1. Document the prerequisites and main sequence for running scripts
It is important to document, preferably in a single file with a self-describing title (such as "README-testing.txt"), the main ideas behind the function testing, including the prerequisites, the setup for the server and the client, the overall (or detailed) sequence of the scripts to follow, how to check for success/failures of the scripts, how to perform the cleanup, and to restart the testing.
2. Divide the actions into logical groups
If you have a very small list of actions to be performed, then you could put them all in a single shell script.
However, if you have a large list of actions, it is good to group them into logical sets, such as the server actions in one file and the client actions in another. This way, you will have finer granularity to perform the testing and to maintain the test cases.
3. Develop an execution sequence based on a common usage scenario
Once you have decided on the grouping of the actions, you need to think of performing the actions in a sequence that follows a common usage scenario. The idea is to simulate a real-life end-user situation. As a general rule, try to focus on the 20% of usage cases that test about 80% of the most commonly invoked functions.
For example, let's assume that the application requires three test groups in a specific sequence. Each test group could be in a file, with a self-describing filename (where possible), and a number that helps to indicate the order of each file in the sequence, such as:
1. fvt-setup-1: To perform initial setup.
2. fvt-server-2: To perform server commands.
3. fvt-client-3: To perform client commands.
4. fvt-cleanup: To cleanup the temporary files,
in order to prepare for the repetition
of the above test cases. |
4. Provide comments and instructions in each shell script
It is good coding practice to provide pertinent comments and instructions in the header of each shell script. That way, when another tester is assigned to run the scripts, the tester will get a good idea of the scope of the testing done in each script, as well as any prerequisites and warnings.
An example is shown below, from the sample Bash script "test-bucket-1".
#!/bin/bash # # Name: test-bucket-1 # # Purpose: # Performs the test-bucket number 1 for Product X. # (Actually, this is a sample shell script, # which invokes some system commands # to illustrate how to construct a Bash script) # # Notes: # 1) The environment variable TEST_VAR must be set # (as an example). # 2) To invoke this shell script and redirect standard # output and standard error to a file (such as # test-bucket-1.out) do the following (the -s flag # is "silent mode" to avoid prompts to the user): # # ./test-bucket-1 -s 2>&1 | tee test-bucket-1.out # # Return codes: # 0 = All commands were successful # 1 = At least one command failed, see the output file # and search for the keyword "ERROR". # ######################################################## |
5. Make an initial backup to create a baseline
You may need to perform the function testing several times. The first time you run it, you will likely find some errors in your scripts or in the procedures. Therefore, to avoid wasting too much time in recreating the server environment from scratch -- especially if a database is involved -- you may want to make a backup just before starting with the testing.
After you run the function test cases, then you could restore the server from the backup, and you would be ready for the next round of testing.
6. Check for input parameters and environment variables
It is a good idea to validate the input parameters and to check if the necessary environment variables are properly set. If there are problems, display the reason for the problem and how to fix it, and terminate the script.
The tester who is going to run this script will generally appreciate it if the script terminates shortly after being invoked in case a variable is not correct. No one likes to wait a long time in the execution of the script to find out that a variable was not properly set.
# -------------------------------------------- # Main routine for performing the test bucket # -------------------------------------------- CALLER=`basename $0` # The Caller name SILENT="no" # User wants prompts let "errorCounter = 0" # ---------------------------------- # Handle keyword parameters (flags). # ---------------------------------- # For more sophisticated usage of getopt in Linux, # see the samples file: /usr/lib/getopt/parse.bash TEMP=`getopt hs $*` if [ $? != 0 ] then echo "$CALLER: Unknown flag(s)" usage fi # Note quotes around `$TEMP': they are essential! eval set -- "$TEMP" while true do case "$1" in -h) usage "HELP"; shift;; # Help requested -s) SILENT="yes"; shift;; # Prompt not needed --) shift ; break ;; *) echo "Internal error!" ; exit 1 ;; esac done # ------------------------------------------------ # The following environment variables must be set # ------------------------------------------------ if [ -z "$TEST_VAR" ] then echo "Environment variable TEST_VAR is not set." usage fi |
Note the following about this script:
- The statement
CALLER=`basename $0`is used to get the name of the script being run. In that way, you do not need to hard-code the script name in the script. Thus, when you make a copy of the script, it will take less work to adapt the newly derived script. - The statement
TEMP=`getopt hs $*`is used to get the input arguments when the script is invoked (such as the -h for help and -s for silent mode). - The statements
[ -z "$X" ]andecho "The environment variable X is not set."andusageare used to test if the string is null (-z) and if so, then perform the echo statement saying that it is not set and invoke the "usage" function discussed below. - If your script does not use flags, then you can use the variable "$#", which returns the number of arguments that are being passed to the script.
7. Try to provide "usage" feedback
It is a good idea to provide a "usage" statement that explains how to use the script:
# ----------------------------
# Subroutine to echo the usage
# ----------------------------
usage()
{
echo "USAGE: $CALLER [-h] [-s]"
echo "WHERE: -h = help "
echo " -s = silent (no prompts)"
echo "PREREQUISITES:"
echo "* The environment variable TEST_VAR must be set,"
echo "* such as: "
echo " export TEST_VAR=1"
echo "$CALLER: exiting now with rc=1."
exit 1
} |
This "usage" statement can be called when the script is invoked with the "-h" flag, such as:./test-bucket-1 -h
8. Try to provide a "silent" running mode.
You may want a script to have two running modes:
- A "verbose" mode (you might want this as the default) in which the user is prompted to enter a value or to simply press Enter to continue.
- A "silent" mode, in which the user is not prompted for data.
The following excerpt illustrates the handling of the invocation flag "-s" to run the script in silent mode:
# ------------------------------------------------- # Everything seems OK, prompt for confirmation # ------------------------------------------------- if [ "$SILENT" = "yes" ] then RESPONSE="y" else echo "The $CALLER will be performed." echo "Do you wish to proceed [y or n]? " read RESPONSE # Wait for response [ -z "$RESPONSE" ] && RESPONSE="n" fi case "$RESPONSE" in [yY]|[yY][eE]|[yY][eE][sS]) ;; *) echo "$CALLER terminated with rc=1." exit 1 ;; esac |
9. Provide one function to terminate the script when there are errorsIt is a good idea to provide a central function to terminate the execution of the script when critical errors are encountered. This function could provide additional instructions on what to do in such situations:
# ----------------------------------
# Subroutine to terminate abnormally
# ----------------------------------
terminate()
{
echo "The execution of $CALLER was not successful."
echo "$CALLER terminated, exiting now with rc=1."
dateTest=`date`
echo "End of testing at: $dateTest"
echo ""
exit 1
} |
10. When possible, provide functions that perform a simple task well
For example, instead of issuing a big list of long line commands, such as:
# --------------------------------------------------
echo ""
echo "Creating Access lists..."
# --------------------------------------------------
Access -create -component Development -login ted -authority plead -verbose
if [ $? -ne 0 ]
then
echo "ERROR found in Access -create -component Development -login ted
-authority plead"
let "errorCounter = errorCounter + 1"
fi
Access -create -component Development -login pat -authority general -verbose
if [ $? -ne 0 ]
then
echo "ERROR found in Access -create -component Development -login pat
-authority general"
let "errorCounter = errorCounter + 1"
fi
Access -create -component Development -login jim -authority general -verbose
if [ $? -ne 0 ]
then
echo "ERROR found in Access -create -component Development -login jim
-authority general"
let "errorCounter = errorCounter + 1"
fi |
... you could create a function such as the following, which also handles the return code and, if needed, increases the error counter:
CreateAccess()
{
Access -create -component $1 -login $2 -authority $3 -verbose
if [ $? -ne 0 ]
then
echo "ERROR found in Access -create -component $1 -login $2 -authority $3"
let "errorCounter = errorCounter + 1"
fi
} |
... and then invoke this function in a manner that is easy to read and to expand:
# ------------------------------------------- echo "" echo "Creating Access lists..." # ------------------------------------------- CreateAccess Development ted projectlead CreateAccess Development pat general CreateAccess Development jim general |
11. Capture the output of each script, while displaying the output being produced
If the script does not automatically send the output to a file, you can exploit some features of the Bash shell to capture the output of the execution of the script, such as:
./test-bucket-1 -s 2>&1 | tee test-bucket-1.out |
Let's analyze the above command:
- The "2>&1" command:
Using "2>&1" , we redirect the standard error to standard output. The string "2>&1" indicates that any errors should be sent to the standard output, that is, the UNIX/Linux file id of 2 for standard error, and the file id of 1 for standard output. If you do not use this string, then you will be capturing only the good messages, and the error messages will not be captured.
- The pipe "|" and the "tee" command:
There is a good analogy between the UNIX/Linux processes and simple plumbing concepts. In this case, we want to make a pipeline in which the input to the pipeline is the output of the desired script. The next thing to decide is what to do with the output of the pipeline. In this case, we want to capture it in an output file, named "test-bucket-1.out" in our example.
However, besides capturing the output, we also want to watch the output being produced while the script is running. To this end, we attach a "tee" (T-shape pipe) that permits two things at the same time: placing the output into a file AND displaying the output into the screen. The plumbing analogy would be:
process --> T ---> output file | V screenIf you only want to capture the output and you do not want to see the output being displayed on the screen, then you can omit the extra plumbing:
./test-bucket-1 -s 2>&1 > test-bucket-1.out
The plumbing analogy in this case would be:
process --> output file
12. Inside each script, capture the return code of each line command
One way to determine the success or failure of the function testing is by counting the line commands that have failed, that is, that have a return code different than 0. The variable "$?" provides the return code of the command recently invoked; in the example below, it provides the return code of the execution of the "ls" command.
# -------------------------------------------
# The commands are called in a subroutine
# so that return code can be
# checked for possible errors.
# -------------------------------------------
ListFile()
{
echo "ls -al $1"
ls -al $1
if [ $? -ne 0 ]
then
echo "ERROR found in: ls -al $1"
let "errorCounter = errorCounter + 1"
fi
} |
13. Keep track of the number of failed transactions
One way to determine the success or failure in function testing is by counting the line commands that return a value other than 0. However, in my personal experience, I am accustomed to handling only strings in my Bash shell scripts, rather than integers. The manuals I consulted were not too clear on how to use integers, which is why I want to expand a little bit here on how to use integers and additions to count the number of errors (failures of line commands):
First, you need to initialize the counter variable as follows:
let "errorCounter = 0" |
Then, issue the line command and capture the return code using the $? variable. If the return code is different than 0, then increment the counter by one (see the statement in bold blue):
ListFile()
{
echo "ls -al $1"
ls -al $1
if [ $? -ne 0 ]
then
echo "ERROR found in: ls -al $1"
let "errorCounter = errorCounter + 1"
fi
} |
By the way, the integer variables can be displayed as other variables using "echo".
14. Highlight the error messages for easy identification in the output file
When an error (or failed transaction) is encountered, besides increasing the error counter, it is a good idea to print an indication that there was an error. Ideally, the string of characters should have a substring such as ERROR or something similar (see the statement in bold blue), which will allow the tester to quickly find the error in the output file. This output file could be large, and it is important to quickly locate errors.
ListFile()
{
echo "ls -al $1"
ls -al $1
if [ $? -ne 0 ]
then
echo "ERROR found in: ls -al $1"
let "errorCounter = errorCounter + 1"
fi
} |
15. When possible, generate files "on the fly"
In some cases it is necessary to handle files that will be used by the application. You could use existing files or you could add statements in the script to create them. If the files to be used are long, then it is better to have them as separate entities. If the files are small and the contents simple or not relevant (the important point is to have a text file, regardless of its contents), then you could decide to create these temporary files "on the fly".
The following lines of code show an example of how a temporary file is created "on the fly":
cd $HOME/fvt echo "Creating file softtar.c" echo "Subject: This is softtar.c" > softtar.c echo "This is line 2 of the file" >> softtar.c |
The first echo statement uses the single > to force the creation of a new file. The second echo statement uses the double >> to append data to the bottom of an existing file. By the way, in case the file does not exist, then the file will be created.
16. Provide feedback on the progress of the execution of the script
It is a good idea to include echo statements in the script to indicate the logical progress of its execution. You can add something that will quickly identify the purpose of the output.
If the script is going to take more than few seconds to execute, you may want to print the date at the beginning and at the end of the execution of the script. This will allow you to compute the elapsed time.
In the sample script, some echo statements that provide the indication of the progress are shown:
# -------------------------------------------- echo "Subject: Product X, FVT testing" dateTest=`date` echo "Begin testing at: $dateTest" echo "" echo "Testcase: $CALLER" echo "" # -------------------------------------------- # -------------------------------------------- echo "" echo "Listing files..." # -------------------------------------------- # The following file should be listed: ListFile $HOME/.profile ... # -------------------------------------------- echo "" echo "Creating file 1" # -------------------------------------------- |
17. Provide a summary of the execution of the script
If you are counting the errors or failed transactions, it is good to indicate whether there were errors. The idea is that the tester could see the bottom of the output file and quickly tell if there were errors or not.
In the following sample script, the code statements provide such a summary of the execution:
# -------------- # Exit # -------------- if [ $errorCounter -ne 0 ] then echo "" echo "*** $errorCounter ERRORS found during ***" echo "*** the execution of this test case. ***" terminate else echo "" echo "*** Yeah! No errors were found during ***" echo "*** the execution of this test case. Yeah! ***" fi echo "" echo "$CALLER complete." echo "" dateTest=`date` echo "End of testing at: $dateTest" echo "" exit 0 # end of file |
18. Try to provide an output file that is easy to interpret
It is very helpful to provide some key information in the actual output that is generated by the script. In that way, the tester could easily determine if the file that is being viewed is relevant and current. The addition of the date-time stamp is important to give a sense of currency. Also, the summary report helps to determine whether there were errors; if there were errors, then the tester will have to search for the specified keyword, such as ERROR, and identify the individual transactions that failed.
A truncated sample output file is shown below:
Subject: CMVC 2.3.1, FVT testing, Common, Part 1
Begin testing at: Tue Apr 18 12:50:55 EDT 2000
Database: DB2
Family: cmpc3db2
Testcase: fvt-common-1
Creating Users...
User pat was created successfully.
...
Well done! No errors were found during the
execution of this test case :)
fvt-common-1 complete.
End of testing at: Tue Apr 18 12:56:33 EDT 2000 |
An example of the bottom of the output file when errors are encountered is shown below:
ERROR found in Report -view DefectView *** 1 ERRORS found during the execution of this test case. *** The populate action for the CMVC family was not successful. Recreating the family may be necessary before running fvt-client-3 again, that is, you must use 'rmdb', 'rmfamily', 'mkfamily' and 'mkdb -d', then issue: fvt-common-1 and optionally, fvt-server-2. fvt-client-3 terminated, exiting now with rc=1. End of testing at: Wed Jan 24 17:06:06 EST 2001 |
19. When possible, provide cleanup scripts and a way to return to the baseline
The test scripts may generate temporary files; in that case, it is a good practice to have a script that will delete those temporary files. This will avoid mistakes in which the tester may not delete all the temporary files, or worse, delete some needed files that were not temporary.
Running the function-testing Bash shell script
This section describes how to run the Bash shell scripts for function testing. I'm assuming that you've executed all the steps in the previous sections.
Setting up required environment variables
Specify the following environment variables in the .profile or manually on demand. This variable is used to illustrate how to handle in the script, the verification that required environment variables must be defined prior to running the script.
export TEST_VAR=1 |
Copying the Bash shell scripts into the proper directory
The Bash shell scripts and the associated files need to be copied into the directory structure of the user id who is going to conduct the function testing.
- Log into the account. You should be in the home directory. Let's assume that it is /home/tester.
- Create a directory for the test cases:
mkdir fvt - Copy the Bash shell scripts and the associated files. Obtain the zip file (seeResources) and place it under $HOME. Then unzip it as follows:
unzip trfvtbash.zip - Change the proper file permissions, in order to execute the files:
chmod u+x * - Change the name to remove the file suffix:
mv test-bucket-1.bash test-bucket-1
Running the script
To run the script, perform the following:
- Log into the tester user id.
- Change to the directory where the scripts were copied:
cd $HOME/fvt - From $HOME/fvt run the script:
./test-bucket-1 -s 2>&1 | tee test-bucket-1.out - Look at the bottom of the output file "test-bucket-1.out" and see the conclusion of the summary report.
- Download trfvtbash.zip, which contains the sample code and tools referenced in this article. The tools might be updated in the future.
- To unzip the files, try the Info-ZIP software. Because of the general value of the tools, it is recommended that you add the unzip and zip tools in a directory in the PATH that is accessible to all the users for the machine.
How to unzip the files:To view the contents of the zip file (without actually unpackaging and uncompressing the files), do:
unzip -l trfvtbash.zipTo unpackage and uncompress the zip file, do:unzip trfvtbash.zip
- Read Daniel Robbins' three-part series on bash programming on developerWorks:Part 1,Part 2, andPart 3.
- Visit GNU's bash home page.
Angel Rivera is an advisory software engineer with the VisualAge TeamConnection technical support team, where he is currently the team lead. He has an M.S. in Electrical Engineering from The University of Texas at Austin, and a B.S. in Electronic Systems Engineering from the Instituto Tecnologico y de Estudios Superiores de Monterrey, Mexico. He joined IBM in 1989. He can be reached atrivera@us.ibm.com
In developing this article, Angel would like to acknowledge the contribution of Lee Perlov in WebSphere technical support.
reresources :
#!/bin/bash
#
# Name: test-bucket-1
#
# Purpose:
# Performs the test-bucket number 1 for Product X.
# (Actually, this is a sample shell script, which invokes some
# system commands to illustrate how to construct a Bash script)
#
# Notes:
# 1) The environment variable TEST_VAR must be set (as an example).
# 2) To invoke this shell script and redirect standard output and
# standard error to a file (such as test-bucket-1.out) do the
# following (the -s flag is "silent mode" to avoid prompts to the
# user):
#
# ./test-bucket-1 -s 2>&1 | tee test-bucket-1.out
#
# Return codes:
# 0 = All commands were successful
# 1 = At least one command failed, see the output file and search
# for the keyword "ERROR".
#
######################################################################.#########
# ----------------------------
# Subroutine to echo the usage
# ----------------------------
# ----------------------------
usage()
{
echo "USAGE: $CALLER [-h] [-s]"
echo "WHERE: -h = help "
echo " -s = silent (no prompts)"
echo "PREREQUISITES:"
echo "* The environment variable TEST_VAR must be set, such as: "
echo " export TEST_VAR=1"
echo "$CALLER: exiting now with rc=1."
exit 1
}
# ----------------------------------
# Subroutine to terminate abnormally
# ----------------------------------
terminate()
{
echo "The execution of $CALLER was not successful."
echo "$CALLER terminated, exiting now with rc=1."
dateTest=`date`
echo "End of testing at: $dateTest"
echo ""
exit 1
}
# ---------------------------------------------------------------------------
# ---------------------------------------------------------------------------
# The commands are called in a subroutine so that return code can be
# checked for possible errors.
# ---------------------------------------------------------------------------
ListFile()
{
echo "ls -al $1"
ls -al $1
if [ $? -ne 0 ]
then
echo "ERROR found in: ls -al $1"
let "errorCounter = errorCounter + 1"
fi
}
################################################################################
# --------------------------------------------
# Main routine for performing the test bucket
# --------------------------------------------
CALLER=`basename $0` # The Caller name
SILENT="no" # User wants prompts
let "errorCounter = 0"
let "errorCounter = 0"
# ----------------------------------
# Handle keyword parameters (flags).
# ----------------------------------
# For more sophisticated usage of getopt in Linux, see
# the samples file: /usr/lib/getopt/parse.bash
TEMP=`getopt hs $*`
if [ $? != 0 ]
then
echo "$CALLER: Unknown flag(s)"
usage
fi
# Note the quotes around `$TEMP': they are essential!
eval set -- "$TEMP"
while true
do
case "$1" in
-h) usage "HELP"; shift;; # Help requested
-s) SILENT="yes"; shift;; # Prompt is not needed
--) shift ; break ;;
*) echo "Internal error!" ; exit 1 ;;
esac
done
# ------------------------------------------------
# The following environment variables must be set
# ------------------------------------------------
# The following environment variables must be set
# ------------------------------------------------
[ -z "$TEST_VAR" ] && { echo "The environment variable TEST_VAR is not set."; usage; }
# --------------------------------------------------
# Everything seems to be OK, prompt for comfirmation
# --------------------------------------------------
if [ "$SILENT" = "yes" ]
then
RESPONSE="y"
else
echo "The $CALLER will be performed."
echo "Do you wish to proceed [y or n]? "
read RESPONSE # Wait for response
[ -z "$RESPONSE" ] && RESPONSE="n"
fi
case "$RESPONSE" in
[yY]|[yY][eE]|[yY][eE][sS])
;;
*)
echo "$CALLER terminated with rc=1."
exit 1
;;
esac
# --------------------------------------------------
echo "Subject: Product X, FVT testing"
dateTest=`date`
echo "Begin testing at: $dateTest"
echo ""
echo "Testcase: $CALLER"
echo ""
# --------------------------------------------------
# --------------------------------------------------
echo ""
echo "Listing files..."
# --------------------------------------------------
# The following file should be listed:
ListFile $HOME/.profile
# The following file should NOT be listed:
ListFile test-1
# --------------------------------------------------
echo ""
echo "Creating file 1"
# --------------------------------------------------
echo "This is file: test1" > test1
if [ $? -ne 0 ]
then
if [ $? -ne 0 ]
then
echo "ERROR found in: creating file test1"
let "errorCounter = errorCounter + 1"
fi
# --------------
# Exit
# --------------
if [ $errorCounter -ne 0 ]
then
echo ""
echo "*** $errorCounter ERRORS found during the execution of this test case. ***"
terminate
else
echo ""
echo "*** Yeah! No errors were found during the execution of this test case. Yeah! ***"
fi
echo ""
echo "$CALLER complete."
echo ""
dateTest=`date`
echo "End of testing at: $dateTest"
echo ""
exit 0