Variations and Purpose of Test Design Technique

Quality Assurance (QA) plays an important role during the creation of an IT product. In order to eliminate errors and “bugs”, QA engineers apply test design techniques among other tools.

Test design means the development and creation of tests. Each test is aimed at checking a specific assumption. For example: “What happens if a user mistakenly clicks here, using the right mouse button instead of the left one”. 

QA simulates a set of test scenarios (test cases) to see how the application behaves in different conditions. The QA specialist’s task is to find a balance and identify the maximum number of errors within the required minimum of test scenarios. In this case, you need to check the most important test cases, since the testing time is limited.

Types of Testing

Static testing, as it implies, does not require running a program or application and allows you to find the most obvious errors at the early stages of product creation. For example, part of the static testing is checking the software parameters for compliance with the requirements of the technical specifications, code proofing.
Dynamic testing requires software to be tested in action. This type, in its turn, is also divided into three groups:

• White box techniques (they are also called structural testing techniques) are used if the specialist knows the architecture of the product, its code, and the “stuffing” – so he can navigate in the program itself.
• The black box techniques allow you to check the operation of the product without any knowledge about the internal structure of the system. At the same time, testing is carried out on the basis of the requirements pointed out in the project specification.
• The gray box techniques allow you to test the product when a specialist partially knows its internal structure. To perform the “gray box” testing, access to the source code is not necessary.

Testing Steps

1. Preparation. At this stage QA engineer reads the design documentation, finds out the requirements for the product, works out a plan, thinks out a strategy, sets tasks according to priority and analyzes possible risks. 

2. Testing as it is. Experts analyze the information collected earlier, compile a list of tested functions, get acquainted with already known bugs, if any, write test cases.

We emphasize once again: it is fundamentally important to strive for the minimum possible number of tests, while the scripts have to find the largest number of high-priority defects. 

3. Analysis of the results and reporting. While creating tests, a QA specialist focuses not only on the documentation, but also on oral information from other QA specialists, analysts, developers, and project managers. 

Examples of Test Design Techniques

Let’s take a look at few basic techniques, keeping in the head that they are often used in combination. One technique may not be enough, because it will not provide maximum coverage of test scenarios. 

Equivalence Partitioning

The equivalence partitioning method allows you to minimize the number of tests without creating a script for each possible value, but choosing only one value from a whole class and taking the fact that the result will be similar for all values in this group as an axiom. 

For example, we test the functionality of an application that allows you to buy air and train tickets online. The ticket price will depend on the age of the passenger, as children, students and senior citizens belong to preferential categories. 

We have four age groups: under 15 years old, from 15 to 25 years old, over 25 and under 60 years old, and people over 60. At the same time, only two characters are placed in the age field, so it’s technically impossible to specify an age of more than 99 years.

 A QA specialist does not need to write 99 tests for each age, five will suffice: one for each age group (say, 10, 18, 35 and 75 years) and one for the case if the person is over 99 years old. Yes, the last test is not feasible in practice (since it is impossible to enter more than two characters in the age field), and yet you should not forget about this test.

Boundary Values

The boundary values technique is based on the assumption that most errors can occur at the boundaries of equivalent classes. It is closely related to the above technique of equivalence partitioning, this is why it is often used in conjunction with it. Then, for the example from the previous paragraph, the boundaries will be the values 0, 15, 25, 60, and 99. The boundary values will be 0, 1, 14, 15, 16, 24, 25, 26, 59, 60, 61, 98, 99, 100.

Difficulties often arise if age categories are indicated by "overlap", for example, 0-12, 12-25 years, etc.

Decision Table

Another name for the method is the decision matrix. This technique is suitable for more complex systems, for example, two-factor authentication. Suppose that in order to enter the system, the user must first enter the username and password, and then confirm his identity by sending an SMS code.

The possible scenarios are:

1. The correct username and password.
2. The correct username, the wrong password.
3. Wrong login, correct password.
4. Wrong login, wrong password.

The first of these scenarios is accompanied by either the correct or incorrect input of the SMS code, so we get 5 tests. In this case, only one of the scenarios will lead to a positive result (the user successfully logs in), and the rest will fail. 

However, the system may raise different messages depending on at which stage the error occurred, say: invalid login, invalid password. Thus, more groups will be needed, and the table will become larger. 

This method is good by showing all possible scenarios at once in a form understandable even to a non-specialist.

Pairwise Testing

The essence of this method is that each value of each parameter to be tested should be tested for interaction with each value of all other parameters. After compiling such a matrix, we remove duplicate tests leaving maximum coverage with the minimum required set of scenarios.

Pairwise testing allows you to detect maximum errors without redundant checks. 

Pairwise testing example 

For pairwise it is enough when each value of all parameters is combined with other values ​​of the remaining parameters at least once. The matrix can be significantly reduced in this case. For example:

While compiling a decision matrix for two browsers, two OSs, and two languages, 8 scenarios would have needed. In pairwise testing four are enough. 

All these things could be calculated manually, but not necessarily – it is much more convenient to automate the process. There is a special program of pairwise independent combined testing that deals with this task – Pairwise Independent Combinatorial Testing (PICT). The specialist creates a text file with a list of possible values, and then runs PICT through command line. Combined tests are displayed as a table in the console itself. The results could be optionally downloaded to an Excel file.

Cause and Consequence

Simple verification of basic actions and their results. For example, if you click the cross in the upper right corner of the window (cause), it closes (consequence), etc. This method allows you to check all the features of the system, as well as detect bugs and improve the technical documentation of the product. 

An approximate algorithm for using the technique:

1. We highlight the causes and consequences in the specifications.
2. We connect the causes and consequences.
3. We take into account the "impossible" combination of causes and consequences.
4. We draw up a “decision table”, where each column contains a combination of inputs and outputs, ie each column is a finished test case.
5. Set priorities.

This technique helps:

• To determine the minimum number of tests to find the maximum number of errors.
• To clarify all the causes and consequences – in this way, we will make sure that the system will have an answer to any manipulations with it.
• To find possible flaws in the application description logic (which will help to improve the documentation).

For example, a QA specialist is testing a “Notebook” application. After entering all the data of the new contact and clicking the “Create” button (cause), the application should automatically create a card with a phone number, photo and full name of the person (consequence). Tests will show whether it is possible to leave one or several fields empty, whether the system recognizes the Cyrillic alphabet, Latin alphabet or both alphabets, as well as other parameters.

Error Prediction

Using his knowledge of the system, a QA specialist can “predict” under what input conditions there is a risk of errors. This requires experience and knowledge of the product, as well as the ability to build communication with colleagues. 

For example, the specification states that a field should accept a four-digit code. Possible tests include:

• What happens if you do not enter the code?
• What happens if you do not enter special characters?
• What happens if you enter not numbers, but other characters?
• What happens if you enter not four numbers, but a different number?

Advantages:
1. This test is effective as a complement to other techniques.
2. Identifies test cases that “should never happen.”

Disadvantages:

1. Technique is largely based on intuition.
2. Experience in testing such systems is required.
3. Small test coverage.

Let’s Sum Up

Of course, this list is far from being complete and gives only the general idea of ​​the principles of testing and test design techniques. For example, exhaustive testing that covers all possible scenarios and detects all errors, exists only in theory. This is the result of that fact that checking all parameters and states will take too much time. However, the more experienced a QA specialist is, the better results he can achieve, and the ability to choose and combine techniques correctly is important for that.