Generators

Generators are responsible for turning your model into a test suite. Tomato currently offers three generator types, each with its own set of options. To create a test suite click the GENERATE TESTS button close the the function name. The generator window should appear. After (optionally) changing generation options, click on the GENERATE button at the bottom of the screen. That’s it!

Accessing generator options

Fig. 1 – Accessing generator options.

Generator options

The default generation options should be enough for most scenarios. However, you can always modify them.

Algorithm

Currently, three types of generators are implemented. Each of them comes with specific options.

N-Wise

The generator outputs test cases that cover all n-tuples of the space.

N-Wise generator options

Fig. 2 – N-Wise generator options.

Options:

  • N – The size of the tuple. For example N=2 (pairwise) covers all possible pairs of values of the input parameters. This is also the default value.
  • Coverage – The percentage of tuples that must be covered. The default value is 100.
  • Tuples – A set of parameters that must be covered by tuples. When it is left blank, all parameters are considered as the input set (what is the default value). In this example, the “coverage” value applies only to “weapon” and “location” parameters.

Random

The generator outputs test cases with parameters selected randomly.

Random generator options

Fig. 3 – Random generator options.

Options:

  • Length – How many test cases should be generated.
  • Duplicates – Defines whether two identical test cases can be generated.
  • Adaptive – Generate tests that are as different from already generated tests as possible. The metric of how different two tests are is the Hamming distance (number of elements that differ). For each step, tomato will look up to max 100 tests back and calculate a test that differs the most from all of them.

Cartesian

Cartesian generator

Fig. 4 – Cartesian generator.

The generator outputs all possible combinations of parameter values, which are valid according to defined constraints. Keep in mind that this can easily explode into thousands or even millions of test cases, which is why it’s rarely the best choice. This generator does not take any additional options.

Filters

Generator filters

Fig. 5 – Generator filters.

Filters define which elements should be taken into consideration while generating test cases. There are two main types of filters:

  • Blacklist – Remove elements from generation.
  • Whitelist – Only selected elements can be used during the generation process.

In this example, we exclude the “weapon” parameter from generation. Note, that it is possible to create as many filters as required.

Options

Generator options

Fig. 6 – Generator options.

There are also a few advanced options that can change the generation process. For instance, you can invert logic or ignore it entirely.The last group of settings, i.e. ‘Other’, affects the structure and formatting of the generated CSV file.