Modern systems rarely fail because of simple inputs. They fail because of interactions — combinations of parameters, hidden dependencies, and edge cases that no one thought to test together. As systems grow, the input space expands exponentially, and manual test design quickly turns into guesswork. Very quickly, you face one of two problems:
- You test only a small, arbitrary subset of combinations.
- You attempt full coverage and create an unmaintainable explosion of tests.
We created tomato to solve exactly this problem.
Tomato is a command-line tool for model-based and combinatorial test generation. It allows you to describe your input domain once — in a structured YAML model — and automatically generate a systematic, reproducible suite of test cases in CSV format.
Tomato follows a simple principle: define your domain’s structure, and the engine explores combinations. It finds an optimal test set that systematically covers the input domain while keeping the test suite manageable. When requirements change, update the model and regenerate the suite. The model serves as the single source of truth.
There is more than that
Although tomato is the core engine, it comes as part of a small CLI suite, testomaton, supporting the entire combinatorial testing workflow.
The suite includes three complementary tools:
- tomato – model analysis and combinatorial test generation
- beaver – evaluates dynamic expressions inside generated CSV files
- jigsaw – manipulates and reshapes CSV data
Each tool has a clear responsibility. All three follow the classic Unix philosophy. They read input from standard input (stdin) and write results to standard output (stdout). This makes them composable in Unix-like pipelines. You can chain them or combine them with external tools like grep, awk, sed, sort, or custom formatters without intermediate files.
This stream-oriented design keeps the tools simple, flexible, and highly scriptable, making them easy to integrate into CI pipelines and larger automation systems.
Integration with test frameworks
In practice, generated test suites are most valuable when they can be executed directly within existing test frameworks. During development we verified that integrating tomato into popular testing ecosystems is both straightforward and pleasant.
We successfully integrated the generated suites with frameworks such as pytest, jest, and vitest. Because tomato produces simple CSV output, the generated combinations can easily be consumed by test runners, parameterized tests, or small helper scripts that feed the data into the framework of choice.
Examples of these integrations appear in a dedicated tomato-demo repository on Bitbucket.
Free and Open Source
tomato, together with the rest of the suite, is released as free and open source software. The tools are distributed as a Python package and can be installed directly from PyPI using pip.
pip install testomaton
After installation, all CLI tools — tomato, beaver, and jigsaw — become available in your environment and can immediately be used in scripts, pipelines, or CI systems.
The source code for the Testomaton suite tools is available in a public repository on Bitbucket. The readme file in that repo contains the latest documentation about the suite.
That’s not all
What you have seen is only the beginning. Upcoming articles will explore more of the tomato ecosystem and practical techniques to maximize efficiency of combinatoric testing.
We will introduce the GUI model editor with AI assistance, discuss private installations for organizations with strict privacy needs, and share tips for modelling complex domains and integrating generated suites with test frameworks.
Stay tuned — there is more to uncover.
Back to blog