Test automation has become a staple in software industry, but is generally restricted to lower-level, code-oriented testing. GUI testing, which is commonly used in industry to verify the correctness of a system under test’s (SUT) behavior or visual appearance, is, therefore, still are mostly manual, and costly, practice. GUI test automation tools are available, but require expert technical knowledge to develop and maintain test scripts. This requirement prohibits domain experts, e.g. stakeholders with unique needs and perspectives of how the SUT will be used, from contributing to test automation.

GUI tests are used for two purposes; regression testing where predefined, scripted, test scenarios are executed to verify that changes to the SUT have been implemented correctly or exploratory testing where undefined, unscripted, test scenarios are executed by a human to find new faulty behaviors. For regression testing, automation provides exact test replication and reduces human error but requires technical knowledge and is costly to maintain. For exploratory testing, new faults can effectively be identified but requires human cognition to innovate test scenarios and can therefore not be automated.

The aim of T.A.R.G.E.T. is to address these core challenges with GUI testing through innovation, realization and validation of demonstrators that incorporate state-of-the-art solutions in Augmented testing and generative AI. This work is performed in a consortium consisting of Synteda, QESTIT and Blekinge Institute of Technology, where experts in AI-, testing- and tool development will merge competences to co-produce a collaborative GUI testing solution that incorporates AI in a concept we refer to as human-machine symbiosis.

The project will address, mitigate and solve GUI testing’s requirements on specialized knowledge (e.g. scripting knowledge), constrained effectiveness (e.g. due to testers’ lack of domain knowledge) and poor efficiency (e.g. due to unsupported manual practices). This idea is captured in the project’s goal to innovate new knowledge and technology to support easy-to-use, end-to-end AI, for efficient and effective GUI testing that is of value to software industry and enables human-machine symbiosis.

Research

Evaluation of the Choice of LLM in a Multi-agent Solution for GUI-Test Generation

Dr. Stevan Tomic, AI/ML developer and researcher at Synteda, has conducted a study on PathFinder, accepted for ICST 2025, a Multi-Agent LLM framework designed to automate GUI testing. The research evaluates how different LLMs—Mistral-Nemo, Gemma2, and Llama3.1—impact test automation across e-commerce sites. Findings reveal that while a single LLM works best for specific platforms, a hybrid approach improves performance across diverse environments. This study underscores the critical role of LLM selection in AI-driven software testing.

Read Research Paper

LLM-based Reporting of Recorded Automated GUI-based Test cases

Dr. Diogo Buarque, AI/ML developer and researcher at Synteda, to be published in ICST 2025, on a new study that explores how LLMs enhance GUI test automation by automatically labeling test states and improving reports. Integrated into the HiveMind tool, this approach reduces manual effort and improves readability. Experiments show that LLM-enhanced reports match expert-written ones, making testing faster and more efficient.

Read Research Paper

Dr. Emil Alégroth, BTH
emil.alegroth@bth.se
(+46)703-33 80 22

Maycel Isaac, Synteda
(+46) 31 20 24 70

Carla Johansson, Synteda
carla@synteda.com
(+46) 31 20 24 70