Author: Dr Asma Zoghlami, Test Lead at Acutest.

How they differ and why it matters for quality assurance and software testing. Artificial Intelligence (AI) is becoming increasingly present in our daily lives, particularly since the emergence of Large Language models like ChatGPT. However, confusion can arise when discussing AI and its capabilities, particularly between Generative AI and Machine Learning (ML) which are two related but distinct fields within AI. In the Quality Assurance and Testing world, it is crucial that we distinguish between these two concepts as they raise different opportunities and challenges. Leveraging ML and Generative AI can significantly enhance productivity, speed and creativity in software testing. Recent research from Gartner highlights that integrating AI into testing workflows can increase test coverage by up to 30% and reduce bug escape rates by 20%.

The purpose of this article is to explain the difference between ML and Generative AI as well as highlight how software testers and QA engineers can use the capabilities of both approaches in diverse ways for effective software testing. It is easy to think that Machine Learning and Generative AI are the same thing since they both relate to AI. There can also be the view that Machine learning is somehow different from Generative AI without knowing exactly what distinguishes them.

What is AI?

This is simply a broad term that covers any system or process capable of performing tasks that typically require human intelligence, such as reasoning, learning, or decision making.

What is Machine Learning?

ML is a subset of AI that enables systems to learn from data and make predictions or decisions based on it. It uses various techniques, including supervised and unsupervised learning.

Supervised ML

Supervised ML involves a human that feeds the system with labelled data or examples containing a combination of inputs and expected outputs. The system learns to recognise patterns or rules that relate the training data to the labels, and then applies this knowledge to new data. The primary goal is to accurately classify data or predict outcomes.

For instance, we can consider Supervised ML in regression testing and test case prioritisation. During data labelling, the QA supervisor assigns priority labels (such as high, medium, or low) to test cases, establishing a reliable ground truth for the training of the ML model. The trained model can then proactively identify relevant test cases that are likely to fail and require immediate attention. As the model undergoes training, it will uncover relationships among various factors (e.g., past failures, performance metrics, module dependencies, code changes) connecting specific inputs to their corresponding priority level outputs. When fresh test cases emerge, the model leverages its acquired knowledge to predict their priorities with precision.

Unsupervised ML

Unsupervised ML models operate with unlabelled data. These models aim to discover patterns or structures in the data, such as clusters without explicit guidance or instruction.

An example use case for unsupervised ML lies in enhancing system reliability and product quality by detecting anomalies. One common approach in anomaly detection involves clustering. Think of it as grouping similar patterns together and spotting deviations. Imagine scenarios like a sudden spike in server response time or unexpected high CPU usage. Unsupervised models help to identify such irregularities when they could not be identified for a suitable data set labelling. This technique analyses system logs and behaviours, identifying clusters of normal activity. When an observation falls outside these established clusters, it is flagged as an anomaly.

Generative AI:

This is a branch of Machine Learning that focuses on generating new content or data from existing sources, including images, text, audio, and video. It excels at tasks like document summarisation, text generation and question answering.

Emerging areas like AI-driven test generation will be a game-changer in quality engineering by drastically improving test coverage and software quality. Imagine automatically creating test cases based on user requirements, generating test data, scripting automated tests, defining test scenarios, and generating test reports. Using Generative AI to rapidly produce human like content is certainly a captivating prospect. However, it does present new challenges to QA engineers and testers:

• Assessing validity, reliability, and usefulness to ensure the quality of the generated content, we need to develop new evaluation methods and metrics. 
• Identifying and addressing bias and discrimination present in the generated output to ensure fairness and inclusivity.
• Aligning Generative AI systems with ethical guidelines to avoid harmful or inappropriate output.
• Managing hallucination to maintain accuracy, where the content produced may seem plausible, but it lacks grounding.

QA engineers will need to master new skills, such as the art of prompt engineering. This is the essential process of creating appropriate and effective inputs or prompts for generative AI systems to produce the required output.

AI is transforming the QA and software testing landscape, unlocking new possibilities and solutions. However, this transformation also requires new skills and competencies. Organisations, delivery teams and QA engineers and testers will have to adjust to these changes and embrace the power of AI, while also ensuring its quality, reliability, and ethical application.

About the author: Dr Asma Zoghlami gained a PhD in computer science from the University Paris 8 (France) in 2013.  Their research focused on the intersection of three disciplines AI, computer science and geomatics. They are currently a Test Lead at Acutest.